KR101501531B1 - Stereo Vision-based Pedestrian Detection System and the method of - Google Patents

Abstract

The present invention relates to a system and a method for detecting a pedestrian and, specifically, to a system and a method for detecting a pedestrian in an image obtained based on stereo vision by obtaining a disparity map using a stereo vision system. The system according to the present invention comprises: a disparity map creation unit for creating a disparity map of an image by obtaining a disparity value of the image inputted from left and right sides; a data extraction unit for estimating location information on a road in the inputted image, an object existing on the road, and an contact point formed by the road and the object based on the created disparity map; and a pedestrian area detection unit for detecting a pedestrian area by estimating the height of the object based on the location information obtained from the data extraction unit and/or the disparity map; and a pedestrian detection unit for detecting a pedestrian by forming windows according to the disparity map and/or the estimated height of the object in the detected disparity area.

Description

[0001] Stereo Vision-based Pedestrian Detection System and Method [0002]

The present invention relates to a system and method for detecting a pedestrian, and more particularly, to a system and a method for detecting a pedestrian by acquiring a depth map in an image based on stereo vision.

Recently, much research has been conducted on a method of detecting a pedestrian on a road using an image. Video-based pedestrian detection technology is an important technology that can be applied to various fields such as intelligent automobile, security / surveillance.

As such a method for detecting a pedestrian detection object and an obstacle during driving, there are a method using a stereo vision and a method using only a mono image (e.g., a gray image).

A method of detecting an obstacle through a stereo image can improve the detection performance through the three-dimensional image information. For example, when a stereo vision system is used to obtain a depth map, which is three-dimensional information, and an obstacle is detected using the stereo vision system, problems of poor detection performance due to background patterns or shadows around the obstacle can be reduced. In addition, using the parallax map can use the three-dimensional distance information, improving the separation between the object and the background, and using various information, thereby improving the object detection performance.

However, when a pedestrian is detected by a single image, it is necessary to perform a pedestrian template of various sizes to detect pedestrians of various sizes. Therefore, it takes a lot of time and deteriorates the detection performance There is a problem.

FIG. 1 shows a problem in detection of pedestrians of various sizes in detecting a single image-based pedestrian. FIG. 1-a shows the input gray image, and FIG. 1-b shows a method of detecting pedestrians of various sizes based on a conventional single image.

As shown in FIG. 1-b, in order to detect pedestrians of various sizes on a conventional single image basis, it is necessary to determine whether a pedestrian is an entire area of the input image using various sizes of pedestrian templates, The pedestrian must be detected by using the template, but changing the size of the input image.

The methods of detecting the pedestrian using the single image described above are time-consuming because of many repetitive operations, and the performance is degraded.

In addition, since the separation of the object and the background is degraded by using only a single image, in order to detect pedestrians of various sizes, a pedestrian template of various sizes needs to be performed on the entire input image, .

In order to solve the above-described problems, the present invention provides a system and method for detecting a pedestrian area using a stereo map-based disparity map (Disparity map) and detecting a pedestrian within the detected pedestrian area The purpose is to provide.

A system for detecting a pedestrian from an image based on a stereo vision according to an embodiment of the present invention obtains a parallax value of an input image on the left and right sides and calculates a parallax map for a parallax based on a Depth / A data extracting unit for estimating position information of a road, an object existing on the road, and a contact formed by the road and the object, based on the generated parallax map; A pedestrian area detecting unit for detecting a pedestrian area by estimating the height of the object on the basis of the positional information and / or the parallax map acquired from the pedestrian area, and a window based on the height of the parallax map and / or the estimated object in the detected pedestrian area And a pedestrian detection unit for detecting the pedestrian.

The pedestrian detecting unit of the present invention further includes a noise removing unit for filtering the noise of the detected object.

In addition, the pedestrian detection unit of the present invention determines whether or not an object detected on the road is a pedestrian, then checks whether the pedestrian is erroneously detected through a parallax value distribution, color, and texture of the image, .

According to another aspect of the present invention, there is provided a method of detecting an obstacle from an image based on stereo vision, comprising: generating a disparity map by receiving an image based on a plurality of stereo vision; The method comprising the steps of: acquiring position information of a road, an object, and a contact formed by the road and the object; estimating a height of an object existing on the road of the input image based on the obtained position information; Detecting a pedestrian area of the image from the positional information and the parallax map, setting a window according to the height of the object for detecting the pedestrian present in the detected pedestrian area, and detecting the pedestrian in the set window .

The step of acquiring the positional information of the present invention estimates the feature of the road included in the image through the V-parallax map.

In addition, the step of acquiring the position information of the present invention includes a step of acquiring the position information of the road through the U-parallax map, the V-parallax map, and / or the column detection and detecting an object included in the image based on the obtained positional information .

The step of detecting the pedestrian zone of the present invention further includes the step of filtering the noise of the detected area.

The step of detecting a pedestrian according to the present invention further includes a step of examining whether the object is erroneously detected as a pedestrian, and a step of correcting the object when the object is erroneously detected as a pedestrian.

The present invention detects a pedestrian area of an input image accurately using a stereoscopic map based on stereo vision and performs pedestrian detection using only a template having a height at which a pedestrian can exist within the detected area, Speed and system performance.

In addition, the present invention provides advantages that can be used for detecting obstacle objects in various fields such as intelligent automobiles, robots, security / surveillance, and medical fields.

1 shows an embodiment of a pedestrian detection method according to the prior art;
FIG. 2 illustrates a parallax map-based pedestrian detection embodiment according to an embodiment of the present invention. FIG.
3 is a system configuration diagram according to an embodiment of the present invention;
4 is a flowchart of a pedestrian detection operation according to an embodiment of the present invention;
FIG. 5 illustrates an embodiment of a pedestrian detection process according to an embodiment of the present invention. FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. And is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined by the claims.

It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. &Quot; comprises " and / or "comprising" when used in this specification is taken to specify the presence or absence of one or more other components, steps, operations and / Or add-ons. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a diagram illustrating a parallax map-based pedestrian detection method according to an embodiment of the present invention.

In an embodiment according to the present invention, as shown in FIG. 2-a, only the pedestrian area is accurately detected using the parallax map, and the pedestrian area To detect pedestrians.

3 is a block diagram of a pedestrian detection system according to an embodiment of the present invention.

The present invention relates to a system for detecting a pedestrian from an image based on a stereo vision and acquiring a parallax value of an input image on the left and right sides and generating a parallax map (Depth / Disparity map) A data extracting unit 320 for estimating the position information of the roads in the input image, the objects existing on the roads and the contacts formed by the roads and the object based on the generated parallax map, A pedestrian area detecting unit 330 for detecting a pedestrian area by estimating the height of the object based on the positional information and / or the parallax map obtained from the extracting unit 320, And a pedestrian detector 340 for detecting a pedestrian by forming a window according to the height of the object.

In one embodiment of the present invention, the parallax map generation unit 310 acquires parallax information of an object formed on an image of a road taken as an object using two stereo cameras to generate a parallax map, A parallax map can be generated by extracting and expressing the distance information using a time difference, a base line between the binocular cameras, and a camera focal length.

The pedestrian detection unit 340 determines whether or not the object detected on the road is a pedestrian by using a noise removing unit 341 for filtering the noise of the detected object, And a pedestrian verification unit 342 for correcting erroneous detection in the case of erroneous detection.

As one embodiment of detecting a pedestrian using the parallax map, there is a method using a V-displacement map (parallax map). In the V-parallax map, the horizontal axis represents the frequency of the parallax values, and the vertical axis represents the vertical line. The distribution of the value of the parallax map (for example, 0 to 255) is plotted on the vertical line.

That is, by displaying the frequency by the corresponding time difference value, data with a high frequency is bright and data with a low frequency is dark. Thereafter, data having a small frequency is deleted and binarized.

 In the binarized V-parallax map, planar components such as roads appear as diagonal lines (red lines), and obstacle objects appear as vertical lines (blue lines).

Thus, only an obstacle can be detected. First, the road component is extracted from the V-parallax map, and it is checked whether there is a parallax value higher than the corresponding road component value for each vertical line, and then extracted. The component obtained after extraction corresponds to the obstacle. If the obstacle is extracted from the V-parallax map, the vertical line section with the obstacle can be extracted. If the portion having the parallax value is extracted within the vertical line section, the obstacle can be detected from the parallax map.

4 is a flowchart of a pedestrian detection operation according to an embodiment of the present invention.

As shown in FIG. 4, first, a parallax map, which is three-dimensional information, is generated from two gray images input from left and right using stereo vision (S410).

The feature information of the road contained in the input image is extracted from the generated parallax map (S420).

Since all obstacles exist on the road, the road feature information is important information for judging the presence of an obstacle. In particular, since road feature information is used for accurate pedestrian area detection, it is very important to accurately estimate road feature information.

In an embodiment of the present invention, the position information of the road is obtained through U-parallax map, V-parallax map, and / or column detection, and an object included in the image is detected based on the positional information.

The V-parallax map can generally be used to extract road feature information. The V-parallax map is obtained by horizontally accumulating the obtained parallax values and shows the road characteristic information according to the parallax value distribution and the state of the detected object.

In addition, only the feature information of the road can be extracted by using the dynamic programming (DP) and the maximum frequency value in the V-parallax map. The object is detected based on the extracted road feature information. U-parallax map, V-parallax map method, and column detection method are typical methods for object detection.

Processing for removing noise is performed through a morphology or the like of the detected area (S440).

Then, the road contact point is estimated for each vertical column in the detected image (S450). In step S450, a point located at the bottom of the positions of pixels having a value for each image vertical line is searched and a height, which is a region where a pedestrian exists, is estimated based on the position of the road contact estimated for each vertical line S460). When estimating the height, the following Equation 1 is used.

[Equation 1]

는 두 카메라가 지면에 대해서 기울어져 있는 각도, b는 base lin 으로서 두 카메라의 간격을 각각 나타낸다. In Equation (1), h is the vertical height (the number of vertical pixels) in the image, d is the parallax value (pixel value of the parallax map)

Is the angle at which the two cameras are tilted with respect to the ground, and b is the base lin.

In addition, H is a vertical size of an actual pedestrian, and generally has a value of 1.8 to 2 m. That is, by using the height (H) of the actual pedestrian and the various parameters in each of the parallax values d, the range (height) in which the pedestrian can exist in each vertical line can be obtained, The pedestrian area where the pedestrian may exist can be detected based on the contact position (S470).

Then, the pedestrian detection is performed in the detected pedestrian area. First, a window for each object height is set (S480). The window is a template suitable for the pedestrian size for pedestrian detection, and the ratio of the length to the width of the window is set to roughly 2: 1.

When the vertical line of the pedestrian area is determined, the width of the window is determined to be about 1/2 of the vertical line size, and the pedestrian detection is performed in the gray image for each window (S490).

Various methods such as SVM (Support Vector Machine) based on Histograms of Oriented Gradient (HOG) and Adaboost algorithm based on Integral Channel Features can be used as pedestrian detection methods.

Since the result of the pedestrian detection may include various erroneous detection, the pedestrian verification is performed using various characteristics such as a time difference value distribution, a color, and a texture (S500). In the pedestrian verification stage, if an object other than a pedestrian is erroneously detected as a pedestrian, it can be corrected.

FIG. 5 shows an embodiment of a pedestrian detection process according to an embodiment of the present invention.

 As shown in FIG. 5, the proposed pedestrian detection method shows a parallax map in FIG. 5-a and a V-parallax map generated in a parallax map in FIG. 5-b.

5C shows the result of extracting only the road feature information from the V-parallax map, and FIG. 5D shows the result of performing the object detection using the road feature information.

FIG. 5-e is a result of estimating an object road contact point for every vertical line, and FIG. 5-f is a result of estimating an object height based on a road contact point. FIG. 5-g shows the result of detecting the final pedestrian zone, and FIG. 5-h shows setting of a window for each object height in the area.

FIG. 5-i shows a process of detecting a pedestrian in each window, and FIG. 5-j shows a pedestrian detection result. Finally, the result of verifying the pedestrian finally to reduce false detection is shown in FIG. 5-k.

As described above, the present invention provides a method of accurately detecting a pedestrian area using a parallax map and detecting a pedestrian using a pedestrian template of an appropriate size within the detected pedestrian area, This provides the advantage.

The foregoing description is merely illustrative of the technical idea of the present invention and various changes and modifications may be made without departing from the essential characteristics of the present invention. Therefore, the embodiments described in the present invention are not intended to limit the scope of the present invention, but are intended to be illustrative, and the scope of the present invention is not limited by these embodiments. It is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents, which fall within the scope of the present invention as claimed.

Claims (8)

A system for detecting a pedestrian from a video based on stereo vision,
A parallax map generating unit for obtaining a parallax value of an input image from left and right sides and generating a parallax map (Depth / Disparity map) for the image;
A data extracting unit for estimating position information of a road in the input image, an object existing on the road, and a contact formed by the road and the object based on the generated parallax map;
A pedestrian zone detection unit for detecting a pedestrian zone by estimating a height of the object based on the positional information obtained from the data extraction unit and the parallax map;
And a pedestrian detection unit for detecting a pedestrian by forming a window according to the height of the object and the parallax map in the detected pedestrian area,
The pedestrian detection unit may determine whether or not the pedestrian is erroneously detected through a parallax value distribution, a color, and a texture of the image after determining whether the object detected on the road is a pedestrian, Further comprising a pedestrian verification unit
Pedestrian detection system.
The method according to claim 1, wherein the pedestrian detection unit
A noise canceling unit for filtering the noise of the object; Further comprising
Pedestrian detection system.
delete A method for detecting an obstacle from an image based on stereo vision,
Generating a disparity map by receiving an image based on a plurality of stereo visions;
Obtaining position information of a road, an object included in the image, and a contact formed by the road and the object based on the generated parallax map;
Estimating a height of an object existing on a road of the input image based on the obtained position information;
Detecting a pedestrian area of the image from height, position information of the estimated object, and the parallax map;
Setting a window according to a height of the object for detecting a pedestrian present in the detected pedestrian zone; And
Detecting a pedestrian in the set window,
The step of detecting the pedestrian further includes checking whether the object is erroneously detected as a pedestrian, and modifying the erroneous detection when the object is erroneously detected as a pedestrian
/ RTI >
5. The method of claim 4, wherein obtaining the location information comprises:
Estimating the feature of the road included in the image through the V-parallax map
/ RTI > 6. The method of claim 5, wherein obtaining location information comprises:
Obtaining a position of the road based on any one of a U-parallax map, a V-parallax map, and a column detection, and detecting an object included in the image based on the position of the road; Including
/ RTI >
5. The method of claim 4, wherein detecting the pedestrian zone of the image comprises:
Filtering the noise of the detected region; Further comprising
/ RTI >
delete

Images