KR101644190B1 - Apparatus and method for detecting pedestrian using edge projections - Google Patents

Abstract

An apparatus and a method for detecting a pedestrian using edge information projected in one dimension are disclosed. In the present invention, it is determined whether a pedestrian is present by analyzing image feature information such as distance information and edges according to the relative stability of a disparity map obtained by a cross correlation method. That is, in the area where the distance information is extracted stably, the obstacle is detected according to the width and the width of the distance profile and the degree of the connection with the ground. In the area where there are many errors in the parallax map, the edge information of the input image is used without depending on the distance information. To this end, reference data for pedestrians are pre-stored in an outdoor environment through a user-friendly S / W interface. The histogram distribution in which the edges of the pedestrian zone are one-dimensionally projected in the horizontal and vertical directions is stored in advance, and the pedestrian is robustly detected in real time by cross-correlation calculation with the actual input image.

Online electric vehicle, intelligent driving assistance system, stereo vision system, obstacle detection and tracking, edge information

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a pedestrian detection apparatus and a pedestrian detection method using one-

The present invention relates to an apparatus and method for detecting a pedestrian using a one-dimensional projected edge information, and more particularly to an apparatus and a method for detecting an obstacle such as a pedestrian present in a front boundary region in an on- .

When the power segmentation needed to drive an online electric vehicle is powered, nearby vehicles or people can be exposed to electromagnetic waves. Therefore, if there is a pedestrian in front of the vehicle, the power supplied to the segment must be cut off in the shortest time. To this end, the conventional art for detecting a pedestrian present in front of a vehicle detects a candidate region of the obstacle by using the distance information obtained by the stereo vision system. However, it is difficult to estimate the distance information from the stereo image stably, as shown in FIG. 1, in a situation where the surrounding background is complicated, various obstacles occur as the object moves, or environmental factors such as weather and time change. There is a case where the actual obstacle is not detected as a candidate area for judging whether or not the pedestrian is present. Also, if the obstacle is judged only by the parallax map distribution having a certain width and height, a situation where the obstacle is erroneously judged as shown in Fig. 2 also occurs. That is, in general, the parallax map is not stably estimated, and inaccurate three-dimensional information is obtained depending on the distance between the background and the object. Therefore, in order to stably detect an obstacle in the front area, a technique of simultaneously using image feature information such as the shape of a pedestrian in an alarm area is required.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a pedestrian detection apparatus and a pedestrian detection apparatus using one-dimensional projected edge information capable of accurately detecting an obstacle such as a pedestrian present in a boundary region even in a situation where it is difficult to stably estimate a time- Method.

SUMMARY OF THE INVENTION The present invention provides a pedestrian detection method using one-dimensional projected edge information that can accurately detect an obstacle such as a pedestrian present in a boundary area even in a situation where it is difficult to stably estimate a parallax map due to the surrounding environment There is provided a computer-readable recording medium storing a program for causing a computer to execute the program.

According to an aspect of the present invention, there is provided an apparatus for detecting a pedestrian using edge information projected on a screen, comprising: a parallax map calculator for calculating a parallax map based on a stereo image; A reliability region determination unit for determining a reliability region and a low region in the parallax map based on a preset reference value; A first candidate region determining unit for determining a pedestrian candidate region based on whether an average width of a pedestrian region is higher than a height of the first region and whether the pedestrian region is connected to a ground; A second candidate region determining unit for determining a pedestrian candidate region based on an edge distribution of a one-dimensional projected image of the second region with respect to a second region having low reliability; And a pedestrian detector for detecting a pedestrian by matching the pedestrian candidate area determined by the first candidate area determining part and the second candidate area determining part with the reference patch image collected in advance.

According to the apparatus and method for detecting a pedestrian using one-dimensional projected edge information, in addition to the parallax map distribution, simultaneous use of image feature information such as the shape of a pedestrian in an alarm area enables the parallax map to be stably It is possible to accurately detect an obstacle such as a pedestrian present in the boundary area even in a situation where estimation is difficult.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a preferred embodiment of an apparatus and method for detecting a pedestrian using one-dimensional projected edge information according to the present invention will be described in detail with reference to the accompanying drawings.

3 is a block diagram illustrating a configuration of a preferred embodiment of a pedestrian detection apparatus using edge information projected according to the present invention.

3, a preferred embodiment of a pedestrian detection apparatus using one-dimensional projected edge information according to the present invention includes an image input unit 310, a parallax map calculation unit 320, a confidence region determination unit 330, 1 candidate region determining unit 340, a second candidate region determining unit 350, a pedestrian detecting unit 360, and an alarm signal generating unit 370.

The image input unit 310 receives a stereo image from a stereo camera mounted on the vehicle. The parallax map calculator 320 calculates a parallax map based on the stereo image input through the image input unit 310. [ Most of the conventional arts have a processing performance of less than 10 frames per second in terms of the calculation of the parallax map, and thus it is difficult to apply to the real-time system such as the detection of the front obstacle of the vehicle in operation. Therefore, the parallax map calculator 320 calculates the parallax map from the stereo image using the SAD method or the cross-correlation method. At this time, each parallax map calculation algorithm is implemented on a graphic processing unit (GPU). In the case of the SAD method, the size of the search mask for setting the correspondence point is set to 5 × 5, 9 × 9, 11 × 11, 13 × 13, and the parallax resolution range is set to 32, The processing time can be 0.73, 0.81, 0.88, and 0.94 msec, respectively, in real time. However, since it is difficult to determine a threshold for judging whether or not the disparity information is trusted in a left-right check process for determining a corresponding region, the SAD method is applied to a matching region in a stereo image and a non- It is difficult to accurately distinguish between them. An alternative solution to this problem is to check the matching area using a cross-correlation method. The cross-correlation method has an advantage that the error region of the ground can be effectively removed while leaving the obstacle region as it is. Since the stable parallax map result is useful for the process of determining the road and boundary region thereafter, it is preferable to obtain the parallax map by the cross-correlation method rather than the SAD method in spite of a relatively large amount of calculation.

The confidence region judging unit 330 distinguishes the high reliability region from the low reliability region in the obtained parallax map. It is difficult to accurately obtain the parallax information from a stereo camera image mounted on a moving vehicle in a dynamic environment in which a plurality of target objects move in a complex background. Accordingly, obstacles such as a pedestrian are detected by performing different processes on the regions having high reliability and the regions having low reliability, which are classified by the reliability region determining unit 330. [

The first candidate region determining unit 340 determines whether or not the obstacle candidates are candidates for the obstacle candidates in consideration of the average width and height of the pedestrian region and whether the pedestrian region is connected to the ground, Area. In order to stably detect an obstacle existing in an incorrect parallax map region (i.e., a region with low reliability) in the front boundary region, the second candidate region determining unit 350 may use a one-dimensional projected edge Distribution. On the other hand, in order to detect only a pedestrian present on the power segment, the edge information may be further analyzed on the candidate obstacle interpreted from the distance information.

The pedestrian detection unit 360 detects the pedestrian from the candidate region determined by the first candidate region determination unit 340 and the second candidate region determination unit 350. To this end, the pedestrian detection unit 360 has a patch image to be a reference collected in advance from the actual environment, and the reference patch image may be stored in a separate storage unit. As a result of analyzing multiple input sequences, a person is free to move, such as stopping, walking or running, and taking various positions accordingly. Also, the same person can exist in various directions such as front, back, and side. Therefore, the pedestrian area having various posture and motion on the input image sequence is acquired as a template image which is a main reference data of matching. Based on the reference patch image, the pedestrian detecting unit 360 collects a patch image, which is a reference in the matching process, from the actual environment in advance and analyzes various feature distributions for the pedestrian. Then, the area corresponding to the pedestrian in the input image sequence is template-matched with the reference data.

In order to detect the pedestrians, brightness, color distribution, distance measure, etc. are used in the matching process of existing researches. In order to robustly detect pedestrians using such information, various kinds of SVM (Support Vector Machine) Methods have been proposed. However, in most of the studies, the constraint was set up such as when the target obstacle moves under a static background acquired from a fixed camera. Even in a dynamic situation, the calculation complexity of the algorithm is very high, or thousands of reference data are acquired Sometimes you have to. In consideration of this, in the present invention, in order to mount a pedestrian detection algorithm on a running vehicle, the cross-correlation of the edge histogram vector for the corresponding region is used in consideration of real-time implementation and convenient expansion of the reference image. Various reference databases are preconfigured through a simple user interface operation at the driving site, and pedestrians are detected through correlation between edge histogram distribution vectors for candidate regions of the input image. FIGS. 4 and 5 illustrate a method of setting a pedestrian area predetermined by the user and an edge histogram distribution of the pedestrian in the actual input image, respectively.

In general, since the data to be matched increases with the use of the brightness distribution in the template area, it is difficult to process a large number of pedestrians in real time. Accordingly, in the present invention, a search mask corresponding to the average size of a pedestrian is set for an area in which parallax information is not correct in a boundary area, and then horizontal and vertical edges are extracted with a Sobel edge filter. Then, the pedestrian area is extracted as shown in FIG. 4 according to the horizontal and vertical scan directions using negative and positive phase information in the gradient information of the edge. That is, the pedestrian detecting unit 360 determines an edge region corresponding to the first positive phase and the negative negative phase according to the phase of the edge in each scan line with the width and height information of the obstacle candidates, respectively, with respect to the low reliability region . The histogram distribution, which means the width and height of the pedestrian zone, is divided into ten bins each in the horizontal and vertical directions and stored as vector-type reference data. There are a number of areas that are partially opened as shown in Fig. 5, rather than closed edges of the pedestrian edge information in the actual input image. However, since the reference data of the pedestrian is directly acquired and updated from the input image sequence in the actual driving environment, a very stable matching result can be obtained. In addition, the background edge in the search window is not asymmetrical and non-uniform relative to the pedestrian area, so it does not affect the process of determining the pedestrian area. Side and back reference data in addition to the frontal view of the pedestrian at various distances are shown in Fig.

와

는 각각의 평균값이고, s_x와 s_y는 각각의 표준편차 값이다. 여기서 상관계수 r은 -1≤r≤1를 항상 만족하며, 상관계수가 0보다 클수록 양의 상관관계를 갖는다고 표현하고 두 데이터가 서로 비슷함을 의미한다.Next, the pedestrian detection unit 360 normalizes the histogram data of the extracted template image to a predetermined size because the histogram data of the extracted template image is different from the size of the candidate region of the obstacle. At this time, the comparison with the reference data determines the cross-correlation coefficient between the normalized histogram information by the following equation. For each of the two measured x, y values, n measured values of the normalized dimension are entered

Wow

Is the mean value of each, and s _x and s _y are the respective standard deviation values. Here, the correlation coefficient r always satisfies -1? R? 1, and a correlation coefficient of greater than 0 means that the correlation has a positive correlation, meaning that the two data are similar to each other.

The alarm signal generator 370 generates an alarm signal when an obstacle such as a pedestrian is detected in the boundary area. The alarm signal generated by the alarm signal generator 370 is transmitted to the control processor of the power segment of the online automobile via RS232 communication or the like.

FIG. 7 is a flowchart illustrating a method of detecting a pedestrian using one-dimensional projected edge information according to an exemplary embodiment of the present invention. Referring to FIG.

Referring to FIG. 7, the parallax map calculator 320 calculates a parallax map based on the stereo image input through the image input unit 310 (S700). Next, the confidence region determination unit 330 distinguishes the high reliability region from the low reliability region in the parallax map based on the preset reference value (S710). In operation S720, the first candidate region determining unit 340 generates a V-parallax map for the first region having high reliability, and then extracts the road information in operation S730. In addition, the first candidate area determining unit 340 determines a front boundary area (Alarm Area) based on the extracted road information (S740). In operation S750, the first candidate area determination unit 340 determines a pedestrian candidate area based on whether the average width of the pedestrian area, the height, and the ground are connected to each other based on the V-parallax map. On the other hand, in step S720, the second candidate area determination unit 350 determines a pedestrian candidate area based on the edge distribution of the one-dimensional projected image of the second area for the second area having low reliability. Next, the pedestrian detection unit 360 detects the pedestrian by matching the pedestrian candidate area determined by the first candidate area determination unit 340 and the second candidate area determination unit 350 with the reference patch image collected in advance (S770 ). Finally, the alarm signal generator 370 generates an alarm signal when a pedestrian is detected within the boundary area (S780).

An experimental system for verifying the performance of a pedestrian detection apparatus and method using one-dimensional projected edge information according to the present invention comprises a stereo camera system and a high-performance computing apparatus. The front stereo image is input from a stereo camera (for example, a Flea2-08S2C camera manufactured by Pointgrey). The Flea2 camera processes 1032 × 776 images at 30 frames per second (fps), and a focal length of 5 mm is attached to ensure a sufficient time. We also implemented a cross correlation method on graphics hardware (GPU) to obtain a stable parallax map from a stereo input image. Recently, graphics hardware, which is actively developed by NVIDIA, etc., provides higher computation performance than a central processing unit (CPU) by arranging a number of high performance processors in parallel. The computing system is an HP Workstation XW4600 with Intel Core2 Quad 2.66MHz CPU, 4G main memory and Nvidia GTX 260 board. The cross correlation method has a processing speed degradation of about 35% compared to SAD, but a parallax map of 320 × 240 resolution can be obtained at a speed of 25 frames per second or more on the configured system.

In actual outdoor experiments, the histogram data of the corresponding area was used as reference data by the edge distribution of 65 pedestrians from various image sequences, and when 6 or more reference templates having a correlation coefficient of 0.6 or more were detected, it was judged to be a pedestrian. FIG. 8 shows a pedestrian area and a histogram distribution detected by the present invention, and an obstacle area detected in a situation where a plurality of pedestrians exist. The time required for the cross-correlation operation between the 65 reference data for one input area is only 0.03 m sec., And even if the number of reference data is expanded for a more robust matching result, many obstacles existing in the boundary area are simultaneously processed And confirmed the processing performance.

Although the pedestrian having an appearance similar to the reference data set in advance is correctly detected, the present invention sometimes erroneously recognizes a pedestrian even if it is an object of a type that is not stored or similar to a person. In order to solve this problem, various types of pedestrian areas should be secured as reference data. In addition, the detection performance of the target pedestrian can be improved if a more robust pedestrian detection algorithm is applied while maintaining the real-time property. In addition, stereo matching performance is greatly affected by ambient lighting environment (brightness, shadow), background, and surrounding objects in obtaining the parallax information from the stereo vision system. Therefore, if only the parallax information is used, the obstacle information at the front can not be detected stably. In this regard, in order to stably detect a pedestrian present in a boundary area in front, a histogram distribution in which edge information of a candidate obstacle area for judging whether a pedestrian is determined is projected one-dimensionally in the horizontal and vertical directions is extracted. Then, the pedestrian is detected by cross-correlation calculation between the reference data constructed in advance and the candidate region of the input video sequence. In the experiment, cross-correlation between 65 reference data for the target obstacle is made within 0.03m second. Therefore, even if there are many obstacles in the boundary area using more reference data, stable real-time detection of obstacles is possible.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer-readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation in the embodiment in which said invention is directed. It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the scope of the appended claims.

FIG. 1 is a diagram showing a detection result (middle) and an original image (right) of a confidence region in a parallax map (left) obtained by a cross-correlation method,

FIG. 2 is a view showing a result of detection and non-detection of a pedestrian according to the prior art,

3 is a block diagram showing a configuration of a preferred embodiment of a pedestrian detecting apparatus using edge information projected according to the present invention;

4 and 5 are diagrams showing a method of setting a pedestrian area predetermined by a user and an edge histogram distribution of a pedestrian in an actual input image,

Fig. 6 shows side and back reference data outside the front of the pedestrian at various distances; Fig.

FIG. 7 is a flowchart illustrating a method of detecting a pedestrian using one-dimensional projected edge information according to an embodiment of the present invention.

8 is a view showing a pedestrian area and a histogram distribution detected by the present invention, and Fig.

9 is a view showing an obstacle area detected by the present invention in a situation where a plurality of pedestrians exist.

Claims (1)

A parallax map calculator for calculating a parallax map based on the stereo image; A first candidate area determining unit for determining a pedestrian candidate area based on whether an average width of the pedestrian area and a height are connected to the ground for the first area of the parallax map; A second candidate area determining unit for determining a pedestrian candidate area based on an edge distribution of the one-dimensional projected image of the second area with respect to the second area of the parallax map; And And a pedestrian detector for detecting a pedestrian by matching a pedestrian candidate area determined by the first candidate area determining part and a second candidate area determining part with a reference patch image collected in advance, Wherein the first area and the second area are divided according to a reference value of an error in the parallax map.

Images