KR20160063039A - Method of Road Recognition using 3D Data - Google Patents

Abstract

The present invention relates to a road recognition method which comprises the following steps of: generating a 3D disparity map with respect to an image obtained through a stereoscopic camera; generating a U disparity map in the 3D disparity map; processing and simplifying the U disparity map; detecting a boundary candidate based on the 3D disparity map in the U disparity map; and filtering the detected boundary candidate to determine a boundary of a road. Therefore, the road recognition method does not perform an additional process of generating V displacement to provide accurate road recognition information to a user in real time.

Description

Field of the Invention [0001] The present invention relates to a road recognition method using 3D data,

More particularly, the present invention relates to a method of detecting a boundary of a road using a 3D disparity map.

A transportation means such as a vehicle can photograph a periphery using a three-dimensional camera, and can recognize a boundary surface of a road, a peripheral obstacle, and the like based on the photographed image.

In the conventional road recognition technology, the u-displacement and the v-displacement, which divide and accumulate the disparity map of the stereo camera in longitudinal and lateral directions, respectively, are discriminated to discriminate the ground area. This conventional technique has difficulties in providing real-time information by using a complicated method such as clustering and regression analysis to generate u displacement and v displacement, respectively, and using a long computation time such as RANSAC or Hough transformation for modeling .

An object of the present invention to solve such conventional problems is to generate a U disparity map of three-dimensional image data without generating a separate V displacement to provide accurate road information to a user.

To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described herein, there is provided a road recognition method comprising: generating a 3D disparity map for an image acquired through a stereo camera; generating a U disparity map in the 3D disparity map; Processing the U disparity map to simplify and processing the U disparity map, detecting a boundary candidate based on the 3D disparity map in the U disparity map, and filtering the detected boundary candidate to determine a boundary of the road can do.

According to various embodiments, processing and simplifying the U disparity map may include applying a distance weight to the U disparity map and binarizing the U disparity map with a threshold . And the distance weight increases as the distance from the object increases. The threshold value may be determined based on standard deviation information of a road area set in advance. The step of performing the binarization by the threshold may include setting a value of a point having a U disparity value lower than the threshold to zero.

According to various embodiments, the step of detecting the boundary candidate may comprise the steps of finding a disparity value of a pixel corresponding to a boundary in a designated column and determining a position in the 3D disparity map having the disparity value .

According to various embodiments, the step of determining the boundary of the road includes detecting peaks by connecting the detected points with the boundary candidate, and determining the boundary of the road based on the detected peak width can do. The detecting of the peak may include detecting a peak by connecting the points detected by the boundary candidate in a first direction and connecting the points detected by the boundary candidate in a second direction opposite to the first direction, And a step of detecting. The step of determining the boundaries of the road based on the detected peak width may include removing the detected peak when the width of the detected peak is less than a specified range.

The transportation means according to the present invention includes a stereo camera, a three-dimensional map generator for generating a three-dimensional disparity map for an image acquired through the stereo camera, a U map for generating a U disparity map in the three- A U-map processing unit for processing the U-disparity map to simplify the U-disparity map, a boundary detector for detecting a boundary candidate based on the 3D disparity map in the U-disparity map, And a filtering unit for determining the boundary of the image.

As described above, the road recognition method according to the present invention does not perform a process for generating another V displacement, and can provide accurate road recognition information to a user in real time.

In addition, the road recognition method according to the present invention can detect unnecessary peaks through bidirectional filtering to determine accurate road boundaries.

1 is a configuration diagram of a road recognition system according to the present invention.
2 is a block diagram showing a configuration of a U-map processing unit according to various embodiments.
3 is a flowchart illustrating a road recognition process according to various embodiments.
Figure 4 is an exemplary diagram of applying a learning-based threshold according to various embodiments.
5 is an exemplary diagram illustrating road boundary candidate detection in accordance with various embodiments.
6 is an illustration of an example of error cancellation filtering performed in a filtering unit according to various embodiments.
FIG. 7 is a diagram illustrating an example of applying a filtering process to an actual image according to various embodiments.
8 is a diagram illustrating boundary detection according to various embodiments.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the exemplary embodiments of the present invention, descriptions of techniques which are well known in the art and are not directly related to the present invention will be omitted. This is to omit the unnecessary description so as to convey the key of the present invention more clearly without fading.

1 is a configuration diagram of a road recognition system according to the present invention. The road recognition system 100 may be mounted on a vehicle such as a vehicle. The information processed through the road recognition system 100 may be transmitted to the driver, used for the user's operation assistance program, and used for the unmanned operation system.

1, the road recognition system 100 may include a stereo camera 110, a 3D map generator 120, a U map generator 130, a U map processor 140, a boundary detector 150, a filtering unit 160, and an output unit 170 .

The stereo camera 110 can photograph two or more photographing lenses at specified intervals to take an image or an image. The stereo camera 110 can simultaneously photograph an object through two photographing lenses and obtain two images at the same time.

The three-dimensional map generating unit 120 may generate the three-dimensional map by analyzing the image information photographed through the stereo camera 110. The three-dimensional map generation unit 120 can analyze an object photographed through each lens, a size of the object, a distance to the object, and the like through comparison. For example, a stereo image taken through each lens included in the stereo camera 110 may be a two-dimensional image that can be represented by x and y axes. The three-dimensional map generating unit 120 may compare the images photographed by the respective lenses at the same time, and may generate a three-dimensional image that can be represented by x-axis, y-axis, and z-axis reflecting the distance between the lenses .

According to various embodiments, the three-dimensional map generator 120 may generate a three-dimensional disparity map according to the difference caused by the position of the photographing lenses, the distance between the lenses, and the installation angle.

The U map generating unit 130 may generate a U disparity map in the 3D disparity map. The U disparity map can be obtained by accumulating histograms of distance values located on the same horizontal axis by using values of x axis and z axis among three-dimensional images.

In the conventional road recognition technology, the u-displacement and the v-displacement, which divide and accumulate the disparity map of the stereo camera in longitudinal and lateral directions, respectively, are discriminated to discriminate the ground area. This conventional technique has difficulties in providing a real-time system by using a complicated method such as clustering and regression analysis to generate u displacement and v displacement, respectively, and employing a long computation time such as RANSAC or Hough transformation for modeling .

On the other hand, the road recognition technology according to the present invention does not use a separate V disparity map and does not use a complicated algorithm such as RANSAC or Hough transformation used for straight line detection, and thus can maintain a high data processing speed. The processed data can be delivered to the driving support system in real time, and can provide road awareness information to the user.

The U map processing unit 140 may process the generated U disparity map, and may apply weighting and binarization processing. The U map processing unit 140 can increase the reliability of data by using not only the number of pixels but also the actual distance information by multiplying the weight according to the U disparity value. In addition, the U map processing unit 140 can perform a binarization operation using a learning-based threshold. Information regarding the operation of the U map processing section 140 may be provided through FIG. 2 and FIG.

The boundary detection unit 150 may connect boundary points in the processed U disparity map. The boundary detector 150 may refer to the 3D disparity map to display the position of the boundary points in the image. Detailed information regarding the operation of the boundary detection unit 150 may be provided through FIG.

The filtering unit 160 may filter out some of the peaks generated by the points detected as boundary candidates. The filtering unit 160 may remove a peak below a predetermined width in the process of connecting the detected points to the boundary candidates, thereby detecting the boundary of the road that can give a substantial meaning to the user. Detailed information regarding the operation of the filtering unit 160 may be provided through Figs. 6 and 7. Fig.

The output unit 170 may output an image or a range value reflecting the filtered boundary value. The output unit 170 may display an image reflecting the filtered boundary value on the screen or provide it to a driving support system so as to use it (narrowing recognition, forward vehicle tracking, collision avoidance, etc.).

2 is a block diagram showing a configuration of a U-map processing unit according to various embodiments.

Referring to FIG. 2, the U map processing unit 140 may include a weight calculation unit 210 and a threshold value application unit 220.

The weight calculation unit 210 may reflect the distance weight to the U disparity value provided by the U map generation unit 130. [ The weight calculation unit 210 may multiply the U disparity value by a weight according to distance (a higher weight as the distance increases) so that the actual distance information as well as the simple pixel count can be utilized. The distance weight can be reflected by the following equation (1).

[Equation 1]

: U 디스패리티 값

: U disparity value

: 거리 가중치

: Distance Weight

The threshold value applying unit 220 can simplify a part of the area corresponding to the road by binarizing. The threshold value applying unit 220 may store standard deviation information on the disparity distribution of the area corresponding to the road. The standard deviation information may be set based on height information of a general road topography.

The threshold value applying unit 220 may determine a threshold for removing a U disparity value below a predetermined range according to the standard deviation information. For example, the threshold applying unit 220 may determine a threshold for removing points within a range of 60% or less based on the standard deviation information among all the U disparity points. The threshold value application unit 220 may set values less than a certain height to 0, and output only values higher than the height. 5, information on the operation of the threshold applying unit 220 may be provided.

3 is a flowchart illustrating a road recognition process according to various embodiments.

Referring to FIG. 3, in operation 310, the three-dimensional map generator 120 may generate a three-dimensional disparity map according to the difference caused by the position of the photographing lenses, the distance between the lenses, and the installation angle.

In operation 320, the U map generating unit 130 may generate a U disparity map in the 3D disparity map. The U-map generating unit 130 can be obtained by accumulating histograms of distance values located on the same horizontal axis by using values of the x-axis and the z-axis among the three-dimensional images.

In operation 330, the U map processing unit 140 may process the generated U disparity map to perform weighting and binarization processing. The U map processing unit 140 may reflect the distance weight to the U disparity value provided by the U map generating unit 130. In addition, the U map processing unit 140 can simplify a part of the area corresponding to the road by binarizing.

In operation 340, the boundary detector 150 may concatenate boundary points in the processed U disparity map. The boundary detector 150 may refer to the 3D disparity map to display the position of the boundary points in the image.

In operation 350, the filtering unit 160 may filter out some of the points detected as boundary candidates. The filtering unit 160 may remove a peak below a predetermined width in the process of connecting the detected points to the boundary candidates, thereby detecting the boundary of the road that can give a substantial meaning to the user.

In operation 360, the output unit 170 may display an image reflecting the filtered boundary value on the screen or provide it to the driving support system for use (narrowing recognition, forward vehicle tracking, collision avoidance, etc.).

Figure 4 is an exemplary diagram of applying a learning-based threshold according to various embodiments.

Referring to FIG. 4, the threshold value application unit 220 can simplify a part of the area corresponding to the road by binarizing the area.

The first image 410 represents the U disparity distribution before applying the threshold. The first image 410 reflects merely the normal height difference of the road in the middle / lower part of the image in addition to the part that becomes the boundary of the actual road.

The threshold value applying unit 220 may store standard deviation information on the disparity distribution of the area corresponding to the road. The standard deviation information may be set based on height information of a general road topography.

The threshold value applying unit 220 may determine a threshold for removing a U disparity value below a predetermined range according to the standard deviation information. For example, the threshold applying unit 220 can obtain the second image 420 by removing the points in the range of 60% or less based on the standard deviation information among the entire U disparity points of the first image 410.

The second image 420 can be converted into a state in which the U-disparity distribution in the lower / upper part of the image is all set to 0 according to the application of the threshold, so that the actual boundary of the road can be easily determined.

5 is an exemplary diagram illustrating road boundary candidate detection in accordance with various embodiments.

Referring to FIG. 5, the boundary detector 150 may connect boundary points in the binarized U disparity map 420. The boundary detection unit 150 may refer to the 3D disparity map 510, which is an input image, in order to display the position of the boundary points in the image.

The boundary detector 150 can find the disparity value (row number) of the pixel corresponding to the boundary in the specific column of the binarized U disparity map 420. The boundary detector 150 can scan the 3D disparity map 510 by scanning the position having the disparity value from bottom to top. The boundary detection unit 150 may store position information of a matching point.

For example, the boundary detector 150 can find the position of the boundary 411 in the binarized U disparity map 420 by referring to the 3D disparity map 520. The boundary detection unit 150 can check the disparity value (line number) of the boundary point 411 and scan the 3D disparity map 510 from below to find the matched point 511. The boundary detection unit 150 may determine the point 521 as a result of matching the position of the point 511 in the actual image and store the result.

The road recognition technology according to the present invention does not use a separate V disparity map and does not use a complicated algorithm such as RANSAC or Hough transformation used for straight line detection, thereby enabling a fast data processing speed to be maintained. The processed data can be delivered to the driving support system in real time and can be provided to the user.

6 is an illustration of an example of error cancellation filtering performed in a filtering unit according to various embodiments.

Referring to FIG. 6, the filtering unit 160 may remove a portion where an error occurs when filtering the points detected as boundary candidates through filtering. The filtering unit 160 can detect peaks in both directions and calculate a width through the following equation (2) to remove peaks less than a predetermined width.

&Quot; (2) "

In FIG. 6, the filtering unit 160 may connect the detected points in the first direction (e.g., from left to right) at the pre-filtering boundary 610. Accordingly, the filtering unit 160 can detect the first peak 601 and the third peak 603. The filtering unit 160 may detect the second peak 602 by connecting the detected points in the second direction (e.g., right to left direction).

Filtering section 160 may determine the width for each peak at boundary 620 where a peak is detected. The filtering unit 160 can remove the peak whose width of the detected peak is smaller than the width determined by the above equation.

For example, when the first peak 601, the second peak 602 and the third peak 603 are detected, the filtering unit 160 maintains the first peak 601 having a width greater than the specified width, and the second peak 601, The peak 602 and the third peak 603 may be removed to obtain the boundary 630. In various embodiments, the filtering unit 160 may further perform a box filtering operation to soften the result of the boundary 630 so that it can be recognized as a continuous boundary.

FIG. 7 is a diagram illustrating an example of applying a filtering process to an actual image according to various embodiments.

Referring to FIG. 7, the filtering unit 160 can detect a peak through a first derivative. The filtering unit 160 can detect peaks by connecting detected points in a first direction (e.g., left to right direction), and connects detected points in a second direction (e.g., right to left direction) to detect peaks can do. Through this, the boundary 710 before filtering can be determined.

The filtering unit 160 can determine the width of each of the detected peaks. The filtering unit 160 can remove a peak 721 of less than a specified width among the detected peaks. The peak 722 of the specified width or more can be detected as a road boundary without being removed. The filtering unit 160 may output the boundary 730. [

8 is a diagram illustrating boundary detection according to various embodiments.

Referring to FIG. 8, the road recognition system 100 may detect a boundary 801 between a road and surrounding objects in each of the images acquired through the stereo camera 110.

The road recognition system 100 can generate and process a U disparity map, and determine the positions of boundary points by referring to the 3D disparity map.

The road recognition system 100 can determine a boundary 801 that can be substantially utilized by the user by detecting only meaningful peaks and eliminating unnecessary peaks through a filtering process.

The road recognition system 100 does not use a separate V disparity map and does not use a complicated algorithm such as RANSAC or Hough transformation used for straight line detection, and thus can maintain a high data processing speed. The processed data can be delivered to the driving support system in real time and can be provided to the user.

Although the present invention has been described in connection with what is presently considered to be preferred embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, The present invention is not limited thereto. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments described above.

100: Road recognition system 110: Stereo camera
120: three-dimensional map generating unit 130: U map generating unit
140: U map processing unit 150:
160: Filtering unit 170: Output unit

Claims (10)

Generating a 3D disparity map for an image acquired through a stereo camera;
Generating a U disparity map in the 3D disparity map;
Processing and simplifying the U disparity map;
Detecting a boundary candidate based on the 3D disparity map in the U disparity map; And
And determining the boundaries of the road by filtering the detected boundary candidates. 2. The method of claim 1, wherein processing and simplifying the U disparity map comprises:
Assigning a distance weight to the U disparity map; And
And binarizing the U disparity map with a threshold value. 3. The method of claim 2, wherein the distance weight
And the distance between the object and the object increases. 3. The method of claim 2,
Wherein the road information is determined based on standard deviation information of a predetermined road area. 3. The method of claim 2, wherein the step of binarizing comprises:
And setting a value of a point whose U disparity value is lower than the threshold to zero. 2. The method of claim 1, wherein detecting the boundary candidate comprises:
Finding a disparity value of a pixel corresponding to the boundary in the designated column; And
And determining a position in the 3D disparity map having the disparity value. 2. The method of claim 1, wherein determining the boundary of the road comprises:
Detecting peaks by connecting points detected by the boundary candidate;
And determining a boundary of the road based on the detected peak width. 8. The method of claim 7, wherein detecting the peak comprises:
Detecting a peak by connecting points detected by the boundary candidate in a first direction; And
And detecting peaks by connecting the points detected by the boundary candidates in a second direction opposite to the first direction. 8. The method of claim 7, wherein determining the boundary of the road based on the detected peak width comprises:
And removing the detected peak when the width of the detected peak is less than a specified range. Stereo camera;
A three-dimensional map generator for generating a three-dimensional disparity map of an image acquired through the stereo camera;
A U map generator for generating a U disparity map in the 3D disparity map;
A U map processing unit for processing and simplifying the U disparity map;
A boundary detector for detecting a boundary candidate based on the 3D disparity map in the U disparity map; And
And a filtering unit for filtering the detected boundary candidates to determine a boundary of the road.

Images