KR101428403B1 - Apparatus and method for detecting obstacle in front - Google Patents

Abstract

The present invention relates to an apparatus and a method for detecting a front obstacle. The apparatus according to the present invention includes: a view converting unit which performed view conversion of a front image and a road surface image which is extracted from the front image based on front depth information which is obtained from a camera or a sensor; a line detecting unit which detects a line component from the view-converted image; an image analyzing unit which separates the background area and road surface area of the front image based on the line component and detects an obstacle by analyzing the road surface area; and an obstacle processing unit which maps the detected obstacle on the front image.

Description

[0001] Apparatus and method for detecting obstacles in front [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to an apparatus and method for detecting a front obstacle, and a technology for detecting an obstacle ahead using three-dimensional information.

The obstacle detection system of the vehicle detects an obstacle ahead, for example, another vehicle or a pedestrian by using a camera or a sensor. In this case, since the camera or the sensor checks whether an obstacle exists in the entire forward image, it takes a long time to process, and unnecessary processing is performed because the obstacle is detected in the background area, and an error May occur.

In addition, when an obstacle is detected using data of a camera or a sensor, there is a problem that the result is largely determined depending on the weather or the illuminance or the surrounding environment.

SUMMARY OF THE INVENTION An object of the present invention is to provide a forward obstacle detection device that detects a road surface region using three-dimensional information obtained from a camera or sensor based on a time of flight (ToF) and detects an obstacle ahead of the detected road surface region, Method.

It is another object of the present invention to provide an apparatus and method for detecting a front obstacle that enables an obstacle to be detected even in a situation where it is difficult for a driver to accurately grasp a situation ahead.

According to an aspect of the present invention, there is provided a front obstacle detecting apparatus comprising a view conversion unit for performing a view conversion of a forward image and an image of a road surface area extracted from the forward image based on front depth information acquired from a camera or a sensor, An image analyzing unit for analyzing the road surface area to separate obstacles from the background area and the road surface area of the forward image based on the line component, And an obstacle processing unit for mapping the forward image.

The view transformation unit transforms the forward image into a v-depth map by projecting the forward image on a v-depth plane based on the v-coordinate using the u and v coordinate information and the depth information on the forward image, And a depth map converting unit.

The view conversion unit may further include a v-disparity map conversion unit that converts the v-depth map into a v-disparity map.

And the line detection unit detects line components from the v-disparity map.

Wherein the view transformation unit projects the image of the road surface region detected from the forward image on the u-depth plane using the u coordinate and the depth information using the u and v coordinate information and the depth information of the forward image, And a u-depth map converting unit for converting the u-depth map into a u-depth map.

The image analyzing unit analyzes an image on the u-depth map and detects a shaded area for the road surface area.

The image analyzing unit detects the obstructed area detected in the road surface area as an obstacle.

The camera or sensor operates on a Time of Flight (ToF) basis.

In addition, the front obstacle detecting apparatus according to the present invention may further include an output unit for outputting the front image to which the detected obstacle is mapped.

According to another aspect of the present invention, there is provided a method of detecting a front obstacle, the method including detecting a forward image based on front depth information acquired from a camera or sensor based on a time of flight (ToF) Converting a v-depth map into a v-disparity map, converting the v-depth map from a v-disparity map to a v-depth map using coordinate information and depth information, Detecting the image of the road surface region from the front image by separating the background region and the road surface region of the forward image on the basis of the detected line component, detecting u and v coordinate information and depth Depth map by analyzing an area of the road surface area projected on the u-depth map and analyzing the obstacle area of the road surface area, Detecting water, and mapping the detected obstacle to the forward image.

According to the present invention, it is possible to detect an obstacle by utilizing three-dimensional information (u, v coordinates and depth) acquired from a camera or sensor based on a time of flight (ToF) There is an advantage that obstacle detection is easy.

In addition, the present invention is advantageous in that obstacle can be more closely detected by detecting a road surface area from a front image using three-dimensional information and detecting an obstacle from an image of the detected road surface area.

1 is a block diagram showing the configuration of a front obstacle detecting apparatus according to the present invention.
FIG. 2 is a block diagram showing a detailed configuration of the view conversion unit of FIG. 1 according to the present invention.
3A to 3G are views for explaining the operation of the front obstacle detecting device according to the present invention.
4 is a flowchart illustrating an operation flow for a front obstacle detection method according to the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a front obstacle detecting apparatus according to the present invention. Referring to FIG. 1, a front obstacle detection apparatus according to the present invention includes a signal processing unit, a camera, a sensor, an output unit, a storage unit, a view conversion unit, a line detection unit, an image analysis unit, and an obstacle processing unit. Here, the signal processing unit processes signals transmitted between respective parts of the front obstacle detecting device.

The camera may be a stereo camera having two or more cameras, and the stereo camera may obtain depth information of the forward image using the parallax of the image captured by each camera. In addition, the camera may be a TOF-based depth camera.

The sensor may be a ToF-based optical sensor, wherein the optical sensor may be an infrared, ultrasonic sensor. At this time, the sensor transmits the optical signal forward, calculates the phase difference of the optical signal reflected from the front, calculates the distance, and can grasp the depth information of the forward image.

At least one of the camera and the sensor outputs the depth information of the captured front to the signal processing unit, and the depth information of the output front is transmitted to the view conversion unit for the view conversion of the forward image.

The output unit may include display means such as a monitor or a screen of a navigation provided on an instrument panel of the vehicle, and may include voice output means such as a speaker or a buzzer.

At this time, the output unit may output the obstacle detection result of the front obstacle detecting device through the display unit and the sound output unit, or may output the obstacle information and the alarm signal when the obstacle is detected in the front.

The storage unit may store set values and the like required for the operation of the front obstacle detecting device, and information captured by the camera or the sensor may be stored. Further, the storage unit may store the processing results of the respective parts of the tetragonal obstacle detection device. For example, the storage unit may store a v-depth map, a v-disparity map, and an u-depth map for a forward image and a forward image acquired in front of the vehicle, and an analysis result of each image may be stored .

The view conversion unit converts view images of the road surface region extracted from the forward image and the forward image based on the front depth information acquired from the camera or the sensor.

At this time, the view transform unit may include a v-depth map transform unit, a v-disparity map transform unit, and a u-depth map transform unit, as shown in FIG.

Here, the v-depth map conversion unit may convert the forward image into a v-depth map by projecting the forward image on a depth plane using u and v coordinate information and depth information on the forward image. In addition, the v-disparity map converts the previously transformed v-depth map into a v-disparity map.

Here, the line detection unit detects a line component from the v-disparity map. The information on the line component detected by the line detection unit may be provided to the image analysis unit and used to detect the road surface area from the forward image.

In other words, the image analyzing unit separates the front image into the background area and the road surface area using the information of the line component detected by the line detecting unit, and detects the image of the road surface area at this time, and provides the image to the view conversion unit. Here, the image analyzing unit maps the detected line component to the forward image considering the position in the image, and determines the area below the line component mapped to the forward image as the road surface area. Therefore, the upper area of the line component mapped to the forward image is determined as the background area.

The view transform unit may further include a u-depth map transform unit. Here, the u-depth map transform unit transforms the image of the road surface region detected from the forward image by the image analysis unit to the u-depth plane using the u and v coordinate information and the depth information of the forward image, can be converted into a u-depth map.

Also, the image analyzing unit analyzes the image on the u-depth map, and detects the obstructed area for the road surface area. At this time, the image analyzing unit can detect the obstacle by analyzing the obstructed area in the image of the road surface area projected on the u-depth map. As an example, if a person is located on the road surface area, the road surface may be covered by the person. Accordingly, the image analyzing unit can determine that an obstacle is located in the obstructed area in the image of the road surface area.

The obstacle processing unit may map the detected obstacle to the forward image and output it through the output unit, if it is confirmed that the obstacle is detected in the image of the road surface area as a result of analysis by the image analysis unit. At this time, the obstacle processing unit can map the obstructed area detected in the road surface area to the forward image.

3A to 3G are views for explaining the operation of the front obstacle detecting device according to the present invention. 3A shows an operation of acquiring three-dimensional information using a phase difference of a front image obtained through a stereo camera, and u and v coordinates and depth information of a forward image can be obtained at this time.

FIG. 3B shows an embodiment in which a forward image is converted into a v-depth map using the three-dimensional information obtained in FIG. 3A. As shown in FIG. 3B, the front obstacle detecting apparatus generates a v-depth map using the u, v coordinates and depth information of the forward image acquired in FIG. 3A, and the v-depth map of the v- The v-depth map corresponding to the forward image can be generated.

FIG. 3C shows an embodiment in which the v-depth map generated in FIG. 3B is converted into a v-disparity map. As shown in Fig. 3C, the front obstacle detecting device converts the v-depth map into the v-disparity map. At this time, the front obstacle detection apparatus can convert the v-depth map into the v-disparity map using the image conversion algorithm. Here, the technique of converting the v-depth map into the v-disparity map is a technique generally used in the field of image conversion, and a detailed description thereof will be omitted.

The v-disparity map may have line components. Therefore, the front obstacle detecting apparatus can be applied to detecting the line component from the v-disparity map and detecting the road surface area.

FIG. 3D shows an embodiment in which a front image is divided into a background area and a road surface area. As shown in FIG. 3D, the front obstacle detecting apparatus maps the line component detected in FIG. 3C to the forward image, and separates the upper background region and the lower road surface region. Here, the front obstacle detecting apparatus can acquire the image of the road surface region by detecting the lower region based on the line component mapped to the forward image.

FIG. 3E shows an embodiment in which a forward image is converted into an u-depth map. As shown in FIG. 3E, the front obstacle detecting apparatus generates the u-depth map using u, v coordinates and depth information of the forward image acquired in FIG. 3A. Specifically, the front obstacle detection apparatus can generate the u-depth map by utilizing the three-dimensional coordinates of the image of the road surface area detected from the forward image. At this time, the front obstacle detection device can generate an u-depth map corresponding to the image of the road surface area by projecting the image of the road surface area to the u-depth plate of the u-depth map.

FIG. 3F shows an embodiment in which an obstacle region is detected from an image of a u-depth map. As shown in FIG. 3F, the front obstacle detecting apparatus analyzes the image of the road surface region projected on the u-depth map, and detects the obstructed area by the obstacle or the like. At this time, the front obstacle detecting apparatus identifies the obstructed area through the scanning of the line from the long distance in the image of the road surface area projected on the u-depth map, and determines the obstructed area as the obstructed area.

FIG. 3G shows an embodiment in which the obstacle region is mapped to the forward image. As shown in FIG. 3G, the front obstacle detecting device maps the forward obstacle region (original image) in consideration of the position of the obstacle region detected in FIG. 3F, thereby allowing the position of the obstacle region to be confirmed through the forward image .

In this way, the front obstacle detecting apparatus according to the present invention detects obstacles by using three-dimensional information on the forward image, maps the obstacles to the original image, and outputs the obstacles, so that even when the user can not grasp the forward situation, It becomes possible to grasp the position of the obstacle through the obstacle area.

The operation flow of the front obstacle detecting apparatus according to the present invention will now be described in detail.

4 is a flowchart illustrating an operation flow for a front obstacle detection method according to the present invention. Referring to FIG. 4, the front obstacle detection apparatus acquires a forward image (S100), and extracts forward depth information using a camera or a sensor (S110).

The front obstacle detecting apparatus converts the forward image acquired in the process 'S110' into the v-depth map using the u and v coordinates of the forward image and the forward depth information extracted in the process 'S110' (S120) Depth map is converted into a v-disparity map (S130). Here, the front obstacle detection apparatus can detect the line information from the v-disparity map in step S130 (S140), map the line information extracted in step S140 to the forward image acquired in step S100 The road surface area and the background area are separated based on the line information (S150).

In addition, the front obstacle detecting apparatus generates the u-depth map using the three-dimensional information of the image of the road surface region in step S150 (S160), and adds the u-depth map to the u-depth map generated in step S160 The image is projected (S170) and analyzed (S180). Here, the front obstacle detecting apparatus detects the obstructed area of the road surface area from the analysis result of step S180 and detects the obstructed area as the obstacle area (S190).

Finally, the front obstacle detecting device maps the obstacle area detected in the process 'S190' to the original image (S200), and outputs an image in which the obstacle area is mapped (S210).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, .

110: signal processing unit 120: camera
125: sensor 130:
140: storage unit 150: view conversion unit
151: v-depth map conversion unit 153: v-disparity map conversion unit
155: u-depth map conversion unit 160:
170: Image analysis unit 180: Obstacle processing unit

Claims (10)

A view conversion unit for performing a view conversion of a forward image and an image of a road surface area extracted from the forward image based on the depth information acquired from a camera or a sensor;
A line detector for detecting a line component from the view-converted image;
An image analyzer for separating a background region and a road surface region of the forward image based on a line component and analyzing the road surface region to detect an obstacle; And
And an obstacle processing unit for mapping the detected obstacle to the forward image. The method according to claim 1,
The view converter may further include:
A v-depth map transforming unit for transforming the forward image into a v-depth map by projecting the forward image onto a v-depth plane based on the v-coordinate using u and v coordinate information and depth information of the forward image, Wherein the front obstacle detecting device comprises: The method of claim 2,
The view converter may further include:
And a v-disparity map converting unit for converting the v-depth map into a v-disparity map. The method of claim 3,
Wherein the line detecting unit comprises:
And detects a line component from the v-disparity map. The method according to claim 1,
The view converter may further include:
The image of the road surface region detected from the forward image is projected onto the u-depth plane using the u coordinate and the depth information using the u and v coordinate information and the depth information of the forward image, And a u-depth map converting unit for converting the u-depth map into a u-depth map. The method of claim 5,
Wherein the image analyzing unit comprises:
Wherein the obstacle detecting unit detects an obstacle area for the road surface area by analyzing the image on the u-depth map. The method of claim 6,
Wherein the image analyzing unit comprises:
And detects the obstacle area detected in the road surface area as an obstacle. The method according to claim 1,
The camera or sensor may comprise:
And a time-of-flight (ToF) basis. The method according to claim 1,
And an output unit for outputting the information of the front image or the detected obstacle to which the detected obstacle is mapped. Based on the front depth information acquired from a camera or sensor based on a ToF (Time of Flight), the front image is converted into a v-depth map using the u and v coordinate information and the depth information of the front image step;
Converting the v-depth map into a v-disparity map;
Detecting a line component from the v-disparity map;
Detecting an image of a road surface region from the forward image by separating a background region and a road surface region of the forward image based on the detected line component;
Converting the detected image of the road surface region into u-depth map using u, v coordinate information and depth information of the forward image;
Analyzing an obstructed area of the image of the road surface region projected on the u-depth map to detect an obstacle; And
And mapping the detected obstacle to the forward image.

Images