KR20170079373A - Apparatus for object detection on the road and method thereof - Google Patents

Apparatus for object detection on the road and method thereof Download PDF

Info

Publication number
KR20170079373A
KR20170079373A KR1020150189849A KR20150189849A KR20170079373A KR 20170079373 A KR20170079373 A KR 20170079373A KR 1020150189849 A KR1020150189849 A KR 1020150189849A KR 20150189849 A KR20150189849 A KR 20150189849A KR 20170079373 A KR20170079373 A KR 20170079373A
Authority
KR
South Korea
Prior art keywords
road
road information
estimated
detecting
depth image
Prior art date
Application number
KR1020150189849A
Other languages
Korean (ko)
Other versions
KR101756698B1 (en
Inventor
임영철
강민성
Original Assignee
재단법인대구경북과학기술원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 재단법인대구경북과학기술원 filed Critical 재단법인대구경북과학기술원
Priority to KR1020150189849A priority Critical patent/KR101756698B1/en
Publication of KR20170079373A publication Critical patent/KR20170079373A/en
Application granted granted Critical
Publication of KR101756698B1 publication Critical patent/KR101756698B1/en

Links

Images

Classifications

    • G06K9/00805—
    • G06K9/3233—
    • G06K9/4633—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The present invention relates to an apparatus and method for detecting an object on a road.
An apparatus for detecting an object on a road according to the present invention includes a depth image generator for generating a depth image of the road from a left and right image of a road taken from a stereo camera; Estimating the road information by using the depth image of the generated road, and estimating the road information by reflecting previously estimated road information when it is determined that an error occurred during the road information estimation; An interest region extractor for extracting a region of interest to be searched using the estimated road information; And an object detecting unit for searching for the extracted ROI and detecting an object existing on the road.
As described above, according to the present invention, when an error occurs in the current image while estimating the road information for detecting an object on the road, the posterior probability is calculated using the road information estimated from the previous image and the road information estimated from the current image The object detection speed and the detection performance are improved as the search area is reduced by searching for a specific object such as a pedestrian or a vehicle in the ROI accurately extracted from the road information estimated to be high.

Description

[0001] APPARATUS FOR OBJECT DETECTION ON THE ROAD AND METHOD THEREOF [0002]

The present invention relates to an apparatus and method for detecting an object on a road, and more particularly, to an apparatus and method for detecting an object on a road, in which, when a loss occurs in the road information estimated in the current image, And a method for detecting an object on the road.

Conventionally, a method using a single camera has been mainly used for detecting an object on a road. FIG. 1 is a diagram for explaining an object detection process using a conventional single camera. In order to detect an object using a single camera, as shown in FIG. 1, a method of detecting an object by searching all regions in the image using a multi-scale sliding window technique is used. Since this method has a large number of search regions, it requires a long calculation time and increases the undetectable rate.

In order to solve this problem, in recent years, many researches have been conducted on a method of detecting an object on the road using a stereo camera in the fields of intelligent automobiles and robots.

The object detection method using a stereo camera has an advantage of extracting a depth image, estimating road information, and searching for an object only in a region of interest on the road, thereby drastically reducing the search area. As a result, the object detection speed and the detection performance can be improved at the same time.

In order to provide excellent performance while maintaining the fast searching speed, it is most important to accurately extract the region of interest by robustly estimating the road information in various external environments. However, in general, a depth image calculated from a stereo image causes a stereo image matching error due to a difference in brightness between left and right pixels, a difference in viewpoint between left and right cameras, and a shielding. An estimation error occurs.

FIG. 2 is a view showing a stereo matching error due to shielding in an object detecting process using a conventional stereo camera, and FIG. 3 is an example of a road information loss caused by a front obstacle in the object detecting process using a conventional stereo camera FIG.

That is, as the stereo matching error occurs due to the partial shielding as shown in FIG. 2, an estimation error may also occur when estimating the road information. In addition, as shown in FIG. 3, when there are a large number of obstacle objects in front, it is difficult to accurately estimate the road information from the depth image because the road information is lost. ROI: region of interest), which may cause a problem of increasing errors of object detection as a whole.

The technology of the background of the present invention is disclosed in Korean Patent Registration No. 10-0959246 (published on May 20, 2010).

According to an aspect of the present invention, there is provided a method for estimating road information for detecting an object on a road, the method comprising the steps of: And an object and a method for detecting an object on the road that detects a specific object such as a pedestrian or a vehicle by searching in a region of interest accurately extracted from the road information estimated to be the highest.

According to an aspect of the present invention, there is provided an apparatus for detecting an object on a road, the apparatus comprising: a depth image generator for generating a depth image of the road from a left and right image of a road taken by a stereo camera; Estimating the road information by using the depth image of the generated road, and estimating the road information by reflecting previously estimated road information when it is determined that an error occurred during the road information estimation; An interest region extractor for extracting a region of interest to be searched using the estimated road information; And an object detecting unit for searching for the extracted ROI and detecting an object existing on the road.

The road information estimating unit generates a v-disparity map by accumulating in the y-axis using the generated depth image, and calculates a v-disparity map by using a Hough transformation algorithm for a portion corresponding to the gradient in the generated v-disparity map And estimates the road information.

The road information estimating unit estimates the road information by reflecting the road information estimated by the depth image of the road generated before the depth image of the generated road, when it is determined that an error has occurred during the road information estimation, The road information may be updated by assigning weights according to the degree of similarity.

The ROI extractor may extract the ROI using the stixel technique that expresses the estimated road information and the image in a group of a predetermined width.

Wherein the object detection unit searches the ROI to determine whether or not an object exists on the road, and when it is determined that the object exists, The type of object can be detected.

According to another aspect of the present invention, there is provided a method of detecting an object on a road using an object detection apparatus, the method comprising: generating a depth image of the road from a left and right image of a road taken by a stereo camera; Estimating the road information by using the depth image of the road; estimating the road information by reflecting previously estimated road information when an error occurs during the road information estimation; Extracting a region of interest to be searched using the estimated road information; And detecting an object existing on the road by searching the extracted ROI.

As described above, according to the present invention, when an error occurs in the current image while estimating the road information for detecting an object on the road, the posterior probability is calculated using the road information estimated from the previous image and the road information estimated from the current image The object detection speed and the detection performance are improved as the search area is reduced by searching for a specific object such as a pedestrian or a vehicle in the ROI accurately extracted from the road information estimated to be high.

In addition, the present invention can accurately extract a region of interest even when a stereo matching error occurs when processing left and right images photographed using a stereo vision camera or when there is a large loss of road information due to a front obstacle, It is possible to accurately detect a specific object by determining the presence or absence of the object using the classifier learned in advance.

Further, the present invention can be applied to the fields of an unmanned vehicle, a robot, and the like that can detect an object such as a front pedestrian or a vehicle, and estimate a road area without an obstacle by autonomous travel.

FIG. 1 is a diagram for explaining an object detection process using a conventional single camera.
FIG. 2 is a diagram showing a stereo matching error due to shielding in an object detection process using a conventional stereo camera.
3 is a diagram illustrating an example of road information loss caused by a forward obstacle in the object detection process using a conventional stereo camera.
4 is a block diagram showing an apparatus for detecting objects on a road according to an embodiment of the present invention.
5 is a flowchart illustrating an operational flow of a method for detecting an object on a road according to an embodiment of the present invention.
6 is a flowchart showing a detailed operation flow of a road information estimation step of a method for detecting an object on a road according to an embodiment of the present invention.
7 is a diagram for explaining a process of estimating stereo-based road information in a method for detecting an object on a road according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An apparatus and method for detecting an object on a road according to an embodiment of the present invention will be described with reference to the accompanying drawings. In this process, the thicknesses of the lines and the sizes of the components shown in the drawings may be exaggerated for clarity and convenience of explanation.

Further, the terms described below are defined in consideration of the functions of the present invention, which may vary depending on the intention or custom of the user, the operator. Therefore, definitions of these terms should be made based on the contents throughout this specification.

First, an apparatus for detecting an object on a road according to an embodiment of the present invention will be described with reference to FIG.

4 is a block diagram showing an apparatus for detecting objects on a road according to an embodiment of the present invention.

4, an apparatus 100 for detecting an object on a road according to an exemplary embodiment of the present invention includes a depth image generator 110, a road information estimator 120, a ROI extractor 130, And a detection unit 140.

The depth image generation unit 110 generates a depth image of a road from the left and right images of the road taken from a stereo camera (not shown).

In detail, the road gradient is extracted using the left and right images of the road taken from the stereo camera, and a depth image is generated using the stereo matching algorithm.

The road information estimating unit 120 estimates the road information using the depth image of the road generated by the depth image generating unit 110. If the road information estimating unit 120 determines that an error has occurred during the road information estimation, Thereby estimating road information.

More specifically, the depth image generated by the depth image generation unit 110 is accumulated on the y-axis to generate a v-disparity map. At this time, the road information is estimated using the Hough transformation algorithm or the RANSAC algorithm for the portion corresponding to the slope in the generated v-disparity map.

At this time, when it is determined that an error has occurred during the road information estimation, the road information estimating unit 120 estimates the road information by reflecting the road information estimated by the depth image of the road generated before the depth image of the generated road, The road information is updated and estimated by assigning weights according to the degree of similarity.

In this case, a case where the road information can not be estimated, such as an error in the stereo matching due to the partial shielding or a road information loss due to the presence of a large number of obstacle objects ahead, can be defined as an error occurrence determination element.

The ROI extracting unit 130 extracts a region of interest (ROI) to be searched using the road information estimated by the ROI estimating unit 120.

That is, the present invention has an effect of extracting a region of interest to be searched and improving object detection speed and detection performance as the search region is reduced.

At this time, the ROI extracting unit 130 may extract ROIs estimated by the ROI estimation unit 120 and ROIs using a stixel technique.

Here, the stick cell technique refers to a method of representing an image in the same group as a stick of a certain width instead of a pixel on the assumption that obstacle objects standing upright on the road have the same parallax information, May be extracted.

The object detecting unit 140 detects an object existing on the road by searching the ROI extracted by the ROI extracting unit 130.

In detail, the presence or absence of an object existing on the road is searched by searching the ROI, and if it is determined that the object exists, the type of the object corresponding to the object determined using the pre- .

At this time, the presence or absence of an object may be determined using a model learned by machine learning (for example, a classifier such as SVM (support vector machine), boosting, neural network, etc.).

For example, it is possible to determine whether an object exists by using a model learned by machine learning, and to detect the kind of an object such as a person and an automobile when the object exists.

Hereinafter, a method for detecting an object on a road according to an embodiment of the present invention will be described with reference to FIGs. 5 and 6. FIG.

FIG. 5 is a flowchart showing an operational flow of a method for detecting an object on a road according to an embodiment of the present invention, and a specific operation of the present invention will be described with reference to the flowchart.

According to the method for detecting an object on the road according to the embodiment of the present invention, the depth image generator 110 of the object detecting apparatus 100 generates a depth image of the road from the left and right images of the road taken from the stereo camera (S510).

6 is a diagram for explaining a process of estimating stereo-based road information in a method for detecting an object on a road according to an embodiment of the present invention.

More specifically, in step S510, the slope (a) of the road is extracted using the left and right images of the road taken from the stereo camera, and a depth image of the road is generated using the stereo matching algorithm.

Next, the road information estimating unit 120 estimates road information using the depth image of the road generated in step S510 (S520).

6 is a flowchart showing a detailed operation flow of a road information estimation step of a method for detecting an object on a road according to an embodiment of the present invention.

More specifically, as shown in FIG. 6, the depth image generated in step S510 is accumulated in the y-axis to generate a v-disparity map (S521). At this time, the road information is estimated using the Hough transformation algorithm or the RANSAC algorithm for the portion corresponding to the slope in the generated v-disparity map (S522).

If it is determined that an error has occurred during the road information estimation, the road information estimation unit 120 reflects the road information estimated by the depth image of the road generated before the depth image of the road generated in step S510, And estimates the road information by assigning weights according to the similarity to the observations.

Here, the case where the road information can not be estimated, such as the occurrence of an error in the stereo matching due to the partial shielding or the loss of the road information due to the presence of a large number of obstacle objects ahead, can be defined as an error occurrence determination element.

In addition, the road information estimating unit 120 estimates the road information by using two observation variables

Figure pat00001
N, which are statistically higher in frequency, are selected (S523). N selected variables
Figure pat00002
And previous state variables
Figure pat00003
The posterior probability, that is, the road information to be updated
Figure pat00004
Is calculated as shown in the following equation (1), and the current state variable
Figure pat00005
(S524).

Figure pat00006

here,

Figure pat00007
Represents a normalization constant,
Figure pat00008
N variables
Figure pat00009
And current state variables
Figure pat00010
Pre-observations of
Figure pat00011
And a weight corresponding to the distance from the center.

That is, the current road information is updated by assigning a larger weight to the observation variables that are close to the previous observation value of the previous state variable (S525).

At this time,

Figure pat00012
Is the frequency of voting in the Hough transform
Figure pat00013
Is calculated by the relative probability as shown in Equation (2) below.

Figure pat00014

That is, it is determined that the slope a having a high frequency is highly reliable with respect to the slope a, and the road information is updated and estimated using the slope.

At this time, the updated road information can be modeled with a Gaussian probability, on the assumption that the road gradient does not change abruptly.

Then, the ROI extractor 130 extracts a region of interest (ROI) to be searched using the road information estimated in operation S520 in operation S530.

That is, the present invention has an effect of extracting a region of interest to be searched and improving object detection speed and detection performance as the search region is reduced.

At this time, the ROI extracting unit 130 may extract ROIs estimated by the ROI estimation unit 120 and ROIs using a stixel technique.

Here, the stick cell technique refers to a method of representing an image in the same group as a stick of a certain width instead of a pixel on the assumption that obstacle objects standing up on the road have the same parallax information. May be extracted.

Finally, the object detection unit 140 searches the ROI extracted by the ROI extraction unit 130 to detect an object existing on the road (S540).

In detail, the presence or absence of an object existing on the road is searched by searching the ROI, and if it is determined that the object exists, the type of the object corresponding to the object determined using the pre- .

At this time, the presence or absence of an object may be determined using a model learned by machine learning (for example, a classifier such as SVM (support vector machine), boosting, neural network, etc.).

For example, it is possible to determine whether an object exists by using a model learned by machine learning, and to detect the kind of an object such as a person and an automobile when the object exists.

As described above, an apparatus and method for detecting an object on a road according to an embodiment of the present invention can detect an object on the road when an error occurs in the current image while estimating road information for detecting an object on the road, By searching for a specific object such as a pedestrian or a vehicle in a region of interest accurately extracted from the road information estimated to have the highest posterior probability using the road information and the road information estimated from the current image, Speed and detection performance can be improved.

In addition, it is possible to accurately extract a region of interest even when a stereo matching error occurs when processing left and right images photographed using a stereo vision camera, or when road information is lost due to a front obstacle, The presence or absence of the object is determined by using the learned classifier, so that the specific object can be accurately detected.

In addition, the present invention can be applied to the fields of an unmanned vehicle, a robot, and the like that can detect an object such as a forward pedestrian or a vehicle and estimate a road area without an obstacle.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined by the appended claims. will be. Accordingly, the true scope of the present invention should be determined by the following claims.

100: object detecting apparatus 110: depth image generating unit
120: Road information estimation unit 130: ROI extracting unit
140: Object detection unit 150:

Claims (10)

A depth image generator for generating a depth image of the road from a left and right image of a road taken from a stereo camera;
Estimating the road information by using the depth image of the generated road, and estimating the road information by reflecting previously estimated road information when it is determined that an error occurred during the road information estimation;
An interest region extractor for extracting a region of interest to be searched using the estimated road information; And
And an object detection unit searching the extracted ROI to detect an object existing on the road. The method according to claim 1,
The road information estimating unit estimates,
A v-disparity map is generated by accumulating in the y-axis using the generated depth image, and a portion corresponding to the gradient in the generated v-disparity map is transformed using a Hough transformation algorithm, The object detection apparatus comprising: 3. The method of claim 2,
The road information estimating unit estimates,
And estimating the road information by reflecting the road information estimated by the depth image of the road generated before the depth image of the generated road when the error is estimated to have occurred during the road information estimation. And updates the road information to estimate the road information. The method according to claim 1,
Wherein the ROI extractor comprises:
And extracts the ROI from the estimated ROI using a stixel technique that expresses the estimated road information and an image in a group of a predetermined width. The method according to claim 1,
Wherein the object detection unit comprises:
The method comprising the steps of: detecting an existence of an object existing on the road by searching the ROI, determining whether the object exists, detecting a type of the object corresponding to the object using the pre- The object detection apparatus comprising: A method for detecting an object on a road using an object detecting apparatus,
Generating a depth image of the road from left and right images of a road taken from a stereo camera;
Estimating the road information by using the depth image of the road; estimating the road information by reflecting previously estimated road information when an error occurs during the road information estimation;
Extracting a region of interest to be searched using the estimated road information; And
And detecting an object existing on the road by searching the extracted ROI. The method according to claim 6,
The step of estimating the road information includes:
A v-disparity map is generated by accumulating in the y-axis using the generated depth image, and a portion corresponding to the gradient in the generated v-disparity map is transformed using a Hough transformation algorithm, Of the object. 8. The method of claim 7,
The step of estimating the road information includes:
And estimating the road information by reflecting the road information estimated by the depth image of the road generated before the depth image of the generated road when the error is estimated to have occurred during the road information estimation. And estimates the road information by updating the road information. The method according to claim 6,
Wherein the extracting of the ROI comprises:
And extracting the ROI from the estimated ROI using a stixel technique that expresses the estimated road information and an image in a group of a predetermined width. The method according to claim 6,
Wherein detecting the object comprises:
The method comprising the steps of: detecting an existence of an object existing on the road by searching the ROI, determining whether the object exists, detecting a type of the object corresponding to the object using the pre- Object detection method.
KR1020150189849A 2015-12-30 2015-12-30 Apparatus for object detection on the road and method thereof KR101756698B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150189849A KR101756698B1 (en) 2015-12-30 2015-12-30 Apparatus for object detection on the road and method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150189849A KR101756698B1 (en) 2015-12-30 2015-12-30 Apparatus for object detection on the road and method thereof

Publications (2)

Publication Number Publication Date
KR20170079373A true KR20170079373A (en) 2017-07-10
KR101756698B1 KR101756698B1 (en) 2017-07-12

Family

ID=59352753

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150189849A KR101756698B1 (en) 2015-12-30 2015-12-30 Apparatus for object detection on the road and method thereof

Country Status (1)

Country Link
KR (1) KR101756698B1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101977503B1 (en) * 2017-12-11 2019-08-28 재단법인대구경북과학기술원 Method and apparatus of detecting object based on distribution of road components
CN111284501A (en) * 2018-12-07 2020-06-16 现代自动车株式会社 Apparatus and method for managing driving model based on object recognition, and vehicle driving control apparatus using the same
RU2730687C1 (en) * 2018-10-11 2020-08-25 Тиндей Нетворк Технолоджи (Шанхай) Ко., Лтд. Stereoscopic pedestrian detection system with two-stream neural network with deep training and methods of application thereof
KR102476374B1 (en) * 2021-07-16 2022-12-08 세종대학교산학협력단 System for remote control of unmanned aerial vehicle and method performed thereby

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101371275B1 (en) * 2012-11-05 2014-03-26 재단법인대구경북과학기술원 Method for multiple object tracking based on stereo video and recording medium thereof

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101977503B1 (en) * 2017-12-11 2019-08-28 재단법인대구경북과학기술원 Method and apparatus of detecting object based on distribution of road components
RU2730687C1 (en) * 2018-10-11 2020-08-25 Тиндей Нетворк Технолоджи (Шанхай) Ко., Лтд. Stereoscopic pedestrian detection system with two-stream neural network with deep training and methods of application thereof
CN111284501A (en) * 2018-12-07 2020-06-16 现代自动车株式会社 Apparatus and method for managing driving model based on object recognition, and vehicle driving control apparatus using the same
KR102476374B1 (en) * 2021-07-16 2022-12-08 세종대학교산학협력단 System for remote control of unmanned aerial vehicle and method performed thereby

Also Published As

Publication number Publication date
KR101756698B1 (en) 2017-07-12

Similar Documents

Publication Publication Date Title
CN105023278B (en) A kind of motion target tracking method and system based on optical flow method
US10395377B2 (en) Systems and methods for non-obstacle area detection
CN107851318B (en) System and method for object tracking
KR102462278B1 (en) Method for estimating ego motion of an object
KR101756698B1 (en) Apparatus for object detection on the road and method thereof
JP2016081525A (en) Vehicular image recognition system and corresponding method
Llorca et al. Stereo regions-of-interest selection for pedestrian protection: A survey
EP2840528B1 (en) Method and apparatus for tracking object
CN111322993B (en) Visual positioning method and device
KR101988551B1 (en) Efficient object detection and matching system and method using stereo vision depth estimation
CN103077521A (en) Area-of-interest extracting method used for video monitoring
Lin et al. Detecting moving objects using a camera on a moving platform
KR101901487B1 (en) Real-Time Object Tracking System and Method for in Lower Performance Video Devices
JP5228148B2 (en) Position estimation method, position estimation apparatus, and position estimation program for estimating position from image data
Ozgunalp Robust lane‐detection algorithm based on improved symmetrical local threshold for feature extraction and inverse perspective mapping
CN104240265B (en) Multi-target tracking method and system based on global restrictions
Huang et al. Multi‐class obstacle detection and classification using stereovision and improved active contour models
KR101910256B1 (en) Lane Detection Method and System for Camera-based Road Curvature Estimation
KR102172849B1 (en) Detecting system for approaching vehicle in video and method thereof
Ramirez et al. Go with the flow: Improving Multi-View vehicle detection with motion cues
Karthiprem et al. Recognizing the moving vehicle while driving on Indian roads
Agunbiade et al. Enhancement optimization of drivable terrain detection system for autonomous robots in light intensity scenario
Manoharan et al. Detection of unstructured roads from a single image for autonomous navigation applications
Chang et al. Real-time vehicle detection and tracking algorithm for forward vehicle collision warning
Murase Image recognition for driver assistance in intelligent vehicles

Legal Events

Date Code Title Description
GRNT Written decision to grant