KR20200102108A - Apparatus for detecting object of vehicle and method thereof - Google Patents

Abstract

The present invention relates to an apparatus for detecting an object of a vehicle and a method thereof. The apparatus comprises: a camera pose estimation unit which estimates camera pose information by converting vehicle pose information acquired through a vehicle wheel speed measurement system into a coordinate system of a camera of the vehicle; a feature point tracking unit which extracts a point existing on the image acquired through the camera, projects the point on a reference line existing on the image, and tracks, as a feature point, the point projected on the reference line according to a predefined feature point tracking algorithm; a three-dimensional environment map generating unit which calculates three-dimensional coordinates of the plurality of feature points tracked by the feature point tracking unit using the camera pose information estimated by the camera pose estimation unit and generates a three-dimensional environment map based on a plurality of three-dimensional feature points having the calculated three-dimensional coordinates; and a stationary object detection unit which clusters the three-dimensional feature points by projecting, onto a reference plane, the three-dimensional feature points on the three-dimensional environment map generated by the three-dimensional environment map generating unit, and detects a stationary object around the vehicle based on the clustering result.

Description

Vehicle object detection device and method {APPARATUS FOR DETECTING OBJECT OF VEHICLE AND METHOD THEREOF}

The present invention relates to an apparatus and method for detecting an object of a vehicle, and more particularly, to an apparatus and method for detecting an object of a vehicle for detecting a stationary object and a moving object existing on a trajectory of a vehicle.

As a technology for detecting an object using a conventional camera, an object by restoring 3D points of pixels observed in an image image acquired through multiple cameras or by classifying 3D points acquired through cameras and other distance detection sensors. A method of detecting is applied. When an object is detected using a monocular camera, a method of detecting an object by constructing a 3D point of matched pixels by tracking pixels in a continuous image for each image is used. A method of tracking pixels in a continuous image includes a method of tracking a feature point through a Kanade-Lucas-Tomasi (KLT) tracking algorithm, or a method of extracting and matching a feature point through a Scale-Invariant Feature Transform (SIFT) algorithm. The feature point matching method is also used to match feature points between images at the same moment in multiple cameras.

On the other hand, if you want to detect an object using only one camera mounted on the vehicle, you can detect the object on the 2D image acquired through the camera, but the accurate distance information about how far the object is actually from the vehicle is There is a problem that cannot be obtained. In addition, the conventional technology that uses the camera and the vehicle's speed measurement system together has difficulty in detecting objects such as obstacles because the system is operated based on the aforementioned KLT Tracking algorithm or SIFT algorithm, and all pixels present in the image are used. Conventional techniques using a dense method have a disadvantage in that the time required for object detection is very large.

The background technology of the present invention is disclosed in Korean Patent Application Publication No. 10-2011-0060600 (published on June 08, 2011).

The present invention was invented to solve the above-described problem, and an object according to an aspect of the present invention is to construct a 3D environment map around a vehicle using an image image acquired through a vehicle camera and a vehicle wheel speed measurement system. And, based on the established 3D environment map, the vehicle object detection apparatus and method for improving the driving convenience of the driver by precisely detecting the stationary and moving objects around the vehicle through a method of detecting objects on the vehicle trajectory. To provide.

The object detection apparatus of a vehicle according to an embodiment of the present invention is a camera pose estimating unit for estimating camera pose information by converting the vehicle pose information obtained through the vehicle wheel speed measurement system into the coordinate system of the camera of the vehicle, A feature point tracking unit for extracting a point existing in the image acquired through the camera and projecting it onto a reference line existing in the image, and tracking the point projected on the reference line as a feature point according to a predefined feature point tracking algorithm, the The three-dimensional coordinates of the plurality of feature points tracked by the feature point tracking unit are calculated using the camera pose information estimated by the camera pose estimating unit, and based on the plurality of three-dimensional feature points having the calculated three-dimensional coordinates, 3 A 3D environment map generation unit that generates a dimensional environment map, and the 3D feature points on the 3D environment map generated by the 3D environment map generation unit are clustered through a method of projecting onto a reference plane, and based on the result of the clustering It characterized in that it comprises a stationary object detection unit for detecting a stationary object around the vehicle.

In the present invention, the camera pose estimating unit may estimate the camera pose information by applying an external parameter of the camera to the vehicle pose information determined based on a rear wheel axis of the vehicle.

In the present invention, the feature point tracking unit is characterized in that the feature point is tracked by extracting a point on an edge existing in the image and projecting it onto the reference line.

In the present invention, the feature point tracking unit projects a point on the edge onto an epipolar line, which is the reference line, and a point projected onto the epipolar line within a range satisfying an epipolar constraint. Is tracked according to the KLT (Kanade-Lucas-Tomasi) Tracking algorithm, which is the feature point tracking algorithm.

In the present invention, the 3D environment map generating unit is characterized in that after filtering the plurality of 3D feature points through reliability according to an error, and then generating the 3D environment map using the filtered 3D feature points. .

In the present invention, the reliability is determined based on the angle between the epipolar line and the edge and the number of images tracked by the feature point tracking unit.

In the present invention, the reference plane is determined as a ground plane on which a grid is formed, and the still object detection unit includes only three-dimensional feature points existing on the reference plane among three-dimensional feature points on the three-dimensional environment map. It is characterized by detecting a stationary object around the vehicle by projecting onto the reference plane and calculating the number of projected 3D feature points for each grid, and then classifying each grid based on the calculation result.

In the present invention, the stationary object detection unit classifies each grid using a Connected Component algorithm based on the number of three-dimensional feature points calculated for each grid, and determines that a grid having the same component corresponds to the same object and stops. The object is detected, but the position and height of the corresponding still object are calculated based on the position of the corresponding grid and the number of 3D feature points projected on the corresponding grid to detect the corresponding still object.

In the present invention, the camera pose estimating unit tracks a feature point present in the image acquired through the camera according to a KLT Tracking algorithm, optimizes the camera pose information based on the tracked feature point, and calculates the camera pose information. In the optimization process, it is characterized in that it further comprises a moving object detection unit for detecting a moving object around the vehicle based on the feature points tracked according to the KLT Tracking algorithm.

In the present invention, the moving object detection unit accumulatively analyzes the difference between the position coordinates and the three-dimensional coordinates of the pixel on the image with respect to the feature point tracked according to the KLT Tracking algorithm, and detects the feature point estimated as a moving object to move. When the score is increased and the movement score counted for each grid formed in the image exceeds a preset reference value, the corresponding grid is determined to correspond to a moving object, and the moving object is detected.

In the method of detecting an object of a vehicle according to an aspect of the present invention, the camera pose estimating unit converts vehicle pose information obtained through a wheel speed measurement system of the vehicle into a coordinate system of the vehicle camera to estimate camera pose information. , A feature point tracking unit extracts a point existing in the image acquired through the camera and projects it onto a reference line existing in the image, and tracks the point projected on the reference line as a feature point according to a predefined feature point tracking algorithm. Step, the 3D environment map generation unit calculates the 3D coordinates of the plurality of feature points tracked by the feature point tracking unit using the camera pose information estimated by the camera pose estimation unit, and calculates the calculated 3D coordinates Generating a three-dimensional environment map based on a plurality of three-dimensional feature points, and a method in which the still object detection unit projects three-dimensional feature points on the three-dimensional environment map generated by the three-dimensional environment map generator onto a reference plane And detecting a stationary object around the vehicle based on the clustering result.

According to an aspect of the present invention, the present invention constructs a 3D environment map around the vehicle using an image image acquired through a vehicle camera and a vehicle wheel speed measurement system, and based on the constructed 3D environment map, the vehicle The driving convenience of a driver can be improved by precisely detecting a stationary object and a moving object around the vehicle through a method of detecting an object on a trajectory.

In addition, the present invention is applied to an autonomous driving system or a parking assistance system of a vehicle to detect an object such as an obstacle around the vehicle and to accurately control the autonomous driving or parking of the vehicle based on the detection result. And parking control performance may be improved.

1 is a block diagram illustrating an apparatus for detecting an object of a vehicle according to an exemplary embodiment of the present invention.
2 and 3 are exemplary diagrams for explaining an operation of a camera pose estimation unit in the apparatus for detecting an object of a vehicle according to an embodiment of the present invention.
4 is an exemplary view for explaining the operation of a feature point tracking unit in the object detection apparatus of a vehicle according to an embodiment of the present invention.
5 to 7 are exemplary views for explaining the operation of a stationary object detection unit in the object detection apparatus of a vehicle according to an embodiment of the present invention.
8 and 9 are exemplary views for explaining an operation of a moving object detection unit in the object detection apparatus of a vehicle according to an embodiment of the present invention.
10 is a flowchart illustrating a method of detecting an object of a vehicle according to an embodiment of the present invention.

Hereinafter, an embodiment of an apparatus and method for detecting an object of a vehicle according to the present invention will be described with reference to the accompanying drawings. In this process, the thickness of the lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of description. In addition, terms to be described later are terms defined in consideration of functions in the present invention and may vary according to the intention or custom of users or operators. Therefore, definitions of these terms should be made based on the contents throughout the present specification.

1 is a block diagram illustrating an object detection apparatus of a vehicle according to an embodiment of the present invention, and FIGS. 2 and 3 are an operation of a camera pose estimation unit in an object detection apparatus of a vehicle according to an embodiment of the present invention. 4 is an exemplary view for explaining the operation of a feature point tracking unit in an object detection device of a vehicle according to an embodiment of the present invention, and FIGS. 5 to 7 are An exemplary diagram for explaining an operation of a stationary object detection unit in an object detection apparatus of a vehicle according to the present invention, and FIGS. 8 and 9 are examples for explaining the operation of a moving object detection unit in the object detection apparatus of a vehicle according to an embodiment of the present invention. Is also.

Referring to FIG. 1, a vehicle object detection apparatus according to an embodiment of the present invention includes a camera pose estimation unit 100, a feature point tracking unit 200, a 3D environment map generation unit 300, and a stationary object detection unit 400. And a moving object detection unit 500. Still objects around the vehicle may be detected through the camera pose estimating unit 100, the feature point tracking unit 200, the 3D environment map generation unit 300, and the still object detection unit 400, and the camera pose estimating unit 100 ) And the moving object detection unit 500 may detect a moving object around the vehicle. Hereinafter, a process of detecting a stationary object around a vehicle will be described first.

The camera pose estimating unit 100 may estimate camera pose information by converting vehicle pose information acquired through a wheel speed measurement system mounted on the vehicle into a coordinate system of a camera mounted on the vehicle. Here, the vehicle pose information may include information on the posture and position of the vehicle obtained through the wheel speed measuring system, and accordingly, the camera pose information may include information on the posture and position of the camera mounted on the vehicle. .

In this case, the camera pose estimating unit 100 may estimate the camera pose information by applying an external parameter of the camera to the vehicle pose information determined based on the rear wheel axis of the vehicle.

Specifically, the camera pose estimating unit 100 according to the present embodiment uses Monocular Visual Odometry in which a scale value is fused to Wheel Odometry that provides the attitude and position of the vehicle in a short section using CAN information. Can be estimated. That is, since the vehicle pose information obtained through Wheel Odometry is information based on the rear wheel axis, the camera pose estimating unit 100 adds the vehicle pose information based on the rear wheel axis as shown in FIG. Transformation matrix) can be applied to estimate camera pose information. In FIG. 3, ① denotes vehicle pose information based on the rear wheel axis, and ② and ③ denote camera pose information converted from vehicle pose information based on the rear wheel axis. Accordingly, camera pose information can be estimated by Equation 1 below.

On the other hand, the camera pose estimating unit 100 may track a feature point existing in an image acquired through a camera according to a Kanade-Lucas-Tomasi (KLT) tracking algorithm, and optimize camera pose information based on the tracked feature point. . That is, the camera pose estimating unit 100 tracks the feature points by applying the KLT Tracking algorithm to the image acquired through the camera, and the reprojection error determined by projecting the tracked feature points onto the previously acquired image (frame). ) Can be optimized to minimize the camera pose information. In the process of optimizing camera pose information, the feature points tracked according to the KLT Tracking algorithm may be used in the process of detecting a moving object, not a stationary object around the vehicle, and a detailed description thereof will be described later. In addition, when optimizing camera pose information, the camera pose estimating unit 100 may reduce the time required for optimizing camera pose information by using only an image (frame) within a sliding window of a predetermined size. .

The feature point tracking unit 200 extracts a point existing in the image acquired through the camera and projects it onto a reference line existing in the image, and tracks the point projected on the reference line as a feature point according to a predefined feature point tracking algorithm. have.

At this time, the feature point tracking unit 200 extracts a point on an edge existing in the image and projects it on an epipolar line, which is a reference line, and the epipolar constraint is satisfied within a range that satisfies the epipolar constraint. The point projected on the polar line can be tracked according to the KLT (Kanade-Lucas-Tomasi) tracking algorithm, which is a feature point tracking algorithm.

A typical KLT Tracking algorithm is an algorithm that tracks a point corresponding to a corner present in an image, and there is a limit to providing inaccurate results for points on an edge existing in an image. Since a fixed point in space moves along an epipolar line in a continuous image, in this embodiment, the feature point is not estimated according to the conventional KLT Tracking algorithm, but on the edge existing in the image using the Epipolar KLT Tracking algorithm. It employs a configuration that accurately detects still objects by sturdy tracking points. FIG. 4 shows a feature point of a tracked current frame and a feature point of a tracked previous frame divided into a conventional KLT Tracking algorithm and an Epipolar KLT Tracking algorithm.

Accordingly, the feature point tracking unit 200 extracts a point on an edge existing in the image and projects it on an epipolar line, and the epipolar constraint (when a point in space is projected from one image to another image, the projected point is A point projected on an epipolar line can be tracked according to the KLT Tracking algorithm within the range of satisfying the constraints that must exist on the epipolar line (i.e., the edge point of the image is tracked as a feature point according to the Epipolar KLT Tracking algorithm). can do).

The 3D environment map generation unit 300 calculates 3D coordinates of a plurality of feature points tracked by the feature point tracking unit 200 using the camera pose information estimated by the camera pose estimation unit 100, and the calculated A 3D environment map may be generated based on a plurality of 3D feature points having 3D coordinates. The 3D environment map generation unit 300 may calculate 3D coordinates for a plurality of feature points using triangulation based on camera pose information.

At this time, the 3D environment map generation unit 300 may filter a plurality of 3D feature points through confidence according to an error, and then generate a 3D environment map using the filtered 3D feature points. May be determined based on the angle between the epipolar line on which the above-described edge point is projected and the edge and the number of images tracked by the feature point tracking unit 200.

Specifically, since the Epipolar KLT Tracking algorithm has similar directions between epipolar lines and edges and the error tends to increase as the number of tracked images decreases, the 3D environment map generator 300 By calculating the reliability and reflecting the 3D feature point to the 3D environment map generation only when the calculated reliability is greater than or equal to a preset reference value, a stationary object is detected based on a 3D feature point having a reliability greater than or equal to the reference value, thereby improving the detection accuracy. Can be improved. The 3D environment map generator 300 may determine the reliability of the 3D feature point according to Equation 2 below.

In Equation 2, C is the reliability of the corresponding three-dimensional feature point, α is the ratio between the angle between the epipolar line and the edge and the number of traced images, and the COST _angle is the cost for the angle between the epipolar line and the edge, and COST _age Means the cost for the number of traced images.

The stationary object detection unit 400 clusters the 3D feature points on the 3D environment map generated by the 3D environment map generation unit 300 on a reference plane, and stops around the vehicle based on the clustering result. Objects can be detected. Here, the reference plane on which the 3D feature points are projected may be determined as a ground plane on which a grid is formed, and the ground may be defined based on external parameters of the camera.

At this time, the still object detection unit 400 projects only the 3D feature points existing on the reference plane (ground) among the 3D feature points on the 3D environment map to the reference plane, and calculates the number of projected 3D feature points for each grid. Thereafter, a stationary object around the vehicle may be detected by classifying each grid based on the calculation result.

Specifically, as shown in FIG. 5, the still object detection unit 400 may remove 3D feature points existing under the ground from among 3D feature points on the 3D environment map and extract only 3D feature points existing on the ground. . In addition, the still object detection unit 400 may project the extracted 3D feature points onto the ground as shown in FIG. 6 (left view of FIG. 6) and calculate the number of projected 3D feature points for each grid (Fig. 6, middle drawing).

Thereafter, the stationary object detection unit 400 classifies each grid using the Connected Component algorithm based on the number of 3D feature points calculated for each grid, and determines that the grid having the same component corresponds to the same object, and thus identifies the stationary object. In this case, based on the position of the corresponding grid and the number of 3D feature points projected on the corresponding grid, the position and height of the corresponding still object may be calculated, and the corresponding still object may be detected.

That is, a label number is assigned to each grid according to the Connected Component algorithm, and accordingly, the stationary object detection unit 400 determines that the same object exists in a grid having the same component (ie, a grid with the same label number), and Can be detected (right side of Fig. 6). In this case, the stationary object detection unit 400 may detect the stationary object by calculating the position of the stationary object based on the position of the grid and calculating the stationary object based on the number of 3D feature points projected on the grid. . The left drawing of FIG. 7 shows a still object finally detected by the still object detection unit 400 (objects corresponding to the left and right boxes represent stationary objects that exist outside the trajectory of the vehicle, and objects corresponding to the middle box are of the vehicle. A still object existing in the trajectory), and the right diagram of FIG. 7 shows graphs for the number of feature points projected for each grid, and label numbers classified among grids corresponding to the same object.

Next, a process of detecting a moving object around the vehicle will be described.

The moving object detection unit 500 may detect a moving object around the vehicle based on a feature point tracked according to a KLT Tracking algorithm in a process of optimizing camera pose information by the camera pose estimating unit 100. That is, since the moving object does not follow the epipolar constraint, the aforementioned Epipolar KLT Tracking algorithm cannot be applied. Therefore, when detecting a moving object, the camera pose estimator 100 is used in the process of estimating camera pose information. It uses a general KLT Tracking algorithm that does not follow the polar constraint. FIG. 8 shows the operating principle of the moving object detection unit 500. That is, in the case of a stationary object, it is expected to move in the reverse direction as much as the moving distance of the camera, and thus the expected position of the corresponding object on the global ground plane And the actual position is compared, and the greater the difference, the higher the probability of the moving object.

When the operation of the moving object detection unit 500 is described in detail based on the above description, the moving object detection unit 500 is the position of a pixel in the image with respect to the feature point tracked according to the KLT Tracking algorithm used in the camera pose estimation process. The difference between the coordinates and the 3D coordinates is accumulated and analyzed, and the movement score is increased by detecting a feature point estimated as a moving object, and when the movement score counted for each grid formed in the image exceeds a preset reference value, the corresponding grid corresponds to the moving object. It is determined that the moving object can be detected.

That is, the moving object detection unit 500 accumulates and analyzes the difference between the position coordinates and the global 3D coordinates of the pixels on the 2D image for the feature points tracked according to the KLT Tracking algorithm as shown in FIG. It detects the presumed feature points to increase the movement score, counts the movement score for each grid formed in the image, and if the value exceeds a preset reference value, the corresponding grid is classified as a dynamic box, and the moving object exists in the dynamic box. The moving object can be detected through a method of determination.

10 is a flowchart for explaining a method of detecting an object of a vehicle according to an embodiment of the present invention. With reference to FIG. 10, a method of detecting an object of a vehicle according to an embodiment of the present invention is described. The overlapping description with will be omitted.

First, the camera pose estimating unit 100 estimates the camera pose information by converting the vehicle pose information obtained through the vehicle wheel speed measurement system into a coordinate system of the vehicle camera (S100). In step S100, the camera pose estimating unit 100 estimates the camera pose information by applying an external parameter of the camera to the vehicle pose information determined based on the rear wheel axis of the vehicle. Further, in step S100, the camera pose estimating unit 100 tracks feature points present in the image acquired through the camera according to the KLT Tracking algorithm, and optimizes camera pose information based on the tracked feature points.

Subsequently, the feature point tracking unit 200 extracts a point existing in the image acquired through the camera and projects it onto a reference line existing in the image, and tracks the point projected on the reference line as a feature point according to a predefined feature point tracking algorithm. Do (S200). In step S200, the feature point tracking unit 200 extracts a point on an edge existing in the image and projects it onto an epipolar line, which is a reference line, and the range in which the epipolar constraint is satisfied. The point projected on the epipolar line is tracked according to the KLT (Kanade-Lucas-Tomasi) Tracking algorithm, which is a feature point tracking algorithm.

Subsequently, the 3D environment map generation unit 300 calculates 3D coordinates of the plurality of feature points tracked in step S200 using the camera pose information estimated in step S100, and calculates the 3D coordinates having the calculated 3D coordinates. A 3D environment map is generated based on the feature points (S300). In step S300, the 3D environment map generation unit 300 filters a plurality of 3D feature points through reliability according to an error, and then generates a 3D environment map using the filtered 3D feature points. The reliability may be determined based on the angle between the epipolar line and the edge and the number of images tracked by the feature point tracking unit 200.

Subsequently, the stationary object detection unit 400 clusters the 3D feature points on the 3D environment map generated by the 3D environment map generation unit 300 on a reference plane, and based on the clustering result, A still object of is detected (S400). In step S400, the still object detection unit 400 projects only the 3D feature points existing on the reference plane among the 3D feature points on the 3D environment map to the reference plane, and calculates the number of projected 3D feature points for each grid. , Based on the number of 3D feature points calculated for each grid, each grid is classified using the Connected Component algorithm, and a grid having the same component is determined to correspond to the same object, and a still object is detected. At this time, the stationary object detection unit 400 detects the stationary object by calculating the position and height of the stationary object based on the location and number of feature points projected on the grid.

Meanwhile, the moving object detection unit 500 detects a moving object around the vehicle based on the feature point tracked according to the KLT Tracking algorithm in the process of optimizing the camera pose information in step S100 (S500). In step S500, the moving object detection unit 500 accumulates and analyzes the difference between the position coordinates and 3D coordinates of pixels on the image with respect to the feature point tracked according to the KLT Tracking algorithm, and detects the feature point estimated as the moving object to calculate the moving score. If the movement score is increased and counted for each grid formed in the image exceeds a preset reference value, the corresponding grid is determined to correspond to a moving object, and the moving object is detected. Step S500 is a parallel configuration performed independently from steps S100 to S400, and the operation sequence is not limited to the above described order.

As described above, this embodiment constructs a 3D environment map around the vehicle by using the image image acquired through the vehicle's camera and the vehicle's wheel speed measurement system, and detects an object on the vehicle trajectory based on the constructed 3D environment map. Through this method, it is possible to improve the driving convenience of the driver by precisely detecting the stationary and moving objects around the vehicle.

In addition, this embodiment is applied to an autonomous driving system or a parking assistance system of a vehicle to detect an object such as an obstacle around the vehicle and control autonomous driving of the vehicle by accurately controlling the autonomous driving or parking of the vehicle based on the detection result. Performance and parking control performance can be improved.

The implementation described herein may be implemented in, for example, a method or process, an apparatus, a software program, a data stream or a signal. Although discussed only in the context of a single form of implementation (eg, only as a method), the implementation of the discussed features may also be implemented in other forms (eg, an apparatus or program). The device may be implemented with appropriate hardware, software and firmware. The method may be implemented in an apparatus such as a processor, which generally refers to a processing device including, for example, a computer, microprocessor, integrated circuit or programmable logic device, or the like. Processors also include communication devices such as computers, cell phones, personal digital assistants ("PDAs") and other devices that facilitate communication of information between end-users.

The present invention has been described with reference to the embodiments shown in the drawings, but these are only exemplary, and those of ordinary skill in the art to which the present technology pertains, various modifications and other equivalent embodiments are possible. I will understand. Therefore, the true technical protection scope of the present invention should be determined by the following claims.

100: camera pose estimation unit
200: feature point tracking unit
300: 3D environment map generation unit
400: still object detection unit
500: moving object detection unit

Claims (18)

A camera pose estimating unit for estimating camera pose information by converting vehicle pose information acquired through a vehicle wheel speed measurement system into a coordinate system of the vehicle camera;
A feature point tracking unit for extracting a point existing in the image acquired through the camera, projecting it onto a reference line existing in the image, and tracking the point projected on the reference line as a feature point according to a predefined feature point tracking algorithm;
The three-dimensional coordinates of the plurality of feature points tracked by the feature point tracking unit are calculated using the camera pose information estimated by the camera pose estimating unit, and based on the plurality of three-dimensional feature points having the calculated three-dimensional coordinates A 3D environment map generator that generates a 3D environment map; And
A stationary object detector configured to cluster by projecting 3D feature points on a 3D environment map generated by the 3D environment map generator onto a reference plane, and detect a stationary object around the vehicle based on the clustering result;
Vehicle object detection apparatus comprising a.
The method of claim 1,
And the camera pose estimating unit estimates the camera pose information by applying an external parameter of the camera to the vehicle pose information determined based on a rear wheel axis of the vehicle.
The method of claim 1,
The feature point tracking unit extracts a point on an edge existing in the image and projects it onto the reference line to track the feature point.
The method of claim 3,
The feature point tracking unit projects the point on the edge onto an epipolar line, which is the reference line, and tracks the point projected onto the epipolar line within a range satisfying an epipolar constraint. An object detection device of a vehicle, characterized in that tracking according to the algorithm KLT (Kanade-Lucas-Tomasi) Tracking algorithm.
The method of claim 3,
The 3D environment map generator, after filtering the plurality of 3D feature points through reliability according to an error, generates the 3D environment map using the filtered 3D feature points. Device.
The method of claim 5,
The reliability is determined based on an angle between the epipolar line and the edge and the number of images tracked by the feature point tracking unit.
The method of claim 1,
The reference plane is determined as a ground plane on which a grid is formed,
The still object detection unit projects only three-dimensional feature points existing on the reference plane among three-dimensional feature points on the three-dimensional environment map to the reference plane, calculates the number of projected three-dimensional feature points for each grid, and An object detection device for a vehicle, characterized in that for detecting a stationary object around the vehicle through a method of classifying each grid based on a calculation result.
The method of claim 7,
The still object detection unit classifies each grid using a Connected Component algorithm based on the number of three-dimensional feature points calculated for each grid, determines that a grid having the same component corresponds to the same object, and detects the still object. And calculating the position and height of the corresponding stationary object based on the position of the corresponding grid and the number of 3D feature points projected on the corresponding grid to detect the corresponding stationary object.
The method of claim 1,
The camera pose estimating unit tracks feature points present in the image acquired through the camera according to the KLT Tracking algorithm, and optimizes the camera pose information based on the tracked feature points,
And a moving object detection unit configured to detect a moving object around the vehicle based on a feature point tracked according to the KLT Tracking algorithm in the process of optimizing the camera pose information.
The method of claim 9,
The moving object detection unit accumulates and analyzes the difference between the position coordinates and 3D coordinates of pixels on the image with respect to the feature point tracked according to the KLT Tracking algorithm, detects the feature point estimated as a moving object, and increases the moving score. And when the movement score counted for each grid formed on the image exceeds a preset reference value, the corresponding grid is determined to correspond to a moving object and detects the moving object.
Estimating camera pose information by converting, by a camera pose estimating unit, vehicle pose information obtained through a wheel speed measurement system of the vehicle into a coordinate system of the camera of the vehicle;
A step of extracting, by a feature point tracking unit, a point present in the image acquired through the camera, projecting it onto a reference line present in the image, and tracking the point projected on the reference line as a feature point according to a predefined feature point tracking algorithm ;
The three-dimensional environment map generation unit calculates the three-dimensional coordinates of the plurality of feature points tracked by the feature point tracking unit by using the camera pose information estimated by the camera pose estimating unit, and a plurality having the calculated three-dimensional coordinates Generating a three-dimensional environment map based on the three-dimensional feature points of; And
The stationary object detection unit clusters by projecting 3D feature points on the 3D environment map generated by the 3D environment map generation unit onto a reference plane, and detects stationary objects around the vehicle based on the clustering result. Step to do;
Object detection method of a vehicle comprising a.
The method of claim 11,
In the estimating step, the camera pose estimation unit,
And estimating the camera pose information by applying an external parameter of the camera to the vehicle pose information determined based on a rear wheel axis of the vehicle.
The method of claim 11,
In the tracking step, the feature point tracking unit,
A point on the edge existing in the image is extracted and projected on the epipolar line, which is the reference line, and a point projected on the epipolar line within a range satisfying the epipolar constraint The object detection method of a vehicle, characterized in that tracking according to the feature point tracking algorithm KLT (Kanade-Lucas-Tomasi) tracking algorithm.
The method of claim 11,
In the generating step, the 3D environment map generation unit,
After filtering the plurality of 3D feature points through reliability according to an error, the 3D environment map is generated using the filtered 3D feature points, wherein the reliability is an angle between the epipolar line and the edge, and the The object detection method of a vehicle, characterized in that it is determined based on the number of images tracked by the feature point tracking unit.
The method of claim 11,
The reference plane is determined as a ground plane on which a grid is formed,
In the detecting step, the still object detection unit,
Among the three-dimensional feature points on the three-dimensional environment map, only three-dimensional feature points existing on the reference plane are projected onto the reference plane, the number of projected three-dimensional feature points is calculated for each grid, and then each grid is calculated based on the calculation result. An object detection method of a vehicle, characterized in that to detect a stationary object around the vehicle through a classification method.
The method of claim 15,
In the detecting step, the still object detection unit,
Based on the number of three-dimensional feature points calculated for each grid, each grid is classified using the Connected Component algorithm, and a grid having the same component is determined to correspond to the same object, and a stationary object is detected. An object detection method for a vehicle, comprising detecting a corresponding stationary object by calculating a position and a height of a corresponding stationary object based on the number of 3D feature points projected on a corresponding grid.
The method of claim 11,
In the estimating step, the camera pose estimation unit,
A feature point present in an image acquired through the camera is tracked according to a KLT Tracking algorithm, and the camera pose information is optimized based on the tracked feature point,
The moving object detection unit further comprising: detecting a moving object around the vehicle based on a feature point tracked according to the KLT Tracking algorithm in the process of optimizing the camera pose information. .
The method of claim 17,
In the step of detecting the moving object, the moving object detection unit,
Accumulatively analyzing the difference between the position coordinates and 3D coordinates of pixels on the image for the feature points tracked according to the KLT Tracking algorithm, detecting the feature points estimated as moving objects, increasing the movement score, and a grid formed on the image When the movement score counted for each exceeds a preset reference value, the corresponding grid determines that the moving object corresponds to and detects the moving object.

Images