JP5482672B2 - Moving object detection device - Google Patents

Description

  The present invention relates to a moving object detection apparatus that detects a moving object that crosses a traveling direction of a vehicle using an imaging unit mounted on the vehicle.

  It is known that the number of fatalities and injuries that occur when pedestrians and bicycles cross the course of vehicles on public roads account for a large percentage of the total number of traffic casualties.

  Therefore, for example, Patent Document 1 discloses an apparatus that can detect an object in an image that moves so as to cross a moving direction in a situation where a vehicle equipped with an image input unit is moving. ing.

  In the apparatus described in Patent Document 1, an optical flow indicating a distribution state of velocity vectors on an image is calculated from an input moving image, and the minimum velocity vector component in the horizontal direction is calculated from the optical flow. Find the value and find the direction of the infinity point. Then, the moving object is detected from the polarity of the velocity vector component in the horizontal direction of the optical flow with reference to the direction of the infinity point. That is, in the apparatus described in Patent Document 1, the horizontal component of the velocity vector of the moving object that is going to cross the traveling direction of the vehicle has the same polarity as that of the horizontal component of the background velocity vector generated by the vehicle's self-motion. The difference is used to detect a moving object that is about to cross the traveling direction of the vehicle.

  Further, Patent Document 2 discloses an apparatus that detects a pedestrian, a roadside structure, a stopped vehicle, a vehicle that is traveling in a transverse direction with respect to the host vehicle, and the like, excluding preceding vehicles and oncoming vehicles. Have been described.

  This apparatus includes a camera and a laser radar, and first extracts an object to be processed in front of the host vehicle based on the position and speed of the object detected by the radar. Then, by comparing the direction and size of the motion vector of the feature point of the object with the theoretical background vector in the image captured by the camera for the extracted object, the motion vector is moved. Alternatively, it is detected whether the object is a stationary object.

JP 7-249127 A JP 2008-171141 A

  In the apparatus described in Patent Document 1, if a moving object that is going to cross the traveling direction of the host vehicle exceeds a line along the traveling direction of the host vehicle, it is very difficult to detect the moving object. There is a problem of becoming. This is because the horizontal component of the velocity vector of the moving object and the background velocity vector after the moving object exceeds the line along the traveling direction of the own vehicle (the line connecting the own vehicle and the infinity point). This is because the polarity with respect to the horizontal direction component becomes the same. Therefore, there is a possibility that a situation may occur in which the moving object cannot be detected even though the moving object is still in the vicinity of the traveling direction of the host vehicle.

  Further, in the apparatus described in Patent Document 2, an object having a relative speed of a predetermined value or less is extracted as a processing target by a laser radar in front of the traveling direction of the host vehicle. Although it can be detected, there is a problem that a laser radar is required in addition to the camera.

  The present invention has been made in view of the above points, and can accurately detect a moving object that crosses the traveling direction of a vehicle from an image captured by an imaging unit mounted on the vehicle without using a radar. It is an object of the present invention to provide a moving object detection device capable of performing the above.

In order to achieve the above object, a moving object detection apparatus according to claim 1 is provided.
Photographing means mounted on the vehicle so as to photograph the traveling direction of the vehicle;
In the image photographed by the photographing means, the feature point is extracted, and the optical flow calculating means for calculating the optical flow of each extracted feature point;
From the optical flow of each feature point calculated by the optical flow calculation means, vanishing point calculation means for determining the vanishing point of each feature point existing on the extension line of the optical flow,
The angle formed by the line segment connecting the infinity point in the image captured by the image capturing unit and the focal point of the image capturing unit and the line segment connecting the vanishing point of each feature point and the focal point of the image capturing unit is set to An angle calculating means for calculating the relative approach angle of the feature point;
Determining means for determining that the moving object indicated by the corresponding feature point is a moving object crossing the traveling direction of the vehicle when the relative approach angle calculated by the angle calculating means is equal to or greater than the threshold value; It is characterized by.

  As will be described in detail later, the actual motion vector of the moving object viewed from the vehicle (the motion vector in the three-dimensional space) connects the vanishing point of the moving object and the focus of the imaging means in the image of the moving object. It becomes parallel to the line segment. When the moving object crosses the traveling direction of the vehicle, the direction of the actual motion vector of the moving object and the traveling direction of the vehicle are not parallel but have an angle.

  Therefore, in the moving object detection device according to claim 1, the line segment connecting the infinity point of the image and the focus of the imaging unit corresponding to the traveling direction of the vehicle, and the direction of the actual motion vector of the moving object viewed from the vehicle Is calculated as a relative approach angle of each feature point with respect to the traveling direction of the vehicle. Then, it is determined whether or not the calculated relative approach angle of each feature point is equal to or greater than a predetermined threshold angle. Thereby, it is possible to determine whether or not the moving object is a moving object that crosses the traveling direction of the vehicle. Furthermore, by using such a relative approach angle, a moving object that exceeds the traveling direction of the vehicle can be determined as a moving object that crosses the traveling direction of the vehicle.

  According to a second aspect of the present invention, the vanishing point calculating means preferably obtains the intersection of the virtual horizontal line extending horizontally from the infinity point in the image and the extension line of the optical flow as the vanishing point of each feature point. . Assuming a flat ground without inclination, the infinity point and the vanishing point exist on the horizon. For this reason, the intersection of the virtual horizontal line extending horizontally from the infinity point and the extension line of the optical flow can be the vanishing point of each feature point. Thus, by simply determining the vanishing point of each feature point, the amount of calculation for calculating the vanishing point can be reduced.

  As described in claim 3, at least from the vanishing point position of each feature point calculated by the vanishing point calculation means, feature points sharing the same vanishing point are grouped as feature points indicating the same moving object. It is preferable that a grouping unit is provided, and the determination unit determines whether the moving object indicated by the grouped feature points is a moving object that crosses the traveling direction of the vehicle. Thus, by grouping the feature points indicating the same moving object, it is possible to clarify how many moving objects cross the traveling direction of the vehicle. Further, by grouping feature points under the condition that they share the same vanishing point, outliers (outliers that are erroneously calculated due to incorrect correspondence) mixed in the optical flow can be removed.

  According to a fourth aspect of the present invention, the vehicle includes a collision time calculation unit that calculates a collision time for each feature point until the distance between the vehicle and each feature point becomes zero in the traveling direction of the vehicle. It is preferable to group the feature points belonging to the predetermined time width as the feature points indicating the same moving object. This is because feature points indicating the same moving object have similar collision times. Then, in addition to the condition that the same vanishing point is shared, the similarity of the collision time is also adopted as the grouping condition, so that the outlier can be removed more reliably. Furthermore, since the collision time becomes clear, it is possible to determine the degree of emergency of the situation at that time from the length of the collision time.

  Furthermore, as described in claim 5, the grouping means may group feature points existing within a predetermined distance range as feature points indicating the same moving object. This is because the feature points indicating the same moving object exist at positions close to each other.

  According to a sixth aspect of the present invention, the determination means includes a warning means for issuing a warning to the driver of the vehicle when a determination result is obtained that the moving object crosses the traveling direction of the vehicle. The warning mode may be changed in accordance with the number of moving objects determined as moving objects crossing the traveling direction of the vehicle by the determining means. For example, a warning sound may be generated for the number of moving objects. Thereby, not only that the moving object which crosses the front of a vehicle exists but the information regarding the number of the moving objects can be provided with respect to the passenger | crew of a vehicle.

  According to a seventh aspect of the present invention, the determination means comprises a warning means for issuing a warning to the driver of the vehicle when a determination result with a moving object crossing the traveling direction of the vehicle is obtained, The warning mode may be changed according to the length of the collision time calculated by the collision time determination means. Thus, not only that there is a moving object that crosses the front of the vehicle but also information regarding the time that the vehicle occupant may collide with the moving object can be provided to the vehicle occupant.

  In the moving object detection device according to claim 8, the determination unit is a moving object before crossing before crossing a line extending a line segment connecting the focal point of the imaging unit and the infinity point as a moving object crossing the traveling direction of the vehicle. And the moving object after crossing after crossing. In other words, in the moving object detection device according to the present invention, the relative approach angle is used to determine whether the moving object is a moving object that crosses the traveling direction of the vehicle. It is possible to determine a moving object after crossing as a target.

  In the moving object detection device according to the ninth aspect, the optical flow calculation means calculates an optical flow from which a component generated by the movement of the photographing means in the rotational direction accompanying the movement of the vehicle is removed. This is because, when the photographing unit is mounted on a moving body such as a vehicle, a component caused by the movement of the photographing unit in the rotation direction becomes a significant error factor in calculating the optical flow of the moving object.

It is a block diagram which shows the structure of the moving body detection apparatus by embodiment of this invention. It is a flowchart which shows the flow of the process performed in a mobile body detection apparatus. It is explanatory drawing for demonstrating an infinite point (FOE). (A), (b) is explanatory drawing for demonstrating a relative approach angle. It is explanatory drawing for demonstrating the method of calculating | requiring the relative approach angle of each feature point from the image image | photographed with the video camera. This is for explaining the classification of an approaching moving object, where (a) represents a moving object in a state before crossing a straight line representing the predicted course of the video camera, and (b) represents the predicted course of the video camera. (C) represents a moving object in a state where it does not cross the straight line representing the predicted course of the video camera. (A)-(c) is a figure which shows the mode at the time of carrying out the perspective projection of the moving object of 3 states shown to Fig.6 (a)-(c), respectively to an image. (A)-(c) is explanatory drawing for demonstrating the grouping process of a feature point.

  Hereinafter, a mobile object detection device according to an embodiment of the present invention will be described in detail with reference to the drawings.

  As shown in FIG. 1, the moving body detection apparatus 100 according to the present embodiment includes a video camera 10 as a photographing unit mounted on a vehicle, and an image processing ECU 20 that processes an image photographed by the video camera 10. ing.

  The video camera 10 is constituted by, for example, a CCD camera, and is installed near the ceiling in the vehicle interior of the vehicle or at the front end (front bumper) of the vehicle. At the time of installation on this vehicle, the optical axis (Z axis) of the video camera 10 coincides with the traveling direction of the vehicle, and the horizontal axis (X axis) of the image taken by the video camera 10 is parallel to the ground. The installation angle of the video camera 10 is adjusted. As a result, the vertical axis (Y axis) of the image coincides with the direction perpendicular to the ground. The video camera 10 periodically captures images while the vehicle is traveling, and outputs the captured images to the image processing ECU 20.

  The image processing ECU 20 stores an A / D converter for converting an image output from the video camera 10 into a digital image, and a converted digital image, in addition to a microcomputer including a CPU, ROM, RAM, etc. (not shown). An image memory or the like is provided. Note that the image memory has a storage capacity capable of storing a plurality of images.

  The image processing ECU 20 executes various processes according to a program stored in advance. For example, the image processing ECU 20 calculates the optical flow of each feature point included in the image from the change of the same feature point in a plurality of images output in time series from the video camera 10. Then, the position of the infinity point (FOE) of the image is calculated mainly based on the optical flow of the feature point of the stationary object. Further, the vanishing point of the moving object is calculated from the optical flow of the feature point of the moving object. Further, using the FOE position of the image and the vanishing point position of the moving object, it is determined whether or not the moving object is a moving object that crosses the course direction of the vehicle. In this way, when the image processing ECU 20 determines that the object is a moving object that crosses the course direction of the vehicle, the determination result is output to the control device 30.

  For example, when the image processing ECU 20 determines that there is a moving object that crosses the direction of the vehicle, the control device 30 notifies the driver of the vehicle and avoids a collision with the moving object. The brake device is controlled to generate a braking force on the vehicle.

  Next, various processes executed in the image processing ECU 20 will be described in detail based on the flowchart of FIG. The process shown in the flowchart of FIG. 2 is repeatedly executed when the engine of the vehicle is turned on or when the vehicle starts running.

  First, in step S100, an image taken by a video camera is captured and stored in an image memory. In the subsequent step S110, when a new digital image is stored in the image memory, an object corner or the like is extracted as a feature point in the digital image. Then, by associating the same feature point in two digital images stored in the image memory at different timings, an optical flow corresponding to the motion vector of the feature point is calculated. As an optical flow calculation method, for example, the LK method can be cited. The LK method searches for a small region in another image having a high correlation with a small region in the image centered on a feature point by a gradient method. is there.

  In the above-described example, the points such as the corners of the object are calculated as the feature points. However, the feature points can be calculated by other methods.

  In step S120, the motion parameter of the video camera 10 is estimated. The video camera 10 is mounted on a vehicle, and the video camera 10 also performs a translational motion and a rotational motion in accordance with the translational motion and rotational motion of the vehicle. For this reason, in step S120, the motion parameter is estimated from the translational momentum and the rotational movement amount of the video camera 10.

  Here, the moving body detection apparatus according to the present embodiment detects a moving object crossing a course that crosses the traveling direction (path) of the vehicle, but is important for determining whether or not the moving object is a moving object that crosses the course. The relative approach angle (which will be described later in detail) having such a technical meaning is based on the premise that the coordinate system of the video camera 10 moves in translation. In other words, if the optical flow includes a component due to the rotational motion of the video camera 10, the relative approach angle cannot be calculated with high accuracy, and the accuracy of discriminating the crossing path moving object also decreases. become. Therefore, it is necessary to remove the component generated by the rotational motion of the video camera from the optical flow calculated in step S110, and therefore, the motion parameter of the video camera 10 is estimated.

The motion parameters of the video camera 10 are represented by the following formula 1, the three-dimensional translational movement amount V = (V _x , V _y , V _z ) of the video camera 10 and the rotational movement amount Ω = (Ω _x , Ω _y , Ω _z ), the coordinates of the stationary point (x, y, z), and the optical flow v = (v _x , v _y ). Note that f is the focal length of the video camera 10.

Here, when the definition formula (x _F , y _F ) = (fVx / Vz, fVy / Vz) of the infinity point (FOE) is used, the following formula 2 is established. Note that the infinity point (FOE) indicates the center point of the optical flow of the feature points of the stationary object that radiates as shown in FIG.

By solving the mathematical formula 2 as a number of simultaneous equations of feature points, the infinity point (x _F , y _F ) representing the direction of translation of the video camera 10 and the rotational movement of the video camera 10 The optical flow (u _x , u _y ) from which the generated optical flow component is removed can be obtained.

Note that the above formulas 1 and 2 are established only for a stationary point, and the values of the formulas have a large error at points having moving points and outliers (outliers that are erroneously calculated due to incorrect correspondence). It is thought to take. For this reason, it is desirable to use a solution method based on robust estimation when obtaining the infinite point (x _F , y _F ) and the optical flow (u _x , u _y ). As robust estimation, for example, M estimation (reference: PJ Huber and EM Ronchetti, “Robust Statistics, 2nd Edition”, Wiley Interscience, 2009.) can be considered.

  Alternatively, the motion parameter of the video camera 10 (optical flow from which a component generated by the rotational movement of the camera is removed) can be calculated using detection values of a vehicle speed sensor or a yaw rate sensor attached to the vehicle. . Further, as is well known, the infinite point may be obtained from the optical flow of the separated stationary point by separating the moving point and stationary point at the feature point by a technique such as RANSAC method or clustering method.

By the process of step S120, the optical flow (u _x , u _y ) from which the component generated by the rotational movement of the video camera 10 and the infinity point (x _F , y _F ) representing the direction of translation of the video camera 10 are removed. Then, in step S130, feature point filtering is performed.

In the filtering process of feature points, first, a relative approach angle is calculated for each feature point using an optical flow (u _x , u _y ) from which an optical flow component generated by the rotational motion of the camera is removed.

  This relative approach angle is demonstrated with reference to Fig.4 (a), (b).

  As shown in FIG. 4 (a), when there is a crossing path moving object that tries to cross the front of the video camera 10, the movement of the crossing path moving object is viewed from the vehicle, that is, the video camera 10 that translates. This can be described as a motion having a certain angle with respect to the traveling direction of the video camera 10. In the present embodiment, as shown in FIG. 4B, this angle is referred to as a “relative approach angle”.

A stationary point can be regarded as a moving object with zero speed, and the relative approach angle is 0 degree. On the other hand, FIG. 4 (a), the (b), the order translational relative angle of approach the cross track moving objects across the path of movement video camera 10 which is larger than 0 degree, an appropriate threshold value theta _th By setting, it becomes possible to distinguish from a stationary point.

  For example, when the vehicle moves at a low speed, the relative approach angle becomes larger than when the vehicle moves at a high speed. Therefore, it can be said that the relative approach angle is a good index for separating the moving object crossing the course and the stationary object, particularly when traveling at a low speed. However, since the calculation accuracy of the relative approach angle changes depending on the resolution of the video camera 10, it is preferable to set a threshold value for discriminating the moving object crossing the course based on this.

  As an example, the threshold of the relative approach angle can be set as follows from the speed of the target vehicle and the speed of the detection target. For example, if the upper limit of the target vehicle speed is Vmax and the lower limit of the detection target speed that crosses the road at right angles is Wmin, the relative approach angle threshold can be set to be less than arctan (Wmin / Vmax). Good. For example, when Vmax = 20 km / h and Wmin = 4 km / h, since arctan (Wmin / Vmax) = 11.3 °, it is reasonable to set the relative approach angle threshold to 10 °. However, this threshold value may be set to a smaller angle from the upper limit of the vehicle speed at which the crossing path moving object is detected, the calculation accuracy of the relative approach angle, and the like.

  Next, a method for obtaining the relative approach angle of each feature point from an image taken by the video camera 10 will be described. As shown in FIG. 5, when a moving object crossing a course is photographed by the video camera 10, an image obtained by perspective projection of the moving object crossing the course is obtained. Note that when the captured image of the video camera 10 is affected by lens distortion, the image is not correctly perspective-projected. Therefore, it is desirable to execute distortion correction processing for removing the influence of lens distortion.

  Assuming that the movement of the crossing path moving object is a rigid translational motion, the optical flow of the crossing path moving object has a direction that spreads radially around a point called the vanishing point. This vanishing point indicates an infinite point where the moving object crossing the course disappears as a result of the constant-velocity linear motion of the moving object and the video camera 10 continuing for an infinite time.

  As is clear from FIG. 5, the actual motion vector (motion vector in the three-dimensional space) of the crossing path moving object viewed from the vehicle is the disappearance of the crossing path moving object in the image of the crossing path moving object. It is parallel to the line segment connecting the point and the focal point of the video camera 10. Therefore, the line segment connecting the FOE and the camera focus corresponding to the traveling direction of the vehicle, the vanishing point of each feature point corresponding to the direction of the actual motion vector of the moving object viewed from the vehicle, and the camera focus. The angle formed by the connecting line segments represents the relative approach angle of each feature point with respect to the traveling direction of the vehicle. In other words, the angle on the focal point side of the triangle whose apex is the vanishing point of the crossing path moving object, the camera focus, and the FOE represents the relative approach angle of the crossing path moving object.

Since the focal length f of the video camera 10 is known, the relative approach angle can be obtained by obtaining the FOE and the vanishing point of the moving object crossing the course. The relative approach angle is given by Equation 3.

P _F is the FOE coordinates (x _F , y _F , f) when the video camera focus is the origin, and P _∞ is the vanishing point coordinates (x _∞ , y _∞ , f).

The FOE coordinates (x _F , y _F , f) are calculated in step S120. The vanishing point coordinates (x _∞ , y _∞ , f) can be calculated as follows.

First, as shown in FIG. 5, in the image photographed by the video camera 10, a straight line y = y _F is called a horizon. Assuming a flat ground with no inclination, the FOE and vanishing point exist on the horizon. For this reason, the vanishing point is an intersection of the extension line of the optical flow from which the rotational component is removed and the horizon. Therefore, for each feature point, the vanishing point can be obtained from the coordinates and the optical flow (u _x , u _y ). Thus, by simply determining the vanishing point of each feature point, the amount of calculation for calculating the vanishing point can be reduced.

  However, the vanishing point of each feature point may be calculated with higher accuracy by another method. For example, intersections of extension lines obtained by extending the optical flow of each feature point may be obtained, the intersections existing at close positions may be grouped, and a position that can be regarded as the center of the grouped intersection position may be determined as a vanishing point.

  Once the relative approach angle is calculated for each feature point in this way, in step S130, the feature point that is a candidate point for the cross-route moving object that crosses the course of the vehicle is separated from the other feature points. Execute the filtering process.

  In the present embodiment, a moving object that crosses the course of the vehicle that crosses the course of the vehicle is the detection target. When the vehicle is moving in the traveling direction, such a cross-track moving object becomes an approaching moving object that approaches the video camera 10 in the Z-axis (camera optical axis) direction as time passes.

  Here, the approaching moving object can be classified into three types as shown in FIGS. A pre-crossing (moving object before crossing) shown in FIG. 6A represents a moving object in a state before crossing a straight line representing a predicted course of the video camera 10. A post-crossing (moving object after crossing) shown in FIG. 6B represents a moving object in a state after crossing a straight line representing a predicted course of the video camera 10. A never-crossing (non-crossing moving object) shown in FIG. 6C represents a moving object that does not cross a straight line representing a predicted course of the video camera 10.

  In the present embodiment, pre-crossing and post-crossing are defined as objects that cross the course and are set as detection targets. The reason why never-crossing is not detected is, for example, the possibility that pedestrians on the sidewalk next to the roadway may apply to never-crossing, and it is considered that there is little need to alert the driver to this target. Because. As long as the relative approach angle is non-zero, never-crossing can be included in the detection target.

The moving objects in the three states shown in FIGS. 6A to 6C are perspectively projected onto the image by the video camera 10 as shown in FIGS. 7A to 7C. Here, the pre-crossing state shown in FIG. 7A includes FOE coordinates (x coordinate: x _F ), vanishing point coordinates (x coordinate: x _∞ ), feature point coordinates (x coordinate: x), and Regarding the relative approach angle θ, the relationship shown in the following Expression 4 is established.

In the post-crossing state shown in FIG. 7B, the relationship shown in the following Expression 5 is established with respect to the FOE coordinate, the vanishing point coordinate, the feature point coordinate, and the relative approach angle.

Furthermore, in the never-crossing state shown in FIG. 7C, the relationship shown in the following Expression 6 is established with respect to the FOE coordinates, vanishing point coordinates, feature point coordinates, and relative approach angle.

  By using these relationships, the moving object in any of the three states is indicated for each feature point from the FOE coordinates, vanishing point coordinates, image coordinates of the feature points, and relative approach angles. It is possible to distinguish whether it is a thing. In the filtering process of step S130, the feature points classified as pre-crossing or post-crossing are determined as candidate points indicating a crossing path moving object. This candidate point becomes an input (target) for the grouping process in step S140 described below.

  In step S140, feature points that can be regarded as indicating the same crossing path moving object under various conditions are extracted and grouped from the feature points that are candidate points indicating the crossing path moving object. Since the candidate points for the path crossing object are just a collection of points, it is difficult to understand information (number, relative approach angle, collision time, etc.) regarding the moving object crossing the path. In addition, optical flows are often mixed with outliers that are erroneously calculated due to an erroneous correspondence called an outlier. Therefore, grouping is performed for the purpose of collecting candidate points for each moving object and removing outliers.

  The first condition for performing grouping is that feature points exist in the vicinity of each other. In this neighborhood determination, as shown in FIG. 8A, for each feature point that is a candidate point, a square region centered on each feature point is given, and the feature points where these square regions overlap each other are Group as if it were in the vicinity. On the other hand, isolated points whose square area does not overlap with the square areas of other feature points are regarded as outliers and excluded from grouping targets.

  The second condition for performing grouping is that the relative approach angles substantially match. As shown in FIG. 5, the feature points of the same moving object share the vanishing point, and as a result, the relative approach angles substantially match.

  In this relative approach angle coincidence determination, as shown in FIG. 7B, the distribution state of vanishing point coordinates is obtained for each group determined to be close, and it is considered that the same vanishing point is shared from the distribution state. Statistically divide feature points from feature points that are not considered to share the same vanishing point. In addition, for example, a feature that estimates the true vanishing point position from the vanishing point coordinate distribution position and shares the same vanishing point based on the angle error between the optical flow and the straight line connecting the vanishing point and the feature point. The points may be divided into feature points that do not share the same vanishing point. And the feature point which does not share the same vanishing point is a feature point by an outlier, for example, and is excluded from the grouping target. On the other hand, it is determined that the feature points sharing the same vanishing point have the same relative approach angle.

The third condition for performing grouping is that the collision times T of the feature points are similar. Here, the collision time T is defined as the time until the object existing in front of the video camera 10 (Z> 0) reaches the image plane (substantially, the time until the vehicle collides). The collision time T is calculated by using the optical flow (u _x , u _y ) from which the optical flow component generated by the rotational movement of the video camera 10 is removed and the vanishing point coordinates (x _∞ , y _∞ ). Can be calculated.

That is, it is possible to obtain the collision time T for each feature point using Equation 7. Specifically, for each feature point grouped according to the first and second conditions described above, the vanishing point coordinates, the feature point coordinates, and the optical flow (u _x , u _y ) are substituted into Equation 7 to determine the collision time. T is calculated. The collision time T is calculated as a time having a certain range in consideration of an optical flow calculation error.

  Here, the collision time T of the same moving object is similar as long as general detection targets (people, bicycles, vehicles, etc.) on the road are taken into consideration, although there are some deviations depending on the part of the object. It is natural to think. Therefore, the similarity determination of the collision time T is performed on the characteristic points grouped according to the conditions 1 and 2 with respect to the collision time T obtained for each characteristic point as shown in FIG. That is, given an error bar with a preset time width ΔT, if the collision time T matches within the error bar range, it is determined that the collision time T is similar, and there is a feature point where the collision time T does not match within the error bar range. Exclude it from the group.

  Thus, in this embodiment, since a plurality of conditions are used as the grouping conditions, the outlier can be removed more reliably. Furthermore, since the collision time T of the moving object crossing the path becomes clear, it is possible to determine the degree of emergency of the situation at that time from the length of the collision time.

  In the subsequent step S150, it is determined whether or not there are feature points grouped by the grouping process in step S140. In this determination process, when it is determined that there are grouped feature points, it is considered that there is a path crossing moving object that crosses the path of the vehicle, and the process proceeds to step S160. On the other hand, if it is determined that there are no grouped feature points, the above-described processing is repeated again from step S100.

  In step S160, the control device 30 is instructed to execute an alarm for the driver, a control for automatically braking the vehicle, and the like, whereby the control device 30 executes the alarm and the vehicle control. To do. In this alarm process, the mode of the alarm may be changed according to the number of detected groups. For example, an alarm sound may be generated for the number of detected moving objects crossing the path (when three moving objects crossing the path are detected, a sound such as “beep, between, beep, between, etc.”) .) Also, an upper limit may be set for the number of alarm sounds in order to avoid the troublesomeness of alarms. Alternatively, the frequency of the warning sound may be changed according to the number of detected moving objects crossing the course. Thus, not only the fact that there is a moving object crossing the course but also information regarding the number of moving objects crossing the course can be provided to the driver.

  Further, the average value of the collision times T of the feature points of the same group may be calculated, and this may be used as the collision time of the moving object, and the warning mode may be changed depending on the length of the collision time. For example, in the case of an alarm device using a monitor, the blinking speed of a rectangle surrounding a path crossing object on the monitor may be changed according to the collision time. Thereby, it is also possible to provide the vehicle driver with information regarding the time when there is a possibility of collision with a moving object.

10 video camera 20 image processing ECU
30 Control device 100 Moving object detection device

Claims (9)

Photographing means mounted on the vehicle so as to photograph the traveling direction of the vehicle;
In the image photographed by the photographing means, the feature point is extracted, and an optical flow calculating means for calculating the optical flow of each extracted feature point;
From the optical flow of each feature point calculated by the optical flow calculation means, vanishing point calculation means for determining the vanishing point of each feature point existing on the extension line of the optical flow;
An angle formed by a line segment connecting the infinity point in the image captured by the image capturing unit and the focal point of the imaging unit, and a line segment connecting the vanishing point of each feature point and the focal point of the imaging unit is defined as the vehicle. Angle calculating means for calculating the relative approach angle of each feature point with respect to the traveling direction of
Determining means for determining that the moving object indicated by the corresponding feature point is a moving object crossing the traveling direction of the vehicle when the relative approach angle calculated by the angle calculating means is equal to or greater than a threshold; A moving object detection device comprising:   2. The vanishing point calculating unit obtains an intersection point between a virtual horizontal line extending horizontally from an infinite point in the image and an extension line of the optical flow as a vanishing point of each feature point. The moving object detection apparatus described in 1. At least grouping means for grouping feature points that share the same vanishing point as feature points indicating the same moving object from the position of the vanishing point of each feature point calculated by the vanishing point calculating means,
The moving object detection according to claim 1 or 2, wherein the determination unit determines whether or not the moving object indicated by the grouped feature point is a moving object that crosses a traveling direction of the vehicle. apparatus. Collision time calculation means for calculating the collision time for each feature point until the distance between the vehicle and each feature point in the traveling direction of the vehicle becomes zero,
The moving object detection apparatus according to claim 3, wherein the grouping unit groups the characteristic points belonging to the collision time within a predetermined time width as characteristic points indicating the same moving object.   5. The moving object detection apparatus according to claim 3, wherein the grouping unit groups feature points existing within a predetermined distance range as feature points indicating the same moving object. In the determination means, when a determination result with a moving object crossing the traveling direction of the vehicle is obtained, the alarm means for issuing an alarm to the driver of the vehicle,
6. The alarm unit according to claim 3, wherein the alarm unit changes an alarm mode according to the number of moving objects determined by the determination unit as moving objects crossing the traveling direction of the vehicle. The moving object detection device described. In the determination means, when a determination result with a moving object crossing the traveling direction of the vehicle is obtained, the alarm means for issuing an alarm to the driver of the vehicle,
The moving object detection apparatus according to claim 4 or 5, wherein the warning unit changes a warning mode according to the length of the collision time calculated by the collision time determination unit.   The determining means includes a moving object before crossing a line extending the line connecting the focal point of the imaging means and an infinite point as a moving object that crosses the traveling direction of the vehicle, and a post-crossing movement after crossing. The moving object detection device according to claim 1, wherein an object is determined.   9. The movement according to claim 1, wherein the optical flow calculation unit calculates an optical flow from which a flow component generated by the movement of the photographing unit in the rotation direction accompanying the movement of the vehicle is removed. Object detection device.

Images