JP2012027595A - Start notification device, start notification method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform a notification at an appropriate timing by highly accurately detecting a start of a stopped vehicle.SOLUTION: When an area of a region including grouped optical flow is a threshold or more, this start notification device notifies a driver of a start of a forward vehicle whose moving direction is indicated by the optical flow. When an area of a region including feature points uninvolved in the regulation of the optical flow is less than the threshold, this start notification device notifies of the start of the forward vehicle. This method can thus notify the start of the forward vehicle that is stopped in front thereof to the driver driving the vehicle at an appropriate timing.

Description

  The present invention relates to a start notification device, a start notification method, and a program. More specifically, the present invention relates to a start notification device that notifies the start of a vehicle stopped in front of the host vehicle, and to notify the start of the vehicle stopped in front of the host vehicle. It is related with the start alerting method and program.

  2. Description of the Related Art In recent years, a driving support system that monitors behaviors such as start and stop of a vehicle located in the vicinity of the host vehicle based on an image output from a camera installed in the vehicle has been put into practical use. This type of driving support system monitors the behavior of a vehicle located in the vicinity of the host vehicle based on the magnitude of a vector (optical flow) indicating a time-series position change of feature points included in an image (for example, a patent) References 1 to 3).

JP 2009-0667240 A JP-A-7-239998 JP 2007-207274 A

  The device described in Patent Literature 1 extracts pixels constituting a brake lamp of a stopped vehicle from an image showing a vehicle that stops in front of the host vehicle (hereinafter also referred to as a stopped vehicle). Then, the start of the stopped vehicle is detected by utilizing the fact that the luminance of the pixels constituting the brake lamp changes between when the brake light of the stopped vehicle is turned on and when it is turned off.

  However, the behavior of a stopped vehicle does not necessarily link with a change in brake lights. For this reason, it may be difficult to reliably detect the behavior of the stopped vehicle.

  The apparatus described in Patent Document 2 divides an image of a stopped vehicle into a plurality of blocks. And based on the optical flow prescribed | regulated for every block, the start of a stop vehicle is detected.

  However, when detection based on an optical flow defined for each block is performed, the stopped vehicle may not be identified well depending on, for example, the lighting conditions for the stopped vehicle and the surrounding environment. Further, if detection is performed in consideration of the surrounding environment of the stopped vehicle, it is expected that the processing for performing detection becomes complicated.

  The device described in Patent Document 3 defines a monitoring line in an image of a stopped vehicle. Then, the start of the stopped vehicle is detected based on the amount of movement of the feature points on the monitoring line.

  However, when performing detection based only on the feature points on the monitoring line, it is conceivable that the feature points on the monitoring line cannot be accurately detected due to the influence of illumination or the like. In this case, it is expected that the detection of the behavior of the stopped vehicle will be inaccurate.

  The present invention has been made under the above-described circumstances, and an object thereof is to accurately detect the start of a stopped vehicle and to notify at an appropriate timing.

In order to achieve the above object, the start notification device of the present invention is:
A first feature point included in a first image obtained by photographing a moving body stopped in front of the host vehicle and a second feature point corresponding to the first feature point included in a second image are extracted. Extraction means;
Optical flow defining means for defining an optical flow starting from the first feature point and ending at the second feature point;
Grouping processing means for grouping the optical flow for each moving object;
First computing means for calculating an area of a first region including optical flows grouped as optical flows related to the same moving body;
Informing means for informing the start of the moving body stopped in front of the host vehicle when the area of the first region is equal to or greater than a threshold;
Is provided.

  The first area may be rectangular.

  A start point of a part of the grouped optical flows may be located at an outer edge of the first region.

The start notification device
A second computing unit that calculates an area of a second region that does not include the start point and the end point of the optical flow defined by the optical flow defining unit within a predetermined region;
The notification means may notify the start of the moving body stopped in front of the host vehicle when the area of the second region is equal to or smaller than a threshold value.

In addition, the start notification device
A second computing unit that calculates an area of a second region that does not include the start point and the end point of the optical flow defined by the optical flow defining unit within a predetermined region;
The notification means may suppress notification of the start of the moving body stopped in front of the host vehicle when the area of the second region is equal to or greater than a threshold value.

  The second area may be rectangular.

  The predetermined area may be an area in which the size in the horizontal direction corresponds to the width of the vehicle and the size in the vertical direction corresponds to the height of the vehicle.

  The predetermined area may be an area whose lateral size corresponds to a width of a lane in which the host vehicle travels.

The extraction means extracts the first feature point and the second feature point from detection areas of the first image and the second image,
The second calculation means may calculate the area of the second region within the predetermined region in the detection region.

The start notification method of the present invention includes:
Extracting a first feature point included in a first image obtained by photographing a moving body stopped in front of the own vehicle;
Extracting a second feature point corresponding to the first feature point included in a second image obtained by photographing a moving body stopped in front of the host vehicle;
Defining an optical flow having the first feature point as a start point and the second feature point as an end point;
Grouping the optical flows for each of the moving objects;
Calculating an area of a first region including optical flows grouped as optical flows related to the same moving object;
A step of notifying the start of the moving body stopped in front of the host vehicle when the area of the first region is equal to or greater than a threshold;
including.

The program of the present invention
On the computer,
A procedure for extracting a first feature point included in a first image obtained by photographing a moving object stopped in front of the host vehicle;
A procedure for extracting a second feature point corresponding to the first feature point included in a second image obtained by photographing a moving object stopped in front of the host vehicle;
A procedure for defining an optical flow having the first feature point as a start point and the second feature point as an end point;
A procedure for grouping the optical flows for each of the moving objects;
A step of calculating an area of a first region including an optical flow grouped as an optical flow related to the same moving body;
When the area of the first region is equal to or greater than a threshold, a procedure for notifying the start of the moving body stopped in front of the host vehicle;
Is executed.

  According to the present invention, when the area of the region including the optical flow is equal to or larger than the predetermined threshold, the vehicle start in which the moving direction is indicated by the optical flow is notified. For this reason, when there is a vehicle stopped in the immediate vicinity of the own vehicle, for example, even if a vehicle at a position away from the own vehicle starts, the start is not erroneously notified. Therefore, it is possible to accurately detect the start of the target vehicle and notify the vehicle at an appropriate timing.

1 is a block diagram of a start notification system according to a first embodiment. It is a figure for demonstrating the attachment position of an imaging device. It is a figure which shows the relative positional relationship of the own vehicle and a stop vehicle. FIG. 3 is a first diagram illustrating an image photographed by the photographing apparatus. It is FIG. (2) which shows the image image | photographed with the imaging device. It is a figure which shows an optical flow. It is a figure which shows the area | region containing an optical flow. It is a figure which shows the area | region containing the feature point which does not prescribe | regulate an optical flow. It is a figure for demonstrating the effect of a start alerting | reporting system. It is a block diagram of the start notification system concerning a 2nd embodiment. It is a flowchart (the 1) for demonstrating operation | movement of a start alerting | reporting apparatus. It is a flowchart (the 2) for demonstrating operation | movement of a start alerting | reporting apparatus.

<< First Embodiment >>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a start notification system 10 according to the present embodiment. The start notification system 10 is a system that detects the start of a stopped vehicle stopped in front of the host vehicle (vehicle 100) and notifies the driver of the start of the stopped vehicle. As shown in FIG. 1, the start notification system 10 includes a photographing device 20 and a start notification device 30.

  The photographing device 20 is a device that converts an image acquired by photographing a subject into an electric signal and outputs the electric signal. The imaging device 20 is attached to the upper part of the front window of the vehicle 100, for example, as shown in FIG. The photographing device 20 photographs the front of the vehicle 100. Then, information related to the image acquired by shooting is output to the start notification device 30.

  FIG. 3 is a diagram illustrating a relative positional relationship between the vehicle 100 and the vehicles 101 and 102. For example, consider a case where the vehicle 101 is stopped at the position indicated by the arrow a1 and the vehicle 102 is stopped at the position indicated by the arrow b1. From this state, when the vehicle 101 starts and reaches the position indicated by the arrow a2, the photographing device 20 firstly sets the vehicle 101 at the position indicated by the arrow a1 with reference to FIG. The vehicle 102 at the position indicated by the arrow b1 is photographed. Next, the vehicle 101 at the position indicated by the arrow a2 and the vehicle 102 at the position indicated by the arrow b1 are photographed.

  FIG. 4 is a diagram showing an image PH1 obtained by photographing the vehicle 101 at the position indicated by the arrow a1 in FIG. FIG. 5 is a diagram showing an image PH2 obtained by photographing the vehicle 101 at the position indicated by the arrow a2 in FIG. When the image capturing device 20 captures the images PH1 and PH2, the image capturing device 20 outputs information related to the images PH1 and PH2 to the start notification device 30.

  In the present embodiment, with respect to the above-described images PH1 and PH2, a point corresponding to the optical center of the imaging device 20 (an image center that is the intersection of the optical axis of the imaging device 20 and its image plane) is defined as an origin Xc (FOE: Focus of). Define the xy coordinate system as (Expansion). The origin Xc of the xy coordinate system coincides with the centers of the images PH1 and PH2.

  Returning to FIG. 1, the start notification device 30 is a device that notifies the start of a stopped vehicle that stops in front of the host vehicle. As shown in FIG. 1, the start notification device 30 includes a storage unit 31, a feature point extraction unit 32, a correlation value calculation unit 33, an optical flow definition unit 34, a grouping processing unit 35, a first calculation unit 36, and a second calculation. The unit 37 and the notification unit 38 are included.

  The storage unit 31 stores information on images sequentially output from the imaging device 20 in time series. Further, information as a processing result of each of the units 32 to 38 is sequentially stored.

  As can be seen from FIG. 4 and FIG. 5, the feature point extraction unit 32 has, for example, a rectangular detection region whose center coincides with the origin Xc for the images PH1 and PH2 based on the information stored in the storage unit 31. Define DA. This detection area DA is defined based on, for example, information such as the lateral width and height of the host vehicle, the focal length of the photographing apparatus 20, and the angle of view (viewing angle) stored in advance in the auxiliary storage unit 30c.

  Next, the feature point extraction unit 32 calculates a feature amount for each of the pixels in the detection area DA defined in the image PH1 and the detection area DA defined in the image PH2. Then, feature points included in the detection area DA defined in the images PH1 and PH2 are extracted based on the feature amounts. For example, the feature quantity f (x, y) of the pixel M (x, y) shown in the xy coordinate system defined on the image PH1 shown in FIG. 4 is the function I (x, y) indicating the luminance gradient of the image. ) Is expressed by the following equation (1).

  However, Ixx, Iyy, and Ixy are respectively expressed by the following equations (2) to (4). K is a constant.

The feature point extraction unit 32 first calculates a feature quantity f (x, y) for each pixel M (x, y) in the detection area DA of the image PH1 using Expression (1). Next, the feature point extraction unit 32 calculates the average value AVG (x, y) of the brightness of the pixels around the pixel M (x, y), and calculates the feature value f (x, y) as the average brightness. Divide the value AVG (x, y) by the fourth power. The feature point extraction unit 32 compares the comparison value V (= f (x, y) / AVG (x, y) ⁴ ) obtained thereby with a preset threshold value, and the comparison value V is equal to or greater than the threshold value. In this case, the pixel M (x, y) at this time is extracted as a feature point. The feature point extraction unit 32 also performs the above-described processing on the image PH2. Here, the reason why the feature quantity f (x, y) is divided by the fourth power of the average luminance value AVG (x, y) is to normalize the feature quantity f (x, y) with respect to the brightness. .

  FIG. 4 shows feature points P1, P2, P3, P4 related to the vehicle 101 extracted from the detection area DA of the image PH1, and feature points P5, P6, P7, P8 related to the vehicle 102. . FIG. 5 also shows feature points Q1, Q2, Q3, Q4 related to the vehicle 101 extracted from the detection area DA of the image PH2, and feature points Q5, Q6, Q7, Q8 related to the vehicle 102. ing. As can be seen by referring to the images PH1 and PH2, the feature point extraction unit 32 constitutes discontinuous points where the contours of the vehicles 101 and 102 reflected in the images PH1 and PH2 change sharply, and the vehicles 101 and 102. The points where the shape of the part changes discontinuously are extracted as feature points.

  When the feature point extraction unit 32 completes the extraction of the feature points for the images PH1 and PH2, the feature point extraction unit 32 outputs information about the extracted feature points to the storage unit 31 and also indicates that the feature point extraction has been completed. To notify. Here, for convenience of explanation, a case where eight feature points are extracted from each of the images PH1 and PH2 is described, but actually, a plurality of feature points are extracted from one image.

  Returning to FIG. 1, the correlation value calculator 33 sequentially selects the feature points P1 to P8 of the image PH1. Then, a correlation value between the selected feature point and the feature points Q1 to Q8 of the image PH2 is calculated.

  Specifically, as shown in FIG. 4, the correlation value calculation unit 33 defines a rectangular image (partial image of the image PH1) centered on the feature point P1 of the image PH1 as a template TF1. The template TF1 is an image composed of pixels arranged in a matrix of M rows and N columns. In the following description, the coordinates of the template TF1 refer to the coordinates of the center of the template TF1.

Next, the correlation value calculator 33 sequentially calculates the correlation value R of the template TF1 with respect to the image PH2, while moving the template TF1 in the vicinity of the feature points Q1 to Q8 of the image PH2. The correlation value R can be calculated using, for example, the following equation (5) indicating normalized cross-correlation. T (i, j) is the luminance of the pixel located in the i-th row and the j-th column of the template TF1. I (i, j) is the luminance of the pixel located in the i-th row and the j-th column of the partial image of the image PH2 overlapping the template TF1. Further, _IAVG is an average value of luminances of pixels constituting the partial image. _TAVG is an average value of the luminance of the pixels constituting the template. I _AVG and T _AVG are represented by the following formulas (6) and (7).

  When the correlation value calculation unit 33 calculates the correlation value R based on the above equation (5), the correlation value R is stored in the storage unit 31 in association with the position coordinates in the xy coordinate system of the template TF1. . The correlation value calculation unit 33 executes the same process as described above for the feature points P2 to P8 of the image PH1. When the correlation values R for all the feature points P1 to P8 are calculated, the optical flow defining unit 34 is notified that the calculation of the correlation values R is complete.

  Returning to FIG. 1, based on the correlation value R calculated by the correlation value calculation unit 33, the optical flow defining unit 34 defines an optical flow having the feature point of the image PH1 as the start point and the feature point of the image PH2 as the end point. To do.

  For example, the optical flow defining unit 34 specifies the feature point of the image PH2 closest to the coordinates of the template TF1 when the correlation value R calculated using the template TF1 is maximized. The feature point P <b> 1 is a feature point corresponding to the right end portion of the rear bumper of the vehicle 101. For this reason, the correlation value R calculated using the template TF1 is maximized when the center of the template TF1 substantially matches the feature point Q1 of the image PH2. Therefore, here, the feature point Q1 is specified as the feature point corresponding to the feature point P1.

  As shown in FIG. 6, the optical flow defining unit 34 defines an optical flow OP1 having a feature point P1 as a start point and a feature point Q1 as an end point in an xy coordinate system.

  In the same manner as described above, the optical flow defining unit 34 defines the optical flow OP2 having the feature point P2 as the start point and the feature point Q2 as the end point. Further, an optical flow OP3 having a feature point P3 as a start point and a feature point Q3 as an end point is defined. Further, an optical flow OP4 having a feature point P4 as a start point and a feature point Q4 as an end point is defined.

  Note that if the vehicle stopped in front of the vehicle 100 has not started when the photographing by the photographing device 20 is sequentially performed, the coordinates of the feature point of the image PH1 and the image corresponding to the feature point are displayed. The coordinates of the feature points of PH2 coincide. Then, the size of the optical flow defined by these both feature points becomes zero. In this case, the optical flow defining unit 34 does not define an optical flow. Therefore, when only the vehicle 101 is traveling, as shown in FIG. 6, only the optical flows OP1 to OP4 for the vehicle 100 are defined, and the optical flows for the characteristic points P5 to P8 and Q5 to Q8 are defined. Will not be.

  When the optical flow defining unit 34 defines the optical flows OP1 to OP4, the optical flow defining unit 34 outputs information on the optical flows OP1 to OP4 to the storage unit 31 and notifies the grouping processing unit 35 that the definition of the optical flow has been completed.

  The grouping processing unit 35 performs grouping of a specified group of optical flows. As shown in FIG. 6, in the present embodiment, a case where there are four optical flows related to the vehicle 101 is described for convenience of explanation. However, in practice, tens or hundreds of feature points can be extracted from an image obtained by photographing the vehicle 101. Dozens or hundreds of optical flows can be defined.

  The grouping processing unit 35 excludes optical flows including many noise components from tens or hundreds of optical flows, and groups the remaining optical flows. For example, when the vehicle 101 is in a complete linear motion, each straight line that matches each optical flow should intersect at the vanishing point VP. Therefore, the grouping processing unit 35 excludes the optical flow when the straight line that matches the optical flow is significantly away from the vanishing point VP, and regards the remaining optical flow as the optical flow related to the same mobile object. Group. The grouping is disclosed in detail in, for example, Japanese Patent Application Laid-Open No. 2008-97126. Here, it is assumed that the optical flows OP <b> 1 to OP <b> 4 related to the vehicle 101 are grouped as the optical flows of the vehicle 101.

  When the grouping of the optical flows is completed, the grouping processing unit 35 outputs information on the grouped optical flows to the storage unit 31 and notifies the first calculation unit 36 that the grouping is completed.

  The first calculation unit 36 calculates the area of the grouping region including the optical flow for each group including the grouped optical flows.

  For example, as shown in FIG. 7, the first arithmetic unit 36 defines a rectangular grouping area AR1 including the grouped optical flows OP1 to OP4. Then, the area S1 of the grouping area AR1 is calculated. When the area S1 of the grouping area AR1 is minimized, the start points of some optical flows are located at the outer edge of the grouping area AR1.

  When calculating the area S1 of the grouping area AR1, the first calculation unit 36 outputs the calculation result to the storage unit 31 and notifies the second calculation unit 37 that the calculation is completed.

  As can be seen with reference to FIGS. 4 and 5, the second calculation unit 37 defines a rectangular stationary object detection area SA whose center coincides with the origin Xc in the detection area DA. Then, the area of the rectangular stationary object detection area SA including the feature points that are in the stationary object detection area SA and do not define the optical flow (not the start point and end point of the optical flow) is calculated.

  In the detection area DA, the length in the X-axis direction is a length obtained by adding a margin width (for example, 2 m) to both sides of the width of the own vehicle (the vehicle 100) at a position 3 meters ahead of the vehicle 100, for example. The length in the direction corresponds to a region where the height of the host vehicle is added to the height of the host vehicle (for example, 1.5 m) at a position, for example, 3 m ahead of the vehicle 100. In this case, the detection area DA is an area in which an area where the detection target (vehicle 101 or the like) does not clearly exist is excluded from the images PH1 and PH2.

  This stationary object detection area SA has a length in the x-axis direction, which is a length obtained by adding a margin width (for example, 1 m) to both sides of the width of the vehicle 100 at a position 3 m ahead of the vehicle 100, for example. The length is an area corresponding to a rectangular area having a length obtained by adding an extra height (for example, 50 cm) to the height of the vehicle 100 at a position, for example, 3 m ahead of the vehicle 100. The stationary object detection area SA is preferably an area in which only the vehicle existing in front of the lane on which the vehicle 100 travels is reflected.

  Next, as can be seen with reference to FIG. 8, the second computing unit 37 does not define the optical flows OP1 to OP4 among the feature points in the stationary object detection area SA defined in the image PH2. Q7 is extracted. Next, the second calculation unit 37 defines a rectangular stationary feature point existence area AR2 including the extracted feature points Q5 and Q7. Then, the area S2 of the stationary feature point existence area AR2 is calculated. When the area of the stationary feature point existence area AR2 is minimized, some feature points are located on the outer edge of the stationary feature point existence area AR2.

  When calculating the area S2 of the stationary feature point existence area AR2, the second calculation unit 37 outputs the calculation result to the storage unit 31 and notifies the notification unit 38 that the calculation is completed.

  The notification unit 38 compares the area S1 of the grouping area AR1 stored in the storage unit 31 with the first threshold value, and compares the area S2 of the stationary feature point existence area AR2 with the second threshold value. Then, when the area S1 of the grouping area AR1 is equal to or larger than the threshold value and the area S2 of the stationary feature point existence area AR2 is equal to or smaller than the threshold value, the vehicle 101 is started in which the moving direction is indicated to the driver of the vehicle 100 by the optical flows OP1 to OP4. Is notified.

  As described above, in the present embodiment, when the area S1 of the grouping area AR1 including the grouped optical flows is equal to or larger than the threshold value, the vehicle 101 whose direction of movement is indicated to the driver by the optical flow of the grouping area AR1. The start of is notified. Thereby, it is possible to accurately notify the driver who drives the vehicle 100 of the start of the vehicle that has stopped forward at an appropriate timing. This will be specifically described below.

  For example, as shown in FIG. 3, consider a case where the vehicle 102 that stops at a position deviated from the travel route of the vehicle 100 and starts at the end of the field of view of the photographing apparatus 20 starts. In this case, as shown in FIG. 9, optical flows OP5 to OP8 starting from the characteristic points P5 to P8 of the image PH1 and ending with the characteristic points Q5 to Q8 of the image PH2 are defined. These optical flows OP5 to OP8 are defined at the left end of the image PH2.

  The area of the grouping region in which an optical flow resulting from the start of a vehicle deviating from the travel route of the vehicle 100 or a vehicle that is stopped at a position relatively far from the vehicle 100 appears is a vehicle that is stopped immediately in front of the vehicle 100. The area is smaller than the area of the grouping region in which the optical flows OP1 to OP4 resulting from the start of 101 appear.

  Specifically, as can be seen with reference to FIGS. 8 and 9, the area of the grouping area AR <b> 3 (see FIG. 9) where the optical flow due to the start of the vehicle 102 appears is the optical flow due to the start of the vehicle 101. The area of the appearing grouping area AR1 (see FIG. 8) is smaller. For this reason, when the area of the grouping region including the optical flow is equal to or greater than a predetermined threshold, the vehicle is stopped immediately in front by notifying the driver of the vehicle 100 of the start of the preceding vehicle. Regardless, it is possible to prevent the start from being notified by mistake. Therefore, it is possible to notify the driver of the vehicle 100 of the start of the preceding vehicle at an appropriate timing.

  Further, in the present embodiment, when the area S2 of the stationary feature point existence area AR2 including the feature points that are not involved in the definition of the optical flow is equal to or less than the threshold value, the vehicle is started in which the moving direction is indicated by the optical flow to the driver. Informed. Thereby, it is possible to accurately notify the driver who drives the vehicle 100 of the start of the vehicle that has stopped forward at an appropriate timing. This will be specifically described below.

  For example, consider a case where the vehicle 101 at the position indicated by the arrow a1 in FIG. 3 does not start. In this case, an optical flow having the feature points P1 to P4 of the image PH1 as the start points and the feature points Q1 to Q4 of the image PH2 as the end points is not defined. Of the feature points Q1 to Q8 of the image PH2, the feature points Q1 to Q4 related to the vehicle 101 are stationary feature points Q1 to Q4 that do not define an optical flow. When the vehicle 101 is stopped immediately in front of the vehicle 100, the stationary feature point existence area AR4 including the stationary feature points Q1 to Q4 caused by the vehicle 101 occupies the image PH2, as shown in FIG. The proportion increases to some extent.

  In the present embodiment, when the area of the stationary feature point existence area including the stationary feature point caused by the vehicle positioned immediately in front is equal to or smaller than a predetermined threshold, the driver of the vehicle 100 is notified of the start of the preceding vehicle. Then, when the area of the stationary feature point existence region is equal to or larger than a predetermined threshold, notification of the start of the preceding vehicle is suppressed. Thereby, it can be avoided that the start is inadvertently notified even though the vehicle is stopped immediately in front. Therefore, it is possible to notify the driver of the vehicle 100 of the start of the preceding vehicle at an appropriate timing.

  In the present embodiment, the process for extracting the feature points and the process for defining the optical flow are performed on the detection area DA defined in the images PH1 and PH2. For this reason, it is not necessary to execute the process necessary for notifying the start of the entire image, and the processing time can be shortened.

  The stationary object detection area SA is defined by the detection area DA. In the present embodiment, the stationary object detection area SA has a length in the x-axis direction that is a length obtained by adding a margin width (for example, 1 m) on both sides of the width of the vehicle 100 and a length in the y-axis direction. The region on the image corresponds to a rectangular region having a length obtained by adding a marginal height (for example, 50 cm) to the height of 100. Not limited to this, the length in the X-axis direction may be a length corresponding to the lane width in which the vehicle 100 travels, for example, at a position 3 m ahead of the vehicle 100. Here, the lane width may be a predetermined design width, or may be a width calculated based on a white line detected from an image.

<< Second Embodiment >>
Next, a second embodiment of the present invention will be described with reference to the drawings. In addition, about the structure same or equivalent to said each embodiment, while using an equivalent code | symbol, the description is abbreviate | omitted or simplified.

  The start notification system 10 according to this embodiment differs from the start notification system 10 according to the above embodiment in that the start notification device 30 is realized by the same configuration as a general computer or a device such as a microcomputer. It is different.

  FIG. 10 is a block diagram showing a physical configuration of the start notification system 10. As shown in FIG. 10, the start notification system 10 includes a photographing device 20 and a start notification device 30 including a computer.

  The start notification device 30 includes a central processing unit (CPU) 30a, a main storage unit 30b, an auxiliary storage unit 30c, a display unit 30d, an input unit 30e, an interface unit 30f, and a system bus 30g that interconnects the above units. It is configured.

  CPU30a performs the process mentioned later with respect to the images PH1 and PH2 acquired by the imaging device 20 according to the program memorize | stored in the auxiliary storage part 30c.

  The main storage unit 30b includes a RAM (Random Access Memory) and the like, and is used as a work area for the CPU 30a.

  The auxiliary storage unit 30c includes a nonvolatile memory such as a ROM (Read Only Memory), a magnetic disk, and a semiconductor memory. The auxiliary storage unit 30c stores programs executed by the CPU 30a, various parameters, and the like. In addition, information about an image output from the photographing apparatus 20 and information including a processing result by the CPU 30a are sequentially stored.

  The display unit 30d includes a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), or the like, and displays the processing result of the CPU 30a.

  The input unit 30e includes a key switch and a pointing device. The operator's instruction is input via the input unit 30e and notified to the CPU 30a via the system bus 30g.

  The interface unit 30f includes a serial interface or a LAN (Local Area Network) interface. The imaging device 20 is connected to the system bus 30g via the interface unit 30f.

  The flowcharts of FIGS. 11 and 12 correspond to a series of processing algorithms of a program executed by the CPU 30a. Hereinafter, the process which the start alerting | reporting apparatus 30 performs is demonstrated, referring FIG.11 and FIG.12. This process is executed when the start notification system 10 is activated and information related to an image captured by the imaging device 20 is output. As a premise, it is assumed that the image capturing device 20 sequentially outputs an image PH1 shown in FIG. 4 and an image PH2 shown in FIG.

  First, in the first step S101, the CPU 30a initializes timers, flags, parameters, variables, and the like (not shown) as initialization processing immediately after the activation of the CPU 30a. As a result, the timer is initialized and measurement of the elapsed time from the initialized time is started.

  In the next step S102, the CPU 30a determines whether or not the vehicle 100 is stopped based on the output from the speedometer of the vehicle 100. When the speed of the vehicle 100 is equal to or higher than the threshold value, the CPU 30a determines that the vehicle is traveling (step S102: No), and proceeds to step S112.

  In step S112, the CPU 30a initializes timers, flags, parameters, variables, and the like as in step S101. As a result, the timer is initialized and measurement of the elapsed time from the initialized time is started. After completing the process in step S112, the CPU 30a returns to step S102, and thereafter repeatedly executes the processes in step S102 and step S112 until the determination in step S102 is affirmed.

  On the other hand, when the speed of the vehicle 100 is equal to or less than the threshold value in step S102, the CPU 30a determines that the vehicle 100 is stopped (step S102: Yes), and proceeds to the next step S103.

  In step S103, the CPU 30a obtains information related to the latest image PH2 stored in the auxiliary storage unit 30c and information related to the image PH1 captured one frame before the image PH2.

  In the next step S104, the CPU 30a defines the detection area DA. This detection area DA is defined based on, for example, information such as the lateral width and height of the host vehicle, the focal length of the photographing apparatus 20, and the angle of view (viewing angle) stored in advance in the auxiliary storage unit 30c.

  In the next step S105, the CPU 30a extracts feature points from the detection area DA defined in the image PH1 and the detection area DA defined in the image PH2. For example, the CPU 30a calculates a feature amount for each pixel in the detection area DA of the images PH1 and PH2, and based on the feature amount, calculates a feature point included in the detection area DA defined in the images PH1 and PH2. Extract. For example, as shown in FIG. 4, feature points P1, P2, P3, and P4 related to the vehicle 101 and feature points P5, P6, P7, and P8 related to the vehicle 102 are extracted from the image PH1. Further, as shown in FIG. 5, feature points Q1, Q2, Q3, and Q4 related to the vehicle 101 and feature points Q5, Q6, Q7, and Q8 related to the vehicle 102 are extracted from the image PH2.

  In the next step S106, the CPU 30a sequentially selects feature points P1 to P8 of the image PH1. Then, a correlation value between the selected feature point and the feature points Q1 to Q8 of the image PH2 is calculated. Specifically, first, the CPU 30a sequentially calculates the correlation value R of the template TF1 with respect to the image PH2 while moving the template TF1 centered on the feature point P1 of the image PH1 in the vicinity of the feature points Q1 to Q8 of the image PH2. To do. CPU30a performs the above-mentioned process also about the feature points P2-P8.

  In the next step S107, as can be seen with reference to FIG. 6, based on the calculated correlation value R, the CPU 30a has optical flows OP1 to OP4 with the feature point of the image PH1 as the start point and the feature point of the image PH2 as the end point. Is specified. Here, it is assumed that the optical flow is not defined when it can be determined that the feature point of the image PH1 and the coordinate of the feature point of the image PH2 corresponding to the feature point match.

  In the next step S108, the CPU 30a performs grouping of a specified group of optical flows. As shown in FIG. 6, in the present embodiment, a case where there are four optical flows related to the vehicle 101 is described for convenience of explanation. However, in practice, tens or hundreds of feature points can be extracted from an image obtained by photographing the vehicle 101. Dozens or hundreds of optical flows can be defined. Therefore, the CPU 30a excludes optical flows including a lot of noise components from tens or hundreds of optical flows, and groups the remaining optical flows. Thereby, the optical flows OP1 to OP4 defined by the feature points related to the vehicle 101 are grouped.

  In the next step S109, the CPU 30a executes a subroutine 200 shown in FIG. 12 in order to detect the start of the vehicle stopped in front of the vehicle 100.

  In the first step S201 of the subroutine 200, the CPU 30a calculates the area of the grouping region including the optical flow for each group including the grouped optical flows. For example, as illustrated in FIG. 7, the CPU 30a defines a rectangular grouping area AR1 including the grouped optical flows OP1 to OP4. Then, the area S1 of the grouping area AR1 is calculated.

  In the next step S202, as shown in FIGS. 4 and 5, the CPU 30a defines a rectangular stationary object detection area SA whose center coincides with the detection area DA in the detection area DA. Then, an area of a rectangular stationary feature point existing area that includes a stationary feature point that is in the stationary object detection area SA and that does not define an optical flow (not a starting point and an ending point of the optical flow) is calculated. Specifically, as can be seen with reference to FIG. 8, feature points Q5 and Q7 that do not define the optical flows OP1 to OP4 are extracted from the feature points in the stationary object detection area SA defined in the image PH2. Next, the CPU 30a defines a stationary feature point existence area AR2 including the extracted feature points Q5 and Q7, and calculates an area S2 of the stationary feature point existence area AR2.

  In the next step S203, the CPU 30a determines whether or not the area S1 of the grouping area AR1 is equal to or greater than a threshold value. When the area S1 is equal to or greater than the threshold (step S203: Yes), the CPU 30a proceeds to the next step S204.

  In the next step S204, the CPU 30a determines whether or not the area S2 of the stationary feature point existence area AR2 is equal to or less than a threshold value. When the area S2 is equal to or smaller than the threshold (step S204: Yes), the CPU 30a proceeds to the next step S205.

  In the next step S205, the CPU 30a determines whether or not the elapsed time measured by the timer exceeds a preset time (set time). If the elapsed time exceeds the set time, the CPU 30a proceeds to the next step S206 and turns on the start flag. Then, the subroutine 200 is terminated and the process proceeds to the next step S110.

  On the other hand, if the determination in step S203 described above is negative (step S203: No), or if the determination in step 204 is negative (step S204: No), the CPU 30a proceeds to step S207, Initialize the timer. Then, the CPU 30a proceeds to step S208 to turn off the start flag, then ends the subroutine 200 and proceeds to the next step S110.

  If the determination in step S205 is negative (step S205: No), the CPU 30a proceeds to step S208 to turn off the start flag, and then ends the subroutine 200 to execute the next step. The process proceeds to S110.

  In the next step S110, the CPU 30a determines whether or not the vehicle 101 stopped in front of the host vehicle has started. Specifically, when the start flag is off, the CPU 30a determines that the preceding vehicle 101 has not started (step S110: No), and returns to step S102. Thereafter, the CPU 30a repeatedly executes the processing from step S102 to step S110 until the determination in step S110 is affirmed.

  On the other hand, when the start flag is on, the CPU 30a determines that the preceding vehicle 101 has started, and proceeds to the next step S111.

  In the next step S111, the CPU 30a notifies the start of the preceding vehicle to the outside. The notification of the start in step S11 may be, for example, outputting a signal indicating the start of the preceding vehicle to an external device such as a speaker.

  When executing the processing in step S111, the CPU 30a returns to step S102, and thereafter repeatedly executes the processing from step S102 to step S111.

  As described above, in this embodiment, when the area S1 of the grouping area AR1 including the grouped optical flows is equal to or larger than the threshold value, the driver is notified of the start of the vehicle 101 whose movement direction is indicated by the optical flows. Is done. Thereby, it is possible to accurately notify the driver who drives the vehicle 100 of the start of the vehicle that has stopped forward at an appropriate timing.

  Further, in the present embodiment, when the area S2 of the stationary feature point existence area AR2 including the feature points that are not involved in the definition of the optical flow is equal to or less than the threshold value, the vehicle is started in which the moving direction is indicated by the optical flow to the driver. Informed. Thereby, it is possible to accurately notify the driver who drives the vehicle 100 of the start of the vehicle that has stopped forward at an appropriate timing.

  As mentioned above, although embodiment of this invention was described, this invention is not limited by the said embodiment.

  For example, in the above embodiment, the feature quantity f (x, y) is calculated using the equation (1), but the present invention is not limited to this. For example, the feature amount may be a so-called KLT feature amount min (λ1, λ2). In this case, the comparison value V can be calculated by dividing the feature amount by the square of the average value AVG (x, y) described above. Note that λ1 and λ2 are represented by the following equations (8) and (9), respectively.

  A to C are represented by the following formula (10). Here, Ix and Iy indicate gradients in the X-axis direction and Y-axis direction of the luminance I (x, y) at the position (x, y) on the image. Specifically, it is represented by the following formula (11) and formula (12).

  Further, the function of the start notification device 30 according to each of the above embodiments can be realized by dedicated hardware or by a normal computer system.

  In the second embodiment, the programs stored in the auxiliary storage unit 30c of the start notification device 30 are a flexible disk, a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), an MO (Magneto). A device that executes the above-described processing may be configured by storing and distributing in a computer-readable recording medium such as -Optical disk) and installing the program in the computer.

  Further, the program may be stored in a disk device or the like included in a predetermined server device on a communication network such as the Internet, and may be downloaded onto a computer by being superimposed on a carrier wave, for example.

  It should be noted that the present invention can be variously modified and modified without departing from the broad spirit and scope of the present invention. Further, the above-described embodiment is for explaining the present invention, and does not limit the scope of the present invention.

  The start notification device, the collision time calculation method, and the program according to the present invention are suitable for notification of the start of the preceding vehicle.

DESCRIPTION OF SYMBOLS 10 Start notification system 20 Imaging device 30 Start notification device 30a CPU
30b main storage unit 30c auxiliary storage unit 30d display unit 30e input unit 30f interface unit 30g system bus 31 storage unit 32 feature point extraction unit 33 correlation value calculation unit 34 optical flow definition unit 35 grouping processing unit 36 first calculation unit 37 second Arithmetic unit 38 Informing unit 100, 101, 102 Vehicle AR1 Grouping area AR2 Stationary feature point existence area AR3 Grouping area AR4 Stationary feature point existence area DA detection area OP1-OP8 Optical flow P1-P8 Feature point PH1, PH2 Image Q1-Q8 Feature Point S1, S2 Area SA Stationary object detection area TF1 Template VP Vanishing point Xc Origin

Claims (11)

A first feature point included in a first image obtained by photographing a moving body stopped in front of the host vehicle and a second feature point corresponding to the first feature point included in a second image are extracted. Extraction means;
Optical flow defining means for defining an optical flow starting from the first feature point and ending at the second feature point;
Grouping processing means for grouping the optical flow for each moving object;
First computing means for calculating an area of a first region including optical flows grouped as optical flows related to the same moving body;
Informing means for informing the start of the moving body stopped in front of the host vehicle when the area of the first region is equal to or greater than a threshold;
A start notification device.   The start notification device according to claim 1, wherein the first region is rectangular.   The start notification device according to claim 2, wherein a start point of a part of the grouped optical flows is located at an outer edge of the first region. A second computing unit that calculates an area of a second region that does not include the start point and the end point of the optical flow defined by the optical flow defining unit within a predetermined region;
4. The start notification device according to claim 1, wherein when the area of the second region is equal to or less than a threshold value, the notification unit notifies the start of the moving body stopped in front of the host vehicle. . A second computing unit that calculates an area of a second region that does not include the start point and the end point of the optical flow defined by the optical flow defining unit within a predetermined region;
The start according to any one of claims 1 to 3, wherein the notification means suppresses a start notification of the moving body that has stopped in front of the host vehicle when the area of the second region is equal to or greater than a threshold value. Notification device.   The start notification device according to claim 4 or 5, wherein the second region is rectangular.   The predetermined region is a region in which a horizontal size corresponds to a width of the own vehicle and a vertical size corresponds to a height of the own vehicle. The start notification device described.   The start notification device according to any one of claims 4 to 6, wherein a size of the predetermined region corresponds to a width of a lane in which the host vehicle travels. The extraction means extracts the first feature point and the second feature point from detection areas of the first image and the second image,
The start notification device according to any one of claims 4 to 8, wherein the second calculation means calculates an area of the second region within the predetermined region in the detection region. Extracting a first feature point included in a first image obtained by photographing a moving body stopped in front of the own vehicle;
Extracting a second feature point corresponding to the first feature point included in a second image obtained by photographing a moving body stopped in front of the host vehicle;
Defining an optical flow having the first feature point as a start point and the second feature point as an end point;
Grouping the optical flows for each of the moving objects;
Calculating an area of a first region including optical flows grouped as optical flows related to the same moving object;
A step of notifying the start of the moving body stopped in front of the host vehicle when the area of the first region is equal to or greater than a threshold;
A start notification method including: On the computer,
A procedure for extracting a first feature point included in a first image obtained by photographing a moving object stopped in front of the host vehicle;
A procedure for extracting a second feature point corresponding to the first feature point included in a second image obtained by photographing a moving object stopped in front of the host vehicle;
A procedure for defining an optical flow having the first feature point as a start point and the second feature point as an end point;
A procedure for grouping the optical flows for each of the moving objects;
A step of calculating an area of a first region including an optical flow grouped as an optical flow related to the same moving body;
When the area of the first region is equal to or greater than a threshold, a procedure for notifying the start of the moving body stopped in front of the host vehicle;
A program for running

Images