JP2008257378A - Object detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detection device 10 capable of always setting an effective caution region and detecting various objects which are likely to collide with a present vehicle. <P>SOLUTION: This object detection device is provided with: an image acquisition means 12 for acquiring a time-series image in the periphery of its own vehicle; a moving vector extraction means 16 for extracting the moving vector of an object existing in the image based on the acquired image; a disappearing point calculation means 18 for calculating an FOE indicating the traveling direction of its own vehicle; a caution region setting means 14 for setting a caution region in an image based on the calculated FOE; and an object decision means 22 for deciding an obstacle which is likely to be brought into contact with its own vehicle in the set caution region. The caution region setting means 14 sets a plurality of caution regions, and the obstacle detection means 22 decides the obstacle based on the direction of the moving vector to the FOE and which caution region does the moving vector exist in. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an object detection device.

There has been proposed an apparatus for detecting an obstacle ahead of the host vehicle by mounting a camera on a car, photographing the surrounding road environment with the camera, and analyzing the photographed image.
For example, in the technique described in Patent Document 1, a captured image of a travel destination of the host vehicle is acquired from one camera, and a detection region virtually fixed in real space is set in the captured image. Then, an optical flow in the set detection area is calculated, and a flying object is detected based on the optical flow.

  When the captured image is read, a lane mark (white line) indicating a boundary portion of the road is first extracted from the captured image. The lane mark is extracted from the captured image by extracting a vertical edge from the captured image using an image filter called a Sobel filter and performing a process called Hough transform on the edge to obtain a linear shape. be able to. When the lane mark is extracted from the captured image, a detection area for detecting an object from the captured image is set in the captured image. At this time, the setting position of each detection area is determined based on the position of the extracted lane mark, and the detection area is arranged at predetermined intervals along the lane mark so that it follows the road on which the host vehicle is traveling. The detection area is set at predetermined intervals. Further, at this time, two detection areas are arranged side by side at predetermined horizontal intervals. Two rows of interdigital detection areas are set along the outer and inner sides and along the road.

The area set outside in the above detection area setting is set as a detection area, and the area set inside is set as a determination area. Based on the optical flows respectively calculated in the detection area and the determination area, the moving speed of the object in the real space is calculated for both the detection area and the determination area. The movement speeds are compared, and if the difference is equal to or less than a predetermined value, the detected object is detected as a flying object. As a result, it is possible to prevent all objects moving in the road direction from being detected as flying objects, and to detect even objects at positions away from the road by mistake.
JP 2003-281700 A

  In the conventional technology, since the detection area and the determination area are set using the lane mark, there is a problem that the area cannot be set when the lane mark cannot be extracted. It is out. That is, it is impossible to extract a moving object that pops out in a mountain road without a white line, an intersection where a white line is broken, or a scene such as when traveling on a curve. Further, in the prior art, only an object flying out from the lateral direction is targeted, and an opposing object and an overtaking object cannot be detected.

  Therefore, an object of the present invention is to provide an object detection device that can always set an effective attention area and detect various obstacles that may collide with the host vehicle.

  In order to solve the above-mentioned problem, the invention according to claim 1 is directed to an image acquisition unit (for example, the image acquisition unit 12 in the embodiment) that acquires a time-series image around the host vehicle, and the image based on the image. A movement vector extraction means for extracting a movement vector of an object existing in the vehicle (for example, movement vector extraction means 16 in the embodiment), and a vanishing point calculation means for calculating a vanishing point representing the traveling direction of the host vehicle (for example, the embodiment) Vanishing point calculating means 18), attention area setting means for setting an attention area in the image based on the vanishing point (for example, attention area setting means 14 in the embodiment), and the host vehicle within the attention area An obstacle detection means (for example, the obstacle determination means 22 in the embodiment) for determining an obstacle that may come into contact with the object detection device (for example, implementation) In the object detection apparatus 10), the attention area setting means sets a plurality of attention areas, and the obstacle determination means determines the direction of the movement vector with respect to the vanishing point and the movement vector in any attention area. The obstacle is determined based on whether it exists.

  The invention according to claim 2 is characterized in that the attention area setting means sets the first attention area (for example, attention area 1 in the embodiment) around the vanishing point in the image, and the obstacle determination means. Determines that the object is the obstacle (for example, a jumping-out vehicle in the embodiment) when the movement vector exists in the first attention area and moves toward the vanishing point. And

  In the invention according to claim 3, the caution area setting means sets the first caution area around the vanishing point in the image, and the obstacle determination means has the movement vector as the first movement vector. It is characterized in that the object is determined to be the obstacle (for example, an oncoming vehicle in the embodiment) when it exists in the attention area and moves away from the vanishing point.

  In the invention according to claim 4, the caution area setting means sets the second caution area (for example, caution area 2 in the embodiment) in at least one of the lower right end and the lower left end in the image, and the obstacle The object determination means determines that the object is the obstacle (for example, the overtaking vehicle in the embodiment) when the movement vector exists in the second attention area and heads toward the vanishing point. It is characterized by.

  The invention according to claim 5 is characterized in that the attention area setting means sets the size of the attention area based on the position and direction of the movement vector.

  The invention according to claim 6 is characterized by comprising noise removal means for removing the movement vector having a correlation value with a plurality of movement vectors less than a predetermined value among the movement vectors of the object.

  The invention according to claim 7 is characterized in that the movement vector extraction means adjusts the acquisition interval of the image when extracting the movement vector based on the speed of the object.

  According to the first aspect of the present invention, since the attention area is set in the image based on the vanishing point without using the lane mark or the like, it is possible to always set the effective attention area. Further, since the obstacle is determined based on the direction of the movement vector with respect to the vanishing point and in which attention area the movement vector exists, various obstacles that may collide with the host vehicle are accurately detected. be able to.

  Since the flying vehicle appears in the vicinity of the vanishing point, and its movement vector is directed in the direction approaching the vanishing point, the flying vehicle can be detected with high accuracy.

  Since the oncoming vehicle appears in the vicinity of the vanishing point and its movement vector goes in a direction away from the vanishing point, the oncoming vehicle can be detected with high accuracy.

  Since the overtaking vehicle appears at the lower right end or the lower left end in the image, and its movement vector goes in the direction approaching the vanishing point, the overtaking vehicle can be detected with high accuracy.

  According to the fifth aspect of the invention, since the attention area is set based on the detected movement vector of the obstacle, it is possible to set the attention area where the obstacle can be detected continuously.

  According to the invention which concerns on Claim 6, an obstruction can be detected accurately.

  According to the seventh aspect of the present invention, the movement vector of the object moving at a low speed can be reliably extracted by adjusting the image acquisition interval.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram of an object detection apparatus according to this embodiment. The object detection apparatus 10 according to the present embodiment includes an image acquisition unit (imaging unit) 12 that acquires a time-series image around the host vehicle, and a movement that extracts a movement vector of an object existing in the image based on the image. A vector extracting unit 16; a vanishing point calculating unit 18 for calculating a vanishing point representing a traveling direction of the host vehicle; a caution region setting unit 14 for setting a caution region in the image based on the vanishing point; The vehicle includes an obstacle determination unit 22 that determines a partner vehicle (obstacle) 30 that may contact the host vehicle in an area, and a notification unit 24 that notifies the presence of the partner vehicle. The movement vector extraction means 16, the vanishing point calculation means 18, the attention area setting means 14, and the obstacle determination means 22 are constructed in a computer 20 mounted on the host vehicle.

(Image acquisition means)
The image acquisition unit 12 acquires a time-series image around the host vehicle, and specifically includes a single CCD camera, CMOS, or the like. In addition, since the object detection device can be configured with one camera in this embodiment compared to the case where the object detection device is configured with two cameras to use stereo vision, the vehicle cost can be reduced. Can do. This image acquisition means 12 is fixed in the vicinity of a rearview mirror in the passenger compartment in order to capture an image in front of the host vehicle. An image taken by the image acquisition unit 12 is recorded in a memory (not shown) and output to the movement vector extraction unit 16 and the like.

(Moving vector extraction means)
The movement vector extracting means 16 extracts a movement vector of an object existing in the image for each pixel of the image.
FIG. 2 is an explanatory diagram of the movement vector. In general, an optical flow is known as one of methods for extracting a motion of an object from a moving image. The optical flow is a movement vector 32 of the object 30, and is based on the movement amount of a point having the same level of luminance under the assumption that the change in luminance of the same object is small between two temporally continuous images V0 and V1. Calculated.

  Now, let the luminance value of the point P (x, y) on the image at time t be E (x, y, t). Assuming that P moves by (Δx, Δy) from time t to t + Δt and the luminance value does not change, Formula 1 is established.

数式１の右辺をテイラー展開して、２次以降の項は微小値であるため無視すると、数式２が得られる。

If the right side of Equation 1 is Taylor-expanded and the terms after the second order are negligible values, Equation 2 is obtained.

数式２の両辺をΔｔで割ると、数式３が得られる。

When both sides of Equation 2 are divided by Δt, Equation 3 is obtained.

この数式３を、以下の数式４を用いて書き直すと、数式５が得られる。

By rewriting Equation 3 using Equation 4 below, Equation 5 is obtained.

この数式５を満たす（ｕ，ｖ）が、オプティカルフローである。
しかし、求める変数が（ｕ，ｖ）の２つであるのに対して、関係式が数式５の１つなので、これのみではオプティカルフローを求めることはできない。そこで、オプティカルフロー（ｕ，ｖ）の算出にあたり、数式５のほかに、「オプティカルフローは滑らかに変化する」という条件を付け加える。オプティカルフロー（ｕ，ｖ）の変化量は、次の数式６で表される。

The optical flow is (u, v) that satisfies Equation (5).
However, since there are two variables (u, v) to be obtained, the relational expression is one of Equation 5, so that it is not possible to obtain an optical flow by this alone. Therefore, in calculating the optical flow (u, v), in addition to Equation 5, a condition that “the optical flow changes smoothly” is added. The amount of change in the optical flow (u, v) is expressed by the following formula 6.

数式５および数式６より、次の数式７を立てる。

From Equation 5 and Equation 6, the following Equation 7 is established.

この数式７を最小化する（ｕ，ｖ）が、求めるオプティカルフローである。この数式７は、一般的な変分法を用いて解くことができる。

The optical flow to be obtained is (u, v) that minimizes Equation 7. Equation 7 can be solved using a general variational method.

It is desirable that the movement vector extraction means 16 shown in FIG. 1 can adjust the image acquisition interval when extracting the movement vector.
FIG. 3 is an explanatory diagram of a movement vector alternative calculation method. A vehicle moving at a low speed has a small amount of movement between successive images. Therefore, it is difficult to extract an optical flow between the current image V0 (time t) and the image V-1 immediately before (time t−Δt). Therefore, the image acquisition interval when extracting the optical flow is lengthened, and the optical flow is extracted between the current image V0 (time t) and the image V-n at time t−nΔt. By adjusting the image acquisition interval based on the speed of the vehicle in this way, even if the vehicle is moving at a low speed, the amount of movement between the images increases, so that the optical flow 32 can be reliably extracted. it can. If the optical flow extracted in this way is multiplied by 1 / n, it can be handled in the same manner as the optical flow extracted between successive images. In this embodiment, n = 2.

(Noise removal means)
The movement vector extraction means 16 shown in FIG. 1 preferably includes noise removal means (not shown). The noise removing means removes a movement vector having a correlation value with a plurality of movement vectors less than a predetermined value from movement vectors of the object.
FIG. 4 is an explanatory diagram of noise removal of the movement vector. The movement vector 32 obtained as described above may include noise (noise component) 32n. The noise 32n can be removed by the following method using the fact that the noise 32n has a small correlation with the surrounding movement vector 32.

  In general, the optical flow 32 in the local region in the same object has properties that are similar in direction and size (high correlation). On the other hand, the noise 32n is different in direction and size from the surrounding movement vector (correlation is small). Therefore, an arbitrary flowA shown in Formula 8 in which the start point coordinates are (xa0, ya0) and the end point coordinates are (xa1, ya1) is considered.

このflowAの周囲Ｎ×Ｎピクセルに存在するｎ個のflowXkと、flowAとの平均相関値cor_aveを、数式９により算出する。

An average correlation value cor_ave between n flowXk existing in N × N pixels around this flowA and flowA is calculated by Equation 9.

なお本実施形態では、Ｎ＝５およびｎ＞１に設定した。
この平均相関値cor_aveが、閾値Threshold_Noise以下であるか、数式１０により判断する。

In this embodiment, N = 5 and n> 1.
Whether the average correlation value cor_ave is equal to or less than the threshold value Threshold_Noise is determined by Expression 10.

なお本実施形態では、Threshold_Noise＝０．８に設定した。数式１０を満たす場合にはflowAをノイズであるとみなす。このようにノイズとみなされたflowAを移動ベクトルから除去することにより、障害物を精度良く検出することができる。

In this embodiment, Threshold_Noise = 0.8 is set. When Formula 10 is satisfied, flowA is regarded as noise. Thus, by removing the flow A regarded as noise from the movement vector, the obstacle can be detected with high accuracy.

(Vanishing point calculation means)
The vanishing point calculating means 18 shown in FIG. 1 calculates a vanishing point that represents the traveling direction of the host vehicle.
FIG. 5 is an explanatory diagram of a vanishing point calculation method. In the present embodiment, the vanishing point is obtained from the optical flow 93 of the surrounding stationary object 92 generated by the traveling of the host vehicle. In general, when the host vehicle travels forward, an optical flow 93 of a surrounding stationary object 92 is generated so as to spring out from an image vanishing point (hereinafter referred to as “FOE”) 90. Therefore, the FOE 90 can be calculated as a spring point of the plurality of optical flows 93.

  However, since the FOE changes depending on the traveling direction of the own vehicle, it is necessary to calculate the FOE for each image, which increases the calculation cost. Therefore, FOE coordinates are estimated by the following method. The FOE coordinates (foe_x (t + Δt), foe_y (t + Δt)) at time t + Δt can be expressed by Equation 11 using the FOE coordinates (foe_x (t), foe_y (t)) at time t.

数式１１のΔｘおよびΔｙは、時間Δtの経過による自車進行方向の変化量であり、自車速度および操舵角から求めることができる。
図６は、ＦＯＥ座標の変化量テーブルの説明図である。このテーブルには、横軸に自車速度をとり、縦軸に操舵角をとって、予め計算や実測によって求めた自車進行方向の変化量が記載されている。

Δx and Δy in Expression 11 are changes in the traveling direction of the host vehicle with the passage of time Δt, and can be obtained from the host vehicle speed and the steering angle.
FIG. 6 is an explanatory diagram of a FOE coordinate change amount table. In this table, the amount of change in the traveling direction of the vehicle, which is obtained in advance by calculation or actual measurement, with the vehicle speed on the horizontal axis and the steering angle on the vertical axis, is described.

  Thus, the subsequent FOE coordinates can be easily estimated by repeatedly adding Δx and Δy obtained in FIG. 6 to the FOE coordinates (foe_x (t), foe_y (t)) at time t. Accordingly, it is not necessary to calculate the FOE as a source of a plurality of optical flows for each image, and the calculation cost can be reduced.

(Caution area setting means)
The attention area setting means shown in FIG. 1 sets an attention area in which an obstacle that may collide with the host vehicle is to be detected.
FIG. 7 is an explanatory diagram of the attention area. In the present embodiment, there are three types of obstacles to be detected: a jump-out vehicle, an oncoming vehicle, and an overtaking vehicle. Among them, the jump-out vehicle and the oncoming vehicle appear in the vicinity of the traveling direction of the host vehicle. Therefore, a rectangular attention area 1 is set around the FOE. In addition, the overtaking vehicle appears on both sides of the own vehicle. Therefore, a square attention area 2L is set at the lower left corner of the image, and a square attention area 2R is set at the lower right edge.

  The attention area 1 is set to a quadrangle centered on the FOE, and the initial value of the size is set to the width W10 and the height H10. In the present embodiment, the width W10 = 300 pixels and the height H10 = 200 pixels are set for an image of 640 × 480 pixels.

  FIG. 8 is an explanatory diagram of the size of the obstacle in the image. When the jumping vehicle 30 is detected in the caution area 1, it is necessary to detect the same jumping vehicle 30 from the subsequent image for tracking monitoring. However, the size of the flying vehicle 30 in the image increases as the host vehicle advances. In this case, there is a possibility that the flying vehicle 30 cannot be detected if the size of the attention area 1 remains the initial value.

  Therefore, the size of the attention area 1 is enlarged by the following method. Of the movement vectors of the flying vehicle 30 detected in the attention area 1, attention is paid to the movement vector whose start point is farthest from the FOE in the X direction (start point 34s of the movement vector 34). In the time series image, the position of the start point 34s gradually moves outward in the X direction. Therefore, if the size of the attention area 1 is set so that the start point 34s is included, the flying vehicle 30 can always be detected.

  FIG. 9 is an explanatory diagram of the operation for enlarging the attention area. If the X coordinate of the start point 34s of the movement vector 34 is xs1 and the X coordinate of the FOE is xfoe, the width (provisional value) W1 ′ of the new attention area 1 is expressed by Equation 12.

ただし、新たな注意領域１の幅Ｗ１が初期値Ｗ１０より小さくなるのは適当でない。そこで、数式１３の規則に従って、新たな注意領域１の幅Ｗ１を決定する。

However, it is not appropriate that the width W1 of the new attention area 1 is smaller than the initial value W10. Therefore, a new width W1 of the attention area 1 is determined according to the rule of Expression 13.

  Similarly, attention is paid to the movement vector of the flying vehicle 30 detected in the attention area 1 whose start point is farthest from the FOE in the Y direction (start point 35s of the movement vector 35). If the Y coordinate of the starting point 35s is ys1 and the Y coordinate of the FOE is yfoe, the height (provisional value) H1 ′ of the new attention area 1 is expressed by Equation 14.

ただし、新たな注意領域１の高さＨ１が初期値Ｈ１０より小さくならないように、数式１５の規則に従って新たな注意領域１の高さＨ１を決定する。

However, the height H1 of the new attention area 1 is determined according to the rule of Equation 15 so that the height H1 of the new attention area 1 does not become smaller than the initial value H10.

  Returning to FIG. 7, the attention area 2L and the attention area 2R are set to squares at the lower left corner and the lower right corner of the image, and the initial values of the sizes are set to the width W20 and the height H20. Note that the initial value of the size of the attention area 2L and the initial value of the size of the attention area 2R may be set to different values.

  FIG. 10 is an explanatory diagram of the enlargement operation of the attention area 2R. When the overtaking vehicle 31 is detected in the caution area 2R, it is necessary to detect the same overtaking vehicle 31 from the subsequent images for tracking monitoring. However, the passing vehicle 31 approaches the traveling direction of the host vehicle with the passage of time. In this case, there is a possibility that the overtaking vehicle 31 cannot be detected if the size of the attention area 2R remains the initial value.

  Therefore, the size of the attention area 2R is enlarged by the following method. Of the movement vectors of the overtaking vehicle 31 detected in the attention area 2, attention is paid to the movement vector 36 closest to the FOE. In the time series image, the position of the end point 36e of the movement vector 36 gradually approaches the FOE. Thus, if the size of the attention area 2R is set so that the end point 36e is included, the overtaking vehicle 31 can always be detected.

  Specifically, if the coordinates of the end point 36e of the movement vector 36 are (xe1, ye1) and the X coordinate of the right end of the attention area 2R is xr, the width (provisional value) W2R ′ of the new attention area 2R is expressed by the mathematical formula. 16.

ただし、注意領域２Ｒの幅Ｗ２Ｒが初期値Ｗ２０より小さくなるのは適当でない。そこで、数式１７の規則に従って、新たな注意領域２Ｒの幅Ｗ２Ｒを決定する。

However, it is not appropriate that the width W2R of the attention area 2R is smaller than the initial value W20. Therefore, the width W2R of the new attention area 2R is determined according to the rule of Expression 17.

  Similarly, the height (provisional value) H2R ′ of the new attention area 2R is expressed by Expression 18.

ただし、注意領域２Ｒの高さＨ２Ｒが初期値Ｈ２０より小さくなるのは適当でない。そこで、数式１９の規則に従って、新たな注意領域２Ｒの高さＨ２Ｒを決定する。

However, it is not appropriate that the height H2R of the attention area 2R is smaller than the initial value H20. Therefore, the height H2R of the new attention area 2R is determined according to the rule of Expression 19.

なお注意領域２Ｌの大きさについても、上述した注意領域２Ｒと同様に設定する。

The size of the attention area 2L is set in the same manner as the attention area 2R described above.

(Obstacle determination means)
The obstacle determination means shown in FIG. 1 determines an obstacle that may come into contact with the host vehicle from the attention area. In the present embodiment, it is determined whether or not the movement vector in the attention area satisfies the discrimination condition for obstacles (flying-out vehicle, oncoming vehicle, and overtaking vehicle) defined by the positional relationship with the FOE. When the determination condition is satisfied, it is determined that an obstacle exists in the image area occupied by the movement vector. For this purpose, first, the movement vectors of the vehicle candidates are extracted, and then various vehicles are discriminated against them.

  First, a movement vector as an obstacle candidate is extracted from the attention area. Specifically, focusing on the positional relationship between the start point (xs, ys) and end point (xe, ye) of each movement vector and FOE (xfoe, yfoe), a movement vector that satisfies the following conditions is used as an obstacle candidate. Extract.

  A candidate for the oncoming vehicle is a movement vector that moves away from the FOE in the attention area 1. Specifically, a movement vector that satisfies Equation 20 is a candidate for an oncoming vehicle.

  A candidate for the overtaking vehicle is a movement vector toward the FOE in the attention area 2L or the attention area 2R. Specifically, a movement vector that satisfies Formula 21 is a candidate for an oncoming vehicle.

  A candidate for a jump-out vehicle is a movement vector from the side toward the FOE in the attention area 1. Specifically, a movement vector that satisfies Formula 22 is a candidate for a flying vehicle.

Next, for a plurality of movement vectors that are candidates for an obstacle, an area in the image that satisfies the following three conditions is determined as an obstacle existing area.
Obstacle movement vectors should be dense in the local region. Therefore, as condition 1, whether or not the average distance dist_ave between the start points of two adjacent movement vectors among the plurality of movement vectors is equal to or smaller than the threshold dist_ave_Threshold is determined by Expression 23.

本実施形態では、dist_ave_Threshold＝２０ピクセルとする。

In the present embodiment, dist_ave_Threshold = 20 pixels.

  Also, as condition 2, in the area where condition 1 is satisfied, whether the total number n_FlowNum of movement vectors is equal to or greater than n_FlowNum_Threshold is determined by Expression 24.

本実施形態では、n_FlowNum_Threshold＝５とする。

In this embodiment, n_FlowNum_Threshold = 5.

  Moreover, the movement vector of an obstacle has the property that direction and magnitude | size are similar (a correlation is large). Therefore, as condition 3, in the region where condition 1 is satisfied, it is determined by Expression 25 whether the average correlation value n_Correlation between all the movement vectors is equal to or greater than the threshold value n_Correlation_Threshold.

本実施形態では、n_Correlation_Threshold＝０．８とする。
以上により、画像から障害物が検出されるようになっている。

In this embodiment, n_Correlation_Threshold = 0.8.
Thus, an obstacle is detected from the image.

(Notification means)
The notification means 24 shown in FIG. 1 notifies the presence of an obstacle to the driver of the host vehicle. As a specific notification means 24, a display device or the like is disposed in the vicinity of the instrument panel in the passenger compartment.
The notification unit 24 receives an image taken by the image acquisition unit 12 and information on the obstacle detected by the obstacle determination unit 22. The notification unit 24 displays the image taken by the image acquisition unit 12, and when an obstacle exists in the image, displays a polygonal frame or the like along the outer periphery thereof. For example, a rectangular frame having two sides with the width and height of the obstacle is displayed. At the same time as the frame, an arrow corresponding to the movement vector of the obstacle may be displayed. Moreover, you may change a alerting | reporting form according to the danger level of an obstruction. Furthermore, the display form may be selected according to the driving skill of the driver.

(Object detection method)
Next, an object detection method using the object detection apparatus will be described with reference to FIGS. 11 to 14 in addition to FIG.
11 to 14 are flowcharts of the object detection method according to this embodiment. First, the image acquisition means 12 acquires an image at the current time T around the host vehicle (S10), records it in the memory, and outputs it to the movement vector extraction means.

Next, the movement vector extraction means 16 extracts the movement vector of the object existing in the image. Specifically, first, an image before Δt time is read from the memory (S12), and an image before nΔt time is also read from the memory (S14). Then, an optical flow is calculated from these images, and a movement vector is extracted (S50).
FIG. 12 is a flowchart of the movement vector extraction subroutine. First, an optical flow is calculated from an image at the current time and an image before Δt time (S52). Next, the optical flow of the low-speed vehicle is calculated from the image at the current time and the image before the time nΔt (S54). The movement vector may be extracted by calculating the average value of the optical flows in S52 and S54. Further, the optical flow of either S52 or S54 may be extracted as a movement vector as it is. If the optical flow is successfully calculated in S52, S54 can be omitted. Finally, the movement vector noise (noise component) is removed (S56).

Next, the vanishing point calculating means 18 estimates a vanishing point representing the traveling direction of the host vehicle (S60).
FIG. 13 is a flowchart of the own vehicle traveling direction estimation subroutine. First, the traveling speed of the host vehicle is acquired (S62), and the steering angle is acquired (S64). Next, the acquired own vehicle speed and steering angle are applied to the FOE coordinate change amount table shown in FIG. The current FOE position is estimated by adding the obtained change amount of the FOE coordinate to the previously estimated FOE coordinate (S66).

Next, the attention area setting unit 14 sets an attention area in the image (S70).
FIG. 14 is a flowchart of the attention area setting subroutine. Here, the size of the attention area for which the initial value has already been set is reset. First, out of the movement vectors in the attention area 1, the one whose start point is farthest from the FOE is extracted in the X direction and the Y direction (S 72). Then, it is determined whether or not the attention area 1 needs to be enlarged (S73). Specifically, the starting point coordinates of the movement vector extracted in S72 are substituted into Equations 12 and 14, and the width W1 ′ and the height H1 ′ of the temporary attention area 1 are calculated. Next, it is determined by Equations 13 and 15 whether the provisional value is larger than the initial value. If the determination is YES, the process proceeds to S74, and the width W1 and the height H1 of the attention area 1 are enlarged and set to the provisional values. If the determination is NO, the width W1 and height H1 of the attention area 1 are maintained at the initial values, and the process proceeds to S76.

  Next, in S76, out of the movement vectors in the attention area 2R, the one whose end point is closest to the FOE is extracted (S76). Then, it is determined whether or not the attention area 2R needs to be enlarged (S77). Specifically, the end point coordinates of the movement vector extracted in S76 are substituted into Expression 16, and the width W2R ′ and the height H2R ′ of the provisional attention area 2R are calculated. Next, it is determined by Equation 17 whether the provisional value is larger than the initial value. If the determination is YES, the process proceeds to S78, and the width W2R and the height H2R of the attention area 2R are enlarged and set to the provisional values. If the determination is NO, the width W2R and the height H2R of the attention area 2R are maintained at the initial values, and the process proceeds to S80.

  Next, in S80, the movement vector in the attention area 2L is extracted that has the end point closest to the FOE (S80). Then, it is determined whether or not the attention area 2L needs to be enlarged (S81). More specifically, the width W2L ′ and the height H2L ′ of the provisional attention area 2L are calculated using the end point coordinates of the movement vector extracted in S80. Next, it is determined whether the provisional value is larger than the initial value. If the determination is YES, the process proceeds to S82, and the width W2L and the height H2L of the attention area 2L are enlarged and set to the provisional values. When the determination is NO, the width W2L and the height H2L of the attention area 2L are maintained at the initial values.

  Next, the obstacle determination means 22 determines an obstacle that may come into contact with the host vehicle from the movement vector in the set attention area. As shown in FIG. 11, first, a jump-out vehicle candidate is extracted (S20). Specifically, it is determined whether or not there is a movement vector toward FOE (satisfying Formula 22) in attention area 1. If the determination is YES, the process proceeds to S21 to detect a flying vehicle. Specifically, a plurality of movement vector existing areas that satisfy Expressions 23, 24, and 25 are determined as flying vehicles. If the determination is NO, it is determined that there is no flying vehicle, and the process proceeds to S23.

  Next, an oncoming vehicle candidate is extracted in S23 (S23). Specifically, it is determined whether or not there is a movement vector that leaves the FOE (satisfies Equation 20) in the attention area 1. When judgment is YES, it progresses to S24 and an oncoming vehicle is detected. Specifically, a plurality of movement vector existing areas that satisfy Expressions 23, 24, and 25 are determined as oncoming vehicles. If the determination is NO, it is determined that there is no oncoming vehicle, and the process proceeds to S26.

  Next, in S26, an overtaking vehicle candidate is extracted (S26). Specifically, it is determined whether or not there is a movement vector toward FOE (satisfying Expression 21) in attention area 2L or attention area 2R. If the determination is YES, the process proceeds to S27 to detect the overtaking vehicle. Specifically, the presence areas of a plurality of movement vectors that satisfy Expressions 23, 24, and 25 are determined as overtaking vehicles. If the determination is NO, it is determined that there is no overtaking vehicle, and the process ends.

When at least one of the jumping-out vehicle, the oncoming vehicle, and the overtaking vehicle is detected, the notification unit 24 notifies the driver of the host vehicle of the presence of the obstacle.
Thus, an object that may collide with the host vehicle is detected.

  As described in detail above, the object detection apparatus 10 according to the present embodiment includes the image acquisition unit 12 that acquires a time-series image around the host vehicle, and the movement of an object that exists in the image based on the acquired image. A movement vector extracting means 16 for extracting a vector, a vanishing point calculating means 18 for calculating an FOE representing the traveling direction of the host vehicle, and an attention area setting means 14 for setting an attention area in the image based on the calculated FOE. Obstacle determining means 22 for determining obstacles that may come into contact with the host vehicle within the set attention area. The attention area setting means 14 sets a plurality of attention areas, and the obstacle determination means 22 Is configured to determine the obstacle based on the direction of the movement vector with respect to the FOE and in which attention area the movement vector exists.

  According to this configuration, since the attention area is set in the image based on the FOE without using the lane mark or the like, it is possible to always set an effective attention area. Further, since the obstacle is determined based on the direction of the movement vector with respect to the FOE and in which caution area the movement vector exists, various obstacles that may collide with the host vehicle are accurately detected. Can do. In addition, since the obstacle is determined within the attention area, it is possible to efficiently determine the important obstacle, and the calculation cost can be reduced.

  The attention area setting means 14 sets the attention area 1 centering on the FOE in the image, and the obstacle determination means 22 jumps the object when the movement vector exists in the attention area 1 and goes to the FOE. It was set as the structure determined to be a taking-out vehicle. The flying vehicle appears in the vicinity of the FOE, and its movement vector is directed in a direction approaching the FOE. Therefore, the flying vehicle can be detected with high accuracy by the above configuration.

  Further, the attention area setting unit 14 sets the attention area 1 around the FOE in the image, and the obstacle determination unit 22 determines that the movement vector exists in the attention area 1 and moves away from the FOE. The object is determined to be an oncoming vehicle. Since the oncoming vehicle appears in the vicinity of the FOE, and its movement vector goes in a direction away from the FOE, the oncoming vehicle can be detected with high accuracy by the above configuration.

  The attention area setting unit 14 sets the attention area 2R and the attention area 2L at least one of the lower right end and the lower left end in the image, and the obstacle determination unit 22 sets the movement vector within the attention area 2R or the attention area 2L. When the vehicle exists and heads for FOE, the object is determined to be an overtaking vehicle. The overtaking vehicle appears at the lower right end or the lower left end in the image, and its movement vector is directed in a direction approaching the FOE. Therefore, the overtaking vehicle can be detected with high accuracy by the above configuration.

The present invention is not limited to the embodiment described above.
For example, in the above-described embodiment, a jump-out vehicle, an oncoming vehicle, and an overtaking vehicle are detected, but it is also possible to detect a vehicle other than the above that may collide with the own vehicle. Moreover, although the vehicle was detected in the said embodiment, it is also possible to detect moving objects, such as a motorcycle, a bicycle, and a pedestrian. Moreover, in the said embodiment, although the display apparatus was employ | adopted as an alerting | reporting means, other alarm sound output apparatuses etc. are also employable.

1 is a block diagram of an object detection device according to an embodiment. It is explanatory drawing of a movement vector. It is explanatory drawing of the alternative calculation method of a movement vector. It is explanatory drawing of the noise removal of a movement vector. It is explanatory drawing of a vanishing point calculation method. It is explanatory drawing of the variation | change_quantity table of FOE coordinate. It is explanatory drawing of a caution area | region. It is explanatory drawing of the magnitude | size of the obstruction in an image. 10 is an explanatory diagram of an enlargement operation of the attention area 1. FIG. It is explanatory drawing of expansion operation of attention area | region 2R. It is a flowchart of the object detection method which concerns on embodiment. It is a flowchart of a movement vector extraction subroutine. It is a flowchart of the own vehicle traveling direction estimation subroutine. It is a flowchart of an attention area setting subroutine.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Attention area (1st attention area) 2L, 2R ... Attention area (2nd attention area) 10 ... Object detection apparatus 12 ... Image acquisition means 14 ... Attention area setting means 16 ... Movement vector extraction means 18 ... Vanishing point calculation means 22 ... Obstacle determination means 24 ... Notification means

Claims (7)

Image acquisition means for acquiring time-series images around the host vehicle;
A movement vector extracting means for extracting a movement vector of an object existing in the image based on the image;
Vanishing point calculating means for calculating a vanishing point representing the traveling direction of the host vehicle;
Attention area setting means for setting an attention area in the image based on the vanishing point;
In an object detection device comprising obstacle determination means for determining an obstacle that may contact the host vehicle within the attention area,
The attention area setting means sets a plurality of attention areas,
The obstacle detection means determines the obstacle based on a direction of the movement vector with respect to the vanishing point and in which attention area the movement vector exists. The attention area setting means sets the first attention area around the vanishing point in the image,
2. The obstacle determination unit according to claim 1, wherein the object is determined to be the obstacle when the movement vector is present in the first attention area and heads toward the vanishing point. The object detection apparatus described in 1. The attention area setting means sets the first attention area around the vanishing point in the image,
The obstacle determination means determines that the object is the obstacle when the movement vector exists in the first attention area and moves in a direction away from the vanishing point. The object detection apparatus according to claim 1 or 2. The attention area setting means sets the second attention area in at least one of the lower right end and the lower left end in the image,
2. The obstacle determination unit according to claim 1, wherein the object is determined to be the obstacle when the movement vector is present in the second attention area and heads toward the vanishing point. The object detection apparatus according to claim 3.   5. The object detection device according to claim 1, wherein the attention area setting unit sets a size of the attention area based on a position and a direction of the movement vector.   The noise removal means for removing the movement vector having a correlation value with a plurality of the movement vectors less than a predetermined value among the movement vectors of the object is provided. The object detection device according to claim 1.   The object detection according to any one of claims 1 to 6, wherein the movement vector extraction unit adjusts an acquisition interval of the image when extracting the movement vector based on a speed of the object. apparatus.

Images