JP2010073036A - Vehicle surroundings monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle surroundings monitoring device for simplifying calculation of optical flow in images on the right or left sides captured when a vehicle turns to the right or left, and facilitating image processing for recognition of a moving object. <P>SOLUTION: The vehicle surroundings monitoring device includes: a vehicle periphery image generating means 2 for generating a vehicle periphery image Gf which is a bird's eye view seen from a virtual view point Bp above a vehicle based on a side image Gs which is captured by a plurality of imaging units 1 which are arranged in different places from each other in the vehicle C; a feature point extracting means 3 for extracting a feature point P in the vehicle periphery image and calculating the motion vector of the feature point; a yaw rate calculating means 4 for calculating a yaw rate ω for the vehicle from the vector of movement for the feature point; a monitor 5 which displays the side image on a screen; and a facing object recognition means 6 for recognizing a facing object in the monitor image based on a correction feature point Q' which is obtained by reducing the yaw rate from the flow in the feature point Q that is extracted from the side image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle periphery monitoring device that displays a nose view image obtained by imaging the side of a vehicle as a side image on a monitor screen, and more particularly to a vehicle periphery monitoring device that can display only when necessary.

  When a vehicle enters an intersection while traveling on a narrow road, the left and right side areas, which are the direction of the intersection road, often have poor visibility from the driver's seat. Therefore, when the vehicle enters the intersection, for safety confirmation, a camera (nose view camera) built in the front of the vehicle is used to capture a scene in the left and right side area at the front position of the vehicle, and the left and right side images are captured. 2. Description of the Related Art A vehicle periphery monitoring device that displays in parallel on a monitor provided in a driver's seat of a vehicle is known.

For example, the nose view monitor device disclosed in Patent Document 1 displays the left and right side images on a monitor provided in the driver's seat of the vehicle, and calculates the optical flow for the feature points extracted from the side images. . Here, there is a technology for selecting only those having a vector in the approach direction from the feature points in the left and right side images as approach feature points, thereby detecting an approaching object to the vehicle and notifying the occupant of the approaching object information. Disclosed.
Or the camera apparatus for vehicles which measures the distance to a moving object combining a distance sensor and a camera is also known (for example, refer patent document 2).

  Further, in Patent Document 3, a video around the host vehicle is shot by a shooting unit, an image overlooking the host vehicle is generated based on the shot video, and driving operation assistance information is generated on the generated image. As an example, there is disclosed a driving support device that displays an inverted L-shaped guide for preventing contact with a front obstacle of the host vehicle and a planned parking position guide indicating a planned parking position of the host vehicle superimposed on a display unit. With this device, the initial position for parking operation is determined through actual driving operation, and contact confirmation between the vehicle and obstacles parked at the left front of the host vehicle and the host vehicle is performed in advance of the parking operation. Techniques for doing so are disclosed.

  Furthermore, Patent Document 3 discloses that when a captured image of a rear view camera mounted on a vehicle is converted into an image looking down from a virtual upper viewpoint position, and when this viewpoint converted image is displayed on the screen, the presence of the rear part of the vehicle A captured image display device of an in-vehicle camera that displays a warning line that warns the range over the converted image is disclosed. With this device, a technique is disclosed in which an image obtained by converting the viewpoint of a captured image of a rear view camera is displayed on a screen to warn the driver of the existence range of the host vehicle.

  Further, in Patent Document 4, a video around the host vehicle is shot by a shooting unit, an image overlooking the host vehicle is generated based on the shot video, and driving operation assistance information is generated on the generated image. As an example, there is disclosed a driving support device that displays an inverted L-shaped guide for preventing contact with a front obstacle of the host vehicle and a planned parking position guide indicating a planned parking position of the host vehicle superimposed on a display unit. With this device, the initial position for the parking operation can be determined through the actual driving operation. In addition, a technique is disclosed in which contact confirmation between a vehicle or an obstacle stopped at the front left of the host vehicle and the host vehicle is performed in advance of a parking operation.

JP 2005-276056 A JP 2001-39248 A JP 2002-316602 A JP 2006-027334 A

  As described above, in the conventional vehicle periphery monitoring device, as proposed in Patent Documents 2 to 4 and the like, the side image is displayed on the monitor provided in the driver's seat of the vehicle, and the driver can improve the driving safety. Therefore, image processing is performed so that a feature point is extracted from the side image and a moving object can be recognized more easily according to the optical flow of the feature point.

  By the way, a camera (nose view camera) that is installed at the front position of the vehicle and captures a scene in the side area is used in the left and right side images Pl and Pr as shown in FIG. The speed vector of the feature point a toward the side of the oncoming vehicle becomes a value corresponding to the vehicle speed. However, when the vehicle turns left and right, as shown in FIG. 13B, the left and right side images Pl 'and Pr' are affected by different velocity displacements. For this reason, the calculation of the optical flow of the feature point a ′ of the side images Pl ′ and Pr ′ is complicated. That is, when the host vehicle is rotating at yaw ωn, the left and right side images Pl ′ and Pr ′ overlap with the movement of the host vehicle's yaw rotation ωn, making the image processing logic difficult. In this case, since it cannot be solved only by the motion on the image, complicated processing is required, which leads to an increase in calculation cost, making it difficult to mount on the ECU, and improvement is desired.

Note that the speed displacement of the left and right side images Pl ′ and Pr ′ due to such a left and right turn of the vehicle complicates the calculation of the optical flow of the feature point a ′ of the side images Pl ′ and Pr ′. Each of the techniques 1 to 4 does not disclose a solution.
The present invention has been made based on the problems as described above, and the object is to simplify the calculation of optical flow in the left and right side images by turning the vehicle left and right and to facilitate the image processing for moving object recognition. It is in providing the vehicle periphery monitoring apparatus which can plan.

  The invention according to claim 1 of the present application is a bird's-eye view image viewed from a virtual viewpoint above the host vehicle, based on side images taken by a plurality of imaging devices installed at different locations of the host vehicle. Vehicle surrounding image generating means for generating a vehicle surrounding image; feature point extracting means for extracting a feature point from the vehicle surrounding image and calculating a motion vector of the feature point; and the host vehicle from the motion vector of the feature point Obtained by subtracting the yaw rate from the flow of the side feature points extracted from the side image, the image display control means for displaying the side image on the monitor, and the yaw rate calculation means for calculating the yaw rate And opposed object recognition means for performing recognition processing on the opposed object in the image of the monitor based on the corrected feature point.

  The invention according to claim 2 of the present application is the vehicle periphery monitoring device according to claim 1, wherein a nose view camera attached to a front portion of the vehicle among the imaging devices to capture the field of view in the vehicle width direction as a side image. The opposing object recognition means detects a side feature point in the side image and calculates a motion vector of the detected side feature point; and a side image feature point extraction means for calculating the motion vector of the detected side feature point; Correction feature point creation means for creating a correction feature point by subtracting the yaw rate from the flow of the side feature point, and a direction approaching the host vehicle in the side image from the motion vector of the correction feature point Moving object detecting means for detecting an object having the vector of the approaching moving object, and object image area setting means for setting an image area in the side image of the approaching moving object, and the image display control. It means for highlighting an image area of the approaching moving object in the side image, and wherein the.

  The invention according to claim 3 of the present application is the vehicle periphery monitoring device according to claim 1, wherein a nose view camera attached to a front portion of the vehicle among the imaging devices to capture the field of view in the vehicle width direction as a side image. The opposing object recognizing unit detects a feature point in the side image and calculates a motion vector of the detected feature point; and a flow of the feature point in the side image. Correction feature point creating means for creating a correction feature point by further reducing the yaw rate, and an object having a vector in a direction approaching the host vehicle in the side image from a motion vector of the correction feature point Moving object detection means for detecting as an approaching moving object, feature point tracking means for tracking the corrected feature point of the detected approaching moving object, and the approaching moving object from the tracking correction feature point It is characterized by comprising a passage time calculating means for calculating a time through the front of the both.

  According to the first aspect of the present invention, a vehicle surrounding image that is a bird's-eye view image is generated based on an image around the host vehicle, and a yaw rate of the vehicle is calculated from a motion vector of a feature point that can be located in the vehicle surrounding image. On the basis of the corrected feature point obtained by subtracting the yaw rate from the feature point flow calculated in the side image, recognition processing for the opposite object in the monitor image is performed. Because it is possible to perform recognition processing for opposing objects with correction feature points that have cleared the influence of yaw rotation, the in-vehicle image processing logic is relatively simple, the calculation cost is reduced, and it is easy to install in the ECU. Driving can be facilitated.

  According to the second aspect of the present invention, the yaw rate of the vehicle is calculated from the vehicle periphery image, and the yaw rate of the host vehicle can be detected even if there is a deviation in the flow of the side image due to the left or right turn of the host vehicle at an intersection or the like. The optical flow of the corrected feature point is calculated using the corrected feature point that has cleared the influence of rotation. For this reason, the logic of the in-vehicle image processing becomes simple, the optical flow of the side feature points in the corrected side image can be easily calculated even when turning, and a set of feature points having the same motion vector is regarded as one approaching moving object. Recognition and detection of an approaching moving object can be performed easily and reliably, the calculation cost can be reduced even when turning, and the mounting on the ECU is facilitated.

  According to the third aspect of the present invention, the yaw rate of the vehicle is calculated from the vehicle periphery image, and the yaw rate of the host vehicle can be detected even if there is a deviation in the flow of the side image due to the left or right turn of the host vehicle at an intersection or the like. Using the corrected feature point that cleared the influence of rotation, the optical flow of the corrected feature point is calculated to detect the approaching moving object, and the approaching moving object passes the front of the vehicle from the corrected feature point obtained by this calculation. Since the estimated passing time is calculated, it is possible to reliably predict the time when the moving object passes the front of the vehicle, the calculation cost is reduced even when turning, and the mounting on the ECU is facilitated.

Hereinafter, a vehicle periphery monitoring device according to an embodiment of the present invention will be described.
1 and 2 show a schematic configuration diagram and a functional block diagram of a main part of a vehicle periphery monitoring device according to this embodiment.
This vehicle periphery monitoring device is installed at different locations on the vehicle C, and is provided with a plurality of imaging devices 1n, 1s, 1r (hereinafter collectively referred to as side images Gs (Gs1, Gs2, etc.) around the vehicle). Is a bird's-eye view image viewed from a virtual viewpoint Bp (not shown) above the host vehicle C based on the side image Gs around the vehicle (see FIGS. 6A and 6B). b) reference) to generate a vehicle surrounding image generation means 2 and a feature for calculating a motion vector of feature points P (P1, P2, P3,...: see FIGS. 6A and 6B) in the vehicle surrounding image. The point extracting means 3, the yaw rate calculating means 4 for calculating the yaw rate ω of the vehicle C, the monitor 5 capable of displaying the side images Gs (Gs1, Gs2) on the screen, and the opposing object recognition process in the side images are performed. And an opposing object recognition means 6.

Here, an image control device ECU is mounted as a control means on the vehicle C, and the image control device ECU is constituted by an electronic control unit mainly including a microcomputer. Note that a vehicle speed sensor 11 for inputting a vehicle speed Vc, an inhibitor switch 12 for inputting gear speed information Rv, D, a main switch 21, and a monitor 5 are connected to the image control device ECU. In addition, the image control device ECU basically has the function of the image display control means (monitor control means) 7 and is connected so that the navigation device 30 and the monitor 5 can be used together (see FIG. 2).
As shown in FIG. 2, the image control apparatus ECU includes the vehicle surrounding image generation means 2, the feature point extraction means 3, the yaw rate calculation means 4, the image display control means (monitor control means) 7 of the monitor 5, and the opposed object recognition. Each control function with the means 6 is provided, and the algorithm shown in FIG. 3 performs control for processing for the vehicle peripheral image Gf that is a bird's-eye view video and processing for the left and right front side images Gs1 and Gs2 from the nose view camera 1n. .

As shown in FIGS. 1 and 2, a plurality of image pickup apparatuses 1 are installed at different locations on the vehicle C, mounted on the front end of the vehicle C in a protruding state, and mounted on the left and right vehicle body side outer walls. A left and right side camera 1s that captures left and right side images of the vehicle, and a rear view camera 1r that is attached to the vehicle body outer wall at the rear of the vehicle C and captures a rear image of the vehicle.
Here, the nose view camera 1n is a lateral camera, and has a function of capturing the left and right front side images Gs1 and Gs2 by imaging the left and right sides of the front part of the vehicle as shown in FIG. As shown in FIG. 4, the image display control means (monitor control means) 7 has a function of dividing the screen of the monitor 5 into left and right parts and displaying the left and right front side images Gs1 and Gs2 in parallel, respectively. When driving 30, the navigation image display is switched.

Next, processing for the overhead view image performed by the image control apparatus ECU will be described.
The vehicle periphery image generating means 2 of the image control device ECU is installed at different locations of the host vehicle C, and the left and right front side images Gs1, Gs2 from the nose view camera 1n, and the left and right side images from the left and right side cameras 1s Based on the rear image from the rear view camera 1r, the vehicle peripheral image Gf (see FIGS. 6 (a) and 6 (b)) is an overhead image viewed from a virtual viewpoint Bp (not shown) above the host vehicle. 5 shows an example of a vehicle periphery image displayed on the screen of No. 5).

The feature point extraction means 3 extracts feature points P (P1.P2.P3) shown in FIGS. 6A and 6B based on the vehicle periphery image Gf (see FIG. 4) which is an overhead image. Specifically, a feature point P whose position can be specified is extracted from the vehicle peripheral image Gf, for example, such that the image signal level (luminance) and hue are significantly different from the peripheral portion.
The image I, neighborhood region D (p), and pixel p used in this feature point extraction algorithm have the relationship shown in FIG.
First, in the image I, partial differential images ∂I / ∂x and に 対 す る I / ∂y with respect to the x and y directions are obtained. Next, a spatial gradient matrix G (p) with respect to the pixel p as shown in the following expression (a) in the image I is obtained for all pixels.

Next, eigenvalues λmin (p) and λmax (p) for the spatial gradient matrix G (p) are calculated, and a position-identifiable point having a predetermined value considered to have a feature is extracted. To do. As a method for selecting the feature point P, a method disclosed in Japanese Patent Application Laid-Open No. 2004-198211 may be used.
Next, a feature point tracking process 3-1 (see FIG. 2) is performed. Here, an optical flow is calculated for the extracted feature point P whose position can be specified, and the change in the position of the feature point over time is adjusted. The optical flow calculation uses a multi-resolution method.

  Here, the multi-resolution method is to hierarchically prepare multiple images with different resolutions for one image, perform pattern matching in order from the image with coarse resolution, find the optical flow, and finally This is a technique for obtaining an optical flow in an image with a fine resolution, as described in detail in Japanese Patent Application Laid-Open No. 2004-56763 and the above-mentioned Patent Document 1.

In other words, as shown in FIGS. 6A, 6 </ b> B, and 7, each vehicle surrounding image Gf (herein, Gfa, Gfb is a generic name) that is a continuous overhead image captured in a predetermined cycle. ) Between the common feature points P (P1, P2, P3,...) To detect whether the feature point P is moving. Further, when the coordinates are changed, the direction of the movement (coordinate change) and the degree of the magnitude are respectively calculated. That is, it is done by calculating the moving direction and moving distance of the feature point P as a flow vector (see FIG. 7). In addition, a flow vector is calculated in all areas in the captured image, and information such as the position and moving direction of the moving object in the image can be recognized.
For example, as shown in FIGS. 6A and 6B, when the vehicle C heads for the parking zone E, three straight lines surrounding the parking zone E from the vehicle peripheral image Gf, that is, the front Threshold line L0 and left and right parking partition lines L1, L2 and third parking partition line L3 are detected. Based on this, three points where the front threshold line L0 intersects with the parking lot lines L1, L2 and the third parking lot line L3 are evaluated as feature points P1, P2, P3 (using three representative points for explanation). And will be determined. Note that the feature points P1, P2, and P3 are an example of these three points and are extracted from the vehicle peripheral image Gf.

Next, the yaw rate calculating means 4 for calculating the yaw rate ω of the vehicle C will be described. Here, the yaw rate of the host vehicle is calculated from the optical flow that is the motion vector of the feature point.
For example, the yaw rate ω of the vehicle C is calculated using the feature points P1, P2, and P3 in the vehicle peripheral image Gf (Gfa, Gfb) shown in FIGS. 6 (a) and 6 (b).
In this case, as shown in FIG. 7, the motion [ub, vb] at the pixel [x, y] in the vehicle peripheral image Gf of one feature point P1 (where the feature point number is i) is as follows. The yaw rate ω can be calculated from the optical flow according to the equation (b).

[Ub, vb] = [ωy, −ωx−Vc] (b)
In other words, from the result of feature point extraction and feature point tracking on the vehicle periphery image Gf (Gfa, Gfb), which is a bird's-eye view video, the feature point with the feature point number i appears on the vehicle periphery image Gf between time t and t + Δt. ,
x _i ^(b) (t) = [x _i ^(b) (t), y _i ^(b) (t)] ^T
X _i ^(b) (t + Δt) = [x _i ^(b) (t + Δt), y _i ^(b) (t + Δt)] ^T
(For example, refer to FIGS. 6A and 6B).

At this time, a speed vector on the vehicle peripheral image Gf which is an overhead video of the feature point P of the feature point number i:
v _i ^(b) (t) = [u _i ^(b) (t), v _i ^(b) (t)] ^T
Is calculated as follows:

  If the feature point i is on the road surface in the vehicle peripheral image Gf and the host vehicle is moving at the yaw rate ω and the speed V, the following relationship is established.

From equation (2), the yaw rate ω is calculated from the Y component y _i ^(b) (t) at the position of the feature point i on the vehicle peripheral image Gf2 and the X component u _i ^(b) (t) of the velocity vector as follows: Desired.

Here, ^ (ω) is the estimated yaw rate, R is a set of feature point numbers on the road surface on the vehicle peripheral image Gf2, and ‖ ● ‖ is the parameter of the set.
Note that the yaw rate ω may be estimated from the least square method as in the following equation.

Next, processing for the left and right side images Gs1 and Gs2 (see FIGS. 4 and 5), which are overhead images performed by the image control apparatus ECU, will be described.
The opposing object recognition means 6 of the image control device ECU recognizes the opposing object in the images GS1 and GS2 based on the front left and right side images GS1 and GS2 (see FIGS. 4 and 5) from the left and right nose view cameras 1n. Image processing is performed.
Here, as shown in FIG. 2, the opposed object recognition means 6 functions as a side image feature point extraction means 6a, a correction feature point creation means 6b, a moving object detection means 6c, and an object image region setting means 6d. To do.

The side image feature point extraction means 6a detects a plurality of side feature points Q1, Q2, Q3 in the left and right side images GS1, GS2 and calculates motion vectors of the detected feature points (calculation of optical flow vectors). .
Specifically, it is a velocity vector on the nose view video with respect to the feature point of the feature point number j at time t by tracking the side feature point Q1 (here, the feature point number is i) on the nose view video. The optical flow vector is calculated as 'v' _j ⁽ⁿ⁾ (t) = [u _j ⁽ⁿ⁾ (t), v _j ⁽ⁿ⁾ (t)] ^T.

In such a velocity vector 'v' _j ⁽ⁿ⁾ (t), when the host vehicle has the yaw rate ω, the nose view video reflected in the nose view camera is influenced by the yaw rate estimation result ^ (ω). Yes.
Therefore, the corrected feature point creation means 6b is a velocity vector 'v' _j ⁽ⁿ⁾ (which is an optical flow vector) at the feature point of the feature point number j at time t in the nose view video (left and right side images GS1, GS2). From t), the influence of the yaw rate ω can be cleared to zero as follows.

By using the velocity vector 'v' _j ⁽ⁿ⁾ (t) that clears the yaw rate ω component in the equation (5), for example, if the process disclosed in Japanese Patent Publication No. 1 is performed, it is easily included in the approaching object. It is possible to select only corrected feature points (referred to as approach feature points).
Further, the moving object detection means 6c is an optical flow vector having a vector component in the traveling direction of the vehicle in the left and right side images GS1, GS2 among the velocity vectors 'v' _j ⁽ⁿ⁾ (t) which are optical flow vectors. An approaching object is detected based on

Further, the object image region setting means 6d has an optical flow having a vector component in the traveling direction of the vehicle in the left and right side images GS1 and GS2 of the velocity vector 'v' _j ⁽ⁿ⁾ (t) which is an optical flow vector. An approaching object is detected based on the vector. That is, a group of a plurality of side feature points (Q1, Q2, Q3...) Including a feature point with feature point number j is detected as an approaching moving object approaching the host vehicle, and left and right of the approaching moving object are detected. An image region Ep in the side images GS1 and GS2 is set.

Specifically, as shown in FIG. 4, the optical flow vector by having a vector component in the right direction among the optical flow vectors in the left side image GS1 (region) and the optical flow vector in the right side image GS2 (region). Of these, the approaching object is detected based on the optical flow vector cy having the vector component in the left direction.
As shown in FIG. 4, the image display control means (monitor control means) 7 divides the screen of the monitor 5 into left and right parts, displays the left and right front side images Gs1 and Gs2 in parallel, and in particular displays an approaching moving object. The image area Ep is highlighted in the left and right side images GS1, GS2 on the screen of the monitor 5 (see FIG. 8).

  In this case, all of the image area Ep of the approaching moving object or the outline portion of the image area Ep is displayed in red, and the illuminance is increased / decreased at a constant period to facilitate the driver's recognition.

Next, a display control process performed by the image control apparatus ECU of the vehicle periphery monitoring apparatus in FIG. 1 will be described together with a flowchart of the image display control routine in FIG.
Here, basically, the image control device ECU also serves as the monitor 5 of the navigation device 30 and reaches step s1 when the main switch 21 is turned on.

In step s1, a navigation image from the navigation device 30 is displayed, and the process proceeds to step s2. Here, it is determined whether the vehicle speed Vc is equal to or less than the slow speed Vc1, for example, 10 Km / H.
On the other hand, when step s3 is reached at the slow speed Vc1 or less, it is determined here whether the current stage is the reverse stage Rv, and if yes, the process proceeds to step s4. Otherwise, the process proceeds to step s5 in forward slow speed.
When step s4 is reached in the reverse stage Rv, the vehicle peripheral image generating means 2 creates, that is, the left and right front and side images Gs1, Gs2 of the nose view camera 1n, the left and right side images of the left and right side cameras, and the rear view camera 1r. The vehicle surrounding image Gf (see FIGS. 6A and 6B), which is an overhead view image based on the rear image, is displayed on the screen of the monitor 5 instead of the navigation screen, and the process returns to step s1.

As a result, the driver can easily perform parking processing while avoiding interference with surrounding opposing objects using the vehicle peripheral image Gf that is a bird's-eye view image when the vehicle C is moving backward such as when the vehicle C is parked.
On the other hand, when step s5 is reached in the D range or the N range, it is necessary here for the vehicle approaching and slowly approaching the intersection to check the oncoming vehicle on the intersection road Rd1 side (see CO in FIGS. 4 and 5). As a result, the left and right front and side images Gs1 and Gs2 are displayed on the monitor.
Next, in step s6, the vehicle periphery image Gf data, which is an overhead image created by the vehicle periphery image generation means 2, is taken in, and the feature point extraction means 3 specifies the position based on the vehicle periphery image Gf (processing on the data). Possible feature points P are extracted. For example, the feature points P1h and P2h that can be identified by the intersection of the road width display lines in FIG. 1 are extracted.

  Further, in step s7, a feature point tracking process 301 (see FIG. 2) is performed, and an optical flow is calculated for the extracted feature points P (P1h, P2h). Next, in step s8, it functions as the yaw rate calculation means 4, and from the optical flow that is the motion vector of the feature point P (P1h, P2h), that is, the feature point P (P1h, P2h) is used to calculate the estimated yaw rate ^ (ω) of the vehicle C.

Next, when step s9 is reached, the side image feature point extraction means 6a detects a plurality of side feature points Q1, Q2, Q3,... In the left and right side images GS1, GS2 (see FIGS. 4 and 5). The motion vector of the detected side feature point is calculated (optical flow vector is calculated). That is, the optical flow vector is ^expressed as “v” _j ⁽ⁿ⁾ (t) = [u _j ⁽ⁿ⁾ (t), v _j ⁽ⁿ⁾ (t)] ^T
Calculate as

The velocity vector 'v' j (n) (t) is affected by the yaw rate estimation result ^ (ω).
Accordingly, when step s10 is reached, the correction feature point creation means 6b performs a process of zero-clearing the influence of the yaw rate ω using the above equation (5).
In such a (5), using the yaw rate Clear ω component velocity vector 'v' j (n) ( t), the moving object detection unit 6c is a optical flow vector velocity vector 'v' j (n ) Among (t), an object that approaches a group of a plurality of corrected feature points (Q1 ′, Q2 ′, Q3 ′...) Including the feature point with the feature point number j is set as an approaching moving object. To detect. Then, the object image area setting unit 6d sets an image area Ep (see FIG. 4A) in the left and right side images GS1 and GS2 of the approaching moving object.

  Next, when step s11 is reached, as a function of the image display control means 7, the image area Ep of the approaching moving object is highlighted in the left and right side images GS1, GS2 on the screen of the monitor 5 (see FIG. 8). In other words, the image area Ep of the approaching moving object is displayed in red, and the illuminance is switched between increasing and decreasing at regular intervals, the driver's recognition is facilitated, the driver is alerted to the oncoming vehicle on the oncoming road, and the process returns to step s1. In addition to the processing at step s11, a buzzer (not shown) may be driven to further facilitate the driver's recognition.

As described above, the vehicle periphery monitoring device in FIG. 1 generates the vehicle periphery image Gf that is an overhead image around the host vehicle C, and calculates the vehicle yaw rate ω from the motion vector of the feature point P in the vehicle periphery image Gf. Obtained by subtracting the yaw rate ω from the flow of the side feature point Q (optical flow vector 'v' _j ⁽ⁿ⁾ (t)) calculated in the side images GS1 and GS2 Based on the corrected feature point Q ′, recognition processing for the opposite object in the monitor image is performed, that is, corrected feature points (Q1 ′, Q2) that have cleared the influence of yaw rotation of the host vehicle from the position information of the side feature point Q. ', Q3'... For this reason, as shown by a two-dot chain line in FIG. 1, even when the vehicle enters the intersection in a turning state, the in-vehicle image processing logic becomes relatively simple, the calculation cost is reduced, and the mounting on the ECU is facilitated. In addition, safe driving can be facilitated.

The opposing object recognition means 6 of the vehicle periphery monitoring device in FIG. 1 sets an image area Ep in the side image of the approaching moving object, and highlights the image area Ep in the side images GS1 and GS2. Instead of this, a vehicle periphery monitoring device in which only the opposed object recognition means 6A as shown in FIG. 11 is replaced may be configured.
In this case, the opposed object recognizing means 6A is the same as the opposed object recognizing means 6 in FIG. 2, in addition to the side image feature point extracting means 6a, the corrected feature point creating means 6b, and the moving object detecting means 6c, and the feature point tracking. The function of the means 6e and the passage time calculation means 6f is provided.

The feature point tracking means 6e tracks the corrected feature points (Q1 ′, Q2 ′, Q3 ′...) Of the detected approaching moving object. At this time, as shown in FIG. 5, the correction that the approaching moving object is regarded as the direction passing through the front of the host vehicle C from the flow of the feature point P (optical flow vector 'v' _j ⁽ⁿ⁾ (t)). One of the feature points (Q1 ′, Q2 ′, Q3 ′...) Is selected and tracked. Furthermore, feature points for which no moving vector is detected are excluded from recognition processing targets as being fixed objects (backgrounds) such as buildings.

  Next, the passage time calculation means 6f obtains an expected passage time tc at which the approaching moving object passes the front of the host vehicle C from one of the correction feature points (Q1 ', Q2', Q3 '...) to be tracked.

As the control processing of the feature point tracking unit 6e and the passage time calculation unit 6f, those described in detail in JP-A-2006-253932 and the like can be used.
Note that the image display control means 6 highlights the image area Ep of the approaching moving object in the left and right side images GS1, GS2 on the screen of the monitor 5. In other words, the image area Ep of the approaching moving object is displayed in red, and the illuminance is switched between increasing and decreasing at regular intervals, facilitating the driver's recognition, and prompting the driver to pay attention to the oncoming vehicle on the oncoming road.

  FIG. 12 shows a flowchart of image display processing performed by the image control apparatus ECU of the opposed object recognition unit 6A. Here, steps s1 to s10 are the same processing as in FIG. 10, and redundant description is omitted. Here, step s10 is reached to step s11a. Here, the correction feature points (Q1 ', Q2', Q3 '...) of the approaching moving object are tracked. Next, in step s11b, an expected passing time tc of the approaching vehicle is obtained from the correction feature points (Q1 ', Q2', Q3 '...) to be tracked. Next, in step s11c, it is determined whether the expected passage time tc is earlier than the predetermined expected passage time Th. If it is late, the process directly returns to step s1, and if it is early, the image area Ep of the approaching moving object is displayed on the screen of the monitor 5 in step s11d. Is highlighted in the left and right side images GS1, GS2, and the process returns to step s1. That is, the image area Ep of the approaching moving object is displayed in red, and the driver can be easily recognized.

  Thus, even when approaching the intersection in a turning state, the predicted passing through which the approaching moving object passes the front of the vehicle accurately from the correction feature points (Q1 ′, Q2 ′, Q3 ′...) Of the approaching moving object is accurate. The time tc can be obtained with high accuracy. Furthermore, when the expected passage time tc is earlier than the predetermined passage assumption time Th, the image area Ep of the approaching moving object is highlighted, so that the driver can easily recognize the approaching moving object and can safely travel. The in-vehicle image processing logic becomes relatively simple, the calculation cost is reduced, and the mounting on the ECU becomes easy.

  The vehicle periphery monitoring device of the present invention has been described above, but the specific configuration is not limited to the embodiment, and unless it departs from the gist of the invention according to each claim of the claims, Design changes and additions are allowed.

1 is a schematic configuration diagram of a vehicle including a vehicle periphery monitoring device as one embodiment of the present invention. It is a block diagram showing the control function part of the vehicle periphery monitoring apparatus of FIG. It is a figure explaining the algorithm in control of the periphery monitoring apparatus for vehicles of FIG. It is the schematic explaining the feature point of the side image which the monitor of the vehicle periphery monitoring apparatus of FIG. 1 displays. It is explanatory drawing of the velocity vector used by the approach time calculation of an approaching vehicle in the side image which the monitor of the vehicle periphery monitoring apparatus of FIG. 1 displays. FIG. 1 is a bird's-eye view image when the vehicle is parked, which is displayed by the monitor of the vehicle periphery monitoring device in FIG. 1. FIG. Show. It is explanatory drawing of each speed vector of the some feature point which the monitor of the vehicle periphery monitoring apparatus of FIG. 1 displays. It is the schematic explaining the case where the moving object in the side image which the monitor of the vehicle periphery monitoring apparatus of FIG. 1 displays is highlighted. These are explanatory drawings which show the relationship between the image I used by the feature point extraction algorithm, the vicinity area | region D (p), and the pixel p. It is a flowchart of the image display control routine which the vehicle periphery monitoring apparatus of FIG. 1 performs. It is a block diagram showing the control function part of the periphery monitoring apparatus for vehicles which is other embodiment. It is a flowchart of the image display control routine which the vehicle periphery monitoring apparatus which is other embodiment performs. It is explanatory drawing of the side image which the conventional vehicle displays, (a) shows the image at the time of going straight, (b) shows the image at the time of turning.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 1n Nose view camera 1s Side camera 1r Rear view camera 2 Vehicle periphery image generation means 3 Feature point extraction means 4 Yaw rate calculation means 5 Monitor 501 Monitor control means 6, 6A Opposite object recognition means 6a Side image feature point extraction means 6b correction feature point creation means 6c moving object detection means 6f feature point tracking means 6g passage time calculation means 7 image display control means 11 vehicle speed sensor 20 navigation device ω yaw rate ^ (ω) estimated yaw rate tc expected passage time Bp virtual viewpoint C vehicle Ep Image area ECU Image control device Gf Vehicle peripheral image Gs Side image Gs1, Gs2 Left and right front side images Gf Vehicle peripheral image P, P1, P2, P3 Feature points Q1, Q2, Q3 Side feature points Q1 ', Q2', Q3 'correction feature point' v 'feature point speed vector Vc vehicle speed

Claims (3)

A vehicle peripheral image that generates a vehicle peripheral image, which is an overhead image viewed from a virtual viewpoint above the host vehicle, based on side images captured by a plurality of imaging devices installed at different locations of the host vehicle. Generating means;
A feature point extracting means for extracting a feature point from the vehicle peripheral image and calculating a motion vector of the feature point;
A yaw rate calculating means for calculating the yaw rate of the host vehicle from the motion vector of the feature point;
Image display control means for displaying the side image on a monitor;
Opposing object recognition means for performing recognition processing on the opposing object in the image of the monitor based on the corrected feature point obtained by subtracting the yaw rate from the flow of the side feature point extracted from the side image;
A vehicle periphery monitoring device comprising: In the vehicle periphery monitoring device according to claim 1,
A nose view camera that is attached to the front portion of the vehicle among the imaging device and captures a field of view in the vehicle width direction as a side image,
The opposing object recognition means includes
Side image feature point extracting means for detecting a side feature point in the side image and calculating a motion vector of the detected side feature point;
Correction feature point creating means for creating a correction feature point by subtracting the yaw rate from the flow of the side feature points in the side image;
Moving object detection means for detecting an object having a vector in a direction approaching the host vehicle in the side image from the motion vector of the correction feature point as an approaching moving object;
An object image region setting means for setting an image region in the side image of the approaching moving object,
The vehicle periphery monitoring device, wherein the image display control means highlights an image area of the approaching moving object in the side image. In the vehicle periphery monitoring device according to claim 1,
A nose view camera that is attached to the front portion of the vehicle among the imaging device and captures a field of view in the vehicle width direction as a side image,
The opposing object recognition means includes
A side image feature point extracting means for detecting a side feature point in the side image and calculating a motion vector of the detected feature point;
Correction feature point creating means for creating a correction feature point by subtracting the yaw rate from the flow of the side feature points in the side image;
Moving object detection means for detecting an object having a vector in a direction approaching the host vehicle in the side image from the motion vector of the correction feature point as an approaching moving object;
Feature point tracking means for tracking the corrected feature point of the detected approaching moving object;
Passing time calculation means for obtaining a time for the approaching moving object to pass the front of the vehicle from the correction feature point to be tracked.

Images