JP2005276057A - Nose view monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily and precisely detect an approaching object at the side part of the nose part of a vehicle when the vehicle revolves by a simple logic in order to announce approaching object information to a crew. <P>SOLUTION: This nose view monitoring device comprises an image pickup means 3 for picking up the image of the side part of a vehicle, an optical flow vector calculating means 11 for calculating a flow vector, a revolving direction detecting means 7 for detecting the revolving direction, revolving state correspondence value detecting means 4 and 7 for detecting a parameter corresponding to the revolving state, a revolving optical flow vector setting means 14 for setting a revolving flow vector generated by revolution, a correction optical flow vector calculating means 15 for calculating a correction flow vector by subtracting the revolving flow vector from the flow vector, an approaching object detecting means 12 for detecting an approaching object based on the corrected flow vector and an announcing means 5 for announcing the detection of the approaching object by displaying the image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a nose view monitor device that detects an optical flow vector of a captured image based on a captured image of a nose view camera.

  2. Description of the Related Art Conventionally, there has been a technique of providing an imaging camera for imaging the left and right sides at the nose (front end) portion of a vehicle, displaying the captured image (so-called nose view image) on a monitor device, etc., and assisting the occupant with the naked eye. Has been developed. In such a technique, generally, a monitor device that displays a captured image is also used as an in-vehicle monitor that displays a television image, a car navigation image (navigation image), and the like. A navigation image is displayed, and when the vehicle approaches an intersection or a T-junction and stops temporarily, the display is automatically switched to a nose view image display.

  Patent Document 1 discloses a vehicle camera device that is attached to a front portion of a vehicle, captures left and right side images, and displays them on a monitor device (display) in a vehicle compartment, before the vehicle stops or at a predetermined low level. A configuration is described in which the apparatus is automatically turned on according to the vehicle speed and the magnitude of deceleration before traveling at a high speed. According to such a configuration, for example, when the vehicle approaches an intersection and the driver tries to check the left and right, the device is turned on before the temporary stop and the left and right imaging screens from the nose portion are displayed. The driver can check the left and right speedily and smoothly with a margin, and on the other hand, the device is not turned on when driving in a nonroro due to traffic jams, so that the TV image, navigation image, etc. are not disturbed. Can be done.

  On the other hand, a technique for detecting a moving object in a captured image using an optical flow in a captured image of such a monitor device has been developed. The optical flow is a two-dimensional velocity vector field on an image, that is, an apparent velocity field of a moving object in a moving image. In such a technique, for example, a point that can be recognized as the same object is set as a feature point (detected by arithmetic processing) between two consecutive images captured at a predetermined period, and the feature point is moved. (Moving direction and moving distance) is calculated as a vector (this vector is an optical flow vector, also simply referred to as a flow vector, and this vector may also be referred to as an optical flow). Then, by calculating the feature points and flow vectors in the entire area in the captured image, information such as the position and moving direction of the moving object in the image can be recognized.

  Further, for example, in Patent Document 2, calculation is performed by omitting an area corresponding to a landscape outside the road on the image in the calculation process for obtaining the optical flow of the captured image in the traveling direction of the vehicle (the forward direction that is the traveling direction of the vehicle). The structure which performs is described. Specifically, the optical flow is obtained only for an area including a lower part surrounded by straight lines drawn from the infinity point on the image to the lower corners of the screen and a part around the infinity point. Yes. As a result, the amount of calculation can be reduced compared to obtaining the optical flow in the entire region in the image, and the processing time can be shortened and the processing speed can be increased.

  By the way, when thinking about recognizing a moving object such as a vehicle approaching the host vehicle using an optical flow from a nose view image obtained by imaging the left and right sides of the vehicle, if the host vehicle is in a stopped state, A flow vector is not generated for an object whose relative position with respect to the host vehicle is not changed (for example, a background portion in an image such as a building or a roadside tree), and a flow vector is generated only for an actually moved object. . That is, a moving object exists at the place where the flow vector is generated, and the moving object can be accurately detected based on the presence or absence of the flow vector.

  However, when the vehicle is advanced with the steering turned off, the entire nose view image moves according to the turning radius and turning speed of the vehicle. In other words, when the imaging camera itself moves, a flow vector is generated in the entire image, and the optical flow generated by the movement of the host vehicle and the optical flow by the actual moving object are detected in a mixed state. Therefore, there is a problem that it is difficult to detect these separately.

In response to such a problem, for example, Patent Document 3 discloses a moving body recognition apparatus that calculates an optical flow from an image obtained by photographing the front of the vehicle, and detects movement direction data and the optical flow of the host vehicle when the vehicle turns. A technique is disclosed in which an apparent optical flow generated by the movement of the host vehicle is calculated based on the distance to the target object, and a moving object is detected by compensating for the influence of the movement of the host vehicle on the optical flow. .
Japanese Patent No. 3287817 Japanese Patent No. 3398934 Japanese Patent No. 3239521

However, in the technique described in Patent Document 3, in calculating the apparent optical flow, that is, the optical flow generated by the movement of the host vehicle, not only the movement direction data such as the traveling speed and yaw rate of the host vehicle but also the detection target It is necessary to grasp the distance to a certain object, the configuration becomes complicated, and the calculation processing may be complicated.
In addition, for example, in an image captured by a camera that captures the front and rear of a vehicle, the apparent moving speed of an object to be detected as a moving object is the difference between the traveling speed of the host vehicle and the traveling speed of the object (relative speed). ), The image is picked up at a moving speed smaller than the moving speed of the actual object, and the size of the optical flow of the object detected from the picked-up image is often relatively small.

  On the other hand, in the nose view camera that images the side of the vehicle, the target vehicle is an image of an ordinary vehicle that is normally traveling while the host vehicle is stopped or slowing down. A relatively large optical flow is obtained when the image is captured at a movement speed approximately equal to the movement speed of the object (that is, the apparent movement speed on the captured image of the object is not so small because the movement speed of the host vehicle is small). This is often detected, and the amount of calculation processing increases in the calculation of the optical flow. Therefore, in order to accurately recognize a moving object such as a vehicle approaching the host vehicle from the nose view image using the optical flow, it is necessary to mount an arithmetic processing device having a high processing capacity. There is a problem of increasing the cost. As described above, when a moving object is detected from an image captured by a nose view camera using an optical flow, it has been a conventional problem to reduce the arithmetic processing.

  The present invention has been devised in view of such a problem, and with simple logic, an approaching object on the side of the nose portion of the vehicle at the time of turning of the vehicle can be easily and accurately detected to approach the occupant. An object of the present invention is to provide a nose view monitor device capable of notifying object information.

  In order to achieve the above object, a nose view monitor device according to the present invention (claim 1) is provided on the front surface of a nose of a vehicle and takes an image of a vehicle side, and calculates an optical flow vector based on the image. An optical flow vector calculating means, a turning direction detecting means for detecting the turning direction of the vehicle, a turning state corresponding value detecting means for detecting a parameter corresponding to the turning state of the vehicle, the turning direction and the parameter A turning optical flow vector setting means for setting a turning optical flow vector generated by turning of the vehicle in the vehicle side image, and the turning optical flow vector from the optical flow vector calculated by the optical flow vector calculating means. Swivel optical flow vector set by setting means Optical flow vector correction means for subtracting and correcting the error, approaching object detection means for detecting an approaching object based on the optical flow vector corrected by the optical flow vector correction means, and displaying the image and the approaching object And an informing means for informing the detection of the above.

Further, the turning optical flow vector setting means sets the direction of the turning optical flow vector in the horizontal direction and outside the turning direction detected by the turning direction detection means, and the turning optical flow vector of the turning optical flow vector is set. It is preferable that the magnitude is set in correspondence with the value of the parameter detected by the turning state corresponding value detecting means (claim 2).
Further, the approaching object detection means detects an approaching object based on an optical flow vector having a vector component in the traveling direction of the vehicle in the image among the optical flow vectors corrected by the optical flow vector correction means. (Claim 3).

Further, it is preferable that the turning state corresponding value detecting means detects an angular velocity at the time of turning of the vehicle as a parameter corresponding to the turning state of the vehicle (Claim 4), or the traveling speed and steering of the vehicle. It is preferable to detect the corner (claim 5).
In addition, the notification means includes a plurality of notification modes according to the size or number of optical flow vectors having a vector component in the traveling direction of the vehicle among the optical flow vectors corrected by the optical flow vector correction means. It is preferable to switch to display the image and to notify the detection of the approaching object.

  Further, the nose view monitor device of the present invention (claim 7) is provided with an imaging means provided on the front side of the nose of the vehicle for capturing an image on the side of the vehicle, and an optical flow vector calculation for calculating an optical flow vector based on the image. Means for detecting the turning direction of the vehicle, and the optical flow vector calculated by the optical flow vector calculating means at the time of turning of the vehicle detected by the turning direction detecting means. An approaching object is detected based on an optical flow vector that is present in a lateral region in the vehicle turning outward direction and has a vector component in the traveling direction of the vehicle in the image, and when the approaching object is detected, Optica present in a lateral area in the direction of turning inside the vehicle detected by the turning direction detecting means And approaching object detection means without reference to the flow vector is characterized by comprising an informing means for informing a detection of 該接 near object and displays the image.

  Further, the notification means switches a plurality of notification modes according to the magnitude or number of optical flow vectors having a vector component in the traveling direction of the vehicle, displays the image and notifies the detection of the approaching object. (Claim 8).

  According to the nose view monitor device of the present invention (Claim 1), the turning flow vector generated by turning of the vehicle can be set based on the turning direction and turning state of the vehicle, that is, turning of the host vehicle. The optical flow vector correction calculation can be simplified. Further, erroneous recognition of an approaching object can be reduced by the correction calculation using the turning flow vector, and the detection rate of the approaching object can be improved.

Further, according to the nose view monitor device of the present invention (claim 2), the turning flow vector generated by turning of the vehicle can be calculated easily and easily. In addition, this makes it possible to simplify the calculation process for detecting an approaching object when the vehicle is turning, and to reduce the calculation amount.
Further, according to the nose view monitor device of the present invention (Claim 3), it is possible to easily recognize an approaching object to the host vehicle on the side of the vehicle when the vehicle turns. Moreover, the amount of calculation can be reduced by making the direction of the optical flow concerning the recognition of the approaching object to the own vehicle the vehicle traveling direction. Furthermore, even when the vehicle moves at a low speed, it is possible to prevent the movement of the background image in the captured image from being mistakenly recognized as an object approaching the host vehicle, and the detection accuracy is improved by accurately detecting the moving object. Can be made. Therefore, erroneous notification of an approaching object to the host vehicle can be reduced.

Further, according to the nose view monitor device of the present invention (claims 4 and 5), it is possible to grasp the turning state of the vehicle using parameters that are easy to detect.
Further, according to the nose view monitor device of the present invention (Claims 6 and 8), the risk of an approaching object to the host vehicle is reduced by switching a plurality of notification modes according to the size and number of optical flow vectors. It is possible to make a determination, and effective notification according to the degree of risk can be performed, and safety can be improved.

  In addition, according to the nose view monitor device of the present invention (Claim 7), by referring only to the optical flow vector existing in a direction in which an object approaching the host vehicle can be reliably detected at the time of turning, the host vehicle can be turned. Such optical flow vector correction calculation can be simplified. In addition, when detecting an approaching object, the optical flow vector that exists in the lateral region inside the turning direction of the vehicle is not referred to, so that erroneous recognition of an approaching object can be reduced and the detection rate of the approaching object is improved. Can be made.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIGS. 1-7 shows the nose view monitor apparatus as embodiment of this invention, FIG. 1 is a typical block diagram which shows the vehicle provided with the nose view monitor apparatus as 1st Embodiment of this invention, FIG. 2 is a schematic diagram of display screen contents as an example of a monitor display by this apparatus, and FIGS. 3A to 3F are schematic diagrams for explaining calculation processing in the approaching object detection means of this apparatus. The left half shows the nose view and flow vector on the left side of the vehicle, the right half shows the nose view and flow vector on the right side of the vehicle, and FIG. 4 is a flowchart for explaining the control in this apparatus. FIG. 5 is a schematic plan view showing an imaging region of a nose view camera in the present apparatus, and FIG. 6 is a schematic configuration diagram showing a vehicle equipped with a nose view monitor apparatus as a second embodiment of the present invention. Is a flow chart for explaining the control in this device.
[First Embodiment]
Referring to FIG. 1, there is shown a vehicle 2 equipped with a nose view monitor device 1 as a first embodiment of the present invention. The vehicle 2 includes a nose view camera (imaging unit) 3 that images the left and right sides of the vehicle 3, a vehicle speed sensor (turning state corresponding value detection unit) 4 that detects a traveling speed signal of the vehicle 2, and an operation switch for the nose view camera 3. A nose camera switch 6, a steering angle sensor (turning direction detecting means / turning state corresponding value detecting means) 7 for detecting a steering angle (or steering wheel steering angle) signal of a steering wheel by an occupant, an electronic control unit (ECU) 10 And a monitor (notification means) 5 for displaying an image picked up by the nose view camera 3.

A pair of nose view cameras 3 are provided at the left and right end portions of the nose (front end) portion of the vehicle 2 so that the left side and the right side of the vehicle can be imaged simultaneously.
The monitor 5 displays left and right side images taken by the nose view camera 3. In the present embodiment, as shown in FIG. 2, the image on the right side of the vehicle 2 is displayed in the right half area on the monitor screen, and at the same time, the image on the left side of the vehicle 2 is displayed in the left half area on the monitor screen. It is like that. As a result, the occupant can check the left and right sides of the vehicle at the same time.

  In this embodiment, the regions imaged by the left and right nose view cameras 3 are on the left and right sides in the vehicle width direction of the vehicle 2 and are perpendicular to the traveling direction of the vehicle 2, as shown in FIG. It is designed to point slightly forward rather than the correct direction. Thereby, in the left side image of the vehicle 2, the traveling direction of the vehicle is the right side direction on the image, while in the right side image, the traveling direction of the vehicle is the left side direction on the image. Further, as shown in FIG. 2, the vehicle 21 approaching the vehicle 2 on the road ahead of the vehicle 2 in the image on the left side is displayed so as to expand while moving in the right direction on the image. On the other hand, the vehicle 22 approaching the vehicle 2 on the road ahead of the vehicle 2 in the image on the right side is displayed so as to expand while moving in the left direction on the image.

  The vehicle speed sensor 4 detects wheel rotation number information and inputs the rotation number information to the ECU 10. Similarly, the steering angle sensor 7 detects the steering angle information of the steering wheel steered by the occupant and inputs it to the ECU 10. The ECU 10 calculates the traveling speed V of the vehicle 2 based on the input wheel speed information, and calculates the steering angle θ of the steered wheels based on the steering angle information. Then, the vehicle turning speed (for example, turning angular speed) is calculated from the traveling speed V and the steering angle θ.

Further, the ECU 10 can determine whether the turning direction of the vehicle 2 is the right direction or the left direction based on the sign of the steering angle θ of the steered wheels. For example, on the basis of a neutral state where the steering wheel is not steered, the steering angle θ is calculated as a positive value in the clockwise direction and a negative value in the counterclockwise direction.
The nose camera switch 6 is a switch for switching on / off the operation of the nose view camera 3. The nose view camera 3 is not operated when the nose camera switch 6 is turned off. Further, when the nose camera switch 6 is turned on, the nose view camera 3 is activated when a predetermined condition (nose view camera operating condition) is established.

The nose view camera operating condition is, for example, a predetermined speed V ₀ (a speed that is not excessively large corresponding to a low-speed state, for example, 5 km / h) in which the traveling speed V of the vehicle 2 calculated by the ECU 10 is preset. And the steering angle θ of the steered wheels is smaller than a predetermined angle θ ₀ set in advance (that is, the steering state of the steering wheel is not steered excessively).

The monitor 5 displays the captured image when the nose view camera 3 is operating, but displays another image such as a television image or a car navigation image when the nose view camera 3 is not operating. It functions as an in-vehicle monitor.
If the nose camera switch 6 is turned on in a state in which TV images and car navigation images are continuously displayed during normal driving, when the vehicle 2 is about to enter an intersection or T-junction, the rudder When the angle θ is smaller than the predetermined angle θ ₀ and the speed of the vehicle is decelerated from the predetermined speed V ₀ , the nose view camera 3 is automatically operated, and left and right side images are displayed on the monitor 5. It is like that. In other words, the image is automatically switched from the TV image or the car navigation image to the left and right side images without being conscious of the passenger. Further, when the steering angle θ is greater than or equal to the predetermined angle θ ₀ or when the traveling speed V is greater than or equal to the predetermined speed V ₀ , the nose view camera 3 does not operate and a TV image or car navigation image is displayed as a general vehicle monitor. It is supposed to continue.

Note that the magnitudes of the predetermined speed V ₀ and the predetermined angle θ ₀ are set in consideration of the magnitude of the flow vector of the background portion in the captured image of the nose view camera 3, and will be described later.
Further, if the nose camera switch 6 is turned off, the nose view camera 3 can be prevented from operating even if the above-described nose view camera operating conditions are satisfied.

  The ECU 10 is able to detect a moving object approaching the host vehicle by calculating an optical flow of each of the left and right captured images captured by the nose view camera 3. First, the ECU 10 has an optical flow calculation unit (optical flow vector calculation means) 11 for calculating an optical flow vector of a captured image, and a turning flow vector setting unit (a turning optical vector) for calculating a turning optical flow vector generated by turning of the vehicle 2. A flow vector setting unit (14) and a flow vector correction unit (optical flow vector) for subtracting and correcting the turning optical flow vector set by the turning flow vector setting unit 14 from the optical flow vector calculated by the optical flow calculation unit 11 Correction means) 15, an approaching object detection unit (approaching object detection means) 12 that detects an approaching object to the host vehicle based on the optical flow vector corrected by the flow vector correction unit 15, and calculation and detection results thereof. Is constructed and an output unit 13 for outputting. In the following description, each optical flow vector is simply referred to as a flow vector, and a set of these flow vectors is referred to as an optical flow.

  The optical flow calculation unit 11 can individually calculate the optical flows of the left and right side images captured by the nose view camera 3, and the left side image (that is, the left half in FIG. 2). The optical flow of the right side image (that is, the image of the right half region in FIG. 2) is obtained in the left side region optical flow calculation unit 1B. Each is calculated. Regarding the calculation of the optical flow, a point corresponding to the same object is calculated (detected by arithmetic processing) between two consecutive images among the images captured by the nose view camera 3, and this is detected. A method of calculating a moving direction and a moving distance of a feature point as a flow vector is used. In addition, a flow vector is calculated in the entire area in the captured image, and information such as the position and moving direction of the moving object in the image can be recognized.

  The turning flow vector setting unit 14 determines both the left and right sides of the vehicle based on the turning direction of the vehicle 2 determined based on the sign of the steering angle θ of the steered wheel, the traveling speed V of the vehicle 2 and the steered angle θ of the steered wheel. In each of the images, a turning flow vector (turning optical flow vector) generated by turning of the vehicle is set. The turning flow vector set here is a flow vector generated by the movement of the entire captured image of the nose view camera 3 due to the turning of the vehicle 2. For example, as shown in FIG. It corresponds to a vector (outlined arrow).

First, the turning flow vector set here is set only for the feature points calculated by the optical flow calculation unit 11 in the captured image of the nose view camera 3.
Next, the direction of the turning flow vector is set to the outside direction of the turning direction of the vehicle 2 [here, the horizontal direction on the image (true lateral direction)]. For example, when the sign of the steering angle θ is positive, the vehicle 2 is turning to the right, so the direction of the turning flow vector is set to the left direction (left horizontal direction) in the captured image, and the sign of the steering angle θ When is negative, since the vehicle 2 is turning left, the direction of the turning flow vector is set to the right direction (right horizontal direction) in the captured image.

  In addition, the magnitude of the turning flow vector is the vehicle turning speed (turning angular speed) calculated from the traveling speed V and the steering angle θ for each of the left side area and the right side area of the image captured by the nose view camera 3. ) Based on a unique setting. For example, in the left side region of the captured image, all the background flow vectors generated by the turning of the vehicle 2 are set as vectors having the same magnitude, and the magnitude of the vector is 1: 1 with the turning speed of the vehicle 2. It is set to correspond.

  That is, when trying to accurately calculate the flow vector (turning flow vector) of the background portion generated by turning of the vehicle 2, the distance to each stationary object constituting the background and the angle with respect to the imaging direction of the nose view camera 3 are determined. After detection, the turning flow vector of each stationary object must be calculated. However, in this embodiment, the turning flow vector is set with the simple logic as described above. Therefore, in this embodiment, the turning flow vector setting unit 14 corresponds to the background for each of the left side region and the right side region of the captured image as long as the steering angle θ and the traveling speed V of the vehicle 2 are input. All the swirl flow vectors of the part can be set collectively.

  The flow vector correction unit 15 subtracts the turning flow vector calculated by the turning flow vector setting unit 14 from the optical flow calculated by the optical flow calculation unit 11 to obtain a corrected flow vector (hereinafter, corrected by the flow vector correction unit 15). The flow vector calculated as a result of the above is calculated and set). In other words, here, the optical flow detected in the captured image of the nose view camera 3 functions to remove the turning flow vector corresponding to the background portion.

  The approaching object detection unit 12 detects an object approaching the vehicle 2 based on the correction flow vector set by the flow vector correction unit 15. Specifically, an object approaching the host vehicle 2 is detected based on a flow vector having a vector component in the traveling direction of the vehicle 2 in the left and right side images. For example, a correction flow vector having a vector component in the right direction on the image is extracted from the image on the left side, while a correction flow vector having a vector component in the left direction on the image is extracted from the image on the right side. The extracted corrected flow vector is a flow vector based on an approaching object approaching the vehicle 2 (that is, an object approaching the host vehicle 2 instead of the background among all moving objects having a flow vector in the captured image). And the approaching object is recognized.

  That is, it is difficult to determine whether or not the moving object is approaching the host vehicle 2 simply by recognizing the moving object using the optical flow, but in the present embodiment, the approaching object detection unit 12 extracts and selects a flow vector of an object approaching the host vehicle 2 among moving objects recognized by the optical flow calculation unit 11 based on a region where the vehicle exists and its direction, and recognizes the flow vector. A moving object that is approaching the host vehicle 2 and that may be dangerous for the host vehicle 2 is recognized.

  In the present embodiment, in order to absorb the setting error in the turning flow vector setting unit 14 that sets the turning flow vector, all the correction flow vectors detected by the above-described logic are determined by an object approaching the host vehicle 2. Instead of determining that the flow vector is a flow vector, it is determined that the corrected flow vector is a flow vector by an object approaching the host vehicle 2 when the magnitude of the corrected flow vector is a predetermined amount or more. ing.

  In addition, when the approaching object detection unit 12 detects an approaching object to the host vehicle 2, the output unit 13 displays on the monitor 5 that the approaching object has been detected, and notifies the occupant by voice or the like. It has become. Here, the output unit 13 switches a plurality of notification modes based on the magnitude and number of correction flow vectors having a vector component toward the traveling direction of the vehicle 2 detected by the approaching object detection unit 12, Is displayed, and the detection of the approaching object is notified.

  That is, if attention is paid to the magnitude of the correction flow vector having a vector component toward the traveling direction of the vehicle 2, if the correction flow vector is large, the approaching object generating the correction flow vector is the own vehicle. Even if it is at a distance away from the vehicle 2, it is likely to be dangerous for the vehicle 2 because it is approaching the vehicle 2 at a high speed. Even if the vehicle 2 is not approaching, if the vehicle 2 is at a short distance, the moving object is still likely to be dangerous, and in this case as well, it has a vector component toward the traveling direction side of the vehicle 2. The correction flow vector becomes large.

Therefore, as the magnitude of the correction flow vector having the vector component toward the traveling direction side of the vehicle 2 increases, the risk level of the approaching object to the vehicle 2 is increased, and a plurality of notification modes corresponding to the risk level are provided. Is switched to perform screen display and notification.
Similarly, when there are many approaching objects to the host vehicle 2 or the distance between the host vehicle 2 and the approaching object is short, the number of correction flow vectors having vector components toward the traveling direction of the vehicle 2 increases. Therefore, as the number of correction flow vectors increases, the danger level of the approaching object to the vehicle 2 is increased, and a plurality of notification modes corresponding to the risk level are switched to perform screen display and notification. .

In the present embodiment, the output unit 13 includes a plurality of notification modes such as “low risk level notification mode” and “high risk level notification mode”, and the flow vector of the approaching object detected by the approaching object detection unit 12 is displayed. These notification modes are configured to be switched based on the size and number.
That is, when a corrected flow vector having a vector component toward the traveling direction side of the vehicle 2 is detected that has a magnitude larger than a predetermined magnitude, or a predetermined number or more is detected. When the output unit 13 is set, the output unit 13 is set to the “high risk level notification mode”. In other cases, the output unit 13 is set to the “low risk level notification mode”.

In the “small danger level notification mode”, the output unit 13 displays one of the correction flow vectors based on the approaching object with an arrow and notifies the detection of the approaching object by voice (for example, “Please pay attention to the surroundings” Announcement etc.)
In the “high risk level notification mode”, the output unit 13 highlights an area on the screen corresponding to the approaching object detected by the approaching object detection unit 12 (for example, changes the luminance and color tone), and the like. All correction flow vectors due to approaching objects are displayed on the screen as arrows, and the passenger is informed of the high degree of danger by voice (for example, “Please note that there are vehicles approaching”) It is like that.
Note that the output unit 13 notifies the occupant when the nose view camera operating condition is no longer satisfied (V ≧ V ₀ or θ ≧ θ ₀ ).

  The nose view monitor device 1 according to the first embodiment is configured as described above, and is controlled as follows according to the flowchart shown in FIG. This flow is always executed in the ECU 10 every predetermined cycle (for example, a cycle synchronized with the imaging cycle of the nose view camera 3).

Steps A10 to A30 are flows for determining whether or not the nose view camera 3 is operating as a premise for calculating the optical flow.
First, in step A10, it is determined whether or not the nose camera switch 6 is turned on. If it is on, the process proceeds to step A20, and if it is off, this flow is terminated. Then, in step A20, the travel speed V of the vehicle is determined whether less than the predetermined speed V _0, the process proceeds to step A30 in the case of V <V _0, the flow in the case of V ≧ V ₀ Exit. Subsequently, in Step A30, it is determined whether or not the steering angle θ of the steered wheel is smaller than a predetermined angle θ _0. If θ <θ ₀ , the process proceeds to Step A40, and if θ ≧ θ ₀ , this End the flow.

That is, when the nose view camera 3 is not operating, the flow is terminated without proceeding to step A40 and after, and the process proceeds to step A40 and after only after the nose view camera 3 is operated.
In step A40, the optical flow calculation unit 11 individually calculates the feature points in the left and right side images captured by the nose view camera 3, and in step A50 calculates the flow vectors for all the feature points. That is, here, a moving object in each of the left and right side images is recognized.

  Subsequently, in step A60, the turning flow vector setting unit 14 sets a turning flow vector based on the steering angle θ and the traveling speed V of the steered wheels. The turning flow vector set here is limited to the flow vector for only the feature point calculated in step A50. In step A70, the flow vector correction unit 15 subtracts the turning flow vector set in step A60 from the flow vector calculated in step A50 to calculate a corrected flow vector. That is, here, the correction flow vectors for all the feature points calculated in step A50 are calculated.

  Subsequently, in step A80, the approaching object detection unit 12 determines whether or not all the correction flow vectors calculated in step A70 have a vector component toward the traveling direction side of the vehicle 2. For the correction flow vector on the left side image of the vehicle 2, it is determined whether or not it has a vector component in the right direction on the image. On the other hand, for the correction flow vector on the right side image of the vehicle 2, Thus, it is determined whether or not it has a vector component in the left direction on the image. That is, here, among the moving objects recognized in step A40, the moving object approaching the vehicle 2 (approaching object) is recognized separately from the others.

  Here, when a corrected flow vector having a vector component toward the traveling direction side of the vehicle 2 is not detected (or when the detected corrected flow vector is less than a predetermined amount), an approaching object is detected. This flow is terminated because there is no (or a moving object within the detection error range). However, when the above flow vector is detected (and the detected correction flow vector is greater than or equal to a predetermined amount). If YES, go to Step A90.

In step A90, the output unit 13 displays on the monitor 5 that the approaching object has been recognized, and notifies the occupant by voice. Here, notification to the passenger in the output unit 13 is switched between notification modes according to the magnitude and number of correction flow vectors having vector components in the traveling direction of the vehicle 2 detected in step A70.
Of the approaching object correction flow vectors detected in step A60, if the magnitude of the approaching flow vector is greater than a preset predetermined magnitude, or if a preset number of approaching objects is corrected. When the flow vector is detected, the output unit 13 is set to the “high risk level notification mode”, the area on the screen corresponding to the approaching object is highlighted, and all the correction flow vectors by the approaching object are displayed. The screen is displayed with an arrow, and the passenger is notified that the degree of danger is high by voice, and this flow is terminated.

  Further, the correction flow vector of the approaching object detected in step A60 is not detected when its magnitude is larger than a preset predetermined magnitude, and more than the preset predetermined number of approaching objects. When the correction flow vector is not detected, the output unit 13 is set to the “small risk notification mode”, one of the correction flow vectors based on the approaching object is displayed on the screen as an arrow, and the approaching object is detected by voice. The passenger is notified and this flow is finished.

Specifically, the following operations and effects are achieved by the control as described above.
In a T-shaped road as shown in FIG. 5, when the vehicle 2 tries to enter the main road while checking left and right, first, when the nose camera switch 6 of the vehicle 2 is turned on, the vehicle 2 is connected to the main road. When the traveling speed is made smaller than the predetermined speed V _{0 in} front, the nose view camera 3 is automatically operated, and left and right side images are displayed on the monitor 5. Thereby, the left and right side images can be automatically displayed on the monitor 5 without making the occupant aware of the switching of the image on the monitor 5. When the nose view camera 3 is activated, the ECU 10 starts calculating the optical flow in the captured image.

[When stopped]
Here, when the vehicle 2 is stopped at the position shown in FIG. 5 in a neutral state, the position of the nose view camera 3 is fixed as shown in FIG. The flow vector is not generated in the background part (the part that is the background of the vehicles 21 and 22 on the image, and refers to the non-moving object such as roads, buildings, guardrails, and sky) in the image, Only flow vectors (black arrows in FIG. 3A) are generated by the vehicles 21 and 22 as the approaching objects approaching the vehicle 2. At this time, since the turning speed calculated from the steering angle θ and the traveling speed V is 0 and the vehicle 2 is not turning, the magnitude of the turning flow vector is set to 0, and the vehicle 2 is approaching. The flow vector by the vehicles 21 and 22 as objects becomes the corrected flow vector as it is.

At this time, the direction of the correction flow vector (that is, the flow vector by the vehicles 21 and 22) is that the vehicle 21 has a vector component in the right direction in the left side image, that is, the traveling direction of the vehicle 2 in the image. Will have vector components to On the other hand, the vehicle 22 has a vector component in the left direction in the screen on the right side, and has a vector component in the traveling direction of the vehicle 2 in the image.
Therefore, the approaching object detection unit 12 can detect the vehicles 21 and 22 as approaching objects based on the flow vector having a vector component toward the traveling direction of the vehicle in the left and right side images.

[When turning]
Further, when the vehicle 2 is turning right from the position shown in FIG. 5 at a low speed (traveling speed V <V ₀ ), that is, the occupant slowly advances the vehicle 2 while checking left and right and wants to proceed in the right direction. 3 (d), the movement of the imaging position of the nose view camera 3 causes the background portion in the captured image to also flow in the background vector (in FIG. 3 (d)). White arrow) will occur. The flow vectors of the vehicles 21 and 22 are the flow vector that will be generated when the vehicle 2 is stopped (that is, the flow vector of the black arrow shown in FIG. 3A), and the nose view camera. As a result, a flow vector (black arrow in FIG. 3D) is generated as a sum of the flow vector of the background portion (flow vector of the white arrow) generated by the position movement of 3. The flow vector of the background portion is generated as a flow vector having a vector component in the left direction of the vehicle 2 on the image when the vehicle 2 turns to the right.

  At this time, since the vehicle 2 is turning right, the turning flow vector setting unit 14 sets the direction of the turning flow vector to the left in the image, and each of the left side region and the right side region of the captured image. On the other hand, the magnitude of the turning flow vector is set on a one-to-one basis based on the turning speed calculated from the steering angle θ and the traveling speed V (white arrow in FIG. 3E). Then, the flow vector correction unit 15 subtracts the turning flow vector shown in FIG. 3 (e) from the flow vector shown in FIG. 3 (d), and the corrected flow vector shown in FIG. 3 (f). Is calculated.

  At this time, the direction of the correction flow vector by the vehicles 21 and 22 is a vector direction having a vector component on the traveling direction side of the vehicle 2 in the image as shown in FIG. A direction having a vector component in the direction, and a direction having a vector component in the left direction in the right side region. In other words, in the left region, the flow vector has a direction in the range of 180 degrees clockwise from the vertical direction, and in the right region, the flow vector has a direction in the range of 180 degrees counterclockwise from the vertical direction.

Therefore, the approaching object detection unit 12 can detect the vehicles 21 and 22 as approaching objects based on the flow vector having a vector component toward the traveling direction of the vehicle in the left and right side images. Further, the output unit 13 of the ECU 2 outputs to the monitor 5 that an approaching object has been detected, and the passenger can be alerted.
Moreover, since the output part 13 switches notification mode according to the danger level with respect to the own vehicle 2 of an approaching object, it can notify a passenger | crew effectively and can improve safety | security. be able to.

Note that when the vehicle 2 moves forward too quickly, the flow vector of the background portion generated by the movement of the position of the nose view camera 3 increases, so the direction of the flow vector by the vehicles 21 and 22 is the vehicle 2 on the image. However, in the present embodiment, when the traveling speed V of the vehicle 2 is equal to or higher than a predetermined speed V ₀ (0 <V ₀ ), the nose view camera 3 is considered. Since the operation of is stopped, erroneous recognition of an approaching object can be prevented. Similarly, since the operation of the nose view camera 3 is stopped when the steering angle θ of the vehicle 2 exceeds a predetermined angle θ ₀ (0 <θ ₀ ), an approaching object is erroneously recognized by a turning movement. Nor. As a result, the detection accuracy of the approaching object by the optical flow can be improved.

Even when the nose view camera operating condition is not established and the operation of the nose view camera 3 is stopped, the output unit 13 notifies the occupant to that effect, so that the occupant is cautioned and safety is increased. Can be improved.
The setting of the magnitudes of the predetermined speed V ₀ and the predetermined angle θ ₀ is arbitrary depending on the embodiment, but when the vehicle 2 turns, the larger the traveling speed V and the steering angle θ, The flow vector of the background part becomes large. Therefore, in order to detect an object with a fast approach speed, there is no problem even if the set values of the predetermined speed V ₀ and the predetermined angle θ ₀ are increased. On the other hand, to detect an object with a slow approach speed, It is desirable that the vehicle 2 is stopped. However, when the vehicle 2 is turning, by utilizing the difference in directionality between the flow vectors of the vehicles 21 and 22 as the approaching objects traveling on the general public road and the flow vectors generated in the background portion ( In other words, the vehicles 21 and 22 as the approaching objects can be distinguished from the background portion by the logic as described above.

  Thus, according to the nose view monitor device of the present invention as the first embodiment, even when the vehicle 2 is turning, the turning flow vector generated by turning of the vehicle is determined as the turning direction of the vehicle. It can be set based on the turning state, and the correction calculation related to the turning of the host vehicle, that is, the calculation in the turning flow vector setting unit 14 can be simplified. Further, since the direction of the turning flow vector is set to the outer side of the turning direction, and the magnitude thereof is uniquely set based on parameters (steering angle θ and traveling speed V) corresponding to the turning state, Arithmetic processing is simplified and the amount of calculation can be reduced.

  In addition, the logic as described above can easily and reliably recognize an approaching object on the side of the vehicle when the vehicle is turning, and can prevent erroneous recognition of the movement of the background image as an approaching object. Accuracy can be improved. Further, among the moving objects recognized based on the optical flow, the configuration for recognizing an approaching object that is dangerous with respect to the host vehicle 2 is also simple, and the amount of calculation of the ECU 10 relating to the recognition of the approaching object is further reduced. Can be made. Moreover, safety can be improved by notification according to the degree of danger.

  Regarding the method of setting the turning flow vector in the turning flow vector setting unit 14, in the first embodiment described above, the turning flow vector turns in the direction outside the turning direction of the vehicle 2 (the lateral direction on the image). The magnitude of the flow vector is set to a magnitude based on the turning speed (turning angular speed) of the vehicle, but is not limited to such a setting method, for example, the magnitude of the turning speed of the vehicle. The direction of the turning flow vector may be corrected according to the height, or a predetermined gain may be set in advance according to the position of the feature point on the captured image. A vector obtained by multiplying the magnitude of the vector set based on the speed by a predetermined gain may be set as the turning flow vector.

  In the first embodiment described above, the turning flow vectors having the same size are set in the left side region and the right side region of the captured image. The size of the swirl flow vector set in step 2 does not have to be set to the same size for each of the left-side area and the right-side area of the captured image. For example, depending on the position on the captured image Thus, the setting may be made based on the turning speed (turning angular speed) of the vehicle calculated from the traveling speed V and the steering angle θ.

  In the first embodiment described above, the traveling speed V of the vehicle 2 and the steering angle θ of the steered wheels are used as parameters corresponding to the turning state of the vehicle 2. You may make it the structure used as the parameter corresponding to a turning state. In this case, an angular velocity sensor for detecting the angular velocity of the vehicle 2 and a gyro device are provided. Moreover, the combination of these parameters is arbitrary.

As described above, the setting method based on the parameters corresponding to the turning direction and turning state of the vehicle 2 is arbitrary.
In the first embodiment described above, the approaching object detection unit 12 detects an approaching object based on the correction flow vector having a vector in the moving direction of the vehicle in the image. The approaching object may be detected by another logic, and the approaching object detection method is arbitrary.

[Second Embodiment]
Next, a nose view monitor device 21 as a second embodiment of the present invention will be described. Hereinafter, the same components as those in the first embodiment are not described and are denoted by the same reference numerals.
FIG. 6 shows a vehicle 2 equipped with the nose view monitor device 21. The vehicle 2 is a nose view camera (imaging means) 3 that images the left and right sides of the vehicle 3. A vehicle speed sensor 4 to detect, a nose camera switch 6 as an operation switch of the nose view camera 3, a steering angle sensor (turning direction detection means) 7 to detect a steering angle (or steering angle of a steered wheel) signal of a steering wheel by an occupant, A monitor (notification means) 5 for displaying an image captured by the electronic control unit (ECU) 10 and the nose view camera 3 is provided.

  The ECU 10 can calculate the optical flow of each of the left and right side captured images captured by the nose view camera 3 and detect a moving object approaching the host vehicle. An optical flow calculation unit (optical flow vector calculation means) 11 that calculates a flow, and an approaching object detection unit (approaching object detection) that detects an approaching object to the host vehicle based on the optical flow vector calculated by the optical flow calculation unit 11 Means) 32 and an output unit 13 for outputting the calculation and detection results.

The approaching object detection unit 32 detects an object approaching the vehicle 2 based on the flow vector calculated by the optical flow calculation unit 11. However, the approaching object detection unit 12 detects an approaching object by referring only to the flow vector in the lateral region outside the turning direction of the vehicle 2 detected by the steering angle sensor. .
Specifically, first, when the steering angle θ of the vehicle 2 is positive and is equal to or larger than a preset reference value θ ₁ (where θ ₁ <θ ₀ ) (that is, the rightward turning speed is slightly high). Case), an approaching object is detected based on the flow vector in the left side region of the vehicle 2. That is, a flow vector having a vector component in the right direction is extracted on the left-side region image, and the extracted flow vector is an approaching object approaching the vehicle 2 (that is, among moving objects having a flow vector, It is determined that the flow vector is based on an object approaching the host vehicle 2, and the approaching object is recognized. At this time, the flow vector in the right side region of the vehicle 2 is not referred to. That is, the approaching object is not detected for the right side.

On the other hand, when the steering angle θ of the vehicle 2 is negative and is equal to or less than the reference value −θ ₁ (that is, when the turning speed in the left direction is slightly high), based on the flow vector in the right side region of the vehicle 2. , Detecting an approaching object. That is, a flow vector having a vector component in the left direction is extracted from the image on the right side region, and the extracted flow vector is determined to be a flow vector by an approaching object approaching the vehicle 2, and an approaching object is recognized. To do. At this time, the flow vector in the left side region of the vehicle 2 is not referred to. That is, the approaching object is not detected for the left side.

When the steering angle θ of the vehicle 2 is less than the reference value θ ₁ (that is, when the turning speed is low), an approaching object is detected based on the flow vectors in the left and right side regions of the vehicle 2. It is like that.
The approaching object detection unit 32 is configured as described above for the following reason. That is, when considering a moving object such as a vehicle approaching the host vehicle 2 using an optical flow from a nose view image obtained by imaging the left and right sides of the vehicle 2, the host vehicle 2 temporarily stops. If it is in the state, no flow vector is generated in the object whose relative position with respect to the host vehicle 2 is not changed. Therefore, there is a moving object at the position where the flow vector is generated. A moving object can be detected.

  However, when the vehicle 2 is turning, the angle (composition) itself of the nose view image obtained by imaging the left and right sides moves in the direction opposite to the turning direction. For example, when the vehicle 2 is turning right, as shown in FIG. 3D, the left and right side images captured by the nose view camera 3 both move to the left and flow to the left. A vector (a flow vector by turning of the vehicle 2, that is, a white arrow in FIG. 3D) is detected on the entire screen. Therefore, when there is a moving object such as a vehicle that actually approaches the vehicle 2, in the left-side image of the vehicle 2, the flow vector (black arrow in the left-side image in FIG. 3D) and the vehicle The flow vector due to the turn of 2 (the white arrow in the left side image in FIG. 3D) can be distinguished and distinguished from the difference in the directionality of the vector. Is the flow vector due to the approaching vehicle (black arrow in the right side image in FIG. 3D) and the flow vector due to turning of the vehicle 2 (white arrow in the right side image in FIG. 3D) are vector directions Are similar and difficult to distinguish and recognize. Then, as the turning angle (or turning speed, for example) of the vehicle 2 increases, the flow vector resulting from the turning of the vehicle 2 also increases, and the similarity in the direction of both vectors increases, so that both are distinguished and recognized. It becomes more difficult to do.

  Therefore, in the second embodiment, when the vehicle 2 is turning and the turning speed is slightly high, the approaching object detection unit 32 refers to only the flow vector in the lateral region outside the turning direction. By configuring so as to detect an object, it is possible to distinguish and recognize the flow vector due to the approaching vehicle and the flow vector due to the turning of the vehicle 2 based on the difference in vector directionality, thereby reducing erroneous recognition. It is.

The nose view monitor device 21 according to the second embodiment is configured as described above, and is controlled as follows according to the flowchart shown in FIG. This flow is always executed in the ECU 10 every predetermined cycle (for example, a cycle synchronized with the imaging cycle of the nose view camera 3).
Steps B10 to B30 are flows for determining whether or not the nose view camera 3 is operating as a precondition for calculating the optical flow.

First, in step B10, it is determined whether or not the nose camera switch 6 is turned on. If it is on, the process proceeds to step B20, and if it is off, this flow is terminated. Next, in step B20, the travel speed V of the vehicle is determined whether less than the predetermined speed V _0, the process proceeds to step B30 in the case of V <V _0, the flow in the case of V ≧ V ₀ Exit. Subsequently, in step B30, it is determined whether or not the steering angle θ of the steered wheel is smaller than a predetermined angle θ _0. If θ <θ ₀ , the process proceeds to step B40, and if θ ≧ θ ₀ End the flow.

In other words, if the nose view camera 3 is not operating, the flow is terminated without proceeding to step B40 and thereafter, and the process proceeds to step B40 and thereafter only after the nose view camera 3 is operated.
In step B40, the optical flow calculation unit 11 individually calculates the feature points in the left and right side images captured by the nose view camera 3, and in step B50 calculates the flow vectors for all the feature points. That is, here, a moving object in each of the left and right side images is recognized.

Subsequently, in step B60, it is determined whether or not the steering angle θ is smaller than a reference value θ ₁ (0 <θ ₁ <θ ₀ ) set as a value smaller than the predetermined value θ ₀ . That is, here, the degree of similarity between the flow vector due to the turning of the vehicle 2 and the flow vector due to the approaching vehicle is determined by the magnitude of the steering angle θ.
Here, | theta | <in case of theta _1, the program proceeds to step B70 as the steering angle is small, the flow vector of the approaching object detection section 32, the optical flow calculating unit 11 left and right both sides calculated in the image Is extracted and the process proceeds to Step B110. If | θ | <θ ₁ , the process proceeds to the determination flow in step B80 on the assumption that the steering angle is slightly larger in either the left or right direction.

In step B80, the steering angle theta is whether or not the reference value theta ₁ or more is determined. Here, if θ ≧ θ ₁ , the steering angle is slightly larger in the right direction (ie, the vehicle 2 is turning right), the process proceeds to step B90, and the approaching object detection unit 32 calculates the optical flow. Only the flow vector of the left side image is extracted from the flow vectors of the image calculated by the unit 11, and the process proceeds to Step B110. In the case of θ <θ ₁ , θ ≦ −θ ₁ from the conditions of steps B30, B60, and B80, so that the steering angle is slightly larger in the left direction (that is, the left turn of the vehicle 2 is large). ), The approaching object detection unit 32 extracts only the flow vector of the right side image from the flow vectors of the image calculated by the optical flow calculation unit 11, and proceeds to step B110.

  In Step B110, it is determined whether or not the flow vector extracted in Step B70, B90, or B100 has a vector component toward the traveling direction side of the vehicle 2. Here, for the flow vector on the left side image of the vehicle 2, it is determined whether or not it has a vector component in the right direction on the image. On the other hand, it is determined whether or not it has a vector component in the left direction on the image. That is, here, among the moving objects recognized in step B50, the moving object approaching the vehicle 2 (approaching object) is recognized separately from the others.

Here, when a flow vector having a vector component toward the traveling direction side of the vehicle 2 is not detected, it means that there is no approaching object, so this flow ends, but the above flow vector is detected. If yes, go to Step B120.
In step B120, the output unit 13 displays on the monitor 5 that the approaching object has been recognized, and notifies the passenger by voice. Here, the notification to the passenger in the output unit 13 is switched between notification modes according to the magnitude and number of flow vectors having vector components in the traveling direction side of the vehicle 2 detected in step B110.

  Among the approaching object flow vectors detected in step B110, when the magnitude of the approaching object is greater than a preset predetermined size, or when the approaching object flow vector is greater than or equal to a preset number. Is detected, the output unit 13 is set to the “high risk level notification mode”, the area on the screen corresponding to the approaching object is highlighted, and all flow vectors due to the approaching object are displayed with arrows. The occupant is informed that the degree of danger is further displayed by voice.

  In addition, among the approaching object flow vectors detected in step B110, a flow vector whose magnitude is larger than a preset predetermined size is not detected, and a flow of approaching objects of a preset number or more is detected. When the vector is not detected, the output unit 13 is set to the “low risk level notification mode”, one of the flow vectors by the approaching object is displayed on the screen with an arrow, and the detection of the approaching object is notified to the occupant by voice. Is done.

Specifically, the following operations and effects are achieved by the control as described above.
In a T-shaped road as shown in FIG. 5, when the vehicle 2 tries to enter the main road while checking left and right, first, when the nose camera switch 6 of the vehicle 2 is turned on, the vehicle 2 is connected to the main road. When the traveling speed is made smaller than the predetermined speed V _{0 in} front, the nose view camera 3 is automatically operated, and left and right side images are displayed on the monitor 5. Thereby, the left and right side images can be automatically displayed on the monitor 5 without making the occupant aware of the switching of the image on the monitor 5. When the nose view camera 3 is activated, the ECU 10 starts calculating the optical flow in the captured image.

[When stopped]
Here, when the vehicle 2 is stopped at the position shown in FIG. 5, the position of the nose view camera 3 is fixed as shown in FIG. An approach that approaches the vehicle 2 without generating a flow vector in the image (the portion that is the background of the vehicles 21 and 22 on the image, and here indicates a non-moving object such as a road, a building, a guardrail, and the sky) Only flow vectors (black arrows in FIG. 3A) by the vehicles 21 and 22 as objects are generated. At this time, the direction of the flow vector by the vehicles 21 and 22 is such that the vehicle 21 has a vector component to the right in the left side image, that is, a vector component to the traveling direction side of the vehicle 2 in the image. Will have. On the other hand, the vehicle 22 has a vector component in the left direction in the screen on the right side, and has a vector component in the traveling direction side of the vehicle 2 in the image.
Therefore, the approaching object detection unit 12 can detect the vehicles 21 and 22 as approaching objects based on the flow vector having a vector component toward the traveling direction of the vehicle in the left and right side images.

[When moving forward]
Further, when the vehicle 2 is traveling at the low speed (traveling speed V <V ₀ ) in the position shown in FIG. 5, that is, when the occupant slowly advances the vehicle 2 while checking the left and right, As shown in FIG. 3B, the movement of the imaging position of the nose view camera 3 causes a flow vector (a white arrow in FIG. 3B) to occur in the background portion in the captured image. Become. The flow vectors of the vehicles 21 and 22 are the flow vector that will be generated when the vehicle 2 is stopped (that is, the flow vector of the black arrow shown in FIG. 3A), and the nose view camera. Therefore, a flow vector (black arrow in FIG. 3B) is generated as a sum of the flow vector of the background portion (flow vector of the white arrow) generated by the position movement of 3. In addition, the flow vector of the background portion is generated as a flow vector having a vector component in the direction opposite to the forward direction of the vehicle 2 on the image when the vehicle 2 moves forward.

  Therefore, by slowly moving the vehicle 2 forward, the magnitude and direction of the flow vector by the vehicles 21 and 22 are subject to deformation. However, when the vehicle 2 is slowing down at low speed, the vehicle 2 is not subject to significant deformation. It can have a vector component toward the traveling direction side of the vehicle 2 above. Moreover, even if the vector component toward the traveling direction side of the vehicle and the flow vector of the background portion among the flow vectors of the vehicles 21 and 22 are equal, the flow vector of the vehicles 21 and 22 is directed to the vehicle 2. Since there is also a vector that expands on the image by the approach of the vehicle, that is, a vector in the vertical direction on the image, the flow vector by the vehicles 21 and 22 is not canceled.

  At this time, the direction of the flow vector by the vehicles 21 and 22 is a vector direction having a vector component on the traveling direction side of the vehicle 2 in the image, as shown in FIG. This is a direction having a vector component in the right direction, and a direction having a vector component in the left direction in the right side region. In other words, in the left region, the flow vector has a direction in the range of 180 degrees clockwise from the vertical direction, and in the right region, the flow vector has a direction in the range of 180 degrees counterclockwise from the vertical direction.

The approaching object detection unit 12 can detect the vehicles 21 and 22 as approaching objects based on a flow vector having a vector component toward the traveling direction of the vehicle in the left and right side images. Therefore, the output unit 13 of the ECU 2 outputs to the monitor 5 that an approaching object has been detected, and can alert the passenger.
Moreover, since the output part 13 switches notification mode according to the danger level with respect to the own vehicle 2 of an approaching object, it can notify a passenger | crew effectively and can improve safety | security. be able to.

Note that when the vehicle 2 moves forward quickly, the flow vector of the background portion generated by the position movement of the nose view camera 3 increases, so the direction of the flow vector by the vehicles 21 and 22 is the progression of the vehicle 2 on the image. Although it is conceivable that there is no vector component in the direction side, in this embodiment, the operation of the nose view camera 3 is stopped when the traveling speed V of the vehicle 2 exceeds a predetermined speed V _0. Therefore, erroneous recognition of an approaching object can be prevented. Similarly, since the operation of the nose view camera 3 is stopped when the steering angle θ of the vehicle 2 is equal to or greater than the predetermined angle θ ₀ , an approaching object is not erroneously recognized by turning movement. As a result, the detection accuracy of the approaching object by the optical flow can be improved.

Even when the operation of the nose view camera 3 is stopped, the output unit 13 notifies the occupant to that effect, so that the occupant can be alerted and safety can be improved.
The setting of the magnitudes of the predetermined speed V ₀ and the predetermined angle θ ₀ is arbitrary depending on the embodiment, but when the vehicle 2 moves, as the traveling speed V and the steering angle θ increase, The flow vector of the background part becomes large. Therefore, in order to detect an object with a fast approach speed, there is no problem even if the set values of the predetermined speed V ₀ and the predetermined angle θ ₀ are increased. On the other hand, to detect an object with a slow approach speed, It is desirable that the vehicle 2 is stopped. However, when the vehicle 2 is moving, by using the difference in direction between the flow vectors of the vehicles 21 and 22 as the approaching object traveling on the general public road and the flow vector generated in the background portion ( In other words, the vehicles 21 and 22 as the approaching objects can be distinguished from the background portion by the logic as described above.

[When turning]
Further, when the vehicle 2 is turning right from the position shown in FIG. 5 at a low speed (traveling speed V <V ₀ ), that is, the occupant slowly advances the vehicle 2 while checking left and right and wants to proceed in the right direction. 3 (d), the movement of the imaging position of the nose view camera 3 causes the background portion in the captured image to also flow in the background vector (in FIG. 3 (d)). White arrow) will occur. The flow vectors of the vehicles 21 and 22 are the flow vector that will be generated when the vehicle 2 is stopped (that is, the flow vector of the black arrow shown in FIG. 3A), and the nose view camera. As a result, a flow vector (black arrow in FIG. 3D) is generated as a sum of the flow vector of the background portion (flow vector of the white arrow) generated by the position movement of 3. The flow vector of the background portion is generated as a flow vector having a vector component in the left direction of the vehicle 2 on the image when the vehicle 2 turns to the right.

Here, the approaching object detection unit 32 selects a flow vector to be referred to according to the magnitude of the steering angle of the vehicle 2. That is, when the vehicle is turning right at the steering angle θ ≧ θ ₁ , the flow vector in the left side region is extracted, and the approaching object is detected therefrom. In this case, out of the flow vectors shown in FIG. 3D, a flow vector having a vector component in the right direction is extracted from the image on the left side region, and the extracted flow vector approaches the vehicle 2. It is determined that the flow vector is based on an object (that is, a moving object having a flow vector that approaches the host vehicle 2), and the approaching object is recognized. Therefore, the approaching vehicle 21 can be accurately detected.

  Also, among the flow vectors shown in FIG. 3D, the flow vector on the screen in the right side area is not referred to. Therefore, the flow vector due to the turning of the vehicle 2 (white arrow in the right side image in FIG. 3B) is erroneously recognized as the flow vector due to the approaching vehicle (black arrow in the right side image in FIG. 3B). It is possible to prevent this, and the calculation processing for detecting an approaching vehicle can be halved.

Further, the same even if you are left pivot steering angle theta ≦ - [theta] _1, the approaching object detection section 32 extracts the flow vector in the left side area, to detect the approaching object from. In this case, a flow vector having a vector component in the left direction on the image on the right side region is extracted, and the extracted flow vector is determined to be a flow vector by an approaching object approaching the vehicle 2, and an approaching object is recognized. . Therefore, the approaching vehicle 21 can be accurately detected. Moreover, since the flow vector on the screen in the left side area is not referred to, erroneous recognition can be prevented and the amount of calculation can be halved.

As described above, according to the nose view monitor device of the present invention as the second embodiment, only the optical flow vector existing in the direction in which the approaching object to the host vehicle can be reliably detected when the vehicle 2 is turning. By referring to, erroneous recognition of an approaching object can be reduced, and the detection rate of the approaching object can be improved. Even when the vehicle 2 is stopped, traveling at a low speed (V <V ₀ ), or when the steering angle is smaller than a predetermined angle (θ <θ ₀ ), the vehicle side The approaching object in the direction can be easily and reliably recognized, the background image can be prevented from being erroneously recognized as the approaching object, and the detection accuracy of the approaching object can be improved. Further, among the moving objects recognized based on the optical flow, the configuration for recognizing an approaching object that is dangerous to the host vehicle 2 is simple, and the amount of calculation of the ECU 10 relating to the recognition of the approaching object is reduced. be able to. Moreover, safety can be improved by notification according to the degree of danger.

Although the first and second embodiments of the present invention have been described above, the present invention is not limited to such embodiments, and various modifications can be made without departing from the spirit of the present invention.
For example, in the above-described first and second embodiments, the nose view cameras 3 as the imaging means are provided at the left and right end portions of the nose portion of the vehicle 2, but only one of them is provided. Alternatively, the left and right sides of the vehicle may be simultaneously imaged using a wide-angle camera having a wide imaging area.

In addition, the imaging area of the nose view camera 3 may be configured to be adjusted in the horizontal direction according to the angle formed by the traveling direction of the vehicle 2 and the direction of the main road that the vehicle 2 wants to enter. You may comprise so that it may adjust to a perpendicular direction according to inclination.
Further, the predetermined speed V ₀ and the predetermined angle θ ₀ in the first and second embodiments described above are arbitrarily set according to the magnitude of the flow vector of the moving object to be recognized. For example, even if the traveling speed V of the vehicle 2 is higher than a predetermined speed V ₀ , the movement moves with a flow vector larger than the flow vector of the background portion generated by the position movement of the vehicle 2 (nose view camera 3) An object can be recognized by the above-described configuration.

The notification method by the output unit 13 in the first and second embodiments described above is arbitrary. For example, when an approaching object is highlighted in the image on the monitor 5, the approaching object on the image is enlarged and displayed. Alternatively, the occupant may be notified by operating the braking device of the vehicle 2.
Moreover, you may comprise so that the some alerting | reporting mode according to a danger level may be set in multistep. For example, the notification mode for switching based on the magnitude and number of the flow vector of the approaching object detected by the approaching object detection unit 12 is further increased, and the possibility that the approaching object is dangerous to the host vehicle 2 is determined in multiple stages. As a result, more detailed notification can be performed, safety can be improved, and reliability of the device itself relating to risk determination can be improved.

It is a typical lineblock diagram showing vehicles provided with a nose view monitor device as a 1st embodiment of the present invention. It is a schematic diagram of the display screen content as a monitor display example by the nose view monitor device as the first and second embodiments of the present invention. It is a schematic diagram for demonstrating the arithmetic processing in the approaching object detection means of the nose view monitor apparatus as 1st, 2nd embodiment of this invention, and the left half is the nose view and flow vector of the left side of a vehicle in all The right half shows a nose view and a flow vector on the right side of the vehicle, (a) is a flow vector generated when the vehicle is stationary, and (b) is a flow generated when the vehicle is moving forward. (C) is a flow vector direction range for extracting only an approaching object, (d) is a flow vector generated when the vehicle is turning right, and (e) is when the vehicle is turning right. A turning flow vector to be set, (f) indicates a correction flow vector calculated when the vehicle is turning right. It is a flowchart for demonstrating the control in the nose view monitor apparatus as 1st Embodiment of this invention. It is a typical top view which shows the imaging area of the nose view camera in the nose view monitor apparatus as 1st, 2nd embodiment of this invention. It is a typical block diagram which shows the vehicle provided with the nose view monitor apparatus as 2nd Embodiment of this invention. It is a flowchart for demonstrating the control in the nose view monitor apparatus as 2nd Embodiment of this invention.

Explanation of symbols

1,21 Nose view monitor device 2 Vehicle (own vehicle)
3 Nose view camera (imaging means)
4 Vehicle speed sensor (turning state corresponding value detection means)
5 Monitor (notification means)
6 Nose camera switch 7 Steering angle sensor (turning direction detection means / turning state corresponding value detection means)
10 Electronic control unit (ECU)
11 Optical flow calculation unit (optical flow vector calculation means)
11A Left side region optical flow calculation unit 11B Right side region optical flow calculation unit 12, 32 Approaching object detection unit (approaching object detection means)
13 output unit 14 turning flow vector setting unit (turning optical flow vector setting means)
15 Flow vector correction unit (optical flow vector correction means)

Claims (8)

An imaging means that is provided in front of the nose of the vehicle and captures an image of the side of the vehicle;
Optical flow vector calculation means for calculating an optical flow vector based on the image;
A turning direction detecting means for detecting a turning direction of the vehicle;
A turning state corresponding value detecting means for detecting a parameter corresponding to the turning state of the vehicle;
A turning optical flow vector setting means for setting a turning optical flow vector generated by turning of the vehicle in the vehicle side image based on the turning direction and the parameter;
Optical flow vector correction means for subtracting and correcting the turning optical flow vector set by the turning optical flow vector setting means from the optical flow vector calculated by the optical flow vector calculation means;
Approaching object detection means for detecting an approaching object based on the optical flow vector corrected by the optical flow vector correction means;
A nose view monitor device comprising: an informing means for displaying the image and informing the detection of the approaching object. The turning optical flow vector setting means sets the direction of the turning optical flow vector in the horizontal direction and outside the turning direction detected by the turning direction detection means, and the magnitude of the turning optical flow vector. 2. The nose view monitor device according to claim 1, wherein is set in correspondence with the value of the parameter detected by the turning state corresponding value detecting means. The approaching object detection means detects an approaching object based on an optical flow vector having a vector component in the traveling direction of the vehicle in the image among the optical flow vectors corrected by the optical flow vector correction means. The nose view monitor device according to claim 1, wherein the nose view monitor device is characterized. The nose according to any one of claims 1 to 3, wherein the turning state corresponding value detecting means detects an angular velocity at the time of turning of the vehicle as a parameter corresponding to the turning state of the vehicle. View monitor device. The said turning state corresponding value detection means detects the travel speed and steering angle of this vehicle as a parameter corresponding to the turning state of this vehicle, The any one of Claims 1-3 characterized by the above-mentioned. Nose view monitor device. The notification unit switches a plurality of notification modes according to the magnitude or number of optical flow vectors having a vector component in the traveling direction of the vehicle among the optical flow vectors corrected by the optical flow vector correction unit, The nose view monitor device according to any one of claims 1 to 5, wherein the image is displayed and the detection of the approaching object is notified. An imaging means that is provided in front of the nose of the vehicle and captures an image of the side of the vehicle;
Optical flow vector calculation means for calculating an optical flow vector based on the image;
A turning direction detecting means for detecting a turning direction of the vehicle;
Of the optical flow vectors calculated by the optical flow vector calculation means when the vehicle is turning, the optical flow vectors are present in a lateral region of the vehicle outside the turn detected by the turning direction detection means, and in the image An approaching object is detected based on an optical flow vector having a vector component in the traveling direction of the vehicle, and when the approaching object is detected, a side of the vehicle turning inside direction detected by the turning direction detecting means is detected. An approaching object detection means that does not refer to an optical flow vector existing in the region;
A nose view monitor device comprising: an informing means for displaying the image and informing the detection of the approaching object. The notification means switches a plurality of notification modes according to the magnitude or number of optical flow vectors having a vector component in the traveling direction of the vehicle, displays the image and notifies the detection of the approaching object. The nose view monitor device according to claim 7.

Images