JP2009231938A - Surrounding monitoring device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a surrounding monitoring device for a vehicle, enabling a user to easily comprehend the moving state of a moving object in an image of a blind area. <P>SOLUTION: The surrounding monitoring device for vehicle is used to monitor a blind area in an area which extends on right and left on the front viewed from a driver. The monitoring device has: real image pickup means 2, 4, 6 provided on the front of a vehicle so as to pick up real images of a blind area; a moving object detecting means 32 for extracting a moving object from the real image picked up by the real image pickup means and detecting the moving state of the moving object; an imaginary image generating means 38 for generating an imaginary image of the moving object detected by the moving object detecting means; an imaginary image defining means 38 for defining the position and size showing the imaginary image of the moving object generated by the imaginary image generating means so as to show the moving state of the moving object; and a display means 26 for displaying the imaginary image of the moving object so as to be overlapped on the actual image in the position and size defined by the imaginary image defining means. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle surrounding monitoring device, and more particularly to a vehicle surrounding monitoring device that monitors a blind spot area of a driver.

  2. Description of the Related Art Conventionally, devices that capture and display a blind spot image in the left-right direction on the front side of a vehicle are known (Patent Documents 1 and 2).

JP 2007-172462 A JP 11-338074 A

  The apparatus as described above merely displays a blind spot area in the left-right direction as it is on a monitor, for example. The driver looks at such a display to determine the status of the blind spot area.For example, when there is a moving body such as a vehicle approaching from the left and right, it is difficult to distinguish it from a stationary object. It was difficult to understand how fast the vehicle was at a certain distance.

  The present invention has been made to solve the above-described problems of the prior art, and it is an object of the present invention to provide a vehicle surrounding monitoring device that makes it easy to grasp the moving state of a moving object in an image of a blind spot area. .

  In order to achieve the above object, according to the present invention, there is provided a vehicle surrounding monitoring device for monitoring a blind spot area in an area that spreads left and right in front of a driver so as to capture a real image of the blind spot area. Real image capturing means provided in the front part of the vehicle, moving body detecting means for extracting a moving body from the real image captured by the real image capturing means, and detecting the moving state of the moving body, and the movement Virtual image defining means for defining a display mode of the predetermined virtual image in order to display a predetermined virtual image representing the moving state of the moving object detected by the body detecting means, and the virtual image of the moving object as a virtual image Display means for superimposing and displaying the actual image in a display mode defined by the defining means.

  In the present invention configured as described above, the virtual image of the moving object is displayed so as to be superimposed on the real image in a display mode that represents the moving state, so that the driver can easily recognize the moving state of the moving object. Therefore, it is easy to distinguish the moving object from the stationary object and to easily grasp the moving state of the moving body in the image of the blind spot area.

In the present invention, preferably, the blind spot area of the driver is a blind spot area around the intersection of the intersection on the front side of the vehicle, and the real image capturing means is provided at the front end of the vehicle, and the vehicle is in the intersection. When approaching, a real image around the intersection of the intersection that can be imaged from the front end of the vehicle is captured.
In the present invention configured as described above, it is possible to easily grasp the movement state of the left and right regions around the intersection where collision accidents frequently occur.

In the present invention, it is preferable that the virtual image defining means displays the moving object's virtual image at the estimated arrival position after a predetermined time on the extension line of the moving object as the moving state of the moving object. A display mode for displaying a virtual image of a moving object is defined.
In the present invention configured as described above, the virtual image of the moving object is defined by the virtual image defining means on the estimated arrival position after a predetermined time on the extension line of the moving object, and this is superimposed on the actual image by the display means. Therefore, the position of the moving body after a predetermined time can be grasped intuitively, and the driver can more easily determine, for example, whether or not the host vehicle can be started.

In the present invention, it is preferable that the second display means further displays a numerical display of a predicted arrival time from the current position of the displayed moving body based on a predetermined time in accordance with the virtual image of the moving body. Have.
In the present invention configured as described above, the position of the moving body after a predetermined time can be grasped more intuitively by the time display using numerical values.

In the present invention, it is preferable that the virtual image defining means displays the moving image of the moving object as an afterimage at regular intervals at a past position where the moving object has moved. A display mode for displaying a virtual image of a moving object is defined.
In the present invention configured as described above, the virtual image of the moving body is displayed by the virtual image defining means so that the virtual image of the moving body is displayed as an afterimage at equal time intervals at the past position where the moving body has moved. The display mode to be displayed is defined, and this is displayed superimposed on the actual image by the display means, so that it is possible to intuitively grasp the moving state of the moving body up to the present, and for the driver, for example, start the own vehicle It is possible to make it easier to determine whether or not to allow it.

In the present invention, it is preferable that vector display is further performed in accordance with afterimages at equal time intervals.
In the present invention configured as described above, it is possible to more intuitively understand the moving state of the moving body up to the present by the vector display in accordance with the afterimages at equal time intervals.

In the present invention, it is preferable that the display unit enlarges the virtual image of the moving object when the distance from the own vehicle of the moving object is a predetermined distance or more than when the distance is smaller than the predetermined distance. indicate.
In the present invention configured as described above, it is possible to easily grasp the moving state of the moving body at a long distance.

In the present invention, it is preferable that the display means displays the virtual image of the moving object when the distance of the moving object from the own vehicle is within a predetermined distance range or the arrival time of the moving object at the own vehicle. Is performed within a predetermined time range.
In the present invention configured as described above, for example, when there is a moving body within a predetermined distance that makes it difficult to determine whether to start or stop the host vehicle, the driver surely moves the moving body. The state can be grasped.

In the present invention, it is preferable that the display unit displays the virtual image of the moving body so as to be distinguishable from the substance and the actual state of the moving body.
In the present invention configured as described above, it is easy for the driver to visually recognize the moving object.

In the present invention, it is preferable that when a plurality of moving bodies are detected by the moving body detecting means, the virtual image defining means is the moving body having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle. The display mode of the virtual image is defined, and the display means displays the defined virtual image superimposed on the actual image.
In the present invention configured as described above, when there are a plurality of moving objects, a virtual image of the moving object having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle is generated. It is possible to reliably grasp the moving state of a moving body that may collide with the host vehicle.

In the present invention, it is preferable that a plurality of moving bodies are detected by the moving body detecting means, and when the display means displays all the virtual images of the plurality of moving bodies, the plurality of moving bodies and their virtual images are displayed in an overlapping manner. The virtual image defining means defines the display mode of the virtual image of the moving body having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle, and the display means defines the defined virtual image. Is superimposed on the actual image and displayed.
In the present invention configured as described above, when there are a plurality of moving bodies and all the virtual images of the moving bodies are displayed, the plurality of moving bodies and their virtual images are displayed in an overlapping manner. Instead of generating all the virtual images of the moving object, a virtual image of the moving object having the shortest distance to the host vehicle or the shortest arrival time to the host vehicle is generated. Without being confused with the virtual images of the moving bodies, it is possible to grasp the moving state of the moving body that may collide with the host vehicle more reliably.

In the present invention, it is preferable that the virtual image defining means displays a virtual road surface image indicating the moving state of the moving body on the road surface in front of the moving body in order to represent the moving state of the moving body. Stipulates the display mode.
In the present invention configured as described above, the virtual road surface image is virtually displayed on the road surface on which the moving body is traveling so as to indicate the moving state of the moving body. And the movement state can be grasped.

In the present invention, it is preferable that the virtual image defining means corresponds to the arrival time of the own vehicle and the arrival time of the moving body up to the intersection of the intersection where the own vehicle and the moving body are directed, or the own vehicle and the moving body. The display mode is defined so that a predetermined virtual road surface image is displayed on the road surface ahead of the moving body in the traveling direction according to the passing time of the host vehicle and the arrival time of the moving body at the intersection of the crossing road.
In the present invention configured as described above, predetermined on the road surface on the front side in the traveling direction of the moving body according to the arrival time of the own vehicle to the intersection of the intersection where the own vehicle and the moving body go and the arrival time of the moving body Since the virtual road surface display is superimposed and displayed, the direction and moving state in which the moving body exists can be grasped more reliably and accurately.

In the present invention, it is preferable that the virtual image defining unit changes the virtual road surface display according to the certainty of the calculation of the arrival time of the host vehicle and the arrival time of the moving body. The display mode is defined.
In the present invention configured as described above, the virtual road surface display is changed and displayed in accordance with the certainty of calculation of the arrival time of the own vehicle to the intersection and the arrival time of the moving object, and thus more reliably and accurately. It is possible to grasp the direction and moving state of the moving body. As the certainty factor, for example, it can be determined that the certainty factor is high in the order of autonomous sensor information, inter-vehicle communication information, road-to-vehicle communication information, and information center distribution information.

In the present invention, it is preferable that when a plurality of moving bodies are detected by the moving body detecting means, the virtual image defining means is the moving body having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle. The display mode of the virtual road surface image is defined for the image, and the display means displays only the defined virtual road surface image as a virtual image superimposed on the real image.
In the present invention configured as described above, when there are a plurality of moving objects, a virtual image of the moving object having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle is generated. It is possible to reliably grasp the moving state of a moving body that may collide with the host vehicle.

In the present invention, it is preferable that a plurality of moving bodies are detected by the moving body detecting means, and when the display means displays all the virtual images of the plurality of moving bodies, the plurality of moving bodies and their virtual images are displayed in an overlapping manner. The virtual image defining means defines the display mode of the virtual road surface image for the moving body having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle, and the display means Only the virtual road surface image is superimposed and displayed on the real image as a virtual image.
In the present invention configured as described above, when there are a plurality of moving bodies and all the virtual images of the moving bodies are displayed, the plurality of moving bodies and their virtual images are displayed in an overlapping manner. Instead of generating all the virtual images of the moving object, a virtual image of the moving object having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle is generated, so a plurality of virtual images are displayed. Thus, the driver can be prevented from confusing those virtual images, and the moving state of the moving body that may collide with the host vehicle can be grasped more reliably.

In the present invention, it is preferable that the display unit displays the virtual image so as to be distinguishable from the real image of the moving object.
In the present invention configured as described above, the driver can recognize the virtual image as a virtual image, and the real image and the virtual image are not confused.

In the present invention, it is preferable that the display unit displays a virtual image or a virtual road surface image of the moving body superimposed on the current background of the moving body and the actual image of the moving body.
In the present invention configured as described above, the driver can see the latest video at the present time as a real image, and a virtual image is displayed on the real image. The movement state of can be grasped.

  According to the vehicle surrounding monitoring apparatus of the present invention, it is easy to grasp the moving state of the moving body in the image of the blind spot area.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
First, with reference to FIGS. 1 and 2, a basic configuration of a vehicle surrounding monitoring apparatus according to each embodiment common to the first to third embodiments of the present invention will be described. FIG. 1 is an overall configuration diagram of a vehicle to which a vehicle surrounding monitoring apparatus according to first to third embodiments of the present invention is applied, and FIG. 2 is a vehicle surrounding according to first to third embodiments of the present invention. It is a figure which shows an example (a) of the imaging range by the camera of a monitoring apparatus, and an example (b) of the imaged image.

  As shown in FIG. 1, the vehicle 1 has a left monitoring camera 2, a front monitoring camera 4, and a right monitoring camera 6, and these cameras are attached to the front end of the vehicle. These cameras are CCD cameras using a wide-angle lens, and for example, can capture a range as shown in FIG. 2A and obtain an image as shown in FIG.

  The vehicle 1 also has a vehicle speed sensor 10, a GPS device 12, a navigation device 13, an image display monitor 14, and a speaker 15. The monitor 14 is arranged at a position where the driver of the driver's seat 16 can see. Further, the vehicle 1 has an ECU 18. The ECU 18 receives signals from the cameras 2, 4 and 6, the vehicle speed sensor 10, and the GPS device 12, and displays a predetermined image on the monitor 14. A predetermined sound is emitted from the speaker 15.

  Furthermore, the vehicle 1 has a laser radar 7 for detecting a distance from another vehicle. Instead of the laser radar 7, the distance from the other vehicle may be grasped by vehicle-to-vehicle communication, or the distance from the other vehicle may be grasped by road-to-vehicle communication.

Next, with reference to FIG. 3, driving behavior when a driver enters an intersection will be described. FIG. 3 is a diagram for explaining driving behavior when a driver enters an intersection.
First, stop before the intersection. After that, the vehicle proceeds slowly to the nasal position, proceeds to the camera position (see FIG. 2A), and proceeds to the cue position (see FIG. 4). Here, the nasal position, the camera position and the cue position will be described. FIG. 4 is an overhead view of the road and the vehicle for explaining the cueing position of the own vehicle.

  First, the nose out position is a position that informs other vehicles and pedestrians of the presence of the host vehicle. As shown in FIG. 2 (a), the camera exit position is a state in which the front end of the vehicle enters the intersection, and the driver cannot see the intersection, but the camera 2 attached to the front end of the vehicle, 4 and 6 are positions where the intersection can be seen. As shown in Fig. 4, the cueing position is the position when the driver's seat where the driver sits enters the intersection, and the driver can directly see inside the intersection with his own eyes. is there.

  In the vehicle surrounding monitoring apparatus according to the embodiment of the present invention, a virtual image to be described later is displayed between the temporary stop and the cueing position. When the cueing position is reached, the virtual image blinks, and after passing through the cueing position, the display of the virtual image is stopped.

Next, referring to FIG. 5, a control flow for determining whether or not to perform the virtual image display processing of the vehicle surrounding monitoring apparatus according to each embodiment common to the first to third embodiments of the present invention will be described. FIG. 5 is a flowchart showing a process for determining whether or not to display a distance grasping virtual image of the vehicle surrounding monitoring apparatus according to the first to third embodiments of the present invention.
As shown in FIG. 5, in S <b> 1, the virtual image processing execution determination unit 52 (see FIGS. 6, 9, and 13) determines that the subject vehicle is near the intersection based on the signal from the GPS 12 and the signal from the navigation device 13. It is determined whether or not. Next, in S <b> 2, the virtual image processing execution determination unit 52 (see FIGS. 6, 9, and 13) determines whether the vehicle speed is lower than V <b> 1 based on the signal from the vehicle speed sensor 10. The vehicle speed V1 is a value that allows the driver to determine that the vehicle is safe even when the vehicle approaches the intersection and decelerates and the driver looks at the monitor 14.

  If the vehicle speed is less than V1 and the vehicle is decelerated, the process proceeds to S3, where an actual image is first displayed based on the image signal from the image correction unit 24. Next, proceeding to S4, the virtual image processing execution determination unit 52 (see FIGS. 6, 9, and 13) determines whether or not the vehicle speed is lower than V2. The vehicle speed V2 is a value at which it can be determined that the host vehicle is almost stopped or stopped. When the vehicle speed is lower than V2, it is determined that the host vehicle is temporarily stopped, and the process proceeds to a distance grasping virtual image display process in S5. If it is determined that the vehicle speed is not less than the intersection in S1 and the vehicle speed is V1 or more in S2 and the vehicle speed is V2 or more in S4, the process does not proceed to the distance grasping virtual image display process in S5.

  Next, the configuration of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 6 is a block diagram showing the configuration of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention.

  In the first embodiment, a virtual vehicle is displayed on a real image of a moving body that actually exists (such as another vehicle that is approaching an intersection) and the position of the moving body after a predetermined time. Thus, the driver recognizes the moving state of the moving body. Thereby, for example, it is possible to help the driver determine whether or not to start the host vehicle.

  As shown in FIG. 6, image signals from the cameras 2, 4, 6 configured as the periphery monitoring sensor 20 are input to the image acquisition unit 22 in the ECU 18. Since the input image signal is a wide-angle image and is distorted, the image correction unit 24 corrects the image. The image signal from the image correction unit 24 is input to the composite image creation unit 26 as a real image.

  The moving body detection unit 30 detects whether or not a moving body such as another vehicle, a motorcycle, or a bicycle is present in the input image from the image correction unit 24. When the moving body does not exist, the composite image creation unit 26 does not display a virtual image to be described later, so a signal instructing the virtual image is sent to the composite image creation unit 26, and when the mobile body exists, the composite image creation unit Since a virtual image, which will be described later, is displayed at 26, a signal instructing that a moving body is present is sent to the composite image creating unit 26. The composite image created by the composite image creation unit 26 is displayed on the monitor 14.

  Further, the moving body moving state detection unit 32 receives a signal indicating that the moving body exists from the moving body detection unit 30, and the distance from the own vehicle to the moving body approaching the own vehicle by the laser radar 7. The speed and the traveling direction are detected, and the arrival time to the host vehicle is calculated. Instead of the laser radar 7, the distance, speed, and traveling direction of the moving body from the own vehicle are detected by inter-vehicle communication or road-to-vehicle communication, and the arrival time to the own vehicle is calculated. good.

The moving position estimation unit 34 after a predetermined time of the moving body estimates the position of the moving body after a predetermined time (in this embodiment, every 10 seconds).
The virtual image creating unit 38 creates a virtual image so that the moving body is displayed at the position estimated by the moving position estimating unit 34 after a predetermined time of the moving body. The created virtual image is superimposed on the actual image obtained by each camera 2, 4, 6 by the composite image creation unit 26 and displayed on the monitor 14.

  Next, a cue position determination unit 50 exists in the ECU 18. The cueing position determination unit 50 determines that the vehicle has cueed in the intersection (see FIG. 4). If the cue has been performed, the driver can already see the intersection, so a predetermined signal is sent to stop the virtual image creation in the virtual image creation unit 38 so that the virtual display is not performed on the monitor 14. To do.

  Further, the ECU 18 includes a virtual image processing execution determination unit 52. The virtual image processing execution determination unit 52 determines whether the vehicle is located near the intersection based on the map data obtained from the navigation device 13, the position data obtained from the GPS device 12, and the vehicle speed data obtained from the vehicle speed sensor 10. And whether or not it is temporarily stopped. As will be described later, when the vehicle is in the vicinity of an intersection and is temporarily stopped, a predetermined signal is sent to the composite image creating unit 26 in order to display a virtual image.

  Next, with reference to FIGS. 7 and 8, the control contents of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention will be described. FIG. 7 is a flowchart showing display processing of a distance grasping virtual image of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention, and FIG. 8 is a diagram of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention. It is a figure which shows an example (b) of the imaging | photography range by a camera, an example (a) of a moving body, and the image | video by which the virtual image was superimposed on the real image. In FIG. 7, S indicates each step.

  As shown in FIG. 7, first, in S11, the image acquisition unit 22 (see FIG. 6) acquires images of the left monitoring camera 2, the front monitoring camera 4, and the right monitoring camera 6, for example, at an intersection, In S12, the image correction unit 24 corrects the distortion of the acquired wide-angle image.

  Next, in S13, the moving object detection unit 30 detects the moving object from the actual images acquired in S11 and S12. The detection of the moving body in S13 is performed by optical flow processing. In addition to the optical flow, the moving object may be detected by background difference, inter-frame difference processing, laser, millimeter wave, or the like. When the moving object is not detected, the process proceeds to S14, and the actual image acquired in S11 and S12 input to the composite image creating unit 26 (see FIG. 6) is displayed as it is, for example, as being able to proceed safely on the intersection. To do. That is, display of a virtual image is prohibited.

  When the moving body is detected in S13, the process proceeds to S15, and the moving body moving state detection unit 32 detects the distance, speed, and traveling direction of the moving body from the own vehicle, and the moving body reaches the own vehicle. Calculate time. When there are a plurality of moving objects, basically, the distance from the own vehicle to the moving object that has the shortest distance from the own vehicle, or the moving object that has the shortest arrival time to the own vehicle. In addition, the traveling direction is detected, and the arrival time of the moving body to the host vehicle is calculated. On the other hand, as will be described later, if virtual images are displayed for a plurality of moving objects, but they do not overlap each other, the distance and speed of the moving object from the host vehicle with respect to the following moving object (such as a following vehicle) In addition, the traveling direction is detected, and the arrival time of the moving body to the host vehicle is calculated.

  Next, in S16, it is determined whether the distance of the moving body from the own vehicle is within a predetermined distance or whether the arrival time of the moving body at the own vehicle is within a predetermined time. The predetermined distance is, for example, 10 to 30 m, and the predetermined time is, for example, 2 to 3.5 seconds.

  If the distance of the moving body from the host vehicle is not within the predetermined distance, or if the arrival time of the moving body to the host vehicle is not within the predetermined time, the virtual display is unnecessary and the process proceeds to S14, and the actual image is displayed. Is displayed as it is. For example, when the moving object is so close that the distance of the moving object from the own vehicle is, for example, 10 m or less, or when the moving object reaches the own vehicle within two seconds, the virtual display Since the moving state of the moving body can be understood well without performing virtual display, virtual display is not performed.

  In S16, when the distance of the moving body from the host vehicle is within the predetermined distance, or when the arrival time of the moving body to the host vehicle is within the predetermined time, the process proceeds to S17. In S17, the moving position estimation unit 34 after a predetermined time of the moving body estimates the position of the moving body after a predetermined time (in this embodiment, every 10 seconds). When there are a plurality of moving objects, the position of the moving object after a predetermined time is estimated for a moving object having the shortest distance from the own vehicle or a moving object having the shortest arrival time to the own vehicle.

Next, in S18, the time from the current position of the moving body to the position where the virtual display of the moving body is displayed is defined. That is, for virtual display, it is defined how many seconds later and every second (predetermined time) the state of the moving body is displayed. In addition to this provision, it may be prescribed how much distance (predetermined distance) the state of the moving body is displayed.
Next, in S19, the position and size to be displayed on the extension line in the traveling direction of the moving body are defined as the virtual moving state of the moving body in the predetermined time (how many seconds and every second) defined in S18. To do.

Next, in S20, as shown in FIG. 8 (b), the predetermined time (after several seconds) defined in S18 is displayed superimposed on the real image, and the mobile object defined in S19 is displayed. The virtual displays V11 and V12 of the moving body according to the position and size displayed on the extension line in the traveling direction are superimposed on the actual image and displayed on the monitor 14. In FIG. 8 (b), the vehicle (moving body) C11, C12 existing at the current position and the road surface condition currently visible are displayed on the monitor 14 as a real image.
In S20, the virtual image of the moving object is always displayed so as to be superimposed on the background and the moving object at the current time (latest) so that the driver can see the latest video at the current time as the actual image.

As shown in FIG. 8B, when there are a plurality of mobile bodies C11 and C12, in this embodiment, the mobile body (C11) present at the shortest distance from the own vehicle or the arrival time to the own vehicle is the shortest. Virtual displays V11, V12, T10, T11, and T12 are displayed only for the moving object (C11). In FIG. 8B, when the virtual display is performed on the moving body C12 and the moving body C11 itself does not overlap with the virtual display, the virtual display may also be performed on the moving body C12. . In other words, when the virtual display is performed on the moving body C12, when the moving body C11 itself and the virtual display overlap with each other, the display of the virtual image of the following vehicle is prohibited.
In the virtual display (V11, V12), it is preferable to display only the outline, display a transparent light color, or display blinking so as to be distinguished from the actual image. The virtual display (V11, V12) is displayed at equal time intervals.

  Next, it progresses to S21 and it is determined by the cue position determination part (refer FIG. 6) 50 whether the own vehicle passed the cue position (refer FIG. 4). There are several methods for determining the cueing position. For example, a method of determining from the position of the own vehicle obtained by the GPS device 12 and the map data of the navigation device 13, the position of the stop line obtained by the cameras 2, 4, 6 and the moving distance by a moving distance sensor (not shown) From the stop line, when the left and right line-of-sight conditions improve from the images from the left and right cameras 2 and 6 (when there is nothing to block the line-of-sight at a short distance) And a method of determining from the position of the stop line obtained by the cameras 2, 4 and 6 and an image around the intersection.

  When the host vehicle is cueing, the process proceeds to S22, where the composite image creation unit 26 (see FIG. 6) causes the virtual image displayed in S20 to blink, and then stops displaying the virtual image. The flashing of the virtual image allows the driver to know that the driver can view the image, not the display on the monitor 14, or that the image is different from what is actually visible. On the other hand, when cueing is not being performed, the display of the virtual image and the real image in S20 is continued.

Next, the function and effect of the first embodiment of the present invention will be described.
According to the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention, it is possible to monitor a blind spot area in an area extending left and right in front of the driver. In the present embodiment, cameras 2, 4, and 6 are provided in the front part of the vehicle so as to capture a real image of the blind spot area, and the moving object detection unit 30 captures the actual images captured by these cameras 2, 4, and 6. A moving object is extracted from the image. Further, the moving state of the moving body is detected by the moving body moving state detecting unit 32, and the position and size of the moving body displaying the virtual image of the moving body by the moving position estimating unit 34 after a predetermined time of the moving body is moved. It is defined to represent the state. Then, a virtual image of the moving object is generated by the virtual image generating unit 38, and the position and size defined by the moving position estimating unit 34 after a predetermined time of the moving object are generated by the composite image generating unit 26. The image is superimposed on the actual image and displayed on the monitor 14.

  Accordingly, since the virtual image of the moving object is displayed superimposed on the real image at a position and size that represents the moving state, it becomes easier for the driver to recognize the moving state of the moving object and distinguish it from a stationary object. It is easy to grasp the moving state of the moving body in the image of the blind spot area.

  Next, according to the first embodiment, the blind spot area of the driver is a blind spot area around the intersection of the intersection on the front side of the vehicle, and the cameras 2, 4, and 6 are provided at the front end of the vehicle. When the vehicle approaches the intersection, it captures the actual image around the intersection of the intersection that can be imaged from the front end of the vehicle, so it is easy to grasp the movement status of the left and right areas around the intersection where collisions frequently occur I can do it.

  Next, according to the first embodiment, the moving position estimation unit 34 after a predetermined time of the moving body moves to the estimated arrival position after a predetermined time on the traveling direction extension line of the moving body as the moving state of the moving body. The position and size for displaying the virtual image of the moving body are defined so that the virtual image of the body is displayed, and this is displayed by being superimposed on the real image by the display means. This makes it easier for the driver to determine, for example, whether the host vehicle is allowed to start.

  Next, according to the first embodiment, a numerical display of the predicted arrival time from the current position of the displayed moving body based on a predetermined time is displayed in accordance with the virtual image of the moving body. By the time display with various numerical values, the moving state of the moving body can be grasped more intuitively.

Next, according to the first embodiment, the composite image creating unit 26 displays the virtual image of the moving object when the distance from the own vehicle of the moving object is within a predetermined distance or to the own vehicle of the moving object. This is done when the arrival time of the vehicle is within a predetermined time. For example, when there is a moving object within a predetermined distance that makes it difficult to determine whether to start or stop the host vehicle, the driver reliably moves the vehicle. The movement state of the body can be grasped.
Next, according to the first embodiment, the composite image creation unit 26 displays the virtual image of the moving body so that it can be distinguished from the actual vehicle body and actual state of the moving body, so that the driver can visually recognize the moving body. Easy.

  Next, according to the first embodiment, when a plurality of moving bodies are detected by the moving body detection unit 30, the virtual image generation unit 38 has the shortest distance to the own vehicle or the arrival time to the own vehicle. Since the shortest virtual image of the moving body is generated, the moving state of the moving body that may possibly collide with the host vehicle can be grasped more reliably.

  Next, according to the first embodiment, when all the virtual images of a plurality of moving objects are displayed by the composite image creating unit 26, when the plurality of moving objects and their virtual images are displayed in an overlapping manner, the virtual image generating unit 38 does not generate all the virtual images of these moving objects, but generates a virtual image of the moving object having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle. Thus, the moving state of the moving body that may collide with the host vehicle can be grasped more reliably without confusing the moving state of each moving body.

  Next, according to the first embodiment, the composite image creating unit 26 can display the virtual display so that it can be distinguished from the real image of the moving object, for example, only the contour line, the transparent light color display, or the blinking display. Therefore, the driver can recognize the virtual image as a virtual image, and the real image and the virtual image are not confused.

  Next, according to the first embodiment, the composite image creation unit 26 displays a virtual display or a virtual road surface display of the moving object superimposed on the current background of the moving object and the current moving object. As a real image, the latest video at the present time can be seen, and a virtual image is displayed for it, so that the moving state of the moving object can be grasped more reliably along with the surrounding situation.

  Next, the configuration of the vehicle surrounding monitoring apparatus according to the second embodiment of the present invention will be described with reference to FIG. FIG. 9 is a block diagram showing a configuration of the vehicle surrounding monitoring apparatus according to the second embodiment of the present invention.

  In the second embodiment, any position before a predetermined time until the moving body comes to the current position by superimposing it on a real image of a moving body that actually exists (such as another vehicle approaching the intersection). The driver recognizes the moving state of the moving body by virtually displaying the after-image as an afterimage. Thereby, for example, it is possible to help the driver determine whether or not to start the host vehicle.

  As shown in FIG. 9, image signals from the cameras 2, 4, 6 configured as the periphery monitoring sensor 20 are input to the image acquisition unit 22 in the ECU 18. Since the input image signal is a wide-angle image and is distorted, the image correction unit 24 corrects the image. The image signal from the image correction unit 24 is input to the composite image creation unit 26 as a real image.

  The moving body detection unit 30 detects whether or not a moving body such as another vehicle, a motorcycle, or a bicycle is present in the input image from the image correction unit 24. When the moving body does not exist, the composite image creation unit 26 does not display a virtual image to be described later, so a signal instructing the virtual image is sent to the composite image creation unit 26, and when the mobile body exists, the composite image creation unit Since a virtual image, which will be described later, is displayed at 26, a signal instructing that a moving body is present is sent to the composite image creating unit 26.

  Further, the moving body moving state detection unit 32 receives a signal indicating that the moving body exists from the moving body detection unit 30, and the distance from the own vehicle to the moving body approaching the own vehicle by the laser radar 7. The speed and the traveling direction are detected, and the arrival time to the host vehicle is calculated. Instead of the laser radar 7, the distance, speed, and traveling direction of the moving body from the own vehicle are detected by inter-vehicle communication or road-to-vehicle communication, and the arrival time to the own vehicle is calculated. good.

The moving body information accumulating unit 33 obtains and accumulates the position information of the moving body detected by the moving body detecting unit 30 from the moving body moving state detecting unit 32. In the moving object afterimage image defining unit 35, the afterimage image of the moving object is based on the past position of the moving object accumulated in the moving object information accumulating unit 33 and the current position obtained from the moving object movement state detecting unit 32. Is specified. Specifically, the position and size of the moving body in the past predetermined time are defined at predetermined time intervals.
Further, in this moving body afterimage defining part 35, the moving body afterimage position stored in the moving body information storage section 33 and the current position obtained from the moving body moving state detection section 32 are used. A vector representing the moving direction is calculated.

  Then, the virtual image creation unit 38 generates a virtual image (afterimage) of the past position and size of the moving object defined by the moving object afterimage defining unit 35. Then, the created virtual image is superimposed on the actual image obtained by each of the cameras 2, 4, 6 by the composite image creating unit 26 and displayed on the monitor 14. Further, the speaker 15 emits a predetermined sound indicating that the virtual image is updated.

  Next, a cue position determination unit 50 exists in the ECU 18. The cueing position determination unit 50 determines that the vehicle has cueed in the intersection (see FIG. 4). If the cue has been performed, the driver can already see the intersection, so a predetermined signal is sent so as to stop the virtual display of the moving body in the virtual image creation unit 38.

  Further, the ECU 18 includes a virtual image processing execution determination unit 52. The virtual image processing execution determination unit 52 determines whether the vehicle is located near the intersection based on the map data obtained from the navigation device 13, the position data obtained from the GPS device 12, and the vehicle speed data obtained from the vehicle speed sensor 10. And whether or not it is temporarily stopped. As will be described later, when the vehicle is in the vicinity of an intersection and is temporarily stopped, a predetermined signal is sent to the composite image creating unit 26 in order to display a virtual image.

  Next, the control contents of the vehicle surrounding monitoring apparatus according to the second embodiment of the present invention will be described with reference to FIGS. FIG. 10 is a flowchart showing the display processing of the distance grasping virtual image of the vehicle surroundings monitoring device according to the second embodiment of the present invention, and FIG. 11 is a diagram of the vehicle surroundings monitoring device according to the second embodiment of the present invention. FIG. 12 is a diagram illustrating an imaging range by a camera, an example of a moving object (a), and an example of an image (b) in which a virtual image is superimposed on a real image, and FIG. 12 illustrates a virtual image according to the second embodiment of the present invention. It is an example of the image | video superimposed on the image. In FIG. 10, S indicates each step.

  As shown in FIG. 10, first, in S30, the image acquisition unit 22 (see FIG. 9) acquires videos of the left monitoring camera 2, the front monitoring camera 4, and the right monitoring camera 6, for example, at an intersection. In S31, the image correction unit 24 corrects the distortion of the acquired wide-angle image.

Next, in S32, the moving object detection unit 30 detects the moving object from the actual images acquired in S30 and S31. The detection of the moving body in S32 is performed by optical flow processing. In addition to the optical flow, the moving object may be detected by background difference, inter-frame difference processing, laser, millimeter wave, or the like.
When the moving body is not detected, the process proceeds to S33, and the actual images acquired in S30 and S31 input to the composite image creating unit 26 (see FIG. 9) are displayed as they are, for example, as those that can safely proceed on the intersection. To do. That is, display of a virtual image is prohibited.

  If a moving object is detected in S32, the process proceeds to S34, where the moving object information accumulating unit 33 (see FIG. 9) uses the moving object as the information on the moving object. The information on the size of the moving object at is stored.

  Next, in S35, the moving body moving state detection unit 32 detects the distance, speed, and traveling direction of the moving body from the host vehicle, and calculates the arrival time of the moving body on the host vehicle. When there are a plurality of moving objects, basically, the distance from the own vehicle to the moving object that has the shortest distance from the own vehicle, or the moving object that has the shortest arrival time to the own vehicle. In addition, the traveling direction is detected, and the arrival time of the moving body to the host vehicle is calculated. On the other hand, as will be described later, if virtual images are displayed for a plurality of moving objects, but they do not overlap each other, the distance and speed of the moving object from the host vehicle with respect to the following moving object (such as a following vehicle) In addition, the traveling direction is detected, and the arrival time of the moving body to the host vehicle is calculated.

Next, in S36, it is determined whether the distance of the moving body from the host vehicle is within a predetermined distance, or whether the arrival time of the moving body at the host vehicle is within a predetermined time. The predetermined distance in S36 is, for example, 0 to 10 m, and the predetermined time is, for example, 0 to 1 second.
If it is determined in S36 that the distance of the moving body from the host vehicle is within a predetermined distance, or the arrival time of the moving body to the host vehicle is within a predetermined time, that is, in this embodiment, the movement When the body exists at a short distance, the process proceeds to S33, and the actual image acquired in S30 and S31 input to the composite image creation unit 26 (see FIG. 9) is displayed as it is. That is, display of a virtual image is prohibited. In other words, if it is a short distance, the driver only needs to know the presence of the moving body, and may pay attention and wait for starting and passing.

If it is not determined in S36 that the distance of the moving body from the host vehicle is within a predetermined distance or the arrival time of the moving body in the host vehicle is not within the predetermined time, the process proceeds to S37.
In S37, it is determined whether the distance of the moving body from the host vehicle is greater than or equal to a predetermined distance, or whether the arrival time of the moving body to the host vehicle is greater than or equal to a predetermined time. The predetermined distance in S37 is, for example, 30 m, and the predetermined time is, for example, 3 seconds.

  In S37, when it is determined that the distance of the moving body from the own vehicle is not less than a predetermined distance or the arrival time of the moving body to the own vehicle is not more than the predetermined time, the The distance is 10 to 30 m, or the arrival time of the moving body to the host vehicle is 1 to 3 seconds. In this embodiment, the process proceeds to S38 when the moving body exists at a medium distance. In this way, when the moving body is present at a medium distance from the host vehicle, the driver is at a distance or time at which the driver waits for the passing of the moving body or whether the host vehicle starts first. This is a case where the user wants to recognize the distance, speed and traveling direction of the moving body as accurately as possible.

In such a case, in S38, the position and size of the afterimage display of the moving body are defined, and the sound output in accordance with the timing for updating the afterimage display is defined. These rules include the rule of how many seconds the afterimage is displayed and how many seconds the afterimage is displayed. In the present embodiment, the afterimage display is updated between 0.5 seconds and 2 seconds, and the time intervals for displaying are updated at equal intervals.
Next, in S39, a vector display for connecting afterimages is defined. The vector display is defined so as to connect the center of gravity of the two-dimensional image of each moving body or the center of gravity of the moving body obtained from the three-dimensional coordinates of the moving body. This ensures the direction of travel.

  In S40, for example, as shown in FIG. 11B, virtual images V21 and V22, which are afterimage displays of the moving body, are displayed on the monitor 14 at the position and size defined in S38. Further, as shown in FIG. 11B, the vector Ve defined in S39 is superimposed on the actual image and displayed on the monitor 14. Further, as shown in FIG. 11B, T20, T21 and T22 indicating how many seconds before the virtual images V1 and V2 are superimposed on the actual image and displayed on the monitor 14. In FIG. 11 (b), as actual images, the vehicles (moving bodies) C21 and C22 existing at the current position and the road surface condition currently visible are displayed on the monitor 14.

  In S40, the virtual image of the moving body is always displayed superimposed on the background and the moving body at the current time (latest) so that the driver can see the latest video at the current time as the actual image.

As shown in FIG. 11B, when there are a plurality of mobile bodies C21 and C22, in this embodiment, the mobile body (C21) existing at the shortest distance from the own vehicle or the arrival time to the own vehicle is the shortest. Virtual displays V21, V22, T20, T21, and T22 are displayed only for the moving object (C21). In FIG. 11B, if the moving object C22 does not overlap with the moving object C21 itself and the virtual display, the moving object C22 may be displayed in a virtual manner. . In other words, when virtual display is performed on the moving body C22, when the moving body C1 itself and the virtual display overlap with each other, display of the virtual image of the following vehicle is prohibited.
Further, when there is a subsequent moving body (following vehicle), the number of the following moving bodies is displayed as indicated by reference numeral N in FIG.

  Note that the virtual display (V21, V22) is preferably displayed with only a contour line, a transparent light color display, or a blinking display so as to be distinguished from the actual image. In S <b> 40, a sound such as “beep” is output from the speaker 15 in accordance with the timing at which the afterimage display is updated. The update timing is, for example, about 0.5 second to 2 seconds so that the time for the driver to watch the moving object is included. Since the driver cannot grasp the sense of speed unless he / she keeps watching the monitor 14 and learns the update timing, such a sound is output.

  Next, if it is determined in S37 that the distance of the moving body from the host vehicle is greater than or equal to a predetermined distance, or the arrival time of the moving body to the host vehicle is greater than or equal to a predetermined time, the vehicle moves with the host vehicle. The distance from the body is 30 m or more, or the arrival time of the moving body to the host vehicle is 3 seconds or more. In this embodiment, the process proceeds to S41 when the moving body exists at a long distance. In this way, when the moving body exists at a long distance from the host vehicle, the driver wants to be confident that there is no danger of a collision, and when the driver wants to accurately grasp the speed change and traveling direction change of the moving body. is there.

  In such a case, first, in S41, the position and size of the moving object for displaying the afterimage are defined. In the present embodiment, when the moving body is present at a long distance, the moving body and the afterimage display are enlarged and displayed on the monitor 14, and thus the position and size in the enlarged display of such a moving body and the afterimage display are also defined. . In addition, the output of sound in accordance with the timing for updating the afterimage display is defined. Further, it defines how many seconds the afterimage is displayed and how many seconds the afterimage is displayed.

  Next, in S42, a vector display for connecting afterimages is defined. The vector display is defined so as to connect the center of gravity of the two-dimensional image of each moving body or the center of gravity of the moving body obtained from the three-dimensional coordinates of the moving body. This ensures the direction of travel.

Next, in S40, as shown in FIG. 12, for example, a moving vehicle C23 is enlarged and displayed on the upper left of the screen of the monitor 14. The position to be enlarged may be anywhere as long as the display of the moving object C23 in the actual image is not hindered. Also in this case, in S40, the virtual display or virtual display of the moving body is always superimposed on the background and the moving body at the current time (latest) so that the driver can see the latest video as the actual image. Displays the road surface display.
In the example shown in FIG. 12, virtual images V23 and V24, which are afterimage displays of the moving object, are displayed on the monitor 14 in a superimposed manner on the real image at the position and size defined in S41.

  Similarly to FIG. 11B, the vector Ve and times T20, T21, and T22 are also displayed. Similarly to the case described with reference to FIG. 11 (b), when there are a plurality of moving bodies, the moving body existing at the shortest distance from the own vehicle or the moving body having the shortest arrival time to the own vehicle. Only the virtual display is displayed. In addition, when virtual display is performed on the foremost moving body, when the subsequent moving body itself and the virtual display overlap with each other, display of the virtual image of the subsequent vehicle is prohibited.

Note that the virtual display (V21, V22) is preferably displayed with only a contour line, a transparent light color display, or a blinking display so as to be distinguished from the actual image.
In S <b> 40, a sound such as “beep” is output from the speaker 15 in accordance with the timing at which the afterimage display is updated. The update timing is, for example, about 0.5 seconds to 2 seconds.

  Next, it progresses to S43 and it is determined by the cue position determination part (refer FIG. 9) 50 whether the own vehicle passed the cue position (refer FIG. 4). When the host vehicle is cueing, the process proceeds to S44, where the composite image creation unit 26 (see FIG. 9) causes the virtual image displayed in S40 to blink, and then stops displaying the virtual image.

Next, the function and effect of the second embodiment of the present invention will be described.
According to the vehicle surroundings monitoring apparatus according to the second embodiment of the present invention, it is possible to monitor a blind spot area in an area that spreads left and right in front of the driver. In the present embodiment, cameras 2, 4, 6 are provided in the front part of the vehicle so as to capture a real image of the blind spot area, and an actual image captured by these cameras 2, 4, 6 by the moving body detection unit 30. A moving object is extracted from the image. Further, the moving state of the moving body is detected by the moving body moving state detection unit 32, and the position and the size of the moving body to display the virtual image by the moving body afterimage definition unit 35 for a predetermined time before the moving body are displayed. It is defined to represent the movement state of Then, a virtual image of the moving object is generated by the virtual image generating unit 38, and the virtual image of the moving object is generated by the composite image generating unit 26 at a position specified by the moving object afterimage defining unit 35 a predetermined time before the moving object. The image is superimposed on the actual image and displayed on the monitor 14.

  Next, according to the second embodiment, the moving object afterimage defining unit 35 sets the moving state of the moving object as the afterimage of the moving object at an equal time interval in the past position where the moving object has moved. The position and size for displaying the virtual image of the moving object is defined so that it can be displayed, and this is displayed as an afterimage at regular time intervals, so it is possible to intuitively grasp the moving state of the moving object up to the present time. For the driver, for example, it is possible to more easily determine whether or not the host vehicle is allowed to start.

Next, according to the second embodiment, it is possible to more intuitively grasp the moving state of the moving body up to the present by the vector display in accordance with the afterimages at equal time intervals.
Next, according to the second embodiment, the virtual image of the moving body is displayed in an enlarged manner when the distance of the moving body from the host vehicle is a predetermined distance or more than when the distance is smaller than the predetermined distance. Therefore, it is possible to easily grasp the moving state of the moving body at a long distance.

Next, according to the second embodiment, the composite image creating unit 26 displays the virtual image of the moving object when the distance from the own vehicle of the moving object is within a predetermined distance or to the own vehicle of the moving object. This is done when the arrival time of the vehicle is within a predetermined time. For example, when there is a moving object within a predetermined distance that makes it difficult to determine whether to start or stop the host vehicle, the driver reliably moves the vehicle. The movement state of the body can be grasped.
Next, according to the second embodiment, the composite image creation unit 26 displays a virtual image of the moving body so as to be distinguishable from the actual vehicle actual and actual state of the moving body, so that the driver can visually recognize the moving body. Easy.

Next, according to the second embodiment, when a plurality of moving bodies are detected by the moving body detection unit 30, the virtual image creation unit 38 has the shortest distance to the host vehicle or the arrival time to the host vehicle. Since the shortest virtual image of the moving body is generated, the moving state of the moving body that may possibly collide with the host vehicle can be grasped more reliably.
Next, according to the second embodiment, when all the virtual images of a plurality of moving objects are displayed by the composite image generating unit 26, when the plurality of moving objects and their virtual images are displayed in an overlapping manner, the virtual image generating unit 38 does not generate all the virtual images of these moving objects, but generates a virtual image of the moving object having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle. Therefore, it is possible to grasp the moving state of a moving body that may collide with the host vehicle more reliably without being confused.

  Next, according to the second embodiment, the composite image creating unit 26 can display the virtual display so as to be distinguishable from the real image of the moving object, for example, display only the contour line, display a transparent light color, or blink. Therefore, the driver can recognize the virtual image as a virtual image, and the real image and the virtual image are not confused.

  Next, according to the second embodiment, the composite image creation unit 26 displays a virtual image of the moving object superimposed on the current background of the moving object and the moving object at the current time. Since the latest video at the present time can be viewed and a virtual image is displayed on the video, the moving state of the moving body can be grasped more reliably along with the surrounding situation.

Next, the configuration of the vehicle surrounding monitoring apparatus according to the third embodiment of the present invention will be described with reference to FIG. FIG. 13 is a block diagram showing the configuration of the vehicle surrounding monitoring apparatus according to the third embodiment of the present invention.
As shown in FIG. 13, image signals from the cameras 2, 4, 6 configured as the periphery monitoring sensor 20 are input to the image acquisition unit 22 in the ECU 18. Since the input image signal is a wide-angle image and is distorted, the image correction unit 24 corrects the image. The image signal from the image correction unit 24 is input to the composite image creation unit 26 as a real image.

  The moving body detection unit 30 detects whether or not a moving body such as another vehicle, a motorcycle, or a bicycle is present in the input image from the image correction unit 24. When the moving body does not exist, the composite image creation unit 26 does not display a virtual image to be described later, so a signal instructing the virtual image is sent to the composite image creation unit 26, and when the mobile body exists, the composite image creation unit Since a virtual image, which will be described later, is displayed at 26, a signal instructing that a moving body is present is sent to the composite image creating unit 26.

  Further, the moving body moving state detection unit 32 receives a signal indicating that the moving body exists from the moving body detection unit 30, and the distance from the own vehicle to the moving body approaching the own vehicle by the laser radar 7. The speed and the traveling direction are detected, and the arrival time to the own vehicle or the intersection is calculated. Instead of the laser radar 7, the distance, speed, and traveling direction of the moving body from the own vehicle are detected by inter-vehicle communication or road-to-vehicle communication, and the arrival time to the own vehicle is calculated. good.

  In addition, by performing image processing on images captured by the cameras 2, 4, and 6, for example, the state of the blinker is detected, and the result of whether the moving body turns right or left or goes straight is detected. In addition, you may detect the left-right turn of a mobile body by vehicle-to-vehicle communication or road-to-vehicle communication.

  In addition, the moving object arrival time t2 calculating unit 36 determines whether the moving object is moving automatically from the distance, speed, and traveling direction of the moving object approaching the own vehicle detected by the moving object moving state detecting unit 32. The time t2 for reaching the vehicle is calculated. The position of the host vehicle is determined by the navigation device 13 or the GPS device 12. Note that, instead of the time t2 when the moving body reaches the host vehicle, the time when the moving body reaches the intersection may be calculated. In that case, the position of the intersection is determined by the navigation device.

  In addition, the time t1 calculation unit 54 required for the vehicle to pass through the intersection receives each information of the navigation device 13, the GPS device 12, and the vehicle speed sensor 10, and the time required for the vehicle to pass through the intersection (or the vehicle becomes the intersection). T1 is calculated.

Then, in the virtual image creation unit 38, as will be described later, the determination result as to whether or not the mobile body makes a right or left turn, the time required for the host vehicle to pass through the intersection, or the time required for the host vehicle to reach the intersection. A virtual road surface image is generated on the road surface according to t1 and the degree of risk determined according to the time t2 when the moving body reaches the host vehicle. The virtual image creation unit 38 defines the position and size of the virtual road surface image. For example, the virtual road surface image defines the position and size so as to extend in front of the moving body with the same road surface width, lane width, or traveling lane obtained from the actual image.
Then, the created virtual road surface image is superimposed on the actual image obtained by each camera 2, 4, 6 by the composite image creation unit 26 and displayed on the monitor 14.

  Next, a cue position determination unit 50 exists in the ECU 18. The cueing position determination unit 50 determines that the vehicle has cueed in the intersection (see FIG. 4). If the cueing has been performed, the driver can already see the intersection, so a predetermined signal is sent so as to stop the virtual display of the virtual road surface image in the virtual image creation unit 38.

  Further, the ECU 18 includes a virtual image processing execution determination unit 52. The virtual image processing execution determination unit 52 determines whether the vehicle is located near the intersection based on the map data obtained from the navigation device 13, the position data obtained from the GPS device 12, and the vehicle speed data obtained from the vehicle speed sensor 10. And whether or not it is temporarily stopped. As will be described later, when the vehicle is in the vicinity of an intersection and is temporarily stopped, a predetermined signal is sent to the composite image creating unit 26 in order to display a virtual image.

  Next, the control contents of the vehicle periphery monitoring device according to the third embodiment of the present invention will be described with reference to FIGS. FIG. 14 is a flowchart showing display processing of a distance grasping virtual image of the vehicle surrounding monitoring apparatus according to the third embodiment of the present invention, and FIG. 15 is a diagram of the vehicle surrounding monitoring apparatus according to the third embodiment of the present invention. FIG. 16 is an example (a) of an imaging range by a camera and a moving body, and an example (b) of an image in which a virtual image is superimposed on a real image, and FIG. 16 is a camera of a vehicle surrounding monitoring apparatus according to a third embodiment of the present invention FIG. 6 is a diagram illustrating an example of an imaging range and a moving object (a), an example of a video (b) in which a virtual image is superimposed on a real image, and an example (c) of a video in which a virtual image is superimposed on a real image. In FIG. 14, S indicates each step.

  In this third embodiment, how much time or distance the moving body is on the own vehicle or the intersection is superimposed on the surrounding real image including the moving body that actually exists (such as another vehicle approaching the intersection). The driver recognizes the moving state of the moving body by virtually superimposing and displaying a virtual road surface image on the road surface in accordance with the risk of reaching. Thereby, for example, it is possible to help the driver determine whether or not to start the host vehicle.

  As shown in FIG. 14, first, in S50, the images of the left monitoring camera 2, the front monitoring camera 4, and the right monitoring camera 6 are acquired by the image acquisition unit 22 (see FIG. 13), for example, at an intersection. In S51, the image correction unit 24 corrects the distortion of the acquired wide-angle image.

Next, in S52, the ECU 18 detects a moving body from the actual images acquired in S50 and S51. The detection of the moving body in S52 is performed by optical flow processing. In addition to the optical flow, the moving object may be detected by background difference, inter-frame difference processing, laser, millimeter wave, or the like.
When the moving object is not detected, the process proceeds to S53, and the actual images acquired in S50 and S51 input to the composite image creating unit 26 (see FIG. 13) are displayed as they are, for example, as those that can safely proceed on the intersection. To do. That is, display of a virtual image is prohibited.

  If a moving body is detected in S52, the process proceeds to S54, and the moving body moving state detecting unit 32 and the moving body arrival time t2 calculating unit 36 (see FIG. 13) detect the moving body approaching the host vehicle. The distance, speed, and traveling direction from the vehicle are detected, and the arrival time t2 to the host vehicle is calculated. The arrival time t2 to the host vehicle may be an arrival time to the intersection. When there are a plurality of moving objects, basically, the distance from the own vehicle to the moving object that has the shortest distance from the own vehicle, or the moving object that has the shortest arrival time to the own vehicle. In addition, the traveling direction is detected, and the arrival time of the moving body to the host vehicle is calculated.

Next, in S55, a time t1 calculating unit 54 (see FIG. 13) required for the own vehicle to pass through the intersection calculates a time t1 required for the own vehicle to pass through the intersection.
Next, proceeding to S56, whether or not the mobile body makes a right or left turn at an intersection from the detection result obtained by the mobile body movement state detection unit 32 (see FIG. 13) whether the mobile body goes straight or turns right or left. Determine whether.

  When the mobile body makes a right or left turn, the process proceeds to S57, where it is determined that the traveling direction of the mobile body makes a right or left turn at the intersection, and the virtual image creation unit 38 determines that the traveling direction is the traveling direction of the mobile body. A virtual road surface display is generated that is superimposed and displayed on each real image of the road surface of the straight line portion up to the intersection and the road surface in the direction of travel after the left turn. Here, since the degree of danger is small, the virtual road surface display is blue. If no color is to be added, the display is shaded, for example, lightly displayed.

  Then, the process proceeds to S58, and the virtual road surface display created in S57 is displayed superimposed on the actual image, for example, as indicated by R1 in FIG. By this virtual road surface display, the driver can immediately recognize that the moving body is turning left, and for example, it is easy to determine whether or not the host vehicle can be started.

  Next, when it is determined in S56 that the moving body travels straight, the process proceeds to S59 first. In S59, it is determined whether or not the difference between the time t2 when the moving body reaches the host vehicle and the time t1 when the host vehicle reaches the intersection is less than a predetermined value k0.

  If it is less than the predetermined value k0, the process proceeds to S60, where both the moving body and the host vehicle reach the intersection at substantially the same time. A virtual road surface display that is displayed by being superimposed on the actual image of the road surface of the straight line portion including the intersection is created. Here, since the degree of danger is large, the virtual road surface display is red. If no color is to be added, a shade is added, for example, it is darkened.

  Then, the process proceeds to S58, and the virtual road surface display created in S60 is displayed superimposed on the actual image, for example, as indicated by R2 in FIG. By this virtual road surface display, the driver can immediately recognize that the moving body is going straight ahead and is in a very close position, and for example, it is easy to make a determination for waiting the host vehicle.

In S59, when t2-t1 is equal to or greater than k0, the process proceeds to S61, where the difference between the time t2 when the moving body reaches the host vehicle and the time t1 when the host vehicle reaches the intersection is equal to or greater than the predetermined value k0; It is determined whether it is less than a predetermined value k1.
If it is greater than or equal to the predetermined value k0 and less than the predetermined value k1, the process proceeds to S62, where there is a certain time difference between the time when the moving body arrives and the time when the host vehicle arrives. The image creation unit 38 creates a virtual road surface display that is displayed superimposed on the actual image of the road surface of the straight line portion including the intersection that is the traveling direction of the moving body. Here, since the degree of danger is medium, the virtual road surface display is yellow. If no color is to be added, a shading is applied, for example, a medium darkness.

  Then, the process proceeds to S58, and the virtual road surface display created in S62 is displayed superimposed on the actual image, for example, as indicated by R3 in FIG. With this virtual road surface display, the driver can immediately recognize that the moving body is going straight ahead and that the moving body is approaching in a few seconds, for example, whether to wait for the vehicle to start. Judgment becomes easier.

  In S61, when the difference between the time t2 when the moving body reaches the own vehicle and the time t1 when the own vehicle reaches the intersection is equal to or greater than the predetermined value k1, the process proceeds to S63, and the time when the moving body reaches the own vehicle Since there is a large time difference in the arrival time of the vehicle, the virtual road surface display is displayed by superimposing on the real image of the road surface of the straight line portion including the intersection that is the traveling direction of the moving body by the virtual image creation unit 38 as the risk level is small Create Here, since the degree of danger is small, the virtual road surface display is blue. If no color is to be added, a shading is applied, for example, lightening.

  In step S58, the virtual road surface display created in step S63 is displayed superimposed on the actual image. With this virtual road surface display, the driver can immediately recognize that the moving body is going straight ahead and that the moving body is approaching considerably later. For example, it is easy to determine that the host vehicle can be started. Become.

  In S58, as described above, the virtual road surface display is changed depending on the arrival times t1 and t2 to the intersection, but is changed and displayed according to the certainty of the calculation of the arrival time of the moving body. You may do it. The certainty is the accuracy of information such as autonomous sensor information, vehicle-to-vehicle communication information, road-to-vehicle communication information, and information center distribution information that are information sources. For example, it can be determined that the certainty is high in the order of autonomous sensor information, vehicle-to-vehicle communication information, road-to-vehicle communication information, and information center distribution information. If the certainty factor is high, the driver may be able to determine the certainty factor by blinking the virtual road surface display or changing the blinking interval.

  Following S58, the process proceeds to S64, and the cueing position determination unit (see FIG. 13) 50 determines whether or not the host vehicle has passed the cueing position (see FIG. 4). When the host vehicle is cueing, the process proceeds to S65, where the composite image creation unit 26 (see the drawing) causes the virtual image displayed in S58 to blink, and then stops displaying the virtual image.

Next, functions and effects according to the third embodiment of the present invention will be described.
According to the vehicle surrounding monitoring apparatus according to the embodiment of the present invention, it is possible to monitor a blind spot area in an area that spreads left and right in front of the driver. In the present embodiment, cameras 2, 4, 6 are provided in the front part of the vehicle so as to capture a real image of the blind spot area, and an actual image captured by these cameras 2, 4, 6 by the moving body detection unit 30. A moving object is extracted from the image. Further, the moving state of the moving body is detected by the moving body moving state detecting unit 32, and the reaching time of the moving body is calculated by the reaching time t2 calculating unit 36 of the moving body. In addition, the time t1 calculating unit 54 required for passing the intersection of the host vehicle calculates the time required for passing the intersection of the host vehicle, and the virtual image creating unit 38 defines a display mode for displaying the virtual road surface image based on the calculated time. Then, a virtual road surface image is generated by the virtual image creating unit 38, and the synthesized image creating unit 26 superimposes the virtual road surface image on the actual image in a prescribed display manner and displays it on the monitor 14.

  Next, according to the third embodiment, since a predetermined virtual road surface image is superimposed and displayed on the road surface in front of the moving body in accordance with the moving state of the moving body, in addition to the virtual display of the moving body, Thus, the road surface on which the moving body is traveling is virtually displayed, and the direction and moving state in which the moving body exists can be grasped more reliably.

  Next, according to the third embodiment, on the road surface ahead of the moving body in the advancing direction according to the arrival time of the host vehicle to the intersection of the intersection where the vehicle and the moving body go and the arrival time of the moving body, Since the virtual road surface display is superimposed and displayed, it is predetermined on the road surface in front of the traveling direction of the moving body according to the arrival time of the own vehicle to the intersection of the intersection where the own vehicle and the moving body go and the arrival time of the moving body. The virtual road surface display is superimposed and displayed, and the direction and moving state in which the moving body exists can be grasped more reliably and accurately.

  Next, according to the third embodiment, the virtual road surface display is changed and displayed in accordance with the certainty of calculation of the arrival time of the own vehicle to the intersection and the arrival time of the moving body, so the own road to the intersection is displayed. The virtual road surface display is changed and displayed according to the certainty of calculation of the arrival time of the vehicle and the arrival time of the moving body, and the direction and moving state in which the moving body exists can be grasped more reliably and accurately. In addition, as a certainty factor, it is preferable to determine that a certainty factor is high in order of autonomous sensor information, vehicle-to-vehicle communication information, road-to-vehicle communication information, and information center distribution information, for example.

  Next, according to the third embodiment, when the moving object detection unit 30 detects a plurality of moving objects, the virtual image generation unit 38 has the shortest distance to the own vehicle or the arrival time to the own vehicle. Since a virtual road surface display is generated for the shortest moving body, when there are a plurality of moving bodies, a virtual image of the moving body having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle is generated. Thus, it is possible to grasp the moving state of the moving body that may collide with the host vehicle more reliably.

  Next, according to the third embodiment, when all the virtual images of a plurality of moving bodies are displayed, the plurality of moving bodies and their virtual images are displayed in an overlapping manner, and all the virtual images of the moving bodies are generated. Instead, a virtual image of the moving body that has the shortest distance to the host vehicle or the shortest arrival time to the host vehicle is generated, so that the plurality of virtual images are not confused with each other more reliably. It is possible to grasp the moving state of a moving body that may collide.

  Next, according to the third embodiment, the composite image creation unit 26 displays, for example, only a contour line, a transparent light color display, or a blinking display so that the virtual image can be distinguished from the real image of the moving object. Therefore, the driver can recognize the virtual image as a virtual image, and the real image and the virtual image are not confused.

  Next, according to the third embodiment, the composite image creation unit 26 displays a virtual image or a virtual road surface image of the moving object superimposed on the current background of the moving object and the moving object at the current time. As a real image, the latest video at the present time can be seen, and a virtual image is displayed for it, so that the moving state of the moving object can be grasped more reliably along with the surrounding situation.

  In the first to third embodiments described above, the description has been given mainly assuming a vehicle as a moving body, but the moving body may include a four-wheeled vehicle, a two-wheeled vehicle, a bicycle, and a pedestrian. . In addition, examples of the stationary object include a stopped four-wheeled vehicle, a stopped two-wheeled vehicle, a stationary pedestrian and bicycle, a roadside object, and a building.

1 is an overall configuration diagram of a vehicle to which a vehicle surrounding monitoring device according to first to third embodiments of the present invention is applied. It is a figure which shows an example (a) of the imaging range by the camera of the surroundings monitoring apparatus for vehicles by the 1st thru | or 3rd embodiment of this invention, and an example (b) of the imaged image. It is a figure for demonstrating the driving action at the time of the crossing approach by a driver. It is a bird's-eye view of the road and vehicles for explaining the cueing position of the own vehicle. It is a flowchart which shows the process of whether to display the distance grasping | ascertainment virtual image of the vehicle periphery monitoring apparatus by 1st thru | or 3rd Embodiment of this invention. 1 is a block diagram illustrating a configuration of a vehicle surroundings monitoring device according to a first embodiment of the present invention. It is a flowchart which shows the display process of the virtual image for distance grasping | ascertainment of the vehicle periphery monitoring apparatus by 1st Embodiment of this invention. It is a figure which shows the imaging range by the camera of the surroundings monitoring apparatus for vehicles by 1st Embodiment of this invention, an example (a) of a moving body, and an example (b) of the image | video on which the virtual image was superimposed on the real image. It is a block diagram which shows the structure of the vehicle periphery monitoring apparatus by 2nd Embodiment of this invention. It is a flowchart which shows the display processing of the distance grasping | ascertainment virtual image of the vehicle periphery monitoring apparatus by 2nd Embodiment of this invention. It is a figure which shows the imaging range by the camera of the surroundings monitoring apparatus for vehicles by 2nd Embodiment of this invention, an example (a) of a moving body, and an example (b) of the image | video on which the virtual image was superimposed on the real image. It is an example of the image | video on which the virtual image by 2nd Embodiment of this invention was superimposed on the real image. It is a block diagram which shows the structure of the vehicle periphery monitoring apparatus by 3rd Embodiment of this invention. It is a flowchart which shows the display process of the distance grasping | ascertainment virtual image of the vehicle periphery monitoring apparatus by 3rd Embodiment of this invention. It is a figure which shows an example (a) of the imaging range by the camera of the surroundings monitoring apparatus for vehicles by 3rd Embodiment of this invention, an example (a) of a moving body, and an example (b) of the image | video on which the virtual image was superimposed on the real image. Example (a) of an imaging range by a camera and an example of a moving body (a), an example of a video in which a virtual image is superimposed on a real image, and a virtual image are superimposed on a real image according to the third embodiment of the present invention It is an example (c) of the produced | generated image | video.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle 2, 4, 6 Monitoring camera 7 Laser radar 10 Vehicle speed sensor 12 GPS apparatus 13 Navigation apparatus 14 Image display monitor 15 Speaker 18 ECU

Claims (18)

A vehicle surrounding monitoring device that monitors a blind spot area in an area that extends from side to side in front of a driver,
Real image capturing means provided at the front of the vehicle so as to capture a real image of the blind spot area;
A moving body detecting means for extracting a moving body from the real image captured by the real image capturing means and detecting a moving state of the moving body;
Virtual image defining means for defining a display mode of the predetermined virtual image in order to display a predetermined virtual image representing the moving state of the moving object detected by the moving object detecting means;
Display means for superimposing and displaying the virtual image of the moving body on the real image in a display mode defined by the virtual image defining means;
A vehicle surrounding monitoring device characterized by comprising: The driver's blind spot area is a blind spot area around the intersection of the intersection on the front side of the vehicle,
The said real image imaging means is provided in the front end of a vehicle, and when a vehicle approaches an intersection, the real image around the intersection of the intersection which can be imaged from the front end of the vehicle is imaged. Vehicle surrounding monitoring device.   The virtual image defining means is a virtual image of the moving object such that the virtual image of the moving object is displayed at the estimated arrival position after a predetermined time on the extension line of the moving object as the moving state of the moving object. The vehicle surroundings monitoring device according to claim 1 or 2, wherein a display mode for displaying is displayed.   The second display unit according to claim 3, further comprising: a second display unit configured to display a numerical display of a predicted arrival time from the current position of the displayed moving body based on the predetermined time in accordance with a virtual image of the moving body. Vehicle surrounding monitoring device.   The virtual image defining means is a moving image of the moving object such that a virtual image of the moving object is displayed as an afterimage at equal time intervals at a past position where the moving object has moved. The vehicle surroundings monitoring device according to claim 1 or 2, wherein a display mode for displaying is displayed.   The vehicle surrounding monitoring apparatus according to claim 5, further comprising third display means for performing vector display in accordance with the afterimages at equal time intervals.   The display means displays the virtual image of the moving body in an enlarged manner when the distance from the vehicle of the moving body is a predetermined distance or more than when the distance is smaller than the predetermined distance. The vehicle surrounding monitoring apparatus according to claim 6.   The display means displays the virtual image of the moving body so that the distance of the moving body from the host vehicle is within a predetermined distance range or the arrival time of the moving body to the host vehicle is within a predetermined time range. The vehicle surrounding monitoring device according to any one of claims 1 to 7, which is sometimes performed.   The vehicle surrounding monitoring apparatus according to any one of claims 1 to 8, wherein the display means displays a virtual image of the moving body so as to be distinguishable from an entity and a real state of the moving body.   When a plurality of moving bodies are detected by the moving body detecting means, the virtual image defining means is configured to display a virtual image of the moving body having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle. The vehicle surrounding monitoring apparatus according to any one of claims 1 to 9, wherein the display means defines and displays the defined virtual image superimposed on a real image.   When the plurality of moving bodies are detected by the moving body detecting means and the display means displays all the virtual images of the plurality of moving bodies, the plurality of moving bodies and their virtual images are displayed in an overlapping manner. The virtual image defining means defines a display mode of a virtual image of a moving body having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle, and the display means displays the defined virtual image as a real image. The vehicle surrounding monitoring device according to any one of claims 1 to 9, wherein the vehicle surrounding monitoring device is displayed in a superimposed manner.   The virtual image defining means displays the virtual road surface image indicating the moving state of the moving body on the road surface ahead of the moving body in order to represent the moving state of the moving body. The vehicle surrounding monitoring device according to claim 1 or 2, wherein the vehicle surrounding monitoring device is defined.   The virtual image defining means is different depending on the arrival time of the own vehicle and the arrival time of the moving body until the intersection of the own vehicle and the moving body, or the intersection where the own vehicle and the moving body are headed. The display mode of the virtual road surface image is defined so that a different virtual road surface image is displayed on the road surface in front of the moving body in accordance with the passing time of the own vehicle at the intersection and the arrival time of the moving body. The vehicle surrounding monitoring apparatus according to claim 12.   The virtual image defining means changes the display mode of the virtual road surface display so as to change the virtual road surface image according to the certainty of calculation of the arrival time of the host vehicle to the intersection and the arrival time of the moving body. The vehicle surrounding monitoring device according to claim 12 or 13, wherein the vehicle surrounding monitoring device is defined.   When a plurality of moving bodies are detected by the moving body detecting means, the virtual image defining means is configured to display the virtual road surface image with respect to the moving body having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle. The vehicle surrounding monitoring according to any one of claims 12 to 14, wherein the display means displays only the specified virtual road surface image as a virtual image superimposed on the real image. apparatus.   When the plurality of moving bodies are detected by the moving body detecting means and the display means displays all the virtual images of the plurality of moving bodies, the plurality of moving bodies and their virtual images are displayed in an overlapping manner. The virtual image defining means defines the display mode of the virtual road surface image for a moving object having the shortest distance to the own vehicle or the shortest arrival time to the own vehicle, and the display means The vehicle surrounding monitoring apparatus according to any one of claims 12 to 14, wherein only a road surface image is displayed as a virtual image superimposed on the real image.   The vehicle surrounding monitoring apparatus according to claim 1, wherein the display unit displays the virtual image so as to be distinguishable from a real image of the moving body.   18. The display unit according to claim 1, wherein the display unit displays a virtual image or a virtual road surface image of the moving body superimposed on a current background of the moving body and a real image of the moving body at the current time. The vehicle surrounding monitoring device as described.

Images