JP2009231937A - Surroundings monitoring device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a surroundings monitoring device for a vehicle, enabling a driver to more surely comprehend the state of a blind area. <P>SOLUTION: The surroundings monitoring device for a vehicle is used to monitor a blind area in a region which extends on right and left on the front viewed from the driver. The monitoring device is provided with: real image pickup means (2, 6) provided on the rear part of a vehicle so as to pick up a real image of the blind area; a real image storage means (30) for storing real images picked up by the real image pickup means; an imaginary image generating means (38) for generating images of the blind area of the real image pickup means as imaginary images from past real images stored in the real image storage means; and display means (26, 14) for displaying the virtual images generated by the imaginary image generating means together with the real image picked up by the real image pickup means. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle surrounding monitoring device, and more particularly to a vehicle surrounding monitoring device that monitors a blind spot area of a driver.

  2. Description of the Related Art Conventionally, devices that capture and display a blind spot image in the left-right direction on the front side of a vehicle are known (Patent Documents 1 and 2).

JP 2007-172462 A JP 11-338074 A

  The apparatus as described above displays the blind spot area in the left and right driver imaged by the camera or the like on the monitor. For example, when there is an obstacle that blocks the camera view at the corner of the intersection or the camera itself When the vehicle has a blind spot area, the driver cannot obtain an image of the blind spot area unless the vehicle has sufficiently entered the intersection. In other words, in front of such an intersection, not only the driver has a blind spot area, but also a blind spot area is generated in the camera, so there are cases where the driver cannot be provided with sufficient information.

  The present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to provide a vehicle surrounding monitoring device that allows a driver to more reliably grasp the condition of a blind spot area. .

  In order to achieve the above object, according to the vehicle surrounding monitoring apparatus of the present invention, there is provided a vehicle surrounding monitoring apparatus that monitors a blind spot area in an area that spreads left and right in front of the driver. An actual image capturing means provided in the front part of the vehicle so as to capture an image, an actual image storage means for storing the actual image captured by the actual image capturing means, and an image of the blind spot area of the actual image capturing means Virtual image generation means for generating a virtual image from a past real image stored by the real image storage means, and a display for displaying the virtual image generated by the virtual image generation means together with the real image captured by the real image imaging means And means.

  In the present invention configured as described above, the image of the blind spot area of the real image capturing unit is generated and displayed as a virtual image from the past real image stored by the real image storing unit, so that the blind spot is displayed on the real image capturing unit. Even if there is, the driver can get information on his blind spot area.

In the present invention, it is preferable that the virtual image generating means includes a shielding object detecting means for detecting the presence of a shielding object that blocks a part of the surrounding line of sight imaged by the real image imaging means. When the shielding object is detected by the means, an image of the area beyond the shielding object is generated as a virtual image from the past real image stored by the real image storage means.
In the present invention configured as described above, the image of the area on the other side of the shielding object is generated as a virtual image from the past real image stored by the real image storage means. Information can be obtained.

In the present invention, preferably, the blind spot area of the driver is a blind spot area around the intersection of the intersection on the front side of the vehicle, and the real image capturing means is provided at the front end of the vehicle, and the vehicle is in the intersection. When approaching, a real image around the intersection of the intersection that can be imaged from the front end of the vehicle is captured.
In the present invention configured as described above, the driver can obtain information on the left and right intersections of intersections where collision accidents frequently occur, and safer driving is possible.

In the present invention, it is preferable that the apparatus further includes a moving body detection unit that detects a moving body from the real image stored by the real image storage unit, and the virtual image generation unit is a state before being shielded by the shielding object. A virtual image of the moving object is generated, and the display unit displays the generated virtual image of the moving object when the moving object is shielded by the shielding object.
In the present invention configured as described above, since the virtual image of the moving body before being shielded by the shielding object is displayed when the moving object is shielded by the shielding object, the driver may collide. The existence of a certain moving body can be recognized.

In the present invention, it is preferable that the virtual image generation unit generates a virtual image of the moving object so that the moving object is displayed at a position immediately before the moving object is blocked by the shielding object.
In the present invention configured as described above, the driver can recognize the existence of a moving body that may collide.

In the present invention, it is preferable to further include a moving body state estimating means for detecting the moving state of the moving body detected by the moving body detecting means and estimating the position and size of the moving body at the current time point. The virtual image generation unit generates the virtual image so that the moving body is displayed at the current position and size estimated by the moving body state estimation unit.
In the present invention configured as described above, since the moving object is displayed as a virtual image with the estimated position and size at the current time point, the position of the moving object that may collide can be more accurately determined. Can be recognized.

In the present invention, it is also preferable that the second virtual image generation unit further generates a shielding object in the actual image captured by the actual image capturing unit as a second virtual image that is transmitted so that the other side can be seen. The display means displays the second virtual image superimposed on the virtual image of the moving object.
In the present invention configured as described above, since the virtual image of the shielding object that is transmitted so that the other side of the shielding object can be seen is superimposed on the virtual image of the moving object, the presence of the moving object is more effective. Can be recognized.

In the present invention, it is also preferable that the shielding object in the actual image captured by the actual image capturing unit is further generated as a second virtual image at a position shifted from the original position so that the other side can be seen. 2 virtual image generating means, and the display means displays the moving body virtual image and the second virtual image side by side.
In the present invention configured as described above, since the virtual image of the shielding object is displayed side by side with the virtual image of the moving object, the existence of the moving object can be recognized more effectively.

In the present invention, it is preferable that the virtual image generation unit generates a past image at the closest point in the past or a past image at a time when the vehicle passes in the past, and the generated past in the blind spot area. An image is displayed, and an actual image captured by the actual image capturing unit is displayed in other areas.
In the present invention configured as described above, the past image at the closest point in the past in the blind spot area or the past image at the time when the vehicle passed in the past is displayed, and the actual image is displayed in the other areas. Therefore, the driver can more effectively recognize the state of the blind spot area of the real image capturing means (for example, the road structure or the presence of a building). Further, if a past image at the closest point in the past is displayed in the blind spot area, it is possible to recognize a moving object in the visual area.

In the present invention, it is preferable that the virtual image generation unit generates a stationary object at the time when the vehicle has passed previously as a past image.
In the present invention configured as described above, it is possible to grasp the state of a stationary object such as a road structure or a sign even in a visual region.

  According to the vehicle surrounding monitoring apparatus of the present invention, the driver can more reliably grasp the state of the blind spot area.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
First, with reference to FIGS. 1 and 2, a basic configuration of a vehicle surrounding monitoring apparatus according to an embodiment of the present invention will be described. FIG. 1 is an overall configuration diagram of a vehicle to which a vehicle surroundings monitoring device according to an embodiment of the present invention is applied. FIG. 2 is an example of an imaging range by a camera of a vehicle surroundings monitoring device according to an embodiment of the present invention ( It is a figure which shows an example (b) of a and the image | video imaged.

  As shown in FIG. 1, the vehicle 1 has a left monitoring camera 2, a front monitoring camera 4, and a right monitoring camera 6, and these cameras are attached to the front end of the vehicle. These cameras are CCD cameras using a wide-angle lens, and for example, can capture a range as shown in FIG. 2A and obtain an image as shown in FIG.

  The vehicle 1 includes a vehicle speed sensor 10, a GPS device 12, a navigation device 13, and an image display monitor 14. The monitor 14 is arranged at a position where the driver of the driver's seat 16 can see. Further, the vehicle 1 has an ECU 18, and signals from the cameras 2, 4, 6, the vehicle speed sensor 10, and the GPS device 12 are input to the ECU 18 and a predetermined image is displayed on the monitor 14. The vehicle 1 also has a storage device 19 capable of storing video.

Next, the control contents of the vehicle surrounding monitoring apparatus according to the embodiment of the present invention will be described with reference to FIG. FIG. 3 is a conceptual diagram for explaining the contents of the vehicle surroundings monitoring apparatus according to the embodiment of the present invention, and FIG. 4 is a block diagram showing the control contents of the vehicle surroundings monitoring apparatus according to the embodiment of the present invention. is there.
In the embodiment of the present invention, for example, as shown in FIG. 3, the other vehicle V3 is detected by the camera from the vehicle V1 approaching the intersection, but from the camera of the vehicle V2 further approaching the intersection, a shielding object that blocks the field of view. When the other vehicle V4 becomes invisible due to A, the vehicle V4 after the movement of the vehicle V3 is mainly used as a virtual image on the monitor of the vehicle V2 based on the detected stored image of the other vehicle V3 in the past. Is displayed virtually, and at that time, a part of the shield A is transmitted and displayed as a virtual image.

  Next, as shown in FIG. 4, image signals from the cameras 2, 4, 6 configured as the periphery monitoring sensor 20 are input to the image acquisition unit 22 in the ECU 18. Since the input image signal is a wide-angle image and is distorted, the image correction unit 24 corrects the image. The image signal from the image correction unit 24 is input to the composite image creation unit 26 as a real image.

The captured image storage unit 30 stores the input image from the image correction unit 24 in the storage device 19 provided with the ECU 18.
Next, the moving body detection unit 32 detects a moving body such as a four-wheeled vehicle approaching the host vehicle from the stored image stored in the captured image storage unit 30. Specifically, the type of moving object (four-wheeled vehicle, pedestrian, etc.) is detected by pattern matching, and the moving direction and moving speed of the moving object are also detected by optical flow, background difference, inter-frame difference processing, etc. . In the present embodiment, the moving body is mainly detected from an image captured when there is no shielding object A (see FIG. 3) before entering the intersection.

The moving body information storage unit 34 stores information such as the position, size, speed, and direction data of the moving body detected by the moving body detection unit 32 in the storage device 19. In the mobile object position estimation unit 36, based on each piece of information of the mobile object stored in the storage device 19, the current position and the apparent size of the mobile object are determined by the time when the information is stored and the current time. Estimate.
The virtual image creation unit 38 creates a virtual image of the moving body with the estimated position and size of the moving body.

  On the other hand, in the stationary object detection unit 40, for example, the shielding object A (see FIG. 3) before entering the intersection is mainly used in this embodiment by image processing from the stored image stored in the captured image storage unit 30. A stationary object is detected from the captured image when there is no such object. The stationary object information accumulation unit 42 accumulates information such as the position and size of the stationary object detected by the stationary object detection unit 40 in the storage device 19. The virtual image creation unit 38 creates a virtual image of the stationary object with the detected position and size of the stationary object.

  Further, the shielding object detection unit 44 detects a shielding object that blocks the field of view of the camera, such as a guard rail or a wall, from the actual image obtained by the image correction unit 24 by a method such as pattern matching. Further, in the shielding object image creation determination unit 46, based on the position and size of the shielding object detected by the shielding object detection unit 44, the virtual image is not created but the actual image is displayed on the monitor 14 or the virtual image is displayed. Whether to superimpose on the actual image is determined.

In the virtual image creation unit 38, the virtual image of the shielding object is displayed such that the position of the detected shielding object is changed and displayed, or a part of the shielding object is transmitted and displayed so that the moving object behind can be seen. Create
As described above, the composite image creation unit 26 creates each virtual image created by the virtual image creation unit 38 as a composite image together with the actual image.

Next, a cue position determination unit 50 exists in the ECU 18. Here, the cueing position will be described with reference to FIG. FIG. 5 is an overhead view of the road and the vehicle for explaining the cueing position of the own vehicle.
As shown in FIG. 5, the cue position is the position when the driver's seat where the driver sits enters the intersection, and the driver can directly see inside the intersection with his own eyes. It is. As shown in FIG. 2 (a), the camera position is the state where the tip of the vehicle is in the intersection, and the driver cannot see the intersection, but the camera attached to the front end of the vehicle 2, 4, 6 is a position where you can see through the intersection. The nose-out position is a position that informs other vehicles and pedestrians of the presence of the host vehicle.

  As will be described later, the cue position determination unit 50 determines that the vehicle has cueed in the intersection. If the cue has been performed, the driver can already see the intersection, so a predetermined signal is sent to the virtual image creation unit 38 to stop the virtual image creation or the virtual display of the existing object, and the monitor 14 The virtual image is not displayed.

  Further, the ECU 18 includes a virtual image processing execution determination unit 52. The virtual image processing execution determination unit 52 determines whether the vehicle is located near the intersection based on the map data obtained from the navigation device 13, the position data obtained from the GPS device 12, and the vehicle speed data obtained from the vehicle speed sensor 10. And whether or not it is temporarily stopped. As will be described later, when the vehicle is temporarily stopped near the intersection, a predetermined signal is sent to the composite image creation unit 26 in order to display the above-described landmark or existing virtual image.

Next, with reference to FIG. 6, driving behavior when a driver enters an intersection will be described. FIG. 6 is a diagram for explaining the driving behavior when the driver enters the intersection.
First, stop before the intersection. Thereafter, the vehicle proceeds slowly to the nose out position, proceeds to the camera out position (see FIG. 2A), and then proceeds to the cue position (see FIG. 5). In the vehicle surrounding monitoring apparatus according to the embodiment of the present invention, a virtual image is displayed between the temporary stop and the cueing position. When the cueing position is reached, the virtual image is blinked, and after passing through the cueing position, the display of the virtual image is stopped. Thereafter, start determination and start operation are performed.

Next, with reference to FIG. 7, an outline of the control of the vehicle surrounding monitoring apparatus according to the embodiment of the present invention will be described. FIG. 7 is a flowchart showing a process for determining whether or not to display a blind spot visual image of the vehicle periphery monitoring device according to the embodiment of the present invention. In FIG. 7, S indicates each step.
First, as shown in FIG. 7, in S <b> 1, the image acquisition unit 22 (see FIG. 4) acquires videos of the left monitoring camera 2, the front monitoring camera 4, and the right monitoring camera 6, for example, at an intersection. In S2, the image correction unit 24 corrects the distortion of the acquired wide-angle image. In S3, the captured image is stored in the storage device 19. In the present embodiment, camera images are always acquired and stored before entering the intersection by the processes of S1 to S3.

  Next, in S4, the moving body detection unit 32 (see FIG. 4) detects the moving body from the actual images acquired in S1 and S2. The detection of the moving body in S4 is performed by optical flow processing. In addition to the optical flow, the moving object may be detected by background difference, inter-frame difference processing, laser, millimeter wave, or the like.

  When the moving body is detected in S4, the process proceeds to S5, where the moving body detecting unit 32 (see FIG. 4) calculates the position, size, moving direction, and moving speed of the moving body. In S6, the moving body information The storage unit 34 (see FIG. 4) stores the position, size, moving direction, and moving speed of the moving body calculated in S5 in the storage device 19. In this embodiment, in S6, data of moving body information from a predetermined time before the vehicle enters the intersection is stored in time series. For example, data on the position, size and moving direction of the moving body with respect to time is accumulated. When the moving body is not detected, the processes of S5 and S6 are not performed.

  Next, in S7, the stationary object detection unit 40 detects a stationary object from the actual images acquired in S1 and S2. The detection of the stationary object in S7 is performed by image processing. When a stationary object is detected in S7, the process proceeds to S8, where the stationary object detection unit 40 calculates the position and size of the stationary object. In S9, the stationary object information storage unit 42 (see FIG. 4) performs S8. The position and size of the stationary object calculated in (1) are stored in the storage device 19. In this embodiment, in S9, data on stationary objects that are originally moving, such as cars and pedestrians, are accumulated in a time series from a predetermined time before the vehicle enters the intersection. Still objects such as signs and traffic lights always accumulate. When no stationary object is detected, the processes of S8 and S9 are not performed.

  Next, in S10, the virtual image processing execution determination unit 52 (see FIG. 4) determines whether or not the host vehicle is in the vicinity of the intersection based on the signal from the GPS 12 and the signal from the navigation device 13. Next, in S <b> 11, the virtual image processing execution determination unit 52 (see FIG. 4) determines whether the vehicle speed is lower than V <b> 1 based on the signal from the vehicle speed sensor 10. The vehicle speed V1 is a value that allows the driver to determine that the vehicle is safe even when the vehicle approaches the intersection and decelerates and the driver looks at the monitor 14.

  If the vehicle speed is less than V1 and the vehicle is decelerated, the process proceeds to S12. In S12, an actual image is first displayed based on the image signal from the image correction unit 24. Next, in S13, the virtual image processing execution determination unit 52 (see FIG. 4) determines whether or not the vehicle speed is lower than V2. The vehicle speed V2 is a value at which it can be determined that the host vehicle is almost stopped or stopped. If the vehicle speed is less than V2, the process proceeds to S14 assuming that the host vehicle is temporarily stopped.

  In S14, it is determined whether or not the host vehicle has passed the cueing position (see FIG. 5). If the driver has passed the cueing position, the driver can overlook the surrounding situation, so the process proceeds to S15, and the actual images acquired in S1 and S2 are displayed on the monitor 14 as they are. If it has not passed through the cueing position, the process proceeds to S16, and a blind spot visual image display process is performed.

Next, the contents of the blind spot visual image display process of the vehicle periphery monitoring device according to the embodiment of the present invention will be described with reference to FIG. FIG. 8 is a flowchart showing a blind spot visual image display process of the vehicle surrounding monitoring apparatus according to the embodiment of the present invention. In FIG. 8, S indicates each step.
First, in S20, it is determined whether or not accumulated data of a moving object or accumulated data of a stationary object is accumulated. Specifically, it is determined whether moving object information is accumulated in S6 of FIG. 7 or stationary object information is accumulated in S9 of FIG. 7 within a predetermined time before entering the intersection. When there is no accumulated data, the blind spot visual image is not displayed.

  Next, when there is accumulated data, the process proceeds to S21, and the current position of the moving object is estimated by the moving object position estimating unit 36 (see FIG. 4). The estimated position is displayed as a virtual image, as will be described later. However, the process of S21 is not performed, the position is not estimated, and the past position is displayed as it is to determine the position of the moving body. You can leave it up to the driver. In this case, for example, a vector display indicating the moving direction and speed may be attached to the moving body. Next, the process proceeds to S22, where the position data of the stationary object accumulated by the stationary object information accumulation unit 42 is acquired.

  Next, in S <b> 23, it is determined whether or not there is a shielding object that blocks the visual field within the visual field of the left and right cameras 2 and 6. For example, an object that blocks the line of sight within a predetermined short distance is determined as a shielding object. For example, as shown in FIG. 9, when the intersection is not orthogonal, there is no shielding object in the field of view of the right camera 6, but the other vehicle V cannot be seen when actually viewed from the driver. Become.

  In such a case, it progresses to S24 and it is determined whether the position of a moving body or a stationary object exists in a complementary position. Here, the complementary position means a case where there is no shielding object in the field of view of the camera as shown in FIG. 9, but there is a moving body or a stationary object in a portion that cannot be seen from the camera or the driver. In FIG. 9, such a complementary position is illustrated as a complementary image region.

  If there is a moving object or a stationary object at the supplementary position, the process proceeds to S25, and the actual image actually acquired by the camera and the supplementary image are displayed in parallel. This complementary image displays information on past moving objects and stationary objects specified from moving object accumulation data and stationary object accumulation data as a virtual image.

  For example, in the case shown in FIG. 9, as shown in FIG. 10, the composite image creation unit 26 (see FIG. 4) displays the actual image actually acquired by the camera on the left side and the shielding object on the right side. A virtual image such as a virtual image Av of A (see FIG. 9), a virtual image Bv of a sign B (see FIG. 9), and a virtual image Vv of another vehicle V (see FIG. 9) as a moving body is displayed. A simple image is created and displayed on the monitor 14. As a result, it is possible to grasp the state of an area that becomes a blind spot when viewed from the camera and the driver by using past data. In addition, about a mobile body, it is good to display on the present position of the mobile body estimated by S21.

  Next, in S23, when it is determined that there is an obstacle that blocks the field of view within the field of view of the left and right cameras 2 and 6, for example, within the field of view of the camera as viewed from the current vehicle position as shown in FIG. If the shielding object A exists, the process proceeds to S26 to determine whether the position of the moving body or the stationary object is located behind the shielding object. If it is not located rearward, the process proceeds to S24 and the above-described processing is performed.

  When it is determined that the position of the moving object or the stationary object is located behind the shielding object, the process proceeds to S27, and the composite image creating unit 26 (see FIG. 4) determines the virtual image of the moving object or the stationary object. A composite image with the acquired real image is created and displayed on the monitor 14. The moving body may be displayed as it is based on past information, or may be displayed at the current position of the moving body estimated in S21. The stationary object is displayed at the position accumulated in the past. The past includes both the time immediately before entering the intersection and the past time. As for the moving object, the past represents the time immediately before entering the intersection.

  In S27, for example, as shown in FIG. 12, a part of the shielding object A, that is, the shielding object Av of the virtual image in which the part where the moving object or the stationary object should be visible is made translucent is displayed. The moving body Vv of the virtual image obtained from the past information is displayed behind. Further, as shown in FIG. 13, the entire shielding object Av may be displayed while being shifted laterally so that the moving body Vv obtained from past information can be clearly seen.

Next, functions and effects according to the embodiment of the present invention will be described.
The vehicle surroundings monitoring device according to the embodiment of the present invention is configured to monitor a blind spot area in an area that spreads left and right in front of the driver and captures a real image of the blind spot area. Cameras 2, 4, 6 are provided, real images captured by the cameras 2, 4, 6 are stored in the storage device 19, and images of the blind spots of the cameras 2, 4, 6 are stored in the cameras 2, 4, 6. Is generated as a virtual image by the virtual image creation unit (virtual image generation means) 38 from the past real image stored in step S4, and the generated virtual image is displayed on the monitor 14 together with the real image captured by the cameras 2, 4, and 6. indicate. In the embodiment of the present invention configured as described above, the images of the blind spots of the cameras 2, 4, 6 are generated and displayed as virtual images from the stored past real images, so that the cameras 2, 4, 6 are displayed. Even if there is a blind spot, the driver can obtain information on his / her blind spot area.

  Here, the blind spot area of the driver is a blind spot area around the intersection of the intersection on the front side of the vehicle. When the vehicle approaches the intersection, the cameras 2, 4 and 6 provided at the front end of the vehicle are Since a real image around the intersection of the intersection that can be imaged from the front end of the vehicle is captured, the driver can obtain information on the left and right intersections of the intersection where collision accidents frequently occur, thereby enabling safer driving.

  Further, according to the present embodiment, when the shielding object detecting unit 44 detects the presence of a shielding object that blocks a part of the surrounding line of sight imaged by the cameras 2, 4, and 6, and when the shielding object is detected, The virtual image creation unit 38 generates an image of the area beyond the shielding object as a virtual image from the past real image stored by the storage unit 19. Therefore, since the image of the area beyond the obstacle is generated as a virtual image from the stored past real image, the driver can obtain information on the blind spot area even if there is an obstacle.

  Further, according to the present embodiment, the moving body detection unit 32 detects a moving body from the real image stored in the storage device 19, and the virtual image creation unit 38 moves the moving body before being shielded by the shielding object. When the moving object is shielded by the shielding object, the composite image creating unit 26 superimposes the generated virtual image of the moving object on the real image and displays it on the monitor 14. Therefore, since the virtual image of the moving object before being shielded by the shielding object is displayed when the moving object is shielded by the shielding object, the driver recognizes the existence of the moving object that may collide. I can do it.

  Further, according to the present embodiment, the virtual image creation unit 38 generates a virtual image of the moving body so that the moving body is displayed at a position immediately before the moving body is shielded by the shielding object, so that the driver can collide. It is possible to recognize the existence of a mobile object with sex.

  Further, according to the present embodiment, the moving body detecting unit 32 detects the moving state of the moving body, the moving body position estimating unit 36 estimates the position and size of the moving body at the current time point, and the virtual image creating unit 38 generates a virtual image so that the moving object is displayed with the position and size at the current time point estimated by the moving object position estimating unit 36. Therefore, since the moving object is displayed as a virtual image with the estimated position and size at the current time, the position of the moving object that may collide can be recognized more accurately.

  In addition, according to the present embodiment, the virtual image creation unit 38 transmits the shielding object in the real image captured by the cameras 2, 4, and 6 so that the other side can be seen, and the composite image creation unit 26 moves the virtual image. The virtual image is displayed on the monitor 14 so as to be superimposed on the virtual image of the body. Therefore, since the virtual image of the shielding object that is transmitted so that the other side of the shielding object can be seen is superimposed on the virtual image of the moving object, the existence of the moving object can be recognized more effectively.

  Further, according to the present embodiment, the virtual image creation unit 38 creates a virtual image at a position shifted from the original position so that the other side of the shielding object in the real image captured by the cameras 2, 4, 6 can be seen. Then, the composite image creation unit 26 displays the virtual image of the moving object and the virtual image side by side on the monitor 14. Therefore, since the virtual image of the shielding object is displayed side by side with the virtual image of the moving object, the presence of the moving object can be recognized more effectively.

  The virtual image creation unit 38 generates a past image at the closest point in the past or a past image at a time when the vehicle has passed in the past, and the generated past image is displayed in the blind spot area. Then, the actual image captured by the actual image capturing means is displayed. Therefore, the driver can more effectively recognize the state of the blind spot area of the real image capturing means (for example, the road structure or the presence of a building). Further, if a past image at the closest point in the past is displayed in the blind spot area, it is possible to recognize a moving object in the visual area.

  Since the virtual image creation unit 38 generates a stationary object at the time when the vehicle has passed previously as a past image, it is possible to grasp the state of a stationary object such as a road structure or a sign even in the visual region.

  In the above-described embodiment, the description has been given mainly assuming a vehicle as the moving body, but the moving body may include a four-wheeled vehicle, a two-wheeled vehicle, a bicycle, and a pedestrian. In addition, examples of the stationary object include a stopped four-wheeled vehicle, a stopped two-wheeled vehicle, a stationary pedestrian and bicycle, a roadside object, and a building.

1 is an overall configuration diagram of a vehicle to which a vehicle surrounding monitoring device according to an embodiment of the present invention is applied. It is a figure which shows an example (a) of the imaging range by the camera of the surroundings monitoring apparatus for vehicles by embodiment of this invention, and an example (b) of the imaged image. It is a conceptual diagram for demonstrating the content of the vehicle periphery monitoring apparatus by embodiment of this invention. It is a block diagram which shows the control content of the surroundings monitoring apparatus for vehicles by embodiment of this invention. FIG. 5 is an overhead view of the road and the vehicle for explaining the cueing position of the own vehicle. It is a figure for demonstrating the driving action at the time of the crossing approach by a driver. It is a flowchart which shows the process of whether the virtual image for blind spot visual recognition of the surroundings monitoring apparatus for vehicles by embodiment of this invention is displayed. It is a flowchart which shows the virtual image display process for blind spot visual recognition of the periphery monitoring apparatus for vehicles by embodiment of this invention. It is a bird's-eye view showing the field of view and surrounding road conditions of the vehicle and camera on which the vehicle surrounding monitoring device according to the embodiment of the present invention is mounted. It is an example of the panoramic image of the real image and complementary image which are displayed on a monitor by the vehicle periphery monitoring apparatus by embodiment of this invention. It is a bird's-eye view showing the field of view and surrounding road conditions of the vehicle and camera on which the vehicle surrounding monitoring device according to the embodiment of the present invention is mounted. It is an example of the composite image of the real image and virtual image which are displayed on a monitor by the vehicle periphery monitoring apparatus by embodiment of this invention. It is an example of the composite image of the real image and virtual image which are displayed on a monitor by the vehicle periphery monitoring apparatus by embodiment of this invention.

Explanation of symbols

1 Vehicle 2, 4, 6 Monitoring camera 10 Vehicle speed sensor 12 GPS device 13 Navigation device 14 Image display monitor 18 ECU
19 Storage device

Claims (10)

A vehicle surrounding monitoring device that monitors a blind spot area in an area that extends from side to side in front of a driver,
Real image capturing means provided at the front of the vehicle so as to capture a real image of the blind spot area;
Real image storage means for storing a real image imaged by the real image imaging means;
Virtual image generation means for generating an image of the blind spot area of the real image imaging means as a virtual image from past real images stored by the real image storage means;
Display means for displaying the virtual image generated by the virtual image generating means together with the real image captured by the real image capturing means;
A vehicle surrounding monitoring device characterized by comprising: A shielding object detecting means for detecting the presence of a shielding object that blocks a part of the surrounding line of sight imaged by the real image capturing means;
When the shielding object is detected by the shielding object detection means, the virtual image generation means generates an image of the area beyond the shielding object as a virtual image from the past real image stored by the real image storage means. The vehicle surrounding monitoring apparatus according to claim 1. The driver's blind spot area is a blind spot area around the intersection of the intersection on the front side of the vehicle,
The said real image imaging means is provided in the front end of a vehicle, and when a vehicle approaches an intersection, the real image around the intersection of the intersection which can be imaged from the front end of the vehicle is imaged. The vehicle surrounding monitoring apparatus according to 2. Furthermore, it has a moving body detection means for detecting a moving body from within the real image stored by the real image storage means,
The virtual image generation means generates a virtual image of the moving body before being shielded by the shielding object,
The vehicle surrounding monitoring apparatus according to any one of claims 1 to 3, wherein the display means displays a virtual image of the generated moving body when the moving body is shielded by a shielding object.   The vehicle surrounding monitoring apparatus according to claim 4, wherein the virtual image generation unit generates a virtual image of the moving body so that the moving body is displayed at a position immediately before the moving body is shielded by a shielding object. Furthermore, it has a moving body state estimating means for detecting the moving state of the moving body detected by the moving body detecting means and estimating the position and size of the moving body at the current time point,
5. The vehicle surrounding monitoring according to claim 4, wherein the virtual image generation unit generates the virtual image so that the moving body is displayed at a position and a size at a current time point estimated by the moving body state estimation unit. apparatus. Furthermore, it has a second virtual image generating means for generating a shield in the real image captured by the real image capturing means as a second virtual image that is transmitted so that the other side can be seen.
The vehicle surrounding monitoring apparatus according to any one of claims 3 to 6, wherein the display means displays the second virtual image superimposed on the virtual image of the moving body. And a second virtual image generating means for generating a second virtual image at a position shifted from the original position so that the other side of the shielding object in the real image captured by the real image capturing means can be seen. ,
The vehicle surrounding monitoring apparatus according to any one of claims 3 to 6, wherein the display means displays the moving object's virtual image and the second virtual image side by side.   The virtual image generation means generates a past image at the closest point in the past or a past image at a time when the vehicle has passed in the past, displays the generated past image in the blind spot area, and displays other areas. The vehicle surrounding monitoring apparatus according to claim 1, wherein an actual image captured by the actual image capturing unit is displayed.   The vehicle surrounding monitoring apparatus according to claim 9, wherein the virtual image generation unit generates a stationary object at a time when the vehicle has passed previously as a past image.

Images