JP2009230225A - Periphery monitoring device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a periphery monitoring device for a vehicle for making it easy to intuitively grasp the feeling of depth in an image of a blind area. <P>SOLUTION: This periphery monitoring device for a vehicle for monitoring a blind area in a region spread right and left at the front when it is viewed from a driver is provided with: real-image pick-up means (2, 4, 6) installed at the front part of a vehicle so as to pick up the real image of a blind area; a virtual light source setting means (38) for setting a virtual light source so that a virtual shadow can be applied to a static object and/or a moving object imaged by an actual video imaging means; a virtual shadow image generation means (34) for generating, as a virtual shadow image, a shadow when the static object and/or moving object receive the rays of light from a virtual light source set by the virtual light source setting means; and display means (26, 14) for displaying the generated virtual shadow image by superimposing it on the actual image so that the generated virtual shadow image can be extended form the road surface ground position of the static object and/or the moving object to the ground surface. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle surrounding monitoring device, and more particularly to a vehicle surrounding monitoring device that monitors a blind spot area of a driver.

  2. Description of the Related Art Conventionally, devices that capture and display a blind spot image in the left-right direction on the front side of a vehicle are known (Patent Documents 1 and 2).

JP 2007-172462 A JP 11-338074 A

  The apparatus as described above, for example, is such that the left and right blind spot area is displayed as it is on the monitor, and it is difficult to intuitively grasp the sense of depth. The driver looks at such a display to determine the state of the blind spot area, but because the depth is not known, for example, the sense of distance to the vehicle approaching from the left and right direction is difficult to understand intuitively. . In particular, in the infrared image and the high dynamic range camera, the contrast is low, and such a problem is remarkable.

  The present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to provide a vehicle surrounding monitoring device that makes it easy to intuitively grasp the sense of depth in an image of a blind spot area. .

  In order to achieve the above object, according to the present invention, there is provided a vehicle surrounding monitoring device for monitoring a blind spot area in an area that spreads left and right in front of a driver so as to capture a real image of the blind spot area. Real image capturing means provided in the front part of the vehicle, virtual light source setting means for setting a virtual light source that gives a virtual shadow to a stationary object and / or a moving body imaged by the real image capturing means, and the virtual light source setting means Virtual shadow image generation means for generating a shadow when a stationary object and / or a moving body receives light from the virtual light source set by the light source setting means as a virtual shadow image; and And / or display means for displaying the image superimposed on the actual image so as to extend from the road surface grounding position of the moving body to the ground.

  In the present invention configured as described above, a virtual shadow image when a stationary object or a moving body receives light from a virtual light source is converted into a real image so as to extend from the ground contact position of the stationary object or the moving body to the ground. Since it is displayed in a superimposed manner, the driver sees the virtual shadow image shadow displayed on the ground in the image of the blind spot area, so that the sense of distance and depth to moving objects and stationary objects can be intuitively viewed. I can grasp it.

Further, in the present invention, preferably, it further includes virtual shadow image defining means for defining the direction and size of the shadow of the virtual wall image based on the position of the virtual light source and the shape of the moving body and / or the stationary object, The display means extends the virtual shadow image of the stationary object and / or the moving body from the road contact surface position of the stationary object and / or the moving body with the direction and size of the shadow defined by the virtual shadow image defining means. It is displayed superimposed on the actual image.
In the present invention configured as described above, the direction and size of the shadow of the virtual wall image is defined based on the position of the virtual light source and the shape of the moving object and / or the stationary object. It is possible to grasp the sense of distance and depth to moving objects and stationary objects.

In the present invention, preferably, the position of the virtual light source is the position of the sun calculated from the latitude, longitude, and time.
In the present invention configured as described above, the sun is set as a more natural virtual light source, so the driver can more intuitively grasp the sense of distance and depth to moving objects and stationary objects. I can do it.

In the present invention, preferably, the position of the virtual light source is a position set above the position of the road surface that is a predetermined distance away from the intersection where the host vehicle enters.
In the present invention configured as described above, a virtual shadow image of a moving object or a stationary object is viewed while taking the position of the virtual light source into consideration, and a sense of distance or depth to the moving object or the stationary object can be obtained with higher accuracy. Can be grasped.

In the present invention, it is preferable that the predetermined distance from the intersection is a distance to an appropriate road surface position for determining whether the own vehicle starts or stops when another vehicle approaches.
In the present invention configured as described above, virtual shadow images of moving objects and stationary objects are generated based on the position of the road surface suitable for determining whether the host vehicle is starting or stopping when another vehicle approaches. By seeing, it is possible to determine whether the own vehicle starts or stops more safely.

In the present invention, preferably, the predetermined distance from the intersection can be arbitrarily set by the driver.
In the present invention configured as described above, since a virtual light source can be set in the sky where it is easy for the driver to determine, it is possible to more accurately grasp the sense of distance and depth to a moving object or a stationary object. I can do it.

In the present invention, it is preferable that the display unit changes the display form of the virtual shadow image before and after passing and / or during the passing of the vehicle at a predetermined distance away from the intersection.
In the present invention configured as described above, since the display form is changed before and after passing through a predetermined distance, it is possible to grasp the sense of distance and the sense of depth to a moving body or a stationary object with higher accuracy.

In the present invention, preferably, the position of the virtual light source is the position of a streetlight.
In the present invention configured as described above, a sense of distance and a sense of depth to a moving body or a stationary object can be grasped more intuitively at night, for example.

In the present invention, preferably, the blind spot area of the driver is a blind spot area around the intersection of the intersection on the front side of the vehicle, and the real image capturing means is provided at the front end of the vehicle, and the vehicle is in the intersection. When approaching, a real image around the intersection of the intersection that can be imaged from the front end of the vehicle is captured.
In the present invention configured as described above, the driver can obtain information on the left and right intersections of intersections where collision accidents frequently occur, and safer driving is possible.

Further, in the present invention, preferably, it further includes a cue position calculating means for calculating a cue position at which the driver's line of sight is improved at the intersection, and the display means performs virtual Cancels the shadow image display.
In the present invention configured as described above, the display of the virtual shadow image is stopped, so that the driver grasps that the own vehicle is in the cue position and grasps the road condition with his own eyes. I can do it.

In the present invention, preferably, further, a moving body detecting means for detecting a moving body from within the real image captured by the real image capturing means, and when the moving body is detected by the moving body detecting means, the movement is detected. Moving body state detecting means for detecting the type and / or moving speed and / or moving direction of the body, and the display means displays the virtual shadow according to the type and / or moving speed and / or moving direction of the moving body. Change the image display.
In the present invention configured as described above, since the display of the virtual shadow image is changed according to the type and / or moving speed and / or moving direction of the moving object, the driver can obtain information on the moving object (type and speed). , The moving direction) can be obtained from the virtual shadow image, it is possible to grasp the sense of distance and depth to the moving body and the state of the moving body with higher accuracy.

In the present invention, it is preferable that the display unit stops displaying the virtual shadow image of the moving body after the moving body passes the intersection.
In this invention comprised in this way, it can grasp | ascertain that the mobile body in which a virtual shadow image is not displayed is a safe mobile body for the own vehicle.

In the present invention, it is preferable that the second moving body state further detects whether the moving body is approaching at high speed or meandering when the moving body is detected by the moving body detecting means. When the moving body approaching at high speed or meandering approaching is detected, the display means displays a shadow in a color different from that of the other moving bodies.
In the present invention configured as described above, the driver can easily grasp that there is a moving body approaching at high speed to be more careful or meandering.

In the present invention, it is preferable to further include third moving body state detecting means for detecting the presence or absence of a moving body approaching in a plurality of columns by the moving body detecting means. When the moving body approaching in a plurality of columns is detected by the three moving body state detecting means, the shadow of the following vehicle among the moving bodies approaching in the column is displayed differently from the shadow of the leading vehicle. To do.
In the present invention configured as described above, when the moving body approaches in a column, it can be easily grasped that the moving body approaches in a column, and the following moving body can also be a distance. A feeling and a feeling of depth can be grasped.

  According to the vehicle surrounding monitoring apparatus of the present invention, it is easy to intuitively grasp the sense of depth in the image of the blind spot area.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
First, with reference to FIGS. 1 and 2, a basic configuration of a vehicle surrounding monitoring apparatus according to each embodiment common to the first and second embodiments of the present invention will be described. FIG. 1 is an overall configuration diagram of a vehicle to which a vehicle surrounding monitoring apparatus according to first and second embodiments of the present invention is applied, and FIG. 2 is a vehicle surrounding according to the first and second embodiments of the present invention. It is a figure which shows an example (a) of the imaging range by the camera of a monitoring apparatus, and an example (b) of the imaged image.

  As shown in FIG. 1, the vehicle 1 has a left monitoring camera 2, a front monitoring camera 4, and a right monitoring camera 6, and these cameras are attached to the front end of the vehicle. These cameras are CCD cameras using a wide-angle lens, and for example, can capture a range as shown in FIG. 2A and obtain an image as shown in FIG.

  The vehicle 1 also includes a vehicle speed sensor 10, a gyro 11, a GPS device 12, a navigation device 13, and an image display monitor 14. The monitor 14 is arranged at a position where the driver of the driver's seat 16 can see. Further, the vehicle 1 has an ECU 18, and signals from the cameras 2, 4, 6, the vehicle speed sensor 10, the gyro 11, and the GPS device 12 are input to the ECU 18, and a predetermined image is displayed on the monitor 14. indicate.

  Furthermore, the vehicle 1 has a laser radar 7 for detecting a distance from another vehicle. Instead of the laser radar 7, the distance from the other vehicle may be grasped by inter-vehicle communication, or the distance from the other vehicle may be grasped by the road-to-vehicle history.

Next, with reference to FIG. 3, driving behavior when a driver enters an intersection will be described. FIG. 3 is a diagram for explaining driving behavior when a driver enters an intersection.
First, stop before the intersection. After that, the vehicle proceeds slowly to the nasal position, proceeds to the camera position (see FIG. 2A), and proceeds to the cue position (see FIG. 4). Here, the nasal position, the camera position and the cue position will be described. FIG. 4 is an overhead view of the road and the vehicle for explaining the cueing position of the own vehicle.

  First, the nose out position is a position that informs other vehicles and pedestrians of the presence of the host vehicle. As shown in FIG. 2 (a), the camera exit position is a state in which the front end of the vehicle enters the intersection, and the driver cannot see the intersection, but the camera 2 attached to the front end of the vehicle, 4 and 6 are positions where the intersection can be seen. As shown in Fig. 4, the cueing position is the position when the driver's seat where the driver sits enters the intersection, and the driver can directly see inside the intersection with his own eyes. is there.

  In the vehicle surrounding monitoring apparatus according to the embodiment of the present invention, a virtual image to be described later is displayed between the temporary stop and the cueing position. When the cueing position is reached, the virtual image blinks, and after passing through the cueing position, the display of the virtual image is stopped.

Next, referring to FIG. 5, a control flow for determining whether or not to perform the virtual image display processing of the vehicle surrounding monitoring apparatus according to each embodiment common to the first and second embodiments of the present invention will be described. FIG. 5 is a flowchart showing a process of whether or not to display a distance grasping virtual image of the vehicle surroundings monitoring apparatus according to the first and second embodiments of the present invention.
As shown in FIG. 5, in S <b> 1, it is determined based on the signal from the GPS 12 and the signal from the navigation device 13 whether or not the host vehicle is in the vicinity of the intersection. Next, in S2, based on the signal from the vehicle speed sensor 10, it is determined whether or not the vehicle speed is lower than V1. The vehicle speed V1 is a value that allows the driver to determine that the vehicle is safe even when the vehicle approaches the intersection and decelerates and the driver looks at the monitor 14.
If the vehicle speed is less than V1 and the vehicle is decelerated, the process proceeds to S3, where an actual image is first displayed based on the image signal from the image correction unit 24. Next, it progresses to S4 and a virtual shadow image display process is performed.

Next, the configuration of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 6 is a block diagram showing the configuration of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention.
In the first embodiment, the position of the sun is calculated as a virtual light source. For example, a moving object (such as another vehicle that approaches the intersection) or a stationary object (a sign or When there is a traffic light etc., the shadow of a moving object or a stationary object that would be obtained from the calculated position of the sun is superimposed and displayed as a virtual image. The driver, for example, sees the direction of the shadow and the length of the shadow to determine how far away the moving body is, and as a result, helps the driver determine whether or not the vehicle can be started. I can do it.

  First, as shown in FIG. 5, image signals from the cameras 2, 4, 6 configured as the periphery monitoring sensor 20 are input to the image acquisition unit 22 in the ECU 18. Since the input image signal is a wide-angle image and is distorted, the image correction unit 24 corrects the image. The image signal from the image correction unit 24 is input to the composite image creation unit 26 as a real image.

The captured image storage unit 30 stores the image corrected by the image correction unit 24.
The moving object / stationary object detection unit 32 detects the presence of a moving object such as a vehicle and a stationary object such as a sign in the input image from the image correction unit 24.
Detection of the moving body is performed by optical flow processing. In addition to the optical flow, the moving object may be detected by background difference, inter-frame difference processing, laser, millimeter wave, or the like. Further, detection of a stationary object is performed by image processing.
When there is a stationary object, the position and shape of the stationary object are detected by image processing and laser radar 7 or the like.

  If there is a moving body, the shape and type of the moving body are detected by image processing or the like. Further, as the state of the moving body, the laser radar 7 detects the distance, speed, and traveling direction of the moving body approaching the host vehicle from the host vehicle, and further calculates the arrival time to the host vehicle. In addition, instead of the laser radar 7, the moving body detection unit 30 and the moving body state detection unit 32 perform the type of the moving body, the distance from the host vehicle, the speed, and the progress by the inter-vehicle communication or the road-to-vehicle communication. The direction may be detected.

  Information on the moving object and the stationary object detected by the moving object / stationary object detection unit 32 is sent to the virtual image creation unit 34. In the virtual image creating unit 34, the moving body is based on the information input from the moving body / stationary object detecting unit 32 in the shadow position / shape calculating unit 34 a and the information on the position of the sun as a virtual light source described later. And the shape of the virtual shadow image of the stationary object and the display position of the virtual shadow image are calculated. The created virtual shadow image is corrected by the shadow correction unit 34b as described later according to the state of the moving body or the like.

In the own vehicle position / travel direction detection unit 36, the gyro 11 detects a change in the travel direction, and the gyro 11 and the GPS device 12 detect the position and travel direction of the host vehicle.
The virtual light source position setting unit 38 calculates the position of the sun, which is a virtual light source, based on the latitude and longitude information from the GPS device 12, the vehicle traveling direction information from the gyro 11, and the current time. . Based on the calculated position of the sun, as described above, the shadow position / shape calculation unit 34a calculates the shape of the virtual shadow image of the moving body or the stationary object and the display position of the virtual shadow image. In the correction part 34b, it correct | amends as mentioned later according to states, such as a moving body.

  Next, a cue position determination unit 40 exists in the ECU 18. The cueing position determination unit 40 determines that the vehicle has cueed in the intersection (see FIG. 4). If the cue has been performed, the driver can already see the intersection, so the virtual image creation by the virtual image creation unit 34 is stopped and only the actual image is displayed on the monitor 14.

  Further, the ECU 18 includes a virtual image processing execution determination unit 42. The virtual image processing execution determination unit 42 is based on the map data obtained from the navigation device 13, the position data obtained from the GPS device 12, and the vehicle speed data obtained from the vehicle speed sensor 10 to determine whether the vehicle is located near the intersection. And whether or not it is temporarily stopped. As will be described later, when the vehicle is in the vicinity of an intersection and is temporarily stopped, a predetermined signal is sent to the composite image creating unit 26 in order to display a virtual image.

  Next, with reference to FIGS. 7 and 8, the control contents of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention will be described. FIG. 7 is a flowchart showing display processing of a distance grasping virtual image of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention, and FIG. 8 is a diagram of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention. It is a figure which shows an example (b) of the imaging | photography range by a camera, an example (a) of a moving body, and the image | video by which the virtual image was superimposed on the real image. In FIG. 7, S indicates each step.

  As shown in FIG. 7, first, in S11, the image acquisition unit 22 (see FIG. 3) acquires images of the left monitoring camera 2, the front monitoring camera 4, and the right monitoring camera 6, for example, at an intersection, In S12, the image correction unit 24 corrects the distortion of the acquired wide-angle image. Next, in S13, the captured image is saved.

  Next, in S14, the moving object / stationary object detection unit 32 detects the moving object and the stationary object from the actual images acquired in S11 and S12. When the moving body and the stationary object are not detected, the process proceeds to S15, and the actual images acquired in S11 and S12 are displayed as they are.

  When a moving body or a stationary object is detected in S14, the process proceeds to S16, and the virtual light source position setting unit 38 (see FIG. 6) calculates (estimates) the position of the virtual light source sun as described above. Next, proceeding to S17, the moving body / stationary object detection unit 32 (see FIG. 6) detects the shape, position, moving direction and moving speed of the moving body, and the shape and position of the stationary object as described above. .

  Next, the process proceeds to S18, where the shadow position / shape calculation unit 34a (see FIG. 6) calculates the moving object or the stationary object from the position of the sun estimated in S16 and the shape of the moving object or the stationary object as described above. The shape of the virtual shadow image of the object is calculated.

  In step S19, the shadow correction unit 34b (see FIG. 6) performs a shadow correction process. As one of the corrections, when the moving object / stationary object detection unit 32 (see FIG. 6) detects a vehicle moving at high speed or a meandering vehicle, the color of the original virtual shadow image is A virtual shadow image of a vehicle which is black but considered to be dangerous is displayed in red, for example. As other corrections, depending on the type of moving object or stationary object detected by the moving object / stationary object detection unit 32 (see FIG. 6) (eg, vehicle, two-wheeled vehicle, pedestrian, signal, sign, etc.) The virtual shadow image is displayed in different colors such as black, green, and blue. As another correction, when a plurality of moving bodies are detected by the moving body / stationary object detection unit 32 (see FIG. 6), the virtual shadow image of the following vehicle is blinked or the color is made gray. The display form of the virtual shadow image may be changed according to the moving speed and moving direction of the moving body.

Next, in S20, it is determined whether or not the host vehicle has traveled to the cue position within the intersection. When the vehicle has passed the cueing position (see FIG. 4), the driver can look around, so the process proceeds to S15, and the actual image acquired in S11 and S12 is displayed on the monitor 14 as it is. If it has not passed the cueing position, the process proceeds to S21, and the virtual shadow image calculated in S18 and corrected in S19 is displayed superimposed on the actual image obtained in S11 and S12.
An example is shown in FIG. As shown in FIG. 8A, when another vehicle V or a sign S is present on the road on which the host vehicle is about to enter, a virtual shadow image is displayed on the other vehicle V and the sign S as shown in FIG. V1 and S1 are superimposed and displayed.

  In the first embodiment, a virtual shadow image is generated based on the position of the sun. This may emphasize the shadow of the sun when the sun is actually visible. Similarly, for example, at night, a shadow of a moving body or the like with respect to the light of the streetlight may be generated as a virtual shadow image based on the position of the streetlight, and the shadow may be emphasized.

  In the above-described embodiment, the description has been given mainly assuming a vehicle as the moving body, but the moving body may include a four-wheeled vehicle, a two-wheeled vehicle, a bicycle, and a pedestrian. In addition, examples of the stationary object include a stopped four-wheeled vehicle, a stopped two-wheeled vehicle, a stationary pedestrian and bicycle, a roadside object, and a building.

Next, the function and effect of the first embodiment of the present invention will be described.
According to the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention, a virtual shadow image when a stationary object or moving object receives light from the sun as a virtual light source is obtained from a road surface grounding position of the stationary object or moving object. Since it is displayed superimposed on the real image so as to extend on the ground, the driver sees the virtual shadow image shadow displayed on the ground in the image of the blind spot area, so that the moving object and the stationary object can be seen. A sense of distance and depth can be grasped intuitively.

  In addition, according to the present embodiment, the direction and size of the shadow of the virtual wall image is defined based on the position of the sun as the virtual light source and the shape of the moving object and / or the stationary object, and the stationary object and / or the moving object. Since the virtual shadow image is superimposed and displayed on the actual image with the specified shadow direction and size, the driver can more accurately grasp the sense of distance and depth to a moving object or a stationary object. I can do it.

  In addition, according to the present embodiment, since the virtual light source is the sun, the sun can be set as a more natural virtual light source, and the position of the sun is calculated from the latitude, longitude, and time. Intuitively grasp the sense of distance and depth to moving objects and stationary objects.

  Further, according to the present embodiment, the blind spot area of the driver is a blind spot area around the intersection of the intersection on the front side of the vehicle, the real image capturing means is provided at the front end of the vehicle, and the vehicle is in the intersection. When approaching, a real image around the intersection of the intersection that can be imaged from the front end of the vehicle is captured. Accordingly, the driver can obtain information on the left and right intersections at intersections where collision accidents frequently occur, and safer driving is possible.

  Further, according to the present embodiment, since the display of the virtual shadow image is stopped after the host vehicle reaches the cue position where the driver's line of sight is improved at the intersection, the driver stops the host vehicle at the cue position. It is possible to grasp the road situation with one's own eyes while grasping a certain thing.

  In addition, according to the present embodiment, a moving body is detected from the actual images captured by the cameras 2, 4, and 6, and further, the type and / or moving speed and / or moving direction of the moving body is detected, Since the display of the virtual shadow image is changed according to the type and / or moving speed and / or moving direction of the moving object, the driver obtains information (type, speed, moving direction) about the moving object from the virtual shadow image. It is possible to grasp the sense of distance and depth to the moving body and the state of the moving body with higher accuracy.

  In addition, according to the present embodiment, after the moving body passes the intersection, the display of the virtual shadow image of the moving body is stopped, so that the driver can safely move the moving body on which the virtual shadow image is not displayed to the host vehicle. It can be grasped that it is a moving body.

  Further, according to the present embodiment, when a moving body is detected, it is detected whether or not the moving body approaches or meanders at high speed, and approaches or meanders and approaches at high speed. When a moving object is detected, such a moving object displays a shadow in a color different from that of the other moving objects, so the driver is approaching at a higher speed or meandering with more attention. It can be easily grasped that a moving body exists.

  According to the present embodiment, the presence or absence of a moving body approaching in a plurality of columns is detected, and when a moving body approaching in a plurality of columns is detected, the movement approaching in those columns Since the shadow of the following car in the body is displayed differently from the shadow of the leading car, it can be easily grasped that the moving body approaches in a column, and the following moving body also has a sense of distance. And a sense of depth.

Next, the configuration of the vehicle surrounding monitoring apparatus according to the second embodiment of the present invention will be described with reference to FIG. FIG. 9 is a block diagram showing a configuration of the vehicle surrounding monitoring apparatus according to the second embodiment of the present invention.
In the second embodiment, a virtual light source is set at a virtual position, and if there is a moving object (such as another vehicle that is approaching an intersection) or a stationary object (such as a sign or a traffic light), the set light source A shadow of a moving object or a stationary object that will be obtained from the position is superimposed and displayed as a virtual image. The driver, for example, sees the direction of the shadow and the length of the shadow to determine how far away the moving body is, and as a result, helps the driver determine whether or not the vehicle can be started. I can do it. In the second embodiment, the position of the virtual light source is set to a position on the road surface that is appropriate for determining whether the host vehicle starts or stops at the intersection when another vehicle approaches the road ahead. Yes. This position is about 30 to 50 m from the intersection (virtual light source L1, see FIG. 11).

  First, as shown in FIG. 9, image signals from the cameras 2, 4, 6 configured as the periphery monitoring sensor 20 are input to an image acquisition unit 22 in the ECU 18. Since the input image signal is a wide-angle image and is distorted, the image correction unit 24 corrects the image. The image signal from the image correction unit 24 is input to the composite image creation unit 26 as a real image.

The captured image storage unit 30 stores the image corrected by the image correction unit 24.
The moving object / stationary object detection unit 32 detects the presence of a moving object such as a vehicle and a stationary object such as a sign in the input image from the image correction unit 24.
Detection of the moving body is performed by optical flow processing. In addition to the optical flow, the moving object may be detected by background difference, inter-frame difference processing, laser, millimeter wave, or the like. Further, detection of a stationary object is performed by image processing.
When there is a stationary object, the position and shape of the stationary object are detected by image processing and laser radar 7 or the like.

  When the moving body exists, the shape of the moving body is detected by image processing or the like. Further, as the state of the moving body, the laser radar 7 detects the distance, speed, and traveling direction of the moving body approaching the host vehicle from the host vehicle, and further calculates the arrival time to the host vehicle. In addition, instead of the laser radar 7, the moving body detection unit 30 and the moving body state detection unit 32 perform the type of the moving body, the distance from the host vehicle, the speed, and the progress by the inter-vehicle communication or the road-to-vehicle communication. The direction may be detected. The display form of the virtual shadow image may be changed according to the moving speed and moving direction of the moving body.

  Information on the moving object and the stationary object detected by the moving object / stationary object detection unit 32 is sent to the virtual image creation unit 34. In the virtual image creation unit 34, the shadow position / shape calculation unit 34a, based on the information input from the moving body / stationary object detection unit 32 and the position information of the virtual light source described later, the moving body and the stationary object. The shape of the virtual shadow image and the display position of the virtual shadow image are calculated. The created virtual shadow image is corrected by the shadow correction unit 34b as described later according to the state of the moving body or the like.

In the own vehicle position / travel direction detection unit 36, the gyro 11 detects a change in the travel direction, and the gyro 11 and the GPS device 12 detect the position and travel direction of the host vehicle.
The virtual light source position setting unit 38 sets the virtual light source at a predetermined position. This predetermined position is an appropriate position on the road surface (30 to 50 m in this embodiment) for determining whether the own vehicle starts or stops at the intersection when another vehicle approaches the road ahead. It is a position on the other side of the moving body as seen from the own vehicle so that the shadows of other moving bodies and stationary objects can be seen from the own vehicle (see virtual light source L1, FIG. 11).

  Based on the position of the virtual light source, as described above, the shadow position / shape calculation unit 34a calculates the shape of the virtual shadow image of the moving object or the stationary object and the display position of the virtual shadow image, and the shadow correction unit. In 34b, it correct | amends so that it may mention later according to states, such as a moving body.

  Next, a cue position determination unit 40 exists in the ECU 18. The cueing position determination unit 40 determines that the vehicle has cueed in the intersection (see FIG. 4). If the cue has been performed, the driver can already see the intersection, so the virtual image creation by the virtual image creation unit 34 is stopped and only the actual image is displayed on the monitor 14.

  Further, the ECU 18 includes a virtual image processing execution determination unit 42. The virtual image processing execution determination unit 42 is based on the map data obtained from the navigation device 13, the position data obtained from the GPS device 12, and the vehicle speed data obtained from the vehicle speed sensor 10 to determine whether the vehicle is located near the intersection. And whether or not it is temporarily stopped. As will be described later, when the vehicle is in the vicinity of an intersection and is temporarily stopped, a predetermined signal is sent to the composite image creating unit 26 in order to display a virtual image.

  Next, the control contents of the vehicle surrounding monitoring apparatus according to the second embodiment of the present invention will be described with reference to FIGS. FIG. 10 is a flowchart showing the display processing of the distance grasping virtual image of the vehicle surroundings monitoring device according to the second embodiment of the present invention, and FIG. 11 is a diagram of the vehicle surroundings monitoring device according to the second embodiment of the present invention. It is a figure which shows an example (b) of the imaging | photography range by a camera, an example (a) of a moving body, and the image | video by which the virtual image was superimposed on the real image. In FIG. 10, S indicates each step.

  As shown in FIG. 7, first, in S31, the images of the left monitoring camera 2, the front monitoring camera 4, and the right monitoring camera 6 are acquired by the image acquisition unit 22 (see FIG. 3), for example, at an intersection. In S32, the image correction unit 24 corrects the distortion of the acquired wide-angle image. Next, in S33, the captured image is saved.

  Next, in S34, the moving object / stationary object detection unit 32 detects the moving object and the stationary object from the actual images acquired in S31 and S32. When the moving body and the stationary object are not detected, the process proceeds to S35, and the actual images acquired in S11 and S12 are displayed as they are.

  Next, when a moving object or a stationary object is detected in S34, the process proceeds to S36, where the moving object / stationary object detection unit 32 (see FIG. 6) determines the shape, position, moving direction and the moving object as described above. The moving speed and the shape and position of a stationary object are detected. In step S37, the virtual light source position setting unit 38 (see FIG. 6) sets the position of the virtual light source as described above. As described above, the position of the virtual light source is 30 to 50 m from the intersection, and is a position on the other side of other moving bodies and the like as viewed from the own vehicle.

  As a modification, when the position of the virtual light source is a moving body approaching the intersection from the left or right, the arrival time to the intersection where the host vehicle exists is calculated from the position and moving speed. May be set to a position that takes a predetermined time, for example, 2 seconds (a position that takes 2 seconds to reach the host vehicle). In this case, the predetermined time is set as an appropriate time for judging the start or stop of the host vehicle when another vehicle approaches the road ahead in the intersection. Note that such setting is performed for the top moving body when there are a plurality of moving bodies. The driver may arbitrarily set the predetermined time and the distance from the intersection described above.

  Next, the process proceeds to S38, and the shadow position / shape calculation unit 34a (see FIG. 9) calculates the moving object or the stationary object from the position of the virtual light source set in S37 and the shape of the moving object or the stationary object as described above. The shape of the virtual shadow image of the stationary object is calculated. Specifically, as shown in FIG. 11, since the other vehicle V1 closest to the intersection is in front of the virtual light source L1, the virtual shadow image extends long in front.

Since the subsequent vehicle V2 is near the virtual light source L1, the virtual shadow image extends shortly. Further, since the subsequent other vehicle V3 is farther from the virtual light source L1, the virtual shadow image is calculated to extend long toward the rear side of the road. As can be seen from the virtual shadow images of the other vehicles V1 to V3, the virtual shadow image extends toward the own vehicle. Since the other vehicle V4 has passed the intersection, a virtual shadow image is not calculated.
Further, the color of each virtual shadow image may be changed or blinked before the vehicle passes through the virtual light source L1, just during and after the vehicle passes.

  It should be noted that the left road is preferably located on the far side from the center of the left lane as in the virtual light source L2. Also in this case, it is preferable to estimate the arrival time of the moving body at the intersection and set it to a position that takes 2 seconds as described above, or set the distance from the intersection to 30 to 50 m.

  In step S39, the shadow correction unit 34b (see FIG. 6) performs a shadow correction process. As one of the corrections, when the moving object / stationary object detection unit 32 (see FIG. 6) detects a vehicle moving at high speed or a meandering vehicle, the color of the original virtual shadow image is A virtual shadow image of a vehicle which is black but considered to be dangerous is displayed in red, for example. As other corrections, depending on the type of moving object or stationary object detected by the moving object / stationary object detection unit 32 (see FIG. 6) (eg, vehicle, two-wheeled vehicle, pedestrian, signal, sign, etc.) The virtual shadow image is displayed in different colors such as black, green, and blue. As another correction, when a plurality of moving bodies are detected by the moving body / stationary object detection unit 32 (see FIG. 6), the virtual shadow image of the following vehicle is blinked or the color is made gray. The display form of the virtual shadow image may be changed according to the moving speed and moving direction of the moving body.

Next, in S40, it is determined whether or not the host vehicle has traveled to the cue position within the intersection. If the driver has passed the cueing position (see FIG. 4), the driver can look around, so the process proceeds to S35, and the actual image acquired in S11 and S12 is displayed on the monitor 14 as it is. If it has not passed the cueing position, the process proceeds to S41, and the virtual shadow image calculated in S38 and corrected in S39 is displayed superimposed on the actual image obtained in S31 and S32.
An example is shown in FIG. As shown in FIG. 8A, when another vehicle V or a sign S is present on the road on which the host vehicle is about to enter, a virtual shadow image is displayed on the other vehicle V and the sign S as shown in FIG. V1 and S1 are superimposed and displayed.

  In the second embodiment, the virtual shadow image is generated based on the virtual light source. However, for example, at night, the virtual shadow image is generated based on the position of the streetlight at a predetermined distance from the intersection. Anyway.

  In the above-described embodiment, the description has been given mainly assuming a vehicle as the moving body, but the moving body may include a four-wheeled vehicle, a two-wheeled vehicle, a bicycle, and a pedestrian. In addition, examples of the stationary object include a stopped four-wheeled vehicle, a stopped two-wheeled vehicle, a stationary pedestrian and bicycle, a roadside object, and a building.

Next, the function and effect of the second embodiment of the present invention will be described.
According to the vehicle surrounding monitoring apparatus according to the second embodiment of the present invention, a virtual shadow image when a stationary object or a moving body receives light from the sun as a virtual light source is obtained from a road surface ground position of the stationary object or the moving body. Since it is displayed superimposed on the real image so as to extend on the ground, the driver sees the virtual shadow image shadow displayed on the ground in the image of the blind spot area, so that the moving object and the stationary object can be seen. A sense of distance and depth can be grasped intuitively.

  In addition, according to the present embodiment, the direction and size of the shadow of the virtual wall image is defined based on the position of the sun as the virtual light source and the shape of the moving object and / or the stationary object, and the stationary object and / or the moving object. Since the virtual shadow image is superimposed and displayed on the actual image with the specified shadow direction and size, the driver can more accurately grasp the sense of distance and depth to a moving object or a stationary object. I can do it.

  Further, according to the present embodiment, the position of the virtual light source is a position that is set above the position of the road surface that is a predetermined distance away from the intersection where the host vehicle enters, so the driver considers the position of the virtual light source. While seeing the virtual shadow image of the moving object or the stationary object, it is possible to grasp the sense of distance and the depth to the moving object or the stationary object with higher accuracy.

  Moreover, according to this embodiment, when another vehicle approaches, the position of the road surface suitable for judging whether the own vehicle starts or stops, that is, a predetermined distance (for example, 30 to 50 m) away from the intersection. Since the virtual light source is set above the position, the driver can more safely determine the start or stop of the host vehicle by viewing the virtual shadow image based on such a virtual light source.

  Further, according to the present embodiment, since the driver can arbitrarily set the predetermined distance from the intersection, the virtual light source can be set above the position where the driver can easily determine, and the movement can be performed with higher accuracy. It is possible to grasp the sense of distance and depth to the body and stationary objects.

  In addition, according to the present embodiment, the display form of the virtual shadow image is changed before and after the passage of a predetermined distance away from the intersection and / or during the passage of the other vehicle. And a sense of distance and depth to a stationary object.

  Further, according to the present embodiment, since the position of the virtual light source is the position of the streetlight, for example, at night, it is possible to grasp the sense of distance and depth to a moving body or a stationary object more intuitively.

  Further, according to the present embodiment, the blind spot area of the driver is a blind spot area around the intersection of the intersection on the front side of the vehicle, the real image capturing means is provided at the front end of the vehicle, and the vehicle is in the intersection. When approaching, a real image around the intersection of the intersection that can be imaged from the front end of the vehicle is captured. Accordingly, the driver can obtain information on the left and right intersections at intersections where collision accidents frequently occur, and safer driving is possible.

  Further, according to the present embodiment, since the display of the virtual shadow image is stopped after the host vehicle reaches the cue position where the driver's line of sight is improved at the intersection, the driver stops the host vehicle at the cue position. It is possible to grasp the road situation with one's own eyes while grasping a certain thing.

  Further, according to the present embodiment, the moving body is detected from the actual images captured by the cameras 2, 4, 6, and the type and / or moving speed and / or moving direction of the moving body is detected, Since the display of the virtual shadow image is changed according to the type and / or moving speed and / or moving direction of the moving object, the driver obtains information (type, speed, moving direction) about the moving object from the virtual shadow image. It is possible to grasp the sense of distance and depth to the moving body and the state of the moving body with higher accuracy.

  In addition, according to the present embodiment, after the moving body passes the intersection, the display of the virtual shadow image of the moving body is stopped, so that the driver can safely move the moving body on which the virtual shadow image is not displayed to the host vehicle. It can be grasped that it is a moving body.

  Further, according to the present embodiment, when a moving body is detected, it is detected whether or not the moving body approaches or meanders at high speed, and approaches or meanders and approaches at high speed. When a moving object is detected, such a moving object displays a shadow in a color different from that of the other moving objects, so the driver is approaching at a higher speed or meandering with more attention. It can be easily grasped that a moving body exists.

  According to the present embodiment, the presence or absence of a moving body approaching in a plurality of columns is detected, and when a moving body approaching in a plurality of columns is detected, the movement approaching in those columns Since the shadow of the following car in the body is displayed differently from the shadow of the leading car, it can be easily grasped that the moving body approaches in a column, and the following moving body also has a sense of distance. And a sense of depth.

1 is an overall configuration diagram of a vehicle to which a vehicle surrounding monitoring device according to first and second embodiments of the present invention is applied. FIG. It is a figure which shows an example (a) of the imaging range by the camera of the surroundings monitoring apparatus for vehicles by 1st and 2nd embodiment of this invention, and an example (b) of the imaged image. It is a figure for demonstrating the driving action at the time of the crossing approach by a driver. It is a bird's-eye view of the road and vehicles for explaining the cueing position of the own vehicle. It is a flowchart which shows the process of whether to display the distance grasping | ascertainment virtual image of the vehicle periphery monitoring apparatus by 1st and 2nd embodiment of this invention. 1 is a block diagram illustrating a configuration of a vehicle surroundings monitoring device according to a first embodiment of the present invention. FIG. 3 is a flowchart showing display processing of a distance grasping virtual image of the vehicle surrounding monitoring apparatus according to the first embodiment of the present invention. FIG. It is a figure which shows the imaging range by the camera of the surroundings monitoring apparatus for vehicles by 1st Embodiment of this invention, an example (a) of a moving body, and an example (b) of the image | video on which the virtual image was superimposed on the real image. It is a block diagram which shows the structure of the vehicle periphery monitoring apparatus by 2nd Embodiment of this invention. It is a flowchart which shows the display processing of the distance grasping | ascertainment virtual image of the vehicle periphery monitoring apparatus by 2nd Embodiment of this invention. It is a figure which shows the imaging range by the camera of the surroundings monitoring apparatus for vehicles by 2nd Embodiment of this invention, an example (a) of a moving body, and an example (b) of the image | video on which the virtual image was superimposed on the real image.

Explanation of symbols

1 Vehicle 2, 4, 6 Monitoring camera 7 Laser radar 10 Vehicle speed sensor 11 Gyro 12 GPS device 13 Navigation device 14 Image display monitor 18 ECU

Claims (14)

A vehicle surrounding monitoring device that monitors a blind spot area in an area that extends from side to side in front of a driver,
Real image capturing means provided at the front of the vehicle so as to capture a real image of the blind spot area;
Virtual light source setting means for setting a virtual light source that gives a virtual shadow to a stationary object and / or a moving object imaged by the real image imaging means;
Virtual shadow image generation means for generating a shadow when the stationary object and / or the moving body receives light from the virtual light source set by the virtual light source setting means as a virtual shadow image;
Display means for displaying the generated virtual shadow image superimposed on the actual image so as to extend from the ground contact position of the stationary object and / or the moving body to the ground;
A vehicle surrounding monitoring device characterized by comprising: Furthermore, it has virtual shadow image defining means for defining the direction and size of the shadow of the virtual wall image based on the position of the virtual light source and the shape of the moving body and / or stationary object,
The display means extends the virtual shadow image of the stationary object and / or the moving object from the road contact surface position of the stationary object and / or the moving object with the direction and size of the shadow defined by the virtual shadow image defining means. The vehicle surrounding monitoring apparatus according to claim 1, wherein the vehicle surrounding monitoring apparatus displays the image so as to overlap the real image.   The vehicle surrounding monitoring apparatus according to claim 1 or 2, wherein the position of the virtual light source is a position of the sun calculated from latitude, longitude, and time.   The vehicle surrounding monitoring device according to claim 1 or 2, wherein the position of the virtual light source is a position set above the position of a road surface that is a predetermined distance away from an intersection where the host vehicle enters.   The vehicle surrounding monitoring apparatus according to claim 4, wherein the predetermined distance from the intersection is a distance to an appropriate road surface position for determining whether the own vehicle starts or stops when another vehicle approaches.   The vehicle surrounding monitoring apparatus according to claim 4, wherein the predetermined distance from the intersection can be arbitrarily set by a driver.   7. The display unit according to claim 4, wherein the display unit changes a display form of the virtual shadow image before and after the passage of a predetermined distance away from the intersection and / or during the passage. The vehicle surrounding monitoring device described in 1.   The vehicle surrounding monitoring apparatus according to claim 1, wherein the position of the virtual light source is a position of a streetlight. The driver's blind spot area is a blind spot area around the intersection of the intersection on the front side of the vehicle,
9. The real image pick-up means is provided at a front end of a vehicle, and picks up a real image around an intersection of an intersection that can be imaged from the front end of the vehicle when the vehicle approaches an intersection. The vehicle surrounding monitoring apparatus according to claim 1. Furthermore, it has a cue position calculation means for calculating a cue position where the driver's line of sight is improved at the intersection,
The vehicle surrounding monitoring apparatus according to claim 9, wherein the display unit stops displaying the virtual shadow image after the host vehicle reaches the cueing position. Furthermore, a moving body detecting means for detecting a moving body from within the real image captured by the real image capturing means,
A moving body state detecting means for detecting the type and / or moving speed and / or moving direction of the moving body when the moving body is detected by the moving body detecting means;
The vehicle surrounding monitoring device according to any one of claims 1 to 10, wherein the display means changes the display of the virtual shadow image according to the type and / or moving speed and / or moving direction of the moving body. .   The vehicle surrounding monitoring apparatus according to claim 11, wherein the display unit stops displaying the virtual shadow image of the moving body after the moving body passes an intersection. Furthermore, when the moving body is detected by the moving body detecting means, it has second moving body state detecting means for detecting whether or not the moving body is approaching at high speed or meandering.
The vehicle according to claim 11 or 12, wherein the display means displays a shadow in a color different from that of other moving bodies when a moving body approaching at high speed or meandering is approaching. Ambient monitoring device. Furthermore, it has the 3rd moving body state detection means which detects the presence or absence of the moving body which approaches by the said moving body detection means in several columns,
When the moving body approaching in a plurality of columns is detected by the third moving body state detection means, the display means displays the shadow of the succeeding vehicle among the moving bodies approaching in the column. The vehicle surrounding monitoring device according to claim 11, wherein the vehicle surrounding monitoring device displays the image so as to be different from the shadow of the vehicle.

Images