JP5158051B2 - Driving assistance device - Google Patents

Driving assistance device Download PDF

Info

Publication number
JP5158051B2
JP5158051B2 JP2009216879A JP2009216879A JP5158051B2 JP 5158051 B2 JP5158051 B2 JP 5158051B2 JP 2009216879 A JP2009216879 A JP 2009216879A JP 2009216879 A JP2009216879 A JP 2009216879A JP 5158051 B2 JP5158051 B2 JP 5158051B2
Authority
JP
Japan
Prior art keywords
image
vehicle
display
unit
displays
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2009216879A
Other languages
Japanese (ja)
Other versions
JP2011066763A (en
Inventor
貴士 清水
Original Assignee
三菱自動車工業株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱自動車工業株式会社 filed Critical 三菱自動車工業株式会社
Priority to JP2009216879A priority Critical patent/JP5158051B2/en
Publication of JP2011066763A publication Critical patent/JP2011066763A/en
Application granted granted Critical
Publication of JP5158051B2 publication Critical patent/JP5158051B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Description

  The present invention relates to a driving support device.

  Images of the surroundings of the vehicle are captured by the cameras provided on the front, rear, left and right of the vehicle, and the images captured by these cameras are converted into a bird's-eye view image viewed from above the vehicle as a virtual viewpoint by the image processing unit. A technique for displaying on a displayed display is known (see, for example, Patent Document 1). According to the technology, the driver's field of view can be expanded and the convenience of the driver can be improved.

JP 2006-341441 A

By the way, when the driver parks the host vehicle in the parking lot, the driver looks around the host vehicle from the driver's seat. For this reason, the bird's-eye view image displayed on the display has a different viewpoint from the image actually seen from the driver's seat.
In view of the above circumstances, it is an object of the present invention to provide a driving support device that can improve a driver's convenience by expanding a driver's field of view while suppressing a driver's discomfort.

In order to achieve the above-described problem, the driving support device includes an imaging unit that captures an image of the surroundings of the vehicle, and an image captured by the imaging unit. and converts the image, an image generator for generating a continuous image of a visual direction is continuously changed around the virtual viewpoint, and a display unit for displaying the continuous image generated by the image generating unit, outside An information receiving unit that receives designation information for designating the imaging position of the image capturing unit, and the image generation unit determines the imaging position received by the information receiving unit as a viewpoint of an occupant seated in a seat closest to the imaging position. A designated image converted into an image outside the vehicle viewed as a virtual viewpoint is generated, and the display unit displays the designated image generated by the image generation unit.

In the driving support device, the image generation unit generates a superimposed image obtained by superimposing an image inside the vehicle on an image outside the vehicle as a transmission image, and the display unit generates the superimposed image generated by the image generation unit. May be displayed.
The driving support device includes a sound collecting unit that collects sound outside the vehicle, and an image captured by the image capturing unit of sound collected by the sound collecting unit when the periphery of the vehicle is imaged by the image capturing unit. And a sound generation unit that is generated in the vehicle when the display unit displays.

  According to the above driving assistance device, the driver's convenience can be improved by expanding the driver's field of view while suppressing the driver's discomfort.

It is a block diagram which shows schematic structure of the driving assistance device which concerns on one Embodiment of this invention. It is a top view which shows the own vehicle provided with the driving assistance device which concerns on this embodiment. It is a top view which shows the scene which parks the own vehicle in a parking lot. It is a flowchart for demonstrating operation | movement of a driving assistance device. It is an image figure which shows the image | video displayed on a display. It is an image figure which shows the image | video displayed on a display. It is an image figure which shows the image | video displayed on a display. It is an image figure which shows the image | video displayed on a display. It is an image figure which shows the image | video displayed on a display. It is an image figure which shows the image | video displayed on a display. It is an image figure which shows the image | video displayed on a display. It is an image figure which shows the image | video displayed on a display. It is an image figure which shows the image | video displayed on a display.

Embodiments of the present invention will be described below.
FIG. 1 is a block diagram showing a schematic configuration of a driving support apparatus 10 according to an embodiment of the present invention. FIG. 2 is a plan view showing the host vehicle 100 including the driving support device 10 according to the present embodiment. As shown in these drawings, the driving support apparatus 10 according to the present embodiment includes an ECU (Electronic Control Unit) 80 that controls the entire driving support apparatus 10 and a plurality of (this embodiment) that images the surroundings of the host vehicle 100. 4) cameras (imaging units) 41, 42, 43, 44, an image processing unit (image generation unit) 30 for processing images taken by the cameras 41, 42, 43, 44, and an image processing unit 30. A display (display unit) 20 for displaying the image processed by the above, and a plurality (four in this embodiment) of microphones (sound collecting units) 61, 62 for collecting sounds around the host vehicle 100, 63, 64, a sound processing unit 50 for processing the sound collected by the microphones 61, 62, 63, 64, and a speaker (sound generator) 70 for emitting the sound processed by the sound processing unit 50. There.

When the ECU 80 is in a predetermined operating state, for example, when the shift position is switched from the P (parking) range to the R (reverse) range, or when the shift position is switched from the P range to the D (drive) range. Then, the driving support device 10 is operated.
Further, the ECU 80 switches the operation of the driving support device 10 in accordance with the operation of the operation keys arranged around or in the display 20. For example, a display unit (information receiving unit) 23 (see FIG. 5 ), which will be described later, serves as a touch panel for designating the position of the image outside the vehicle, and will be described later when the position of the image outside the vehicle is designated by the touch panel. The video to be displayed on the display unit 21 (see FIG. 5 ) is switched to the video at the designated position.

2, the camera 41 and microphone 61 are disposed in the vehicle width direction central portion of the vehicle front, the camera 42 and microphone 62, distribution in the side mirror 10 1 of the vehicle left side (passenger seat side surface) It is installed. The camera 43 and the microphone 63 are disposed at the center in the vehicle width direction on the rear surface of the vehicle, and the camera 44 and the microphone 64 are disposed on the side mirror 102 on the right side of the vehicle (side surface of the driver's seat).

The cameras 41 to 44 are wide-angle shooting cameras having an angle of view of 180 °. The camera 41 images the front of the vehicle with an angle of view of 180 °. The camera 42 captures the left side of the vehicle with an angle of view of 180 °. The camera 43 captures the rear of the vehicle with an angle of view of 180 °. Further, the camera 44 images the right side of the vehicle with an angle of view of 180 °.
The image processing unit 30 performs coordinate conversion (viewpoint conversion) on images captured by the cameras 41 to 44, and creates an image around the vehicle viewed from the driver's seat as a virtual viewpoint. The image processing unit 30 has a conversion table storing the pixel arrangement correspondence between the captured image and the image after coordinate conversion for each of the cameras 41 to 44, and the image captured by each of the cameras 41 to 44 is stored. Then, coordinate conversion is performed based on each conversion table. Further, the image processing unit 30 sequentially transmits the created image data to the display 20.

The display 20 sequentially displays images created by the image processing unit 30 based on the image data sequentially transmitted from the image processing unit 30.
The microphones 61 to 64 are unidirectional microphones. The microphone 61 collects sound in front of the vehicle, and the microphone 62 collects sound on the left side of the vehicle. The microphone 63 collects sound behind the vehicle, and the microphone 64 collects sound on the right side of the vehicle.

  The sound processing unit 50 generates sounds collected by the microphones 61 to 64 from the speaker 70 at a predetermined timing. Specifically, when an image captured by the camera 41 is displayed on the display 20, a sound collected by the microphone 61 is generated from the speaker 70, and an image captured by the camera 42 is displayed on the display 20. Sound collected by the microphone 62 is generated from the speaker 70. In addition, when the image captured by the camera 43 is displayed on the display 20, the sound collected by the microphone 63 is generated from the speaker 70. Further, when the image captured by the camera 44 is displayed on the display 20, the sound collected by the microphone 64 is generated from the speaker 70.

  Next, the operation of the driving support device 10 will be described. FIG. 3 is a plan view showing a scene in which the host vehicle 100 is parked in the parking lot. As shown in this figure, the parking area P1 located behind the host vehicle 100 is an area where the host vehicle 100 is parked. Vehicles 103 and 104 are parked in the parking areas P2 and P3 on the left and right sides of the parking area P1, respectively. In addition, vehicles 101 and 102 are parked in parking areas P4 and P5 arranged in front and rear in the front left direction of the parking area P1, respectively. In addition, vehicles 105 and 106 are parked in parking areas P6 and P7 arranged in front and rear in front of the parking area P1, respectively.

  FIG. 4 is a flowchart for explaining the operation of the driving support device 10. As shown in this flowchart, when the start switch of the host vehicle 100 is on, the processing routine is started and the routine proceeds to step 100. In step 100, the ECU 80 determines whether or not the shift position has been switched from the P range, D range, or N range to the R range, or whether or not the shift position has been switched from the P range or N range to the D range. If the determination is affirmative, the routine proceeds to step 102.

  In step 102, the ECU 80 turns on the operation switch of the driving support device 10 and proceeds to step 104. In step 104, the camera 41 in front of the vehicle images the front of the vehicle, and the left camera 42 images the left side of the vehicle. The image processing unit 30 performs coordinate conversion on the images taken by the cameras 41 and 42 and combines them, and an image in the left front of the vehicle when the driver's seat is viewed as a virtual viewpoint and an overhead image when the upper side of the host vehicle is viewed as a virtual viewpoint. And create. Here, the image processing unit 30 alternately arranges pixels of the former and the latter at the joint portion between the image photographed by the camera 41 and the image photographed by the camera 42, so that the image at the joint portion is obtained. An image (so-called seamless image) with a reduced discontinuity is created. The display 20 displays the image created by the image processing unit 30.

A microphone 61 in front of the vehicle collects sound in front of the vehicle, and a microphone 62 on the left side of the vehicle collects sound in the left side of the vehicle. The sound processing unit 50 synthesizes the sounds collected by the microphones 61 and 62 and causes the speaker 70 to generate the image generated by the image processing unit 30 at the same time as it is displayed on the display 20.
FIG. 5 is an image diagram showing a video displayed on the display 20 in this step. As shown in this figure, the display 20 displays not only an out-of-vehicle image I1 in front of the vehicle left when the driver's seat is viewed as a virtual viewpoint, but also an in-vehicle image I2 in front of the virtual viewpoint set in the driver's seat. A display unit 21, a display unit 22 that displays the bird's-eye view video I4, and a display unit 23 that displays position information indicating the shooting position are provided.

The display unit 21 displays the in-vehicle image I2 on the entire surface of the display unit 21, and displays the out-of-vehicle image I1 on the left side of the in-vehicle image I2. Since the vehicles 101 and 102 are located on the left front side of the host vehicle 100, these are displayed on the display unit 21 as the vehicle outside image I1. Here, the display unit 21 displays a region I3 that overlaps the vehicle exterior image I1 in the vehicle interior image I2 as a transmission image.
Further, the display unit 22 displays a bird's-eye view video I4 on the left front side of the vehicle. Further, the display unit 23 displays a portion corresponding to the left front of the vehicle that is the photographing position in a color different from other portions such as red.

Next, in step 106, the camera 42 on the left side of the vehicle images the left side of the vehicle. The image processing unit 30 performs coordinate conversion on the image captured by the camera 42 and creates an image on the left side of the vehicle when the driver's seat is viewed as a virtual viewpoint and an overhead image on the left side of the vehicle when the upper side of the vehicle is viewed as a virtual viewpoint. . The display 20 displays the image created by the image processing unit 30.
Further, the microphone 62 on the left side of the vehicle collects the sound on the left side of the vehicle. The sound processing unit 50 generates the sound collected by the microphone 62 from the speaker 70 at the same time when the image created by the image processing unit 30 is displayed on the display 20.

  FIG. 6 is an image diagram showing a video displayed on the display 20 in this step. As shown in this figure, the display unit 21 of the display 20 superimposes the vehicle exterior image I1 on the left side of the vehicle viewed from the driver's seat as a virtual viewpoint and the vehicle interior image I2 on the left side of the virtual viewpoint set in the driver's seat. To display. The display unit 21 displays the vehicle interior image I2 on the entire surface of the display 20, and displays the vehicle exterior image I1 superimposed on the left side of the vehicle interior image I2. In addition, the display unit 21 displays a region I3 that overlaps the vehicle exterior image I1 in the vehicle interior image I2 as a transmission image. Here, in the display unit 21, a part of the vehicle 102 located on the left front side of the host vehicle 100 is slightly reflected on the right end of the vehicle outside image I1.

  Here, when moving from step 104 to step 106, the image processing unit 30 continuously changes the visual direction from the front left side of the vehicle to the left side of the vehicle around the virtual viewpoint set in the driver's seat. Is generated. And the display part 21 of the display 20 displays the continuous image which changes continuously from the image of the vehicle left front shown in FIG. 5 to the image of the vehicle left shown in FIG.

Further, the display unit 22 displays a bird's-eye view video I4 on the left side of the vehicle. Further, the display unit 23 displays a portion corresponding to the left side of the vehicle, which is the shooting position, in a color different from other portions such as red.
Next, in step 108, the camera 42 on the left side of the vehicle images the left side of the vehicle, and the camera 43 on the rear side of the vehicle images the rear side of the vehicle. The image processing unit 30 performs coordinate conversion on the images taken by the cameras 42 and 43, combines them, an image on the left rear of the driver's seat viewed as a virtual viewpoint, and an overhead view viewed above the host vehicle 100 as a virtual viewpoint. Create images and. Here, the image processing unit 30 alternately arranges pixels of the former and the latter at the joint portion between the image photographed by the camera 42 and the image photographed by the camera 43, so that the image at the joint portion is obtained. Create an image with reduced discontinuity. The display 20 displays the image created by the image processing unit 30.

Further, the microphone 62 of the left side of the vehicle collects the sound of the left of the vehicle, the vehicle rear of the microphone 63 collects the sound of the vehicle rear. The sound processing unit 50 synthesizes the sounds collected by the microphones 62 and 63 and causes the speaker 70 to generate the image created by the image processing unit 30 at the same time as it is displayed on the display 20.
FIG. 7 is an image diagram showing a video displayed on the display 20 in this step. As shown in this figure, the display unit 21 of the display 20 superimposes the vehicle rear left image I1 viewed from the driver's seat as a virtual viewpoint and the vehicle rear image I2 rear left from the virtual viewpoint set in the driver's seat. To display. The display unit 21 displays the vehicle interior image I2 on the entire surface of the display 20, and displays the vehicle exterior image I1 superimposed on the left side of the vehicle interior image I2. In addition, the display unit 21 displays a region I3 that overlaps the vehicle exterior image I1 in the vehicle interior image I2 as a transmission image. Here, since the vehicle 103 is located on the left rear side of the host vehicle 100, the display unit 21 displays the vehicle 103 as the vehicle outside image I1.

  Here, when moving from step 106 to step 108, the image processing unit 30 continuously changes the visual direction from the left side of the vehicle to the left rear side of the vehicle around the virtual viewpoint set in the driver's seat. Is generated. And the display part 21 of the display 20 displays the continuous image which changes continuously from the image | video of the vehicle left side shown in FIG. 6 to the image | video of the vehicle left back shown in FIG.

Further, the display unit 22 displays a bird's-eye view video I4 on the left rear side of the vehicle. Further, the display unit 23 displays a portion corresponding to the left rear of the vehicle that is the shooting position in a color different from other portions such as red.
Next, in step 110, the camera 42 on the left side of the vehicle images the left side of the vehicle, and the camera 43 on the rear side of the vehicle images the rear side of the vehicle. The image processing unit 30 performs coordinate conversion on the image captured by the camera 42 and the image captured by the camera 43, combines them, and displays an image on the left rear side of the vehicle viewed from the driver's seat as a virtual viewpoint and above the host vehicle 100. A bird's-eye view image behind the vehicle viewed as a virtual viewpoint is created. Here, the image processing unit 30 alternately arranges pixels of the former and the latter at the joint portion between the image photographed by the camera 42 and the image photographed by the camera 43, so that the image at the joint portion is obtained. Create an image with reduced discontinuity. The display 20 displays the image created by the image processing unit 30.

The microphone 62 on the left side of the vehicle collects sound on the left side of the vehicle, and the microphone 63 on the rear side of the vehicle collects sound on the rear side of the vehicle. The sound processing unit 50 synthesizes the sounds collected by the microphones 62 and 63 and causes the speaker 70 to generate the image created by the image processing unit 30 at the same time as it is displayed on the display 20.
FIG. 8 is an image diagram showing a video displayed on the display 20 in this step. As shown in this figure, the display unit 21 of the display 20 superimposes the vehicle exterior image I1 on the left rear side of the vehicle viewed from the driver's seat as a virtual viewpoint and the vehicle interior image I2 on the rear left side of the virtual viewpoint set in the driver's seat. To display. The display unit 21 displays the vehicle interior image I2 on the entire surface of the display 20, and displays the vehicle exterior image I1 so as to overlap the vehicle interior image I2. In addition, a region I3 that overlaps the vehicle exterior image I1 in the vehicle interior image I2 is displayed as a transmission image. Here, in the display unit 21, a part of the vehicle 103 located on the left rear side of the host vehicle 100 is slightly reflected on the left end of the outside-vehicle image I1.

  Here, when moving from step 108 to step 110, the image processing unit 30 continuously changes the visual direction from the vehicle rear left side to the vehicle rear left side around the virtual viewpoint set in the driver's seat. Is generated. And the display part 21 of the display 20 displays the continuous image which changes continuously from the image of the vehicle left back shown in FIG. 7 to the image of the vehicle back left shown in FIG.

Further, the display unit 22 displays a bird's-eye view image I4 on the left rear side of the vehicle. Further, the display unit 23 displays a portion corresponding to the left rear side of the vehicle, which is the shooting position, in a color different from other portions such as red.
Next, in step 112, the camera 43 behind the vehicle images the rear of the vehicle, and the camera 44 on the right side of the vehicle images the right side of the vehicle. The image processing unit 30 performs coordinate conversion on the image captured by the camera 43 and the image captured by the camera 44, combines them, and displays an image on the right rear side of the vehicle viewed from the driver's seat as a virtual viewpoint, and above the host vehicle 100. A bird's-eye view image behind the vehicle viewed as a virtual viewpoint is created. Here, the image processing unit 30 alternately arranges pixels of the former and the latter at the joint portion between the image photographed by the camera 43 and the image photographed by the camera 44, so that the image at the joint portion is obtained. Create an image with reduced discontinuity. The display 20 displays the image created by the image processing unit 30.

A microphone 63 at the rear of the vehicle collects sound behind the vehicle, and a microphone 64 at the right side of the vehicle collects sound at the right side of the vehicle. The sound processing unit 50 synthesizes the sounds collected by the microphones 63 and 64 and causes the speaker 70 to generate the image created by the image processing unit 30 at the same time as it is displayed on the display 20.
FIG. 9 is an image diagram showing a video displayed on the display 20 in this step. As shown in the figure, the display unit 21 of the display 20 superimposes the vehicle exterior image I1 on the right rear side of the vehicle viewed from the driver's seat as a virtual viewpoint and the vehicle interior image I2 on the rear right side of the virtual viewpoint set in the driver's seat. To display. The display unit 21 displays the vehicle interior image I2 on the entire surface of the display 20, and displays the vehicle exterior image I1 so as to overlap the vehicle interior image I2. In addition, the display unit 21 displays a region I3 that overlaps the vehicle exterior image I1 in the vehicle interior image I2 as a transmission image. Here, in the display unit 21, a part of the vehicle 104 located on the right rear side of the host vehicle 100 is slightly reflected on the right end of the vehicle outside image I <b> 1.

  Here, when moving from step 110 to step 112, the image processing unit 30 continuously changes the visual direction from the vehicle rear left side to the vehicle rear right side around the virtual viewpoint set in the driver's seat. Is generated. And the display part 21 of the display 20 displays the continuous image which changes continuously from the image | video of the vehicle back left side shown in FIG. 8 to the image | video of the vehicle back right side shown in FIG.

The display unit 22 displays a bird's-eye view video I4 behind the vehicle. Further, the display unit 23 displays a portion corresponding to the right rear side of the vehicle that is the shooting position in a color different from other portions such as red.
Next, in step 114, the camera 43 behind the vehicle images the rear of the vehicle, and the camera 44 on the right side of the vehicle images the right side of the vehicle. The image processing unit 30 performs coordinate conversion on the image captured by the camera 43 and the image captured by the camera 44, combines them, and displays an image on the right rear side of the vehicle viewed from the driver's seat as a virtual viewpoint and above the host vehicle 100. A bird's-eye view image on the left side of the vehicle viewed as a virtual viewpoint is created. Here, the image processing unit 30 alternately arranges pixels of the former and the latter at the joint portion between the image photographed by the camera 43 and the image photographed by the camera 44, so that the image at the joint portion is obtained. Create an image with reduced discontinuity. The display 20 displays the image created by the image processing unit 30.

Further, the vehicle rear of the microphone 6 3 collects the sound of the vehicle after the side, a microphone 6 4 vehicle right to collect the sound of the vehicle ahead. Audio processing unit 50 synthesizes the sound microphone 64,6 3 collects, image created image processing unit 30 is generated from the speaker 70 at the same time as being displayed on the display 20.
FIG. 10 is an image diagram showing a video displayed on the display 20 in this step. As shown in this figure, the display unit 21 of the display 20 superimposes a vehicle exterior image I1 when the driver's seat is viewed as a virtual viewpoint and a vehicle interior image I2 when the driver's seat is set to the driver's seat and the vehicle is viewed right behind the virtual viewpoint. To display. The display unit 21 displays the vehicle interior image I2 on the entire surface of the display 20, and displays the vehicle exterior image I1 superimposed on the right side of the vehicle interior image I2. In addition, the display unit 21 displays a region I3 that overlaps the vehicle exterior image I1 in the vehicle interior image I2 as a transmission image. Here, the display unit 21, the vehicle 104 is located in the left rear of the vehicle 100, displayed as outside image I1.

  Here, when moving from step 112 to step 114, the image processing unit 30 continuously changes the visual direction from the vehicle rear right side to the vehicle right rear centering on the virtual viewpoint set in the driver's seat. Is generated. And the display part 21 of the display 20 displays the continuous video which changes continuously from the image | video of the vehicle back right shown in FIG. 9 to the image | video of the vehicle right back shown in FIG.

The display unit 22 displays a bird's-eye view video I4 on the right side of the vehicle. Further, the display unit 23 displays a portion corresponding to the right rear side of the vehicle that is the shooting position in a color different from other portions such as red.
Next, in step 116, the vehicle right side camera 44 images the vehicle right side. The image processing unit 30 performs coordinate conversion on an image captured by the camera 44, and an image on the right side of the vehicle when the driver's seat is viewed as a virtual viewpoint, and an overhead image on the right side of the vehicle when the upper side of the host vehicle 100 is viewed as a virtual viewpoint. Create The display 20 displays the image created by the image processing unit 30.

In addition, the microphone 64 of the vehicle right side is to gather the sound of the vehicle right side. Audio processing unit 50, a sound microphone 64 collects, image created image processing unit 30 is generated from the speaker 70 at the same time as being displayed on the display 20.
FIG. 11 is an image diagram showing a video displayed on the display 20 in this step. As shown in this figure, the display unit 21 of the display 20 superimposes an in-vehicle image I1 on the right side of the vehicle viewed from the driver's seat as a virtual viewpoint and an in-vehicle image I2 on the right side of the virtual viewpoint set in the driver's seat. To display. The display unit 21 displays the vehicle interior image I2 on the entire surface of the display 20, and displays the vehicle exterior image I1 superimposed on the right side of the vehicle interior image I2. In addition, the display unit 21 displays a region I3 that overlaps the vehicle exterior image I1 in the vehicle interior image I2 as a transmission image. Here, the display unit 21, a part of the vehicle 105 located in the right front of the own vehicle 100 is reflected on the left end of the slightly outside the image I1.

  Here, when moving from step 114 to step 116, the image processing unit 30 continuously changes the visual direction from the vehicle right rear side to the vehicle right side around the virtual viewpoint set in the driver's seat. Is generated. And the display part 21 of the display 20 displays the continuous image which changes continuously from the image | video of the vehicle right back shown in FIG. 10 to the image | video of the vehicle right side shown in FIG.

The display unit 22 displays a bird's-eye view video I4 on the right side of the vehicle. Further, the display unit 23 displays a portion corresponding to the right side of the vehicle that is the shooting position in a color different from other portions such as red.
Next, in step 118, the camera 44 on the right side of the vehicle images the right side of the vehicle, and the camera 41 on the front side of the vehicle images the front side of the vehicle. The image processing unit 30 performs coordinate transformation on the image captured by the camera 44 and the image captured by the camera 41, combines them, and displays an image on the right front of the vehicle viewed from the driver's seat as a virtual viewpoint, and above the host vehicle 100. And a bird's-eye view image in front of the vehicle viewed as a virtual viewpoint. The display 20 displays the image created by the image processing unit 30.

Further, the microphone 64 on the right side of the vehicle collects the sound on the right side of the vehicle, and the microphone 61 on the front side of the vehicle collects the sound on the front side of the vehicle. The sound processing unit 50 synthesizes the sounds collected by the microphones 64 and 61 and causes the speaker 70 to generate the image generated by the image processing unit 30 at the same time as it is displayed on the display 20.
FIG. 12 is an image diagram showing a video displayed on the display 20 in this step. As shown in this figure, the display unit 21 of the display 20 superimposes a vehicle exterior image I1 when the driver's seat is viewed as a virtual viewpoint and a vehicle interior image I2 when the driver's seat is set in the driver's seat and the virtual front view. To display. The display unit 21 displays the vehicle interior image I2 on the entire surface of the display 20, and displays the vehicle exterior image I1 superimposed on the right side of the vehicle interior image I2. In addition, the display unit 21 displays a region I3 that overlaps the vehicle exterior image I1 in the vehicle interior image I2 as a transmission image. Here, the display unit 21 displays the vehicle 102 located in the right front of the host vehicle 100 as the outside image I1.

  Here, when moving from step 116 to step 118, the image processing unit 30 continuously changes the visual direction from the right side of the vehicle to the right front side of the vehicle with the virtual viewpoint set at the driver's seat as the center. Is generated. And the display part 21 of the display 20 displays the continuous image which changes continuously from the image | video of the vehicle right side shown in FIG. 11 to the image | video of the vehicle right front shown in FIG.

Moreover, the display part 22 displays the bird's-eye view image I4 on the right front side of the vehicle. Further, the display unit 23 displays a portion corresponding to the right front of the vehicle that is the shooting position in a color different from other portions such as red.
Next, in step 120, the camera 41 in front of the vehicle images the front of the vehicle. The image processing unit 30 performs coordinate conversion on the image captured by the camera 41, and generates an image in front of the vehicle when the driver's seat is viewed as a virtual viewpoint and an overhead image in front of the vehicle when the upper side of the host vehicle 100 is viewed as a virtual viewpoint. create. The display 20 displays the image created by the image processing unit 30.

Further, the microphone 61 in front of the vehicle collects sound in front of the vehicle. The sound processing unit 50 generates sound from the speaker 70 simultaneously with the sound collected by the microphone 61 and the image created by the image processing unit 30 being displayed on the display 20.
Next, in step 122, the ECU 80 determines whether or not the touch panel of the display unit 23 has been operated. If the determination is affirmative, the ECU 80 proceeds to step 124, but if the determination is negative, the ECU 80 proceeds to step 122. 126.

  Here, the touch panel of the display unit 23 includes a plurality of operation keys that respectively specify vehicle front, vehicle left front, vehicle left, vehicle left rear, vehicle rear, vehicle right rear, vehicle right side, and vehicle right front. ing. In step 124, when the shooting position is designated by the operation key on the touch panel of the display unit 23, the ECU 80 displays the image of the shooting position when the seat closest to the shooting position is viewed as a virtual viewpoint, and the host vehicle 100. The image processing unit 300 creates a bird's-eye view of the shooting position viewed from above as a virtual viewpoint. For example, when an operation key that specifies the left rear side of the vehicle on the touch panel of the display unit 23 is operated, the ECU 80 displays an image of the rear left side of the vehicle when the passenger seat side rear seat is viewed as a virtual viewpoint and an upper side of the host vehicle 100. The image processing unit 300 is made to create a bird's-eye view image on the left rear side viewed as a virtual viewpoint.

  FIG. 13 is an image diagram showing a video displayed on the display 20 in this step. As shown in this figure, the display unit 21 of the display 20 includes a vehicle exterior video I1 with the passenger seat side rear seat as a virtual viewpoint, and a vehicle interior at the left rear of the virtual viewpoint set in the passenger seat side rear seat. The video I2 is overlaid and displayed. The display unit 21 displays the vehicle interior image I2 on the entire surface of the display 20, and displays the vehicle exterior image I1 so as to overlap the vehicle interior image I2. In addition, the display unit 21 displays a region I3 that overlaps the vehicle exterior image I1 in the vehicle interior image I2 as a transmission image. Here, in the display unit 21, a part of the vehicle 103 located on the left rear side of the host vehicle 100 is reflected on the left end of the outside-vehicle image I1.

Next, in step 126, the ECU 80 determines whether or not the shift position has been switched from the R range to the P range, D range or N range, or whether or not the shift position has been switched from the D range to the P range or N range. To do. If the determination is affirmative, the processing routine ends.
As described above, in the driving assistance device 10 according to the present embodiment, when the driver parks the host vehicle 100 in the parking space or when the host vehicle 100 starts from the parking space, the driver When looking around the surroundings for one round, images that pass through the host vehicle 100 and are perceived by the driver's vision are continuously displayed on the display 20. For this reason, the difference between the video displayed on the display 20 and the video around the vehicle that is actually captured by the driver's vision can be reduced compared to the bird's-eye view of the vehicle as a virtual viewpoint. A sense of incongruity can be reduced. Further, the blind spots in all directions around the vehicle can be confirmed on the video displayed on the display 20 without switching the shooting positions such as the side and the rear. Furthermore, when the images captured by the cameras 41 to 44 are combined, a process of relaxing discontinuities at the joints between the images to be combined is performed. Therefore, according to the driving support device 10 according to the present embodiment, the driver's convenience can be improved by widening the driver's field of view while suppressing the driver's discomfort.

  Further, in the driving support device 10 according to the present embodiment, when a shooting position is designated by the touch panel of the display unit 23, an image that can be captured by the sight of a passenger seated in a seat closest to the shooting position is displayed. Is displayed on the display 20. For this reason, when a plurality of occupants are on the host vehicle 100, the occupant who is seated in the most visible seat where he wants to see outside the vehicle advises the driver of the position of the obstacle in the location. The same effect can be obtained. Therefore, according to the driving support device 10 according to the present embodiment, the convenience of the driver can be further improved.

  Further, in the driving support device 10 according to the present embodiment, the outside image I1 viewed from the driver's seat as a virtual viewpoint and the in-vehicle image I2 viewed from the virtual viewpoint are displayed on the display 20 so as to overlap each other. A region I3 overlapping with the image I2 is displayed on the display 20 as a transmission image. In other words, the entire image captured by the driver's vision when the driver looks around the host vehicle 100 is displayed on the display 20. Therefore, according to the driving support device 10 according to the present embodiment, the driver can feel a sense of reality.

Further, in the driving support device 10 according to the present embodiment, the sound collected by the microphones 61 to 64 is generated from the speaker 70 at the same time when the vehicle outside image I1 is displayed on the display 20. Therefore, according to the driving support device 10 according to the present embodiment, the driver can feel a sense of reality.
Further, in the driving support device 10 according to the present embodiment, for example, the outside video I1 of the left front of the vehicle is displayed on the display 20, and at the same time, sounds collected from the left front of the vehicle by the microphones 61 and 62 are generated from the speaker 70. The sound in the same direction as the outside image I1 displayed on the display 20 is generated from the speaker 70. Therefore, according to the driving support device 10 according to the present embodiment, the sense of presence felt by the driver can be further enhanced.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention. For example, in the driving support device 10 according to the present embodiment, the vehicle outside image I1 around the host vehicle 100 is continuously displayed on the display 20. However, the range of the outside image I1 to be displayed on the display 20, such as not displaying the outside image I1 in front of the vehicle on the display 20, or only displaying the outside image I1 on the passenger side on the display 20, is appropriately selected. Can be set.

  In the present embodiment, the four cameras 41 to 44 are provided on the front, rear, left and right sides of the vehicle. However, the number of installed cameras and the installation positions are not limited thereto and can be changed as appropriate. In this embodiment, the cameras 41 to 44 are wide-angle cameras having an angle of view of 180 °. However, the camera is not limited thereto, and other cameras such as a camera having an angle of view smaller than 180 ° may be used. It may be changed. In the present embodiment, the four microphones 61 to 64 are provided on the front, rear, left and right sides of the vehicle. However, the number of microphones installed and the installation positions are not limited thereto and can be selected as appropriate.

  In the present embodiment, the ECU 80 assists driving when the shift position is switched from the P (parking) range to the R (reverse) range, or when the shift position is switched from the P range to the D (drive) range. The operation switch of the device 10 is turned on. However, for example, in a vehicle equipped with a manual transmission, when the N range is switched to the R range, or when the N range is switched to the LOW range, the operation switch of the driving support device 10 is turned ON. You can also

10 Driving support device 20 Display (display unit)
21-23 Display unit 30 Image processing unit (image generation unit)
41-44 camera (imaging part)
50 Audio processing units 61-64 Microphone (sound collecting unit)
70 Speaker (sound generator)
100 Vehicle 101-106 Vehicle

Claims (3)

  1. An imaging unit for imaging the surroundings of the vehicle;
    The image captured by the imaging unit is converted into an image around the vehicle viewed from the viewpoint of an occupant seated in the driver's seat as a virtual viewpoint, and the visual direction is continuously changed around the virtual viewpoint. An image generation unit for generating an image;
    A display unit for displaying the continuous image generated by the image generation unit;
    An information receiving unit for receiving designation information for designating an imaging position outside the vehicle,
    The image generation unit generates a designated image obtained by converting the imaging position received by the information reception unit into an image outside the vehicle viewed from a viewpoint of a passenger seated in a seat closest to the imaging position as a virtual viewpoint,
    The display unit is a driving support device that displays the designated image generated by the image generation unit .
  2. The image generation unit generates a superimposed image in which an image inside the vehicle is overlapped with an image outside the vehicle as a transmission image,
    The driving support device according to claim 1 , wherein the display unit displays the superimposed image generated by the image generation unit.
  3. A sound collection unit for collecting sounds outside the vehicle;
    A sounding unit that generates sound collected by the sound collecting unit when the periphery of the vehicle is imaged by the imaging unit, and is generated inside the vehicle when the display unit displays an image captured by the imaging unit;
    Driving support apparatus according to claim 1 or 2 comprising a.
JP2009216879A 2009-09-18 2009-09-18 Driving assistance device Active JP5158051B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2009216879A JP5158051B2 (en) 2009-09-18 2009-09-18 Driving assistance device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2009216879A JP5158051B2 (en) 2009-09-18 2009-09-18 Driving assistance device

Publications (2)

Publication Number Publication Date
JP2011066763A JP2011066763A (en) 2011-03-31
JP5158051B2 true JP5158051B2 (en) 2013-03-06

Family

ID=43952499

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2009216879A Active JP5158051B2 (en) 2009-09-18 2009-09-18 Driving assistance device

Country Status (1)

Country Link
JP (1) JP5158051B2 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5381847B2 (en) * 2010-03-24 2014-01-08 株式会社デンソー Vehicle display device
JP6014433B2 (en) * 2012-09-19 2016-10-25 富士通テン株式会社 Image processing apparatus, image processing method, and image processing system
JP5977130B2 (en) * 2012-09-25 2016-08-24 富士通テン株式会社 Image generation apparatus, image display system, and image generation method
JP6105244B2 (en) * 2012-09-28 2017-03-29 富士通テン株式会社 Image processing apparatus and image processing system
JP6130118B2 (en) * 2012-09-28 2017-05-17 富士通テン株式会社 Image processing system, image processing apparatus, image processing method, and program
CN105556956B (en) 2013-09-19 2019-01-22 富士通天株式会社 Video generation device, image display system, image generating method and image display method
KR20160111102A (en) 2015-03-16 2016-09-26 두산인프라코어 주식회사 Method of displaying a dead zone of a construction machine and apparatus for performing the same
JP6370833B2 (en) * 2016-05-16 2018-08-08 株式会社デンソーテン Image generation apparatus, image display system, and image generation method
JP2019074850A (en) * 2017-10-13 2019-05-16 株式会社デンソー Image processing apparatus
JP6568981B2 (en) * 2018-05-29 2019-08-28 株式会社デンソーテン Image display device, image display system, image display method, and program

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000217100A (en) * 1999-01-25 2000-08-04 Matsushita Electric Ind Co Ltd On-vehicle camera system
JP4537537B2 (en) * 2000-05-25 2010-09-01 パナソニック株式会社 Driving assistance device
JP4877447B2 (en) * 2004-08-31 2012-02-15 株式会社エクォス・リサーチ Vehicle peripheral image display device
JP5047650B2 (en) * 2007-03-06 2012-10-10 クラリオン株式会社 In-vehicle camera system

Also Published As

Publication number Publication date
JP2011066763A (en) 2011-03-31

Similar Documents

Publication Publication Date Title
US10479275B2 (en) Vehicle peripheral observation device
US9862319B2 (en) Vehicle peripheral observation device using cameras and an emphasized frame
EP2603402B1 (en) Method for displaying images on a display unit and driver assistance system
US8854466B2 (en) Rearward view assistance apparatus displaying cropped vehicle rearward image
KR101375944B1 (en) Parking assistance system
JP6077210B2 (en) Vehicle rearward monitoring device
JP6241687B2 (en) Driving assistance device
DE102006054162B4 (en) Environmental monitoring system for a vehicle
EP2464113B1 (en) Vehicle peripheral image generation device
DE60219730T2 (en) Display device for driving assistance
US9007462B2 (en) Driving assist apparatus, driving assist system, and driving assist camera unit
KR101342033B1 (en) Image generation device and image display system
JP4723703B2 (en) Vehicle driving support device
US9219887B2 (en) Drive assist device
JP5522485B2 (en) Parking assistance device
DE102006018370B4 (en) Parking assistance method and parking assistance device
JP4639753B2 (en) Driving assistance device
EP2739050B1 (en) Vehicle surroundings monitoring system
JP5035643B2 (en) Image display device
JP5099451B2 (en) Vehicle periphery confirmation device
WO2012169355A1 (en) Image generation device
JP4914458B2 (en) Vehicle periphery display device
JP3695319B2 (en) Vehicle periphery monitoring device
JP5684144B2 (en) Peripheral image generation method and apparatus
JP4642723B2 (en) Image generating apparatus and image generating method

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20111007

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20120824

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20120828

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20121019

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20121113

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20121126

R151 Written notification of patent or utility model registration

Ref document number: 5158051

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20151221

Year of fee payment: 3