JP2006054662A - Drive support device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a drive support device for presenting a user with the peripheral status of a vehicle driven by the user in a easily understandable fashion. <P>SOLUTION: The drive support device is provided with: a plurality of cameras 10<SB>1</SB>to 10<SB>4</SB>arranged to perform wide-angle photography by bearing their share of the full azimuth of the periphery of a vehicle; a panorama converting part 11 for converting a plurality of wide angle images obtained by performing wide angle photography with the plurality of cameras into a plurality of plane images without any distortion; a parallax measuring part 12 for generating a parallax image to which a distance to a subject is reflected, based on the parallax of a plurality of plane images obtained by converting the plurality of wide angle images by the panorama converting part; a spatial map generating part 13 for generating a spatial map where a vehicle and its periphery is viewed from sky based on the parallax image generated by the parallax measuring part; and display parts 21 and 22 for displaying the spatial map generated by the spatial map generating part. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a driving support apparatus that provides a user with information for supporting driving of a vehicle, and more particularly, to a technique for transmitting a situation around a vehicle to a driver by display.

  2. Description of the Related Art Conventionally, as one of driving support devices that support driving of a vehicle, a car navigation device that is mounted on a vehicle and provides information for supporting driving to a user is known. In this car navigation apparatus, a map is displayed on a display based on map information stored in advance. The driver can drive while confirming the position of the host vehicle by looking at the map displayed on the display, so that the driver can arrive at the destination reliably and in a short time without getting lost.

  In recent years, as such a driving support device, an in-vehicle camera device that displays an external image captured by a camera on a monitor inside the vehicle has been developed (see, for example, Patent Document 1). This in-vehicle camera apparatus corrects distortion of an image obtained by photographing with a fisheye camera, and cuts out a part of the corrected image according to the traveling state of the vehicle and displays the enlarged image. Specifically, a fisheye camera is connected to the image controller. The image controller includes a coordinate conversion circuit, an image extraction circuit, and an image expansion circuit. The distortion image signal from the fisheye camera is converted into a positive image signal by a coordinate conversion circuit. The image cut-out circuit is connected to a state detection unit for detecting the running state of the vehicle, and cuts out an arbitrary region in the distortion-corrected image according to an input signal from the state detection unit. The image decompression circuit decompresses the clipped image and displays it on the entire display screen of the monitor.

  As another driving support device, there is known a parking support device that displays a parking lot video that complements a video missing range due to a blind spot of a camera (see, for example, Patent Document 2). This parking assist device images the target parking lot by the on-board parking lot imaging means, images the surroundings of the vehicle by the on-vehicle vehicle surrounding imaging means, and the image becomes the blind spot of the vehicle body in the video around the vehicle. The image of the missing range is cut out from the image of the target parking lot, and the image around the vehicle is complemented and displayed. As a result, it is possible to display images around the vehicle with few missing parts due to blind spots of the camera, clearly displaying the white line that divides the parking range and the vehicle stop that is the stopping target, and the position of the parking lot and the vehicle The relationship becomes easier to grasp.

  As another driving support device, there is known a vehicle collision preventing device that detects an obstacle on the traveling path of the host vehicle and makes a collision determination (see, for example, Patent Document 3). In this collision prevention apparatus, a pair of images captured by a stereo optical system is processed by an image processing unit to calculate a three-dimensional distance distribution over the entire image, and a road shape or a three-dimensional three-dimensional object is calculated from the distance distribution information. When the position is detected at high speed, the collision determination unit sets the traveling region of the host vehicle based on the input data from the road shape, the vehicle speed sensor, the yaw rate sensor, and the rudder angle sensor detected by the image processing unit. All of the vehicles and obstacles in or within this travel area are extracted from the plurality of vehicles and obstacles detected by the image processing unit, and the possibility of collision of the own vehicle is extracted for all of the extracted vehicles and obstacles. A collision risk is calculated as an evaluation value, and if there is any one having a high collision risk, it is displayed on the display and a warning is given to the driver.

JP-A-9-202180 JP 2002-29314 A Japanese Patent Laid-Open No. 10-283593

  However, in the above-described conventional driving support apparatus, a camera must be installed in the parking lot, and basically, only the surrounding image centered on the own vehicle is displayed. Therefore, there is a problem that the user cannot intuitively grasp the state of the vehicle.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a driving support device that can easily present a state around the host vehicle to a user.

  A driving support apparatus according to the present invention includes a panorama conversion unit that shares all directions around a vehicle and converts a plurality of wide-angle images obtained by a plurality of cameras into a plurality of flat images without distortion, and a panorama conversion unit A parallax measurement unit that generates a parallax image reflecting the distance to the subject based on the parallax of a plurality of planar images obtained by the conversion in the above, and the vehicle and its surroundings based on the parallax image generated by the parallax measurement unit A spatial map generation unit that generates a spatial map viewed from the above, and the display unit displays the spatial map generated by the spatial map generation unit.

  According to the driving support apparatus according to the present invention, a spatial map in which the vehicle and its surroundings are viewed from the sky is generated and displayed based on images from a plurality of cameras that have taken wide-angle images while sharing all directions around the vehicle. The spatial map displayed on the display unit becomes a target for distinguishing obstacles, parked vehicles, and traveling vehicles that are obstacles to stopping or changing lanes, and the positional relationship with the host vehicle is known. It becomes easy. As a result, the state around the host vehicle can be presented to the user in an easy-to-understand manner, and the user's parking and lane change can be supported.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of the driving support apparatus according to Embodiment 1 of the present invention. This driving support device includes fisheye cameras 10 ₁ to 10 ₄ , a panorama conversion unit 11, a parallax measurement unit 12, a spatial map generation unit 13, an image integration unit 14, a position detection unit 15, a turn detection unit 16, and a brake detection unit 17. The vehicle speed detection unit 18, the handle angle detection unit 19, the image cutout unit 20, and the navigation device 21 are included. The position detection unit 15, the turn detection unit 16, the brake detection unit 17, the vehicle speed detection unit 18, and the steering wheel angle detection unit 19 correspond to the vehicle driving information acquisition unit of the present invention.

The fisheye cameras 10 ₁ to 10 ₄ correspond to a plurality of cameras of the present invention. Each of the fish-eye cameras 10 ₁ to 10 ₄ includes a fish-eye lens and a CCD element (both not shown), and distortion is caused by the light incident through the fish-eye lens being imaged on the surface of the CCD element. A fisheye image is formed. The fisheye image formed on the surface of the CCD element is sent to the panorama conversion unit 11.

For example, as shown in FIG. 2, the _four fisheye cameras 10 ₁ to 10 ₄ are mounted at the four corners of the vehicle 1, and are installed so that the imaging direction is a diagonal direction of the vehicle 1. Each of the fisheye cameras 10 ₁ to 10 ₄ has an angle of view of approximately 180 degrees. Therefore, the fish-eye camera 10 _1, the fisheye image including the subject a and b as shown in FIG. 3 (a) is obtained. Similarly, the fisheye camera 10 ₂ obtains a fisheye image including subjects b and c as shown in FIG. 3B, and the fisheye camera 10 ₃ obtains a subject c as shown in FIG. 3C. and fish-eye image is obtained containing d, the fisheye camera 10 _4, the fish-eye image including the subject d and a, as shown in FIG. 3 (d) is obtained. In addition, in the driving assistance device according to the first embodiment, four fisheye cameras are used, but the number of fisheye cameras is arbitrary as long as it is a configuration capable of photographing the periphery of the vehicle in all directions. Also, a wide-angle camera can be used instead of the fisheye camera.

Panorama converter 11, by which each of the four fisheye image transmitted from the fish-eye camera 10 ₁ to 10 ₄ performs predetermined arithmetic processing, and converts the four undistorted planar image. The four planar images obtained by the panorama conversion unit 11 are sent to the parallax measurement unit 12 and the image integration unit 14.

  The parallax measurement unit 12 measures the parallax on the four planar images sent from the panorama conversion unit 11 using the stereo method, and based on the measured parallax, the parallax image (distance distribution reflecting the distance to the subject) Image). This parallax image has a horizontal angle of view of 360 degrees, like a panoramic image described later. The parallax image obtained by the parallax measurement unit 12 is sent to the spatial map generation unit 13.

  The spatial map generation unit 13 generates a spatial map based on the parallax image sent from the parallax measurement unit 12. Here, the spatial map refers to a map obtained by photographing the vehicle 1 and its surroundings from the sky. The spatial map generation unit 13 acquires a distance from the parallax image to, for example, an obstacle, and draws the obstacle. Thereby, for example, a spatial map including the obstacle 2 existing around the host vehicle 1 as shown in FIG. 4 is obtained. The spatial map generated by the spatial map generation unit 13 is sent to the navigation device 21.

  The image integration unit 14 integrates the four planar images sent from the panorama conversion unit 11 and generates a panoramic image having a horizontal angle of view of 360 degrees as shown in FIG. The panoramic image generated by the image integration unit 14 is sent to the image cutout unit 20.

  The position detector 15 detects a set position of a shift lever (not shown) of the vehicle 1. The set position of the shift lever detected by the position detection unit 15 is sent to the image cutout unit 20 as a position signal. The turn detector 16 detects the set state of a turn indicator (not shown). The set state of the direction indicator detected by the turn detection unit 16 is sent to the image cutout unit 20 as a turn signal.

  The brake detection unit 17 detects a depression state of a brake (not shown). The brake depression state detected by the brake detection unit 17 is sent to the image cutout unit 20 as a brake signal. The vehicle speed detector 18 detects the moving speed of the vehicle 1. The moving speed of the vehicle detected by the vehicle speed detection unit 18 is sent to the image cutout unit 20 as a speed signal. The handle angle detector 19 detects an angle at which a handle (not shown) is rotated. The rotation angle of the handle detected by the handle angle detection unit 19 is sent to the image cutout unit 20 as a handle angle signal.

  The image cutout unit 20 converts the panoramic image sent from the image integration unit 14 into a position signal from the position detection unit 15, a turn signal from the turn detection unit 16, a brake signal from the brake detection unit 17, and a vehicle speed detection unit. 18 is cut out at a position corresponding to the viewpoint determined based on the speed signal from 18 and the steering wheel angle signal from the steering wheel angle detector 19 (these signals are referred to as “vehicle driving information”). The image cut out by the image cutout unit 20 is sent to the navigation device 21.

  The navigation device 21 includes a display 22 and a touch panel 23. The display unit of the present invention is configured by the display 22, and the viewpoint instruction unit is configured by the touch panel 23. The navigation device 21 has a function of displaying a map around the host vehicle 1 as a normal navigation function on the display 22, and a spatial map sent from the spatial map generator 13 and an image cutout unit 20. It has a function of displaying the displayed image on the display 22.

  Next, the operation of the driving assistance apparatus configured as described above according to Embodiment 1 of the present invention will be described with reference to the flowcharts shown in FIGS.

  First, as shown in FIG. 9A, the navigation device 21 displays a map 30 and a spatial map button 31 on the display 22 (step ST10). Next, the navigation device 21 checks whether or not the space map button 31 has been pressed (step ST11). Here, if it is determined that the space map button 31 is not pressed, the sequence returns to step ST10, and the above-described processing is repeated. Thereby, the map 30 displayed on the display 22 is sequentially updated, and so-called navigation is performed.

If it is determined in step ST11 that the spatial map button 31 has been pressed, then a spatial map generation process is performed (step ST12). FIG. 7 is a flowchart showing details of the spatial map generation process. In spatial map generation processing, firstly, the fish-eye image is inputted from the fisheye camera 10 ₁ to 10 ₄ (step ST20). Next, the _four fisheye images input from the fisheye cameras 10 ₁ to 10 ₄ are converted into four flat images without distortion by the panorama conversion unit 11 (step ST21). Next, a parallax image having a horizontal angle of view of 360 degrees is generated by the parallax measuring unit 12 from the four planar images obtained by the panorama converting unit 11 (step ST22). Next, the spatial map generation unit 13 generates a spatial map from the parallax images generated by the parallax measurement unit 12 (step ST23). Next, the spatial map generated in step ST23 is sent to the navigation device 21 (step ST24). Thereafter, the sequence returns to the processing shown in the flowchart of FIG.

When the spatial map generation process is completed, a panoramic image cutting process is then performed (step ST13). FIG. 8 is a flowchart showing details of the panoramic image cut-out process. In cutting process of a panoramic image, first, the fish-eye image is inputted from the fisheye camera 10 ₁ to 10 ₄ (step ST30). Next, the _four fisheye images input from the fisheye cameras 10 ₁ to 10 ₄ are converted into four flat images without distortion by the panorama conversion unit 11 (step ST31). Next, the four planar images obtained by the panorama conversion unit 11 are integrated by the image integration unit 14 to generate a panoramic image having a horizontal angle of view of 360 degrees (step ST32). Next, an optimal viewpoint is determined by the image cutout unit 20 according to the vehicle driving information from the position detection unit 15, the turn detection unit 16, the brake detection unit 17, the vehicle speed detection unit 18, and the steering wheel angle detection unit 19 (step ST33). ). Next, a part of the panoramic image is cut out from the position corresponding to the optimum viewpoint determined in step ST33 (step ST34). Then, the image cut out in step ST34 is sent to the navigation device 21 as a cut out image (step ST35). Thereafter, the sequence returns to the processing shown in the flowchart of FIG.

  When the panoramic image cut-out process is completed, a map 30, a spatial map 32, a cut-out image 33, and a camera button 34 are then displayed on the display 22 as shown in FIG. 9B (step ST14). Next, it is checked whether or not the camera button 34 has been pressed (step ST15). If it is determined that the camera button 34 has not been pressed, the sequence returns to step ST12 and the above-described processing is repeated. Thereby, the map 30, the spatial map 32, and the cutout image 33 displayed on the display 22 are sequentially updated, and so-called navigation is performed.

  If it is determined in step ST15 that the camera button 34 has been pressed, the spatial map 32, the cutout image 33, and the panoramic image 35 are displayed on the display 22 as shown in FIG. ST16). The panoramic image 35 is directly input from the image integration unit 14 to the navigation device 21. During this display, a portion of the panoramic image 35 corresponding to the cut-out image 33 is displayed with a frame 36. The user can arbitrarily change the viewpoint, that is, the position to be cut out from the panoramic image 35 by moving left and right (in the direction of the arrow in the drawing) while touching the frame 36.

  Next, it is checked whether or not the viewpoint has been changed (step ST17). If it is determined that the viewpoint has not been changed, the sequence returns to step ST16 and the above-described processing is repeated. On the other hand, if it is determined that the viewpoint has been changed, the clipped image is updated (step ST18). That is, the image at the position where the frame 36 has been moved is displayed as the cutout image 33. Next, it is checked whether or not it is finished (step ST19). This is performed, for example, by checking whether a predetermined button of the navigation device 21 has been pressed. If it is determined that the process is not finished, the sequence returns to step ST16 and the above-described processing is repeated. On the other hand, when it is determined that the operation is finished, the operation of the driving support device is finished.

  According to the driving support apparatus according to Embodiment 1 of the present invention configured as described above, the following usage is possible.

  Now, as one example of use of this driving support device, consider the case of changing to the right lane while the vehicle 1 is traveling. In this case, it is assumed that a screen as shown in FIG. 10A is displayed on the display 22 of the navigation device 21. When the space map button 31 is pressed in this state, a map 30, a space map 32, a cutout image 33, and a camera button 34 are displayed on the display 22 of the navigation device 21, as shown in FIG. The In this case, the turn signal from the turn detection unit 16 is set to the right, the position signal from the position detection unit 15 indicates forward movement, and the vehicle speed signal from the vehicle speed detection unit 18 moves forward. On the condition that it is in the middle, an image on the right rear side is automatically displayed as the cut-out image 33. Accordingly, the driver can easily determine whether or not the right lane can be changed.

  Moreover, the case where the vehicle 1 is parked is considered as another usage example of this driving support device. In this case, it is assumed that a screen as shown in FIG. 11A is displayed on the display 22 of the navigation device 21. When the space map button 31 is pressed in this state, a map 30, a space map 32, a cut image 33, and a camera button 34 are displayed on the display 22 of the navigation device 21, as shown in FIG. The In this case, the turn signal from the turn detector 16 is set to turn right and left, the position signal from the position detector 15 indicates reverse, and the vehicle speed signal from the vehicle speed detector 18 is backward. On the condition that it indicates that the image is moving, it is automatically determined that the right diagonally rear image is optimal as the cut image 33 and is displayed on the display 22. In this state, when the camera button 34 is further pressed, a panoramic image 35 is displayed together with a frame 36.

As described above, according to the driving support apparatus according to Embodiment 1 of the present invention, based on the images from the fish-eye cameras 10 ₁ to 10 ₄ that have taken the wide angle while sharing all the directions around the vehicle 1. Since the spatial map of the vehicle 1 and its surroundings viewed from above is generated and displayed on the display 22 of the navigation device 21, the spatial map displayed on the display 22 is an obstacle that is an obstacle to stopping or changing lanes. It becomes a target to classify objects, parked vehicles, and traveling vehicles, and it becomes easier to grasp the positional relationship with the own vehicle. As a result, the state around the host vehicle can be presented to the user in an easy-to-understand manner, and the user's parking and lane change can be supported.

A panoramic image is generated based on the images from the fisheye cameras 10 ₁ to 10 _4, and the viewpoint switching of the panoramic image is automatically performed based on the vehicle driving information. Therefore, an image that requires attention while the user is driving the vehicle 1 is automatically displayed, so that the user can intuitively grasp the state of the host vehicle 1.

  In addition, when there is a viewpoint to be confirmed, the user can change the clipping position by moving the panoramic image frame 36 by manual operation. Can be provided to the user, and parking, lane changes, and the like can be supported.

It is a block diagram which shows the structure of the driving assistance apparatus which concerns on Embodiment 1 of this invention. It is a figure for demonstrating arrangement | positioning of the fisheye camera mounted in the driving assistance apparatus which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the fisheye image which can be image | photographed with each of the fisheye camera mounted in the driving assistance apparatus which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the space map obtained with the driving assistance device which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the panoramic image obtained with the driving assistance device which concerns on Embodiment 1 of this invention. It is a flowchart for demonstrating operation | movement of the driving assistance device which concerns on Embodiment 1 of this invention. It is a flowchart which shows the detail of the space map production | generation process shown in FIG. 7 is a flowchart showing details of a panoramic image cutout process shown in FIG. 6. It is a figure for demonstrating the transition of the display screen in the driving assistance device which concerns on Embodiment 1 of this invention. It is a figure for demonstrating one usage example of the driving assistance device which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the other usage example of the driving assistance device which concerns on Embodiment 1 of this invention.

Explanation of symbols

10 ₁ to 10 ₄ fisheye camera, 11 panorama conversion unit, 12 parallax measurement unit, 13 spatial map generation unit, 14 image integration unit, 15 position detection unit, 16 turn detection unit, 17 brake detection unit, 18 vehicle speed detection unit, 19 Handle angle detection unit, 20 Image cutout unit, 21 Navigation device, 22 Display, 23 Touch panel.

Claims (4)

A plurality of cameras arranged to divide all directions around the vehicle and shoot wide angle;
A panorama converter that converts a plurality of wide-angle images obtained by wide-angle imaging with the plurality of cameras into a plurality of flat images without distortion;
A parallax measuring unit that generates a parallax image reflecting a distance to a subject based on parallax of a plurality of planar images obtained by conversion by the panorama converting unit;
A spatial map generation unit that generates a spatial map of the vehicle and its surroundings viewed from above based on the parallax image generated by the parallax measurement unit;
And a display unit that displays the spatial map generated by the spatial map generation unit. An image integration unit that generates a panoramic image in all directions around the vehicle by integrating a plurality of planar images obtained by conversion in the panorama conversion unit;
A vehicle driving information acquisition unit for acquiring vehicle driving information representing the driving state of the vehicle;
An image cutout unit that cuts out a part of the panoramic image generated by the image integration unit at a position corresponding to a viewpoint determined based on the vehicle driving information acquired by the vehicle driving information acquisition unit;
The driving support device according to claim 1, wherein the display unit displays the image cut out by the image cutout unit. The driving support device according to claim 2, wherein the display unit displays the panoramic image generated by the image integration unit. Provided with a viewpoint instruction unit that instructs the viewpoint to cut out the panoramic image,
The driving support device according to claim 3, wherein the image cutout unit cuts out a part of the panoramic image generated by the image integration unit at the viewpoint instructed by the viewpoint instruction unit.

Images