JP5045172B2 - Vehicle driving support device - Google Patents

Description

  The present invention relates to a vehicular driving support apparatus that performs driving support by displaying the surroundings of a host vehicle as an overhead image.

  The reverse parking performed by the driver is one of the most stressful scenes in driving the vehicle, and is an operation with a very heavy load. In addition, as a means to alleviate the burden on the driver by properly showing the driver the information necessary for reverse parking, measures such as securing a direct field of view, installing a mirror that reflects the rear, a camera, and a display that displays the image are taken. ing. For example, Patent Document 1 proposes a technique for generating an image (virtual image) from the second viewpoint based on the first viewpoint image obtained from the camera and providing it to the driver as a backward parking support function. ing. However, the technique described in Patent Document 1 may not function effectively due to the excessive amount of visual information obtained and the fixed viewpoint of the video. That is, the load required for the driver's recognition is large, and it may be particularly difficult to grasp the three-dimensional object. These equipment will only function when reversing, and will not be effective during forward travel.

  On the other hand, in patent document 2, while generating the bird's-eye view image (bird's-eye view image) image | photographed from the virtual viewpoint higher than the own vehicle using the image around the own vehicle, a virtual viewpoint is set according to the vehicle state. Technology to change has been proposed.

Non-patent document 1 discloses that viewpoint control is performed according to a user's head position as control of a virtual camera by driver image recognition.
JP 2005-035542 A Japanese Patent Laying-Open No. 2005-167309 "Control of virtual camera viewpoint based on body motion input by image recognition" Internet <URL: http://www.interaction-ipsj.org/archives/paper2003/pdf2003/008.pdf>

  By the way, since the environment around the host vehicle is constantly changing according to the position and driving state of the vehicle, it is necessary to change not only the virtual viewpoint but also the parameters of the virtual camera according to the location that the driver wants to check.

  The present invention has been made in view of the above-described problems, and the object of the present invention is to provide a driver with an overhead view image in which shooting parameters are continuously changed according to changes in the vehicle state and the driver's posture with respect to an image around the host vehicle. An object of the present invention is to realize a vehicle driving support device that can provide appropriate information according to changes in driver demands and circumstances.

In order to solve the above-described problems and achieve the object, the first aspect of the present invention uses image acquisition means for acquiring image information around the host vehicle and the image information, and is higher than the host vehicle. A bird's-eye view image generating means for generating a virtual bird's-eye view image taken by a virtual camera installed at a virtual viewpoint, a display means for displaying the bird's-eye view image, and a relative relationship between the head of the driver and the display means A relative distance detecting means for detecting a distance; a parameter changing means for continuously changing the shooting parameters of the virtual camera using the detected relative distance;
Have

According to this aspect, the driver can provide an overhead view image in which shooting parameters are continuously changed for an image around the host vehicle in which a place to be confirmed changes in accordance with a change in the vehicle state or the driver's posture. Providing appropriate information according to changes in the situation.

In the second aspect, the image acquisition means includes at least a surrounding photographing camera for photographing the surroundings of the own vehicle, a vehicle-to-vehicle communication means for communicating with another vehicle, and a road-to-vehicle communication means for communicating with a road surface. The image information around the host vehicle is acquired by either. According to this aspect, it is possible to collect a wide range of information that cannot be obtained only from the host vehicle, and to display an appropriate overhead image according to the driver's request or changes in the situation.

The third embodiment further comprises a display density changing means for changing the display density of the overhead image on the display screen in accordance with the prior SL relative distance. According to this aspect, it is possible to display an appropriate bird's-eye view image corresponding to a change in the viewpoint position of the driver.

The fourth embodiment, the parameter changing means, using the vehicle speed and acceleration rate as a run line state changes the position of the virtual camera, the amount of tilt as the imaging parameters by using the relative distance, zoom amount and Change the pan amount. According to this aspect, for example, by changing to the telephoto side or the wide-angle side according to the change in the driver's posture, it is possible to display an appropriate overhead view image according to the running state of the vehicle or the change in the driver's posture.

In the fifth aspect, the relative distance detection means detects, as the relative distance, a linear distance between the dry head and the display means, a vertical separation distance, and a horizontal separation distance, respectively. The parameter changing means changes the tilt amount as the shooting parameter using the vertical distance, the zoom amount using the linear distance, and the pan amount using the left-right distance. According to this aspect, for example, by changing to the telephoto side or the wide-angle side according to the change in the vertical and horizontal posture of the driver, it is possible to display an appropriate bird's-eye view image according to the driving state of the vehicle or the change in the driver's posture. it can.

  ADVANTAGE OF THE INVENTION According to this invention, about the image around the own vehicle, a bird's-eye view image in which the shooting parameters are continuously changed according to the vehicle state and the driver's posture change can be provided to the driver. However, it is possible to appropriately control the parameters of the virtual camera according to the driver's request and changes in the situation, and it is possible to improve the visibility and operability of the image at the time of driving support.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  The embodiment described below is an example as means for realizing the present invention, and the present invention can be applied to a modified or modified embodiment described below without departing from the spirit of the present invention.

  In addition, a computer program corresponding to the control of each embodiment to be described later and a storage medium storing the computer program are supplied to a computer mounted on the vehicle, and the computer reads out the program code stored in the storage medium. May be executed.

[ Reference example ]
1 and 2 are block diagrams showing the overall configuration of a vehicle driving support apparatus of a reference example .

  As shown in FIGS. 1 and 2, the vehicle 1 on which the vehicle driving support device of the present embodiment is mounted has a GPS that detects current position information x (latitude, longitude, altitude) used in the car navigation device. A vehicle position detection sensor 2 such as a sensor; a brake sensor 3 that detects a brake pedal depression amount b; a vehicle speed sensor 4 that detects a vehicle speed v of the host vehicle; a travel range switch 5 that detects forward or backward movement of the vehicle; An accelerator sensor 6 for detecting the steering angle, a steering angle sensor 7 for detecting the steering angle s, a surrounding photographing camera 8 for photographing the surroundings of the host vehicle, a distance detecting camera 9 for photographing the interior of the vehicle, inter-vehicle communication with other vehicles, or a road surface Acquired from the communication device 10 that performs road-to-vehicle communication using DSRC (narrow band wireless communication system), etc., the display device 11 that displays a bird's-eye view image to be described later, and the above-mentioned sensors and cameras From information and / or vehicle-to-vehicle information collected by the communication and road-vehicle communication and an image processing apparatus 12 for generating an overhead image of the virtual.

  As shown in FIG. 1B, the display device 11 is disposed in the vicinity of the sun visor above the driver head in the vehicle interior, and the distance detection camera 9 includes the driver head H and the display device as described later in FIG. 11 is detected (separation distance L, vertical distance u, horizontal distance t). An ultrasonic sensor or the like may be used instead of the distance detection camera 9.

  The communication device 10 acquires ambient environment information d (information on vehicles, terrain, buildings, etc. existing around the host vehicle, image information, etc.) from other vehicles or infrastructure facilities.

  As shown in FIG. 2, the image processing apparatus 12 uses a current position information x, a brake depression amount b, and a vehicle speed v to calculate a virtual camera position (height h, reverse amount z). And virtual camera parameters (tilt amount θ, zoom amount f, pan amount Φ) using virtual camera position h, z, current position information x, vehicle speed v, travel range information, accelerator opening a, and steering angle s. The virtual camera parameter calculation unit 14 to calculate, and the overhead image display density calculation unit to calculate the display density n of the overhead image from the relative distances L, u, t between the driver head detected by the distance detection camera 9 and the display device 11. 15, a virtual camera position h, z, virtual camera parameters θ, f, Φ, a bird's eye image display density n, and an existing computer graphics line used for generating a three-dimensional image from ambient environment information d. Internet (for example, Internet <URL: http://dl.cybernet.co.jp/matlab/support/manual/r14/toolbox/matlab/visualize/?/matlab/support/manual/r14/toolbox/matlab/visualize/ chview7.shtml # 72077> and <http://www.image.esys.tsukuba.ac.jp/~kitahara/data/pubs/2005_JSAI2005_sakamoto.pdf>)) 16 and.

FIG. 3 illustrates a method for calculating the virtual camera position (a) and the virtual camera parameter (b) according to the reference example .

  The virtual camera parameters used in this embodiment are “pan: pan” for horizontal turning of the camera, “tilt” for vertical turning of the camera, and “zoom: zoom” for the change in the focal length of the camera. "

  As an image generation technique for changing the viewpoint of a virtual camera, for example, “2. Free viewpoint video” Internet <URL: http://www.image.esys.tsukuba.ac.jp/~kititahara/data/pubs/ Use the method published in 2005_JSAI2005_sakamoto.pdf>.

First Embodiment
FIG. 4 is a block diagram of the vehicle driving support apparatus according to the first embodiment of the present invention.

  The configuration of FIG. 4 is the same as that of FIGS. 1 and 2 except that the accelerator sensor 6 and the steering angle sensor 7 are omitted from FIGS.

  In the present embodiment, the virtual camera position is calculated using the current position information x, the brake depression amount b, and the vehicle speed v, and the tilt amount θ and the relative distances u, L, and t between the driver head H and the display device 11 are calculated. The pan amount Φ and the zoom amount f are determined.

FIG. 5 illustrates a method for calculating the virtual camera position (a) and the camera parameter (b) according to the first embodiment. FIG. 6 shows the relative distance between the driver head H and the display device 11 as a linear distance L in plan view (a), a left-right direction (vehicle width direction) distance t in plan view (b), and a side view. The vertical direction (vertical direction) distance u in (c) is shown.

  As shown in FIGS. 5 and 6, the image processing apparatus 12 according to the present embodiment uses the virtual camera parameter calculation unit 14 to calculate the tilt amount θ and the driver head H from the vertical distance u between the driver head H and the display device 11. Then, the zoom amount f is calculated from the linear distance L of the display device 11, and the pan amount Φ is calculated from the driver head H and the horizontal distance t of the display device 11. Further, the virtual camera position calculation unit 13 calculates the virtual camera positions h and z and the display density n using the calculation formulas shown in FIGS.

<Overhead image display processing>
FIG. 7 is a flowchart showing the overhead image display processing by the image processing apparatus of the present embodiment.

  In FIG. 7, the image processing device 12 includes current position information x, ambient environment information d, vehicle travel information (vehicle speed v, traveling direction, posture), driving operation information (accelerator opening a, brake depression amount b, steering rudder angle). s, and the variations Δa, Δb, Δs) per unit time, and the relative distance (L, u, t) between the driver head H and the display device 11 are acquired (S1 to S5).

  Next, the image processing apparatus 12 calculates the virtual camera positions h and z from the calculation formula of FIG. 3A by the virtual camera position calculation unit 13 (S6).

Next, the image processing apparatus 12 calculates the virtual camera parameters θ, f, and Φ from the calculation formula of FIG. 3B or FIG. 6 by the virtual camera parameter calculation unit 14 (S7). Here, in the reference example , the virtual camera parameter is calculated from the vehicle travel information and the driving operation information, and in the first embodiment, the virtual camera parameter is calculated from the relative distances u, L, t between the driver head H and the display device 11. .

  Next, the image processing apparatus 12 uses the overhead image display density calculation unit 15 and the overhead image generation unit 16 to generate a virtual overhead image captured from the viewpoint of the virtual camera controlled by the virtual camera parameters (S8). .

  Next, the image processing device 12 calculates the display density n from the calculation formula of FIG. 3B, and displays an overhead image at the display density n on the display device 11 (S9).

  The display process described above is executed at a cycle of 10 Hz or more (100 msec or less).

< Example of virtual camera parameter control and bird's-eye view image display during retreat according to reference example>
8 and 9 show a virtual camera parameter control example and a bird's-eye view image display example when retreating according to a reference example .

  When the vehicle reverses and the state shown in FIG. 8A (vehicle speed 0 km / h) changes to the state shown in FIG. 8B (vehicle speed 2 km / h, accelerator opening 5%), the virtual camera position is P1 ( It moves from h1, z1) to P2 (h2, z2) calculated from vehicle speed v (= 2 km / h) and brake depression amount b (= 0 mm), and tilt calculated from accelerator opening a (= 5%) The amount and the zoom amount are controlled to θ1, f1. When the state shown in FIG. 8B is changed to the state shown in FIG. 8C (vehicle speed 0 km / h, brake depression amount 20 mm), the virtual camera position changes from P2 to the vehicle speed v (= 0 km / h) and It moves to P3 (h3, z3) calculated from the brake depression amount b (= 20 mm), and is controlled to the tilt amount θ and the zoom amount f calculated from the accelerator opening a (= 0%).

  At this time, the virtual camera positions P1, P2, P3 and the overhead images displayed when the backward movement is completed are shown in FIGS. 9A, 9B, 9C, and 9D. When the backward movement is started, as shown in FIG. 9B, the zoom amount f is controlled to the telephoto side, and a bird's-eye view image that is tilted to promote grasping of the surroundings is displayed. When the brake is depressed, as shown in FIG. 9C, the zoom amount f is controlled to the wide-angle side, and a detailed overhead image near the rear bumper is displayed.

< Example of virtual camera parameter control and bird's-eye view image display during forward movement according to reference example>
10 and 11 show a virtual camera parameter control example and a bird's-eye view image display example during forward movement according to a reference example .

  From the state shown in FIG. 10A (vehicle speed 0 km / h, brake depression amount 20 mm, tilt amount θ 2, zoom amount f 2), the acceleration state shown in FIG. 10B (vehicle speed 20 km / h, accelerator opening) 40%), the virtual camera position moves from P4 (h4, z4) to P5 (h5, z5) calculated from the vehicle speed v (= 20 km / h) and the brake depression amount b (= 0 mm), The tilt amount and zoom amount calculated from the accelerator opening a (= 40%) are controlled to θ3 and f3. When the state shown in FIG. 10B changes to the high speed running state shown in FIG. 10C (vehicle speed 50 km / h, accelerator opening 20%), the virtual camera position changes from P5 (h5, z5) to the vehicle speed. It moves to P6 (h6, z6) calculated from v (= 50 km / h) and the brake depression amount b (= 0 mm), and the tilt amount and zoom amount calculated from the accelerator opening a (= 20%) are θ4. , F4.

  At this time, the virtual camera positions P4, P5, P6 and the overhead images displayed at the time of steering are shown in FIGS. 11 (a), 11 (b), 11 (c), and 11 (d). When starting acceleration is started, as shown in FIG. 11B, the zoom amount f is controlled to the telephoto side, and a bird's-eye view image that tilts and prompts to grasp the surroundings is displayed. Further, during high speed traveling, as shown in FIG. 11C, the viewpoint is increased by moving in the direction in which the virtual camera height h is increased. Further, when the steering is performed from FIG. 11C, a panned overhead image is displayed as shown in FIG. 11D.

<Example of virtual camera parameter control and bird's-eye view image display during forward movement according to the first embodiment>
FIG. 12 shows an example of virtual camera parameter control during forward movement according to the first embodiment.

  When the vehicle starts, the acceleration state shown in FIG. 12B (vehicle speed 20 km / h, vehicle speed 0 km / h, brake depression amount 20 mm, u = 200 mm (where Δu = 0)) is shown in FIG. When the accelerator opening degree is changed to 40% and u = 400 mm (where Δu = 0), the virtual camera position changes from P7 (h7, z7) to the vehicle speed v (= 20 km / h) and the brake depression amount b (= 0 mm). Is moved to P8 (h8, z8) calculated from the above, and the tilt amount calculated from the vertical distance u (= 400 mm) is changed from 20 ° to 40 °. When the acceleration state shown in FIG. 12B is changed to the high speed running state shown in FIG. 12C (vehicle speed 50 km / h, accelerator opening 20%, u = 300 mm (where Δu = 0)), The virtual camera position moves from P8 (h8, z8) to P9 (h9, z9) calculated from the vehicle speed v (= 50 km / h) and the brake depression amount b (= 0 mm), and the vertical distance u (= 300 mm). Is changed from 40 ° to 30 °.

  According to the above-described embodiment, an overhead image in which the position of the virtual camera and the camera parameters are continuously changed is displayed. This display process matches the characteristics of human spatial grasping, such as the ability of monocular stereoscopic vision (reconstruction of three-dimensional structures by movement factors), so it is possible to recognize stereoscopic positional relationships and useful information in images. make it easier. Therefore, it is possible to provide the driver with a bird's-eye view image in which the shooting parameters are continuously changed for the image around the host vehicle where the location to be confirmed changes according to the vehicle state or the driver's posture change, and the driver's request or situation changes. Appropriate information can be provided.

In particular, in the reference example , the virtual camera position and the camera parameters are controlled using the vehicle speed v, the brake depression amount b, the accelerator opening a, the steering angle s, etc., thereby contributing to facilitating the grasping of the space. Since the bird's-eye view image in the direction, distance, and zoom magnification required by the driver is displayed, an appropriate bird's-eye view image according to the driving state of the vehicle and the driving operation of the driver can be displayed, and the driver's confirmation burden can be reduced.

Furthermore, in the first embodiment, by controlling the camera parameter using the relative distance between the driver head and the display device, for example, a detailed image is displayed as the relative distance approaches the display device. Further, when the position is shifted in the vertical direction or the horizontal direction, the tilt amount and the pan amount are changed according to the distance. As a result, a bird's-eye view image that more reflects the driver's viewpoint, posture change, and intention can be displayed.

  Further, since the bird's-eye view image is displayed not only at the time of reverse but also at the time of forward movement, the driver can obtain visual information that cannot be recognized directly from the field of view as in the case of backward movement during forward traveling. In addition, the bird's-eye view image, like the speedometer and engine tachometer, not only provides feedback on driving operation, but also provides the driver with the same driving feeling and exhilaration as that provided by the current engine tachometer. It is possible to give

  In addition, since the surrounding environment information d of the own vehicle is acquired by using the vehicle-to-vehicle communication means and the road-to-vehicle communication means, a wide range of information that cannot be obtained from the own vehicle alone is collected, and the driver's request or situation changes It is possible to display an appropriate overhead image.

[Correspondence between claims and description]
In claim 1 , the image acquisition means corresponds to the ambient camera 8 and the communication device 10, the overhead image generation means corresponds to the overhead image generation unit 16, the display means corresponds to the display device 11, and the parameter change means corresponds to the virtual camera parameter calculation unit 14. Yes.

Car both state detection unit vehicle position detection sensor 2, a brake sensor 3, a vehicle speed sensor 4, driving range switch 5 corresponds to the accelerator sensor 6, and the steering angle sensor 7.

The relative distance detection means in claim 1 corresponds to the distance detection camera 9.

The vehicle-to-vehicle communication means and the road-to-vehicle communication means in claim 2 correspond to the communication device 10.

The display density changing means in claim 3 corresponds to the overhead image display density calculation unit 15.

Claims 4 and 5 correspond to the calculation of the virtual camera position and camera parameters using the image processing apparatus 12 shown in FIG.

  Each means is not limited to the above-described configuration, and any configuration or form may be used as long as it has the above-described functions.

It is a figure which shows the structure in the state with which the vehicle driving assistance device of the reference example which concerns on this invention was mounted in the vehicle. It is a block diagram which shows the structure of the driving assistance apparatus for vehicles of the reference example which concerns on this invention. It is a figure which illustrates the calculation method of the virtual camera position (a) and virtual camera parameter (b) by a reference example . 1 is a block diagram of a vehicle driving support apparatus according to a first embodiment of the present invention. It is a figure which illustrates the calculation method of the virtual camera position (a) and camera parameter (b) by 1st Embodiment. As a relative distance between the driver head H and the display device 11, a linear distance L in a plan view (a), a left-right direction distance t in a plan view (b), and a vertical distance u in a side view (c). FIG. It is a flowchart which shows the display process of the bird's-eye view image by the image processing apparatus of this embodiment. It is a figure which shows the virtual camera parameter control example at the time of the reverse by a reference example . It is a figure which shows the example of a bird's-eye view image display at the time of reverse by a reference example . It is a figure which shows the virtual camera parameter control example at the time of advance by a reference example . It is a figure which shows the bird's-eye view image display example at the time of advance by a reference example . It is a figure which shows the virtual camera parameter control example at the time of advance by 1st Embodiment.

DESCRIPTION OF SYMBOLS 1 Vehicle 2 Vehicle position detection sensor 3 Brake sensor 4 Vehicle speed sensor 5 Traveling range switch 6 Acceleration sensor 7 Steering angle sensor 8 Surrounding camera 9 Distance detection camera 10 Communication apparatus 11 Display apparatus 12 Image processing apparatus 13 Virtual camera position calculation part 14 Virtual Camera parameter calculation unit 15 Overhead image display density calculation unit 16 Overhead image generation unit

Claims (5)

Image acquisition means for acquiring image information around the host vehicle;
Using the image information, an overhead image generation means for generating a virtual overhead image captured by a virtual camera installed at a virtual viewpoint higher than the own vehicle;
Display means for displaying the overhead image;
A relative distance detecting means for detecting a relative distance between the head of the driver and the display means;
Parameter changing means for continuously changing the shooting parameters of the virtual camera using the detected relative distance;
A vehicle driving support apparatus comprising: The image acquisition means includes the own vehicle by at least one of a surrounding photographing camera for photographing the periphery of the own vehicle, an inter-vehicle communication means for communicating with another vehicle, and a road-to-vehicle communication means for communicating with a road surface. 2. The vehicle driving support apparatus according to claim 1 , wherein surrounding image information is acquired. Display density changing means for changing the display density of the overhead image with respect to the display screen according to the relative distance;
The vehicle driving support device according to claim 1 , further comprising: Wherein the parameter changing means, using the vehicle speed and acceleration rate as a run line state changes the position of the virtual camera, the amount of tilt as the imaging parameters by using the relative distance, to change the zoom amount and pan weight The vehicle driving support device according to claim 1 , wherein: The relative distance detecting means, as the relative distance, the straight line distance between the driver's head and said display means, the vertical direction of the distance, the distance in the lateral direction by the respective detection,
The parameter changing means changes the tilt amount as the shooting parameter using the vertical separation distance, the zoom amount using the linear distance, and the pan amount using the left-right distance, respectively. The vehicle driving support device according to claim 4 .

Images