JP2004147083A - Driving support apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To intuitively and three-dimensionally view the track of a vehicle on a two-dimensional monitor screen. <P>SOLUTION: The apparatus is provided with: an on-vehicle camera; an on-vehicle monitor which displays the picked up image of the on-vehicle camera; an image composing means which superimposes a vehicle track image showing the predictive moving direction of a vehicle on the picked up image in the case of displaying the picked up image on the on-vehicle monitor; and a driving support data generation means which displays the vehicle track image by a U-shape consisting of a track surface on a ground surface in the moving direction of the vehicle and the track surfaces of both side faces of the vehicle and connecting the track surfaces of both side surfaces of the vehicle by one or two lines 115 to display it on the on-vehicle monitor. Since the lines 115 exits, the track surfaces of both side faces of the vehicle can intuitively grasped as a surface rising from the ground surface, thereby the vehicle track image can intuitively and three-dimensionally be viewed. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a driving support device that displays an image captured by a vehicle-mounted camera on a vehicle-mounted monitor to support vehicle driving, and in particular, a driving support device that allows an intuitive grasp of an area where a vehicle is traveling on a monitor screen. About.
[0002]
[Prior art]
2. Description of the Related Art Driving support devices in which an in-vehicle camera is installed rearward in a rear trunk portion of a vehicle and the like, and an image captured behind the vehicle taken by the in-vehicle camera is presented to a driver have begun to spread. A conventional steering assist device described in Japanese Patent Application Laid-Open No. 2001-180405 (Patent Document 1) superimposes a guide line indicating a vehicle width line on a monitor screen so as to extend to the rear of the vehicle in addition to an image captured by an on-vehicle camera. It is displayed and the direction of this guide line is moved in accordance with the operation of the steering wheel so that the driver can easily recognize the traveling direction of the vehicle.
[0003]
Further, in this conventional technique, of the guide lines indicating the vehicle width line, the guide line on the inside where the vehicle turns, that is, the guide line on the side that is likely to come into contact with another vehicle, is displayed on a curved surface rising from the ground to the bumper position, and the most It is intended to make it easy to determine whether or not the projecting bumper contacts an obstacle.
[0004]
[Patent Document 1]
JP 2001-180405 A (FIG. 7)
[0005]
[Problems to be solved by the invention]
However, when a guide line is superimposed and displayed on an in-vehicle monitor, it is necessary to be able to intuitively recognize what the guide line means at the moment of viewing. The guide line according to the related art described above, in particular, a guide line that is to be displayed on a three-dimensional curved surface rising from the guide line on the ground to the bumper position, when this is displayed on the two-dimensional screen of the vehicle-mounted monitor, There is no guarantee whether the driver who sees the line intuitively grasps the surface as a three-dimensional curved surface, and may erroneously recognize it as a surface drawn on the surface of a road.
[0006]
SUMMARY OF THE INVENTION An object of the present invention is to provide a driving support device that enables a driver or the like to easily understand that a predicted trajectory in the vehicle traveling direction is three-dimensionally displayed on a monitor screen when the predicted trajectory is three-dimensionally displayed. Is to do.
[0007]
[Means for Solving the Problems]
A driving support device that achieves the above object includes an in-vehicle camera, an in-vehicle monitor that displays a captured image of the in-vehicle camera, and a vehicle trajectory image that indicates a predicted traveling direction of the vehicle when displaying the captured image on the in-vehicle monitor. Image synthesizing means for superimposing on the captured image, and when the vehicle trajectory image is displayed in a U-shape including a trajectory surface on the ground in the vehicle traveling direction and a trajectory surface on both sides of the vehicle, a distance between the trajectory surfaces on both sides of the vehicle is displayed. Driving assistance data generating means for displaying the information on the in-vehicle monitor by connecting one or two lines at positions floating from the track surface on the ground.
[0008]
With this configuration, the trajectory surfaces on both sides of the vehicle can be easily recognized as surfaces rising from the ground, the vehicle trajectory can be grasped as a three-dimensional shape on the monitor screen, and the interference with an obstacle overlapping the vehicle trajectory can be easily determined. .
[0009]
Preferably, in the above, the captured image and the vehicle trajectory image are displayed on the on-vehicle monitor as viewpoint-converted images. With this configuration, it is possible to convert the display image on the monitor into an image viewed from various viewpoints and provide the image to the driver.
[0010]
More preferably, in the above, the height of the trajectory surfaces on both sides of the vehicle is set to a vehicle height or a bumper position. With this configuration, a sense of distance in the height direction appears, and a three-dimensional effect is emphasized.
[0011]
More preferably, in the above, the bumper position is highlighted in the trajectory plane on both sides of the vehicle. With this configuration, attention is drawn to the most protruding position on the side of the vehicle, and it is easy to determine whether there is interference between the bumper and an obstacle such as another vehicle.
[0012]
More preferably, in the above, the highlighting of the bumper position is displayed on a plane having a predetermined width. With this configuration, the attention to the bumper position is further enhanced, and the three-dimensional effect is further improved.
[0013]
More preferably, in the above, the distance display is not displayed in the track surface on the ground. With this configuration, complexity on the screen is reduced, and the screen is easy to see.
[0014]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0015]
FIG. 1 is a block diagram showing a configuration of a driving support device according to one embodiment of the present invention. The driving support device according to the present embodiment includes a camera 10 that captures a color image of a scene within an angle of view, a steering wheel angle sensor 20 that detects a turning angle of a steering wheel of a vehicle, and a steering angle based on a detection value of the steering wheel angle sensor 20. A steering angle calculation unit 30 obtained by calculation, an image processing unit 40 which takes in a captured image of the camera 10 and a calculation result of the steering angle calculation unit 30 and generates driving assistance image data as described later, and an image processing unit 40 The display control unit 50 converts the driving assistance image data generated in the step (1) into a video signal, and a monitor 60 such as a color liquid crystal display that displays the video signal from the display control unit 50 on a screen.
[0016]
The image processing unit 40 includes a CPU (central processing unit) 41 that controls the entire image processing, a program memory 42 that stores a control program of the CPU 41, and a work memory 43 that is used for image processing by the CPU 41.
[0017]
The working memory 43 is connected to the image storage unit 44 connected to the camera 10, the viewpoint conversion map data storage unit 45 connected to the image storage unit 44, the steering angle calculation unit 30, and the viewpoint conversion map data storage unit 45. The driving support data generation unit 46, a trajectory map data storage unit 47 connected to the driving support data generation unit 46, and an image synthesis unit 48 connected to the image storage unit 44 and the driving support data generation unit 46.
[0018]
The viewpoint conversion map data storage unit 45 stores predetermined viewpoint conversion map data used for viewpoint conversion. Viewpoint conversion refers to converting an image captured from a real viewpoint of the camera 10 into an image viewed from another desired virtual viewpoint position. The viewpoint conversion map data may be conversion map data including both viewpoint conversion and lens distortion correction.
[0019]
When a real image captured by the camera 10 is captured, the image storage unit 44 rearranges the pixel data of the real image in accordance with the viewpoint conversion map data stored in the viewpoint conversion map data storage unit 45, and performs viewpoint conversion. Store as an image.
[0020]
The trajectory map data storage unit 47 stores the trajectory map data for each wheel steering angle, and the driving support data generation unit 46 responds according to the wheel steering angle calculated by the steering angle calculation unit 30. The trajectory map data is acquired from the trajectory map data storage 47.
[0021]
Further, similarly to the processing performed in the image storage unit 44, the driving support data generation unit 46 performs a predetermined viewpoint conversion on the trajectory map data according to the viewpoint conversion map data stored in the viewpoint conversion map data storage unit 45. The generated locus image is generated.
[0022]
The image synthesizing unit 48 generates driving assistance image data by synthesizing the viewpoint converted image generated by the image storage unit 44 and the trajectory image generated by the driving assistance data generation unit 46, and displays the display control unit at a predetermined timing. Output to 50.
[0023]
Typically, the program memory 42, the viewpoint conversion map data storage unit 45, and the trajectory map data storage unit 47 are configured by a read-only memory such as a read-only memory, and include an image storage unit 44, an image support data generation unit 46, The image synthesizing unit 48 is configured by a readable / writable memory such as a random access memory.
[0024]
FIG. 2 is an image diagram when the driving support device described in FIG. In this vehicle 1, a camera 10 for photographing the surrounding area behind the vehicle 1 is installed at an upper rear portion. Further, the camera 10 is installed so that a part of the vehicle 1 such as the rear bumper 3 of the vehicle is reflected in the photographing range (viewing range).
[0025]
The driver's seat of the vehicle 1 is provided with a monitor 60. When the navigation device is mounted on the vehicle 1, the monitor 60 also functions as a display unit of the navigation device. In this case, the monitor 60 is normally used as a display unit of the navigation device, and is capable of displaying an image captured by the camera 10 when the shift lever 5 provided in the driver's seat is operated to the reverse traveling position. It can be switched to.
[0026]
Similarly, the image processing section 40 performs the above-described image processing at a predetermined cycle (for example, a cycle of processing an image of 30 frames per second) only when the shift lever 5 is operated to the reverse travel position. It is easy to determine whether or not the shift lever 5 has been operated to the reverse running position by separately providing a switch for detecting the position of the shift lever 5.
[0027]
The front wheels 7a and 7b, which are steered wheels, are steered by operating the steering wheel 8. The steering wheel angle sensor 20 is provided at a predetermined position with respect to the steering wheel 8, and detects the turning angle θ of the steering wheel 8 operated by the driver. The steering angle calculation unit 30 calculates a steering angle (tire turning angle) β of the front wheels 7a and 7b from the turning angle θ detected by the steering wheel angle sensor 20. The steering angle β is obtained by multiplying the turning angle θ by a predetermined coefficient K (β = Kθ).
[0028]
The image processing unit 40, the steering angle calculation unit 30, and the display control unit 50 shown in FIG. 1 are installed at appropriate places suitable for mounting electronic devices inside the vehicle 1 (not shown in FIG. 2).
[0029]
Next, a method of creating locus map data stored in advance in the locus map data storage unit 47 will be described.
[0030]
First, trajectory plane data on the ground in the world coordinate system shown in FIG. 3 is designed. In FIG. 3, assuming that the width of the vehicle 1 is W, the length of the wheel base is L, and the distance between the centers of the rear wheels 7c and 7d is Tr, the center of the rear axle (the midpoint between the rear wheels 7c and 7d) Ov. , The turning radius Rout of the intersection of the rear axle center line Ax and the turning center O with the outside of the vehicle (outside the rear wheel 7c), and the turning radius Rout of the rear axle center line Ax and the turning center O inside the vehicle body (the rear wheel). The turning radius Rin at the intersection with (inside 7d) is represented by the following equations (1) to (3) including the steering angle β.
[0031]
Rmid = (L / tanβ) − (Tr / 2) (1)
Rout = Rmid + W / 2
= (L / tanβ)-(Tr / 2) + W / 2 (2)
Rin = Rmid-W / 2
= (L / tanβ)-(Tr / 2) -W / 2 (3)
[0032]
Then, the trajectory data in the world coordinate system, that is, the predicted courses Gin and Gout where the vehicle retreats, are the movement positions of the rear wheels 7c and 7d in a range where the middle point Ov retreats 0 to 300 cm on the trajectory Gmid of the turning radius Rmid. Is calculated at 13 points at 25 cm intervals. The predicted courses Gin and Gout are obtained in increments of 1 degree in a range of values that the steering angle β can take (for example, in the left-right direction from 0 to 37 degrees). Note that the calculation range and the number of locations may be other than the above example. In addition, the unit for incrementing the steering angle β may be other than 1 degree.
[0033]
As a result, trajectory plane data on the ground as shown in FIGS. 4A and 4B is created. FIG. 4A shows trajectory surface data of rectilinear receding, and FIG. 4B shows trajectory surface data of curving and retreating. In the example of FIGS. 4A and 4B, the predicted courses Gin and Gout shown in FIG. 3 are indicated by lines 101 and 102, and the positions when the rear end of the vehicle rear bumper 3 is retracted by 1 m, 2 m and 3 m are indicated by lines. 103, a line 104, and a line 105.
[0034]
Next, the predicted trajectory planes 110 and 111 on both sides of the vehicle 1 are expressed on the predicted paths Gin and Gout of the generated trajectory plane data on the ground as shown in FIGS. The trajectory surfaces 110 and 111 indicate the height from the ground to the top of the vehicle 1 of the vehicle 1, but may be the height from the ground to the bumper position of the vehicle 1. Note that the locus of the side surface of the vehicle 1 does not need to faithfully follow the shape of the side surface of the vehicle, and in this example, is the locus of an external tangent to the side surface of the vehicle.
[0035]
The trajectory plane data on the ground and the trajectory plane data on both sides in the world coordinate system created as described above are stored in the trajectory map data storage unit 47, and the driving support data generation unit 46 The driving support image is generated by performing the viewpoint conversion and the lens distortion correction on the trajectory plane data in the system based on the viewpoint conversion map data, and combining the captured image of the camera 10 with the image obtained by performing the viewpoint conversion. .
[0036]
In this case, according to the present embodiment, when the trajectory plane data in the world coordinate system is subjected to the viewpoint transformation and the lens distortion correction, the vehicle trajectory is tertiarily transformed on the screen of the monitor 60 instead of simply performing the viewpoint transformation and the lens distortion correction. Originally, emphasis correction is performed on an image that is easy to grasp intuitively.
[0037]
FIG. 6 is an image obtained by simply performing viewpoint transformation and lens distortion correction on vehicle trajectory data of rectilinear backward movement in the world coordinate system. The virtual viewpoint position is about 50 cm above the actual installation position of the camera 10 shown in FIG. The trajectory data shown in the image of FIG. 6 includes lines 101a and 102a indicating the vehicle width of the vehicle, lines 103a, 104a and 105a indicating 1m, 2m and 3m behind the vehicle on the ground, and both sides of the vehicle. Trajectory planes 110a and 111a.
[0038]
The image in FIG. 6 can be viewed as a three-dimensional vehicle trajectory if it is consciously viewed as a three-dimensional figure. However, when this image is displayed on the two-dimensional screen of the monitor 60, it is difficult to instantaneously grasp the image as a three-dimensional image, and the track surfaces 110a and 111a on both sides of the vehicle are also recognized as images drawn on the ground. There is a high possibility that it will be done.
[0039]
Therefore, in the present embodiment, the emphasis correction is performed as described with reference to FIG. 7 so that the image in FIG. 6 can be instantly grasped intuitively as a three-dimensional trajectory. Is superimposed on the image captured by the camera.
[0040]
FIG. 7 is an image obtained by enhancing and correcting the image of FIG. In this embodiment, only one emphasis line 115 for stereoscopic vision assistance connecting the trajectory plane 110a indicating the side trajectory of the vehicle and the trajectory plane 111a is displayed. By displaying the emphasis line 115, the trajectory plane 110a and the trajectory plane 111a can be intuitively grasped as a three-dimensional plane rising from the ground. In this example, the start point and the end point of the enhancement line 115 for stereoscopic vision assistance are the upper end positions on the front side of the surfaces 110a and 111a at the height indicating the vehicle height at the position floating from the track surface on the ground. It is not limited to.
[0041]
Further, in the present embodiment, the vehicle rear end 0m connecting the end lines of the locus plane 110a and the locus plane 111a, that is, the end lines 110b and 111b indicating the vehicle rear end 0m and the lines 101a and 102a indicating the vehicle width on the ground. Is also displayed, the trajectory plane on the ground, that is, the frame surrounded by the lines 101a, 102a, 105a, and 106a is blank, and the display of the intermediate distance data is deleted.
[0042]
That is, distance data lines corresponding to the lines 103a and 104a shown in FIG. 6 are not displayed. These lines 103a and 104a indicate the rear end 1m and 2m of the vehicle, respectively, but in the example shown in FIG. 7, the vertical lines 110c and 111c in the locus surfaces 110a and 111a on both sides of the vehicle indicate the distance by 50 cm. It has become.
[0043]
Further, in the present embodiment, driving is performed by displaying brightest lines or changing colors on the locus surfaces 110a, 111a on the side surfaces of the vehicle, by displaying bumper lines 120a, 121a indicating the most protruding portions on the vehicle side, that is, the bumper positions. We call attention to person. In this example, the bumper lines 120a and 121a are highlighted as surfaces having a certain width, such as guardrails, so that the bumper position can be easily grasped as a three-dimensional object.
[0044]
In addition, in the case of highlighting, instead of displaying the guardrail-shaped band portion with a bright line or color-coded display, it is possible to emphasize the band by opening the band portion like a window. In this way, there is an advantage that the captured image of the camera coming in the band portion can be seen without being disturbed.
[0045]
FIG. 8 is a flowchart illustrating a processing procedure performed by the image processing unit 40 of the driving support device. When the shift lever 5 of the vehicle 1 is switched to the reverse travel position, the steering angle β corresponding to the steering angle θ of the steering wheel 8 detected by the steering wheel angle sensor 20 is calculated by the steering angle calculation unit 30 (step S81). ). Next, in the driving support data generation unit 46, the trajectory map data corresponding to the steering angle β is read from the trajectory map data storage unit 47 (Step S82).
[0046]
Next, in the driving support data generation unit 46, necessary viewpoint conversion and enhancement correction described with reference to FIG. 7 are performed to generate a vehicle locus image (step S83). Next, it is determined whether or not a viewpoint converted image captured by the camera 10 and subjected to necessary viewpoint conversion has been written from the image storage unit 44 to the image synthesizing unit 48 (step S84). When the writing is completed, the image compositing section 48 superimposes the vehicle locus image on the viewpoint converted image and outputs it to the display control section 50 (steps S85 and S86). The above processing is repeatedly performed while the shift lever 5 of the vehicle 1 is in the reverse traveling position (Step S87).
[0047]
As described above, according to the driving support apparatus according to the embodiment of the present invention, the trajectory of the vehicle is displayed on the two-dimensional screen of the monitor 60 as an image that can be intuitively viewed stereoscopically. The driver can easily understand the sense of distance between the host vehicle and the other vehicle and the height of the obstacle from the ground.
[0048]
FIG. 9 is a vehicle locus image showing an example of enhancement correction according to another embodiment of the present invention. In this embodiment, bumper lines 122a and 123a for emphasizing the bumper position are displayed on surfaces 110a and 111a indicating the side trajectory of the vehicle, and further, a stereoscopic assisting line connecting end points of lines 122a and 123a is connected. By displaying the emphasis lines 125a and 126a, the trajectory surfaces 110a and 111a on the side of the vehicle are connected, and it is possible to easily grasp that the trajectory surfaces 110a and 111a are surfaces rising from the ground.
[0049]
Also, in this embodiment, as in the embodiment shown in FIG. 7, many lines and surfaces that block the upper surface of the U-shaped vehicle trajectory are not displayed. Thereby, the state on the ground can be seen well through the upper surface portion of the vehicle trajectory, and an easy-to-view image can be provided to the driver.
[0050]
Although the heights of the locus surfaces 110a and 111a on the side surfaces of the vehicle are the vehicle heights in the above-described embodiment, the heights are not necessarily required to be the vehicle heights, and may be the heights up to the bumper positions most protruding to the side of the vehicle.
[0051]
In the driving assistance apparatus according to the above-described embodiment, a case has been described in which a real image taken by one camera 10 is used. However, an image obtained by combining real images taken by two or more cameras is used. It is also possible to use. In addition, the camera is not limited to a camera capable of shooting in color, but a camera for shooting in black and white is sufficient, and a monitor for displaying in black and white is sufficient. In this case, instead of displaying each line or each surface by color, the difference may be represented by a difference in luminance or a difference in line width.
[0052]
Further, the case where the vehicle is moved backward is described as an example. However, the above-described embodiment can be applied to a case where the vehicle traveling prediction trajectory is stereoscopically viewed on the monitor screen when the vehicle is moved forward. Further, in the embodiment, the viewpoint conversion and the lens distortion correction are performed on the captured image of the camera and the trajectory data, but the viewpoint conversion and the lens distortion correction are not always necessary. However, in combination with viewpoint conversion or the like, it is possible to provide a driver with images that are easy to see from various viewpoints.
[0053]
【The invention's effect】
According to the present invention, it is possible to provide a driving support device that can intuitively stereoscopically view a vehicle locus image on a two-dimensional monitor screen.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a driving support device according to one embodiment of the present invention; FIG. 2 is an image diagram when the driving support device according to one embodiment of the present invention is mounted on a vehicle; FIG. 4 is an explanatory diagram when the driving support device according to one embodiment of the present invention calculates trajectory plane data on the ground in a world coordinate system. FIG. 4 is a diagram illustrating the ground calculated by the driving support device according to one embodiment of the present invention. FIG. 5 is a view showing an example of a screen display of the upper trajectory data. FIG. 5 is a diagram in which trajectory data on the side of the vehicle calculated by the driving support device according to one embodiment of the present invention is combined with trajectory data on the ground. FIG. 7 is a view showing an example of an image in which vehicle trajectory data calculated by the driving support device according to one embodiment of the present invention is displayed on a screen. FIG. 7: A vehicle trajectory calculated by the driving support device according to one embodiment of the present invention. FIG. 8 is a diagram showing an example of an image in which data has been enhanced and corrected. [EXPLANATION OF SYMBOLS] shows an enhancement correction example of a vehicle trajectory data in the driving support apparatus according to another embodiment of the flow chart Figure 9] the invention showing a processing procedure in the driving support apparatus according to an embodiment of the invention
Reference Signs List 1 vehicle 3 vehicle rear bumper 5 shift levers 7a to 7d wheels 8 handle 10 camera 20 handle angle sensor 30 steering angle calculation unit 40 image processing unit 41 CPU
42 program memory 43 work memory 44 image storage unit 45 viewpoint conversion map data storage unit 46 driving support data generation unit 47 trajectory map data storage unit 48 image synthesis unit 50 display control unit 60 monitors 110a and 111a trajectory surfaces 115 and 125a on the side of the vehicle , 126a Emphasis lines for stereoscopic vision assistance 120a, 121a, 122a, 123a Emphasis lines indicating bumper positions

Claims (6)

An in-vehicle camera, an in-vehicle monitor that displays a captured image of the in-vehicle camera, and an image synthesizing unit that superimposes, on the captured image, a vehicle trajectory image indicating a predicted traveling direction of a vehicle when displaying the captured image on the in-vehicle monitor. The vehicle locus image is displayed in a U-shape including a locus surface on the ground in the vehicle traveling direction and a locus surface on both sides of the vehicle, and a position floating between the locus surfaces on the ground on both sides of the vehicle. And a driving support data generating means for displaying the driving support data on the in-vehicle monitor connected by one or two lines. The driving support device according to claim 1, wherein the captured image and the vehicle locus image are displayed on the on-vehicle monitor as viewpoint-converted images. The driving support device according to claim 1 or 2, wherein the locus surfaces on both sides of the vehicle have a height up to a vehicle height or a bumper position. The driving support device according to any one of claims 1 to 3, wherein a bumper position is highlighted in a locus plane on both sides of the vehicle. The driving support apparatus according to claim 4, wherein the highlighting of the bumper position is displayed on a plane having a predetermined width. The driving support device according to any one of claims 1 to 5, wherein a distance display is not displayed in the track surface on the ground.

Images