JP2008149879A - Operation assisting system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an operation assisting system capable of achieving accurate parking operation assistance or safety confirmation assistance by performing display change control of an optimal monitor video image according to the operating situation when turning. <P>SOLUTION: The operation assisting system is equipped with a monitor image data generation means for generating monitor image data displayed on a monitor 3 in a vehicle cabin from camera video image data of a rear camera 1 by view point conversion using a virtual camera 8 set at a position different from the rear camera 1 installed in the vehicle and a virtual projection plane set on a subject side displayed by the rear camera 1. The monitor image data generation means is for increasing the angle of view which is a range in which a lens of the virtual camera 8 can project on a virtual 3D CCD 9 as the degree of a steering wheel steering angle θ sensed by a steering wheel steering angle sensor 4 increases from a neutral position when turning and traveling backward, and generating monitor image data displayed on the monitor 3 by view point conversion using the virtual camera 8, a virtual 3D screen 7, and the virtual 3D CCD 9, by view angle control. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a driving support device that generates monitor image data from camera video data by viewpoint conversion using a virtual camera and a virtual stereoscopic projection plane / virtual stereoscopic imaging plane (two-plane stereoscopic model), and in particular, based on steering. The present invention relates to a driving support device that controls an angle of view by a two-plane solid model.

  Conventionally, as a driving support device, in order to provide a driver with an easy-to-understand image by continuously changing the screen configuration and the viewpoint position according to the movement of the vehicle, in conjunction with the movement of the vehicle In addition, there are known ones that switch between a single viewpoint video that displays a plurality of camera videos taken by a plurality of photographing devices as a bird's eye view viewed from one viewpoint and a multi-view video that displays a plurality of camera videos on a divided screen. (For example, refer to Patent Document 1).

In addition, as a conventional image generation apparatus, an image of a plurality of images captured by several cameras is not displayed independently of each other, but the entire area captured by several cameras Is known to display an image synthesized on one sheet (see, for example, Patent Document 2).
JP 2005-236493 A Japanese Patent No. 3286306

  However, in the driving support device of Patent Document 1 and the image generation device of Patent Document 2, the angle of view, which is a range in which the lens of the virtual camera can be projected on the imaging surface, is constant regardless of the steering angle of the steering wheel. Therefore, for example, the monitor video by the viewpoint conversion method with a constant angle of view may remain only a close-up bird's-eye video with the same visual field range, or only a distant perspective video with the same video visual field range. In this way, video is displayed with a fixed visual field.

  In this way, in order to cope with any scene with video display with a fixed visual field, there is no choice but to display the most general or compromise, in the driving assistance device using the rear camera, As a result, the display is based on the angle of view setting so that the vehicle rear can be visually recognized when going straight ahead. In this case, there is a problem that in the driving scene when turning backward, it is not possible to display an image of the diagonally backward traveling direction to be displayed on the monitor screen.

  As a solution to solve this problem, for example, there is a proposal that the rear camera is a zoom camera and the angle of view of the monitor image is adjusted according to the driving scene, but the monitor image is displayed during complicated driving operations such as parking. It is inconvenient to adjust the angle of view by manual operation, and there is a risk that safety may be impaired.

  The present invention has been made paying attention to the above problem, and proposes a view angle control method of a two-dimensional solid model based on steering, and performs optimal display change control of the monitor image according to the driving scene at the time of turning. Thus, an object of the present invention is to provide a driving support device that can achieve accurate parking operation support and safety check support.

In order to achieve the above object, in the present invention, by viewpoint conversion using a virtual camera set at a position different from the real camera installed in the vehicle and a virtual projection plane set on the subject side projected by the real camera, In a driving support apparatus provided with monitor image data generating means for generating monitor image data to be displayed on a monitor in a vehicle interior from camera image data of a real camera,
Virtual stereoscopic projection plane setting means for setting a virtual stereoscopic projection plane having a near-view projection plane that obtains a bird's-eye view image and a distant view projection plane that obtains an oblique viewpoint image as the virtual projection plane;
Virtual stereoscopic imaging plane setting means for setting a virtual stereoscopic imaging plane having a near-view imaging plane for obtaining a bird's-eye view image and a distant view imaging plane for obtaining an oblique viewpoint video as the virtual imaging plane of the virtual camera;
A steering wheel angle detecting means for detecting a steering wheel angle;
The monitor image data generating means enlarges the angle of view, which is a range in which the lens of the virtual camera can be projected on the virtual stereoscopic imaging surface, as the steering angle of the steering wheel from the neutral position increases when the vehicle turns. Monitor image data to be displayed on the monitor is generated by viewpoint conversion using a virtual camera, a virtual stereoscopic projection plane, and a virtual stereoscopic imaging plane by angle control.

  The “view angle”, which is the range that the virtual camera lens can project on the virtual stereoscopic imaging surface, and the “focal length”, which is the distance from the principal point that is the center of the virtual camera lens to the virtual stereoscopic imaging surface. , That relationship corresponds. Therefore, in the present invention, it is described as “view angle control”, but may be described as “focal length control”, and both “view angle control” and “focal length control” are treated as the same control.

Therefore, in the driving support device of the present invention, the steering wheel steering angle is detected by the steering wheel steering angle detecting means. Then, in the monitor image data generation means, when the vehicle turns, the angle of view that is the range in which the lens of the virtual camera can be projected on the virtual stereoscopic imaging surface increases as the steering wheel steering angle from the neutral position increases. Monitor image data to be displayed on a monitor is generated by viewpoint conversion using a virtual camera, a virtual stereoscopic projection plane, and a virtual stereoscopic imaging plane by angle control.
In general, when going straight back, a wide field of view is not required, so a narrow view angle display that shows a larger view of obstacles in the rear landscape than a wide view view of obstacles in the rear view is shown. However, when the vehicle is turning backwards, such as when parked, it is preferable to display a wide angle of view that can display an obstacle or the like in a direction to advance by turning rather than a narrow angle of view.
On the other hand, in the present invention, when the steering angle of the steering wheel is at the neutral position, an overhead video area and an oblique visual area are set on the monitor screen based on a predetermined area distribution, and the steering angle is increased from the neutral position. Indeed, for example, the field angle of the overhead view video area is expanded in the vehicle width direction, and the field angle of the oblique viewpoint video area is expanded in the vehicle width direction and the vehicle upward direction. That is, when turning, optimal monitor image display change control is performed in accordance with the driving scene, in which the angle of view of the monitor screen is enlarged as the steering wheel steering angle increases.
As described above, by performing the angle-of-view change control based on the steering angle of the steering wheel, it is possible to accurately acquire the expected course area information by simply looking at the monitor screen regardless of the size of the steering wheel steering angle. Parking operation support and safety confirmation support are achieved.
As a result, it is possible to achieve appropriate parking operation support and safety confirmation support by performing optimal display video display change control according to the driving scene during turning.

  Hereinafter, the best mode for realizing the driving support device of the present invention will be described based on Examples 1 to 4 shown in the drawings.

First, the configuration will be described.
FIG. 1 is an overall system diagram illustrating a backward driving support device (an example of a driving support device) according to the first embodiment. FIG. 2 is a schematic diagram for explaining an example of a viewpoint conversion method using a virtual camera and a virtual stereoscopic screen / virtual stereoscopic CCD (two-dimensional stereoscopic model) by angle-of-view control based on the steering angle of the steering wheel in the backward driving support device of the first embodiment. FIG. FIG. 3 is a monotone characteristic diagram showing the relationship of the angle of view with respect to the steering angle of the steering wheel in the reverse driving assistance device of the first embodiment. 4A and 4B are monitor images when the vehicle is reversely rotated in the reverse drive assist device of the first embodiment. FIG. 4A shows a monitor image when the steering angle of the steering wheel is in the neutral range, and FIG. 4B shows a large steering angle of the steering wheel. The monitor image at the time is shown.

  As shown in FIG. 1, the backward driving support device in the first embodiment includes a rear camera 1 (actual camera), an image processing controller 2, a monitor 3, and a steering wheel steering angle sensor 4 (steering wheel steering angle detecting means). A shift lever position sensor 5, a virtual camera position adjustment operation dial 6, a virtual stereoscopic screen 7 (virtual stereoscopic projection plane), a virtual camera 8, a virtual stereoscopic CCD 9 (virtual stereoscopic imaging plane), and a steering system handle 10. And a shift lever 11. Here, “CCD” is an abbreviation for “Charge Coupled Device” and refers to a charge coupled device.

  The backward driving support device of Example 1 uses a virtual camera 8 set at a position different from the rear camera 1 installed in the vehicle, and a virtual stereoscopic screen 7 set on the subject side projected by the rear camera 1. This is a device that generates monitor image data to be displayed on the monitor 3 from the camera video data of the rear camera 1 by viewpoint conversion.

  As shown in FIG. 1, the rear camera 1 is attached to a rear position of the vehicle and projects a rear view of the vehicle. Camera video data is acquired from an image projected on the actual imaging surface (rear camera CCD) of the rear camera 1.

On the subject side (lens axis side) of the rear camera 1, as shown in FIGS. 1 and 2, a virtual stereoscopic screen 7 as a two-dimensional stereoscopic model is set as a virtual projection plane (virtual stereoscopic projection plane setting). means).
The virtual stereoscopic screen 7 includes a near view screen 71 (projection plane for near view) that obtains a bird's eye view set along the ground, and a distant view that obtains an oblique viewpoint image set by connecting the near view screen 71 with an upward inclination angle α. Screen 72 (distant view projection plane).

  Further, as shown in FIGS. 1 and 2, the virtual camera 8 is set at a position higher than the rear camera 1 at a position defined by the horizontal distance a from the rear camera 1 and the vertical distance b from the ground. Is done.

  Further, as shown in FIGS. 1 and 2, a virtual stereoscopic CCD 9 as a two-dimensional stereoscopic model similar to the virtual stereoscopic screen 7 is set as a virtual imaging plane of the virtual camera 8 (virtual stereoscopic imaging plane setting means). .

  When the virtual stereoscopic CCD 9 is moved from the back side to the front side of the virtual camera 8, the virtual stereoscopic CCD 9 is set in parallel to the foreground screen 71 to obtain a bird's-eye view image 91 (foreground imaging surface), and upward from the foreground CCD 91. It has a distant view CCD 92 (distant view image pickup surface) that obtains an oblique viewpoint image connected in accordance with an inclination angle β. In the first embodiment, since the relationship between the inclination angle α and the inclination angle β is α = β, the virtual stereoscopic screen 7 and the virtual stereoscopic CCD 9 maintain a similar shape.

  As shown in FIG. 1, the image processing controller 2 includes an inter-decoder conversion unit 21, a coordinate conversion processing unit 22, a ROM 23, a RAM 24, and an encoder conversion unit 25.

  The inter-decoder conversion unit 21 is based on the difference between the decoder joined to the rear camera 1 and the decoder assumed by the coordinate conversion processing unit 22, and the decoder conversion coordinates from the camera input coordinate system between the two decoders. Convert the data coordinate system to a system. The “decoder” refers to software that restores encoded data based on a certain rule and extracts the original data.

  The coordinate conversion processing unit 22 inputs the decoder conversion coordinate system from the inter-decoder conversion unit 21 as single camera video data acquired by the rear camera 1 and uses a mapping table stored in advance in the ROM 23. Then, each pixel of the camera video data is moved onto the virtual stereoscopic CCD 9 of the virtual camera 8 according to the coordinate transformation, and an image projected on the virtual stereoscopic CCD 9 when the virtual stereoscopic screen 7 is viewed from the virtual camera 8 is used as a monitor image. Monitor image data for obtaining this monitor image is generated (monitor image data generating means).

  The ROM 23 designates one pixel position on the virtual stereoscopic CCD 9, determines a first corresponding position corresponding to one designated pixel position on the virtual stereoscopic screen 7, and sets the first corresponding position on the imaging surface of the rear camera 1. This is a memory for storing and setting a mapping table created by performing coordinate conversion of each pixel position in the order of determining the second corresponding position corresponding to the corresponding position.

  When creating this mapping table, coordinate conversion is performed between each pixel position of the single camera video data acquired by the rear camera 1 and each pixel position on the virtual stereoscopic CCD 9 of the virtual camera 8 via the virtual stereoscopic screen 7. Is done. In addition, when creating the mapping table, a conversion amount relationship characteristic between the pixel distance from the optical axis position by the distorted image and the pixel distance from the optical axis position by the undistorted image is determined in advance, and the distance from the optical axis position of each pixel is determined. The coordinate system distortion of each pixel position of single camera video data acquired by the rear camera 1 is corrected and converted using the distance and the conversion amount relation characteristic, and the coordinate system of the virtual stereoscopic CCD 9 is obtained. In the mapping table, a plurality of mapping tables are stored and set according to the position (a, b) of the virtual camera 8 and the angle of view of the two-dimensional stereoscopic model by the virtual stereoscopic screen 7 / virtual stereoscopic CCD 9. ing. Further, for example, an interpolation method is used for fine camera position setting and angle of view setting.

The RAM 24 is a memory for storing and setting rewritable information.
The first embodiment is an example in which a mapping table created based on the viewpoint conversion method is stored and set in the ROM 23 in advance, and coordinate conversion processing is performed to obtain camera image data as monitor image data displayed on the monitor 3. . However, for example, in the case of a system that is equipped with hardware having a high calculation processing speed and generates monitor image data while performing coordinate conversion in real time, a coordinate conversion formula for each pixel or the like is stored and set in the RAM 24. Keep it.

  The encoder conversion unit 25 converts the monitor image data generated by the coordinate conversion processing unit 22 into image data to be displayed on the monitor 3, including, for example, left / right inversion processing corresponding to the driver's viewpoint. The “encoder” refers to software that encodes data based on a certain rule.

  The monitor 3 is set to an instrument panel position in the vehicle interior, etc., and based on the image data from the image processing controller 2, the monitor 3 displays a close-up view of the vehicle rear side view as a bird's-eye view image. Is displayed with an oblique viewpoint image. As shown in FIGS. 4 (a) and 4 (b), the vehicle illustration 12 is displayed on the monitor screen 31 at the lower center of the screen in addition to the overhead view video display of the foreground and the oblique viewpoint video display of the distant view. .

  When the steering system handle 10 is operated, the steering angle sensor 4 detects the steering angle θ from the steering neutral position together with the steering direction. In the system of the first embodiment, handle operation direction information and handle steering angle information are obtained from the handle steering angle sensor 4.

  The shift lever position sensor 5 detects a lever operation position when the shift lever 11 is operated. For example, in the case of the shift lever 11 of an automatic transmission, the shift range position (P) and the drive range position (D ), A sensor signal corresponding to a selected operation position such as a neutral range position (N), a reverse range position (R) or the like is output. In the system of the first embodiment, reverse range position selection information (= reverse information) is obtained from the shift lever position sensor 5.

  The virtual camera position adjustment operation dial 6 is virtual camera position adjustment operation means for setting the virtual camera 8 at an arbitrary space position by an external setting operation by a driver or the like. In the virtual camera position adjusting operation dial 6, as shown in FIG. 1, a horizontal distance a (for example, 2 m) from the rear camera 1 and a vertical distance b (for example, 3 m) from the ground are independently set. It can be set.

  In the coordinate conversion processing unit 22 serving as the monitor image data generating means in the first embodiment, the lens of the virtual camera 8 is copied onto the virtual stereoscopic CCD 9 as the steering wheel steering angle θ from the neutral position increases as the vehicle turns backward. Monitor image data to be displayed on the monitor 3 is generated by enlarging the field angle that can be output and viewpoint conversion using the virtual camera 8, the virtual stereoscopic screen 7, and the virtual stereoscopic CCD 9 by controlling the angle of view.

  In the coordinate conversion processing unit 22, the steering wheel steering angle θ is set to the neutral position when the steering system handle 10 is turned to the right and the vehicle is turned counterclockwise or when the steering system handle 10 is turned to the left and the vehicle is turned backward. At this time, the monitor screen 31 is set to the overhead view video area 3B and the oblique viewpoint video area 3P based on a predetermined area distribution (for example, 50%: 50%) (see FIG. 4A), and the steering angle θ of the steering wheel As the distance from the neutral position increases, the field of view of the overhead view video area 3B is expanded in the vehicle width direction, and the field of view of the oblique viewpoint video area 3P is expanded in the vehicle width direction and the vehicle upward direction (see FIG. 4 (b)). Note that the video display on the monitor screen 31 is a full screen display regardless of the steering wheel steering angle θ.

Further, even when turning, the coordinate conversion processing unit 22 handles the steering angle θ when the steering angle θ from the neutral position is in the range of the insensitive angle θ ₀ or less as shown in FIG. The angle of view is fixed regardless of the slight change in the angle. When turning, when the steering wheel steering angle θ from the neutral position is in an angle region exceeding the dead angle θ ₀ , the monotonic characteristic increases in proportion to the steering wheel steering angle θ (FIG. 3). The control for changing the angle of view is performed.

Next, the operation will be described.
The backward driving support apparatus according to the first embodiment generates monitor image data from camera video data by viewpoint conversion using the virtual camera 8 and the virtual stereoscopic screen 7 / virtual stereoscopic CCD 9 (two-plane stereoscopic model). The video displayed on the monitor 3 is used as driving support information.
The invention point of the first embodiment is to control the angle of view by the virtual camera 8 and the virtual stereoscopic screen 7 / virtual stereoscopic CCD 9 (two-plane stereoscopic model) based on the steering angle θ, in other words, obliquely depending on the steering angle θ. This is to change and control the size of the viewpoint video area.
Hereinafter, “monitor image generation operation from camera image data” and “driving support operation” will be described as operations in the backward driving support device of the first embodiment.

[Monitor video generation from camera video data]
The present inventor has previously proposed a viewpoint conversion technique for a rear camera image using a two-plane stereoscopic screen / two-plane stereoscopic CCD (two-plane stereoscopic model). With this technology, the road surface image of the vehicle close-up view is converted into a bird's-eye view image (= top view image) viewed from directly above, and the distant view image is converted into an oblique view image (= diagonal view image) with a downward view. Are displayed simultaneously.

  However, in the viewpoint conversion technology for rear camera images using a two-dimensional solid model, the angle of view is fixed regardless of the steering angle of the steering wheel, so the viewpoint conversion method using the two-dimensional solid model with this fixed angle of view is used. The monitor video is, for example, a video display in which the visual field range of a near view overhead video region and a far view oblique perspective video region is fixed.

In this way, in video display with a fixed angle of view, a display with a narrow field of view and an emphasis on straight backwards, a display with a wide field of view and an emphasis on reverse rotation, a straight reverse and reverse rotation are compromised. There is no choice but to select from three patterns of display, that is, an angle of view depending on the field of view.
When monitor display with a narrow field of view (telephoto lens) is selected, the presence of distant obstacles can be confirmed when traveling forward or when moving straight back at high speed, but turning at low speed When the vehicle is parked backward, the moving direction image is out of the visual field range, and obstacles and the like existing in the moving direction cannot be confirmed.
In addition, when the monitor display with a wide field of view (wide-angle lens) is selected, obstacles that exist in the direction of travel can be confirmed when parked to turn backward at a low speed. When the vehicle is moving in a straight line at high speed, distant obstacles are projected on the screen, and the presence of the obstacles cannot be confirmed at an early stage.
In addition, if you select the angle of view display based on a compromised field of view (standardized lens) for straight backwards and backwards, you will be able to confirm the presence of obstacles that exist in the direction of travel regardless of whether you are going backwards or going backwards. It becomes half-finished and cannot satisfy the demand for reliably obtaining the expected course area information when entering or leaving the garage.

  On the other hand, in the backward driving support device of the first embodiment, when the vehicle turns backward with the steering system handle 10 turned to the right, or when the vehicle goes backward with the steering wheel 10 turned to the left, the vehicle turns backward. The shift lever position sensor 5 detects a reverse selection operation and the steering wheel steering angle sensor 4 detects a steering wheel steering angle θ. When the vehicle turns backward, the coordinate conversion processing unit 22 enlarges the angle of view that is the range in which the lens of the virtual camera 8 can be projected onto the virtual stereoscopic CCD 9 as the steering wheel angle θ from the neutral position increases. Monitor image data to be displayed on the monitor 3 is generated by viewpoint conversion using the virtual camera 8, the virtual stereoscopic screen 7, and the virtual stereoscopic CCD 9 by angle control.

  In general, when going straight back, a wide field of view is not required, so a narrow view angle display that shows a larger view of obstacles in the rear landscape than a wide view view of obstacles in the rear view is shown. Is preferable. On the other hand, when the vehicle is turned backward during parking or the like, it is desirable to display a wide angle of view that can capture an obstacle or the like in a direction to advance by turning rather than a narrow angle of view.

  On the other hand, in the first embodiment, when the steering wheel steering angle θ is in the neutral position, as shown in FIG. 4A, the overhead view video area 3B and the oblique viewpoint video are displayed on the monitor screen based on a predetermined area distribution. As the area 3P is set and the steering wheel steering angle θ increases from the neutral position, the field angle of the overhead view video area 3B is expanded in the vehicle width direction and the oblique viewpoint video area 3P as shown in FIG. 4B. Is expanded in the vehicle width direction and the vehicle upward direction. As is clear from the comparison between FIGS. 4 (a) and 4 (b), the vehicle illustration 12 is displayed in a smaller size on the monitor screen 31 as the angle of view increases, so that the degree of expansion of the angle of view is observed on the screen. Can be easily determined.

Further, the coordinate conversion processing unit 22 handles the steering wheel angle θ when the steering wheel steering angle θ from the neutral position is in the insensitive angle θ ₀ or less as shown in FIG. Control for fixing the angle of view is performed regardless of the slight change in the angle (see FIG. 4A).

In addition, when the vehicle is turning backward, when the steering wheel angle θ from the neutral position is in an angle region that exceeds the dead angle θ ₀ , as shown in FIG. 3, it is proportional to the steering wheel angle θ. Control for changing the angle of view is performed by the increasing monotonic characteristics.

[Driving support action]
As described above, when the steering wheel is rotated backward, when the steering angle of the steering wheel θ is in the neutral position area equal to or less than the insensitive angle θ _{0, the} angle of view of the monitor screen 31 is fixed, and the magnitude of the steering angle of the steering wheel θ from the neutral position is insensitive. When the angle θ ₀ is exceeded, control is performed to enlarge the angle of view of the monitor screen 31 as the steering angle θ of the steering wheel increases from the neutral position. That is, at the time of turning, optimal monitor image display change control is performed according to the driving scene, in which the angle of view of the monitor image displayed on the monitor screen 31 is enlarged as the steering wheel steering angle θ increases.

  Thus, by performing the angle-of-view change control based on the steering wheel steering angle θ, it is possible to reliably obtain the expected course area information simply by gazing at the monitor screen 31 regardless of the magnitude of the steering wheel steering angle θ. Accurate parking operation support and safety check support are achieved.

  For example, when entering a garage or leaving a garage, a monitor display in which the angle of view becomes larger as the steering wheel steering angle θ is larger can be confirmed without missing an obstacle present in the traveling direction. Accurate driving support is achieved through reliable acquisition of route area information.

  Also, when parking in an outdoor parking lot, etc., when the vehicle is turning backwards with a large steering angle θ, it is possible to obtain the predicted route area information reliably by the monitor display with an enlarged angle of view (reduced image). It becomes parking operation support at the time of parking accompanied by turning operation.

  Also, when going straight ahead with the steering angle θ as the neutral range, the monitor display with a reduced angle of view (enlarged image) allows you to quickly confirm the presence of obstacles, etc. at a distant place, It will be a safety confirmation support when running straight.

Further, in the first embodiment, when the steering wheel steering angle θ from the neutral position is in the region of the dead angle θ ₀ or less, the control for fixing the angle of view of the monitor screen 31 is performed regardless of the minute change in the steering wheel steering angle θ. Done. For this reason, the image on the monitor screen 31 does not fluctuate in response to a slight steering operation during reverse travel while performing corrective steering so as to ensure straight reverse of the vehicle behavior, and straight reverse with corrective steering. At this time, it is possible to secure the monitor screen 31 with a stable bird's-eye view image and an oblique viewpoint image without any blur on the monitor screen 31.

In addition, in the first embodiment, when the steering wheel is moved backward, when the magnitude of the steering wheel steering angle θ from the neutral position is an angle region exceeding the dead angle θ ₀ , it increases in proportion to the steering wheel steering angle θ. Control for changing the angle of view is performed by monotonic characteristics. For this reason, the video of the monitor screen 31 does not give the driver a sense of incongruity as the angle of view increases or the angle of view decreases according to the steering angle θ given by the driver.

Next, the effect will be described.
In the backward driving support device according to the first embodiment, the effects listed below can be obtained.

  (1) From the camera video data of the real camera by viewpoint conversion using the virtual camera 8 set at a position different from the real camera installed in the vehicle and the virtual projection plane set on the subject side projected by the real camera In the driving support apparatus including monitor image data generating means for generating monitor image data to be displayed on the monitor 3 in the passenger compartment, a near-view screen 71 for obtaining a bird's-eye view image and a distant view screen 72 for obtaining an oblique viewpoint image are used as the virtual projection plane. A virtual stereoscopic projection plane setting means for setting a virtual stereoscopic screen 7 having a virtual stereoscopic CCD 7 and a virtual stereoscopic CCD 9 having a near view CCD 91 for obtaining a bird's-eye view image and a far view CCD 92 for obtaining an oblique view point image are set as the virtual imaging surface of the virtual camera 8. A virtual three-dimensional imaging surface setting means for controlling and a steering wheel steering angle sensor 4 for detecting a steering wheel steering angle θ. When the vehicle turns, the data generation means enlarges the angle of view that is the range in which the lens of the virtual camera 8 can be projected on the virtual stereoscopic CCD 9 as the steering angle θ from the neutral position increases, and controls the angle of view. In order to generate monitor image data to be displayed on the monitor 3 by viewpoint conversion using the virtual camera 8, the virtual stereoscopic screen 7 and the virtual stereoscopic CCD 9, optimal monitor image display change control according to the driving scene is performed during turning. By doing so, accurate parking operation support and safety confirmation support can be achieved. In particular, the visibility during the rotational movement is improved as compared with the fixed two-dimensional solid model, which is effective for accurate parking operation and safety confirmation.

  (2) The real camera is the rear camera 1 set at the rear position of the vehicle, and the monitor image data generating means is when the steering system handle 10 is turned backward when turning the steering system handle 10 to the right or when the steering system handle 10 is turned backward. When the steering wheel is turned to the left and the steering wheel steering angle θ is in the neutral position, the monitor screen 31 is set to the overhead view video area 3B and the oblique viewpoint video area 3P based on the predetermined area allocation, and the steering wheel is steered. As the angle θ increases from the neutral position, monitor image data that expands the field of view of the overhead view video area 3B in the vehicle width direction and expands the field of view of the oblique viewpoint video area 3P in the vehicle width direction and the vehicle upward direction. Therefore, the optimal display change control of the monitor image 31 according to the driving scene can be performed when the vehicle is turning backward. In particular, when putting a garage into or leaving a garage, it is possible to provide accurate driving support according to a change in the situation.

(3) The monitor image data generation means, when moving backward, when the steering wheel steering angle θ from the neutral position is in a region where the size of the steering wheel steering angle θ is equal to or less than the insensitive angle θ ₀ , Since the monitor image data to be fixed is generated, a stable monitor image without blurring can be secured on the monitor screen 31 at the time of straight backward movement with correction steering.

(4) The monitor image data generating means is proportional to the magnitude of the steering wheel angle θ when the steering wheel steering angle θ from the neutral position is an angle region exceeding the dead angle θ ₀ when the vehicle is turning backward. Since the angle of view is changed by the monotonic characteristic that increases, the image on the monitor screen 31 changes according to the steering angle θ of the steering wheel given by the driver, and the driver does not feel uncomfortable.

  The second embodiment is an example in which the angle of view is changed by a step type characteristic that increases at a stroke when the steering angle of the steering wheel exceeds the dead angle.

First, the configuration will be described.
FIG. 5 is a step type characteristic diagram showing the relationship of the angle of view with respect to the steering angle of the steering wheel in the backward driving support device of the second embodiment.

In the coordinate conversion processing unit 22 serving as the monitor image data generating means in the second embodiment, when the steering wheel is moved backward, the steering wheel steering is performed when the steering wheel steering angle θ from the neutral position is in the angle region exceeding the dead angle θ _0. When the angle θ exceeds the dead angle θ ₀ , the angle of view is changed by step-type characteristics that increase at a stretch. Since other configurations are the same as those of the first embodiment, illustration and description thereof are omitted.

Next, the operation will be described. In the second embodiment, the control for changing the angle of view by the step type characteristic that increases at once when the steering wheel steering angle θ from the neutral position exceeds the dead angle θ ₀ when the vehicle turns backward is performed. Done.

Therefore, when the steering wheel angle θ exceeding the dead angle θ ₀ is given to the video on the monitor screen 31 based on the driver's intention to turn, for example, as shown in FIG. From the state in which the oblique viewpoint video area 3P is displayed with continuity, as shown in FIG. 4B, the angle of view of the overhead view video area 3B and the oblique viewpoint video area 3P is expanded at a stretch. The angle of view of the oblique viewpoint video in the video area 3P is switched on / off to a display state in which the angle of view is rapidly advanced.

For this reason, instead of manually changing the zoom of the monitor display, the full-screen monitor display and turning with a preferred narrow angle of view is automatically performed depending on whether the steering angle θ exceeds the dead angle θ ₀ or not. It is possible to switch between full-screen monitor display with a wide angle of view that is preferable during reverse travel.

Furthermore, when the steering angle θ increases beyond the dead angle θ ₀ , the angle of view gradually increases, so that the angle of view can be optimized in accordance with the steering operation when the vehicle is turning backward. Other functions are the same as those in the first embodiment.

Next, the effect will be described.
In the backward driving support device of the second embodiment, in addition to the effects (1), (2), and (3) of the first embodiment, the following effects can be obtained.

(5) When the monitor image data generation means is in the angle region where the steering wheel steering angle θ from the neutral position exceeds the dead angle θ ₀ during backward turn, the steering wheel steering angle θ changes to the dead angle θ ₀ . Since the angle of view changes with a step-type characteristic that increases at once, the distance of the steering wheel steering angle θ automatically exceeds the dead angle θ _0. The display can be switched, and when the steering wheel steering angle θ exceeds the dead angle θ ₀ , the angle of view can be optimized in accordance with the steering wheel cutting operation.

  In the third embodiment, the angle of view is changed by an acceleration type characteristic in which the steering angle gradually increases in the small steering angle region where the steering angle exceeds the dead angle, and the increasing gradient becomes steeper as the steering angle becomes farther from the small steering angle region. This is an example.

First, the configuration will be described.
FIG. 6 is an acceleration characteristic diagram showing the relationship of the angle of view with respect to the steering angle of the steering wheel in the reverse driving assistance device of the third embodiment.

In the coordinate conversion processing unit 22 serving as the monitor image data generating means in the third embodiment, when the steering wheel is moved backward, the steering wheel steering is performed when the steering wheel steering angle θ from the neutral position is in the angle region exceeding the dead angle θ _0. The angle of view is changed by the acceleration type characteristic in which the angle θ gradually increases in the small steering angle region where the dead angle θ ₀ exceeds the dead angle θ ₀ and the increasing gradient becomes steeper as the steering wheel steering angle θ is separated from the small steering angle region. Yes. Since other configurations are the same as those of the first embodiment, illustration and description thereof are omitted.

Next, the operation will be described. In the third embodiment, the steering wheel angle θ from the neutral position gradually increases in the small steering angle region where the dead angle θ ₀ exceeds the insensitive angle θ _0, and the steering wheel steering angle θ is reduced to the small steering angle. Control to change the angle of view is performed by an acceleration characteristic in which the increasing gradient becomes steeper as the distance from the region increases.

Therefore, even if the driver gives a steering angle θ that exceeds the dead angle θ ₀ when the driver performs a slow steering operation or repeats the steering operation and the steering wheel return operation, for example, As shown in FIG. 4 (a), a full screen display state in which the field of view of the oblique viewpoint video region 3P of the monitor screen 31 is slightly enlarged is obtained. For this reason, when the driver performs a slow steering operation or repeats the steering wheel cutting operation and the steering wheel returning operation, the angle of view fluctuation due to the enlargement / reduction of the oblique viewpoint video area 3P in the monitor display is reduced. Can be suppressed.
Then, when the driver gives a large steering angle θ with a willingness to turn backward in one direction, all of the overhead view video area 3B and the oblique viewpoint video area 3P by the predetermined area allocation shown in FIG. From the screen display state, as shown in FIG. 4B, the full screen display state is changed so that the angle of view of the overhead view video in the overhead view video region 3B and the oblique viewpoint video in the oblique viewpoint video region 3P is accelerated. Is done.
For this reason, at the time of reverse travel, the monitor display can be optimized in accordance with the driver's intention to travel backward by changing to a display state with a wide angle of view in response to the turning operation of the steering wheel. Other functions are the same as those in the first embodiment.

Next, the effect will be described.
In the backward driving support device of the third embodiment, in addition to the effects (1), (2), and (3) of the first embodiment, the following effects can be obtained.

(6) When the monitor image data generation means is in the angle region where the steering wheel steering angle θ from the neutral position exceeds the dead angle θ ₀ when the vehicle turns backward, the steering wheel steering angle θ changes to the dead angle θ ₀ . The angle of view changes due to the acceleration type characteristics that increase gradually in the small steering angle range that exceeds, and the steering gradient θ becomes steep as the steering angle θ becomes farther away from the small steering angle range. Monitor display state that can suppress fluctuations in the monitor display when performing or repeatedly performing the steering wheel turning operation and the steering wheel return operation, and at the time of the steering wheel turning operation, the monitor display state expands the angle of view with good response in response to the driver's intention to move backward. Can be changed to

  The fourth embodiment is a composite control example in which the angle-of-view change control based on the steering wheel steering angle of the first to third embodiments is combined with the rotation control of the virtual camera based on the steering wheel steering angle.

First, the configuration will be described.
FIG. 7 is a schematic diagram for explaining an example of a viewpoint conversion method using a virtual camera, a virtual stereoscopic screen, and a virtual stereoscopic CCD in which the position / posture is controlled according to the steering angle in the backward driving support device of the fourth embodiment. is there.

  In the coordinate conversion processing unit 22 as the monitor image data generating means in the fourth embodiment, the virtual camera 8 rotates in the turning direction of the vehicle according to the magnitude of the steering angle θ of the steering wheel from the neutral position when the vehicle turns backward. By maintaining the relative positional relationship between the virtual stereoscopic CCD 9 of the rotating virtual camera 8 and the virtual stereoscopic screen 7, and by performing viewpoint conversion using the rotating virtual camera 8, the virtual stereoscopic CCD 9, and the virtual stereoscopic screen 7, And monitor image data for displaying on the monitor 3 an image that seamlessly joins the oblique perspective images of the distant view. Since other configurations are the same as those in the first to third embodiments, illustration and description thereof are omitted.

Next, the operation will be described.
When the left turn of the steering system handle 10 is turned to the right or when the left turn of the steering system handle 10 is turned to the left, the shift lever position sensor 5 detects the reverse selection operation and The steering angle sensor 4 detects the steering wheel steering angle θ. When the vehicle turns backward, the coordinate conversion processing unit 22 rotates and displays the vehicle illustration 12 at the lower center position of the screen in accordance with the steering angle θ, and the predicted position where the vehicle is expected to pass in the future (this predicted position). For example, may be displayed in the upper center of the monitor screen 31.) The virtual camera 8 is rotated in the turning direction of the vehicle and the position of the virtual camera 8 is moved in parallel with the ground so that it does not move.

  Then, monitor image data is generated in the coordinate conversion processing unit 22 by viewpoint conversion using the virtual camera 8 that moves with rotation.

For this reason, the image displayed on the monitor screen 31 by the image data generated by the viewpoint conversion using the rotating virtual camera 8 is rotated in accordance with the rotation operation of the steering system handle 10 in turn, and the entire image behind the vehicle is rotated. Thus, when the vehicle is turning backward, the predicted course area can be captured and displayed on the monitor screen 31 of the monitor 3 at the center of the screen. Since other operations are the same as those in the first to third embodiments, the description thereof will be omitted. Next, the effects will be described.
In the backward driving support device according to the fourth embodiment, the effects listed below can be obtained in addition to the effects of the first to third embodiments.

  (7) The monitor image data generating means rotates the virtual camera 8 in the turning direction of the vehicle according to the magnitude of the steering angle θ of the steering wheel from the neutral position when the vehicle is turning backward, and the virtual camera 8 and the virtual stereoscopic object that rotate. In order to generate monitor image data on the monitor 3 by seamlessly joining a bird's-eye view image of a near view and an oblique view image of a distant view by viewpoint conversion using the CCD 9 and the virtual stereoscopic screen 7, the monitor screen 31 is used during backward rotation. It is possible to perform display change control of the monitor screen 31 with an optimum angle of view according to the driving scene while capturing and displaying the video of the expected course area at the center of the screen.

  (8) The monitor image data generating means rotates the virtual camera 8 in the turning direction of the vehicle according to the magnitude of the steering angle θ of the steering wheel from the neutral position when turning backward, and the virtual image of the rotating virtual camera 8 is rotated. In order to maintain the relative positional relationship between the stereoscopic CCD 9 and the virtual stereoscopic screen 7, when the virtual stereoscopic screen 7 is fixed, the end image of the monitor screen 31 is distorted in a region where the steering angle θ is large. Even in a region where the steering wheel steering angle θ is large, video distortion can be eliminated and an image obtained by seamlessly joining a close-up bird's-eye view image and a distant view oblique viewpoint image can be displayed on the monitor screen 31.

  As mentioned above, although the drive assistance apparatus of this invention has been demonstrated based on Example 1-Example 4, it is not restricted to these Examples about a concrete structure, Each claim of a claim is a claim. Design changes and additions are allowed without departing from the gist of the invention.

In the first to third embodiments, a control example of changing the angle of view of the virtual stereoscopic screen / virtual stereoscopic CCD (two-plane stereoscopic model) by the steering angle of the steering wheel is shown. An example of combined control of "virtual camera rotation control of a two-plane solid model" is shown. However, for “view angle change control of two-dimensional solid model” or “combined control of view angle change control and virtual camera rotation control”,
・ "Two-plane connection position change control for a two-plane solid model (control that enlarges the bird's-eye view image area displayed on the monitor as the steering wheel steering angle increases)"
・ "Two plane crossing angle change control of the two-dimensional solid model (control to convert the oblique viewpoint video area displayed on the monitor into a bird's eye view as the steering wheel steering angle increases)"
・ "Model shape change control of the two-dimensional solid model (control that makes the image displayed on the monitor wider in the left-right direction as the steering wheel steering angle increases)"
Of these, it is possible to add and use at least one control in combination.

  In the first to fourth embodiments, an example in which a virtual stereoscopic screen / virtual stereoscopic CCD (two-plane stereoscopic model) is used as the viewpoint conversion model has been described. However, the viewpoint conversion model is not limited to these models, and is a three-dimensional model composed of a plurality of planes of three or more planes, or a curved surface such as a cylindrical surface or an elliptical cylindrical surface at a joint between the planes and the planes. The present invention can also be applied to a configured three-dimensional model or a three-dimensional model composed of a free-form surface combining innumerable planes.

  In Examples 1 to 4, an example of performing driving support during reverse using the steering wheel steering angle information and reverse information has been shown. For example, when driving support is specified during garage entry, garage removal, or parking, The vehicle angle sensor may be used to execute the angle-of-view change control of the present invention in a low speed range or a stop range where the vehicle speed is equal to or lower than the set vehicle speed.

  In the first to fourth embodiments, the vehicle illustration 12 is displayed at the lower center of the monitor screen 31 and the size of the vehicle illustration 12 is changed according to the change in the angle of view. It may be displayed on the upper part of the central part, or without displaying the vehicle illustration 12, the overhead image of the own vehicle may be displayed as it is.

  In the first to fourth embodiments, the rear camera is used as a real camera, and an example of a reverse driving support device that supports driving when moving backward in a straight line or when turning backward is shown. The front camera or door mirror camera to be acquired can also be applied as a forward driving support device that supports driving at the time of linear advance or turning forward.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall system diagram illustrating a backward driving support device (an example of a driving support device) according to a first embodiment. FIG. 6 is a schematic diagram illustrating an example of a viewpoint conversion method using a virtual camera and a virtual stereoscopic screen / virtual stereoscopic CCD (two-dimensional stereoscopic model) by angle-of-view control based on a steering angle of the steering wheel in the backward driving support device according to the first embodiment. . FIG. 6 is a monotonic characteristic diagram showing the relationship of the angle of view with respect to the steering angle of the steering wheel in the backward driving support device according to the first embodiment. FIG. 4 is a monitor image when the vehicle is reversely rotated in the reverse driving support device of the first embodiment, where (a) shows a monitor image when the steering angle of the steering wheel is in a neutral range, and (b) shows a monitor when the steering wheel is given a large steering angle. Images are shown. FIG. 10 is a step-type characteristic diagram showing the relationship of the angle of view with respect to the steering angle of the steering wheel in the backward driving support device according to the second embodiment. FIG. 10 is an acceleration characteristic diagram showing a relationship of a field angle with respect to a steering angle of a steering wheel in a backward driving support device according to a third embodiment. FIG. 10 is a schematic diagram illustrating an example of a viewpoint conversion method using a virtual camera, a virtual stereoscopic screen, and a virtual stereoscopic CCD in which a position / posture is controlled according to a steering angle in a reverse driving support device according to a fourth embodiment.

Explanation of symbols

1 Rear camera (actual camera)
2 Image processing controller 21 Inter-decoder conversion unit 22 Coordinate conversion processing unit 23 ROM
24 RAM
25 Encoder conversion unit 3 Monitor 4 Handle steering angle sensor (handle steering angle detection means)
5 Shift lever position sensor 6 Virtual camera position adjustment dial 7 Virtual stereoscopic screen (virtual stereoscopic projection plane)
71 Foreground screen (projection surface for foreground)
72 Distant screen (distant projection plane)
8 Virtual camera 9 Virtual stereoscopic CCD (virtual stereoscopic imaging surface)
91 CCD for near view (imaging surface for foreground)
92 CCD for distant view (distant image pickup surface)
10 Steering system handle 11 Shift lever

Claims (8)

Monitor the interior of the vehicle from the camera image data of the real camera by viewpoint conversion using a virtual camera set at a different position from the real camera installed in the vehicle and a virtual projection plane set on the subject side projected by the real camera. In the driving support device provided with the monitor image data generating means for generating the monitor image data to be projected on
Virtual stereoscopic projection plane setting means for setting a virtual stereoscopic projection plane having a near-view projection plane that obtains a bird's-eye view image and a distant view projection plane that obtains an oblique viewpoint image as the virtual projection plane;
Virtual stereoscopic imaging plane setting means for setting a virtual stereoscopic imaging plane having a near-view imaging plane for obtaining a bird's-eye view image and a distant view imaging plane for obtaining an oblique viewpoint video as the virtual imaging plane of the virtual camera;
A steering wheel angle detecting means for detecting a steering wheel angle;
The monitor image data generating means enlarges the angle of view, which is a range in which the lens of the virtual camera can be projected on the virtual stereoscopic imaging surface, as the steering angle of the steering wheel from the neutral position increases when the vehicle turns. A driving support device that generates monitor image data to be displayed on the monitor by viewpoint conversion using a virtual camera, a virtual stereoscopic projection plane, and a virtual stereoscopic imaging plane by angle control. In the driving assistance device according to claim 1,
The real camera is a rear camera set at a vehicle rear position,
The monitor image data generation means is a monitor image when the steering wheel is turned to the right with the steering wheel turned to the right or when the steering wheel is turned to the left with the steering wheel at the neutral position. Is set as a bird's-eye video area and an oblique viewpoint video area based on a predetermined area distribution, and as the steering angle of the steering wheel increases from the neutral position, the field angle of the bird's-eye video area increases in the vehicle width direction and the oblique viewpoint video. A driving support device that generates monitor image data that expands the field of view of a region in a vehicle width direction and a vehicle upper direction. In the driving assistance device according to claim 1 or 2,
The monitor image data generating means generates monitor image data in which the angle of view is fixed regardless of a minute change in the steering angle of the steering wheel when the steering angle from the neutral position is in the range of the dead angle or less during turning. A driving support device characterized by that. In the driving assistance device according to claim 3,
The monitor image data generating means has a monotonous characteristic that expands in proportion to the steering wheel steering angle when the steering wheel steering angle from the neutral position exceeds the dead angle during turning. A driving assistance device characterized by changing a corner. In the driving assistance device according to claim 3,
When the steering wheel steering angle exceeds the dead angle and the steering wheel steering angle exceeds the dead angle when turning, the monitor image data generating means is displayed with a step-type characteristic. A driving assistance device characterized by changing a corner. In the driving assistance device according to claim 3,
The monitor image data generating means calculates a two-plane intersection angle between the virtual stereoscopic projection plane and the virtual stereoscopic imaging plane when the steering wheel steering angle from the neutral position is an angle area exceeding a dead angle during turning. The angle of view is changed by an acceleration type characteristic that gradually increases in the small steering angle region where the steering angle of the steering wheel exceeds the dead angle, and becomes steeper as the steering angle of the steering wheel becomes farther from the small steering angle region. Driving assistance device. In the driving assistance device according to any one of claims 1 to 6,
The monitor image data generating means rotates the virtual camera in the turning direction of the vehicle according to the steering angle from the neutral position when turning the vehicle, and rotates the virtual camera, the virtual stereoscopic imaging surface, and the virtual stereoscopic projection. A driving support device that generates monitor image data that projects a video obtained by seamlessly joining a bird's-eye view image of a near view and an oblique view image of a distant view by a viewpoint conversion using a plane. In the driving assistance device according to claim 7,
The monitor image data generating means rotates the virtual camera in the turning direction of the vehicle according to the steering angle from the neutral position when turning the vehicle, and the virtual stereoscopic imaging surface and virtual stereoscopic of the rotating virtual camera. A driving support apparatus characterized by maintaining a relative positional relationship of projection surfaces.

Images