JP2005313772A - Vehicular head-up display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicular head-up display device for displaying information images in a dynamically superimposing manner on ever-changing objects carefully watched by a driver. <P>SOLUTION: The vehicular head-up display device comprises a display source for projecting a display image indicating predetermined information and a combiner for displaying the display image projected from the display source in the front visual field of a driver's seat. The display device is equipped with a specifying means for specifying a front object of the vehicle carefully watched by the driver, a distance measuring means for measuring a distance to the object watched by the driver specified by the specifying means, and an adjusting means for adjusting the display source and/or the combiner so as to superimpose/image-focus the display image on the object watched by the driver according to the distance measured by the distance measuring means. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention generally relates to a head-up display device for a vehicle, and in particular, adjusts a focal distance to an object being watched by a driver in front of the vehicle and displays an information image such as a sub-operation system interface. The present invention relates to a vehicle head-up display device that superimposes and displays on a vehicle.

  A head-up display (HUD) device is well known as a device for displaying an image indicating vehicle information such as vehicle speed or navigation information in a vehicle.

  In the image display by the HUD device, a display image called a holographic virtual image is projected on a combiner provided on the windshield. The combiner is a semi-reflective glass (half mirror), which appears to be transparent to the driver, but reflects light from the hologram virtual image display source.

  Since the image displayed by the HUD device is an image projected onto the windshield and reflected by the combiner, it seems that the driver is deepened by the light path lengthened by the mirror, that is, in front of the vehicle. It looks as if it exists.

  Therefore, if the driver uses the HUD device, he / she can confirm the displayed information while looking at the vehicle front view without turning the line of sight to the instrument panel and adjusting the focal length of the eyes while driving. can do.

  In such a HUD device, it becomes a problem how much the hologram virtual image to be displayed is set to be seen from the front of the vehicle when viewed from the driver. That is, if the information image displayed by the HUD device is viewed from the driver's viewpoint, the vehicle front object (preceding vehicle, building, etc.) or the background (infinite) that the driver is gazing at is significantly different from the perspective. Eventually, the driver will have to frequently change the focal length of the eye as usual.

Therefore, conventionally, the imaging position of the information image displayed by the HUD device is moved back and forth and / or left and right based on the detected driver's line-of-sight direction and the distance between the preceding vehicle and the driver using the HUD device. Various apparatuses and methods that aim to reduce the burden of focusing the eyes and moving the line of sight when viewing displayed information have been proposed (see, for example, Patent Documents 1 to 7).
Japanese Utility Model Publication No. 61-134437 JP-A-6-247184 JP 7-55941 A Japanese Patent Laid-Open No. 7-257228 JP-A-8-76050 Japanese Patent Application Laid-Open No. 10-1000073 JP 2000-344028 A

  However, in any of the above-described conventional apparatuses and methods, the vehicle front object (including the background at infinity) that the driver is gazing at is not dynamically specified. There may still be situations where the driver needs to move his gaze or refocus to see.

  SUMMARY OF THE INVENTION The present invention has been made in order to solve such problems, and it is a primary object of the present invention to provide a vehicle head-up display device that dynamically displays a driver's gaze object that can be changed and displays an information image. And

One aspect of the present invention for achieving the above object includes a display source for projecting a display image representing predetermined information, and a combiner for projecting the display image projected from the display source into the field of view in front of the driver's seat. A head-up display device for a vehicle,
A specifying means for specifying an object ahead of the host vehicle being watched by the driver;
Distance measuring means for measuring the distance to the driver's gaze object specified by the specifying means;
Adjusting means for adjusting the display source and / or the combiner so that the display image is superimposed on the driver gaze object based on the distance measured by the distance measuring means. It is a head-up display device.

  In this aspect, the predetermined information is, for example, a sub-operation system interface, vehicle information such as a vehicle speed, and the like.

  Moreover, in this one aspect | mode, the said vehicle front object and the said driver | operator gaze object are backgrounds, such as a preceding vehicle, a building, etc., and a mountain and a cloud located at infinity, for example.

  In this one aspect, the specifying unit includes, for example, a detecting unit that detects a driver's line-of-sight direction, an imaging unit that captures a scenery image in front of the host vehicle reflected on the windshield, and the own vehicle obtained by the imaging unit. Conversion means for converting a front view image into a driver viewpoint image; and an intersection between the driver viewpoint image obtained by the conversion means and the driver's line-of-sight direction obtained by the detection means; and the driver viewpoint Identification means for identifying an object present at the intersection on the image.

  According to this aspect, since the object ahead of the host vehicle that the driver is gazing at can be dynamically specified, even if the gazing object changes every moment, it is always superimposed on the object that the driver is gazing at. A display image representing predetermined information can be formed. Thereby, the driver can recognize the displayed information without moving his / her line of sight and without adjusting the focal distance of the eyes.

  In this aspect, the adjusting means may prevent the image display position and / or the display position and / or distance of the information display image from recognizing to the driver due to excessive frequent movement. You may have a speed adjustment means which suppresses the rapid fluctuation | variation of distance.

  ADVANTAGE OF THE INVENTION According to this invention, the head-up display apparatus for vehicles which displays an information image dynamically superimposed on the driver | operator's gaze object which can change every moment can be provided.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the accompanying drawings. Note that the basic concept of the HUD device, the main hardware configuration, the operating principle, the basic control method, and the like are known to those skilled in the art, and thus detailed description thereof is omitted.

  A vehicle head-up display (HUD) device according to an embodiment of the present invention, and its operation and function will be described with reference to FIGS. In this embodiment, the information displayed by the HUD device is, for example, a sub-operation system interface such as an audio, an air conditioner, or a navigation system.

  FIG. 1 shows a schematic configuration of a HUD device 100 according to the present embodiment. As shown in the figure, the HUD device 100 mainly includes an ECU 101 that is a controller of the device, an image processing unit 102 that detects and acquires a driver's line-of-sight direction and a front image of the host vehicle reflected on the windshield, and a driver's gaze. A distance detection unit 103 for measuring the distance from the object being operated, a user input unit 104 for inputting an operation such as user selection / determination for the displayed sub-operation system interface, and a menu that is a sub-operation system interface. And a menu display unit 105 for displaying a displayed display image as a holographic virtual image.

  The image processing unit 102 includes, for example, a first camera (for example, a CCD camera) for detecting the driver's line of sight, a second camera for capturing a landscape image in front of the host vehicle reflected on the windshield, And an arithmetic unit for performing arithmetic processing. The first camera and the second camera may be shared by the same camera.

  The distance detection unit 103 is, for example, for measuring the distance between a radar device (for example, a millimeter wave radar) for measuring the distance from an object existing in front of the host vehicle and the driver's gaze object by a stereo method. It consists of a third camera (for example, a CCD camera), a device for detecting the current position of the host vehicle (for example, GPS, gyroscope, etc.), map information such as a map database, and a calculation unit for performing calculation processing. .

  The user input unit 104 is, for example, a switch or dial (such as haptic use) attached to the steering handle in consideration of operability. You may implement | achieve using a speech recognition technique.

  The menu display unit 105 has the same configuration as that of a conventional HUD device, and includes, for example, a light source, an optical system, a display element, a controller that performs display control, and a combiner (half mirror) provided on a windshield.

  Next, the operation flow of the HUD device 100 will be described with reference to FIGS. FIG. 2 is a flowchart showing the flow of the main operation of the HUD device 100.

  First, when the vehicle is in an unstable or dangerous state, it is preferable not to display the low priority sub-operation system interface to ensure safety, so confirm that the sub-operation system switch flag is on. (S201 in FIG. 2). Here, the flow of a subroutine relating to the processing of S201 is shown in the flowchart of FIG.

  First, it is determined whether or not the secondary operation system switch flag has already been turned on, that is, whether or not the secondary operation system is being operated (S301 in FIG. 3). If the sub-operation system switch flag has already been turned on (“YES” in S301), it is next determined whether or not an obstacle exists around the host vehicle (S302).

  If there is no obstacle (“NO” in S302), it is next determined whether or not the vehicle is in a stable state (S303). If it is determined that the vehicle is in a stable state (“YES” in S303), it is then determined whether or not the driver's line of sight has moved suddenly or rapidly (S304). In the determination in S304, for example, a moving speed ΔG on the windshield surface of an intersection G (detailed later) between the driver's line-of-sight vector and the windshield surface is calculated. If | ΔG | It is determined that there has been movement.

  If there is no sudden movement of the line of sight (“NO” in S304), it is next determined whether or not there is no input for a certain period of time for the activated sub-operation system (S305). If there is no input for a certain period of time for the activated sub-operation system (“NO” in S305), the sub-operation system switch flag is kept ON (S306).

  On the other hand, when there is an obstacle around the host vehicle (“YES” in S302), or when it is determined that the vehicle is not in a stable state, such as when the ABS is operating (“NO” in S303), or If it is determined that the driver's line of sight has moved suddenly ("YES" in S304), or if there is no input for a certain period of time for the activated sub-operation system ("YES" in S305), for safety It is determined that the operation of the secondary operation system should not be permitted, and the secondary operation system switch flag is turned OFF (S307).

  In the process of S307, the sub-operation system interface being projected may be turned off (projection stopped), or may be put in any one of the four corners of the windshield to enter the standby mode. Even in the former case, it is preferable that the content being operated is saved for the next operation. In any case, it is preferable that the driver is provided with voice guidance to the effect that the sub-operation system cannot be operated.

  Returning to S301, if the sub-operation system switch flag is in the OFF state, that is, if the sub-operation system is not being operated ("NO" in S301), it is next determined whether there is an obstacle around the host vehicle. (S308).

  If there is no obstacle (“NO” in S308), it is next determined whether or not the vehicle is in a stable state (S309). If it is determined that the state is stable (“YES” in S309), it is determined that there is no problem even if the operation of the sub-operation system is performed, and the sub-operation system switch flag is turned ON (S306).

  On the other hand, if there is an obstacle around the vehicle (“YES” in S308), or if the vehicle is determined to be in an unstable state such as the ABS is operating (“NO” in S309), the safety Therefore, it is determined that the operation of the secondary operation system should not be permitted, and the operation of the secondary operation system is not permitted (S310).

  In the process of S310, it is preferable that voice guidance indicating that the operation of the sub-operation system is not permitted is provided to the driver.

  Returning to the description of FIG. If it is confirmed by the execution of the subroutine described with reference to FIG. 3 that the sub-operation system switch flag is on, then the line-of-sight direction of the vehicle driver is extracted by the first camera of the image processing unit 102. (S202). The first camera is, for example, a CCD camera installed in the passenger compartment toward the driver.

  More specifically, the driver eye-gaze direction extraction processing is performed by, for example, extracting three points on the driver's face by image processing using a camera in a vehicle coordinate system, and a vector up to the center of the three points and three points. This is performed by obtaining a line-of-sight direction vector from three vectors: a vector from the center to the eyeball center (by model prediction) and a vector to the detected pupil position.

  In this case, as a method of extracting three points on the driver's face, for example, by using the fact that there are many edge portions, the position of the eyes and mouth is detected by edge conversion and this is followed. Extracted.

  Further, the driver line-of-sight direction may be converted into black and white by binarizing a part of both eyes, and a vector up to an intermediate position between both pupil positions may be extracted as a line-of-sight direction vector by detecting the pupil.

  When the driver's line-of-sight direction is extracted in this way, the image reflected on the windshield is then estimated and created by the image processing unit 102 (S203). In this process, first, a landscape image in front of the host vehicle reflected on the windshield is captured using the second camera of the image processing unit 102. The second camera is, for example, a wide-angle CCD camera installed in the vehicle interior facing the windshield. The second camera preferably employs a structure that can be rotated horizontally in accordance with the movement of the driver's head.

  Since the image captured by the second camera is naturally a scene reflected on the windshield viewed from the second camera viewpoint, the arithmetic unit of the image processing unit 102 next converts the captured image to the driver viewpoint. Correction is made using the coordinates, and the image is converted into a landscape viewed from the driver's viewpoint.

  When the scenery reflected on the windshield as viewed from the driver viewpoint is estimated and created by such image capturing and image conversion processing, the driver sight line direction extracted in S202 and the scenery reflected on the windshield estimated and created in S203 are then displayed. Is calculated (S204). Since the design parameters of the windshield are known, the windshield position can be approximated to a plane equation from the windshield mounting coordinates in the vehicle coordinate system.

  When the intersection point G is obtained in this way, the object existing at the intersection point G in the scene reflected on the windshield estimated and created in S203, that is, the object that the driver is gazing at that time is detected. S205). This is shown in FIG.

  Next, the distance from the driver position to the detected driver gaze object is measured (S205). This distance measurement is performed according to a subroutine shown in FIG. Hereinafter, the distance measurement process shown in FIG. 5 will be described.

First, using image processing pattern matching or the like, it is determined whether or not the driver gaze object detected in S205 of FIG. 2 is a preceding vehicle (S501). When the vehicle is not a preceding vehicle (“NO” in S501), the distance d _s from the driver position to the driver gaze object is measured using the stereo method (S502). This is shown in FIG.

When measuring the distance d _s by the stereo method, at least the figure is obtained using the second camera described above and the third camera of the distance detection unit 103 installed on a horizontal plane symmetrical to the position of the second camera. The luminances of the images captured by these two cameras at the height position near the intersection G calculated in S204 of 2 are compared, and the shift amount Z of the imaging surface is obtained. If the focal length of the second and third cameras is f and the horizontal distance between the two cameras is B, the depth (distance) D from the focal length of both cameras to the target (driver's gaze object) is D = B Since it can be expressed as xf / Z, the distance D + f from the horizontal plane where the camera is installed to the target can be obtained. If this is converted into driver viewpoint coordinates, the distance d _s from the driver to the driver gaze object is calculated.

When the distance d _s is measured by the stereo method in this way, next, the map information held in advance by the distance detection unit 103 is illuminated with the current vehicle position and the driver's line-of-sight vector, and the landmarks (buildings and mountains) in the map information are illuminated. find the intersection with the like), and calculates the distance _{d m} to the intersection point (S503). This state is also shown in FIG.

Thus, if the driver gaze object is not a preceding vehicle, the measuring the distance d _s by a stereo method, by calculating the distance d _m from map information, and compares the two (S504). More specifically, the absolute value of the distance difference between the two is compared with a predetermined threshold value d _th .

If the absolute value of the difference obtained by subtracting the d _m based on the map information by the stereo method from the distance d _s is less than or equal to the predetermined threshold value d _th ( "YES" in S504), the vehicle current position on the map information and the driver sight vector it is considered that bets landmarks in the map information obtained in light of the matches the driver gaze object, adopting the distance d _m based on the map information is considered more high accuracy as the distance the driver gaze object (S505).

On the other hand, ( "NO" S504 in) from the distance d _s absolute value of the difference obtained by subtracting the d _m based on the map information when the predetermined larger than the threshold value d _th by stereo method, the driver gaze object in the map information in the preceding vehicle It determines that not a landmark included in, employing the distance d _s by stereo method as distance to the driver gaze object (S506).

  Return to S501. When the driver gaze object detected in S205 in FIG. 2 is a preceding vehicle (“YES” in S501), the distance to the preceding vehicle is measured using the radar device of the distance detection unit 103 (S507). This state is also shown in FIG.

  Returning to FIG. When the distance from the driver's gaze object is measured in this way, the menu display unit 105 then displays a display image representing the sub-operation system interface as a hologram virtual image (S207). This is shown in FIG. At this time, the sub-operation system interface to be displayed is displayed in the driver line-of-sight direction extracted in S202, and the imaging position is adjusted so that it can be seen from the distance measured in S206 when viewed from the driver.

  In other words, the sub-operation system interface display image is pasted on the driver's gaze object as the driver's gaze object has the same vertical and horizontal positional relationship and sense of distance as seen from the driver. It looks like a sticker or sticker or a switch provided on the driver's gaze object.

  The virtual image projection processing in S207 will be described in detail with reference to FIGS. As shown in FIG. 6, a liquid crystal panel is inserted into the windshield 601 in a mesh shape, and a desired position can be made into a half mirror by switching.

  The menu display unit 105 includes a backlight 603, an information display element 604 such as a liquid crystal, and a lens 605. The information display element 604 has a structure in which the display size can be changed according to the size of the driver gaze object A.

  In addition, the backlight 603 and the information display element 604 are integrated so that the distance from the lens 605 is variable. As a result, the hologram virtual image can be formed at an arbitrary distance from the relationship with the focal length of the lens, so that it can be displayed at the same distance as the driver's gaze object A. For example, the virtual hologram image M can be formed at infinity by presenting a real image at the focal length position of the lens.

  Further, the menu display unit 105 itself has a structure that can be moved (rotated and translated) three-dimensionally. Thereby, it is comprised so that a virtual image can be projected even if any location of the windshield 601 is made into a half mirror.

  The actual flow of virtual image projection processing will be described with reference to FIG. First, the intersection point G calculated in S204 of FIG. 2 on the windshield 601 and its periphery are formed into a half mirror to form a combiner 602 (S701).

  Next, the position of the menu display unit 105 is adjusted so that it can be projected near the intersection G (S702).

  Then, the display size, backlight 603, and information display by the information display element 604 are formed so that the sub-operation system interface display image M is formed on the driver gaze object A detected in S205 of FIG. After the distance between the element 604 and the lens 605 is adjusted, the determined display content (sub-operation system interface) is projected onto the combiner 602 (S703).

  In the projection process of S703, since the scene reflected on the windshield 601 has already been grasped in the process of S203 of FIG. 2, the color data of the image is analyzed, and the visual effect is highest for the driver. It is preferable to project the display image (sub-operation system interface) M in color.

  When the display image M representing the sub-operation system interface is projected in this way, the driver then operates the sub-operation system switch using the user input unit 104 to select / determine ON / OFF and various settings. (S208). This state is shown in FIG. Here, for example, as illustrated in FIG. 6, the user input unit 104 is a switch 606 provided on the steering handle, a dial 607 (haptic available), or the like.

  When the selection by the driver is completed, it is then determined whether or not the selection for ending the operation of the secondary operation system and stopping the display of the secondary operation system interface has been selected by the driver (S209). When the operation of the sub operation system is continued (“NO” in S209), the process returns to S201, and the sub operation system interface is displayed again on the object being watched by the driver at that time. That is, by repeatedly executing the flow of FIG. 2, the sub-operation system interface can be displayed on the driver's gaze object that can change from moment to moment.

  On the other hand, when the operation of the sub operation system is ended (“YES” in S209), the sub operation system switch flag is turned OFF (S210).

  Thus, according to the present embodiment, since the information image representing the sub-operation system interface can be displayed dynamically superimposed on the object that the driver is gazing at the moment, the driver can It is possible to easily view the information image of the sub-operation system interface without burdening without changing the focus and refocusing the focus.

  As a modification of the above embodiment, it is also possible to display a three-dimensional display image superimposed on the driver's gaze object using digital hologram technology. This modification is shown in FIG.

  The digital hologram menu display unit 105 ′ shown in FIG. 8 has a three-dimensional display image M ′ superimposed on the laser 801 that emits reproduction light, the acoustooptic device 802, and the driver's gaze object A. The ECU 803, the galvanometer mirror 804, the polygon mirror 805, the exit lens 806, which calculates the interference pattern between the object light and the radar light (reference light) when irradiated with light in real time and presents the interference pattern to the acoustooptic device 802 Consists of.

  By adopting such a configuration, the present invention can be extended to a digital hologram that realizes a three-dimensional display image.

  In any of the above-described embodiments, when the driver frequently changes the gaze object with a relatively short period, the visibility may be deteriorated if the display position of the display image is frequently changed accordingly. There is. Therefore, a dead zone is provided for the change (movement) speed of the driver's line of sight (or the intersection point G) using, for example, a low-pass filter, and the display position change speed so that the display position is not changed too frequently. It is preferable that a setting for suppressing the above is performed.

  In any of the above embodiments, the size of the display image is changed according to the size of the driver's gaze object. However, if the driver's gaze object is small or looks small from the driver's viewpoint, it is adjusted accordingly. If the display image is too small, the visibility may be reduced. Therefore, when the size of the display image changed in accordance with the driver's gaze object is smaller than the predetermined size, it is not forcibly displayed on the driver's gaze object and is slightly deviated from the driver's gaze. Further, a display image having a size that can be sufficiently visually recognized (that is, around the driver's gaze object) may be displayed.

  Further, in any of the above-described embodiments, the sub-operation system interface has been described as an example of the information screen to be displayed. Of course, it is also possible to display as.

  The present invention can be used for a vehicle head-up display (HUD) device. The present invention can also be applied to a vehicle head mounted display (HMD) device using the same principle. The appearance, weight, size, running performance, etc. of the vehicle to be mounted are not limited.

1 is a schematic configuration diagram of a vehicle head-up display device according to an embodiment of the present invention. It is a flowchart which shows the flow of the main operation | movement of the head-up display apparatus for vehicles which concerns on one Example of this invention. It is a flowchart which shows the flow of the sub operation system switch flag ON determination subroutine by the vehicle head-up display apparatus which concerns on one Example of this invention. It is a schematic diagram which shows the outline | summary of the display and operation of a suboperation system interface by the vehicle head-up display apparatus which concerns on one Example of this invention. It is a flowchart which shows the flow of the distance measurement subroutine to a driver | operator gaze object by the vehicle head-up display apparatus which concerns on one Example of this invention. It is the schematic which shows the structure and function of the menu display part of the head-up display apparatus for vehicles which concern on one Example of this invention. It is a flowchart which shows the flow of the display image projection subroutine by the head-up display apparatus for vehicles which concerns on one Example of this invention. It is the schematic which shows the structure and function of the menu display part of the head-up display apparatus for vehicles which concerns on another one Example of this invention.

Explanation of symbols

100 head-up display (HUD) device 101 ECU
DESCRIPTION OF SYMBOLS 102 Image processing part 103 Distance detection part 104 User input part 105,105 'Menu display part 601 Windshield 602 Combiner 603 Backlight 604 Information display element 605 Lens 606 Switch 607 Dial 801 Laser 802 Acoustooptic element 803 ECU
804 Galvano mirror 805 Polygon mirror 806 Outgoing lens A, A 'Driver gaze object M, M' Display image

Claims (3)

A vehicle head-up display device comprising: a display source that projects a display image representing predetermined information; and a combiner that projects the display image projected from the display source into a field of view in front of the driver seat;
A specifying means for specifying an object ahead of the host vehicle being watched by the driver;
Distance measuring means for measuring a distance from the driver's gaze object specified by the specifying means;
Adjusting means for adjusting the display source and / or the combiner so that the display image is superimposed on the driver gaze object based on the distance measured by the distance measuring means. A vehicle head-up display device. The vehicle head-up display device according to claim 1,
The specifying means is:
Detection means for detecting the driver's line-of-sight direction;
Imaging means for capturing a landscape image in front of the vehicle reflected on the windshield;
Conversion means for coordinate-converting the vehicle front view image obtained by the imaging means into an image of the driver's viewpoint;
And an identification means for determining an intersection between the driver viewpoint image obtained by the conversion means and the driver's line-of-sight direction obtained by the detection means, and identifying an object existing at the intersection on the driver viewpoint image. A head-up display device for a vehicle. The vehicle head-up display device according to claim 1 or 2,
The vehicle head-up display device, characterized in that the adjusting means includes speed adjusting means for suppressing rapid fluctuations in the imaging position and / or distance.

Images