JP2018087852A - Virtual image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a virtual image display device for enabling a user to accurately view and recognize an object as a warning target.SOLUTION: A virtual image display device is configured to, when there exists a warning object as a warning target to a crewmember of a vehicle, display a virtual image at a position between the crewmember and the warning object where the virtual image is viewed and recognized so as to be superimposed with at least a part of the warning object by the crewmember, and to, when a visual line of the crewmember is positioned around the warning object afterwards, move the position of the virtual image in a direction of the warning object along a direction connecting the crewmember to the warning object.SELECTED DRAWING: Figure 11

Description

  The present invention relates to a virtual image display device that issues a warning to an object that is a warning target for a user.

  Conventionally, various means have been used as information providing means for providing driving information such as route guidance and obstacle warnings to passengers of moving bodies such as vehicles. For example, display on a liquid crystal display installed on a moving body, sound output from a speaker, and the like. In recent years, as one of such information providing means, a head-up display device (hereinafter referred to as HUD) is used in a space different from a position where an image is actually displayed using an illusion of human eyes. There is a virtual image display device for visually recognizing an image.

  Here, in order to provide more effective information by the virtual image display device, it is important to appropriately set the position for displaying the virtual image (more specifically, the distance from the user viewing the virtual image to the virtual image). It is. For example, Japanese Patent Application Laid-Open No. 2015-74391 discloses displaying a virtual image at a position where an obstacle actually exists when displaying a virtual image of a warning for the obstacle. In addition, when it takes time to move the screen, once the virtual image of the warning is displayed at a position close to the user, and the virtual image is displayed again at the position where the obstacle actually exists after the movement of the screen is finished. It is disclosed.

Japanese Patent Laying-Open No. 2015-74391 (pages 14-15, FIG. 19)

  Here, the virtual image displayed by the virtual image display device is superimposed on the surrounding environment and visually recognized by the user. Therefore, especially for a virtual image located at a position away from the user, the virtual image may melt into the surrounding environment, and the user may not be aware of the virtual image or an obstacle warned by the virtual image.

  In the virtual image display device described in Patent Document 1, the distance from the user to the virtual image is changed depending on the position of the obstacle to be warned, but the position of the user's line of sight is taken into consideration. There wasn't. Therefore, for example, even when an obstacle at a position away from the user's line of sight is warned, a virtual image is displayed from the beginning at the position of the obstacle away from the user if the screen moves in time. As a result, the virtual image guidance may end without the user noticing the virtual image or the obstacle that warns by the virtual image.

  The present invention has been made to solve the above-described conventional problems, and even when the user's line of sight is away from the object to be warned, the user's line of sight is warned using a virtual image. It is an object of the present invention to provide a virtual image display device that can surely visually recognize a target object as a warning target by guiding the target target object.

  In order to achieve the above object, a virtual image display device according to the present invention is a virtual image display device that allows a user to visually recognize a virtual image of the video by allowing the user to visually recognize a video displayed on a video display surface. A virtual image that displays a virtual image at a position that is between the target object from a user and is superimposed on at least a part of the target object and is visually recognized by the user, and a position acquisition unit that acquires the position of the target object to be warned After the virtual image is displayed by the display means, the visual line detection means for detecting the visual line of the user, and the virtual image display means, and when the visual line of the user is positioned around the target object, the user and the target object Virtual image position changing means for moving the position of the virtual image in the direction of the object along the direction connecting the two.

  According to the virtual image display device according to the present invention having the above-described configuration, the position of the virtual image is moved according to the user's line of sight even when the user's line of sight is at a position away from the target object to be warned. Thus, it becomes possible to guide the user's line of sight to a target object to be warned by a virtual image. Thereby, it becomes possible to make the user visually recognize the target object to be warned.

It is the figure which showed the installation aspect to the vehicle of HUD which concerns on this embodiment. It is the figure which showed the internal structure of HUD which concerns on this embodiment. It is the figure which showed the optical path formed in HUD. It is the figure which showed the display area of the image | video of a liquid crystal display. It is a figure explaining the position where a virtual image is generated. It is the block diagram which showed the structure of HUD which concerns on this embodiment. It is a flowchart of the information acquisition processing program which concerns on this embodiment. It is the figure which showed the display line. It is a flowchart of the virtual image display processing program which concerns on this embodiment. It is the figure which showed an example of the virtual image which can be visually recognized from the passenger | crew of a vehicle in a 1st step. It is the figure which showed an example of the virtual image which can be visually recognized from the passenger | crew of a vehicle in a 2nd step.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a virtual image display device according to the invention will be described in detail with reference to the drawings with regard to an embodiment in which the virtual image display device is embodied in a head-up display device mounted on a vehicle.

  First, the configuration of a head-up display device (hereinafter referred to as HUD) 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an installation mode of the HUD 1 according to the present embodiment on the vehicle 2.

  As shown in FIG. 1, the HUD 1 is installed inside the dashboard 3 of the vehicle 2 and includes a liquid crystal display 5 that displays an image therein. Then, the video displayed on the liquid crystal display 5 is reflected on the front window 8 in front of the driver's seat through the reflective polarizing plate 6 and the concave mirror 7 provided in the HUD 1 as will be described later, and visually recognized by the passenger 9 of the vehicle 2. It is configured to let you. The video displayed on the liquid crystal display 5 includes information related to the vehicle 2 and various information used for assisting the driving of the occupant 9. For example, a warning for an object (another vehicle or a pedestrian) that is a warning target for the user, guidance information set by the navigation device, guidance information based on the guidance route (such as an arrow indicating a right / left turn direction), and the like are displayed on the road surface. There are warnings (attention to rear-end collision, speed limit, etc.), current vehicle speed, information signs, map images, traffic information, news, weather forecast, time, connected smartphone screens, TV programs, and the like.

  Further, in the HUD 1 of the present embodiment, when the occupant 9 visually recognizes the image displayed on the liquid crystal display 5 by reflecting the front window 8, the occupant 9 is not at the position of the front window 8 but at the tip of the front window 8. The image displayed on the liquid crystal display 5 at a distant position is visually recognized as a virtual image 10. The virtual image 10 that can be seen by the occupant 9 is an image displayed on the liquid crystal display 5, but the vertical direction and the horizontal direction may be reversed through the reflective polarizing plate 6 and the concave mirror 7. It is necessary to display the video on the liquid crystal display 5 in consideration. In addition, the size is changed through the concave mirror 7.

  Here, regarding the position where the virtual image 10 is generated, more specifically, the distance from the occupant 9 to the virtual image 10 (hereinafter referred to as virtual image generation distance) L, the curvature of the concave mirror 7 included in the HUD 1, the liquid crystal display 5 and the concave mirror 7 It is possible to set as appropriate depending on the relative position of the. For example, if the curvature of the concave mirror 7 is fixed, the virtual image generation distance L is determined by the distance (optical path length) along the optical path from the position where the image is displayed on the liquid crystal display 5 to the concave mirror 7.

  And HUD1 which concerns on this embodiment is the 1st mirror 13 which consists of a total reflection mirror as a reflective mirror which changes the optical path which enters from the liquid crystal display 5 to the direction which goes to the concave mirror 7 as mentioned later, and a 1st mirror is And a second mirror 14 that is a separate half mirror. Of the images displayed on the liquid crystal display 5, a virtual image 10 based on the image reflected by the first mirror 13 (hereinafter referred to as a first virtual image 10A) and a virtual image 10 based on the image reflected by the second mirror 14. (Hereinafter, referred to as second virtual image 10B). The optical path length of the optical path that reflects the first mirror 13 from the liquid crystal display 5 and connects the concave mirror 7 is different from the optical path length of the optical path that reflects the second mirror 14 from the liquid crystal display 5 and connects the concave mirror 7. To do. For example, the optical path length of each optical path is set so that the virtual image generation distance L of the first virtual image 10A is 20 m and the virtual image generation distance L of the second virtual image 10B is 2.5 m.

  In addition, the HUD 1 of the present embodiment has an optical path incident on the concave mirror 7 from the reflective polarizing plate 6 with respect to the liquid crystal display 5, the first mirror 13, the second mirror 14, the reflective polarizing plate 6 and the concave mirror 7 as will be described later. Therefore, the quality of the virtual image 10 can be improved since the direction of the light path and the direction of the optical path reflected by the concave mirror 7 and directed toward the user's eyes are opposite. For example, it is possible to make the user visually recognize a clearer and clearer virtual image 10. Further, by using the reflective polarizing plate 6, the distance (optical path length) along the optical path from the position where the image is displayed on the liquid crystal display 5 to the concave mirror 7 can be secured longer than the volume of the apparatus. doing.

  A front camera 11 is installed above the front bumper of the vehicle, behind the rearview mirror, and the like. The front camera 11 is an imaging device configured by a camera using a solid-state imaging device such as a CCD, for example, and is installed with the optical axis direction facing forward in the traveling direction of the vehicle. Then, by performing image processing on the captured image captured by the front camera 11, the situation of the front environment (that is, the environment in which the virtual image 10 is superimposed) that is visually recognized by the occupant 9 through the front window 8 is detected. Is done. A sensor such as a millimeter wave radar may be used instead of the front camera 11.

  An in-vehicle camera 12 is also installed in the vehicle. The in-vehicle camera 12 uses a solid-state imaging device such as a CCD, for example, and is installed on the upper surface of the dashboard 3 of the vehicle or the ceiling of the vehicle, and is installed with the imaging direction facing the driver's seat. Then, the face of the user who is a passenger sitting in the driver's seat is imaged. Then, as described later, image processing is performed on the captured image captured by the in-vehicle camera 12 to detect the user's eye position (gaze start point) and gaze direction.

  Next, a more specific configuration of the HUD 1 will be described with reference to FIGS. FIG. 2 is a diagram showing an internal configuration of the HUD 1 according to the present embodiment. FIG. 3 is a diagram illustrating an optical path formed by the HUD 1 according to the present embodiment.

  As shown in FIG. 2, the HUD 1 includes a liquid crystal display 5, a reflective polarizing plate 6, a concave mirror 7, a first mirror 13, a second mirror 14, a λ / 4 wavelength plate 15, and a control circuit unit 16. The CAN interface 17 and the heat sink 18 are basically configured.

  Here, the liquid crystal display 5 uses a backlight as a light source, and outputs only the light having a specific polarization direction out of the light from the backlight, thereby displaying an image on the image display surface provided on the front surface. This is a video display device having the function of As the backlight, for example, a CCFL (cold cathode tube) or a white LED is used. As a means for displaying an image, a combination of a liquid crystal projector and a screen may be used in addition to the liquid crystal display.

  In addition, as shown in FIG. 3, the image display surface for displaying an image on the liquid crystal display 5 is arranged at an angle facing downward from the horizontal direction. That is, the light source is disposed at an angle that is located above the liquid crystal layer of the liquid crystal display 5.

  As shown in FIG. 3, the first mirror 13 is disposed between the liquid crystal display 5 and the concave mirror 7 along the optical path connecting the liquid crystal display 5 and the concave mirror 7, and is incident from the liquid crystal display 5 in the first direction. This is light reflecting means for changing the first optical path 19 in a second direction different from the first direction. Similarly, the second mirror 14 is disposed between the liquid crystal display 5 and the concave mirror 7 along the optical path connecting the liquid crystal display 5 and the concave mirror 7, and the second optical path 20 incident in the first direction from the liquid crystal display 5 The light reflecting means changes in a second direction different from the one direction.

  The first mirror 13 and the second mirror 14 are separate mirrors. In particular, the first mirror 13 is a total reflection mirror and the second mirror 14 is a half mirror. The first mirror 13 and the second mirror 14 are arranged in parallel at a predetermined distance, and the second mirror 14 is positioned on the first optical path 19 reflected by the first mirror. As a result, as shown in FIG. 3, the first optical path 19 reflected by the first mirror 13 passes through the second mirror 14 and reaches the concave mirror 7. The second mirror 14 is, for example, a half mirror having the same transmittance and reflectance. However, the half mirror does not necessarily have the same transmittance and reflectance, and transmits a part of the incident light. Any member (beam splitter) that reflects the part may be used. For example, the second mirror 14 may be a half mirror having a transmittance of 40%, a reflectance of 60%, a transmittance of 60%, and a reflectance of 40%. On the other hand, the first mirror 13 does not necessarily have to be a total reflection mirror, and may be a mirror having a higher reflectance than the transmittance. However, a clearer virtual image can be generated by using a mirror with high reflectivity.

  Further, as shown in FIGS. 2 and 3, the first mirror 13 and the second mirror 14 are not directly connected to the liquid crystal display 5 but are directly fixed to the outer wall of the HUD 1. This makes it difficult for the heat of the liquid crystal display 5 to be transmitted to the first mirror 13 and the second mirror 14. In addition, the first mirror 13 and the second mirror 14 are arranged as close as possible to the liquid crystal display 5 in a range where the optical path and the liquid crystal display 5 do not overlap. Here, the range of light becomes wider as the distance from the liquid crystal display 5 increases. Therefore, by arranging the first mirror 13 and the second mirror 14 close to the liquid crystal display 5, the size of the first mirror 13 and the second mirror 14 can be reduced as much as possible, and the entire HUD 1 apparatus. The size can be reduced.

  In the present embodiment, the first optical path 19 reflected by the first mirror 13 and the second optical path 20 reflected by the second mirror 14 overlap with each other, and the incident directions to the concave mirror 7 are the same direction. Thus, the liquid crystal display 5, the first mirror 13 and the second mirror 14 are arranged. As a result, the occupant 9 can view the image reflected by the first mirror 13 and the image reflected by the second mirror 14 at the same time (that is, the first virtual image 10A and the first image based on the image reflected by the first mirror 13). The second virtual image 10B based on the image reflected by the two mirrors 14 can be viewed at the same time).

  Further, the image reflected by the first mirror 13 and the image reflected by the second mirror 14 are images displayed in different areas of the liquid crystal display 5. Specifically, as shown in FIG. 4, the first optical path 19 is an optical path incident from the first area 21 below the liquid crystal display 5, and the virtual image of the image displayed in the first area 21 is the first virtual image. It is visually recognized by the passenger 9 as 10A. On the other hand, the second optical path 20 is an optical path incident from the second region 22 above the liquid crystal display 5, and the virtual image of the image displayed in the second region 22 is visually recognized by the occupant 9 as the second virtual image 10B.

  Further, the optical path length of the first optical path 19 reflected by the first mirror 13 (more specifically, the optical path length of the first optical path 19 connecting the liquid crystal display 5 and the occupant 9) and the second mirror 14 are reflected. The optical path length of the second optical path 20 (more specifically, the optical path length of the second optical path 20 connecting the liquid crystal display 5 and the occupant 9) is different, and the optical path length of the first optical path 19 is longer. Yes. As a result, as shown in FIG. 5, the position of the first virtual image 10 </ b> A that is a virtual image visually recognized by the user through the first optical path 19 is greater than the position of the second virtual image 10 </ b> B that is a virtual image visually recognized by the user through the second optical path 20. Is also located far away. For example, the optical path length is set so that the virtual image generation distance L of the first virtual image 10A is 20 m and the virtual image generation distance L of the second virtual image 10B is 2.5 m. Further, as shown in FIG. 5, the region where the first virtual image 10 </ b> A is visually recognized from the occupant 9 and the region where the second virtual image 10 </ b> B is visually recognized overlap in the sight line direction of the occupant 9.

  Therefore, by switching and displaying the first virtual image 10A and the second virtual image 10B, it is possible to perform guidance for changing the distance from the occupant to the virtual image. For example, the second virtual image 10B is displayed first, then the second virtual image 10B is erased, and the first virtual image 10A having the same shape as the second virtual image 10B is displayed at a position overlapping the displayed second virtual image 10B in the line-of-sight direction. If it shows, it will be visually recognized from a passenger | crew so that the position of the virtual image may have moved to the position 20m away from the position 2.5m away from the passenger | crew. In the present embodiment, virtual images of a video that warns an object (another vehicle or a pedestrian) that is a warning target to an occupant are displayed as a first virtual image 10A and a second virtual image 10B.

  The reflective polarizing plate 6 has a property of reflecting (non-transmitting) light having a specific first polarization direction and transmitting light having a second polarization direction different from the first polarization direction. It is a member. In the reflective polarizing plate 6 of this embodiment, the first polarization direction is designed to be the same as the polarization direction of the light output from the liquid crystal display 5, and the second polarization direction is relative to the first polarization direction. Thus, the polarization direction is designed to have a phase difference of λ / 2 (180 degrees) (that is, a direction orthogonal to the first polarization direction). The reflective polarizing plate 6 is disposed between the first mirror 13 and the second mirror 14 and the concave mirror 7 along the first optical path 19 and the second optical path 20 connecting the liquid crystal display 5 and the concave mirror 7. Then, as shown in FIG. 3, the reflection of the light that changes in the third direction toward the concave mirror 7 with respect to the first optical path 19 and the second optical path 20 incident in the second direction from the first mirror 13 and the second mirror 14. Functions as a means. On the other hand, since the light reflected to the concave mirror 7 passes through the λ / 4 wavelength plate 15 twice as will be described later, the polarization direction is finally changed to the second polarization direction. The first optical path 19 and the second optical path 20 function as light transmitting means.

  On the other hand, the concave mirror 7 is a projection mirror that generates a virtual image of the image in front of the occupant 9 by enlarging and reflecting the image displayed on the liquid crystal display 5 and causing the occupant 9 to visually recognize the image. As the concave mirror 7, a spherical concave mirror, an aspheric concave mirror, or a free-form curved mirror for correcting distortion of a projected image is used.

  Also, as shown in FIG. 3, the concave mirror 7 changes the optical path incident in the third direction from the reflective polarizing plate 6 to the fourth direction that passes through the reflective polarizing plate 6 and moves toward the user's eyes. In particular, in this embodiment, the positions and angles of the liquid crystal display 5, the first mirror 13, the second mirror 14, the reflective polarizing plate 6 and the concave mirror 7 are set so that the third direction and the fourth direction are opposite to each other. To design. The “reverse direction” is not limited to a direction different by 180 degrees, but a direction having a certain range of width including a direction different by 180 degrees. For example, the direction different from 175 degrees or 185 degrees is also set as the opposite direction.

  As a result, the quality of the virtual image can be improved. For example, it becomes possible to make the occupant 9 visually recognize a clearer and more vivid virtual image. Furthermore, it is possible to give a width to the virtual image generation distance L that improves the quality of the virtual image. Therefore, there is a particularly great effect when virtual images are visually recognized at a plurality of positions having different virtual image generation distances L as in the present embodiment.

  The λ / 4 wavelength plate 15 is a polarizing element having a flat plate shape that gives a phase difference of λ / 4 (90 degrees) with respect to the polarization direction of light passing therethrough. In the present embodiment, the concave mirror 7 is disposed separately between the reflective polarizing plate 6 and the concave mirror 7 along the optical path connecting the liquid crystal display 5 and the concave mirror 7. Therefore, as shown in FIG. 3, the light output from the liquid crystal display 5 first passes through the λ / 4 wavelength plate 15 while moving from the reflective polarizing plate 6 to the concave mirror 7. Then, the light passes through the λ / 4 wavelength plate 15 again while being reflected by the concave mirror 7 and moved to the reflective polarizing plate 6. As a result, a phase difference of λ / 2 (180 degrees) is given in increments of λ / 4 with respect to the polarization direction of light. Accordingly, at the stage of output from the liquid crystal display 5, the light polarization direction is the first polarization direction that is a reflection target (non-transmission target) in the reflective polarizing plate 6. By passing through the number of times, the reflective polarizing plate 6 is finally displaced in the second polarization direction to be transmitted.

  Further, the λ / 4 wavelength plate 15 is disposed at an angle that is perpendicular to the first optical path 19 and the second optical path 20 that are transmitted therethrough. Furthermore, the size includes all the light transmitted from the reflective polarizing plate 6 in the second direction and the light transmitted from the concave mirror 7 in the third direction, and the size is as small as possible. In addition, the λ / 4 wavelength plate 15 is disposed in a range that does not overlap the optical path connecting the liquid crystal display 5 and the reflective polarizing plate 6.

  The control circuit unit 16 is an electronic control unit that controls the entire HUD 1. Here, FIG. 6 is a block diagram showing the configuration of the HUD 1 according to the present embodiment.

  As shown in FIG. 6, the control circuit unit 16 includes a CPU 31 as an arithmetic device and a control device, a RAM 32 used as a working memory when the CPU 31 performs various arithmetic processes, a control program, and information acquisition described later. An internal storage device such as a ROM 33 storing a processing program (see FIG. 7), a virtual image display processing program (see FIG. 9), and the like, and a flash memory 34 for storing a program read from the ROM 33 is provided. The control circuit unit 16 is connected to the liquid crystal display 5 and controls driving of the liquid crystal display 5.

  The CAN (controller area network) interface 17 is an interface for inputting / outputting data to / from CAN, which is a vehicle-mounted network standard that performs multiplex communication between various vehicle-mounted devices and vehicle equipment control devices installed in the vehicle. It is. The HUD 1 is connected to a control device (for example, the navigation device 48, the AV device 49, etc.) of various vehicle-mounted devices and vehicle equipment via the CAN so as to be able to communicate with each other. Thereby, the HUD 1 is configured to be able to display information acquired from the navigation device 48, the AV device 49, and the like.

  The heat sink 18 is a heat radiating member that is disposed on the back side of the liquid crystal display 5 and radiates heat generated from the light source of the liquid crystal display 5. Here, the heat sink 18 is made of, for example, aluminum, copper, or the like, and is formed into a shape (for example, a sword mountain shape or a bellows shape) having a large surface area. Further, a fan may be provided in order to enhance the heat dissipation effect. The heat sink 18 is disposed on the back side of the liquid crystal display 5, that is, on the opposite side of the liquid crystal display 5 with respect to the first mirror 13 and the second mirror 14. The heat sink 18 is positioned above the liquid crystal of the liquid crystal display 5, the first mirror 13, and the second mirror 14. Accordingly, it is possible to reduce the adverse effect of the heat radiated by the heat sink 18 on the liquid crystal of the liquid crystal display 5 and other optical members.

  Next, an information acquisition processing program executed by the CPU 31 in the HUD 1 having the above configuration will be described with reference to FIG. FIG. 7 is a flowchart of the information acquisition processing program according to this embodiment. Here, the information acquisition processing program is a program that is repeatedly executed at predetermined time intervals after the ACC power supply (accessory power supply) of the vehicle is turned on, and acquires various information necessary for displaying a virtual image. 7 and 9 are stored in the RAM 32 and the ROM 33 provided in the HUD 1 and executed by the CPU 31.

  First, in step (hereinafter abbreviated as S) 1 in the information acquisition processing program, the CPU 31 has a target object (hereinafter referred to as a warning target object) that is a warning target for the vehicle occupant in front of the traveling direction of the vehicle. It is determined whether or not. Here, the warning object is, for example, an object that is located at a predetermined distance or more (for example, 50 m or more) away from the vehicle occupant and is difficult to see clearly from the occupant, or a blind spot of the vehicle occupant (for example, shielded by a shield) The target object is only partially visible from the occupant. Further, the “object” is an object to be watched when a vehicle occupant travels. For example, in addition to obstacles such as other vehicles, pedestrians, and two-wheeled vehicles, signs, traffic lights, and the like also apply.

  Further, the presence or absence of a warning object ahead of the traveling direction of the vehicle may be detected, for example, by performing image processing on a captured image captured by the front camera 11 or using a sensor such as a millimeter wave radar. It may be detected. Moreover, you may acquire by communication with a vehicle-to-vehicle communication or an external server.

  And when it determines with there being a warning target object ahead of the advancing direction of a vehicle (S1: YES), it transfers to S2. On the other hand, when it is determined that there is no warning object ahead in the traveling direction of the vehicle (S1: NO), the information acquisition processing program is terminated.

  In S <b> 2, the CPU 31 acquires the specific position coordinates of the warning object determined to be ahead of the traveling direction of the vehicle and the distance from the vehicle occupant to the warning object. The position coordinates and distance of the warning object may be acquired by performing image processing on a captured image captured by the front camera 11, for example, or may be acquired using a sensor such as a millimeter wave radar. good. Moreover, you may acquire by communication with a vehicle-to-vehicle communication or an external server.

  Next, in S <b> 3, the CPU 31 acquires the luminance distribution of the surrounding environment ahead of the traveling direction of the vehicle on which the virtual image is to be superimposed. Note that the luminance distribution of the surrounding environment may be acquired from, for example, a captured image captured by the front camera 11 or may be acquired using a sensor such as a luminance sensor.

  Subsequently, in S4, the CPU 31 detects the sight line start point (eye position) and the sight line direction of the occupant based on the captured image of the in-vehicle camera 12. The in-vehicle camera 12 is installed on the dashboard or ceiling of the vehicle as described above, and is installed with the imaging direction facing the driver's seat, and the captured image includes the driver's face. As a method for detecting the line-of-sight start point and the line-of-sight direction, there is a method of detecting using the center position of the pupil or the Purkinje image measured by the corneal reflection method, for example. Since these methods are already known techniques, details are omitted.

  Thereafter, in S5, the CPU 31 generates a display line connecting the position coordinates of the warning object acquired in S2 and the sight line start point of the occupant acquired in S4. For example, as shown in FIG. 8, there is a warning object 51 in front of the traveling direction of the occupant 9 of the vehicle, the position coordinates of the warning object 51 are (x1, y1, z1), and the sight line start point (eye When (position) is detected as (x2, y2, z2), a straight line connecting (x1, y1, z1) and (x2, y2, z2) is generated as the display line 52.

  Next, in S6, the CPU 31 obtains the current vehicle speed of the vehicle via a vehicle speed sensor mounted on the vehicle.

  Thereafter, in S7, the CPU 31 stores the information acquired in S2 to S6 in the flash memory 34 or the like. The stored information is used in a virtual image display processing program (FIG. 9) described later. The processes of S1 to S7 are performed at predetermined intervals while the ACC power supply of the vehicle is turned on, and the latest acquired information is sequentially overwritten in S7.

  Next, a virtual image display processing program executed by the CPU 31 in the HUD 1 will be described with reference to FIG. FIG. 9 is a flowchart of the virtual image display processing program according to the present embodiment. Here, the virtual image display processing program is a program that is executed after an ACC power supply (accessory power supply) of the vehicle is turned on, and displays a virtual image for allowing a vehicle occupant to visually recognize an object to be warned.

  First, in S <b> 11 in the virtual image display processing program, the CPU 31 determines whether or not there is a warning object that is a warning target for the vehicle occupant ahead of the traveling direction of the vehicle. The details are the same as in S1, and a description thereof will be omitted.

  And when it determines with there being a warning target object ahead of the advancing direction of a vehicle (S11: YES), it transfers to S12. On the other hand, when it is determined that there is no warning object ahead in the traveling direction of the vehicle (S11: NO), the virtual image display processing program is terminated without displaying a virtual image on the warning object.

  Next, in S12, the CPU 31 reads the latest information stored in the flash memory 34 in S7 of the information acquisition processing program (FIG. 7) executed in parallel with this program. The information to be read out includes the position coordinates of the warning object, the distance from the vehicle occupant to the warning object, the luminance distribution of the surrounding environment, the sight line start point (eye position) and direction of the occupant, the display line, and the like. is there.

  Subsequently, in S13, the CPU 31 determines whether or not the vehicle occupant can visually recognize the warning object based on the information acquired in S12 and the captured image captured by the front camera 11. Specifically, if the area ratio allows 50% or more of the warning object to be visually recognized by the occupant, it is determined that the warning object is visible. Even if 50% or more of the warning object is visible from the occupant by area ratio, if the warning object is located at a predetermined distance or more (for example, 50 m or more) from the occupant of the vehicle, Since it is difficult to visually recognize the warning object clearly, it may be determined that the warning object is not visible.

  And when it determines with the passenger | crew of a vehicle being able to visually recognize a warning target object (S13: YES), it transfers to S20. On the other hand, when it is determined that the vehicle occupant cannot visually recognize the warning object (S13: NO), the process proceeds to S14.

  In S14, the CPU 31 performs various settings for displaying a virtual image. Specifically, the display position of the video on the liquid crystal display 5, the display brightness of the video, and the size of the video are set. First, the display position of the video is a position for generating a virtual image at a position overlapping the display line in the second region 22 (FIG. 4). The display brightness of the video is higher than the brightness of the surrounding environment where the virtual image is superimposed. The size of the video is at least a size that covers the entire warning object.

  Thereafter, in S15, the CPU 31 outputs an image for generating a virtual image in the second region 22 of the liquid crystal display 5 based on the setting in S14. For example, an icon image indicating that the object is a warning object is output. In addition, the 2nd virtual image 10B which is a virtual image of the image | video displayed on the 2nd area | region 22 is visually recognized in the position 2.5 m away which is the vicinity of the passenger | crew of a vehicle as mentioned above (refer FIG. 5).

  As a result, as shown in FIG. 10, the second virtual image 10B superimposed on the entire warning object 51 in the forward direction of the vehicle is visually recognized at a position 2.5 m away from the vehicle occupant. The position where the second virtual image 10B is displayed is between the vehicle occupant and the warning object, that is, closer to the vehicle occupant side than the warning object. The video output to the second area 22 is continuously performed until the video output to the first area 21 is performed in S22 described later.

  Subsequently, in S16, the CPU 31 determines whether or not the line of sight of the vehicle occupant is approaching the display line, that is, whether or not the line of sight of the vehicle occupant is positioned around the warning object. As described above, the display line is a line connecting the position coordinates of the warning object and the sight line start point of the occupant (FIG. 8). Accordingly, when the latest sight line direction of the occupant acquired in S12 is within a predetermined angle difference (for example, ± 3 degrees) from the display line, it is determined that the sight line of the vehicle occupant is approaching the display line.

  If it is determined that the line of sight of the vehicle occupant is close to the display line, that is, the line of sight of the vehicle occupant is located around the warning object (S16: YES), the process proceeds to S20. To do. On the other hand, when it is determined that the line of sight of the vehicle occupant is not close to the display line, that is, the line of sight of the vehicle occupant is not located around the warning object (S16: NO), the process proceeds to S17. And migrate.

  In S17, the CPU 31 determines whether or not the distance from the vehicle (that is, the vehicle occupant) to the warning object is equal to or less than a threshold value. The threshold value is the shortest stop distance at which the vehicle can stop without emergency stop control, and is calculated based on the current vehicle speed of the vehicle acquired in S12.

  If it is determined that the distance from the vehicle to the warning object is equal to or smaller than the threshold value (S17: YES), it is determined that the warning to the warning object is urgent, and the process proceeds to S20. On the other hand, if it is determined that the distance from the vehicle to the warning object is longer than the threshold (S17: NO), it is determined that the warning to the warning object is not urgent at the present time, and the process proceeds to S18. And migrate.

  In S18, the CPU 31 blinks and displays the image displayed in the second area 22 of the liquid crystal display 5 in S15. As a result, the second virtual image 10B visually recognized by the vehicle occupant is also displayed in a blinking manner, and it is possible to easily guide the line of sight of the vehicle occupant toward the second virtual image 10B.

  Thereafter, in S19, the CPU 31 newly reads out the latest information stored in the flash memory 34 in S7 of the information acquisition processing program (FIG. 7) executed in parallel with this program. The information to be read out includes the position coordinates of the warning object, the distance from the vehicle occupant to the warning object, the luminance distribution of the surrounding environment, the sight line start point (eye position) and direction of the occupant, the display line, and the like. is there. Thereafter, the process proceeds to S14.

  On the other hand, in S20, the CPU 31 outputs a warning sound for notifying that there is a warning object from a speaker in the vehicle. As a result, even if the vehicle occupant is not aware of the warning object, the warning object can be noticed.

  Next, in S21, the CPU 31 performs various settings for displaying a virtual image. Specifically, the display position of the video on the liquid crystal display 5, the display brightness of the video, and the size of the video are set. First, the display position of the video is a position for generating a virtual image at a position overlapping the display line in the first region 21 (FIG. 4). Further, the display brightness of the video is darker than the brightness set in S14, and is about the same as the brightness of the surrounding environment where the virtual image is superimposed. The size of the video is smaller than the size set in S14 and covers only a part of the warning object.

  Thereafter, in S22, the CPU 31 outputs a video for generating a virtual image in the first region 21 of the liquid crystal display 5 based on the setting in S21. The content of the video to be output may be the same video as the video displayed in S15 or may be a different video. For example, a yellow icon image may be output in S15, and a red icon image may be output in S22. If the video is displayed on the second area 22 of the liquid crystal display 5 in S15, the video on the second area 22 is erased as the video is output to the first area 21. Note that the timing for switching the video output from the second area 22 to the first area 21 may be immediately after YES is determined in S16 or S17, or after YES is determined in S16 or S17. It may be after a predetermined time has elapsed. The first virtual image 10A, which is a virtual image of the image displayed in the first region 21, is visually recognized at a position 20 m away from the vehicle occupant as described above (see FIG. 5).

  As a result, as shown in FIG. 11, the first virtual image 10 </ b> A superimposed on only a part of the warning object 51 in the forward direction of the vehicle is visually recognized at a position 20 m away from the vehicle occupant. In this embodiment, by switching from the second virtual image 10B shown in FIG. 10 to the first virtual image 10A shown in FIG. 11, the warning object is displayed along the display line connecting the vehicle occupant and the warning object. It is possible to perform guidance for moving the position of the virtual image in the direction of. As a result, it becomes possible to guide the user's line of sight to the warning object by a virtual image. Thereby, it is possible to make the user visually recognize the warning object. Video output to the first area 21 is basically performed until the warning object disappears from the front of the vehicle thereafter.

  In the present embodiment, the second virtual image 10B shown in FIG. 10 displays a virtual image superimposed on the entire warning target object 51, but the first virtual image 10A shown in FIG. 11 shows only a part of the warning target object 51. A virtual image is displayed superimposed on. As a result, in a situation where the line of sight of the vehicle occupant is not in the vicinity of the warning object 51, it is possible to easily guide the line of sight of the passenger to the vicinity of the warning object 51 by displaying the virtual image as large as possible. Become. On the other hand, after the occupant's line of sight is positioned around the warning object 51, the size of the virtual image is reduced as much as possible, so that the visual recognition of the warning object 51 is not hindered by the virtual image, and the warning object 51 is reliably obtained. Can be visually recognized.

  Furthermore, in the present embodiment, the second virtual image 10B shown in FIG. 10 displays a virtual image with higher brightness than the surrounding environment, but the first virtual image 10A shown in FIG. 11 displays a virtual image with the same brightness as the surrounding environment. To do. As a result, in a situation where the line of sight of the vehicle occupant is not in the vicinity of the warning object 51, it is possible to easily guide the line of sight of the occupant to the vicinity of the warning object 51 by displaying a virtual image with the highest possible brightness. Become. On the other hand, after the occupant's line of sight is positioned in the vicinity of the warning object 51, the brightness of the virtual image is brought close to the surrounding environment, so that the visual recognition of the warning object 51 is not hindered by the virtual image, and the warning object 51 is reliably made. Can be visually recognized.

  The liquid crystal display 5 may display information related to the vehicle 2 and other information used for assisting driving of the occupant 9 in addition to the warning image for the warning object. For example, a guide route (scheduled travel route) set by the navigation device 48, guide information based on the guide route (such as an arrow indicating the traveling direction of the vehicle), current vehicle speed, guide sign, map image, traffic information, news, weather forecast, There are time, connected smartphone screen, TV screen and so on.

  As described above in detail, according to the HUD 1 according to the present embodiment, when there is a warning object to be a warning target for the vehicle occupant, the warning object A virtual image is displayed at a position that is superposed on at least a part and visually recognized by the occupant (S15). After that, when the sight line of the occupant is positioned around the warning object, the virtual image is displayed in a direction connecting the occupant and the warning object. Accordingly, the position of the virtual image is moved in the direction of the warning object (S21, S22), so that even if the vehicle occupant's line of sight is away from the warning object, the virtual image depends on the occupant's line of sight. It is possible to guide the occupant's line of sight to the warning object by a virtual image. As a result, it is possible to make the occupant visually recognize the warning object.

Note that the present invention is not limited to the above-described embodiment, and various improvements and modifications can be made without departing from the scope of the present invention.
For example, in the present embodiment, the virtual image is generated in front of the front window 8 of the vehicle 2 by the HUD 1, but the virtual image may be generated in front of a window other than the front window 8. Further, the object to be reflected by the HUD 1 may be a visor (combiner) installed around the front window 8 instead of the front window 8 itself.

  In the present embodiment, the HUD 1 is installed on the vehicle 2. However, the HUD 1 may be installed on a moving body other than the vehicle 2. For example, it can be installed on a ship or an aircraft. Moreover, you may install in the ride type attraction installed in an amusement facility. In that case, a virtual image can be generated around the ride so that the rider can visually recognize the virtual image.

  In the present embodiment, the display position of the image on the liquid crystal display 5 is switched from the second area 22 to the first area 21 (that is, the optical path length connecting the liquid crystal display 5 and the occupant is changed from the first optical path length to the first optical path length. The position of the virtual image is moved in the direction of the warning object by switching to the second optical path length that is longer than the optical path length), but the optical path length of the optical path connecting the liquid crystal display 5 and the occupant is gradually increased. By doing so, it is good also as a structure which moves the position of a virtual image to the direction of a warning target object. For example, it is possible to realize a configuration in which the position of the mirror that turns the liquid crystal display 5 or the optical path is movable using a driving source such as a motor, and the optical path length is gradually changed by moving the liquid crystal display 5 or the mirror. . As the optical path length is gradually increased, it is desirable that the display size of the image displayed on the liquid crystal display 5 is gradually changed to be controlled so that the luminance is gradually lowered so as to approach the surrounding environment.

  In this embodiment, the virtual image display position is moved from a position 2.5 m away from the vehicle occupant to a position 20 m away, but the warning object is moved along the direction connecting the occupant and the warning object. If the virtual image moves in the direction of the object, the position where the virtual image is displayed can be changed as appropriate. For example, it may be moved from a position 2.5 m away from the passenger to a position 40 m away.

  In the present embodiment, the warning object is an object that is in the blind spot of the vehicle occupant (for example, a position shielded by the shielding object) and is only partially visible by the occupant. Even if the entire object can be visually recognized, an object that is difficult to visually recognize due to other reasons (for example, backlight) may be included in the warning object.

  Moreover, although the embodiment which actualized the virtual image display device according to the present invention has been described above, the virtual image display device can also have the following configuration, and in that case, the following effects can be obtained.

For example, the first configuration is as follows.
A virtual image display device (1) that allows a user to visually recognize a virtual image of the video by allowing the user to visually recognize the video displayed on the video display surface (5), and an object to be a warning target for the user (9) A position acquisition means (31) for acquiring the position of (51), and a virtual image that displays a virtual image at a position that is between the target object and a user and is visually recognized by the user by overlapping at least a part of the target object. After the virtual image is displayed by the display means (31), the visual line detection means (31) for detecting the visual line of the user, and the virtual image display means, and when the visual line of the user is positioned around the object And virtual image position changing means (31) for moving the position of the virtual image in the direction of the target object along the direction connecting the user and the target object.
According to the virtual image display device having the above configuration, even if the user's line of sight is in a position away from the target object to be warned, by moving the position of the virtual image according to the user's line of sight, It is possible to guide the line of sight to a target object to be warned by a virtual image. Thereby, it becomes possible to make the user visually recognize the target object to be warned.

The second configuration is as follows.
The position acquisition means (31) acquires the position of an object that is at a predetermined distance or more away from the user (9) or the position of the object in the blind spot of the user as the position of the object that is a warning target for the user. To do.
According to the virtual image display device having the above-described configuration, when there is an object that is difficult for the user to visually recognize due to being far away from the user or being shielded by the shielding object, the object is finally given to the user. It becomes possible to make it visible reliably.

The third configuration is as follows.
The virtual image position changing means (31) switches the optical path length of the optical path connecting the video display surface (5) and the user (9) from the first optical path length to the second optical path length longer than the first optical path length. To move the position of the virtual image in the direction of the object (51).
According to the virtual image display device having the above-described configuration, it is possible to guide the user's line of sight to the target object to be warned by the virtual image by switching the display position of the virtual image from the vicinity of the user to the distance.

The fourth configuration is as follows.
The virtual image position changing means (31) gradually increases the optical path length of the optical path connecting the video display surface (5) and the user (9), thereby moving the virtual image in the direction of the object (51). Move position.
According to the virtual image display device having the above configuration, by gradually moving the display position of the virtual image from the vicinity of the user to the distance, it is possible to guide the user's line of sight to the target object to be warned by the virtual image. Become.

The fifth configuration is as follows.
It has a distance acquisition means (31) which acquires the distance from the user (9) to the object (51), and the virtual image position changing means (31) is a function of the user's line of sight before the object is positioned around the object. Even so, when the distance from the user to the object becomes equal to or less than the threshold value, the position of the virtual image is moved in the direction of the object.
According to the virtual image display device having the above-described configuration, when the warning to the warning target object is urgent, it is possible to promptly guide the user's line of sight to the target object to be the warning target.

The sixth configuration is as follows.
As the virtual image position changing unit (31) moves the position of the virtual image, the virtual image position changing unit (31) includes a size changing unit (31) that changes the size of the virtual image visually recognized by the user.
According to the virtual image display device having the above configuration, the size of the virtual image is reduced after the user's line of sight is positioned around the target, so that the visual recognition of the target is not hindered by the virtual image. Can be visually recognized.

The seventh configuration is as follows.
The virtual image display means (31) displays the virtual image with an initial size that is superimposed on the entire object (51) and is visually recognized by the user, and the size changing means (31) displays the size of the virtual image as the size of the virtual image. The initial size is superposed on only a part of the object and changed to a size visually recognized by the user.
According to the virtual image display device having the above configuration, in a situation where the user's line of sight is not in the vicinity of the object, the user's line of sight can be easily guided to the periphery of the object by displaying the virtual image as large as possible. It becomes possible. On the other hand, after the user's line of sight is located around the object, it is possible to make the object surely visible without reducing the size of the virtual image as much as possible, thereby preventing the object from being visually recognized by the virtual image. It becomes.

The eighth configuration is as follows.
Luminance acquisition means (31) for acquiring the brightness of the surrounding environment on which the virtual image is superimposed, the virtual image display means (31) displays the virtual image with an initial brightness higher than the brightness of the surrounding environment, and the virtual image In accordance with the movement of the position of the virtual image by the position changing means, there is provided a luminance changing means (31) for bringing the brightness of the virtual image closer to the brightness of the surrounding environment from the initial brightness.
According to the virtual image display device having the above configuration, in a situation where the user's line of sight is not in the vicinity of the target object, the user's line of sight can be easily guided to the periphery of the target object by displaying the virtual image with higher brightness than the surrounding environment. It becomes possible. On the other hand, after the user's line of sight is located in the vicinity of the target object, the virtual image brightness is brought close to the surrounding environment, so that the target object can be surely viewed without being obstructed by the virtual image. It becomes.

DESCRIPTION OF SYMBOLS 1 Head up display apparatus 2 Vehicle 3 Dashboard 5 Liquid crystal display 6 Reflective polarizing plate 7 Concave mirror 8 Front window 9 Crew 10A 1st virtual image 10B 2nd virtual image 11 Front camera 12 Car camera 13 1st mirror 14 2nd mirror 19 1st Optical path 20 Second optical path 31 CPU
32 RAM
33 ROM
34 Flash memory 51 Warning object 52 Display line

Claims (8)

A virtual image display device that allows a user to visually recognize a virtual image of the video by allowing a user to visually recognize a video displayed on a video display surface,
Position acquisition means for acquiring the position of the object to be warned to the user;
A virtual image display means for displaying a virtual image at a position that is visible between the user and the object, and is superimposed on at least a part of the object;
Gaze detection means for detecting the gaze of the user;
After the virtual image is displayed by the virtual image display means, and when the user's line of sight is located around the object, the direction of the object along the direction connecting the user and the object A virtual image display device comprising: virtual image position changing means for moving the position of the virtual image.   The said position acquisition means acquires the position of the target object which exists in the position more than predetermined distance from the user, or the target object in a user's blind spot as a position of the target object used as a warning object with respect to a user. Virtual image display device.   The virtual image position changing means switches the optical path length of the optical path connecting the video display surface and the user from the first optical path length to the second optical path length longer than the first optical path length, thereby moving in the direction of the object. The virtual image display device according to claim 1, wherein a position of the virtual image is moved.   The virtual image position changing means moves the position of the virtual image in the direction of the object by gradually increasing the optical path length of the optical path connecting the video display surface and the user. The virtual image display device described. A distance acquisition means for acquiring a distance from the user to the object;
The virtual image position changing unit is configured to move in the direction of the object when the distance from the user to the object is equal to or less than a threshold even before the user's line of sight is positioned around the object. The virtual image display device according to claim 1, wherein the virtual image display device moves a position of the virtual image.   The virtual image according to any one of claims 1 to 5, further comprising size changing means for changing a size of the virtual image visually recognized by a user as the virtual image position changing means moves the position of the virtual image. Display device. The virtual image display means displays the virtual image with an initial size that is superimposed on the entire object and visually recognized by a user.
The virtual image display device according to claim 6, wherein the size changing unit changes the size of the virtual image from the initial size to a size visually recognized by a user by superimposing only the part of the object. Luminance acquisition means for acquiring the luminance of the surrounding environment where the virtual image is superimposed,
The virtual image display means displays the virtual image with an initial luminance higher than the luminance of the surrounding environment,
8. The apparatus according to claim 1, further comprising: a brightness changing unit configured to bring the brightness of the virtual image closer to the brightness of the surrounding environment from the initial brightness as the position of the virtual image is moved by the virtual image position changing unit. The virtual image display device described.

Images