JPWO2013088486A1 - Display device, display method, and display program - Google Patents

Abstract

  The display device includes an emitting unit that emits image information light and an optical element that receives the image information light, and causes the user's retina to irradiate the user's retina with the image information light so as to visually recognize the image. . The image information light includes first image information light and second image information light, and one of the first image information light and the second image information light is a retina according to the movement of the user's eyeball. Are condensed at different positions, respectively. The control means provides guidance information for prompting the user to move the eyeball so that the second image information light is irradiated on the retina from the state where the first image information light is irradiated on the retina. Control to notify is performed.

Description

  The present invention relates to a display device that visually recognizes an image by projecting light onto a user's retina.

  2. Description of the Related Art Conventionally, a technique for displaying information related to vehicle driving using a head-up display or the like is known. For example, in Patent Document 1, a warning display for alerting the driver to a position that does not interfere with the driver's line of sight during driving is performed, and the driver's line of sight is displayed from the position where the warning display is performed. A technique for performing guidance display for guiding to a position of a guidance target has been proposed.

JP 2003-291688 A

  In the technique described in Patent Literature 1 described above, the driver's gaze guidance is basically performed after alerting in the peripheral vision region of the field of view during forward gaze. However, Patent Document 1 does not describe that information is appropriately displayed outside the peripheral vision by performing line-of-sight guidance to the outer region of the peripheral vision during driving.

  Examples of the problem to be solved by the present invention are as described above. It is an object of the present invention to provide a display device, a display method, and a display program that can appropriately present information over a wide range that is greater than peripheral vision.

  According to the first aspect of the present invention, there is provided an emitting means for emitting image information light and an optical element on which the image information light is incident, and the user's retina is irradiated with the image information light via the optical element. The image information light includes first image information light and second image information light, and the first image information light and the second image information light are Depending on the movement of the user's eyeball, either one is condensed at different positions so that one is irradiated on the retina, and the first image information light is irradiated on the retina. And control means for performing control for notifying guidance information for prompting the user to move the eyeball so that the second image information light is irradiated on the retina.

  According to a fifteenth aspect of the present invention, the image processing apparatus includes an emission unit that emits image information light and an optical element that receives the image information light, and irradiates the user's retina with the image information light via the optical element. The image information light includes a first image information light and a second image information light, and the first image information light and the first image information light are displayed. The two-image information light is condensed at different positions so that either one is irradiated on the retina according to the movement of the user's eyeball, and the first image information light is applied to the retina. A control step of performing control for notifying guidance information for prompting the user to move the eyeball so that the second image information light is irradiated on the retina from the irradiated state; Prepare.

  According to a sixteenth aspect of the present invention, the image processing apparatus includes an emission unit that emits image information light, an optical element that receives the image information light, and a computer, and uses the image information light via the optical element as a user's retina. The image information light includes first image information light and second image information light, and the first image information light The second image information light is condensed at different positions so that either one is irradiated on the retina according to the movement of the user's eyeball, and the first image information light is Control for notifying the user of guidance information for prompting the user to move the eyeball so that the second image information light is irradiated on the retina from the state irradiated on the retina Means, as said con To work Yuta.

  According to an eighteenth aspect of the invention, there is provided an emitting means for emitting image information light and an optical element to which the image information light is incident, and the user's retina is irradiated with the image information light via the optical element. The image information light includes first image information light and second image information light, and the first image information light and the second image information light are In accordance with the movement of the user's eyeball, either one is condensed at different positions so that the main region of the retina is irradiated, and the first image information light is irradiated onto the retina. Control means for performing control for notifying the user of guidance information for prompting the user to move the eyeball so that the second image information light is irradiated on the retina.

  According to a nineteenth aspect of the present invention, there is provided an emitting means for emitting image information light and an optical element on which the image information light is incident, and the user's retina is irradiated with the image information light via the optical element. The image information light includes first image information light and second image information light, and the first image information light and the second image information light are The second image information light is applied to the retina from the state where the first image information light is applied to the retina according to the movement of the user's eyeball. Control means for performing control for notifying the user of guidance information for prompting the user to move the eyeball so that the eyeball moves.

It is the figure which showed schematically the structure of the display apparatus which concerns on 1st Example. It is a figure which shows the specific example of the spot where an image is displayed in 1st Example. It is a figure which shows schematic structure of the light source unit which concerns on 1st Example. The figure for demonstrating an example of the display method which concerns on 1st Example is shown. It is a flowchart which shows the display method which concerns on 1st Example. The figure for demonstrating the other example of the display method which concerns on 1st Example is shown. It is a figure which shows the specific example of the spot where an image is displayed in 2nd Example. The figure for demonstrating an example of the display method which concerns on 2nd Example is shown. 6 is a diagram illustrating a schematic configuration of a hologram element according to Modification Example 1. FIG.

  In one aspect of the present invention, the image processing apparatus includes an emitting unit that emits image information light, and an optical element that receives the image information light, and irradiates a user's retina with the image information light via the optical element. The image information light includes a first image information light and a second image information light, and the first image information light and the second image information light are: Depending on the movement of the user's eyeball, either one is focused on the retina, and the light is condensed at different positions, and the first image information light is irradiated on the retina. Control means for performing control for notifying guidance information for prompting the user to move the eyeball so that the second image information light is irradiated on the retina.

  The display device includes an emitting unit and an optical element (for example, a hologram element), and causes an image to be visually recognized by projecting image information light via the optical element onto the retina. The image information light includes first image information light and second image information light. The first image information light and the second image information light are condensed at different positions by the function of the optical element so that either one is irradiated on the retina according to the movement of the user's eyeball. The The control means provides guidance information for prompting the user to move the eyeball so that the second image information light is irradiated on the retina from the state where the first image information light is irradiated on the retina. Control to notify is performed. Thus, by guiding the user to move his / her line of sight, information can be appropriately presented also to the area outside the area being watched. Therefore, it is possible to appropriately present information in a wider range than peripheral vision.

  In one aspect of the display device, the control unit performs control for displaying a guide image as control for informing the guide information, and the guide image is obtained by projecting the first image information light onto the retina. It is an image corresponding to a second image included in one image and projected by the second image information light onto the retina. By using such a guide image, it is possible to appropriately prompt the user to move the line of sight from the area where the first image is displayed to the area where the second image is displayed.

  In another aspect of the display device, the control unit is a position away from a central region in the first image, and a position corresponding to a direction of an arrangement position of the second image with respect to the first image. Then, the guidance image is displayed. Thereby, a user's eyes | visual_axis can be guide | induced effectively.

  In another aspect of the display device, the control unit displays an image included in the second image as the guide image. Thereby, in a state where the line of sight is directed to the area where the first image is displayed, it is possible to grasp that the second image is displayed and an outline of the second image.

  In another aspect of the above display device, the control unit can perform control to generate sound according to information to be displayed by the second image information light as control for informing the guidance information.

  In another aspect of the above display device, the control means may perform control to vibrate a vehicle handle in accordance with information to be displayed by the second image information light, as control for informing the guidance information. it can.

  In another aspect of the above display device, the control means performs control to notify the guidance information when predetermined information is to be displayed by the second image information light. In a preferred example, the predetermined information is information relating to an emergency vehicle. In another preferred example, the predetermined information is information related to an object that is likely to cause a traffic accident. In another preferred example, the predetermined information is information regarding a predetermined point on the map that is not directly related to driving or navigation of the vehicle.

  In another aspect of the display device, the optical element includes a first region and a second region, and the first image information light is light that has passed through the first region, and the second image information The light is light that has passed through the second region. Preferably, the first area is formed in the optical element such that the first image information light is projected onto the retina when the user's eyeball faces the front, and the second area is The second image information light is formed on the optical element so that the second image information light is projected onto the retina when the user's eyeball faces a predetermined direction other than the front.

  In the above display device, it is preferable that the second image information light is projected onto the retina when the user turns his / her eyeball toward a room mirror or a door mirror while the vehicle is driving. Formed on the optical element. As a result, since a room mirror or a door mirror can be used as a target when moving the line of sight, it is possible to instantaneously move the line of sight and view information. That is, by associating limited specific objects in the vehicle, such as the room mirror and the door mirror, with the position where the line of sight is moved, it is possible to easily determine the position where the line of sight should be moved when displaying the guidance image.

  Preferably, in the above display device, the second area is formed when the user turns his eyeball in a direction corresponding to a direction in which a target of information to be displayed by the second image information light is present. Two image information lights are formed on the optical element so as to be projected onto the retina. Accordingly, the information target can be easily determined by associating the direction in which the line of sight is moved with the direction in which the information target exists.

  In another aspect of the present invention, the image processing apparatus includes an emitting unit that emits image information light and an optical element that receives the image information light, and irradiates the user's retina with the image information light via the optical element. In this display method, the image information light includes first image information light and second image information light, and the first image information light and the second image information light are displayed. The image information light is condensed at different positions so that either one is irradiated on the retina according to the movement of the user's eyeball, and the first image information light is irradiated on the retina. A control step of performing control for notifying the user of guidance information for urging the user to move the eyeball so that the second image information light is irradiated on the retina from a state where the second image information light is irradiated. .

  According to still another aspect of the present invention, the image processing apparatus includes: an emitting unit that emits image information light; an optical element that receives the image information light; and a computer. The image information light includes first image information light and second image information light, and the first image information light The second image information light is condensed at different positions so that either one is irradiated on the retina according to the movement of the user's eyeball, and the first image information light is Control for notifying the user of guidance information for prompting the user to move the eyeball so that the second image information light is irradiated on the retina from the state irradiated on the retina Means, as said compilation To work over data.

  The display program can be suitably handled in a state where it is recorded on a recording medium.

  In still another aspect of the present invention, the image processing apparatus includes an emitting unit that emits image information light and an optical element that receives the image information light, and irradiates the user's retina with the image information light via the optical element. The image information light includes first image information light and second image information light, and the first image information light and the second image information light are In accordance with the movement of the user's eyeball, either one is condensed at different positions so that the main region of the retina is irradiated, and the first image information light is irradiated onto the retina. Control means for performing control for notifying the user of guidance information for prompting the user to move the eyeball so that the second image information light is irradiated on the retina.

  In still another aspect of the present invention, the image processing apparatus includes an emitting unit that emits image information light and an optical element that receives the image information light, and irradiates the user's retina with the image information light via the optical element. The image information light includes first image information light and second image information light, and the first image information light and the second image information light are The second image information light is applied to the retina from the state where the first image information light is applied to the retina according to the movement of the user's eyeball. Control means for performing control for notifying the user of guidance information for prompting the user to move the eyeball so that the eyeball moves.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
First, the first embodiment will be described.

(overall structure)
FIG. 1 is a diagram schematically illustrating a configuration of a display device 100 according to the first embodiment. As shown in FIG. 1A, the display device 100 mainly includes a light source unit 1, a scanning mechanism 2, a hologram element 3, and a beam splitter 4. The display apparatus 100 is an apparatus that visually recognizes an image by projecting image information light including image information onto a user's retina. For example, the display device 100 is used as a head mounted display (HMD).

  The light source unit 1 includes a red LD (laser diode), a green LD, a blue LD, and the like, and emits red, green, and blue laser beams (hereinafter also simply referred to as “light”). In addition, the light source unit 1 includes control circuits such as a video ASIC (Application Specific Integrated Circuit) and a laser driver ASIC. For example, the light source unit 1 performs intensity modulation on the laser light according to the image, and emits the laser light after intensity modulation. The light source unit 1 corresponds to an example of “emission means” and “control means” in the present invention.

  Laser light emitted from the light source unit 1 enters the scan mechanism 2. The scanning mechanism 2 includes a mirror, an actuator, and the like, and deflects the laser light from the light source unit 1 toward the hologram element 3. The scanning mechanism 2 performs a scanning operation for changing the position on the retina where the laser beam is irradiated in order to draw an image to be displayed on the retina.

  The hologram element 3 is configured as a transmission type, and condenses the laser light from the scanning mechanism 2 and emits it toward the beam splitter 4. The hologram element 3 corresponds to an example of an “optical element” in the present invention. The beam splitter 4 is a half mirror in one example, and reflects the laser beam from the hologram element 3 toward the user's eyeball. As a result, the laser light is condensed near the user's pupil (including the pupil, and so on), and the laser light is projected onto the user's retina. As a result, the image formed by the display device 100 is visually recognized by the user.

  The display device 100 uses the principle of Maxwell's view with respect to the point that laser light is once condensed near the pupil. Maxwell's vision is a method in which light corresponding to an image is once converged at the center of the pupil and then projected onto the retina. According to this method, an image can be observed without being affected by the lens's adjustment function. it can.

  With reference to FIG.1 (b), the hologram element 3 which concerns on 1st Example is demonstrated concretely. FIG. 1B is a diagram in which the hologram element 3 is observed from the direction indicated by the arrow X1 in FIG.

  As shown in FIG. 1B, the hologram element 3 has four hologram regions 3a to 3d. The hologram areas 3a to 3d have substantially the same size and are formed at positions that do not overlap each other. Each of the hologram regions 3a to 3d has a function of collecting incident light and a function of deflecting incident light. Specifically, the hologram regions 3a to 3d deflect light in different directions and form focal points at different positions.

  Thereby, as shown in FIG. 1A, the hologram area 3a generates a focal point at the point P1a, the hologram area 3b generates a focal point at the point P1b, and the hologram area 3c generates a focal point at the point P1c. Region 3d generates a focal point at point P1d. In this case, the hologram regions 3a to 3d generate focal points P1a to P1d at positions that are separated to some extent so that light through each of the hologram regions 3a to 3d is not simultaneously irradiated onto the pupil. Therefore, image information from different hologram areas 3a to 3d is not visually recognized at the same time.

  The hologram area 3a corresponds to an example of the “first hologram area” in the present invention, and the hologram areas 3b to 3d correspond to an example of the “second hologram area” in the present invention. The light passing through the hologram area 3a corresponds to “first image information light”, and the light passing through the hologram areas 3b to 3d corresponds to “second image information light”.

  Here, the focal point P1a by the hologram region 3a is generated at a position corresponding to the pupil position when the user's eyeball is facing the front, and the focal points P1b to P1d by the hologram regions 3b to 3d are other than the front of the user's eyeball. Is generated at a position corresponding to the pupil position when facing the predetermined direction. When the user's eyeball is facing the front, only the light passing through the hologram region 3a is condensed near the pupil, so that only the image corresponding to the light passing through the hologram region 3a is projected onto the retina. An image corresponding to light through the areas 3b to 3d is not projected on the retina. Therefore, in this case, the user views only the image corresponding to the light that has passed through the hologram area 3a (corresponding to the “first image” in the present invention).

  On the other hand, when the user's eyeball is facing a predetermined direction other than the front, only the light from one area in the hologram areas 3b to 3d is condensed in the vicinity of the pupil. Only the image corresponding to the reflected light is projected on the retina, and the image corresponding to the light via the hologram region 3a and the other regions in the hologram regions 3b to 3d is not projected on the retina. Therefore, in this case, the user visually recognizes only an image (corresponding to the “second image” in the present invention) corresponding to the light that passes through one of the hologram regions 3b to 3d.

  Next, with reference to FIG. 2, the relationship between the line-of-sight direction of the user (corresponding to the driver of the vehicle; the same shall apply hereinafter) and the images displayed by the hologram regions 3a to 3d will be specifically described. To do. FIG. 2 shows spots SPa to SPd where images are displayed by the hologram areas 3a to 3d so as to be superimposed on the scenery in the passenger compartment.

  The spot SPa corresponds to a region in which an image is displayed when the user is turning his / her head and line of sight in the front direction (that is, when the user's eyeball is facing the front). In the spot SPa, an image corresponding to the light passing through the hologram area 3a is displayed. The spot SPb corresponds to a region where an image is displayed when the user turns his / her line of sight toward the rearview mirror 50 (in other words, the rearview mirror) with the head facing the front. In the spot SPb, an image corresponding to the light passing through the hologram area 3b is displayed. The spot SPc corresponds to a region where an image is displayed when the user turns his gaze in the direction of the left door mirror 51 with the head facing the front. In the spot SPc, an image corresponding to the light passing through the hologram area 3c is displayed. The spot SPd corresponds to a region where an image is displayed when the user turns his gaze in the direction of the right door mirror 52 with the head facing the front. In the spot SPd, an image corresponding to the light passing through the hologram area 3d is displayed.

  In addition, when the user turns his / her line of sight in a predetermined direction (front direction, direction of the rearview mirror 50, direction of the left door mirror 51 and direction of the right door mirror 52) as described above, the direction depends on the direction. Further, the optical design of the hologram element 3 is performed so that an image by a predetermined hologram area (any one of the hologram areas 3a to 3d) is appropriately displayed. For example, in consideration of the vehicle type, the driver's seat position, and the like, the arrangement positions, deflection directions, condensing positions, and the like of the hologram regions 3a to 3d in the hologram element 3 are designed.

(Configuration of light source unit)
Next, a schematic configuration of the light source unit 1 according to the first embodiment will be described with reference to FIG. Here, an outline of control performed by the light source unit 1 will be described.

  As shown in FIG. 3, the light source unit 1 mainly includes a display control unit 10 and a database 14. The display control unit 10 corresponds to a video ASIC, a laser driver ASIC, or the like, a process for generating an image to be displayed, a process for controlling a red LD, a green LD, and a blue LD (not shown) so that the image is displayed. I do. Specifically, the display control unit 10 includes a driving assistance image generation unit 11, an image presentation spot determination unit 12, and a guidance image generation unit 13.

  The driving assistance image generation unit 11 generates a driving assistance image for assisting driving. Basically, the driving assistance image is displayed in a direction other than the front direction (any one of the direction of the rearview mirror 50, the direction of the left door mirror 51, and the direction of the right door mirror 52) during driving. The driving assistance image generation unit 11 generates a driving assistance image based on information acquired from the in-vehicle sensor 21, the navigation device 22, the web (not shown), or the like. In this case, the driving assistance image generation unit 11 refers to the database 14 in which various driving assistance images created in advance are stored, and generates driving assistance images to be displayed. The in-vehicle sensor 21 is, for example, a camera that captures the surroundings of the vehicle (front, rear, or side), a distance sensor that detects the distance between the host vehicle and the following vehicle (inter-vehicle distance), It has a microphone that collects sound.

  The image presentation spot determination unit 12 determines a spot for displaying the driving assistance image generated by the driving assistance image generation unit 11. Specifically, the image presentation spot determination unit 12 determines a spot for displaying the driving assistance image from the above spots SPb to SPd. That is, the image presentation spot determination unit 12 determines whether to display the driving assistance image in any one of the direction of the rearview mirror 50, the direction of the left door mirror 51, and the direction of the right door mirror 52. In this case, the image presentation spot determination unit 12 determines the spots SPb to SPd and the vehicle interior positions (specifically, the rearview mirror 50, the left door mirror 51, and the right door mirror 52) based on the content of information displayed by the driving assistance image. The spot for displaying the driving assistance image is determined with reference to the database 14 in which the data indicating the correspondence between the two is stored.

  In a situation where the user should visually recognize the driving assistance image during driving, the guidance image generation unit 13 changes the user's line of sight from the front direction to a direction other than the front direction (the direction of the rearview mirror 50, the direction of the left door mirror 51, and the right side). In order to guide in the direction of any one of the directions of the door mirror 52, a guide image for prompting the user to move the eyeball is generated. Basically, the guidance image is displayed on the spot SPa used when the user has the head and the line of sight directed in the front direction. Specifically, the guide image generation unit 13 generates a guide image for prompting the user to move the line of sight from the spot SPa to the spot (any one of the spots SPb to SPd) determined by the image presentation spot determination unit 12. Generate. In this case, the guide image generation unit 13 generates a guide image according to the direction of the spot arrangement position determined by the image presentation spot determination unit 12 with respect to the spot SPa.

  As described above, the display control unit 10 performs control to display the guide image generated by the guide image generation unit 13 on the spot SPa, and also determines the spots (spots SPb to SPd determined by the image presentation spot determination unit 12). To control the display of the driving assistance image generated by the driving assistance image generator 11.

(Display method)
Next, the display method according to the first embodiment performed by the display control unit 10 in the light source unit 1 will be specifically described.

  In the present embodiment, the display control unit 10 displays information in a wide visual field range using a plurality of spots SPa to SPd. Here, due to the configuration of the hologram regions 3a to 3d as described above, images of the spots SPa to SPd cannot be seen at the same time. That is, unless the pupil is moved and the light of the adjacent spot is projected onto the retina, information other than the spot being watched cannot be viewed. Therefore, the display control unit 10 displays a guide image for prompting the user to move the line of sight. More specifically, the display control unit 10 displays the driving assistance image generated by the driving assistance image generation unit 11 on the spot determined by the image presentation spot determination unit 12 (any one of the spots SPb to SPd). In addition to performing control, the guide image generated by the guide image generation unit 13 (that is, an image for guiding the user's line of sight from the front direction in a direction corresponding to the spot determined by the image presentation spot determination unit 12) Is displayed on the spot SPa. Thereby, the user can see the driving assistance images presented at various positions by moving the line of sight.

  Below, the specific example of the display method which concerns on 1st Example is demonstrated.

  FIG. 4 is a diagram for explaining an example of the display method according to the first embodiment. The display control unit 10 always displays the speed of the following vehicle and the distance between the own vehicle and the following vehicle as a driving assistance image on the spot SPb including the rearview mirror 50, as indicated by an arrow A1 in FIG. . The user (driver) usually drives with the viewpoint fixed in the area including the spot SPa, but when the user consciously moves his / her line of sight toward the rearview mirror 50, the speed and distance between the following vehicles Distance information can be confirmed. The speed and the inter-vehicle distance of the succeeding vehicle are obtained based on the detection value of the distance sensor that the in-vehicle sensor 21 has.

  Here, it may be possible to display the speed of the following vehicle and the inter-vehicle distance on the spot SPa. However, if there is no trouble in driving, such a trivial information is always shown to the user. Operation may be hindered. On the other hand, in situations where there is a possibility of an accident such as when the following vehicle is rapidly approaching the vehicle and may cause a rear-end collision, information on such a following vehicle should be actively presented. Is desirable. However, basically, the user does not notice the approach of the following vehicle without looking at the rearview mirror 50.

  Therefore, in the present embodiment, the display control unit 10 moves to the arrow in FIG. 4 in a situation that may lead to an accident such as a case where the following vehicle is rapidly approaching the host vehicle and may collide. As indicated by A2, a guide image for prompting the movement of the line of sight from the spot SPa to the spot SPb is displayed on the spot SPa. That is, a guidance image for guiding the user's line of sight from the front direction toward the rearview mirror 50 is displayed. Specifically, the display control unit 10 displays the guidance image at a position in the spot SPa that is away from the central region and according to the direction of the arrangement position of the spot SPb with respect to the spot SPa. Specifically, the display controller 10 causes the upper right end of the spot SPa to blink.

  For example, when the speed of the succeeding vehicle is equal to or higher than a predetermined speed and the distance between the own vehicle and the succeeding vehicle is equal to or less than the predetermined distance, the display control unit 10 quickly approaches the own vehicle. The guidance image as described above is displayed. In this case, the display control unit 10 displays not only the speed and inter-vehicle distance of the following vehicle but also characters such as “possible rear-end collision” on the spot SPb as the driving assistance image.

  Since the size of the following vehicle varies depending on the type of vehicle, it is difficult to instantly determine the speed of the following vehicle, the distance between vehicles, the possibility of a rear-end collision, etc. just by looking at the vehicle reflected in the mirror. By displaying the image and the guidance image, the user can appropriately grasp the speed of the subsequent vehicle, the inter-vehicle distance, the possibility of a rear-end collision, and the like. Further, even when the system cannot recognize the vehicle type such as whether the rear vehicle is an emergency vehicle or a truck, if data indicating that the inter-vehicle distance is rapidly shortened is obtained, the line of sight is viewed as if the rearview mirror 50 is viewed. By guiding the user, the user can determine the vehicle type.

  4 shows an example in which a guide image is displayed at the upper right end of the spot SPa in order to guide the line of sight to the spot SPb including the rearview mirror 50, the spot SPc including the left door mirror 51 is displayed. In order to guide the line of sight, a guide image may be displayed at a position far from the center area in the spot SPa and at the lower left end of the spot SPa. Further, when the line of sight is guided to the spot SPd including the right door mirror 52, a guidance image may be displayed at a position far from the center region in the spot SPa and at the lower right end of the spot SPa. .

  In the above, an example in which a guidance image is displayed in a situation that may lead to an accident such as a case in which the following vehicle is rapidly approaching the host vehicle and may be collided is shown. In the example, the display control unit 10 can display a guidance image so as to guide the line of sight even when the host vehicle may come into contact with a pedestrian or a bicycle. In this example, the display control unit 10 determines whether or not the own vehicle may come into contact with a pedestrian or a bicycle from a captured image of a camera included in the in-vehicle sensor 21, and the own vehicle is a pedestrian or a bicycle. When it is determined that there is a possibility of contact with the pedestrian, a guidance image as described above is displayed to guide the line of sight to a spot (any one of the spots SPb to SPd) corresponding to the direction in which the pedestrian or the bicycle exists. Let In this case, the display control unit 10 displays characters such as “possibility of contact with pedestrians” and “possibility of contact with bicycles” as driving assistance images.

  In yet another example, the display control unit 10 can display a guidance image so as to guide the line of sight even when an emergency vehicle is approaching from behind.

  FIG. 5 is a flowchart showing a display method performed when an emergency vehicle is approaching from behind. This flow is repeatedly executed by the display control unit 10.

  First, in step S <b> 101, the display control unit 10 analyzes sounds around the host vehicle collected by a microphone included in the in-vehicle sensor 21. Then, the process proceeds to step S102. In step S102, the display control unit 10 determines whether or not the siren of the emergency vehicle has been collected based on the analysis result of step S101. If the emergency vehicle siren is collected (step S102: Yes), the process proceeds to step S103. If the emergency vehicle siren is not collected (step S102: No), the process returns to step S101.

  In step S <b> 103, the display control unit 10 analyzes the direction in which the emergency vehicle exists based on an image taken by a camera included in the in-vehicle sensor 21. Then, the process proceeds to step S104. In step S104, the display control unit 10 determines whether or not an emergency vehicle exists behind, based on the analysis result of step S103. When the emergency vehicle exists behind (step S104: Yes), the process proceeds to step S105. On the other hand, when the emergency vehicle does not exist behind (step S104: No), that is, when the emergency vehicle exists ahead, the process returns to step S101.

  In step S105, the display control unit 10 determines whether or not the emergency vehicle is traveling in the lane on the left side of the lane in which the host vehicle is traveling based on the image captured by the camera. If the emergency vehicle is traveling in the left lane of the host vehicle (step S105: Yes), the process proceeds to step S106. In this case, the display control unit 10 displays characters such as “emergency vehicle” as the driving assistance image on the spot SPc including the left door mirror 51 (step S106), and displays the guidance image on the lower left in the front spot SPa. Is displayed blinking (step S107). Then, the process proceeds to step S114.

  On the other hand, when the emergency vehicle is not traveling in the left lane of the host vehicle (step S105: No), the process proceeds to step S108. In step S108, the display control unit 10 determines whether or not the emergency vehicle is traveling in the same lane as the lane in which the host vehicle is traveling, based on the image captured by the camera. If the emergency vehicle is traveling in the same lane as the host vehicle (step S108: Yes), the process proceeds to step S109. In this case, the display control unit 10 displays characters such as “emergency vehicle” as the driving assistance image on the spot SPb including the rearview mirror 50 (step S109), and displays the guidance image in the upper right corner of the front spot SPa. Is displayed blinking (step S110). Then, the process proceeds to step S114.

  On the other hand, when the emergency vehicle is not traveling in the same lane as the own vehicle (step S108: No), the process proceeds to step S111. In step S111, the display control unit 10 determines whether or not the emergency vehicle is traveling in the lane on the right side of the lane in which the host vehicle is traveling based on the image captured by the camera. If the emergency vehicle is traveling in the right lane of the host vehicle (step S111: Yes), the process proceeds to step S112. In this case, the display control unit 10 displays characters such as “emergency vehicle” as the driving assistance image on the spot SPd including the right door mirror 52 (step S112), and displays the guidance image as the lower right corner of the front spot SPa. Is displayed blinking (step S113). Then, the process proceeds to step S114. If the emergency vehicle is not traveling in the right lane of the host vehicle (step S111: No), the process returns to step S101.

  The driving assistance image described above is generated by the driving assistance image generation unit 11, the spot to be displayed is determined by the image presentation spot determination unit 12, and the above-described guidance image is generated by the guidance image generation unit 13. Is done.

  In step S114, the display control unit 10 determines whether or not an emergency vehicle has passed based on, for example, an image taken by a camera. When the emergency vehicle has passed (step S114: Yes), the process proceeds to step S115. In this case, the display control unit 10 erases the display in each spot (step S115). That is, the display of the driving assistance image and the guidance image is deleted. Then, the process ends. On the other hand, when the emergency vehicle has not passed (step S114: No), the process returns to step S103. In this case, the display control unit 10 performs the processes after step S103 described above again.

  In FIGS. 4 and 5, in the exact relative direction of the spot for which the line of sight is to be guided with respect to the spot SPa (specifically, any one of the upper right, lower left and lower right directions of the spot SPa), Although the example which displays a guidance image was shown, it is not limited to this. In another example, instead of such an accurate relative direction, the display control unit 10 generates a guidance image in a pseudo direction (that is, an approximate direction) of the spot where the line of sight is to be guided with respect to the spot SPa. Can be displayed.

  FIG. 6 is a diagram for explaining another example of the display method according to the first embodiment. As indicated by an arrow A3 in FIG. 6, when guiding the line of sight to the spot SPb including the rearview mirror 50, the display control unit 10 is located at a position away from the central region in the spot SPa and the spot SPa. A guide image is displayed on the upper edge of the screen. Further, when the display control unit 10 guides the line of sight to the spot SPc including the left door mirror 51, the display control unit 10 displays a guide image at the left end of the spot SPa at a position away from the center region in the spot SPa. In the case where the line of sight is guided to the spot SPd including the right door mirror 52, the guidance image is displayed at a position away from the central area in the spot SPa and at the right end of the spot SPa. In this way, it is possible to simplify the display position for guiding the line of sight.

  Further, in FIGS. 4 and 5, an example in which the end portion of the spot SPa is blinked and displayed as the guidance image display is not limited to this. In another example, the display control unit 10 causes the end portion of the spot SPa to be lit and displayed as the guidance image. In yet another example, the display control unit 10 displays an icon such as a vehicle or a store at the end of the spot SPa as a guide image display. In yet another example, the display control unit 10 displays the icons of the rearview mirror 50, the left door mirror 51, and the right door mirror 52 at the end of the spot SPa as a guide image display. In yet another example, the display control unit 10 displays a gradation at the end of the spot SPa as a guide image display. In yet another example, the display control unit 10 displays an animation such that the image moves from the central region of the spot SPa toward the end as the guide image display. In yet another example, the display control unit 10 displays an image such that the circle gradually decreases toward the end of the spot SPa as the guidance image.

(Operations and effects of the first embodiment)
As described above, in the first example, an image is displayed using a plurality of spots, and the spot to be watched is changed by prompting the movement of the line of sight by the guidance image. Thus, by moving the line of sight, information can be appropriately displayed outside the peripheral vision during driving. In other words, it is possible to appropriately present information in a wider range than the peripheral vision by performing the gaze movement guidance to the outer region of the peripheral vision during driving.

  Further, in the first embodiment, the information is displayed when the line of sight is moved to the front where the user is gazing during driving and the positions (back mirror 50, left door mirror 51, and right door mirror 52) that are occasionally gazing during driving. As shown, hologram regions 3 a to 3 d are formed in the hologram element 3. Accordingly, since the rearview mirror 50, the left door mirror 51, and the right door mirror 52 can be used as targets when moving the line of sight, it is possible to instantaneously move the line of sight and view information. That is, in the first embodiment, the spots SPb to SPd that move the line of sight are associated with limited specific things in the vehicle such as the rearview mirror 50, the left door mirror 51, and the right door mirror 52. The position where the line of sight should be moved during display can be easily determined.

  Furthermore, according to the first embodiment, information that is not so important during driving is not visible in the area that is being watched during normal driving, and is only visible when the user consciously moves the line of sight. Therefore, it is possible to appropriately ensure driving safety.

[Second Embodiment]
Next, a second embodiment will be described. In the second embodiment, an image relating to a predetermined point on the map that is not directly related to driving or navigation of the vehicle is displayed as a driving assistance image, and an image included in such a driving assistance image is displayed as a guidance image. This is different from the first embodiment. Specifically, in the second embodiment, the display control unit 10 displays navigation information such as a route to the destination in a spot included in an area to be watched during driving, and outside the peripheral vision during driving. Information on stores on the left and right of the road (restaurants, etc.) and information on toilet facilities are displayed at the spot located at.

  In the following, description of the same configuration, processing, and control as in the first embodiment will be omitted as appropriate. In addition, configurations, processes, and controls not specifically described here are the same as those in the first embodiment.

  FIG. 7 shows an example of spots on which images are displayed in the second embodiment. As shown in FIG. 7, in the second embodiment, images are displayed on the spots SPa, SPe, and SPf. The spot SPa is the same as that of the first embodiment, and corresponds to a region where an image is displayed when the user is turning his / her head and line of sight in the front direction. The spot SPe corresponds to an area in which an image is displayed when the user turns his / her line of sight toward the left with the head directed toward the front. The spot SPf corresponds to an area in which an image is displayed when the user turns his gaze rightward with the head facing the front. Basically, information existing on the left front side of the vehicle is displayed in the spot SPe, and information existing on the right front side of the vehicle is displayed in the spot SPf. For example, the spot SPe displays information related to facilities existing on the left side of the road in the field of view of the host vehicle, and the spot SPf displays information related to facilities existing on the right side of the road in the field of view of the host vehicle.

  As described above, when the user turns his / her line of sight in a predetermined direction (front direction, left direction, or right direction), an image of a predetermined hologram area corresponding to the direction is appropriately displayed. As described above, the optical design of the hologram element 3 is performed. The second embodiment and the first embodiment are basically different in the configuration of the hologram region in the hologram element 3 because the number and positions of the spots to be used are different.

  FIG. 8 is a diagram for explaining an example of the display method according to the second embodiment. As indicated by an arrow B1 in FIG. 8, the display control unit 10 displays an arrow indicating route guidance to the destination at a spot SPa included in an area to be watched during driving. The display control unit 10 displays an arrow indicating route guidance based on the route obtained by the navigation device 22. Further, the display control unit 10 performs control so that an arrow indicating route guidance is displayed so as to be superimposed on an actual road corresponding to the arrow. That is, the display control unit 10 performs AR display (AR: Augmented Reality). For example, the display control unit 10 realizes such AR display by analyzing an image captured by a camera included in the in-vehicle sensor 21 and extracting a road corresponding to a route to the destination.

  Further, the display control unit 10 causes the spot SPe located on the left side of the spot SPa to display information relating to the toilet facility existing on the left side in front of the vehicle as a driving assistance image. Specifically, the display control unit 10 displays an icon (WC) indicating a toilet facility on the spot SPe (see arrow B2 in FIG. 8) and an arrow indicating a route to the toilet facility (FIG. 8). (See arrow B3 in FIG. 8). The display control unit 10 performs such display based on map information stored in a database in the navigation device 22. In addition, the display control unit 10 also displays an arrow indicating the route to the toilet facility in the AR manner, similarly to the arrow indicating the route guidance to the destination (see arrow B1). Specifically, the display control unit 10 causes an arrow indicating a route to the toilet facility to be superimposed and displayed on a road that should actually bend to go to the toilet facility. When the toilet facility faces the road, the display control unit 10 displays the toilet facility icon superimposed on the building facility itself.

  In addition, the display control unit 10 displays an icon (WC) indicating a toilet facility in a spot SPa included in an area to be watched during driving as a guidance image (see arrow B4 in FIG. 8). Specifically, the display control unit 10 displays an icon indicating a toilet facility at a position away from the center area in the spot SPa and according to the direction of the arrangement position of the spot SPe with respect to the spot SPa. Specifically, the display control unit 10 displays an icon indicating a toilet facility at the left end of the spot SPa.

  Here, large facilities that can be reached by car such as a gas station have large signs on the side of the road, and it is easy to specify the location, but facilities such as toilets and restaurants tend not to have large signs Yes, it may be difficult to understand where it is. On the other hand, if you suddenly want to go to a restaurant, toilet, etc. on the way, it is cumbersome to search for a stop point and make a route change setting. For this reason, in the second embodiment, as described above, information on a small stop point such as a restaurant or a toilet facility is displayed as a driving assistance image on the spots SPe and SPf located outside the peripheral vision during driving. As a result, it is possible to appropriately present and provide information on stop points that are difficult to understand around the road that is running. Further, in the second embodiment, only the navigation information is displayed on the spot SPa to be watched during driving, and the information on the stop-off point is not displayed. Therefore, the information displayed on the spot SPa to be watched during driving is excessive. Therefore, it is possible to ensure driving safety.

[Modification]
Below, the modification suitable for said Example is demonstrated. It should be noted that the following modifications can be applied to the above-described embodiments in any combination.

(Modification 1)
In the above-described embodiment, when the user turns his / her line of sight in a predetermined direction, the optical design of the hologram element 3 is performed so that an image through a predetermined hologram area corresponding to the direction is appropriately displayed. (See FIG. 1 and FIG. 2). That is, in the above-described embodiment, a plurality of hologram regions are formed in the hologram element 3 in accordance with the positions of a plurality of spots for displaying images. In another example, a plurality of hologram regions that can appropriately display images at various spots are formed on the hologram element, and a hologram region to be used is selected from the plurality of hologram regions according to the spots used for displaying images. You can choose.

  FIG. 9 is a diagram illustrating a schematic configuration of a hologram element 3x according to the first modification. FIG. 9 shows a view of the hologram element 3x observed from a direction along the light traveling direction.

  As shown in FIG. 9, a plurality of hologram areas 3xa are regularly arranged in the hologram element 3x according to the first modification. Specifically, the hologram element 3x is formed with a number of hologram regions 3xa that is at least larger than the number of spots used for image display in an arbitrary situation. When such a hologram element 3x is used, a hologram area 3xa to be used is selected according to a spot for displaying an image, and control is performed so that an image by light through the selected hologram area 3xa is displayed. It can be performed. For example, as in the first embodiment, in the case where an area to be watched during driving, an area including each of the rearview mirror 50, the left door mirror 51, and the right door mirror 52 is used as a spot (see FIG. 2), FIG. The hatched hologram area 3xa can be selected.

  Moreover, in the modification 1, when the direction of a user's head changes, the hologram area 3xa to be used can be changed. For example, when the head orientation changes, a new hologram region 3xa obtained by shifting the hologram region 3xa used before the head orientation changes in a direction corresponding to the head orientation can be used. By doing so, even when the head direction changes, an image corresponding to the gaze direction can be appropriately displayed as in the case where the line-of-sight direction changes. The change in head orientation can be detected by, for example, an acceleration sensor.

(Modification 2)
In the above-described embodiment, an example of performing control for displaying a guidance image to guide the user's line of sight has been shown, but in another example, control for generating sound to guide the user's line of sight instead. It can be performed. Specifically, in this example, it is possible to perform control to generate sound from the direction of the spot where the line of sight should be moved.

  In yet another example, control for vibrating the steering wheel of the vehicle can be performed to guide the user's line of sight. Specifically, in this example, it is possible to perform control to vibrate the position of the handle in accordance with the direction of the spot where the line of sight should be moved.

(Modification 3)
In the embodiment described above, the hologram element 3 is configured as a transmission type (see FIG. 1A), but the hologram element may be configured as a reflection type.

  In the above-described embodiment, the hologram element 3 and the beam splitter 4 are configured separately (see FIG. 1A), but the hologram element 3 and the beam splitter 4 may be configured integrally. . Specifically, one hologram element that has the basic functions of both the hologram element 3 and the beam splitter 4 can be used.

  In the above-described embodiment, the display device 100 is configured as a scanning display (see FIG. 1A), but the display device may be configured as a projection display. When the display device is configured as a projection display, a point light source or a predetermined transmission object (for example, LCD) may be used instead of the light source unit 1 and the scanning mechanism 2.

(Modification 4)
Although the example which applies this invention to a head mounted display (HMD) was shown above, application of this invention is not limited to this. The present invention can be applied to various display devices other than the HMD. For example, it can be applied to a visually impaired person auxiliary display and a driving assistance device.

  As described above, the embodiments are not limited to the above-described embodiments, and can be appropriately changed without departing from the gist or concept of the invention that can be read from the claims and the entire specification.

  The present invention can be used for a display device such as a head-mounted display.

DESCRIPTION OF SYMBOLS 1 Laser light source 2 Scan mechanism 3 Hologram element 3a-3d Hologram area | region 4 Beam splitter 10 Display control part 11 Driving assistance image generation part 12 Image presentation spot determination part 13 Guidance image generation part 14 Database 50 Back mirror 51 Left door mirror 52 Right door mirror 100 Display device

Claims (19)

A display device comprising: an emitting unit that emits image information light; and an optical element on which the image information light is incident. The display device visually recognizes an image by irradiating a user's retina with the image information light via the optical element. Because
The image information light includes first image information light and second image information light,
The first image information light and the second image information light are condensed at different positions so that either one is irradiated on the retina according to the movement of the eyeball of the user,
Guidance for urging the user to move the eyeball so that the second image information light is irradiated onto the retina from a state where the first image information light is irradiated onto the retina A display device comprising control means for performing control for notifying information. The control means performs control to display a guide image as control to notify the guide information,
The guide image is an image corresponding to the second image that is included in the first image that the first image information light projects onto the retina, and the second image information light projects onto the retina. The display device according to claim 1.   The control means causes the guide image to be displayed at a position in the first image that is distant from a central region and according to the direction of the arrangement position of the second image with respect to the first image. The display device according to claim 2.   The display device according to claim 2, wherein the control unit displays an image included in the second image as the guide image.   The display device according to claim 1, wherein the control unit performs control to generate a sound according to information to be displayed by the second image information light as control for informing the guidance information.   2. The display device according to claim 1, wherein the control unit performs control to vibrate a vehicle handle in accordance with information to be displayed by the second image information light as control for informing the guidance information. .   The display according to any one of claims 1 to 6, wherein the control means performs control to notify the guidance information when predetermined information is to be displayed by the second image information light. apparatus.   The display device according to claim 7, wherein the predetermined information is information relating to an emergency vehicle.   The display device according to claim 7, wherein the predetermined information is information related to a subject that is likely to cause a traffic accident.   The display device according to claim 7, wherein the predetermined information is information relating to a predetermined point on the map that is not directly related to driving or navigation of the vehicle. The optical element includes a first region and a second region,
11. The first image information light is light that passes through the first region, and the second image information light is light that passes through the second region. The display device according to one item. The first region is formed in the optical element such that the first image information light is projected onto the retina when the user's eyeball is facing the front.
The second region is formed in the optical element such that the second image information light is projected onto the retina when the user's eyeball faces a predetermined direction other than the front. The display device according to claim 11.   The second region is formed in the optical element so that the second image information light is projected onto the retina when the user turns his / her eyeball toward a room mirror or a door mirror while driving the vehicle. The display device according to claim 12, wherein the display device is a display device.   The second area projects the second image information light onto the retina when the user points the eyeball in a direction corresponding to a direction in which an object of information to be displayed by the second image information light exists. The display device according to claim 12, wherein the display device is formed on the optical element. A display device comprising: an emitting unit that emits image information light; and an optical element on which the image information light is incident. The display device visually recognizes an image by irradiating a user's retina with the image information light via the optical element. A display method executed by
The image information light includes first image information light and second image information light,
The first image information light and the second image information light are condensed at different positions so that either one is irradiated on the retina according to the movement of the eyeball of the user,
Guidance for urging the user to move the eyeball so that the second image information light is irradiated onto the retina from a state where the first image information light is irradiated onto the retina A display method comprising a control step of performing control for notifying information. The image processing apparatus includes an emission unit that emits image information light, an optical element that receives the image information light, and a computer, and causes the user's retina to irradiate the image information light through the optical element so that the user can visually recognize the image. A display program executed by a display device,
The image information light includes first image information light and second image information light,
The first image information light and the second image information light are condensed at different positions so that either one is irradiated on the retina according to the movement of the eyeball of the user,
Guidance for urging the user to move the eyeball so that the second image information light is irradiated onto the retina from a state where the first image information light is irradiated onto the retina A display program that causes the computer to function as a control unit that performs control for informing information.   A recording medium on which the display program according to claim 16 is recorded. A display device comprising: an emitting unit that emits image information light; and an optical element on which the image information light is incident. The display device visually recognizes an image by irradiating a user's retina with the image information light via the optical element. Because
The image information light includes first image information light and second image information light,
The first image information light and the second image information light are condensed at different positions so that one of the first image information light and the second image information light is irradiated to the main region of the retina according to the movement of the user's eyeball. And
Guidance for urging the user to move the eyeball so that the second image information light is irradiated onto the retina from a state where the first image information light is irradiated onto the retina A display device comprising control means for performing control for notifying information. A display device comprising: an emitting unit that emits image information light; and an optical element on which the image information light is incident. The display device visually recognizes an image by irradiating a user's retina with the image information light via the optical element. Because
The image information light includes first image information light and second image information light,
The first image information light and the second image information light are condensed at different positions according to the movement of the user's eyeball,
Guidance for urging the user to move the eyeball so that the second image information light is irradiated onto the retina from a state where the first image information light is irradiated onto the retina A display device comprising control means for performing control for notifying information.

Images