JP7151397B2 - Display device, display method, head-up display and wearable device - Google Patents

Description

The present invention relates to display devices, display methods, head-up displays, and wearable devices.

Patent Documents 1 and 2, for example, disclose wakefulness maintenance devices for maintaining the wakefulness of a user such as a driver of a vehicle by giving a visual stimulus to the user.

JP 2017-156869 A JP 2017-174091 A

However, although the presentation of the visual stimulus disclosed in Patent Documents 1 and 2 is accompanied by movement of the user's line of sight, the visual stimulus itself does not provide information to the user, so the user does not need to pay attention to it. Depending on the application and the content displayed by the display device, it is desirable to be able to maintain alertness and recover attention by presenting visual stimuli without moving the line of sight. For example, when a display device is used to support the driving of a vehicle, it is desirable to avoid movement of the driver's line of sight as much as possible when maintaining the wakefulness of the driver who is the user of the display device and recovering the attention. ing. In other words, the wakefulness maintenance devices disclosed in Patent Documents 1 and 2 have room for improvement in terms of minimizing the movement of the user's line of sight when visually stimulating.

The present invention has been made based on the above, and provides a display device, a display method, a head-up display, and a wearable device that maintain an alert state and recover attention without inducing a user's line of sight movement. With the goal.

The display device of this embodiment is a display device that displays a first image that provides information to a user in a display area, and displays a second image that does not include information to be provided to the user in the display area. a control unit, wherein the control unit displays a plurality of the second images in the display area, the display timings of the plurality of second images overlap in terms of time, and the movement of the user's line of sight is controlled; a line-of-sight movement detection unit that detects the movement of the user's line of sight; The main stimulus image is displayed in the display area when the user's eye movement is detected by the eye movement detection unit after the stimulus image is displayed .

A display method of the present embodiment is a display method for displaying a first image that provides information to a user in a display area, and displays a second image that does not include information to be provided to the user in the display area. a control step, wherein the control step causes a plurality of the second images to be displayed in the display area, the display timings of the plurality of second images overlap in terms of time, and movement of the user's line of sight is controlled; a line-of-sight movement detection step, wherein the plurality of second images includes a dummy stimulus image for inducing line-of-sight movement of the user and a main stimulus image for prompting recovery of attention of the user; After the stimulus image is displayed, the main stimulus image is displayed in the display area when the user's eye movement is detected by the eye movement detection step .

The head-up display of this embodiment has the above-described display device, and is characterized by using the windshield or combiner of the vehicle as the display area.

The wearable device of this embodiment has the display device described above, and is characterized by being worn on the user's body for use.

Advantageous Effects of Invention According to the present invention, it is possible to provide a display device, a display method, a head-up display, and a wearable device that maintain an arousal state and recover attention without inducing movement of the user's line of sight.

FIG. 10 is a front view showing an example of a virtual image displayed in the display area superimposed on the scenery in front of the own vehicle seen by the user through the windshield in the vehicle HUD device. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a partially broken schematic side view schematically showing the configuration of a vehicle equipped with a vehicle HUD device; It is a block diagram which shows typically the internal structure of the optical system of the HUD apparatus for vehicles. 2 is a block diagram showing the internal configuration of a control system of the vehicle HUD device; FIG. 1 is a block diagram showing a schematic configuration of a vehicle HUD device and its peripheral devices; FIG. FIG. 4 is a diagram showing an example of a display image displayed in a display area and a plurality of visual stimulus images; FIG. 10 is a first diagram showing an example of a case where a plurality of visual stimulus images are displayed so as to sandwich the user's visual field center in the display area; FIG. 10 is a second diagram showing an example of a case where a plurality of visual stimulus images are displayed so as to sandwich the user's visual field center in the display area; FIG. 11 is a third diagram showing an example of a case where a plurality of visual stimulus images are displayed so as to sandwich the user's visual field center in the display area; FIG. 10 is a diagram showing an example of a case where a plurality of visual stimulus images are displayed outside the display range of the display image in the display area; FIG. 10 is a diagram showing an example of displaying a plurality of visual stimulus images while changing at least one of brightness, color, and size; FIG. 4 is a block diagram showing an example of the configuration of an optical system and a control system for determining integrated intensity of a plurality of visual stimulus images within a predetermined time; 4 is a timing chart showing an example of display timings of a plurality of visual stimulus images; 4 is a first flow chart showing an example of trigger (timing) conditions (standards) for displaying a plurality of visual stimulus images. FIG. 10 is a second flowchart showing an example of trigger (timing) conditions (standards) for displaying a plurality of visual stimulus images; FIG. FIG. 11 is a third flowchart showing an example of trigger (timing) conditions (standards) for displaying a plurality of visual stimulus images; FIG. FIG. 14 is a fourth flowchart showing an example of trigger (timing) conditions (standards) for displaying a plurality of visual stimulus images; FIG. FIG. 3 is a partially broken schematic side view corresponding to FIG. 2 showing a vehicle HUD device having a selector switch; FIG. 3 is a partially broken schematic side view corresponding to FIG. 2 showing a vehicle HUD device having a field-of-view detection sensor; FIG. 10 is a first diagram showing an example of displaying a plurality of visual stimulus images based on the field of view of the driver; FIG. 11 is a second diagram showing an example of displaying a plurality of visual stimulus images based on the visual field of the driver; FIG. 11 is a third diagram showing an example of displaying a plurality of visual stimulus images based on the visual field of the driver; FIG. 3 is a partially broken schematic side view corresponding to FIG. 2 showing a vehicle HUD device having a visual stimulus presentation unit; FIG. 10 is a diagram showing an example of a case in which a plurality of visual stimulus images through the display area are displayed in a superimposed region of the HUD display range and the driver's peripheral visual field; FIG. 4 is a diagram showing an example of a case where a visual stimulus image through a display area and a visual stimulus through a visual stimulus presentation unit are presented in the driver's peripheral visual field; FIG. 10 is a diagram showing an example of a case in which a plurality of visual stimuli are presented to the driver's peripheral visual field via a visual stimulus presentation unit; 4 is a block diagram showing an example of a linking unit between a display control unit and a visual stimulus presentation unit; FIG. FIG. 4 is a diagram showing an example of display positions of a main stimulus image and a dummy stimulus image; FIG. 10 is a diagram showing an example of the flow from display of a dummy stimulus image to display of a main stimulus image; 4 is a flow chart showing an example of an operation using a main stimulus image and a dummy stimulus image; FIG. 4 is a block diagram showing an example of the configuration of an optical system and a control system for changing the brightness of a plurality of visual stimulus images based on the brightness of the surrounding environment;

An example of the overall configuration of a vehicle HUD device to which a display device is applied will be described with reference to FIGS. 1 to 5. FIG. Here, the display device 1 is applied to a vehicle head-up display (HUD: Head Up Display) device that displays various images (a display image and a visual stimulus image to be described later) on the windshield or combiner of the vehicle. .

FIG. 1 is a front view showing an example of a virtual image displayed in the display area superimposed on the scenery in front of a vehicle 301 viewed from a driver (user) 300 through a windshield 302 in the HUD device 1 for a vehicle. FIG. 2 is a partially cutaway schematic side view schematically showing the configuration of a vehicle equipped with the vehicle HUD device 1. As shown in FIG. FIG. 3 is a block diagram schematically showing the internal configuration of the optical system of the vehicle HUD device 1. As shown in FIG.

As shown in FIG. 2, the vehicle HUD device 1 is installed, for example, in the dashboard of an automobile (self-vehicle 301) that is a moving body (running body). Projection light L, which is image light emitted from the vehicle HUD device 1 in the dashboard, is reflected by a windshield 302 as a light transmitting member and travels toward a driver 300 who is a viewer. As a result, the driver 300 can visually recognize the HUD display image (display image and visual stimulus image) such as a navigation image, which will be described later, as the virtual image G. FIG. A combiner as a light transmission member may be installed on the inner wall surface of the windshield 302 so that the projected light L reflected by the combiner may be used to make the driver visually recognize the virtual image. In addition, the “display image” in this specification is not limited to a virtual image, and may be an image formed directly from an image to be visually recognized. That is, the "display image" in this specification may be any image that can be visually recognized by the driver.

A forward photographing camera 110 for photographing the display information of the vehicle HUD device 1 and its background (scenery through the windshield 302), and the ambient light around the virtual image G (HUD display image) are provided on the upper part of the vehicle inner side of the windshield 302. An ambient light sensor (brightness detector) 150 is provided to detect brightness and chromaticity. The brightness here includes luminance, illuminance, luminous intensity, total luminous flux, and values calculated based on the measurement results thereof.

In the vehicle HUD device 1, the optical system and the like of the vehicle HUD device 1 are configured so that the distance from the driver 300 to the virtual image G is 4 m or more. In a conventional general vehicle HUD device, the distance from the driver 300 to the virtual image G was about 2 m. The driver 300 normally drives the vehicle while paying attention to the preceding vehicle several tens of meters ahead from the point at infinity in front of the vehicle. When the driver 300, who is focusing on such a distant place, tries to visually recognize the virtual image G 2m ahead, the focal length is greatly different, so it is necessary to move the crystalline lens of the eye greatly. As a result, it takes a long time to adjust the focus until the virtual image G is brought into focus, and it takes a long time to recognize the content of the virtual image G, and the eyes of the driver 300 tend to get tired easily. In addition, it is difficult for the driver to grasp the contents of the virtual image G, and there is room for improvement from the viewpoint of appropriately providing information to the driver through the virtual image G.

If the distance from the driver 300 to the virtual image G is 4 m or more, the amount of movement of the crystalline lens of the eyeball is reduced compared to the conventional one, and the focus adjustment time to the virtual image G is shortened so that the contents of the virtual image G can be recognized early. Also, eye fatigue of the driver 300 can be reduced. Furthermore, it becomes easier for the driver 300 to grasp the content of the virtual image G, and it becomes easier to appropriately provide information to the driver 300 through the virtual image G. If the distance from the driver 300 to the virtual image G is 4 m or more, the driver can focus on the virtual image G without causing a convergence movement of the eyeballs. Therefore, it is suppressed that the effect of using motion parallax to perceive a sense of distance (change in perceived distance) and a sense of depth (difference in perceived distance) is weakened by the convergence movement of the eyeball. Therefore, the driver's information perception effect using the sense of distance and depth of the image can be effectively exhibited.

As shown in FIG. 3, the vehicle HUD device 1 includes, in an optical system 230, red, green, and blue laser light sources 201R, 201G, and 201B, and collimator lenses 202, 203, and 204 provided for each laser light source. , and two dichroic mirrors 205 and 206 . The vehicle HUD device 1 further includes a light quantity adjusting unit 207, a light scanning device 208 as light scanning means, a free-form surface mirror 209, a microlens array 210 as a light divergence member, and a projection mirror 211 as a light reflection member. It consists of The light source unit 220 is configured by unitizing laser light sources 201R, 201G, 201B, collimator lenses 202, 203, 204, and dichroic mirrors 205, 206 with an optical housing.

LDs (semiconductor laser elements) are used as the laser light sources 201R, 201G, and 201B. The wavelength of the light flux emitted from the red laser light source 201R is, for example, 640 nm, the wavelength of the light flux emitted from the green laser light source 201G is, for example, 530 nm, and the wavelength of the light flux emitted from the blue laser light source 201B is, for example, 445 nm. .

The vehicle HUD device 1 projects an intermediate image formed on the microlens array 210 onto the windshield 302 of the own vehicle 301 using the free-form surface mirror 209 , thereby displaying an enlarged image of the intermediate image to the driver 300 . is visually recognized as a virtual image G. Color laser beams emitted from laser light sources 201R, 201G, and 201B are collimated by collimator lenses 202, 203, and 204, respectively, and combined by two dichroic mirrors 205 and 206, respectively. The combined laser light is two-dimensionally scanned by the mirror of the optical scanning device 208 after the light amount is adjusted by the light amount adjusting unit 207 . Scanning light L′ that has been two-dimensionally scanned by the optical scanning device 208 is reflected by the free-form surface mirror 209 and corrected for distortion, then converged on the microlens array 210 to draw an intermediate image.

Here, the microlens array 210 is used as a light diverging member that diverges and emits light fluxes for each pixel (one point of the intermediate image) of the intermediate image, but other light diverging members may be used. As a method of forming the intermediate image G', a method using a liquid crystal display (LCD) or a vacuum fluorescent display (VFD) may be used. However, the laser scanning method is preferable for displaying a large virtual image G with high brightness.

In addition, in a method using a liquid crystal display (LCD) or a vacuum fluorescent display (VFD), a non-image portion in the display area where the virtual image G is displayed is slightly irradiated with light, and it is necessary to completely block the light. is difficult. Therefore, there is a demerit that the visibility of the scenery ahead of the own vehicle 301 in the non-image portion is lowered. On the other hand, according to the laser scanning method, for the non-image portion in the display area of the virtual image G, the laser light sources 201R, 201G, and 201B are turned off to completely prevent the non-image portion from being irradiated with light. can be shut off. Therefore, it is possible to avoid a situation in which the visibility of the scenery ahead of the vehicle 301 in the non-image portion is lowered by the light emitted from the vehicle HUD device 1, and there is an advantage that the visibility of the scenery ahead can be kept good.

Furthermore, when the degree of warning is strengthened by stepwise increasing the brightness of the warning image for warning the driver, etc., the warning image displayed in the display area 700 (FIG. 1) Display control is required to gradually increase only the brightness of the image. The laser scanning method is also suitable for performing display control to partially increase the luminance of a partial image in the display area 700 as described above. In a method using a liquid crystal display (LCD), a vacuum fluorescent display (VFD), or the like, it is difficult to change the brightness of only a specific portion of the display area 700. The brightness of the image also increases. Therefore, it is difficult to increase the brightness difference between the warning image and the other images, and it is difficult to increase the level of warning by increasing the brightness of the warning image step by step.

The optical scanning device 208 tilts the mirror in the main scanning direction and the sub-scanning direction using a known actuator drive system such as MEMS (Micro Electro Mechanical Systems), and two-dimensionally scans (raster scans) the laser beam incident on the mirror. . The drive control of the mirrors is performed in synchronization with the light emission timings of the laser light sources 201R, 201G, and 201B. The optical scanning device 208 may be composed of, for example, a mirror system composed of two mirrors that oscillate or rotate about two axes perpendicular to each other.

FIG. 4 is a block diagram showing the internal configuration of the control system 250 of the vehicle HUD device 1. As shown in FIG. A control system 250 of the vehicle HUD device 1 includes an FPGA 251 , a CPU 252 , a ROM 253 , a RAM 254 , an interface (hereinafter referred to as I/F) 255 , a bus line 256 , an LD driver 257 and a MEMS controller 258 . The FPGA 251 controls the operation of the laser light sources 201R, 201G, and 201B of the light source unit 220 with the LD driver 257, and the MEMS 208a of the optical scanning device 208 with the MEMS controller 258. FIG. The CPU 252 controls each function of the vehicle HUD device 1 . The ROM 253 stores various programs such as an image processing program executed by the CPU 252 to control each function of the vehicle HUD device 1 . A RAM 254 is used as a work area for the CPU 252 . The I/F 255 is an interface for communicating with an external controller or the like. connected to Further, the I/F 255 is connected with a forward photographing camera 110 for photographing display information of the vehicle HUD device 1 and its background (landscape through the windshield 302). Also, the I/F 255 is connected to an ambient light sensor 150 that detects the brightness and chromaticity of ambient light.

FIG. 5 is a block diagram showing a schematic configuration of the vehicle HUD device 1 and its peripheral devices. A vehicle navigation device 400, a sensor device 500, and the like are provided as information acquisition means for acquiring driver-provided information to be provided to the driver by means of the virtual image G. FIG. The vehicle HUD device 1 mainly includes an optical system 230 that constitutes image light projection means as image display means, and a control system 250 as display control means.

The vehicle navigation device 400 can widely use a known vehicle navigation device mounted on an automobile or the like. Information necessary for generating a route navigation image to be displayed on the virtual image G is output from the vehicle navigation device 400 , and this information is input to the control system 250 . For example, as shown in FIG. 1, the number of lanes (driving lanes) on the road on which the vehicle 301 is traveling, the distance to the point where the course should be changed (right turn, left turn, branching, etc.), and then the course change It contains an image that shows information such as direction. These pieces of information are input from the vehicle navigation device 400 to the control system 250 . Thus, under the control of the control system 250, the vehicle HUD device 1 displays as follows. That is, navigation images such as a driving lane indication image 711 , an inter-vehicle distance indication image 712 , a route indication image 721 , a remaining distance image 722 , an intersection name image 723 , etc. are displayed in the upper display area A of the display area 700 .

Further, in the display example shown in FIG. 1, an image showing road-specific information (road name, speed limit, etc.) is displayed in the lower display area B of the display area 700 . This road specific information is also input from the vehicle navigation device 400 to the control system 250 . The control system 250 causes the vehicle HUD device 1 to display a road name display image 701 , a speed limit display image 702 , an overtaking prohibition display image 703 , etc. corresponding to the road-specific information in the lower display area B of the display area 700 .

The sensor device 500 is composed of one or two or more sensors for detecting various kinds of information indicating the behavior of the vehicle 301, the state of the vehicle 301, the circumstances around the vehicle 301, and the like. Sensing information necessary for generating an image to be displayed as the virtual image G is output from the sensor device 500 , and this sensing information is input to the control system 250 . For example, in the image example shown in FIG. 1, a vehicle speed display image 704 (in FIG. 1, an image of characters "83 km/h" in FIG. 1) indicating the vehicle speed of the host vehicle 301 is displayed in the lower display area B of the display area 700. . Therefore, the vehicle speed information included in the CAN information of the own vehicle 301 is input from the sensor device 500 to the control system 250, and under the control of the control system 250, the vehicle HUD device 1 displays a character image indicating the vehicle speed in the display area 700. It is displayed in the lower display area B.

The sensor device 500 detects, for example, (1) other vehicles, pedestrians, and structures (guardrails, a laser radar device and an imaging device for detecting the distance from a telephone pole, etc.), a sensor for detecting the external environment information of the own vehicle (external temperature, brightness, weather, etc.), (2) driving operation of the driver 300 (brake (3) a sensor for detecting the amount of fuel remaining in the fuel tank of the vehicle 301; (4) a sensor for detecting the state of various in-vehicle devices such as the engine and battery; sensors that By detecting such information with the sensor device 500 and sending it to the control system 250 , the information is displayed as a virtual image G by the vehicle HUD device 1 and provided to the driver 300 .

Next, the virtual image G displayed by the vehicle HUD device 1 will be described. In the vehicle HUD device 1 , the driver-provided information provided to the driver 300 by the virtual image G may be any information as long as it is useful for the driver 300 . In the following, as an example, the information provided to the driver is roughly classified into passive information and active information.

Passive information is information that is passively recognized by driver 300 at the timing when a predetermined information provision condition is satisfied. Therefore, the information provided to the driver 300 at the setting timing of the vehicle HUD device 1 is included in the passive information, and information having a certain relationship between the timing at which the information is provided and the content of the information is included. Included in passive information. Passive information includes, for example, information related to safety during driving, route navigation information, and the like. Information related to safety during driving includes inter-vehicle distance information between own vehicle 301 and the preceding vehicle (inter-vehicle distance presentation image 712), emergency information related to driving (emergency operation instruction information instructing the driver to perform an emergency operation). such as warning information or alert information). The route navigation information is information for guiding a travel route to a preset destination, and is provided to the driver by a known vehicle navigation system. The route navigation information includes travel lane indication information (travel lane indication image 711) that indicates the lane to travel at the nearest intersection, and course change operation at the next intersection or junction where the course should be changed from the straight ahead direction. Course change operation instruction information to be instructed may be mentioned. As route change operation instruction information, route designation information (route designation image 721) for designating which route to take at an intersection or the like, remaining distance information to the intersection or the like where the route change operation is to be performed (remaining distance image 722). , intersection name information (intersection name image 723).

Active information is information that is actively recognized by the driver 300 at timing determined by the driver himself/herself. The active information is sufficient information if it is provided to the driver 300 at the timing desired by the driver 300. For example, the relationship between the timing at which the information is provided and the content of the information is low or not. Missing information is included in active information. Since the active information is information that the driver 300 acquires at the timing desired by the driver 300, it continues to be displayed for a certain long period of time or all the time. For example, specific information of the road on which the own vehicle 301 is traveling, vehicle speed information of the own vehicle 301 (vehicle speed display image 704), current time information, and the like. The road-specific information includes, for example, road name information (road name display image 701), regulation content information such as the speed limit of the road (speed limit display image 702, overtaking prohibition display image 703), and other information related to the road. Information that is useful to driver 300 may be mentioned.

The passive information and the active information roughly classified in this manner are displayed in corresponding display areas in the display area 700 in which the virtual image G can be displayed. Specifically, the display area 700 is vertically divided into two display areas, of which the upper display area A mainly displays the passive information image corresponding to the passive information, and the lower display area B mainly displays the passive information image corresponding to the passive information. Display an active information image corresponding to the active information. When part of the active information image is displayed in the upper display area A, the active information image is displayed so that the visibility of the passive information image displayed in the upper display area A is prioritized.

Also, as the virtual image G displayed in the display area 700, a stereoscopic image expressed using stereoscopic vision is used. Specifically, the inter-vehicle distance presentation image 712 and the driving lane indication image 711 displayed in the upper display area A of the display area 700 are perspective images expressed by the perspective method.

Specifically, the five horizontal lines forming the inter-vehicle distance presentation image 712 are made shorter toward the upper side, and the inter-vehicle distance presentation image 712 is drawn by perspective drawing so as to face one vanishing point. Law image. In particular, inter-vehicle distance presentation image 712 is displayed so that its vanishing point is located near the gaze point of driver 300, so that inter-vehicle distance presentation image 712 is more likely to be perceived by driver 300 while driving. Further, the perspective image is such that the width of the horizontal line becomes thinner toward the upper side, and the brightness of the horizontal line becomes lower toward the upper side. This makes it easier for the driver 300 who is driving to perceive the sense of depth of the inter-vehicle distance presentation image 712 .

The passive information and active information displayed in the upper display area A and the lower display area B of the display area 700 are displayed as a “display image (first image)” that provides information to the driver (user) 300 . In this embodiment, in addition to the above-mentioned "display image (first image)", for example, the display area 700 maintains the wakefulness of the driver during driving, and also displays the image of the driver in danger of maintaining the wakefulness. In this case, the driver (user) 300 is presented with a visual stimulus as a "visual stimulus image (second image)" for calling attention to the driver and returning to the arousal state (recovering attention). do.

This visual stimulus image is, for example, predetermined visual information for the purpose of maintaining alertness or alerting the driver, and is a so-called dummy image that does not include information related to driving for the driver (information provided to the driver). be. A visual stimulus image has, for example, a predetermined shape (eg, circle, triangle, square, etc.) and a predetermined color (eg, red, blue, yellow). The visual stimulus image is provided to the driver as a virtual image G through the optical system 230 of the vehicle HUD device 1, like the display image. The display mode of the visual stimulus image is controlled by the control system (control section) 250 of the vehicle HUD device 1 .

As a result of intensive research on the presentation of visual stimuli to the user, the present inventors have found that the visual stimuli themselves do not provide information to the user, so that the user does not need to pay attention to them. It has been found that it is extremely useful to present visual stimuli to the user while avoiding movement of the user's line of sight as much as possible depending on the content to be displayed. Considering the case of presenting a visual stimulus to a driver, who is an example of a user, the environment changes significantly during driving, and the driver's line of sight should be directed in the direction of travel of the vehicle. It is preferable to provide a visual stimulus to the driver without inducing It is known that the speed of human gaze movement decreases in a state of low arousal in which attention is reduced, and attention is likely to decrease during driving when the vehicle is moving at high speed. environment can be considered. For example, when driving for a long time in a situation where there are few changes in scenery or driving behavior, such as when driving on an expressway, the driver's attention tends to decrease. Then, when the attention of the driver who is driving a vehicle moving at high speed is declining, the visual stimulus is presented to restore the attention of the driver, thereby moving the driver's line of sight. Inducing is not preferable in view of the fact that the speed of line-of-sight movement of a person whose attention is declining is declining.

Conventionally, as a means to prevent the user's attention from declining, warnings such as voice and special display devices have been used, but in the case of voice, sufficient effects cannot be obtained due to the surrounding environment (e.g. noise). In the case of a display, there is a concern that it may lead to excessive caution and have an adverse effect on the contrary.

Hereinafter, an embodiment for giving a visual stimulus image without inducing movement of the line of sight of the driver, who is an example of the user (while maintaining the driver in the driving direction), will be described in detail with reference to the drawings.

<<First Embodiment>>
Display images (for example, passive information and active information displayed in upper display area A and lower display area B) and visual stimulus images are displayed in display area 700 . The control system 250 of the vehicular HUD device 1 functions as a "control unit" that displays a plurality of visual stimulus images (second images) in the display area 700 and temporally overlaps the display timings of the plurality of visual stimulus images. do. Adjustment of the display mode of a plurality of visual stimulus images, for example, in the optical system 230 of the vehicle HUD device 1, by varying the lighting time and lighting luminance of the laser light sources 201R, 201G, and 201B, and the mirror of the optical scanning device 208 It is realized by varying the rotational position.

FIG. 6 is a diagram showing an example of a display image displayed in the display area 700 and a plurality of visual stimulus images. In FIG. 6, a first visual stimulus image 10 and a second visual stimulus image 20 having square shapes of the same size are displayed. The first visual stimulus image 10 is displayed inside the upper display area A of the display area 700 at the left end in the left-right direction and slightly below the middle part in the up-down direction. The second visual stimulus image 20 is displayed outside the upper display area A of the display area 700 and above (immediately above) the right edge in the horizontal direction.

By displaying the first visual stimulus image 10 and the second visual stimulus image 20 with temporal overlap (simultaneously), the line of sight is directed to either the first visual stimulus image 10 or the second visual stimulus image 20 . The first visual stimulus image 10 and the second visual stimulus image 20 can be presented to the driver while maintaining the line of sight in the driving direction without being pulled. As a result, it is possible to maintain the wakefulness of the driver and recover his attention without inducing the movement of the driver's line of sight.

Here, "temporally overlap the display timing" and "temporally overlap (simultaneously) display" mean, for example, not only the same frame in the display frame interval, but also the time resolution of the human eye. If the frame interval is not separable, that is, if the frame rate is sufficiently high for the temporal resolution of the human eye, it may be displayed in another frame. In other words, in the driver's visual recognition, when the first visual stimulus image 10 and the second visual stimulus image 20 can be grasped at the same time even for a moment, "the display timing overlaps temporally ” and “displaying overlapping (simultaneously) in terms of time”. The display start timing and the display end timing of the first visual stimulus image 10 and the second visual stimulus image 20 may be completely matched, or even if at least one of the display start timing and the display end timing is deviated. good. Alternatively, one of the first visual stimulus image 10 and the second visual stimulus image 20 may be lit all the time, and the other may be lit (blinking) at intervals.

Also, as shown in FIG. 6, the first visual stimulation image 10 and the second visual stimulation image 20 are displayed in the display area 700 by the control system 250 with the center of the display area 700 interposed therebetween. Alternatively, the first visual stimulus image 10 and the second visual stimulus image 20 are displayed in the display area 700 by the control system 250 so as to be symmetrical about the center of the display area 700 . As a result, the first visual stimulus image 10 and the second visual stimulus image 20 are maintained while the line of sight is maintained in the driving direction without being pulled by either the first visual stimulus image 10 or the second visual stimulus image 20. A visual stimulus image 20 can be presented to the driver. As a result, it is possible to maintain the wakefulness of the driver and recover his attention without inducing the movement of the driver's line of sight.

Further, as shown in FIGS. 7 to 9, the first visual stimulation image 10 and the second visual stimulation image 20 are arranged by the control system 250 so that the center of the visual field of the user who observes the display area 700 is sandwiched between the display areas. 700. As a result, the first visual stimulus image 10 and the second visual stimulus image 20 are maintained while the line of sight is maintained in the driving direction without being pulled by either the first visual stimulus image 10 or the second visual stimulus image 20. A visual stimulus image 20 can be presented to the driver. As a result, it is possible to maintain the wakefulness of the driver and recover his attention without inducing the movement of the driver's line of sight.

In the example of FIG. 7, the first visual stimulus image 10 and the second visual stimulus image 20 are displayed so as to be point symmetric with respect to the center of the driver's visual field within the driver's field of view in the display area 700. ing. As a result, the first visual stimulus image 10 and the second visual stimulus image 20 are arranged so as to sandwich the center of the driver's visual field in both the vertical direction and the horizontal direction.

In the example of FIG. 8, the first visual stimulus image 10 and the second visual stimulus image 20 are positioned within the region of the driver's visual field in the display area 700 relative to the axis of symmetry extending vertically through the center of the driver's visual field. are displayed so as to be line symmetrical. Since the vertical line of sight movement during vehicle driving does not remove the line of sight from the traveling direction, the first visual stimulus image 10 and the second visual stimulus image 20 are arranged symmetrically with respect to the vertical axis of symmetry. It is valid to

In the example of FIG. 9, the first visual stimulus image 10 and the second visual stimulus image 20 are displayed in the display range of the display image in the display area 700, with respect to the axis of symmetry extending vertically through the center of the display range. (The first visual stimulus image 10 and the second visual stimulus image 20 are arranged at the lower left end and the lower right end, respectively). In general, the display range of a vehicle HUD is symmetrical with respect to a straight line in the vertical direction passing through the center of the driver's visual field. It is preferable to have symmetry with respect to the center of the display range of the HUD for use.

As shown in FIG. 10, the first visual stimulus image 10 and the second visual stimulus image 20 are displayed in the display area 700 so as to be outside the display range of the display image in the display area 700. good. In the example of FIG. 10, the first visual stimulus image 10 and the second visual stimulus image 20 are spaced apart to the left and right outside the display range in which the display image in the display area 700 is displayed, and the display range is separated from the left and right. They are arranged so that they are sandwiched between them. In this way, the display image and the visual stimulus image are divided and displayed (arranged) so that their display areas do not overlap. In addition, it becomes possible to maintain the wakefulness of the driver and recover his attention. In addition, the display range of the display image may be set narrower than the drawable range as the display area in consideration of the display quality due to the influence of the restriction (restriction) of the angle of view, optical distortion, and the like. In such a case, if the display position of the visual stimulus image is set so as to be outside the display range of the display image, it becomes possible to effectively utilize the drawable range as the display area.

An integrated intensity is expressed by integrating the luminance (intensity) of the visual stimulus image such as the first visual stimulus image 10 and the second visual stimulus image 20 within a predetermined period of time. Also, the degree of influence of the visual stimulus image on the user is expressed as the intensity of the visual stimulus image. As shown in FIGS. 11A to 11C, the first visual stimulation image 10 and the second visual stimulation image 20 are controlled by the control system 250 so that at least one of brightness, color, and size changes over time. , are displayed in the display area 700 . 11A to 11C, the horizontal axis indicates time, and the vertical axis indicates luminance (intensity) of the visual stimulus image. In the example of FIG. 11A, the luminance (intensity) of the visual stimulus image changes over time. In the example of FIG. 11B, the color and luminance (intensity) of the visual stimulus image change over time. In the example of FIG. 11C, the size and brightness (intensity) of the visual stimulus image change over time. The numbers "1" and "4" in FIG. 11C are used as indices indicating the area (size) of the visual stimulus image. By displaying the visual stimulus image while changing at least one of brightness, color, and size in this way, it is possible to display the visual stimulus image that avoids familiarization of the driver.

The integrated intensity of the first visual stimulus image 10 and the second visual stimulus image 20 within a predetermined period of time is set based on the driver's (user's) discrimination threshold. If the visual stimulus is too weak, it will be difficult for the driver to enjoy the visual stimulus effect of maintaining the driver's wakefulness and recovering attention, and if the visual stimulus is too strong, the information that the driver should recognize ( Various information related to driving) may become sluggish. For this reason, it is possible to provide visual stimuli such that the driver maintains the alertness of the driver and restores the attention while appropriately maintaining the attention of the driver to the information to be recognized (various types of information related to driving). A discrimination threshold that can enjoy the effect is set, and the integrated brightness of the first visual stimulus image 10 and the second visual stimulus image 20 within a predetermined time is set at or near the discrimination threshold. Here, since the discrimination threshold varies depending on the surrounding environment (for example, brightness) and the characteristics of the driver, it is preferable to set it with a certain degree of freedom.

For example, if the light emission time of the visual stimulus image during the display period of the visual stimulus image is sufficiently short relative to the time resolution of the human eye, the intensity of the visual stimulus perceived by the user is determined by the integrated intensity during the display period. If the presentation of visual stimuli draws the driver's attention excessively, it will result in a situation that goes beyond the original purpose and will result in putting the cart before the horse. Must be set. As an indicator of not drawing too much attention, the driver's discrimination threshold is set as the intensity of the visual stimulus. The driver's discrimination threshold is the background luminance of the vehicle HUD display (brightness of the scenery in front of the vehicle 301), that is, the luminance of the area superimposed on the HUD display in the forward direction of the vehicle to which the driver is facing, and the surrounding area. Varies with brightness. For this reason, the background luminance may be defined as an arbitrary fixed value from the distribution of the background luminance in the driving environment obtained in advance by test driving, etc., or the background luminance in the driving environment obtained by test driving etc. may be determined based on a lookup table (LUT) prepared by sensory evaluation or the like for the distribution of and the background luminance.

The integrated intensity of the first visual stimulus image 10 and the second visual stimulus image 20 within a predetermined period of time can be set based on the intensity (brightness) of the display image in the display area 700 . The intensity (luminance) of the display image in the display area 700 can be set based on the brightness of the surrounding environment detected by the ambient light sensor (brightness detection unit) 150, for example. By changing the intensity (brightness) of the visual stimulus image in real time according to the intensity (brightness) of the display image in the display area 700, the driver's wakefulness is maintained without causing the driver's line of sight to move. and restore attention. In addition, depending on the type of display image, there are indications such as cautions and warnings that the driver should definitely recognize. It is also conceivable to decide by a relative value to the luminance of .

FIG. 12 is a diagram showing an example of the configuration of the optical system 230 and the control system 250 for determining integrated intensities of the first visual stimulus image 10 and the second visual stimulus image 20 within a predetermined period of time. As shown in FIG. 12, the control system 250 establishes a correspondence between the intensity (luminance) of the display image and the integrated intensity (integrated luminance) of the first visual stimulus image 10 and the second visual stimulus image 20 within a predetermined period of time. It has a storage unit 250B that stores a lookup table (LUT) 250A that indicates the relationship. For example, the lookup table contains the intensity (luminance) of the first visual stimulus image 10 whose integrated intensity (integrated luminance) is slightly higher than the intensity (luminance) of the displayed image and the intensity (luminance) of the second visual stimulus image 10 . The intensity (brightness) of the stimulus image 20 is set.

The first visual stimulation image 10 and the second visual stimulation image 20 are displayed in the display area 700 at arbitrary timing by the control system 250 . For example, the first visual stimulus image 10 and the second visual stimulus image 20 are temporally randomly displayed in the display area 700 . For example, as shown in FIG. 13, an ON period, which is a visual stimulus image display period, and an OFF period, which is a visual stimulus image non-display period, are divided, and in the ON period, the visual stimulus image is displayed and hidden at predetermined time intervals. You may control so that a display may be repeated. In this way, by randomly displaying a plurality of visual stimulus images in terms of time, it is possible to display visual stimulus images that avoid the habituation of the driver.

Further, the first visual stimulation image 10 and the second visual stimulation image 20 are controlled by the control system 250 when the continuous observation time of the display area 700 by the driver (continuous observation time of the display image) exceeds a predetermined time. , are displayed in the display area 700 . For example, as shown in FIG. 14, after the display of the display image is started, it is determined whether or not the continuous observation time has exceeded a predetermined time (step ST1A), and if the continuous observation time has exceeded the predetermined time (step ST1A: Yes), display of the visual stimulus image may be started, and if the continuous observation time does not exceed the predetermined time (step ST1A: No), it may be arranged to wait until the continuous observation time exceeds the predetermined time. When the continuous observation time exceeds a predetermined time, the driver's attention is more likely to decline. Therefore, by displaying the visual stimulus image at that timing, it is possible to present the visual stimulus while avoiding the driver's getting used to it. . In addition, it is possible to reduce the burden (load) on the device due to excessive display of visual stimulus images.

Also, the first visual stimulus image 10 and the second visual stimulus image 20 are displayed in the display area 700 by the control system 250 at the same time as the specific image is displayed in the display area 700 or with a predetermined time lag. Here, the specific image is, for example, a display image for the purpose of alerting or warning the driver (for example, a traffic jam information image or a traffic control information image by the vehicle navigation device 400). For example, as shown in FIG. 15, after the display of the display image is started, it is determined whether or not the display image includes a specific image (for example, a warning image) (step ST1B), and the display image includes a specific image (for example, warning image) is included (step ST1B: Yes), the display of the visual stimulus image is started, and if the display image does not include the specific image (for example, the warning image) (step ST1B: No), the display image includes the specific image (e.g. a warning image) may be included. When a specific image is displayed for the purpose of alerting or warning the driver, the driver's attention is drawn to the specific image. It is useful to display the image.

Also, the first visual stimulation image 10 and the second visual stimulation image 20 are displayed in the display area 700 at the timing when the control system 250 detects that the driver's attention has decreased. Here, the decrease in the attention of the driver includes, for example, at least one of information on shaking of the vehicle body, information on reseating the seat, information on the posture of the driver, and information on extreme or unnatural accelerator operation or brake operation. can be done. For example, as shown in FIG. 16, after the display of the display image is started, it is determined whether or not a decrease in driver's attention is detected (step ST1C), and a decrease in driver's attention is issued. (Step ST1C: Yes), the display of the visual stimulus image is started, and if the decrease in the driver's attention is not detected (Step ST1C: No), it waits for the detection of the decrease in the driver's attention. You may do so. In this way, by directly detecting a decline in the driver's attention and controlling the display of the visual stimulus image according to the detection result, the effectiveness of the visual stimulus image (wakefulness maintenance function, attention calling function) ) can be further enhanced.

Also, the first visual stimulus image 10 and the second visual stimulus image 20 are displayed in the display area 700 when the control system 250 receives any traffic information from a system inside or outside the vehicle. For example, when driving in a city, the scenery in the direction of travel of the vehicle in the driver's field of view changes greatly, so it is difficult for the driver's attention to decline, but when driving on a highway, the direction of travel of the vehicle in the driver's field of vision is less likely to occur. Because the change in scenery is small, it is easy to cause a decrease in attention. Therefore, as shown in FIG. 17, after the display of the display image is started, it is determined whether or not the vehicle is traveling on the highway based on the traffic information received from the system inside or outside the vehicle (step ST1D), when the vehicle is traveling on the highway (step ST1D: Yes), the display of the visual stimulus image is started, and when the vehicle is not traveling on the highway (step ST1D: No), the display of the visual stimulus image is started. You can choose not to. It should be noted that the determination of whether the vehicle is traveling on an expressway may be made based on a criterion such as, for example, that the traveling speed of the vehicle is stable for a predetermined period of time and exceeds a predetermined speed.

As described with reference to the flowcharts of FIGS. 14 to 17, by setting triggers for displaying a plurality of visual stimulus images based on various conditions (criteria), it is possible to display a plurality of visual stimulation images at optimal timing according to the situation. By displaying the visual stimulus image, it is possible to efficiently present the visual stimulus to the user.

On the other hand, there is also a mode in which display execution of the first visual stimulus image 10 and the second visual stimulus image 20 is switched based on subjective judgment such as the driver's feeling of fatigue. For example, as shown in FIG. 18, a changeover switch (switching section) 30 may be provided. When the driver operates the change-over switch 30 , a control signal is transmitted to the receiver 31 regarding whether or not to display the first visual stimulation image 10 and the second visual stimulation image 20 . The control system 250 performs display control of the first visual stimulation image 10 and the second visual stimulation image 20 based on the control signal received by the receiving unit 31 .

<<Second embodiment>>
As shown in FIG. 19 , the vehicle HUD device 1 of the second embodiment has a field-of-view detection sensor (field-of-view detection unit) 40 that detects the field of view of the driver observing the display area 700 . The field-of-view detection sensor 40 can be composed of, for example, a camera capable of capturing at least the movement of the driver's eyes. The visual field detection sensor 40 has a degree of freedom in its specific form, and for example, it estimates the visual field of the driver by comprehensively analyzing various information such as eye movement, driver posture, and weight shift of the driver. may be The field of view of the driver detected by the field of view detection sensor 40 is output to the control system 250 . The control system 250 controls the display modes of the first visual stimulus image 10 and the second visual stimulus image 20 based on the visual field of the driver detected by the visual field detection sensor 40 .

As shown in FIG. 20, the first visual stimulus image 10 and the second visual stimulus image 20 are generated around the driver in the display area 700 by the control system 250 based on the visual field of the driver detected by the visual field detection sensor 40. It is displayed in the display area 700 so as to be positioned in the field of view. Specifically, the first visual stimulus image 10 and the second visual stimulus image 20 are superimposed on the peripheral visual field of the driver and displayed at the lower left end and the lower right end of the HUD display range, respectively. If the HUD display range includes an area that can be superimposed on the peripheral visual field of the driver, it is preferable to display the visual stimulus image in the area that can be superimposed. By displaying two visual stimulus images in the superimposed area of the HUD display range and the driver's peripheral vision, it is possible to maintain the driver's wakefulness and recover the attention without inducing the driver's line of sight movement. be possible.

As shown in FIG. 21, one of the first visual stimulus image 10 and the second visual stimulus image 20 is displayed in the driver's central visual field (lower right end of the HUD display range) in the display area 700, The other of the one visual stimulus image 10 and the second visual stimulus image 20 may be displayed in the driver's peripheral vision (lower left end of the HUD display range) in the display area 700 . By displaying at least one visual stimulus image in the superimposed region of the HUD display range and the driver's peripheral vision, the driver's alertness is maintained and the driver's attention is restored without inducing the driver's line of sight movement. becomes possible. In addition, even if at least one visual stimulus image must be displayed in the driver's central visual field, the visual stimulus image should be positioned as far away from the driver's visual field center as possible (at least a little in the driver's peripheral visual field). close position).

As shown in FIG. 22 , the first visual stimulus image 10 and the second visual stimulus image 20 are generated by the control system 250 based on the visual field of the driver detected by the visual field detection sensor 40 . They are displayed in the display area 700 so as to be symmetrical about the center. Specifically, the first visual stimulus image 10 and the second visual stimulus image 20 are positioned within the region of the driver's field of vision in the display area 700 with respect to the axis of symmetry extending vertically through the center of the driver's visual field. It is displayed so as to be line symmetrical. In this way, by displaying a plurality of visual stimulus images so as to be symmetrical with respect to the center of the driver's field of view rather than the center of the display range of the HUD, the driver's line of sight is not moved. It is possible to maintain the wakefulness of the driver and recover his attention.

When the visual stimulus image is displayed to the driver, if the visual stimulus image is displayed in the central visual field of the driver, the driver may perceive it excessively, which may make the driver feel annoyed. Therefore, in the second embodiment, the driver's visual field that changes from time to time is detected in real time, and a visual stimulus image is displayed in the driver's peripheral visual field based on the visual field, thereby inducing the driver's line of sight movement. It maintains the wakefulness of the driver and restores the attention of the driver.

<<Third Embodiment>>
As shown in FIG. 23 , the vehicle HUD device 1 of the third embodiment provides a visual stimulation image different from the visual stimulation image (for example, the first visual stimulation image 10 and the second visual stimulation image 20) via the display area 700. It has a visual stimulus presenting unit 50 that presents visual stimuli. As described above, when the driver's attention is reduced, the line of sight trapped in the direction of travel of the vehicle due to the main task of driving becomes more likely to move due to other stimuli. In this case, in order to restore attention so as not to cause a shift in the line of sight, it is necessary to set not one point in the driver's visual field (preferably in the peripheral vision), but a plurality of positions (preferably symmetrical with each other in the peripheral vision). It is desired to present the visual stimulus at multiple locations). When the specifications (functionality) of the HUD device are high, the HUD display range is sufficiently wide-angle, and the display range of the display area 700 is not limited (no), as shown in FIG. A stimulus image (here, two visual stimulus images) is displayed in an overlapping area of the HUD display range and the driver's peripheral vision.

On the other hand, if the specification (functionality) of the HUD device is low and the angle of view of the HUD display range is narrow, so that the display range of the display area 700 is greatly restricted, instead of the visual stimulus image via the display area 700 or in addition to the visual stimulus image via the display area 700, by using the visual stimulus by the visual stimulus presentation unit 50, without being affected by the specifications (functionality) of the HUD device and the HUD display range, the driver It is possible to maintain the driver's wakefulness and recover his attention without inducing any movement of the driver's line of sight.

FIG. 25 shows an example of using both a visual stimulus image through the display area 700 and a visual stimulus through the visual stimulus presentation unit 50 . In FIG. 25, the visual stimulus image via the display area 700 is displayed in the superimposed region of the HUD display range and the driver's peripheral vision, and the visual stimulus via the visual stimulus presentation unit 50 is presented in the driver's peripheral vision. Specifically, the visual stimulus image via the display area 700 arranged vertically and the visual stimulus via the visual stimulus presentation unit 50 sandwich the driver's central visual field from above and below. The display timing of the visual stimulus image via the display area 700 and the presentation timing of the visual stimulus (another visual stimulus) via the visual stimulus presentation unit 50 temporally overlap.

FIG. 26 shows an example of using a plurality of visual stimuli via the visual stimulus presentation unit 50 without using visual stimulus images via the display area 700 . In FIG. 26 , a plurality of visual stimuli are presented to the driver's peripheral visual field via the visual stimulus presentation unit 50 . Specifically, two visual stimuli via the visual stimulus presentation units 50 arranged above and below sandwich the driver's central visual field from above and below. Each of the two visual stimuli via the visual stimulus presentation unit 50 is presented at overlapping timings.

FIG. 27 is a block diagram showing an example of a linking unit between the display control unit 240 and the visual stimulus presentation unit 50. As shown in FIG. FIG. 27 shows a block including the optical system 230 and the control system 250 as the display control section 240 . The visual stimulus presentation unit 50 has, for example, an optical system 50A including a light source such as an LED, and a control system 50B that controls the optical system 50A. For example, when the light source of the optical system 230 of the display control unit 240 and the light source of the optical system 50A of the visual stimulus presentation unit 50 have different operation principles or frame rates, the control system 250 of the display control unit 240 and the visual stimulus presentation unit 50 By cooperating with the control system 50B, it is possible to maintain operational stability, reduce the burden on the control device, and reduce the cost.

<<Fourth Embodiment>>
The vehicle HUD device 1 of the fourth embodiment has a field-of-view detection sensor (field-of-view detection unit) 40 that detects the field of view of the driver in the display area 700, similarly to the vehicle HUD device 1 of the second embodiment. (see Figure 19). In the fourth embodiment, the field-of-view detection sensor 40 functions as a "line-of-sight movement detection unit" that detects the line-of-sight movement of the driver.

The vehicle HUD device 1 of the fourth embodiment uses the control system 250 to set the above-described first visual stimulus image 10 and second visual stimulus image 20 as main stimulus images (second images), for example. A dummy stimulus image (second image) different from the stimulus image is displayed in the display area 700 .

FIG. 28 shows an example of display positions of the first visual stimulation image 10 and the second visual stimulation image 20 as the main stimulation images, and the two dummy stimulation images 60 . The first visual stimulus image 10, the second visual stimulus image 20 (main stimulus image), and the two dummy stimulus images 60, respectively, extend vertically through the center of the driver's visual field within the area of the driver's visual field. It is displayed so as to be symmetrical with respect to the extending axis of symmetry. The first visual stimulus image 10 and the second visual stimulus image 20 (main stimulus image) are displayed in an overlapping region of the HUD display range and the driver's central visual field. Two dummy stimulus images 60 are displayed in the overlapping area of the HUD display range and the driver's peripheral vision.

The two dummy stimulus images 60 have the function of inducing movement of the driver's line of sight. The first visual stimulus image 10 and the second visual stimulus image 20 (main stimulus image) have the function of promoting the maintenance of the user's arousal state and recovery of attention. Two dummy stimulus images 60 are displayed in the driver's peripheral vision in display area 700, for example, while the driver is driving. After the two dummy stimulus images 60 are displayed, when the visual field detection sensor 40 detects the movement of the driver's line of sight, the first visual stimulus image 10 and the second visual stimulus image 20 (main stimulus image) are displayed. is displayed in the display area 700 . Furthermore, instead of the two dummy stimulus images 60 displayed in the display area 700, two dummy stimuli presented by the visual stimulus presentation section 50 may be used.

29A to 29C are diagrams showing an example of the flow from display of dummy stimulus images to display of main stimulus images. In FIG. 29A, two dummy stimulus images are displayed in the overlapping region of the HUD display area and the driver's peripheral vision. In FIG. 29B, the display of the two dummy stimulus images in FIG. 29A induces the driver's line of sight movement, and as a result, the two dummy stimulus images are displayed in the superimposed region of the HUD display range and the driver's central visual field. In FIG. 29C, the display of the two dummy stimulus images is discontinued and the two main stimulus images are displayed in the overlapping region of the HUD display area and the driver's peripheral vision.

As the driver's attention level declines, the line of sight, which is trapped in the direction of travel of the vehicle due to the main task of driving, tends to shift due to other stimuli. In other words, when the dummy stimulus image is displayed (or the dummy stimulus is presented), when the driver's line of sight, which should be oriented in the traveling direction of the vehicle, moves toward the dummy stimulus image (or the dummy stimulus), the driver's attention is reduced. presumed to be The driver's gaze movement is acquired when the dummy stimulus image is displayed (or the dummy stimulus is presented), and when the driver's gaze movement is detected by the display of the dummy stimulus image (or the dummy stimulus is presented), the main stimulus is detected. By displaying the image, it is possible to provide a visual stimulus image that combines the main stimulus image and the dummy stimulus image (or dummy stimulus) without acquiring information from other systems in the vehicle (such as the driver monitor). Become.

FIG. 30 is a flow chart showing an example of operation using a main stimulus image and a dummy stimulus image. After the display of the dummy stimulus image is started, it is determined whether or not it is detected that the dummy stimulus image induces the driver's line of sight movement based on the visual field of the driver detected by the visual field detection sensor 40. (Step ST1E). When it is detected that the driver's line of sight movement is induced by the dummy stimulus image (step ST1E: Yes), the display of the main stimulus image is started. If it is not detected that the dummy stimulus image has induced the driver's line of sight movement (step ST1E: No), the dummy stimulus image will not induce the driver's line of sight movement without starting the display of the main stimulus image. wait for

In the fourth embodiment, the number of dummy stimulus images displayed (or the number of dummy stimuli presented) is two. This is to prevent the driver's field of vision from deviating significantly from the direction of travel of the vehicle even when the driver's attention is declining and the driver's line of sight moves to the dummy stimulus image (or dummy stimulus). be. However, as mentioned above, the dummy stimulus image is used to detect whether the driver's attention is declining. Since there is no movement to the stimulus image (or dummy stimulus), the number of dummy stimulus images displayed (or the number of dummy stimuli presented) is 2 from the viewpoint of detecting whether or not the driver's attention is declining. It is not necessary to be limited to one, and by devising a position where the dummy stimulus image is displayed (or the position where the dummy stimulus is presented) so that the driver's field of vision does not deviate greatly from the traveling direction of the vehicle, the dummy The number of stimulus images displayed (or the number of dummy stimuli presented) may be one.

<<Fifth Embodiment>>
In the vehicle HUD device 1 of the fifth embodiment, the detection result of the ambient light sensor (brightness detection unit) 150 (FIGS. 2, 5, 18, 19, and 23) that detects the brightness of the surrounding environment Based on this, the brightness of the first visual stimulus image 10 and the second visual stimulus image 20 (for example, integrated intensity within a predetermined period of time) is changed. Here, the brightness of the surrounding environment includes, for example, luminance, illuminance, luminous intensity, total luminous flux, and values calculated based on these measurement results.

The brightness of the surrounding environment of a running vehicle changes from moment to moment depending on various factors such as time, weather, passage through a tunnel, and the like. Therefore, if the first visual stimulus image 10 and the second visual stimulus image 20 were always displayed with the same intensity (luminance), the intensity of the visual stimulus image would be too strong in a certain situation, and concentration on driving would be disturbed. In other situations, the intensity of the visual stimulus image may be too weak, and the effect of displaying the visual stimulus image (maintaining the driver's arousal state and restoring attention) may be insufficient.

Therefore, the brightness of the surrounding environment is detected in real time by the ambient light sensor (brightness detection unit) 150, and feedback control is executed based on the detection result to maintain the alertness of the driver and restore his attention. Visual stimuli can be presented with appropriate intensity.

In that case, the intensity (luminance) of the visual stimulus image is set to a value that satisfies at least one of, for example, a value that slightly exceeds the brightness of the detected surrounding environment or a value that slightly exceeds the driver's discrimination threshold. do. As described above, the driver's discrimination threshold is the background luminance of the HUD display (brightness of the scenery in front of the vehicle 301), that is, the luminance of the area superimposed on the HUD display in the forward direction of the vehicle that the driver is facing. It changes according to the brightness of its surroundings. For this reason, the background luminance may be defined as an arbitrary fixed value from the distribution of the background luminance in the driving environment obtained in advance by test driving, etc., or the background luminance in the driving environment obtained by test driving etc. may be determined based on a lookup table (LUT) prepared by sensory evaluation or the like for the distribution of and the background luminance. That is, as shown in FIG. 12, the intensity of the visual stimulus image may be determined based on a lookup table (LUT) 250A stored in the storage section 250B of the control system 250. FIG. Alternatively, as shown in FIG. 31, the control system 250 may be provided with a visual stimulus image intensity calculator 250C that calculates the intensity of the visual stimulus image based on the detection result of the ambient light sensor 150. FIG.

The brightness of the surrounding environment may be detected based on an image captured by the front camera 110 instead of being detected by the ambient light sensor 150 . Although the installation position of the forward photographing camera 110 can be inside or outside the vehicle, it is preferable to consider the influence of the windshield (for example, refraction and diffraction) when installing it inside the vehicle.

In the above embodiment, the case where the display device 1 is applied to a vehicle HUD device has been described as an example. For example, it is also possible to apply the display device 1 to a wearable device (for example, a head mount display (HMD)) that is used by being worn on the body of a driver (user).

In the above embodiments, the case where two visual stimulus images are mainly applied as a plurality of visual stimuli has been exemplified and explained, but the number of multiple visual stimulus images is not limited to two, and the user's Three or more visual stimulus images can be applied as long as they do not induce eye movement.

1 Display device (vehicle HUD device)
10 first visual stimulus image (multiple visual stimulus images, main stimulus image, second image)
20 second visual stimulus images (multiple visual stimulus images, main stimulus image, second image)
30 switch (switching unit)
40 field-of-view detection sensor (field-of-view detection unit, line-of-sight movement detection unit)
50 visual stimulus presentation unit 60 dummy stimulus image (second image)
150 ambient light sensor (brightness detection unit)
250 control system (control unit)
700 display area

Claims (22)

A display device for displaying a first image providing information to a user in a display area,
a control unit that displays a second image that does not include information to be provided to the user in the display area;
The control unit causes the display area to display a plurality of the second images,
display timings of the plurality of second images overlap in time ;
further comprising a line-of-sight movement detection unit that detects line-of-sight movement of the user;
The plurality of second images have a dummy stimulus image that induces movement of the user's line of sight and a main stimulus image that prompts the user to recover attention,
After the dummy stimulus image is displayed, the main stimulus image is displayed in the display area when the eye movement detection unit detects the eye movement of the user.
A display device characterized by: The plurality of second images are displayed in the display area across at least one of the center of the display area and the center of the visual field of the user observing the display area.
2. The display device according to claim 1, wherein: The plurality of second images are displayed in the display area so as to be symmetrical with respect to at least one of the center of the display area and the center of the visual field of the user observing the display area.
3. The display device according to claim 1, wherein: The plurality of second images are displayed in the display area while at least one of brightness, color, and size varies.
4. The display device according to any one of claims 1 to 3, characterized in that: The integrated intensity obtained by integrating the luminance of the plurality of second images within a predetermined time is set based on the user's discrimination threshold.
5. The display device according to any one of claims 1 to 4, characterized in that: The integrated intensity obtained by integrating the luminance of the plurality of second images within a predetermined time is set based on the luminance of the first image.
6. The display device according to any one of claims 1 to 5, characterized in that: display timing of the plurality of second images is random;
7. The display device according to any one of claims 1 to 6, characterized in that: The plurality of second images are displayed in the display area simultaneously with the display of the specific image in the first image or with a predetermined time difference,
7. The display device according to any one of claims 1 to 6, characterized in that: The plurality of second images are displayed in the display area when the user's continuous observation time of the display area exceeds a predetermined time.
9. The display device according to any one of claims 1 to 8, characterized by: The plurality of second images are displayed in the display area at a timing when a decrease in attention of the user is detected.
9. The display device according to any one of claims 1 to 8, characterized by: further comprising a switching unit for switching whether to display the plurality of second images in the display area;
11. The display device according to any one of claims 1 to 10, characterized in that: wherein the plurality of second images are displayed outside the display range of the first images in the display area;
12. The display device according to any one of claims 1 to 11, characterized in that: further comprising a field-of-view detection unit that detects the field of view of the user observing the display area;
The plurality of second images are displayed in the peripheral visual field of the user in the display area based on the detection result of the visual field detection unit.
13. The display device according to any one of claims 1 to 12, characterized in that: further comprising a field-of-view detection unit that detects the field of view of the user in the display area;
The plurality of second images are displayed in the display area on either side of the user's visual field center in the display area based on the detection result of the visual field detection unit.
13. The display device according to any one of claims 1 to 12, characterized in that: further comprising a visual stimulus presentation unit that presents a visual stimulus different from the second image;
the display timing of the second image and the presentation timing of the different visual stimulus temporally overlap;
15. The display device according to any one of claims 1 to 14, characterized in that: further comprising a visual stimulus presenting unit that presents a plurality of visual stimuli different from the second image;
each of the plurality of different visual stimuli are presented at overlapping timings;
15. The display device according to any one of claims 1 to 14, characterized in that: further comprising a brightness detection unit that detects the brightness of the surrounding environment;
brightness of the plurality of second images is changed based on the detection result of the brightness detection unit;
17. The display device according to any one of claims 1 to 16, characterized by: The dummy stimulus image is displayed in the user's peripheral visual field in the display area, and the main stimulus image is displayed in the user's visual field center in the display area.
18. The display device according to any one of claims 1 to 17, characterized by: Having the display device according to any one of claims 1 to 18 , wherein the display area is a windshield or a combiner of a vehicle,
A head-up display characterized by displaying the second image when any traffic information is received from a system in-vehicle or out-of-vehicle;
The head-up display according to claim 19 , characterized by: Having the display device according to any one of claims 1 to 18 , and used by being worn on the user's body,
A wearable device characterized by: A display method for displaying a first image providing information to a user in a display area, comprising:
a control step of displaying in the display area a second image that does not contain information to be provided to the user;
The control step displays a plurality of the second images in the display area;
display timings of the plurality of second images overlap in time ;
further comprising a line-of-sight movement detection step of detecting line-of-sight movement of the user;
The plurality of second images have a dummy stimulus image that induces movement of the user's line of sight and a main stimulus image that prompts the user to recover attention,
After displaying the dummy stimulus image, the main stimulus image is displayed in the display area when the user's eye movement is detected by the eye movement detection step.
A display method characterized by:

Images