WO2023003045A1

WO2023003045A1 - Display control device, head-up display device, and display control method

Info

Publication number: WO2023003045A1
Application number: PCT/JP2022/028492
Authority: WO
Inventors: 一成濱田
Original assignee: 日本精機株式会社
Priority date: 2021-07-22
Filing date: 2022-07-22
Publication date: 2023-01-26

Abstract

The present invention makes it difficult for a viewer to have an uncomfortable feeling. A processor acquires eye position-related information including at least one of the eye position, face position, and face direction of a user, displays an AR virtual image V60, executes eye-tracking image correction processing for correcting the position of an image to be displayed on a display device 40 on the basis of at least the eye position-related information in order to adjust the display position of the AR virtual image V60, and when determining that the eye position-related information or a detection operation for the eye position-related information satisfies a predetermined determination condition, executes visibility reduction processing for reducing the visibility of the AR virtual image V60.

Description

Display control device, head-up display device, and display control method

The present disclosure relates to a display control device, a head-up display device, and a head-up display device that are used in a mobile object such as a vehicle and superimpose an image on the foreground of the mobile object (actual view in the forward direction of the mobile object as seen from the vehicle occupant). It relates to a display control method and the like.

In Patent Document 1, display light projected onto a projection target portion such as a front windshield of a vehicle is reflected toward an occupant (observer) of the vehicle inside the vehicle, so that the observer can see the vehicle. describes a head-up display device (an example of a virtual image display device) that allows a user to visually recognize a virtual image that overlaps the foreground of the image. In particular, the head-up display device described in Patent Literature 1 virtually displays a display object at a predetermined position (here, the position is referred to as a target position) in the depth and vertical and horizontal directions in the real space of the foreground. (virtual image), and even if there is a change in the posture of the vehicle or the position of the observer's eyes, the display object will appear as if it were present at the target position in the foreground. Controls the image displayed inside the up display device. That is, such a head-up display device forms augmented reality in which virtual objects are added to the real scenery (foreground) and displayed, and changes in the attitude of the vehicle (this also changes the eye position of the observer with respect to the real scenery). connected) or when the position of the observer's eyes changes in the vehicle, the position of the image displayed inside the head-up display device is adjusted according to the change in the observer's eye position detected by a face detection unit such as a camera. By correcting the above, motion parallax can be given to the virtual object, and the observer can be made to perceive the virtual object as if it were at the target position in the foreground (in the real scene).

Further, in Patent Document 2, the positions of the observer's right eye and left eye detected by a face detection unit such as a camera are tracked, the right-eye display light indicating the right-eye image is directed to the tracked right-eye position, and the tracked image is detected. Binocular parallax is given to the virtual object by controlling the display device so as to direct the display light for the left eye indicating the image for the left eye to the position of the left eye, and whether the virtual object is at the target position in the foreground (in the real scene). A head-up display device is disclosed that allows an observer to perceive in a pseudo manner.

Further, in Japanese Patent Laid-Open No. 2002-200030, from the position of the observer's eyes detected by a face detection unit such as a camera, a specific position on a real object existing in the foreground (or an area around the real object having a specific positional relationship with the real object) is detected. A head-up display device is disclosed that emphasizes the position of a real object existing in the real scene by aligning the display position of an image (virtual image) with a position on a straight line when looking at the position of .

JP 2010-156608 A JP 2019-062532 A WO2019/097918

By the way, when the observer moves his/her head intentionally or when the head moves unintentionally due to vehicle vibration, the face detection unit detects the movement of the eye position and thus also detects the display position of the image (virtual image). can be corrected according to the determined eye position. In such a case, the image corresponding to the eye position is displayed with a delay due to system latency. and an image that does not correspond to the eye position is viewed (the observer can view an image adapted to the first eye position from the second eye position). becomes), it is assumed that the observer feels uncomfortable.

In addition, a face detection unit such as a camera detects the position of the observer's eyes (left and right eye positions) using a complex algorithm from captured images. Depending on the situation, the correction of the display position of the image (virtual image) and the eye position of the observer will not match due to the expansion of the detection error (decrease in detection accuracy) or erroneous detection. is assumed.

A summary of certain embodiments disclosed herein is provided below. It should be understood that these aspects are presented only to provide the reader with an overview of these particular embodiments and are not intended to limit the scope of this disclosure. Indeed, the present disclosure may encompass various aspects not described below.

The outline of the present disclosure relates to making it difficult for the observer to feel uncomfortable. More specifically, the present invention relates to providing a display control device, a head-up display device, a display control method, and the like that make it difficult to visually recognize an image that does not match the user's eye position.

Therefore, the display control device, head-up display device, display control method, etc. described in this specification employ the following means in order to solve the above problems. The present embodiment displays on the display device based on at least the eye position-related information based on the eye position-related information including at least one of the user's eye position, face position, and face direction or the detection operation of the eye position-related information. The gist of this is to reduce the visibility of an AR virtual image that undergoes an eye-following image correction process that corrects the position of the image.

Therefore, the display control device of the first embodiment of the present invention projects a display for displaying an image and the light of the image displayed by the display onto a projection target member, thereby displaying a virtual image of the image to the user of the vehicle in the foreground. 1. A display control device for performing display control in a head-up display device that superimposes on a head-up display device, comprising: one or more processors; a memory; and one or more computer programs configured, wherein the processor acquires eye position-related information including at least one of the user's eye position, face position, and face orientation, and displays AR on the head-up display device In order to display the virtual image and adjust the display position of the AR virtual image, eye-tracking image correction processing is performed to correct the position of the image displayed on the display device based at least on the eye position-related information. Based on this, it is determined whether the eye position-related information or the detection operation of the eye position-related information satisfies a predetermined determination condition, and if it is determined that the determination condition is satisfied, a visibility reduction process is executed to reduce the visibility of the AR virtual image. do, it's supposed to. The first embodiment of the present invention has the advantage of making it difficult to see images that do not match the eye position. Therefore, based on the eye position-related information including at least one of the user's eye position, face position, and face orientation, or based on the detection operation of the eye position-related information, estimating a situation in which an image that does not match the eye position can be viewed, It is possible to reduce the visibility of the AR virtual image on which eye-tracking image correction processing is performed. The AR virtual image is adjusted to match the position, direction, shape, etc. of the real object (analog) in the foreground (real world) as viewed from the eye position (or face position) of the observer. , in which case it is also called a Kontaktanalog image. Note that the AR virtual image is not necessarily limited to a Kontaktanalog image that changes to match the real object (analog). may be an image on which eye-tracking image correction processing (for example, motion parallax addition processing, superimposition processing) that changes is performed.

According to a particularly preferred second embodiment, the determination conditions are conditions for change speed of at least one of eye position, face position, and face orientation, coordinates of at least one of eye position, face position, and face orientation. and at least one of the eye position, face position, and face direction moving time conditions. In this case, estimating a situation in which an image that does not match the eye position can be viewed according to the change speed of at least one of the eye position, the face position, and the face direction, the coordinate condition, or the movement time condition, Visibility of the AR virtual image can be lowered.

According to a particularly preferred third embodiment, the determination conditions are that at least one of the eye position, face position, and face direction change speed is fast; It includes at least one of: the coordinates being within a predetermined range; and at least one of the eye position, face position, and face orientation continuously changing. In this case, there is an advantage that the visibility of the AR virtual image can be reduced on the condition that at least one of the eye position, face position, and face orientation changes quickly. For example, if the change speed is faster than a predetermined threshold, the visibility of the AR virtual image is reduced. Moreover, in this case, there is an advantage that the visibility of the AR virtual image can be reduced on condition that at least one of the coordinates of the eye position, the face position, and the face orientation is within a predetermined range. For example, the visibility of the AR virtual image is reduced in a predetermined range where eye position detection errors are likely to increase (detection accuracy is reduced) or erroneous detections are likely to occur. Moreover, in this case, there is an advantage that the visibility of the AR virtual image can be reduced on condition that a continuous change in at least one of the eye position, face position, and face direction is detected. For example, when it is detected that the eye position continuously changes in one direction, the visibility of the AR virtual image is reduced.

According to a particularly preferred fourth embodiment, the conditions for the detection operation of the eye position-related information are that at least one of the eye position, face position, and face direction cannot be detected, and that the eye position, face position, and face direction cannot be detected. detecting a decrease in detection accuracy of at least one of the orientations. In this case, there is an advantage that the visibility of the AR virtual image can be reduced on the condition that at least one of the eye position, face position, and face direction cannot be detected. Moreover, in this case, there is an advantage that the visibility of the AR virtual image can be reduced on the condition that the detection accuracy of at least one of the eye position, the face position, and the face orientation is lowered.

According to a particularly preferred fifth embodiment, in the visibility reduction process, the processor lowers the visibility differently depending on at least one of the eye position, face position, and face orientation. In this case, at some eye positions, face positions, or face orientations, the AR virtual image is displayed with low visibility that prioritizes preventing the unmatched image from being visually recognized by the observer. , face position, or face orientation, it is possible to display the AR virtual image with moderate visibility considering the visibility of the virtual image while suppressing the unmatched image from being seen by the observer, which is flexible and convenient. has the advantage of being able to provide a system with a high

According to a particularly preferred sixth embodiment, in the visibility reduction processing, the processor lowers the visibility differently according to the change speed of at least one of the eye position, the face position, and the face orientation. ing. In this case, at some eye position, face position, or face direction change speeds, the AR virtual image is displayed with low visibility that prioritizes preventing the unmatched image from being visually recognized by the observer. In the eye position, face position, or face orientation, it is possible to display the AR virtual image with moderate visibility considering the ease of viewing the virtual image, while suppressing the unmatched image from being seen by the observer. It has the advantage of being able to provide a highly convenient system.

According to a particularly preferred seventh embodiment, when the processor determines that the determination condition is not satisfied,
executing a first eye-following image correction process for correcting the position of the image displayed on the display device based at least on the eye position or the face position; and the second correction amount of the image position with respect to the change amount of the eye position or face position is the eye position or face position at the time of the first eye-following image correction processing. or the position of the image relative to at least one of the amount of change in eye position or face position in the vertical direction and the amount of change in eye position or face position in the horizontal direction. is set to zero, and a second image correction process is executed. In this case, there is an advantage that it is possible to suppress the influence of erroneous eye-tracking image correction processing performed with measurement values due to erroneous detection of eye positions or face positions.

According to a particularly preferred eighth embodiment, the processor determines whether the eye-position-related information or the detection operation of the eye-position-related information satisfies a predetermined release condition based on the eye-position-related information, and determines whether the release condition is satisfied. If it is determined to be sufficient, the visibility increasing process is further executed to increase the visibility of the AR virtual image that has been subjected to the visibility decreasing process. In this case, there is an advantage that the image that matches the eye position is less likely to be visually recognized. Therefore, based on the eye position-related information including at least one of the user's eye position, face position, and face direction or the detection operation of the eye position-related information, estimating a situation in which an image that matches the eye position is likely to be viewed, It is possible to improve the visibility of the AR virtual image on which the eye-trackable image correction processing is executed.

According to a ninth embodiment depending on the particularly preferred eighth embodiment, in the visibility increasing process, the processor raises the visibility differently depending on at least one of the eye position, the face position, and the face orientation. Let it be. In this case, for some eye positions, face positions, or face orientations, the AR virtual image is displayed with visibility that prioritizes the visibility of the virtual image, and for some eye positions, face positions, or face orientations, unmatched images It is possible to display the AR virtual image with moderate visibility that considers the visibility of the virtual image while suppressing the visual recognition by the observer, and it is possible to provide a flexible and highly convenient system. have.

According to a tenth embodiment that is particularly dependent on the eighth embodiment, the processor, in the visibility increasing process, comprises different visual It's supposed to be sexually uplifting. In this case, for some of the eye position, face position, or face orientation change speed, the AR virtual image is displayed with visibility that prioritizes the visibility of the virtual image, and for part of the eye position, face position, or face orientation change speed. In part, it is possible to display the AR virtual image with moderate visibility considering the visibility of the virtual image while suppressing the unmatched image from being visually recognized by the observer, providing a flexible and highly convenient system. has the advantage of being able to

According to a particularly preferred eleventh embodiment, a display for displaying an image and light of an image displayed by the display are projected onto a projection target member, so that a virtual image of the image is superimposed on the foreground and visually recognized by the user of the vehicle. a head-up display device that causes a head-up display device, comprising: one or more processors; a memory; one or more computer programs stored in the memory and configured to be executed by the one or more processors; , the processor acquires eye position-related information including at least one of the user's eye position, face position, and face orientation, displays the AR virtual image, and adjusts the display position of the AR virtual image, eye position executing eye-following image correction processing for correcting the position of an image displayed on a display device based at least on the relevant information, and based on the eye-position-related information, the eye-position-related information or the detection operation of the eye-position-related information is performed in a predetermined manner; It is determined whether the determination condition is satisfied, and when it is determined that the determination condition is satisfied, visibility reduction processing is performed to reduce the visibility of the AR virtual image. This provides the advantages described above. Other advantages and preferred features have been particularly mentioned in the embodiments and the description.

According to a particularly preferred eleventh embodiment, a display for displaying an image and light of an image displayed by the display are projected onto a projection target member, so that a virtual image of the image is superimposed on the foreground and visually recognized by the user of the vehicle. A display control method for a head-up display device that allows the head-up display device to display an AR virtual image on the head-up display device; , executing an eye-following image correction process for correcting the position of an image displayed on a display device based at least on the eye position-related information in order to adjust the display position of the AR virtual image; and based on the eye position-related information, Determining whether the eye position-related information or the detection operation of the eye position-related information satisfies a predetermined determination condition, and if it is determined that the determination condition is satisfied, execute a visibility reduction process to reduce the visibility of the AR virtual image. to do and to include; This provides the advantages described above. Other advantages and preferred features have been particularly mentioned in the embodiments and the description.

FIG. 1 is a diagram showing an application example of a vehicle virtual image display system to a vehicle. FIG. 2 is a diagram showing the configuration of the head-up display device. FIG. 3 is a diagram showing an example of a foreground visually recognized by an observer and a virtual image displayed superimposed on the foreground while the host vehicle is running. FIG. 4 shows, in an embodiment in which the HUD device is a 3D-HUD device, a left-viewpoint virtual image and a right-viewpoint virtual image displayed on a virtual image plane, and a perceptual image perceived by an observer based on these left-viewpoint virtual images and right-viewpoint virtual images. , and is a diagram conceptually showing the positional relationship between . FIG. 5 is a diagram conceptually showing a virtual object placed at a target position in the real scene and an image displayed in the virtual image display area such that the virtual object is visually recognized at the target position in the real scene. FIG. 6 is a diagram for explaining a method of motion parallax addition processing in this embodiment. FIG. 7A is a comparative example showing a virtual image visually recognized from the position Px12 shown in FIG. 6 when the motion parallax adding process of this embodiment is not performed. FIG. 7B is a diagram showing a virtual image visually recognized from the position Px12 shown in FIG. 6 when the motion parallax adding process of this embodiment is performed. FIG. 8 is a diagram for explaining a method of motion parallax addition processing by moving eye positions (face positions) in the vertical direction according to the present embodiment. FIG. 9 is a diagram showing an example of a foreground visually recognized by an observer and a virtual image displayed superimposed on the foreground while the own vehicle is running. FIG. 10A is a diagram showing a real object in the foreground and a virtual image displayed by the HUD device that the observer sees when facing forward of the vehicle. The figure below shows an example of this embodiment in which superimposition processing is executed. FIG. 10B is a diagram showing a real object in the foreground and a virtual image displayed by the HUD device that the observer sees when facing forward of the vehicle. The figure below shows an example of this embodiment in which superimposition processing is executed. FIG. 11 is a block diagram of a vehicle virtual image display system according to some embodiments. FIG. 12A is a flow diagram showing a method of executing visibility reduction processing based on the detection result of the observer's eye position, face position, or face direction. FIG. 12B is a flow diagram following FIG. 12A. FIG. 13 is an image diagram showing the eye position (face position), the amount of change in the eye position (face position), the speed of change in the eye position (face position), and the like detected at each predetermined cycle time. FIG. 14 is a diagram showing an example of the foreground visually recognized by the observer, the AR virtual image when the visibility reduction process is executed, and the AR-related virtual image while the host vehicle is running. FIG. 15 is a diagram illustrating a HUD device in some embodiments in which the eyebox can be moved vertically by rotating the relay optics. FIG. 16 is a flow diagram illustrating a method of executing the visibility increasing process while executing the visibility decreasing process.

1 through 16 below provide a description of the configuration and operation of an exemplary vehicular display system. In addition, the present invention is not limited by the following embodiments (including the contents of the drawings). Of course, modifications (including deletion of constituent elements) can be added to the following embodiments. In addition, in the following description, descriptions of known technical matters are omitted as appropriate in order to facilitate understanding of the present invention.

See Figure 1. FIG. 1 is a diagram showing an example of the configuration of a vehicle virtual image display system including a parallax 3D-HUD device. In FIG. 1, the left-right direction of a vehicle (an example of a moving body) 1 (in other words, the width direction of the vehicle 1) is the X-axis (the positive direction of the X-axis is the left direction when the vehicle 1 is facing forward). ), and the vertical direction (in other words, the height direction of the vehicle 1) along a line segment that is orthogonal to the horizontal direction and orthogonal to the ground or a surface corresponding to the ground (here, the road surface 6) is the Y axis (Y axis is the upward direction), and the front-rear direction along a line segment perpendicular to each of the left-right direction and the up-down direction is the Z-axis (the positive direction of the Z-axis is the straight-ahead direction of the vehicle 1). This point also applies to other drawings.

As illustrated, a vehicle display system 10 provided in a vehicle (self-vehicle) 1 displays the positions and line-of-sight directions of a left eye 700L and a right eye 700R of an observer (typically, a driver sitting in the driver's seat of the vehicle 1). A face detection unit 409 for detecting pupils (or faces) to be detected, a vehicle exterior sensor 411 configured by a camera (for example, a stereo camera) for imaging the front (in a broad sense, the surroundings) of the vehicle 1, a head-up display device (hereinafter , a HUD device) 20, a display control device 30 that controls the HUD device 20, and the like.

FIG. 2 is a diagram showing one aspect of the configuration of the head-up display device. The HUD device 20 is installed, for example, in a dashboard (reference numeral 5 in FIG. 1). The HUD device 20 accommodates a stereoscopic display device (an example of a display device) 40, a relay optical system 80, and the display device 40 and the relay optical system 80, and transmits display light K from the display device 40 from the inside to the outside. It has a housing 22 having a light exit window 21 that can be emitted toward.

The display device 40 is a parallax 3D display device here. This display device (parallax type 3D display device) 40 is a naked-eye stereoscopic display device using a multi-viewpoint image display method that can control depth representation by visually recognizing left-viewpoint images and right-viewpoint images. and a light source unit 60 functioning as a backlight.

The display 50 has a spatial light modulation element 51 that optically modulates the illumination light from the light source unit 60 to generate an image, and, for example, a lenticular lens or a parallax barrier (parallax barrier). The light emitted from the light beams K11, K12, and K13 for the left eye (reference symbol K10 in FIG. 1) and the right eye display light beams K21, K22, and K23 for the right eye (see FIG. 1). 1 (K20) and an optical layer (an example of a light beam splitting section) 52. Optical layer 52 includes optical filters such as lenticular lenses, parallax barriers, lens arrays, and microlens arrays. However, this is an example and is not limited. Embodiments of the optical layer 52 are not limited to the above-described optical filters, and the display light for the left eye (K10 in FIG. 1) and the display light for the right eye (K10 in FIG. 1) from the light emitted from the spatial light modulation element 51 K20) includes all forms of optical layers placed on the front or back surface of the spatial light modulator 51. FIG. Some embodiments of the optical layer 52 are electrically controlled so that left-eye display light (K10 in FIG. 1) and right-eye display light (K10 in FIG. 1) are separated from light emitted from the spatial light modulator 51. symbol K20), such as a liquid crystal lens. That is, embodiments of optical layer 52 may include those that are electrically controlled and those that are not.

In addition, the display device 40 is configured by configuring the light source unit 60 with a directional backlight unit (an example of the light beam separating portion) instead of or in addition to the optical layer (an example of the light beam separating portion) 52. left-eye display light such as left-eye light rays K11, K12, and K13 (reference symbol K10 in FIG. 1), right-eye display light such as right-eye light rays K21, K22, and K23, etc. (reference symbol K20 in FIG. 1); may be emitted. Specifically, for example, the display control device 30, which will be described later, causes the spatial light modulation element 51 to display a left viewpoint image when the directional backlight unit emits illumination light directed toward the left eye 700L. Left-eye display light K10 such as light rays K11, K12, and K13 is directed toward the viewer's left eye 700L, and the directional backlight unit emits illumination light toward the right eye 700R. By displaying an image, right-eye display light K20 such as right-eye light rays K21, K22, and K23 is directed to the viewer's left eye 700R. However, the embodiment of the directional backlight unit described above is an example, and is not limited.

The display control device 30, which will be described later, performs, for example, image rendering processing (graphic processing), display device driving processing, and the like, so that the display light K10 for the left eye of the left viewpoint image V10 and the display light K10 for the right eye 700R of the observer's left eye 700L are displayed. By directing the right-eye display light K20 of the right-viewpoint image V20 and adjusting the left-viewpoint image V10 and the right-viewpoint image V20, the aspect of the perceptual virtual image FU displayed by the HUD device 20 (perceived by the observer) is controlled. be able to. The display control device 30, which will be described later, controls the display (display device 50) so as to generate a light field that (approximately) reproduces light rays output in various directions from a point in a certain space. may

The relay optical system 80 has curved mirrors (concave mirrors, etc.) 81 and 82 that reflect the light from the display device 40 and project the image display light K10 and K20 onto the windshield (projection target member) 2 . However, it may further include other optical members (including refractive optical members such as lenses, diffractive optical members such as holograms, reflective optical members, or any combination thereof).

In FIG. 1, the display device 40 of the HUD device 20 displays an image (parallax image) with parallax for each of the left and right eyes. Each parallax image is displayed as V10 and V20 formed on a virtual image display surface (virtual image formation surface) VS, as shown in FIG. The focus of each eye of the observer (person) is adjusted so as to match the position of the virtual image display area VS. Note that the position of the virtual image display area VS is referred to as an "adjustment position (or imaging position)", and a predetermined reference position (for example, the center 205 of the eyebox 200 of the HUD device 20, the observer's viewpoint position, or , a specific position of the vehicle 1, etc.) to the virtual image display area VS (see symbol D10 in FIG. 4) is referred to as an "adjustment distance (imaging distance)".

However, in reality, since the human brain fuses each image (virtual image), the human is at a position farther back than the adjustment position (for example, due to the convergence angle between the left viewpoint image V10 and the right viewpoint image V20). It is a fixed position, and the position perceived as being farther away from the observer as the angle of convergence decreases) is recognized as a perceptual image (here, an arrowhead figure for navigation) FU is displayed. do. Note that the perceptual virtual image FU may be referred to as a "stereoscopic virtual image", and may also be referred to as a "stereoscopic image" when the "image" is taken in a broad sense to include virtual images. It may also be referred to as a "stereoscopic image", "3D display", or the like. The HUD device 20 can display the left-viewpoint image V10 and the right-viewpoint image V20 so that the perceived image FU can be viewed at a position on the front side of the adjustment position.

Next, refer to FIGS. 3 and 4. FIG. 3 is a diagram showing an example of a foreground visually recognized by an observer and a perceptual image superimposed on the foreground and displayed while the vehicle 1 is running. FIG. 4 is a diagram conceptually showing the positional relationship between the left-viewpoint virtual image and the right-viewpoint virtual image displayed on the virtual image plane, and the perceived image perceived by the observer from the left-viewpoint virtual image and the right-viewpoint virtual image. be.

In FIG. 3, the vehicle 1 is traveling on a straight road (road surface) 6. The HUD device 20 is installed inside the dashboard 5 . Display light K (K10, K20) is projected from the light exit window 21 of the HUD device 20 onto the projected portion (front windshield of the vehicle 1) 2. FIG. In the example of FIG. 3, a first content image FU1 that is superimposed on the road surface 6 and instructs the route of the vehicle 1 (here, indicates straight ahead), and similarly indicates the route of the vehicle 1 (here, indicates straight travel). and a second content image FU2 perceived farther than the first content image FU1 is displayed.

As shown in the left diagram of FIG. 4, the HUD device 20 (1) directs the projection target 2 to the left eye 700L detected by the face detection unit 409 at such a position and angle as to be reflected by the projection target 2. Left-eye display light K10 is emitted to form a first left-viewpoint content image V11 at a predetermined position in the virtual image display area VS seen from the left eye 700L, and (2) reflected by the projection target unit 2 to the right eye 700R. Right-eye display light K20 is emitted to the projection target unit 2 at a position and angle such that . The first content image FU1 perceived by the first left-viewpoint content image V11 and the first right-viewpoint content image V21 having parallax is positioned behind the virtual image display area VS by a distance D21 (the above position separated from the reference position by a distance D31).

Similarly, as shown in the right diagram of FIG. 4, the HUD device 20 (1) detects the left eye 700L detected by the face detection unit 409 at a position and angle that are reflected by the projection target unit 2. The left-eye display light K10 is emitted to the portion 2 to form a second left-viewpoint content image V12 at a predetermined position in the virtual image display area VS seen from the left eye 700L, and (2) the projected portion 2 to the right eye 700R. Right-eye display light K20 is emitted to the projection target 2 at a position and an angle such that it is reflected by the second right-viewpoint content image V22 at a predetermined position in the virtual image display area VS seen from the right eye 700R. form an image. The second content image FU2 perceived by the second left-viewpoint content image V12 and the second right-viewpoint content image V22 having parallax is positioned behind the virtual image display area VS by a distance D22 (the above position separated from the reference position by a distance D31).

Specifically, the distance (imaging distance D10) from the reference position to the virtual image display area VS is set to, for example, "4 m", and the first distance shown in the left diagram of FIG. The distance to the content image FU1 (first perceptual distance D31) is set to a distance of, for example, "7 m", and the distance from the reference position to the second content image FU2 shown in the right diagram of FIG. (Second perceptual distance D32) is set to, for example, a distance of "10 m". However, this is an example and is not limited.

FIG. 5 is a diagram conceptually showing a virtual object placed at a target position in the real scene and an image displayed in the virtual image display area such that the virtual object is visually recognized at the target position in the real scene. Note that the HUD device 20 shown in FIG. 5 shows an example of performing 2D display instead of 3D display. That is, the display device 40 of the HUD device 20 shown in FIG. 5 is a 2D display device that is not a stereoscopic display device (even a stereoscopic display device can perform 2D display). As shown in FIG. 5, when viewed from the viewer 700, the depth direction is the Z-axis direction, the left-right direction (the width direction of the vehicle 1) is the X-axis direction, and the vertical direction (the vertical direction of the vehicle 1) is the Y-axis direction. Axial direction. Note that the direction away from the viewer is the positive direction of the Z axis, the leftward direction is the positive direction of the X axis, and the upward direction is the positive direction of the Y axis. direction.

The viewer 700 perceives the virtual object FU at a predetermined target position PT in the real scene by visually recognizing the virtual image V formed (imaged) in the virtual image display area VS through the projection target section 2. . A viewer visually recognizes the virtual image V of the image of the display light K reflected by the projection target section 2 . At this time, if the virtual image V is, for example, an arrow indicating a course, the arrow of the virtual image V is placed in the virtual image display area VS so that the virtual object FU is placed at a predetermined target position PT in the foreground of the vehicle 1 and visually recognized. Is displayed. Specifically, the HUD device 20 (display control device 30) uses the center of the observer's left eye 700L and right eye 700R as the origin of the projective transformation, and the virtual object FU of a predetermined size and shape arranged at the target position PT. An image to be displayed on the display device 40 is rendered such that a virtual image V having a predetermined size and shape that has undergone projective transformation is displayed in the virtual image display area VS. Then, the HUD device 20 (display control device 30) sets the virtual image display area VS so that the virtual object FU is perceived at the same target position PT as before the eye position is moved even when the eye position of the observer is moved. By changing the position of the virtual image V displayed in the target position PT, the virtual object FU (virtual image V) is displayed at the target position PT even though it is displayed at a position (virtual image display area VS) away from the target position PT. It can be recognized as if That is, the HUD device 20 (display control device 30) changes the position (this, size (In other words, the HUD device 20 adds motion parallax to the virtual image (image) by image correction accompanying the movement of the eye position, making it easier to perceive depth). In the description of the present embodiment, image position correction that expresses motion parallax in accordance with such a change in eye position is referred to as motion parallax addition processing (an example of eye-tracking image correction processing). The motion parallax adding process is not limited to image position correction that perfectly reproduces natural motion parallax, but may also include image position correction that approximates natural motion parallax. Note that the HUD device 20 (display control device 30) performs motion parallax addition processing (an example of eye-tracking image correction processing) in response to a change in the eye position 700. Motion parallax addition processing (an example of eye-tracking image correction processing) may be executed based on the face position.

FIG. 6 is a diagram for explaining the method of motion parallax addition processing in this embodiment. The display control device 30 (processor 33) of the present embodiment controls the HUD device 20 to display the virtual images V41, V42, and V43 formed (imaged) in the virtual image display area VS via the projection target section 2. indicate. The virtual image V41 is set at the target position PT11, which is a perceived distance D33 (a position that is a distance D23 behind the virtual image display area VS). The virtual image V43 is set at the target position PT12, which is a position behind the area VS by a distance D24 (>D23)), and the virtual image V43 is positioned at a perceived distance D35 longer than the perceived distance D34 of the virtual image V42 (more than the virtual image display area VS). The target position PT13, which is a position behind by a distance D25 (>D24), is set. Note that since the amount of correction of the image on the display device 40 corresponds to the amount of correction of the virtual image in the virtual image display area VS, in FIG. The same reference numerals C1, C2, and C3 are also used for the correction amounts of the virtual image (the same applies to the reference numerals Cy11 (Cy) and Cy21 (Cy) in FIG. 8).

When the observer's face position (eye position 700) moves rightward (in the negative direction of the X axis) by ΔPx10 from the position indicated by Px11 to the position indicated by Px12, the display control device 30 (processor 33) performs motion parallax addition processing. , the display positions of the virtual images V41, V42, and V43 displayed in the virtual image display area VS are corrected in the same direction as the observer's face position (eye position 700) is moved. Only C1, C2 (>C1) and C3 (>C2) are corrected. FIG. 7A shows a virtual image V901 and a virtual image V42 visually recognized from a position Px12 shown in FIG. V902 viewed from the position Px12 shown in FIG. 6 without performing the addition processing, and V903 viewed from the position Px12 illustrated in FIG. FIG. 7B is a diagram showing virtual images V44, V45, and V46 viewed from position Px12 shown in FIG. 6 when the motion parallax adding process of this embodiment is performed. Note that in FIG. 7B, the difference in the positions of the virtual images V44, V45, and V46 is exaggerated so that the difference in correction amount can be easily understood. That is, the display control device 30 (processor 33) corrects the positions of the plurality of virtual images V41, V42, V43 according to the movement of the eye position due to the difference in the perceived distances D33, D34, D35 of the plurality of virtual images V41, V42, V43. By varying the amounts, the observer can perceive the motion parallax only between the virtual images V41 (V44), V42 (V45), and V43 (V46). More specifically, the display control device 30 (processor 33) increases the amount of correction in the motion parallax adding process as the perceived distance D30 set is longer, so that the plurality of virtual images V41 (V44) and V42 ( V45) and V43 (V46) are added with motion parallax.

Embodiments of the eye-tracking image correction process are not limited to the motion parallax addition process described above, and may include the superimposition process described below. That is, the HUD device 20 (display control device 30) may execute superimposition processing (an example of eye-tracking image correction processing) in accordance with a change in the eye position 700 of the observer (change in face position). .

FIG. 8 is a diagram for explaining a method of motion parallax addition processing when the eye position (face position) moves in the vertical direction in this embodiment. When the observer's face position (eye position 700) moves upward (in the positive Y-axis direction) from the position indicated by reference sign Py12, the display control device 30 (processor 33) performs motion parallax addition processing to create a virtual image. As shown in FIG. 8A, the position where the virtual image V displayed in the display area VS is displayed is set in the same direction (upward (Y-axis positive direction)), the correction amount Cy11 is corrected (the position of the virtual image V is changed from the position of the reference V48 to the reference V47). Further, when the observer's face position (eye position 700) moves downward (Y-axis negative direction) from the position of Py12, the display control device 30 (processor 33) executes motion parallax addition processing. Then, the position where the virtual image V displayed in the virtual image display area VS is displayed is moved in the same direction (upward (Y (The position of the virtual image V is changed from V48 to V49) by the correction amount Cy21 in the direction of negative axis). As a result, although the virtual object FU (virtual image V) is displayed at a position (virtual image display area VS) distant from the target position PT, it can be recognized as if it were at the target position PT ( It is possible to enhance the feeling that the virtual object FU (virtual image V) is at the target position PT).

FIG. 9 is a diagram showing a real object 300 existing in the foreground and a virtual image V displayed by the HUD device 20 of the present embodiment, which is visually recognized when an observer faces forward from the driver's seat of the vehicle 1. . A virtual image V shown in FIG. A non-AR virtual image V70 includes a virtual image whose displayed position, direction, and shape are set regardless of the shape. The AR virtual image V60 is displayed at a position (target position PT) corresponding to the position of the real object 300 existing in the real scene. The AR virtual image V60 is displayed, for example, at a position superimposed on the real object 300 or in the vicinity of the real object 300, and notifies the existence of the real object 300 with emphasis. In other words, the “position corresponding to the position of the real object 300 (target position PT)” is not limited to the position superimposed on the real object 300 and viewed by the observer. There may be. It is preferable that the AR virtual image V60 does not interfere with the visual recognition of the real object 300, but any aspect is possible.

The AR virtual images V60 shown in FIG. 9 include navigation virtual images V61 and V62 that indicate guidance routes, enhanced virtual images V64 and V65 that highlight and notify attention targets, and POI virtual images 65 that indicate targets, predetermined buildings, and the like. . The position (target position PT) corresponding to the position of the real object 300 is the position of the road surface 311 (an example of the real object 300) on which these are superimposed in the navigation virtual images V61 and V62. (an example of the real object 300), in the enhanced virtual image V64 the position near the other vehicle 314 (an example of the real object 300), and in the POI virtual image V65 a building 315 (an example of the real object 300). ). As described above, the display control device 30 (processor 33) increases the correction amount C associated with the movement of the observer's eye position in the motion parallax adding process as the perceived distance D30 set for the virtual image V increases. That is, assuming that the perceptual distance D30 set for the virtual image V shown in FIG. The associated correction amount C is set as follows: correction amount of V65>correction amount of V64>correction amount of V63>correction amount of V62>correction amount of V61. Note that the virtual image V62 and the virtual image V61 are of the same kind and are displayed close to each other. The same correction amount may be set.

Further, the display control device 30 (processor 33) of some embodiments may set the correction amount C associated with the movement of the observer's eye position to zero in the non-AR virtual image V70 ( need not be corrected accordingly).

Also, the display control device 30 (processor 33) of some embodiments may correct the non-AR virtual image V70 according to the movement of the observer's eye position. In the example shown in FIG. 9, the non-AR virtual images V70 (V71, V72) are arranged below the virtual image display area VS, and the area of the road surface 311, which is the real object 300, overlaps with the navigation virtual image V61 of FIG. It is closer to the vehicle 1 than the area of the road surface 311 . That is, the display control device 30 (processor 33) of some embodiments sets the perceived distance D30 of the non-AR virtual image V70 (V71, V72) to the AR virtual image V60 (in a narrow sense, the lowest distance among the AR virtual images V60). ) is set shorter than the perception distance D30 of the navigation virtual image V61) arranged in ( In a narrow sense, it may be set to be smaller than the correction amount C of the navigation virtual image V61 positioned at the lowest position among the AR virtual images V60.

10A and 10B are diagrams showing a real object in the foreground and a virtual image displayed by the HUD device visually recognized when the observer faces the front of the vehicle, and the upper diagram is a comparative example in which superimposition processing is not performed. , and the lower diagram shows an example of this embodiment in which superimposition processing is performed.

In a situation where the HUD device 20 displays a rectangular virtual image that is visually recognized by the observer so as to surround the forward vehicle 301 (an example of the real object 300) existing in the foreground, for example, the observer's eye position 700 is When moving rightward (negative direction of the X-axis), the virtual image V911 of the comparative example, which is not superimposed, shifts leftward with respect to the forward vehicle 301 (real object 300), as shown in the upper diagram of FIG. 10A. be visible. Here, the display control device 30 of the present embodiment executes the superimposing process based on the change in the eye position 700 of the observer (moving to the right), and the superimposed AR virtual image V66 of the present embodiment. is visually recognized without deviation (less deviation) from the forward vehicle 301 (real object 300), as shown in the lower diagram of FIG. 10A. The display control device 30 of this embodiment, based on the observer's eye position 700 (or face position) detected by the face detection unit 409 and the position, direction, and shape of the real object 300 detected by the vehicle exterior sensor 411, The position of the virtual image V displayed in the virtual image display area VS is changed so that the virtual image V and the real object 300 have the specific positional relationship stored in the memory 37 . Here, the "specific positional relationship" is, for example, a position overlapping the real object 300, a vicinity of the real object 300, or a position set with the real object 300 as a reference.

Also, in a situation where the HUD device 20 displays an arrow-shaped virtual image that is visually recognized by the observer so as to follow the running lane 302 (an example of the real object 300) existing in the foreground, for example, the observer's eye position 700 moves downward (negative Y-axis direction), the virtual image V912 of the comparative example that is not superimposed moves upward with respect to the driving lane 302 (real object 300), as shown in the upper diagram of FIG. 10B. It shifts and is visually recognized. Here, the display control device 30 of the present embodiment executes the superimposing process based on the change in the eye position 700 of the observer (moving downward), so that the superimposed AR virtual image V67 of the present embodiment is displayed. , as shown in the lower diagram of FIG. 10B, is visually recognized without deviation (with little deviation) from the running lane 302 (real object 300). The display control device 30 of the present embodiment detects the observer's eye position 700 (or face position) detected by the face detection unit 409, and the driving lane 302 (real object) detected by the vehicle exterior sensor 411 (road information database 403). 300), the position, direction, and shape of the virtual image V displayed in the virtual image display area VS are changed so as to match the position, direction, and shape of the real object 300 . That is, the AR virtual image V60 (V51, V52) on which superimposition processing is performed is the position, direction, and shape of the foreground (real world) real object 300 (analog) viewed from the observer's eye position 700 (or face position). In this case, it is also called a Kontaktanalog image.

FIG. 11 is a block diagram of a vehicle virtual image display system according to some embodiments. The display controller 30 comprises one or more I/O interfaces 31 , one or more processors 33 , one or more image processing circuits 35 and one or more memories 37 . Various functional blocks illustrated in FIG. 11 may be implemented in hardware, software, or a combination of both. FIG. 11 is only one embodiment and the illustrated components may be combined into fewer components or there may be additional components. For example, image processing circuitry 35 (eg, a graphics processing unit) may be included in one or more processors 33 .

As shown, processor 33 and image processing circuitry 35 are operatively coupled with memory 37 . More specifically, the processor 33 and the image processing circuit 35 execute a computer program stored in the memory 37 to generate and/or transmit image data, for example, to the vehicle display system 10 ( A display device 40) can be controlled. Processor 33 and/or image processing circuitry 35 may include at least one general purpose microprocessor (e.g., central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA). , or any combination thereof. Memory 37 includes any type of magnetic media such as hard disks, any type of optical media such as CDs and DVDs, any type of semiconductor memory such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

As shown, processor 33 is operatively coupled with I/O interface 31 . For example, the I / O interface 31 communicates (CAN communication). The communication standard adopted by the I/O interface 31 is not limited to CAN. : MOST is a registered trademark), a wired communication interface such as UART or USB, or a personal area network (PAN) such as a Bluetooth network, a local network such as an 802.11x Wi-Fi network. It includes an in-vehicle communication (internal communication) interface, which is a short-range wireless communication interface within several tens of meters such as an area network (LAN). In addition, the I / O interface 31 is a wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e base (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, An external communication (external communication) interface such as a wide area communication network (for example, Internet communication network) may be included according to a cellular communication standard such as 5G.

As shown, the processor 33 is interoperably coupled with the I/O interface 31 to communicate with various other electronic devices and the like connected to the vehicle display system 10 (I/O interface 31). can be given and received. The I/O interface 31 includes, for example, a vehicle ECU 401, a road information database 403, a vehicle position detection unit 405, an operation detection unit 407, a face detection unit 409, an external sensor 411, a brightness detection unit 413, an IMU 415, a mobile information terminal 417, and an external communication device 419, etc. are operatively coupled. Note that the I/O interface 31 may include a function of processing (converting, calculating, and analyzing) information received from other electronic devices connected to the vehicle display system 10 .

The display device 40 is operatively connected to the processor 33 and the image processing circuitry 35 . Accordingly, the image displayed by spatial light modulating element 51 may be based on image data received from processor 33 and/or image processing circuitry 35 . The processor 33 and image processing circuit 35 control the image displayed by the spatial light modulator 51 based on the information obtained from the I/O interface 31 .

The vehicle ECU 401 detects the state of the vehicle 1 (for example, the ON/OFF state of a start switch (for example, an accessory switch: ACC or an ignition switch: IGN) from sensors and switches provided in the vehicle 1 (an example of start information), Travel distance, vehicle speed, accelerator pedal opening, brake pedal opening, engine throttle opening, injector fuel injection amount, engine speed, motor speed, steering angle, shift position, drive mode, various warning conditions, posture (roll angle and/or pitching angle), vehicle vibration (including magnitude, frequency, and/or frequency of vibration), etc., to collect and manage the state of the vehicle 1 (including control ), and as part of its function, it is possible to output a signal indicating the numerical value of the state of the vehicle 1 (for example, the vehicle speed of the vehicle 1 ) to the processor 33 of the display control device 30 . The vehicle ECU 401 simply transmits a numerical value detected by a sensor or the like (for example, the pitching angle is 3 [degrees] in the forward tilting direction) to the processor 33, or alternatively, transmits the numerical value detected by the sensor. Determination results based on one or more states of the vehicle 1 including (for example, the vehicle 1 satisfies a predetermined forward lean condition), or/and analysis results (for example, the degree of brake pedal opening (combined with the information that the vehicle is leaning forward due to braking) may be sent to the processor 33 . For example, the vehicle ECU 401 may output to the display control device 30 a signal indicating a determination result that the vehicle 1 satisfies a predetermined determination condition stored in advance in a memory (not shown) of the vehicle ECU 401 . Note that the I/O interface 31 may acquire the above-described information from sensors and switches provided in the vehicle 1 without using the vehicle ECU 401 .

In addition, the vehicle ECU 401 may output to the display control device 30 an instruction signal that instructs an image to be displayed by the vehicle display system 10. At this time, the coordinates, size, type, display mode of the image, the need for notification of the image, and so on. Necessity-related information that serves as a basis for determining the degree and/or the necessity of notification may be added to the instruction signal and transmitted.

The road information database 403 is included in a navigation device (not shown) provided in the vehicle 1 or an external server connected to the vehicle 1 via an external communication interface (I/O interface 31). Based on the position of the vehicle 1 acquired from the unit 405, road information (lanes, white lines, stop lines, crosswalks, road width, number of lanes, intersections, curves, forks, traffic regulations, etc.), presence/absence, position (including distance to vehicle 1), direction, shape, type of feature information (buildings, bridges, rivers, etc.) , detailed information, etc. may be read and sent to the processor 33 . The road information database 403 may also calculate an appropriate route (navigation information) from the departure point to the destination, and output to the processor 33 a signal indicating the navigation information or image data indicating the route.

The vehicle position detection unit 405 is a GNSS (global navigation satellite system) or the like provided in the vehicle 1, detects the current position and direction of the vehicle 1, and transmits a signal indicating the detection result via the processor 33. or directly to the road information database 403, a portable information terminal 417 and/or an external communication device 419, which will be described later. The road information database 403, a portable information terminal 417 (to be described later), and/or an external communication device 419 acquire position information of the vehicle 1 from the vehicle position detection unit 405 continuously, intermittently, or at each predetermined event. , information about the surroundings of the vehicle 1 may be selected/generated and output to the processor 33 .

The operation detection unit 407 is, for example, a CID (Center Information Display) of the vehicle 1, a hardware switch provided on an instrument panel or the like, or a software switch combining an image and a touch sensor. It outputs to the processor 33 operation information based on the operation by the passenger (the user sitting in the driver's seat and/or the user sitting in the front passenger seat). For example, the operation detection unit 407 sets the display area setting information based on the operation of moving the virtual image display area VS, the eye box setting information based on the operation of moving the eye box 200, and the observer's eye position 700 by the user's operation. Information based on the operation to be performed is output to the processor 33 .

The face detection unit 409 includes a camera such as an infrared camera that detects the eye position 700 (see FIG. 1) of the observer sitting in the driver's seat of the vehicle 1, and may output the captured image to the processor 33. . The processor 33 acquires a captured image (an example of information that can estimate the eye positions 700) from the face detection unit 409, and analyzes the captured image by a technique such as pattern matching to determine the eye positions 700 of the observer. may be detected and a signal indicating the detected coordinates of the eye position 700 may be output to the processor 33 .

In addition, the face detection unit 409 obtains an analysis result obtained by analyzing the captured image of the camera (for example, it indicates where the observer's eye position 700 belongs in a spatial region corresponding to a plurality of preset display parameters). signal.) may be output to the processor 33 . Note that the method of acquiring the eye position 700 of the observer of the vehicle 1 or the information that can estimate the eye position 700 of the observer is not limited to these, and a known eye position detection (estimation) technique is used. may be obtained by

In addition, the face detection unit 409 detects the speed of change and/or the direction of movement of the observer's eye position 700, and sends a signal indicating the speed of change and/or the direction of movement of the observer's eye position 700 to the processor 33. may be output.

The face detection unit 409 also determines (11) that the newly detected eye position 700 is greater than or equal to the eye position movement distance threshold previously stored in the memory 37 with respect to the previously detected eye position 700 (predetermined (12) a signal indicating that the change speed of the eye position is equal to or greater than the eye position change speed threshold stored in advance in the memory 37; (13) When a signal indicating that the observer's eye position 700 cannot be detected after the movement of the observer's eye position 700 is detected, it is determined that a predetermined determination condition is satisfied, and the state is indicated. The signal may be output to processor 33 .

Also, the face detection unit 409 may have a function as a line-of-sight direction detection unit. The line-of-sight direction detection unit may include an infrared camera or a visible light camera that captures the face of an observer sitting in the driver's seat of the vehicle 1 , and may output the captured image to the processor 33 . The processor 33 acquires a captured image (an example of information that can estimate the line-of-sight direction) from the line-of-sight direction detection unit, and identifies the line-of-sight direction (and/or the gaze position) of the observer by analyzing the captured image. can do. The line-of-sight direction detection unit may analyze the captured image from the camera and output to the processor 33 a signal indicating the line-of-sight direction (and/or the gaze position) of the observer, which is the analysis result. Note that the method of acquiring information that can estimate the line-of-sight direction of the observer of the vehicle 1 is not limited to these, and includes an EOG (Electro-oculogram) method, a corneal reflection method, a scleral reflection method, and Purkinje image detection. may be obtained using other known gaze direction detection (estimation) techniques such as method, search coil method, infrared fundus camera method.

The vehicle exterior sensor 411 detects real objects existing around the vehicle 1 (front, side, and rear). Real objects detected by the sensor 411 outside the vehicle include, for example, obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), road surfaces of driving lanes, lane markings, roadside objects, and/or features (buildings, etc.), which will be described later. and so on. Exterior sensors include, for example, radar sensors such as millimeter-wave radar, ultrasonic radar, and laser radar, cameras, or a detection unit composed of any combination thereof, and the detection data from the one or more detection units is processed. and a processing device that performs data fusion. A conventional well-known technique is applied to object detection by these radar sensors and camera sensors. By object detection by these sensors, the presence or absence of a real object in the three-dimensional space, and if the real object exists, the position of the real object (relative distance from the vehicle 1, the traveling direction of the vehicle 1 in the front-back direction) horizontal position, vertical position, etc.), size (horizontal direction (horizontal direction), height direction (vertical direction), etc.), movement direction (horizontal direction (horizontal direction), depth direction (front-back direction)), change speed (horizontal direction (left-right direction), depth direction (front-back direction)), and/or the type of the real object may be detected. One or a plurality of vehicle exterior sensors 411 detect a real object in front of the vehicle 1 in each detection cycle of each sensor, and detect real object information (presence or absence of a real object, existence of a real object), which is an example of real object information. If so, information such as the position, size and/or type of each real object) can be output to the processor 33 . Note that the real object information may be transmitted to the processor 33 via another device (for example, the vehicle ECU 401). Also, when using a camera as a sensor, an infrared camera or a near-infrared camera is desirable so that a real object can be detected even when the surroundings are dark, such as at night. Moreover, when using a camera as a sensor, a stereo camera that can acquire distance and the like by parallax is desirable.

The brightness detection unit 413 converts the illuminance or luminance of a predetermined range of the foreground existing in front of the passenger compartment of the vehicle 1 into the outside world brightness (an example of brightness information), or converts the illuminance or luminance in the passenger compartment into the vehicle interior brightness (brightness (an example of information about The brightness detection unit 413 is, for example, a phototransistor or a photodiode, and is mounted on the instrument panel, rearview mirror, HUD device 20, or the like of the vehicle 1 shown in FIG.

The IMU 415 includes one or more sensors (e.g., accelerometers and gyroscopes) configured to sense the position, orientation, and changes thereof (velocity of change, acceleration of change) of the vehicle 1 based on inertial acceleration. can include a combination of The IMU 415 outputs detected values (the detected values include signals indicating the position and orientation of the vehicle 1, and changes thereof (change speed and change acceleration)) and the results of analyzing the detected values to a processor. 33. The analyzed result is a signal or the like indicating whether or not the detected value satisfies a predetermined determination condition. Therefore, the signal may be a signal indicating that the behavior (vibration) of the vehicle 1 is small.

The mobile information terminal 417 is a smart phone, a laptop computer, a smart watch, or other information equipment that can be carried by the observer (or other occupants of the vehicle 1). By pairing with the mobile information terminal 417, the I/O interface 31 can communicate with the mobile information terminal 417, and can read data recorded in the mobile information terminal 417 (or a server through the mobile information terminal). to get The mobile information terminal 417 has, for example, the same functions as the road information database 403 and the own vehicle position detection unit 405 described above, acquires the road information (an example of real object related information), and transmits it to the processor 33. may The mobile information terminal 417 may also acquire commercial information (an example of real object related information) related to commercial facilities near the vehicle 1 and transmit it to the processor 33 . In addition, the mobile information terminal 417 transmits schedule information of the owner of the mobile information terminal 417 (for example, an observer), incoming call information at the mobile information terminal 417, mail reception information, etc. to the processor 33, and the processor 33 and the image Processing circuitry 35 may generate and/or transmit image data for these.

The external communication device 419 is a communication device that exchanges information with the vehicle 1, for example, other vehicles connected to the vehicle 1 by vehicle-to-vehicle communication (V2V: vehicle-to-vehicle communication), pedestrian-to-vehicle communication (V2P: vehicle-to-pedestrian ) connected by pedestrians (mobile information terminals carried by pedestrians), network communication equipment connected by road-to-vehicle communication (V2I: Vehicle To roadside Infrastructure), and in a broad sense, communication with vehicle 1 (V2X : Vehicle To Everything). The external communication device 419 acquires, for example, the positions of pedestrians, bicycles, motorcycles, other vehicles (preceding vehicles, etc.), road surfaces, lane markings, roadside objects, and/or features (buildings, etc.), and sends them to the processor 33. can be output. In addition, the external communication device 419 has the same function as the vehicle position detection unit 405 described above, may acquire the position information of the vehicle 1 and transmit it to the processor 33, and furthermore may store the information of the road information database 403 described above. It may also have a function to acquire the road information (an example of real object related information) and transmit it to the processor 33 . Information acquired from the external communication device 419 is not limited to the above.

The software components stored in memory 37 include eye position detection module 502, eye position estimation module 504, eye position prediction module 506, face detection module 508, determination module 510, vehicle state determination module 512, visibility control module 514, It includes an eye-tracking image processing module 516, a graphics module 518, a light source driving module 520, an actuator driving module 522, and the like.

FIGS. 12A and 12B are flowcharts showing a method S100 for executing visibility reduction processing based on the detection result of the observer's eye position, face position, or face direction. The method S100 is performed in a HUD device 20 including a spatial light modulating element and a display control device 30 controlling this HUD device 20 . Some of the acts of method S100 described below are optionally combined, some of the steps of the acts are optionally varied, and some of the acts are optionally omitted.

First, the display control device 30 (processor 33) acquires information indicating the observer's eye position 700, face position (not shown), or face position (not shown) (step S110).

(Example of step S110)
In step S110 in some embodiments, the display control device 30 (processor 33) executes the eye position detection module 502 of FIG. (obtain eye position information indicating the eye position 700). The eye position detection module 502 detects the coordinates indicating the observer's eye position 700 (the positions in the X and Y axis directions, which is an example of eye position information), and indicates the height of the observer's eyes. Detecting coordinates (positions in the Y-axis direction, which is an example of eye position information), coordinates indicating positions in the height and depth directions of the eyes of the observer (positions in the Y- and Z-axis directions, and/or detect the coordinates indicating the observer's eye position 700 (positions in the X, Y, and Z axis directions, which is an example of eye position information). includes various software components for performing various operations related to doing.

Note that the eye position 700 detected by the eye position detection module 502 includes

positions

700R and 700L of the right and left eyes, one predetermined position out of the right eye position 700R and the left eye position 700L, the right eye position 700R and the left eye position 700L. Detectable (easily detectable) position, or a position calculated from the right eye position 700R and the left eye position 700L (for example, the middle point between the right eye position and the left eye position). good. For example, the eye position detection module 502 determines the eye position 700 based on the observation position acquired from the face detection unit 409 immediately before the timing of updating the display settings.

Further, the face detection unit 409 detects the movement direction and/or change speed of the observer's eye position 700 based on a plurality of observation positions with different detection timings of the observer's eyes obtained from the face detection unit 409, A signal indicating the direction of movement and/or the rate of change of the observer's eye position 700 may be output to the processor 33 .

(Example of step S110)
Further, the display control device 30 (processor 33) of some embodiments may acquire information (an example of eye position information) that enables estimation of eye positions by executing the eye position estimation module 504 . Information from which eye positions can be estimated (an example of eye position information) is, for example, a captured image acquired from the face detection unit 409, the position of the driver's seat of the vehicle 1, the position of the observer's face, the observer's face position, and the sitting height. or the observation positions of the eyes of multiple observers. The eye position estimation module 504 estimates the eye positions 700 of the observer of the vehicle 1 from one or more eye position estimable information. The eye position estimation module 504 uses the captured image acquired from the face detection unit 409, the position of the driver's seat of the vehicle 1, the position of the observer's face, the observer's face position, the height of the sitting height, or the eyes of a plurality of observers. includes various software components for performing various operations related to estimating the observer's eye position 700, such as estimating the observer's eye position 700, such as from the observation position of the . That is, the eye position estimation module 504 can include table data, arithmetic expressions, and the like for estimating the eye position 700 of the observer from information that can estimate the eye position.

(Example of step S110)
Also, the display control device 30 (processor 33) of some embodiments may acquire information that can predict the eye positions 700 of the observer by executing the eye position prediction module 506 . Information that can predict the observer's eye position 700 is, for example, the latest observation position obtained from the face detection unit 409, or one or more observation positions obtained in the past. Eye position prediction module 506 includes various software components for performing various operations related to predicting eye positions 700 based on information capable of predicting eye positions 700 of an observer. Specifically, for example, the eye position prediction module 506 predicts the eye position 700 of the observer at the timing when the observer visually recognizes the image to which the new display settings are applied. The eye position prediction module 506 uses one or more past observed positions, for example, using a least squares method, a prediction algorithm such as a Kalman filter, an α-β filter, or a particle filter to calculate the next value. You can make a prediction.

(Example of step S110)
Also, the display control device 30 (processor 33) of some embodiments may acquire face position information indicating the face position and face orientation information indicating the face orientation by executing the face detection module 508. . The face detection module 508 acquires face area detection data (an example of face position information, an example of face orientation information) from the face detection unit 409, and detects facial feature points from the acquired face area detection data. Then, from the arrangement pattern of the detected feature points, face position information indicating the face position of the observer and face direction information indicating the face direction are detected. Note that the face detection module 508 acquires detection data (an example of face position information, an example of face direction information) of the feature points of the face detected by the feature point detection unit 126, and extracts the acquired detection data of the feature points of the face. may be used to detect face position information and face orientation information. Also, the face detection module 508 may simply acquire the face position information and face direction information detected by the face detection unit 409 . Face direction detection processing is based on a method of calculating a face direction angle based on the positional relationship of a plurality of face parts (for example, eyes, nose, mouth, etc.), for example. Alternatively, for example, the face direction detection process is based on a method using the results of machine learning (however, the face direction detection process is not limited to these). Specifically, for example, the face position and face direction are coordinates on the X-axis along the left-right direction, a pitch angle indicating the direction of rotation about the X-axis, coordinates on the Y-axis along the up-down direction, and the Y-axis. As the yaw angle indicating the direction of rotation about the axis, the coordinate on the Z-axis along the depth direction, and the roll angle indicating the direction of rotation about the Z-axis, the position in the three-axis direction and the angle around each axis Calculated.

(Step S120)
Next, the display control device 30 (processor 33) determines whether a predetermined determination condition is satisfied by executing the determination module 510 (step S120).

(Step S130)
In step S120 in some embodiments, the display control device 30 (processor 33) executes the determination module 510 of FIG. Based on this, it is determined whether the eye position 700, face position, or face orientation satisfies a predetermined condition. In the following description, processing using the eye position 700 and face position will be mainly described. The only difference is that the eye position 700 and face position are in a positional coordinate system, and the face orientation is in an angular coordinate system. , and the processing using the change amount and change speed of the face direction, and the description of the processing using the face direction is omitted.

FIG. 13 shows (11) eye position 700 in the vertical direction or face position (or face orientation may be used) Py (Y1, Y2, Y3 . Py3 (=Y4-Y3), . Δt), Vy2 (=Py2/Δt), Vy3 (=Py3/Δt), . ) Px (X1, X2, X3 . X3-X2), Px3 (=X4-X3), . 2 is a table showing (Vx1 (=Px1/Δt), Vx2 (=Px2/Δt), Vx3 (=Px3/Δt), . . . Vx9 (=Px9/Δt)).

(Step S131)
In step S130 in some embodiments, the display control device 30 (processor 33) executes the determination module 510 of FIG. If (Vy) is fast, it is determined that the predetermined determination condition is satisfied. For example, the determination module 510 determines the change speed Vx (Vy) of the eye position 700 or face position (or face orientation) and a predetermined can be compared with a first threshold (not shown), and if the change speed Vx (Vy) of the eye position 700 or the face position (or face orientation) is faster than the predetermined first threshold, a predetermined (however, the method of determining the rate of change is not limited to this).

(Step S132)
Further, in step S130 in some embodiments, the display control device 30 (processor 33) executes the determination module 510 of FIG. If it is within the set first range (not shown), it is determined that the predetermined determination condition is satisfied. For example, the determination module 510 compares the eye position 700 or face position (or face orientation) Px (Py) with a predetermined first range (not shown) pre-stored in the memory 37 to determine the eye position. 700 or face position (or face orientation) Px (Py) is within the first range, it may be determined that a predetermined determination condition is satisfied (however, eye position 700 or face position (or (Face orientation) The method for determining position coordinates or angle coordinates is not limited to this). The first range can be set to a range separated from a predetermined reference position (not shown) by predetermined coordinates. That is, the first range is a first left range deviated from the center 205 of the eyebox 200 (an example of the predetermined reference position) by a predetermined X coordinate in the left direction (X-axis negative direction), and a right direction ( A first right range shifted by a predetermined X coordinate in the positive direction of the X axis, a first upper range shifted by a predetermined Y coordinate in the upward direction (positive direction of the Y axis), and a downward direction (negative direction of the Y axis). It is set to either a first lower range offset by a predetermined Y coordinate, and any combination thereof. According to this, the first range can be set to the outer edge away from the center 205 of the eyebox 200 or the outside of the eyebox 200 . In another embodiment, the determination module 510 determines the difference between the eye position 700 or face position (or face orientation) Px (Py) and a predetermined reference position (not shown) stored in the memory 37 in advance. is calculated, and if the difference between the eye position 700 or the face position (or face orientation) Px (Py) and the predetermined reference position is longer than a predetermined second threshold stored in advance in the memory 37, Eye position 700 or face position (or face orientation may be acceptable) Px (Py) is within a first range separated from the predetermined reference position by the second threshold or more, and it is determined that the predetermined determination condition is satisfied. You may Here, the reference position can be set at the center 205 of the eyebox 200 . In this case, the determination module 510 determines that the predetermined determination condition is satisfied if the eye position 700 or face position (or face orientation) Px (Py) is away from the center 205 of the eyebox 200 . Note that the first range can be changed as the eyebox 200 moves. The display control device 30 (processor 33) controls the first actuator 28 (and/or the second actuator 29), for example, to move the eyebox 200 by controlling the first actuator 28 (and/or the second actuator 29). The first range may be changed based on the control value of .

(Step S133)
Further, in step S130 in some embodiments, the display control device 30 (processor 33) executes the determination module 510 of FIG. If the eye position 700 or the face position (or face orientation may be used) is detected in the second range (not shown) changed by , it may be determined that the predetermined determination condition is satisfied.

In step S133, the eye position estimation module 504 in FIG. 11 sequentially updates the second range based on the fact that the eye position 700 or face position (or face orientation) Px (Py) is in a stable state. You may The second range can be set to a range separated by a predetermined coordinate from the reference position that is changed according to the eye position 700 or the face position (or face orientation). For example, if the eye position 700 or the face position (or the face orientation) Px (Py) remains roughly the same for one second or longer, the eye position estimation module 504 determines that the current state is stable. The eye position 700 or the face position (or face orientation may be used) Px (Py) is registered in the memory 37 as the reference position. A range may be set to the second range. In another embodiment, the eye position estimation module 504 determines that the stable state is present when the eye position 700 or the face position (or face orientation) Px (Py) remains substantially at the same position for one second or longer. An average value of a plurality of eye positions 700 or face positions (or face orientations) Px (Py) acquired in the past may be registered in the memory 37 as the reference position. For example, assuming that the eye position 700 or the face position (or face orientation) Px (Py) has a sample rate of 60 samples/sec and an averaging period of 0.5 sec, the eye position estimation module 504 calculates the eye position 700 Alternatively, if 60 samples of the face position (or face orientation) Px (Py) in 1 second are at approximately the same position, it is determined that the state is stable, and out of 30 samples acquired in the past 0.5 sec An average value of eye positions 700 of the latest 5 samples or face positions (or face orientations) Px (Py) may be registered in the memory 37 as the reference position. For example, the determination module 510 calculates the difference between the eye position 700 or face position (or face orientation) Px (Py) and the predetermined reference position sequentially updated and stored in the memory 37, and If the difference between the position 700 or the face position (or face orientation) Px (Py) and the predetermined reference position is longer than a predetermined third threshold stored in advance in the memory 37, a predetermined determination condition is set. It may be determined that it is sufficient.

(Step S134)
In step S130 in some embodiments, the display control device 30 (processor 33) executes the determination module 510 of FIG. If it changes continuously, it may be determined that a predetermined determination condition is satisfied. For example, the determination module 510 determines that the amount of change ΔPx in the eye position 700 in the left-right direction or the face position (or face orientation) shown in FIG. If it is detected that the movement has continuously changed in one direction (here, right direction) for a predetermined number of times (for example, two times) or more, it is determined that the predetermined determination condition is satisfied. good too.

(Step S141)
Also, in some embodiments, the determination module 510 of FIG. 11 determines whether the observer's eye position 700 (or face position) is in an unstable state, and determines whether the observer's eye position 700 (or face position) is When it is determined that the state is unstable, it may be determined that the predetermined determination condition is satisfied. The determination module 510 determines whether the stability of the observer's eye position is low (unstable). Step S141), includes various software components for performing various operations related to. That is, the determination module 510 uses threshold values, table data, arithmetic expressions, and so on.

(Example of step S141)
The eye position detection module 502 calculates the variance of position data of each of a plurality of observation positions acquired from the face detection unit 409 within a predetermined measurement time, and the determination module 510 calculates the variance of the position data calculated by the eye position detection module 502. When the variance is larger than a predetermined threshold value stored in advance in the memory 37 (or set by the operation detection unit 407), it is determined that the observer's eye position is less stable (unstable). There may be.

(Example of step S141)
) The eye position detection module 502 calculates the deviation of position data of each of the plurality of observation positions acquired from the face detection unit 409 within a predetermined measurement time, and the determination module 510 calculates the deviation of the position data calculated by the eye position detection module 502. If the deviation obtained is greater than a predetermined threshold stored in advance in the memory 37 (or set by the operation detection unit 407), it is determined that the observer's eye position is less stable (unstable) ( non-unstable) form.

(Example of step S141)
Also, without using the variance and deviation in step S141, the eye position detection module 502 divides the eye box 200 into a plurality of partial viewing zones (for example, 25 regions divided vertically into 5 and horizontally into 5). Identifiable, the stability of the observer's eye position is low (unstable) when the number of partial viewing zones to which the eye position 700 has moved per predetermined unit time exceeds a predetermined threshold. (not in an unstable state).

(Example of step S141)
Further, the eye position detection module 502 detects that the total moving distance of the eye position 700 (the sum of the distances between a plurality of observation positions obtained multiple times per unit time) per predetermined unit time exceeds a predetermined threshold value. It may be determined (not in an unstable state) that the stability of the observer's eye position is low (unstable) when it becomes longer.

(Step S142)
Further, in some embodiments, the determination module 510 of FIG. 11 determines whether the detection operation of the observer's eye position 700 is in an unstable state, and if it is determined to be in an unstable state, the predetermined It is determined that the determination condition is satisfied. The determination module 510 (10) determines whether or not the eye position 700 of the observer can be detected, and if the eye position 700 cannot be detected, determines that the state is unstable (an example of step S142), ( 20) Determining whether it can be estimated that the detection accuracy of the eye position 700 of the observer has decreased, and determining that the state is unstable if it can be estimated that the detection accuracy of the eye position 700 has decreased (step An example of S142.), (30) Determining whether or not the observer's eye position 700 is outside the eye box 200, and if it is outside the eye box 200, determining that the state is unstable (step S142). (40) Determine whether the observer's eye position 700 can be estimated to be outside the eye box 200. If it can be estimated to be outside the eye box 200, determine that the state is unstable (step S142). ), or (50) determining whether or not the observer's eye position 700 is predicted to be outside the eyebox 200, and if it is predicted to be outside the eyebox 200, it is determined to be in an unstable state. (an example of step S142), including various software components for performing various operations related to. That is, the determination module 510 determines whether or not the detection operation of the observer's eye position 700 is in an unstable state from the detection information, estimation information, or prediction information of the eye position 700. , arithmetic expressions, and the like.

(Example of step S142)
A method for determining whether or not the eye position 700 of the observer can be detected is (1) obtaining a signal from the face detection unit 409 indicating that the eye position 700 cannot be detected; (3) the eye position detection module 502 cannot detect all or part of the observation positions of the observer's eyes acquired within the period (for example, more than a predetermined number of times); Determining that the observer's eye position 700 cannot be detected (detection of the observer's eye position 700 is in an unstable state) due to the inability to detect the eye position 700 or any combination thereof. The determination method is not limited to these.).

(Example of step S142)
A method for determining that the detection accuracy of the eye position 700 of the observer is degraded includes: (1) obtaining a signal indicating that the detection accuracy of the eye position 700 is estimated to be degraded from the face detection unit 409; (2) some or all of the observation positions of the observer's eyes acquired from the face detection unit 409 within a predetermined period (for example, more than a predetermined number of times) cannot be detected; and (3) an eye position detection module. 502 cannot detect the observer's eye position 700 in normal operation; (4) the eye position estimation module 504 cannot estimate the observer's eye position 700 in normal operation; The prediction module 506 cannot predict the observer's eye position 700 in normal operation, (6) detects a decrease in the contrast of the image of the observer captured by external light such as sunlight, and (7) the hat. and accessories (including eyeglasses) are detected, (8) a part of the observer's face is not detected due to a hat or accessory (including eyeglasses), or any combination of these determining that the detection accuracy of the position 700 is degraded (the determination method is not limited to these).

(Example of step S142)
The method for determining whether or not the observer's eye position 700 is outside the eyebox 200 includes (1) a part of the observer's eye observation position acquired from the face detection unit 409 within a predetermined period (for example, (2) eye position detection module 502 detects eye position 700 of the observer outside eye box 200, or any combination thereof. , determining that the observer's eye position 700 is outside the eye box 200 (the observer's eye position 700 is in an unstable state) (the above-described determination method is not limited to these).

(Example of step S142)
The method for determining whether the observer's eye position 700 is outside the eyebox 200 is determined by: (1) After the face detection unit 409 detects the movement of the observer's eye position 700, the observer's eye position 700 is (2) the eye position detection module 502 detects the observer's eye position 700 near the boundary of the eye box 200; (3) the eye position detection module 502 detects the observer's right eye position 700R and left eye position 700L, or any combination thereof, it can be estimated that the observer's eye position 700 is outside the eyebox 200 (observer's eye position 700L). is in an unstable state) (Note that the determination method is not limited to these.).

(Example of step S142)
The method for determining whether or not the observer's eye position 700 is predicted to be outside the eyebox 200 is as follows. (2) that the eye position 700 newly detected by the eye position detection module 502 is greater than or equal to the eye position movement distance threshold previously stored in the memory 37 with respect to the previously detected eye position 700 ( The change speed of the eye position 700 is equal to or greater than the eye position change speed threshold value stored in advance in the memory 37), or any combination thereof, the observer's eye position 700 is predicted to be outside the eyebox 200. (Note that the determination method is not limited to these.).

(Step S150)
Reference is now made to FIG. 12B. After it is determined in step S120 whether the predetermined determination condition is satisfied, the display control device 30 (processor 33) updates the image displayed on the display device 40. FIG. When it is determined in step S120 that the predetermined determination condition is satisfied, the display control device 30 (processor 33) executes the visibility control module 514 to display on the display device 40 and correspond to the AR virtual image V60. Visibility reduction processing (S180) is executed to reduce the visibility of the image to be processed.

(Step S160)
The visibility control module 514 of FIG. 11 maintains the normal visibility of the AR virtual image V60 when it is determined in step S120 that the predetermined determination condition is not satisfied.

Further, in step S160, when it is determined in step S120 that the predetermined determination condition is not satisfied, the eye-following image processing module 516 of FIG. The vertical position of the virtual image V is corrected by a first correction amount Cy1, and the horizontal position of the virtual image V is corrected according to the amount of change ΔPx in the eye position in the horizontal direction. The first correction amount Cy1 (the same applies to the second correction amount Cy2, which will be described later) is a parameter that gradually increases as the eye position change amount ΔPy in the vertical direction increases. Also, the first correction amount Cy1 (the same applies to the second correction amount Cy2 described later) is a parameter that gradually increases as the perceived distance D30 set to the virtual image V increases. The first image correction processing S160 is to create an image that perfectly reproduces natural motion parallax as if the virtual image V were fixed at the set target position PT even when viewed from each eye position Py in the vertical direction. This includes position correction, and in a broader sense, it may also include correction of the image position so as to approximate natural motion parallax. That is, the first image correction processing S160 sets the display position of the virtual image V to the position of the intersection of the virtual image display area VS and the straight line connecting the target position PT set in the virtual image V and the observer's eye position 700. Align (bring the display position of the virtual image V closer).

(Step S170)
When it is determined in step S120 that the predetermined determination condition is satisfied, the display control device 30 (processor 33) executes at least the visibility reduction process (step S180), and additionally performs eye-following image processing. Module 516 may perform a second image correction process (step S190) described below.

(Step S180)
When it is determined in step S120 that the predetermined determination condition is satisfied, the visibility control module 514 performs visibility reduction processing (step S180) to reduce the visibility of the AR virtual image V60 from the normal visibility in step S160. at least Here, decreasing the visibility means decreasing the brightness of the AR virtual image V60, increasing the transmittance of the AR virtual image V60 (bringing it closer to transmission), and decreasing the brightness of the AR virtual image V60 (bringing it closer to black). , lowering the saturation of the AR virtual image V60 (bringing it closer to an achromatic color), and any combination thereof. The display control device 30 (processor 33) controls the visibility of the image displayed by the display 50 through gradation control in the display 50 and local or overall illumination control in the light source unit 60, thereby making the image Controls the visibility of the corresponding AR virtual image V60.

(Example of step S181)
When it is determined in step S120 that the predetermined determination condition is satisfied, the visibility control module 514 performs a first visibility decrease that abruptly decreases the visibility of the AR virtual image V60 from the normal visibility in step S160. Execute the process. More specifically, when it is determined that the predetermined determination condition is satisfied, the visibility control module 514 sets the desired visibility (lower than the normal visibility) stored in the memory 37 from the normal visibility. ).

Note that the visibility control module 514 can rapidly reduce the visibility of the AR virtual image V60 immediately after it is determined in step S120 that the predetermined determination condition is satisfied.

Also, the visibility control module 514 can rapidly reduce the visibility of the AR virtual image V60 after a predetermined period of time has elapsed after it is determined that the predetermined determination condition is satisfied.

(Example of step S181)
In another embodiment, when it is determined in step S120 that the predetermined determination condition is satisfied, the visibility control module 514 gradually reduces the visibility of the AR virtual image V60 to the normal visibility in step S160 over time. A second visibility lowering process for further lowering the visibility is executed. More specifically, when it is determined that the predetermined determination condition is satisfied, the visibility control module 514 sets the desired visibility (lower than the normal visibility) stored in the memory 37 from the normal visibility. ) gradually.

(Step S183)
When it is determined in step S120 that the predetermined determination condition is satisfied, the visibility control module 514 sets the visibility of the AR virtual image V60 to the changing speed of the eye position 700 or the face position (or face orientation may be used). 2nd visibility lowering processing which lowers to different visibility according to is performed. More specifically, when the eye position 700 or the face position (or face orientation may be changed), the visibility control module 514 sets the visibility significantly lower than the normal visibility (including non-display) when the change speed is fast. ), and if the change speed of the eye position 700 or the face position (or the face orientation is also acceptable) is slow, the visibility is set slightly lower than the normal visibility. Note that the visibility level lower than the normal visibility is not limited to two stages, but may be three stages or more according to the change speed of the eye position 700 or the face position (or face orientation may be used). may Visibility control module 514 may also substantially reduce the level of visibility continuously in response to an increasing rate of change in eye position 700 or face position (or face orientation).

(Step S185)
The visibility control module 514 reduces the visibility of the AR virtual image V60 to different visibility depending on the eye position 700 or the face position when it is determined in step S120 that the predetermined determination condition is satisfied. Execute visibility reduction processing. More specifically, the visibility control module 514 sets the visibility significantly lower than normal when the eye position 700 or face position is far away from a predetermined reference position (eg, the center 205 of the eyebox 200). Visibility (including non-display) is set, and when the eye position 700 or the face position is slightly away from the predetermined reference position, the visibility is set slightly lower than normal visibility. Note that the visibility level lower than normal visibility is not limited to two stages, and may be three stages or more according to the eye position 700 or the face position. Visibility control module 514 may also substantially continuously change the level of visibility depending on eye position 700 or face position.

Note that in steps S181, S183, and S185, the visibility control module 514 reduces the visibility of the entire AR virtual image V60.

In another example, in steps S181, S183, and S185, the visibility control module 514 reduces the visibility of part of the AR virtual image V60. For example, the visibility control module 514 reduces the visibility of the AR virtual image V60 having a perceptual distance D30 set to the AR virtual image V60 longer than a predetermined threshold value (not shown) (performs visibility reduction processing), The visibility of the AR virtual image V60 that is shorter than a predetermined threshold does not have to be lowered (the visibility lowering process is not executed). Further, when the perceived distance D30 set for the AR virtual image V60 is long, the visibility control module 514 sets the visibility (including non-display) significantly lower than the normal visibility, and sets the AR virtual image V60. If the perceived distance D30 is short, the visibility may be slightly lower than the normal visibility.

(Step S187)
The graphic module 518 of FIG. 11 displays an AR-related virtual image related to part or all of the AR virtual image V60 whose visibility is lowered in steps S181, S183, and S185. FIG. 14 is a diagram showing an example of the foreground visually recognized by the observer, the AR virtual image when the visibility reduction process is executed, and the AR-related virtual image while the host vehicle is running. In the example of FIG. 14, the HUD device 20 creates a distant virtual image V1 (for example, a virtual image V64- V65), and a near virtual image V2 perceived at a position closer than the first distance (for example, virtual images V61 to V63 shown in FIG. 14), and the processor 33 performs visibility reduction processing S170 on the distant virtual image V1. The visibility is reduced by executing the visibility reduction processing S170, and the visibility reduction processing S170 is not executed for the neighboring virtual image V2. Also, in the example of FIG. 14, the processor 33 displays an AR-related virtual image V80 (for example, V81 shown in FIG. 14) related to a part (virtual image V64) of the distant virtual image V1 with reduced visibility.

The eye-following image processing module 516 in FIG. 11 switches between the first image correction process (step S160) and the second image correction process (step S190) based on the determination result in step S120. good too.

(Step S190)
If it is determined in step S120 that the predetermined determination condition is satisfied, the eye-following image processing module 516 of FIG. 11 performs eye position 700 (or face position ) is reduced.

(Example of step S190)
If it is determined in step S120 that the predetermined determination condition is satisfied, the eye-following image processing module 516 of FIG. The correction amount may be reduced only with respect to the amount of change in .

In some embodiments, the eye-following image processing module 516 of FIG. 11 corrects the vertical position of the virtual image V by a second correction amount Cy2 corresponding to the amount of change ΔPy in the eye position in the vertical direction. The horizontal position of the virtual image V is corrected by a second correction amount Cx2 corresponding to the eye position change amount ΔPx. Here, the eye-following image processing module 516 makes the second correction amount Cy2 smaller than the first correction amount Cy1 for the eye position change amount ΔPy in the vertical direction in the first image correction process (step S160). , the second correction amount Cx2 is set to be the same as the first correction amount Cy1 for the eye position change amount ΔPx in the horizontal direction in the first image correction process (step S160). Specifically, for example, when the first correction amount Cy1 for the eye position change amount ΔPy in the vertical direction is 100%, the second correction amount Cy2 for the same eye position change amount ΔPy in the vertical direction is 25%. When the first correction amount Cx1 for the eye position change amount ΔPx in the horizontal direction is 100%, the second correction amount Cx2 for the same eye position change amount ΔPx in the horizontal direction is also 100%. In a broad sense, the second correction amount Cy2 should be smaller than the first correction amount Cy1, and therefore should be less than 100% of the first correction amount Cy1. is preferably less than 60% with respect to

(Example of step S190)
If it is determined in step S120 that the predetermined determination condition is satisfied, the eye-following image processing module 516 of FIG. The correction amount may be set to zero only for the amount of change in .

In some embodiments, when it is determined in step S120 that the predetermined determination condition is satisfied, the eye-following image processing module 516 of FIG. The second correction amount Cy2 may be set to zero. For example, the eye position followable image processing module 511 may correct the position of the virtual image V in the horizontal direction only according to the amount of change ΔPx in the eye position in the horizontal direction.

(Example of step S190)
If it is determined in step S120 that the predetermined determination condition is satisfied, the eye-following image processing module 516 of FIG. The correction amount may be smaller than the amount.

In some embodiments, the eye-following image processing module 516 of FIG. 11 corrects the vertical position of the virtual image V by a second correction amount Cy2 corresponding to the amount of change ΔPy in the eye position in the vertical direction. The horizontal position of the virtual image V is corrected by a second correction amount Cx2 corresponding to the eye position change amount ΔPx. Here, the eye-following image processing module 516 makes the second correction amount Cy2 smaller than the first correction amount Cy1 for the eye position change amount ΔPy in the vertical direction in the first image correction process (step S160), The second correction amount Cx2 is made smaller than the first correction amount Cy1 for the eye position change amount ΔPx in the horizontal direction in the first image correction process (step S160). Specifically, for example, when the first correction amount Cy1 for the eye position change amount ΔPy in the vertical direction is 100%, the second correction amount Cy2 for the same eye position change amount ΔPy in the vertical direction is 25%. When the first correction amount Cx1 for the eye position change amount ΔPx in the horizontal direction is 100%, the second correction amount Cx2 for the same eye position change amount ΔPx in the horizontal direction is also 25%.

Further, the eye-following image processing module 516 in FIG. 11 in some embodiments sets the image position correction amount Cx2 with respect to the eye position change amount ΔPx in the horizontal direction in the second image correction process (step S190) to the is set lower than the image position correction amount Cx1 for the eye position change amount ΔPx in the horizontal direction in the image correction process (step S160) in step S160, and the first correction amount Cy1 for the eye position change amount ΔPy in the vertical direction is set to a second correction amount Cy1. (Cx2/Cx1>Cy2/Cy1).

FIG. 16 is a flow diagram showing a method S200 for executing the visibility increasing process while executing the visibility decreasing process. The method S200 is performed in a HUD device 20 including a spatial light modulating element and a display controller 30 controlling this HUD device 20 .

In some embodiments, the display control device 30 (processor 33) determines whether a predetermined cancellation condition is satisfied (step S210), and when it is determined that the cancellation condition is satisfied, visibility reduction processing (step S180) to the visibility increasing process (step S220).

The predetermined cancellation condition includes that a predetermined period of time (for example, 20 seconds) has passed since the visibility reduction process (step S180) was started. The visibility control module 514 executes timekeeping after transitioning to the visibility reduction process (step S180), and the predetermined time stored in advance in the memory 37 (or set by the operation detection unit 407) has passed. In this case, it may be determined that the cancellation condition is satisfied.

Also, the predetermined cancellation condition may include that the predetermined determination condition is no longer satisfied in step S120. That is, the predetermined cancellation condition is that at least one of steps S131 to S134 and steps S141 to S143 has changed from a state in which the predetermined determination condition is satisfied to a state in which the predetermined determination condition is no longer satisfied. detecting. Further, the predetermined cancellation condition may include that a predetermined time (for example, 20 seconds) has elapsed since the predetermined determination condition was not satisfied in step S120.

(Step S220)
The display control device 30 (processor 33) executes visibility increasing processing when it is determined in step S210 that the cancellation condition is satisfied.

(Example of step S181)
When it is determined in step S210 that the canceling condition is satisfied, the visibility control module 514 abruptly changes the visibility of the AR virtual image V60 from the visibility set in the visibility reduction process (step S170) to normal visibility. A first visibility increasing process for increasing the visibility is executed. More specifically, when it is determined that the cancellation condition is satisfied, the visibility control module 514 switches the visibility set in the visibility reduction process (step S170) to the normal visibility.

Note that the visibility control module 514 can rapidly increase the visibility of the AR virtual image V60 immediately after it is determined in step S210 that the cancellation condition is satisfied.

Also, the visibility control module 514 of another embodiment can rapidly increase the visibility of the AR virtual image V60 after a predetermined period of time has passed after it is determined that the cancellation condition is satisfied.

(Example of step S221)
In another embodiment, when it is determined in step S210 that the cancellation condition is satisfied, the visibility control module 514 gradually decreases the visibility of the AR virtual image V60 over time through visibility reduction processing (step S170). A first visibility lowering process is executed to increase the visibility from the set visibility to the normal visibility. More specifically, when it is determined that the cancellation condition is satisfied, the visibility control module 514 gradually switches the visibility set in the visibility reduction process (step S170) to the normal visibility.

(Step S223)
When it is determined in step S210 that the cancellation condition is satisfied, the visibility control module 514 performs a second visibility increase that increases the visibility of the AR virtual image V60 to a different visibility depending on the change speed of the eye position. Execute the process. More specifically, when the speed of change in eye position is fast, the visibility control module 514 sets the visibility to be slightly higher than the visibility set in the visibility reduction process (step S170). If the speed is slow, the visibility is made significantly higher than the visibility set in the visibility lowering process (step S170) (slightly lower than normal visibility or the same as normal visibility). Note that the visibility level higher than the visibility set in the visibility reduction process (step S170) is not limited to two stages, and may be three stages or more according to the change speed of the eye position. good. Visibility control module 514 may also substantially continuously increase the level of visibility as the rate of change in eye position decreases.

(Step S225)
When it is determined in step S210 that the cancellation condition is satisfied, the visibility control module 514 executes a third visibility increase process for increasing the visibility of the AR virtual image V60 to different visibility depending on the eye position. do. More specifically, the visibility control module 514 sets the eye position in the visibility reduction process (step S170) when the eye position is far away from a predetermined reference position (for example, the center 205 of the eyebox 200). If the eye position is slightly away from the predetermined reference position, the visibility is significantly higher than the visibility (normal or the same as normal visibility). Note that the visibility level higher than the visibility set in the visibility reduction process (step S170) is not limited to two stages, and may be three stages or more depending on the eye position. Visibility control module 514 may also substantially continuously change the level of visibility according to eye position.

Note that in steps S221, S223, and S225, the visibility control module 514 increases the visibility of all the AR virtual images V60 whose visibility has been lowered in the visibility reduction process (step S170).

In another example, in steps S221, S223, and S225, the visibility control module 514 sequentially increases the visibility of the plurality of AR virtual images V60 whose visibility has been reduced in the visibility reduction process (step S170). Let For example, the visibility control module 514 may increase the visibility of the distant virtual image V1 after a predetermined period of time has elapsed after increasing the visibility of the near virtual image V2.

(Step S227)
The graphic module 518 of FIG. 11 hides the AR-related virtual images related to part or all of the AR virtual image V60 whose visibility has been increased in steps S221, S223, and S225.

If it is determined in step S210 that the cancellation condition is satisfied, the eye-following image processing module 516 of FIG. from the image correction process (step S190) to the first image correction process (step S160) in which the amount of positional correction of the image with respect to the amount of change in the eye position 700 (or the face position) is greater than that of the second image correction process (step S190). switch to

Refer to FIG. 11 again. Graphics module 518 of FIG. 11 includes various known software components for performing image processing, such as rendering, to generate image data, and to drive display device 40 . The graphic module 518 also controls the type (moving image, still image, shape), arrangement (positional coordinates, angle), size, display distance (in the case of 3D), visual effect (for example, luminance, (transparency, saturation, contrast, or other visual properties) may be included. The graphic module 518 stores the type of image (one example of display parameters), the positional coordinates of the image (one example of display parameters), the angle of the image (the pitching angle with the X direction as the axis, and the Y direction as the axis). yaw rate angle around the axis, rolling angle around the Z direction, etc., which are examples of display parameters), image size (one example of display parameters), image color (hue, saturation , one example of display parameters set by brightness, etc.), and the intensity of perspective expression of an image (one of display parameters set by the position of a vanishing point, etc.) to make an image visible to the observer. Data may be generated to drive the display 50 .

The light source driving module 520 includes various known software components for performing driving the light source unit 24 . The light source driving module 520 can drive the light source unit 24 based on the set display parameters.

Actuator drive module 522 includes various known software components for performing driving first actuator 28 and/or second actuator 29. One actuator 28 and a second actuator 29 can be driven.

FIG. 16 is a diagram illustrating the HUD device 20 according to some embodiments, in which the eyebox 200 can be vertically moved by rotating the relay optical system 80 (curved mirror 81). The display control device 30 (processor 33) in some embodiments rotates the relay optical system 80 (curved mirror 81) by, for example, controlling the first actuator 28 to move the eyebox 200 up and down (Y-axis direction). Typically, when the eyebox 200 is arranged in the relatively upper eyebox 201 shown in FIG. is arranged in the lower eyebox 203 shown in FIG. 16, the position of the virtual image display area VS is the relatively upper position indicated by the symbol VS3. The display control device 30 (processor 33) in some embodiments executes the eye-following image processing module 516 to determine if the eyebox 200 is located above a predetermined height threshold (in other words, the first When the control value of the actuator 28 exceeds the actuator control threshold such that the eyebox 200 is positioned above the predetermined height threshold), the amount of change in eye position (or face position) is displayed on the display 50. The image position correction amount Cy may be reduced. Note that the actuator driving module 522 may automatically change the height of the eyebox 200 according to the vertical position of the eye position 700 (or the face position). , the height of the eyebox 200 may be changed. That is, the eye-following image processing module 516 provides information on the height of the eyebox 200, information on the control values of the actuators, information on the upper and lower eye positions 700 (or face positions) that can automatically adjust the height of the eyebox 200. The correction amount Cx of the position of the image displayed on the display 50 with respect to the amount of change in the eye position (or face position) is determined from the information on the position of the direction or the operation information from the operation detection unit 407 for adjusting the height of the eyebox 200. It can include threshold values, table data, arithmetic expressions, and the like for switching (Cy).

Further, the display control device 30 (processor 33) in some embodiments adjusts the control value of the first actuator 28 as the eyebox 200 becomes higher (in other words, the control value of the first actuator 28 becomes higher). ), the correction amount Cx (Cy) of the position of the image displayed on the display 50 with respect to the amount of change in eye position (or face position) may be reduced stepwise or continuously. That is, the eye-following image processing module 516 provides information on the height of the eyebox 200, information on the control values of the actuators, information on the upper and lower eye positions 700 (or face positions) that can automatically adjust the height of the eyebox 200. The position of the image to be displayed on the display 50 with respect to the amount of change in eye position (or face position) in the vertical direction can be determined from information about the position of the direction or operation information from the operation detection unit 407 for adjusting the height of the eyebox 200 . It may include threshold values, table data, arithmetic expressions, etc. for adjusting the correction amount Cx(Cy).

As described above, the display control device 30 of the present embodiment includes at least the display device 40 that displays an image, and the relay optical system 80 that projects the light of the image displayed by the display device 40 onto the projection target member, A display control device 30 that executes display control in a HUD device 20 that allows a vehicle user to visually recognize a virtual image of an image superimposed on the foreground, comprising one or more processors 33, a memory 37, and stored in the memory 37, and one or more computer programs configured to be executed by one or more processors 33, the processor 33 calculating the user's vehicle vertical eye position (and/or face position) Py Also, the eye position (and/or face position) Px in the horizontal direction of the vehicle is obtained, and the display device 40 is based on at least the eye position (or face position) Py in the vertical direction and the eye position (or face position) Px in the horizontal direction. First image correction processing (step S160) for correcting the position of the image displayed on the display device based on at least the eye position (or face position) Py in the vertical direction and the eye position (or face position) Px in the horizontal direction 40, and the second correction amount Cy2 of the image position with respect to the change amount ΔPy of the eye position (or face position) in the vertical direction is the first image correction process (step S160). is smaller than the first correction amount Cy1 of the image position with respect to the change amount ΔPy of the eye position (or face position) in the vertical direction, or displayed on the display device 40 based on at least the eye position (or face position) Px in the horizontal direction and a second image correction process S170 in which the image position is corrected and the correction amount of the image position with respect to the change amount ΔPy of the eye position (or face position) in the vertical direction is set to zero.

Further, in some embodiments, the processor 33 determines that (1) the horizontal eye position (or face position) Px has changed continuously in one direction, (2) the vertical eye position (and/or A change in eye position (or face position) in the horizontal direction and a change in eye position (and/or face position) in the horizontal direction are detected. and (3) changes in eye position (or face position) Py in the vertical direction and eye position (or face position) Px in the horizontal direction. is detected, and at this time, if at least one of the following conditions is satisfied: that the amount of change ΔPy in the eye position (or face position) in the vertical direction is less than a predetermined second threshold, the second Image correction processing S170 may be selected. As a result, when the observer moves the eye position (face position) in the horizontal direction, the movement of the eye position (face position) in the vertical direction, which the observer is not aware of, is detected. It is possible to reduce the feeling of discomfort.

Further, in some embodiments, after the processor 33 becomes unable to acquire the eye position (and/or face position) Py in the vertical direction and/or the eye position (and/or face position) Px in the horizontal direction, If a change in eye position (or face position) Py in the vertical direction and a change in eye position (or face position) Px in the horizontal direction are detected, the second image correction processing S170 may be selected. In other words, in the first image correction process (step S160), the processor 33 selects one of the eye position Py in the vertical direction, the face position Py in the vertical direction, the eye position Px in the horizontal direction, and the face position Px in the horizontal direction. If one or a plurality of images has been detected but has not been detected, the process may proceed to the second image correction process S170.

In some embodiments, in the second image correction processing S170, the processor 33 performs the eye position (or face position) Py in the vertical direction and the eye position (or face position) in the horizontal direction after a predetermined time has elapsed. ) Px, the position of the image displayed on the display device 40 is corrected based on at least Px. It may be switched to a third image correction process S182 that is smaller than the first correction amount Cy1 in the correction process (step S160) and larger than the second correction amount Cy2 in the second image correction process S170. .

Further, in some embodiments, the processor 33 detects in the second image correction processing S170 that the amount of change ΔPy in the eye position (or face position) in the vertical direction has become larger than a predetermined third threshold. In this case, the position of the image displayed on the display device 40 is corrected based on at least the eye position (or face position) Py in the vertical direction and the eye position (or face position) Px in the horizontal direction, and the eye position (or face position) in the vertical direction is corrected. or face position) is smaller than the first correction amount Cy1 in the first image correction process (step S160) and the second image correction process You may switch to 3rd image correction process S182 larger than 2nd correction amount Cy2 at the time of S170.

Further, in some embodiments, the processor 33 changes the third correction amount Cy3 in the third image correction process S182 from the first correction amount Cy1 in the first image correction process (step S160). It may change over time so that it approaches.

Further, in some embodiments, the HUD device 20 is configured to create a distant virtual image V1 (for example, virtual images V64 to V65 shown in FIG. 9) perceived at a position a first distance away from a reference point set on the vehicle side. ), and a near virtual image V2 (for example, virtual images V61 to V63 shown in FIG. 9) perceived at a position separated by a second distance shorter than the first distance, and the processor 33 displays the far virtual image V1 at a predetermined distance. According to the satisfaction of the determination condition, the display is switched between the first image correction processing (step S160) and the second image correction processing S170, and the neighboring virtual image V2 is displayed when the predetermined determination condition is satisfied. Instead, it may be displayed in the second image correction processing S170. That is, the determination module 510 determines each virtual image from the position information of the real object 300 with which the virtual image V obtained from the vehicle exterior sensor 411 is associated, the information about the perceived distance D30 set to the virtual image V based on the position information of the real object 300, and the like. It may include thresholds, table data, arithmetic expressions, etc. for determining V as a distant virtual image V1 or a near virtual image V2.

Further, in some embodiments, if the area in which the HUD device 20 can display the virtual image V is assumed to be a virtual image display area VS, as shown in FIG. and a lower virtual image V70 displayed in a lower area VSβ below the upper area VSα including the lower end VSb of the virtual image display area VS, and the processor 33 displays , the upper virtual image V60 is displayed by switching between the first image correction processing (step S160) and the second image correction processing S170 according to the satisfaction of a predetermined determination condition, and the lower virtual image V70 is displayed. , the image may be displayed without positional correction based on the eye position or the face position.

In some embodiments, as shown in FIG. 9, the HUD device 20 includes an AR virtual image V60 that changes its display position according to the position of a real object existing in the foreground of the vehicle, and an AR virtual image V60 that changes its display position according to the position of the real object. and the non-AR virtual image V70 whose display position is not changed, and the processor 33 performs the first image correction processing (step S160) and the second The non-AR virtual image V70 may be displayed without image position correction based on the eye position or the face position.

The operations of the processing processes described above can be implemented by executing one or more functional modules of an information processing device such as a general-purpose processor or an application-specific chip. These modules, combinations of these modules, and/or combinations with known hardware that can replace their functions are all within the scope of protection of the present invention.

The functional blocks of vehicle display system 10 are optionally implemented in hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is noted that the functional blocks illustrated in FIG. 11 may optionally be combined or separated into two or more sub-blocks to implement the principles of the described embodiments. will be understood by those skilled in the art. Accordingly, the description herein optionally supports any possible combination or division of the functional blocks described herein.

Reference Signs List 1: vehicle 2: projected part 5: dashboard 6: road surface 10: vehicle display system 20: HUD device (head-up display device)
21 : Light exit window 22 : Housing 24 : Light source unit 28 : First actuator 29 : Second actuator 30 : Display control device 31 : I/O interface 33 : Processor 35 : Image processing circuit 37 : Memory 40 : Display device 50 : display 51 : spatial light modulator 52 : optical layer 80 : relay optical system 81 : curved mirror 205 : center 300 : real object 401 : vehicle ECU
403 : Road information database 405 : Vehicle position detection unit 407 : Operation detection unit 409 : Face detection unit 411 : Exterior sensor 413 : Brightness detection unit 417 : Portable information terminal 419 : External communication device 502 : Eye position detection module 504 : Eye position estimation module 506 : Eye position prediction module 508 : Face detection module 510 : Determination module 511 : Eye position followability image processing module 512 : Vehicle state determination module 514 : Visibility control module 516 : Eye followability image processing module 518 : Graphic module 520: Light source driving module 522: Actuator driving module PT: Target position Px: Eye position (face position)
Py: eye position (face position)
V: Virtual image V1: Distant virtual image V2: Nearby virtual image V60: AR virtual image V70: Non-AR virtual image V80: AR-related virtual image VS: Virtual image display area Vx: Velocity of change Vy: Velocity of change ΔPx: Amount of change ΔPy: Amount of change

Claims

A display device (40) for displaying an image, and a head-up for allowing a vehicle user to visually recognize a virtual image of the image superimposed on the foreground by projecting the light of the image displayed by the display device (40) onto a projection target member. A display control device (30) for performing display control in a display device (20),
one or more processors (33);
a memory (37);
one or more computer programs stored in said memory (37) and configured to be executed by said one or more processors (33);
Said processor (33)
obtaining eye position-related information including at least one of the user's eye position, face position, and face orientation;
Displaying an AR virtual image (V60) on the head-up display device (20),
performing eye-following image correction processing for correcting the position of the image displayed on the display (40) based at least on the eye position-related information in order to adjust the display position of the AR virtual image (V60);
determining whether the eye position-related information or the detection operation of the eye position-related information satisfies a predetermined determination condition based on the eye position-related information;
When it is determined that the determination condition is satisfied, executing a visibility reduction process for reducing the visibility of the AR virtual image (V60),
A display controller (30).
The judgment condition is
a condition of change speed of at least one of the eye position, the face position, and the face orientation;
at least one of a coordinate condition of at least one of the eye position, the face position, and the face orientation, and a moving time condition of at least one of the eye position, the face position, and the face orientation including,
A display control device (30) according to claim 1.
The judgment condition is
At least one of the eye position, the face position, and the face direction changes at a high speed;
coordinates of at least one of the eye position, the face position, and the face orientation are within a predetermined range;
Continuous change of at least one of the eye position, the face position, and the face orientation,
A display control device (30) according to claim 1.
Conditions for detecting the eye position-related information are:
At least one of the eye position, the face position, and the face orientation cannot be detected, and a decrease in detection accuracy of at least one of the eye position, the face position, and the face orientation is detected. including any one
A display control device (30) according to claim 3.
The processor (33), in the visibility reduction process,
reduce visibility differently depending on at least one of the eye position, the face position, and the face orientation;
A display control device (30) according to claim 1.
The processor (33), in the visibility reduction process,
Decrease the visibility differently according to the change speed of at least one of the eye position, the face position, and the face orientation;
A display control device (30) according to claim 1.
Said processor (33)
If it is determined that the above determination conditions are not satisfied,
performing a first eye-following image correction process for correcting the position of the image displayed on the display (40) based at least on the eye position or the face position;
If it is determined that the above determination conditions are satisfied,
correcting the position of the image displayed on the display device (40) based at least on the eye position or the face position, wherein a second correction amount of the position of the image with respect to the amount of change in the eye position or the face position is , smaller than the first correction amount of the position of the image for the same change amount of the eye position or the face position in the first eye-following image correction process, or the eye position or the face position in the vertical direction and the amount of correction of the position of the image with respect to at least one of the amount of change in the eye position in the horizontal direction and the amount of change in the face position in the horizontal direction, performing a second image correction process.
A display control device (30) according to claim 1.
Said processor (33)
determining whether the eye position-related information or the detection operation of the eye position-related information satisfies a predetermined release condition based on the eye position-related information;
When it is determined that the cancellation condition is satisfied, further executing a visibility increase process for increasing the visibility of the AR virtual image (V60) that has been subjected to the visibility reduction process,
A display control device (30) according to claim 1.
The processor (33), in the visibility increasing process,
Different visibility is raised according to at least one of the eye position, the face position, and the face orientation,
A display control device (30) according to claim 8.
The processor (33), in the visibility increasing process,
Different visibility is raised according to the change speed of at least one of the eye position, the face position, and the face orientation;
A display control device (30) according to claim 8.
A display device (40) for displaying an image, and a head-up for allowing a vehicle user to visually recognize a virtual image of the image superimposed on the foreground by projecting the light of the image displayed by the display device (40) onto a projection target member. A display device (20),
one or more processors (33);
a memory (37);
one or more computer programs stored in said memory (37) and configured to be executed by said one or more processors (33);
Said processor (33)
obtaining eye position-related information including at least one of the user's eye position, face position, and face orientation;
Display an AR virtual image (V60),
performing eye-following image correction processing for correcting the position of the image displayed on the display (40) based at least on the eye position-related information in order to adjust the display position of the AR virtual image (V60);
determining whether the eye position-related information or the detection operation of the eye position-related information satisfies a predetermined determination condition based on the eye position-related information;
When it is determined that the determination condition is satisfied, executing a visibility reduction process for reducing the visibility of the AR virtual image (V60),
A head-up display device (20).
A display device (40) for displaying an image, and a head-up for allowing a vehicle user to visually recognize a virtual image of the image superimposed on the foreground by projecting the light of the image displayed by the display device (40) onto a projection target member. A display control method for a display device (20), comprising:
obtaining eye position-related information including at least one of the user's eye position, face position, and face orientation;
displaying an AR virtual image (V60) on the head-up display device (20);
executing eye-following image correction processing for correcting the position of the image displayed on the display (40) based at least on the eye position-related information, in order to adjust the display position of the AR virtual image (V60); When,
Determining whether the eye position-related information or the detection operation of the eye position-related information satisfies a predetermined determination condition based on the eye position-related information;
When it is determined that the determination condition is satisfied, executing a visibility reduction process that reduces the visibility of the AR virtual image (V60),
Display control method.