JP2023034899A - Display control device, head-up display device, and display control method - Google Patents

Abstract

To display an image which easily recognizes information while reducing visual fatigue and/or discomfort.SOLUTION: Disclosed is a processor 33 which is provided on a vehicle and performs the display control on a head-up display device which allows an observer to visually recognize a virtual image of an image by projecting the image onto a member to be projected, which generates the image FU expressing a depth feeling by giving parallax to both eyes of the observer, and which performs a first display processing in which the expression depth E of the image FU is brought loser to a virtual image distance D10 when a predetermined first condition is satisfied, including the lapse of a first predetermined time ΔT1 after at least the image FU is displayed.SELECTED DRAWING: Figure 5

Description

The present disclosure relates to a display control device, a head-up display device, and a head-up display device that are used in a mobile object such as a vehicle and superimpose an image on the foreground of the mobile object (actual view in the forward direction of the mobile object seen from the vehicle occupant). It relates to a display control method and the like.

In Patent Document 1, display light projected onto a projection target such as a front windshield of a vehicle is reflected toward an occupant (observer) of the vehicle inside the vehicle, thereby presenting a virtual image to the observer. A visual head-up display device (an example of a virtual image display device) is described. In particular, the head-up display device described in Patent Document 1 describes a head-up display device that allows the vehicle speed situation to be recognized by changing the expression depth of the virtual image (the distance to the virtual image perceived by the observer). ing. The image (virtual image) can attract the visual attention of the observer by moving to the near side or the far side from the observer's point of view, or can add distance information to the displayed image. It can be added. A head-up display device that changes the depth of expression in this way has a stereoscopic display device inside.

For example, a stereoscopic display device uses a method using a lens array (lenticular method, see Patent Document 2, for example), a method using a parallax barrier (parallax barrier method, see Patent Document 3, for example), or a method using parallax images. There is a time-division display method (see Patent Documents 4, 5, and 6, for example).

In the head-up display device, the display light of the left viewpoint image is irradiated to the position of the left eye, and the display light of the right viewpoint image is irradiated to the position of the right eye, so that the left viewpoint image and the right viewpoint image with parallax are visually recognized by the observer. . The images perceived in the observer's brain by fusing these images appear as if they were displayed at a distance different from the display distance at which the left-viewpoint image (virtual image) and the right-viewpoint image (virtual image) are actually formed. It is perceived as

JP 2019-127065 A JP 2006-115198 A JP 2014-150304 A JP 2011-107382 A JP-A-2000-236561 JP-A-2000-4452

When the representation depth of the virtual object due to the binocular parallax is in front of or behind the virtual image plane, the observer's eyes try to focus the lens on the distance of the virtual image plane. On the other hand, the convergence of both eyes is set to the expression depth in front of or behind the virtual image plane. That is, the distance of accommodation of the observer's eye (the distance corresponding to the virtual image plane) and the distance of convergence (the distance of the depth of representation of the virtual object) are different. or discomfort.

A summary of certain embodiments disclosed herein follows. It should be understood that these aspects are presented only to provide the reader with an overview of these particular embodiments and are not intended to limit the scope of this disclosure. Indeed, the present disclosure can encompass combinations of various aspects not described below.

An overview of the present disclosure relates to providing a display control device, a head-up display device, a display control response, and the like that display an image with high visibility that is easy to see and that information is easy to recognize. More specifically, it also relates to displaying an image that facilitates recognition of information while reducing visual fatigue and/or discomfort.

Therefore, the display control device, the head-up display device, the display control method, and the like described in this specification employ the following means in order to solve the above problems. This embodiment displays an image (virtual image) that expresses a sense of depth by giving parallax to both eyes of the observer, and displays a predetermined time including the elapse of a first predetermined time after the image is displayed. The gist of this is to execute a first display process for bringing the expression depth of the image closer to the virtual image distance when the first condition is satisfied.

Therefore, the display control device according to the first embodiment of the present invention executes display control in a head-up display device that is provided in a vehicle and projects an image onto a projection target member so that an observer can visually recognize a virtual image of the image. a display controller comprising one or more processors, a memory, and one or more computer programs stored in the memory and configured to be executed by the one or more processors The processor generates an image that expresses a sense of depth by giving parallax to both eyes of the observer, and generates an image that expresses a sense of depth. is satisfied, a first display process is executed to bring the expression depth of the image closer to the virtual image distance. In this case, first, the representation depth of the image is set to the front and/or the back of the position (virtual image distance) where the image (virtual image) is displayed (imaged) due to the parallax given to the image. If a predetermined first condition including that one predetermined period of time has passed, the virtual image distance is approached. As a result, the first embodiment of the present invention has the advantage of making it difficult to view an image with a long depth of expression for longer than a predetermined time.

In the display control device according to a preferred embodiment, the processor gradually or continuously changes from the first representation depth to the second representation depth over time in the first display process, and changes the representation depth to the second representation depth. After reaching , the representation depth of the image is maintained, and the second representation depth is the depth equal to the virtual image distance. In this case, there is an advantage that the contradiction in vergence adjustment can be reduced while maintaining stereoscopic vision for at least a certain period of time, and finally the contradiction in vergence adjustment can be eliminated. Also, in the display control device according to a preferred embodiment, the second representation depth is the depth between the first representation depth and the virtual image distance. This has the advantage of reducing vergence accommodation conflict while preserving stereoscopic vision.

In the display control device in a preferred embodiment, the processor switches from the first representation depth to the second representation depth in the first display process, and maintains the representation depth of the image after switching to the second representation depth. , the second representation depth is a depth between the first representation depth and the virtual image distance or a depth equal to the virtual image distance. In this case, there is an advantage that the congestion accommodation contradiction can be quickly reduced or eliminated.

In any display control device in another preferred embodiment, the second representation depth is set differently according to the first representation depth of the image. According to this, since the second representation depth can be varied according to the difference in the first representation depth, it becomes easier to visually distinguish the plurality of images by the difference in the second representation depth, and the information can be recognized. can be made easier.

In any of the display control devices in another preferred embodiment, the processor changes from the second representation depth to the first representation depth if a predetermined second condition is met after the first display process. 2 display processing is further executed. According to this, even after the expression depth of the image approaches the virtual image distance by the first display processing, the image giving priority to the depth expression can be displayed again, and the information indicated by the image can be easily recognized.

In one of the display control devices according to another preferred embodiment, the first display processing is executed when the depth between the first representation depth and the virtual image distance is longer than a predetermined distance. An image with a long depth between representation depth and virtual image distance also has a large vergence accommodation contradiction. Visual fatigue and/or discomfort can be effectively reduced by performing the first display processing on such an image in which the contradiction of vergence adjustment becomes large.

In any of the display control devices in another preferred embodiment, the processor controls the processing of the second content having a first representation depth with a greater depth between the first content and the virtual image distance than the first content. , and in the first condition, the first predetermined time of the second content is shorter than the first predetermined time of the first content. According to this, the convergence adjustment contradiction is quickly reduced for an image with a large vergence adjustment contradiction, and the depth expression of the image is lengthened for an image with a small vergence adjustment contradiction. While reducing visual fatigue and/or discomfort, it is possible to suppress deterioration of information identifiability.

In any one of the display control devices according to another preferred embodiment, the processor, before the first display processing, changes the representation depth of the image and changes the size of the image to perceive a change in perspective,
After the first display processing, the change in perspective is perceived by changing the size of the image while maintaining the depth of expression of the image. As a result, it is possible to improve the identifiability of information while reducing visual fatigue and/or discomfort.

In any display control device in another preferred embodiment, the second representation depth is set constant regardless of the first representation depth of the image. According to this, even if there is a difference in the first representation depth, the second representation depth is the same. Even if the line of sight moves over the image, the image can be visually recognized with the same convergence, and the visual load can be reduced.

FIG. 1 is a diagram showing an application example of a vehicle display system to a vehicle. FIG. 2 is a diagram showing the configuration of the head-up display device. FIG. 3 is a diagram showing an example of a foreground visually recognized by an observer and a perceptual image superimposed on the foreground and displayed while the vehicle is running. FIG. 4 is a diagram conceptually showing the positional relationship between the left-viewpoint virtual image and the right-viewpoint virtual image displayed on the virtual image plane, and the perceived image perceived by the observer from the left-viewpoint virtual image and the right-viewpoint virtual image. be. FIG. 5 shows the virtual image distance of the virtual image plane on which the left-viewpoint virtual image and the right-viewpoint virtual image are displayed, and how the observer perceives the left-viewpoint virtual image and the right-viewpoint virtual image after the first display processing is executed. FIG. 4 is a diagram conceptually showing the positional relationship between the depth of expression of a perceptual image and FIG. FIG. 6 shows the virtual image distance of the virtual image plane on which the left-viewpoint virtual image and the right-viewpoint virtual image are displayed, and the perception of the observer by the left-viewpoint virtual image and the right-viewpoint virtual image after the first display processing is executed. FIG. 4 is a diagram conceptually showing the positional relationship between the depth of expression of a perceptual image and FIG. FIG. 7A is a diagram showing changes in depth of expression in the first display process. FIG. 7B is a diagram showing changes in depth of expression in the first display process. FIG. 8 is a diagram showing changes in depth of expression in the first display process. FIG. 9 is a diagram showing changes in depth of expression in the first display process. FIG. 10 is a diagram showing changes in depth of expression in the second display process. FIG. 11 is a block diagram of a vehicle display system of some embodiments. FIG. 12 is a flow diagram illustrating a method of performing operations that vary the depth of representation of a perceived image, according to some embodiments.

1-12 below provide a description of the configuration and operation of an exemplary vehicular display system. In addition, the present invention is not limited by the following embodiments (including the contents of the drawings). Of course, modifications (including deletion of constituent elements) can be added to the following embodiments. In addition, in the following description, descriptions of known technical matters are omitted as appropriate in order to facilitate understanding of the present invention.

Please refer to FIG. FIG. 1 is a diagram showing an example of the configuration of a vehicle virtual image display system including a parallax 3D-HUD device. In FIG. 1, the left-right direction of a vehicle (an example of a moving body) 1 (in other words, the width direction of the vehicle 1) is the X axis (the positive direction of the X axis is the left direction when the vehicle 1 faces forward). ), and the vertical direction (in other words, the height direction of the vehicle 1) along a line segment that is perpendicular to the left-right direction and is perpendicular to the ground or a surface corresponding to the ground (here, the road surface 6) is the Y-axis (Y-axis is the upward direction), and the front-rear direction along a line segment perpendicular to each of the left-right direction and the up-down direction is the Z-axis (the positive direction of the Z-axis is the straight-ahead direction of the vehicle 1). This point also applies to other drawings.

As illustrated, a vehicle display system 10 provided in a vehicle (vehicle) 1 detects the positions of the left eye 700L and the right eye 700R of an observer (typically, a driver sitting in the driver's seat of the vehicle 1) and the line-of-sight direction. An eye position detection unit (line-of-sight detection unit) 409 for detecting a pupil (or face) to be detected, an exterior sensor 411 configured by a camera (for example, a stereo camera) for imaging the front (in a broad sense, the surroundings) of the vehicle 1, a head-up It has a display device (hereinafter also referred to as HUD device) 20 and a display control device 30 that controls the HUD device 20 .

FIG. 2 is a diagram showing one aspect of the configuration of the head-up display device. The HUD device 20 is installed, for example, in a dashboard (reference numeral 5 in FIG. 1). The HUD device 20 accommodates the stereoscopic display device 40, the relay optical system 80, and the stereoscopic display device 40 and the relay optical system 80, and can emit the display light K from the stereoscopic display device 40 from the inside to the outside. A housing 22 having a light exit window 21 is provided.

The stereoscopic display device 40 is assumed here to be a parallax 3D display device. This stereoscopic display device (parallax type 3D display device) 40 is a display device 50 which is a naked-eye stereoscopic display device using a multi-viewpoint image display method capable of controlling depth expression by visually recognizing a left-viewpoint image and a right-viewpoint image. and a light source unit 60 functioning as a backlight.

The display 50 has a light modulation element 51 that modulates the illumination light from the light source unit 60 to generate an image, and, for example, a lenticular lens or a parallax barrier (parallax barrier). Left eye display light such as light rays K11, K12 and K13 (reference symbol K10 in FIG. 1) and right eye display light such as light rays K21, K22 and K23 (in FIG. 1). symbol K20) and an optical layer (an example of a light beam splitting section) 52. The optical layer 52 includes optical filters such as lenticular lenses, parallax barriers, lens arrays, and microlens arrays. In the embodiments, the optical layer 52 is not limited to the optical filter described above, and includes all types of optical layers arranged on the front or rear surface of the light modulation element 51 . However, this is an example and is not limited.

Also, in the stereoscopic display device 40, the light source unit 60 is configured with a directional backlight unit (an example of the light beam separating unit) instead of or in addition to the optical layer (an example of the light beam separating unit) 52. , left-eye display light such as light rays K11, K12, and K13 (reference symbol K10 in FIG. 1), and right-eye display light such as light rays K21, K22, and K23, etc. (reference symbol K20 in FIG. 1). , may be emitted. Specifically, for example, the display control device 30, which will be described later, causes the light modulation element 51 to display a left-viewpoint image when the directional backlight unit emits illumination light directed toward the left eye 700L. When the left-eye display light K10 such as K11, K12, and K13 is directed toward the viewer's left eye 700L, and the directional backlight unit emits illumination light toward the right eye 700R, the right viewpoint image is displayed on the light modulation element 51. By displaying, right-eye display light K20 such as light rays K21, K22, and K23 for the right eye is directed to the left eye 700R of the observer. However, this is an example and is not limited.

The display control device 30, which will be described later, performs, for example, image rendering processing (graphic processing), display device driving processing, and the like, so that the display light K10 for the left eye of the left viewpoint image V10 and the display light K10 for the left eye and the display light K10 for the right eye 700R of the observer's left eye 700L are displayed. By directing the display light K20 for the right eye of the right viewpoint image V20 and adjusting the left viewpoint image V10 and the right viewpoint image V20, the aspect of the perception image FU displayed by the HUD device 20 (perceived by the observer) is controlled. be able to. Note that the display control device 30, which will be described later, controls the display (display device 50) so as to reproduce a light field that (approximately) reproduces light rays output in various directions from a point in a fixed space. may

The relay optical system 80 has curved mirrors (concave mirrors, etc.) 81 and 82 that reflect the light from the stereoscopic display device 40 and project the image display light K10 and K20 onto the windshield (projection target member) 2 . However, it may further include other optical members (refractive optical members such as lenses, diffractive optical members such as holograms, reflective optical members, or combinations thereof).

In FIG. 1, the stereoscopic display device 40 of the HUD device 20 displays images with parallax (parallax images) for the left and right eyes. Each parallax image is displayed as V10 and V20 formed on a virtual image display surface (virtual image formation surface) VS, as shown in FIG. The focus of each eye of the observer (person) is adjusted so as to match the position of the virtual image display area VS. Note that the position of the virtual image display area VS is referred to as a "virtual image distance (or imaging position)", and a predetermined reference position (for example, the center 205 of the eyebox 200 of the HUD device 20, the observer's viewpoint position, or , a specific position of the vehicle 1, etc.) to the virtual image display area VS (see symbol D10 in FIG. 4) is referred to as a virtual image distance (imaging distance).

However, in reality, since the human brain fuses each image (virtual image), the human is positioned further back than the virtual image distance (for example, due to the convergence angle between the left viewpoint image V10 and the right viewpoint image V20). It is a fixed position, and the position perceived as being farther away from the observer as the angle of convergence decreases) is recognized as a perceptual image (here, an arrowhead figure for navigation) FU is displayed. do. Note that the perceptual image FU may be referred to as a "stereoscopic virtual image", and may also be referred to as a "stereoscopic image" when the "image" is taken in a broad sense to include virtual images. It may also be referred to as a "stereoscopic image", "3D display", or the like. Note that the HUD device 20 can display the left viewpoint image V10 and the right viewpoint image V20 so that the perceived image FU can be visually recognized at a position closer to the viewer than the virtual image distance.

Next, refer to FIGS. 3 and 4. FIG. FIG. 3 is a diagram showing an example of a foreground visually recognized by an observer and a perceptual image displayed in association with a real object existing in the foreground while the vehicle 1 is running. FIG. 4 shows the virtual image distance on the virtual image plane on which the left-viewpoint virtual image and the right-viewpoint virtual image are displayed, and how the left-viewpoint virtual image and the right-viewpoint virtual image affect the distance of the observer before the first display process described later is executed. FIG. 3 is a diagram conceptually showing the positional relationship between the depth of expression of a perceptual image to be perceived;

In FIG. 3 , the vehicle 1 is traveling on a straight road (road surface) 6 . The HUD device 20 is installed inside the dashboard 5 . Display light K (K10, K20) is projected from the light exit window 21 of the HUD device 20 onto the projected portion (front windshield of the vehicle 1) 2. FIG. In the example of FIG. 3, a partially elliptical arc-shaped image FU arranged below a preceding vehicle (real object) 300 traveling in front of the vehicle 1 as seen from the observer is displayed.

As shown in the left diagram of FIG. 4, the HUD device 20 (1) detects the left eye 700L detected by the eye position detection unit 409, and at such a position and angle that the projection target 2 reflects the left eye 700L. to form a first left viewpoint image V11 at a predetermined position in the virtual image display area VS seen from the left eye 700L; Right-eye display light K20 is emitted to the projection target portion 2 at such a position and angle as shown in FIG. The first image FU1 perceived by the first left-viewpoint image V11 and the first right-viewpoint image V21 having parallax is located at a position behind the virtual image display area VS by a distance D21 (from the reference position described above). position separated by a distance D31).

Similarly, as shown in the right diagram of FIG. 4 , the HUD device 20 (1) detects the left eye 700L detected by the eye position detection unit 409 at such a position and angle as to be reflected by the projection target unit 2 . Left-eye display light K10 is emitted to the projection unit 2 to form a second left-viewpoint image V12 at a predetermined position in the virtual image display area VS as seen from the left eye 700L, and (2) to the right eye 700R, the projected unit 2 Right-eye display light K20 is emitted to the projection target 2 at a position and angle such that it is reflected by the right eye 700R, and a second right-viewpoint image V22 is formed at a predetermined position of the virtual image display area VS seen from the right eye 700R. image. The second image FU2 perceived by the second left viewpoint image V12 and the second right viewpoint image V22 having parallax is located at a position behind the virtual image display area VS by a distance D22 (from the reference position described above). position separated by a distance D31).

Specifically, the distance (virtual image distance D10) from the reference position to the virtual image display area VS is set to, for example, "4 m", and the first distance shown in the left diagram of FIG. The distance to the image FU1 (first expression depth E11) is set to, for example, a distance of “7 m”, and the distance from the reference position to the second image FU2 shown in the right diagram of FIG. The representation depth E12) is set to a distance of, for example, "10 m". However, this is an example and is not limited.

The first expression depths E11 and E12 are the line segment connecting the left eye 700L and the left viewpoint image V10 (V11, V12) (in other words, the line-of-sight direction from the left eye 700L toward the left viewpoint image V10), the right eye 700R and the right eye 700R. It is determined according to the size of the angle (convergence angle θc) between the line segment connecting the viewpoint image V20 (V21, V22) (in other words, the line-of-sight direction from the right eye 700R to the right viewpoint image V20).

The first convergence angle θc1 shown in the left diagram of FIG. 4 is larger than the second convergence angle θc2 shown in the right diagram of FIG. Therefore, the distance from the reference position to the first image FU1 (first depth of expression E11) is shorter than the distance from the reference position to the second image FU2 (first depth of expression E12). In the example of FIG. 4, the perceived image FU changes its size while changing the representation depth according to the distance of the preceding vehicle (real object) 300 existing in the foreground, so that the change in perspective is perceived. Specifically, when the real object 300 is close, the perceived image FU has a relatively short expression depth E11 and a relatively large size as shown in the left diagram of FIG. 4, thereby giving an impression of being close. On the other hand, when the real object 300 is far away, as shown in the right diagram of FIG. 4, the perceptual image FU has an expression depth E12 deeper than the expression depth E11 and a relatively small size (smaller than the left diagram of FIG. 4). , gives a distant impression.

When the display control device 30 (HUD device 20) of the present embodiment satisfies a predetermined first condition including at least the elapse of a first predetermined time ΔT1 after the predetermined perceptual image FU is displayed, A first display process for bringing the representation depth of the perceived image FU closer to the virtual image distance D10 is executed.

FIG. 5 shows the virtual image distance of the virtual image plane on which the left-viewpoint virtual image and the right-viewpoint virtual image are displayed, and how the observer perceives the left-viewpoint virtual image and the right-viewpoint virtual image after the first display processing is performed. FIG. 2 is a diagram conceptually showing the positional relationship between the expression depth of a perceptual image and the positional relationship. Here, the expression depth of the image before the first display processing is executed is a first expression depth E10, and the expression depth of the image after the first display processing is executed is a second expression depth E20. called. When the representation depth of the perceptual image FU is the first representation depth E11 (the left diagram in FIG. 4), and the first condition is satisfied and the first display process is executed, the representation depth shown in the left diagram in FIG. As shown, it is changed to a second representation depth E21 closer to the virtual image distance D10 than the first representation depth E11. Further, when the representation depth of the perceived image FU is the first representation depth E12 (the right figure in FIG. 4) which is farther from the virtual image distance D10 than the first representation depth E11 (the left figure in FIG. 4), the first representation depth E12 (the right figure in FIG. 4) condition is satisfied and the first display process is executed, as shown in the right diagram of FIG. . That is, the first display processing of the present embodiment sets the representation depth of the perceived image FU to a virtual image distance greater than the first representation depth E10, regardless of the depth of the first representation depth E10 before the first display processing. Change to one second representation depth E20 closer to D10. Note that the first display process of the present embodiment is not limited to this.

In some embodiments, the second representation depth E20 after the first display processing may be set differently according to the first representation depth E10 of the perceived image FU. That is, the second depth of expression E20 after the second display processing is set to be deeper as the first depth of expression E10 before the first display processing is deeper. FIG. 6 shows the virtual image distance of the virtual image plane on which the left-viewpoint virtual image and the right-viewpoint virtual image are displayed, and the left-viewpoint virtual image and the right-viewpoint virtual image after the first display processing in some embodiments is executed. FIG. 2 is a diagram conceptually showing the positional relationship between the expression depth of a perceived image perceived by an observer and the positional relationship. When the representation depth of the perceived image FU is the first representation depth E12 (the right figure in FIG. 4), which is farther from the virtual image distance D10 than the first representation depth E11 (the left figure in FIG. 4), the first condition is satisfied and the first display process is executed, as shown in the right diagram of FIG. It is changed to E22. For example, the second representation depth E20 may be proportional to the first representation depth E10. In another example, the distance D20 from the virtual image display area VS to the perceived image FU at the second depth of expression E20 is proportional to the distance D20 from the virtual image display area VS to the perceived image FU at the first depth of expression E10. may be Further, in another example, the convergence angle θc at the second representation depth E20 may be proportional to the convergence angle θc at the first representation depth E10. In this way, the second depth of expression E20 after the second display processing is set to be deeper as the first depth of expression E10 before the first display processing is deeper. It becomes easy to visually distinguish a plurality of images due to the difference in the number of images, and information can be easily recognized.

FIGS. 7A to 10 are diagrams showing changes in depth of expression in the first display process. In the display control device 30 (HUD device 20) in some embodiments, the processor 33 switches from the first representation depth E10 to the second representation depth E20 in the first display process, as shown in FIG. 7A, After switching to the second representation depth E20, the representation depth of the image FU is maintained. Here, the second representation depth E20 is the depth between the first representation depth E10 and the virtual image distance D10. However, the present invention is not limited to this, and the second representation depth E20 may be equal to the virtual image distance D10 as shown in FIG. 7B.

As shown in FIG. 8, in one of the display control devices 30 (HUD devices 20) in another preferred embodiment, the processor 33 sets the second representation depth E20 to be constant regardless of the first representation depth E10 of the image. set to The processor 33 can change the first representation depth E10 of the image continuously (dynamically) or stepwise before the first display processing, but after the first display processing, the depth E10 before the first display processing can be changed. The constant second representation depth is maintained regardless of the first representation depth E10. Before the first display processing, the processor 33 appropriately sets the first representation depth E10 of the image, for example, continuously (dynamically) or stepwise according to the difference in distance to the real object 300 with which the image is associated. obtain. Furthermore, preferably, the processor 33 changes the size of the image while changing the depth of expression of the image (first depth of expression E10) before the first display processing, thereby making the change in perspective perceived. On the other hand, after the first display processing, the processor 33 maintains the second representation depth E20 of the image at a constant second representation depth E20 even if there is a difference in the distance to the real object 300 with which the image is associated, for example. can be set to In this case, preferably, after the first display processing, the processor 33 maintains the depth of representation of the image (the second depth of representation E20) constant, and changes the size of the image to perceive changes in perspective. Let Note that the change in perspective of the image (change in depth of expression and/or change in size) is not limited to the change in the distance of the real object 300 with which the image is associated. It can be appropriately set continuously (dynamically) or stepwise according to the desired effect.

Also, in any display control device 30 (HUD device 20) in another preferred embodiment, the processor 33 sets the second representation depth E20 for a plurality of images having different first representation depths E10 of the images. is also set constant.

As shown in FIG. 9, in any display control device 30 (HUD device 20) in another preferred embodiment, the processor 33, in the first display process, changes the depth of representation from the first representation depth E10 to the second representation depth E20 , and after reaching the second representation depth, the representation depth of the image is maintained. As a result, it is possible to reduce the contradiction of vergence accommodation while maintaining stereoscopic vision. In the first display process, the processor 33 continuously (or stepwisely) changes from the first representation depth E10 to the second representation depth E20 over a predetermined time ΔT2. In some embodiments, the predetermined time ΔT2 is constant regardless of the depth of the first representation depth E10. That is, when the depth between the first representation depth E10 and the virtual image distance D10 is long, the change rate of the representation depth within the predetermined time ΔT2 is large. is short, the change rate of the representation depth within the predetermined time ΔT2 is small. Also, in some embodiments, the predetermined time ΔT2 varies depending on the depth of the first representation depth E10. That is, when the depth between the first representation depth E10 and the virtual image distance D10 is long, the predetermined time ΔT2 is long, and when the depth between the first representation depth E10 and the virtual image distance D10 is short, The predetermined time ΔT2 is shortened. Note that the predetermined time ΔT2 may be appropriately changed by the observer's operation of the operation detection unit 407, which will be described later. In addition, the display control device 30 may automatically change the predetermined time ΔT2 according to the conditions of the vehicle, the conditions outside the vehicle, and the conditions of the observer.

Any display control device 30 (HUD device 20) according to another preferred embodiment executes the first display process when the depth between the first representation depth E10 and the virtual image distance D10 is longer than a predetermined distance. However, when the depth between the first expression depth E10 and the virtual image distance D10 is shorter than a predetermined distance, the first display processing is not executed. An image with a long depth between representation depth and virtual image distance also has a large vergence accommodation contradiction. By executing the first display process on such an image with a large contradiction of vergence accommodation, it is possible to effectively reduce visual fatigue and/or discomfort. can be maintained.

In any display control device 30 (HUD device 20) in another preferred embodiment, the processor 33 provides a first content having a greater depth between the first content and the virtual image distance D10 than the first content. and a second content having a depth of representation E12, and in a first condition, the first predetermined time ΔT1 of the second content is shorter than the first predetermined time ΔT1 of the first content. According to this, by quickly reducing the contradiction of vergence accommodation for an image with a large contradiction of vergence accommodation, and by sustaining the depth expression of the image for a long time for an image with a small contradiction of vergence accommodation, It is possible to effectively reduce visual fatigue and/or discomfort while suppressing a decrease in the identifiability of information.

In one of the display control devices (HUD device 20) in another preferred embodiment, the processor 33 reduces the second representation depth E20 to the first depth if a predetermined second condition is met after the first display processing. is further executed to change the expression depth E10. According to this, even after the representation depth of the image approaches the virtual image distance D10 by the first display process, depth representation is prioritized.

FIG. 11 is a block diagram of a virtual image display system for vehicles, according to some embodiments. The display controller 30 comprises one or more I/O interfaces 31 , one or more processors 33 , one or more image processing circuits 35 and one or more memories 37 . Various functional blocks depicted in FIG. 3 may be implemented in hardware, software, or a combination of both. FIG. 11 is only one embodiment and the illustrated components may be combined into fewer components or there may be additional components. For example, image processing circuitry 35 (eg, a graphics processing unit) may be included in one or more processors 33 .

As shown, processor 33 and image processing circuitry 35 are operatively coupled with memory 37 . More specifically, the processor 33 and the image processing circuit 35 execute a program stored in the memory 37 to generate and/or transmit image data, for example, the vehicle display system 10 (display device). 40) can be controlled. Processor 33 and/or image processing circuitry 35 may include at least one general purpose microprocessor (e.g., central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA). , or any combination thereof. Memory 37 includes any type of magnetic media such as hard disks, any type of optical media such as CDs and DVDs, any type of semiconductor memory such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

As shown, processor 33 is operatively coupled with I/O interface 31 . The I/O interface 31 communicates (CAN communication). The communication standard adopted by the I/O interface 31 is not limited to CAN. : MOST is a registered trademark), a wired communication interface such as UART or USB, or a personal area network (PAN) such as a Bluetooth network, a local network such as an 802.11x Wi-Fi network. It includes an in-vehicle communication (internal communication) interface, which is a short-range wireless communication interface within several tens of meters such as an area network (LAN). In addition, the I / O interface 31 is a wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e base (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, An external communication (external communication) interface such as a wide area communication network (for example, Internet communication network) may be included according to a cellular communication standard such as 5G.

As shown, the processor 33 is interoperably coupled with the I/O interface 31 to communicate with various other electronic devices and the like connected to the vehicle display system 10 (I/O interface 31). can be given and received. The I/O interface 31 includes, for example, a vehicle ECU 401, a road information database 403, a vehicle position detection unit 405, an operation detection unit 407, an eye position detection unit 409, an external sensor 411, a brightness detection unit 413, an IMU 415, portable information A terminal 417, an external communication device 419, etc. are operatively coupled. Note that the I/O interface 31 may include a function of processing (converting, calculating, and analyzing) information received from other electronic devices or the like connected to the vehicle display system 10 .

Display device 40 is operatively coupled to processor 33 and image processing circuitry 35 . Accordingly, the image displayed by light modulating element 51 may be based on image data received from processor 33 and/or image processing circuitry 35 . The processor 33 and image processing circuit 35 control the image displayed by the light modulation element 51 based on the information obtained from the I/O interface 31 .

The eye position detection unit 409 may include a camera such as an infrared camera that detects the eye position 700 of the observer sitting in the driver's seat of the vehicle 1 , and may output the captured image to the processor 33 . The processor 33 acquires a captured image (an example of information that can estimate the eye position 700) from the eye position detection unit 409, and analyzes the captured image by a method such as pattern matching to determine the eye positions of the observer. 700 coordinates may be detected.

In addition, the eye position detection unit 409 detects an analysis result obtained by analyzing the captured image of the camera (for example, the observer's eye position 700 is set within the eye box 200 (or within an area including the eye box 200 and its surroundings). ) may be output to the processor 33. Note that the method of acquiring the eye position 700 of the observer of the vehicle 1 or the information that can estimate the eye position 700 of the observer is not limited to these, and a known eye position detection (estimation) technique is used. may be obtained by

Further, the eye position detection unit 409 detects the movement speed and/or movement direction of the observer's eye position 700 (or the observer's head), and detects the observer's eye position 700 (or the observer's head). ) may be output to the processor 33 to indicate the speed and/or direction of movement.

The software components stored in the memory 37 include an eye position detection module 502, an eye position estimation module 504, an eye position prediction module 506, a condition determination module 508, a display parameter setting module 510, a brightness detection module 511, a graphic module 512, A light source drive module 514 and an actuator drive module 516 are included.

FIG. 12 is a flow diagram illustrating a method S100 of performing operations that vary the depth of representation of a perceived image, according to some embodiments. The method S100 is performed in the display device 40 (light modulation element 51) and the display control device 30 that controls this display device 40 (light modulation element 51). Some acts in method S100 are optionally combined, some acts are optionally reordered, and some acts are optionally omitted.

As described below, method S100 provides, among other things, a way to vary the depth of representation of a perceptual image.

The eye position detection module 502 of FIG. 11 detects the eye position 700 of the observer. The eye position detection module 502 detects the coordinates indicating the lateral position of the observer's eyes (the position in the X-axis direction, which is an example of a signal indicating the eye position 700). and the coordinates indicating the position in the depth direction (the position in the X and Z axis directions, which is an example of a signal indicating the eye position 700), and the coordinates indicating the eye height of the observer (Z , which is an example of a signal indicating the eye position 700), and coordinates indicating the position in the eye height and depth directions of the observer (positions in the Y and Z axis directions, and/or coordinates indicating the eye position 700 of the observer (positions in the X, Y, and Z axis directions and an example of a signal indicating the eye position 700). ) includes various software components for performing various operations related to detecting.

Note that the eye position 700 detected by the eye position detection module 502 is one of predetermined positions 700R and 700L of the right eye and left eye, the right eye position 700R and the left eye position 700L, the right eye position 700R and the left eye position. 700L, or a position calculated from the right eye position 700R and the left eye position 700L (for example, the midpoint between the right eye position and the left eye position). For example, the eye position detection module 502 determines the eye position 700 based on the observation position acquired from the eye position detection unit 409 immediately before the timing of updating the display settings.

Further, the eye position detection module 502 detects the movement direction and/or the movement speed of the eye position 700 of the observer based on the observation positions of the eyes of a plurality of observers obtained from the eye position detection unit 409, and A signal indicating the moving direction and/or moving speed of the eye position 700 may be output to the processor 33 .

The eye position estimation module 504 obtains information from which eye positions can be estimated. The information that can estimate the eye position is, for example, the captured image acquired from the eye position detection unit 409, the position of the driver's seat of the vehicle 1, the position of the observer's face, the height of the sitting height, or the eye position of a plurality of observers. observation position and so on. The eye position estimating module 504 estimates the eye position 700 of the observer of the vehicle 1 from the information that can estimate the eye position. The eye position estimation module 504 uses the captured image acquired from the eye position detection unit 409, the position of the driver's seat of the vehicle 1, the position of the observer's face, the height of the sitting height, or the observation positions of the eyes of a plurality of observers. , estimating the eye position 700 of the observer, etc., for performing various operations related to estimating the eye position 700 of the observer. That is, the eye position estimation module 504 can include table data, arithmetic expressions, and the like for estimating the eye position 700 of the observer from information that can estimate the eye position.

The eye position estimation module 504 combines the eye position 700 with the eye observation position acquired from the eye position detection unit 409 immediately before the timing of updating the display parameters and one or more previously acquired eye observation positions. Based on this, for example, it may be calculated by a method such as a weighted average.

The eye position prediction module 506 acquires information that can predict the eye position 700 of the observer. The information that can predict the eye position 700 of the observer is, for example, the latest observation position obtained from the eye position detection unit 409, one or more observation positions obtained in the past, or the like. Eye position prediction module 506 includes various software components for performing various operations related to predicting eye positions 700 based on information capable of predicting eye positions 700 of an observer. Specifically, for example, the eye position prediction module 506 predicts the eye position 700 of the observer at the timing when the observer visually recognizes the image to which the new display settings are applied. The eye position prediction module 506 uses one or more past observed positions, for example, using a least squares method, a prediction algorithm such as a Kalman filter, an α-β filter, or a particle filter to calculate the next value. You can make a prediction.

A condition determination module 508 determines whether a first condition for starting the first display process is satisfied. The first condition includes at least that a first predetermined time ΔT1 has passed since the image having the first representation depth E10 was displayed. In addition to this, the first condition is that the importance of the information indicated by the image is low (lowered), that it can be determined that the observer has recognized the image from the observer's line of sight, etc., and that the vehicle is running. The condition may include a change in risk in the environment in which it is located, or a combination of these. The first condition may include various conditions other than these.

Also, the condition determination module 508 determines whether a second condition for starting the second display process is satisfied. The second condition is, for example, that the importance of the information indicated by the image having the second depth of expression E20 has increased, that the information indicated by the image has changed, or that the observer's line of sight has changed. The condition may include the fact that it can be determined that the image is not recognized, the fact that the risk has changed in the environment in which the vehicle is running, or a combination of these. The second condition may include various conditions other than these.

The display parameter setting module 510 sets the eye positions 700 detected by the eye position detection module 502, the eye positions 700 estimated by the eye position estimation module 504, or the eye positions 700 predicted by the eye position prediction module 506 to a predetermined number. , and set display parameters corresponding to the eye position 700 . The display parameter setting module 510 determines where the X-axis coordinate, the Y-axis coordinate, the Z-axis coordinate, or a combination thereof indicated by the eye position 700 belongs to a plurality of spatial regions, and determines the space to which the eye position 700 belongs. It includes various software components for performing various operations related to setting display parameters corresponding to the desired region. That is, the eye position estimation module 504 may include table data, arithmetic expressions, etc. for specifying display parameters from the X-axis coordinates, Y-axis coordinates, Z-axis coordinates indicated by the eye position 700, or a combination thereof.

The types of display parameters are as follows: (1) an image layout that has a predetermined positional relationship with a real object positioned outside the vehicle 1 when viewed from the eye position 700 (the display device 40 is used to control the image layout); (2) parameters for pre-distorting the image to reduce image distortion that may be caused by, for example, the virtual image optics 90 as viewed from the eye position 700; parameters that control the display device 40 to pre-distort the image displayed on the display device 40, also called warping parameters); Display parameters other than the parameters for directional display that does not direct the light of the image (or weakens the light) (parameters for controlling the display device 40, parameters for controlling the light sources of the display device 40, actuator control (4) parameters for perceiving an image with a desired depth of expression when viewed from the eye position (parameters for controlling the display device 40, the light source of the display device 40 parameters that control, parameters that control actuators, or parameters that include a combination thereof.), and the like. However, the display parameters set (selected) by the display parameter setting module 510 may be parameters that are preferably changed according to the observer's eye position 700, and the types of display parameters are not limited to these.

The display parameter setting module 510 selects one or more display parameters corresponding to the eye position 700 for one type of display parameter. For example, if the eye positions 700 include a right eye position 700R and a left eye position 700L, the display parameter setting module 510 sets the display parameter corresponding to the right eye position 700R and the display parameter corresponding to the left eye position 700L. one can be selected. On the other hand, the eye position 700 is, for example, a predetermined one of the right eye position 700R and the left eye position 700L, one of the right eye position 700R and the left eye position 700L that can be detected (easily detected), or If there is one position calculated from the right eye position 700R and the left eye position 700L (for example, the midpoint between the right eye position 700R and the left eye position 700L), the display parameter setting module 510 creates one display corresponding to the eye position 700. parameters can be selected. Note that the display parameter setting module 510 may also select display parameters corresponding to the eye position 700 and display parameters set around the eye position 700 . That is, display parameter setting module 510 may select three or more display parameters corresponding to eye position 700 .

The display parameter setting module 510 of the present embodiment executes the first display processing (S130) on the image determined to satisfy the first condition by the condition determination module 508, thereby increasing the depth of expression from the first depth of expression E10 to the second depth of expression. to E20. In addition, the display parameter setting module 510 of the present embodiment executes the second display processing (S150) on the image determined to satisfy the second condition by the condition determination module 508, thereby reducing the depth of expression from the second depth of expression E20 to Change to the first representation depth E10.

Please refer to FIG. 11 again. Graphics module 512 includes various known software components for performing image processing, such as rendering, to generate image data, and to drive display device 40 . The graphic module 512 also controls the type, arrangement (positional coordinates, angle), size, display distance (in the case of 3D), visual effects (for example, brightness, transparency, saturation, contrast, or and other visual characteristics), may include various known software components. The graphic module 512 stores the type of image (one example of display parameters), the positional coordinates of the image (one example of display parameters), the angle of the image (pitch angle with the X direction as the axis, Y direction as the axis, yaw rate angle around the axis, rolling angle around the Z direction, etc., which are examples of display parameters), image size (one example of display parameters), image color (hue, saturation , brightness, etc.), image data can be generated and the light modulation element 51 can be driven so as to be visually recognized by the observer.

Light source driving module 514 includes various known software components for performing driving light source unit 60 . The light source driving module 514 can drive the light source unit 60 based on the set display parameters.

Actuator drive module 516 includes various known software components for performing the driving of first actuator 28 and/or second actuator 29. One actuator 28 and a second actuator 29 can be driven.

The operations of the processing processes described above may be implemented by executing one or more functional modules of an information processing device such as a general purpose processor or an application specific chip. These modules, combinations of these modules, and/or combinations with known hardware that can replace their functions are all within the scope of protection of the present invention.

The functional blocks of vehicle display system 10 are optionally implemented in hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is noted that the functional blocks illustrated in FIG. 3 may optionally be combined or separated into two or more sub-blocks to implement the principles of the described embodiments. will be understood by those skilled in the art. Accordingly, the description herein optionally supports any possible combination or division of the functional blocks described herein.

Reference Signs List 1: vehicle 2: projected part 5: dashboard 6: road surface 10: vehicle display system 20: HUD device (head-up display device)
21 : Light exit window 22 : Housing 28 : First actuator 29 : Second actuator 30 : Display control device 31 : I/O interface 33 : Processor 35 : Image processing circuit 37 : Memory 40 : Stereoscopic display device (display device)
50: Display 51: Light modulation element 52: Optical layer 60: Light source unit 80: Relay optical system 90: Virtual image optical system 200: Eye box 205: Center 300: Real object 401: Vehicle ECU
403 : road information database 405 : own vehicle position detection unit 407 : operation detection unit 409 : eye position detection unit 411 : outside sensor 413 : brightness detection unit 417 : portable information terminal 419 : external communication device 502 : eye position detection module 504 : Eye position estimation module 506 : Eye position prediction module 508 : Condition determination module 510 : Display parameter setting module 511 : Brightness detection module 512 : Graphic module 514 : Light source drive module 516 : Actuator drive module 700 : Eye position 700L : Left eye position 700R: Right eye position D10: Virtual image distances E10, E11, E12: First depth of expression E20, E21, E22: Second depth of expression FU: Perceptual image FU1: First image FU2: Second image K: Display light K10: Display light for left eye K20: Display light for right eye V10, V11, V12: Left viewpoint images V20, V21, V22: Right viewpoint image VS: Virtual image display area ΔT1: First predetermined time ΔT2: Predetermined time θc: Convergence angle θc1: first convergence angle θc2: second convergence angle

Claims (10)

A display control device (30) that executes display control in a head-up display device that is provided in a vehicle and projects an image onto a projection target member so that an observer can visually recognize a virtual image of the image,
one or more processors (33);
a memory (37);
one or more computer programs stored in said memory (37) and configured to be executed by said one or more processors (33);
Said processor (33)
generating an image that expresses a sense of depth by providing parallax to both eyes of the observer;
When a predetermined first condition including at least the elapse of a first predetermined time after the image is displayed is satisfied, a first display process is performed to bring the representation depth of the image closer to the virtual image distance.
A display control device (30) characterized by: The processor, in the first display process,
change stepwise or continuously over time from the first representation depth to the second representation depth,
maintaining the representational depth of the image after reaching the second representational depth;
The second representation depth is a depth between the first representation depth and the virtual image distance or a depth equal to the virtual image distance.
A display control device (30) according to claim 1, characterized in that: The processor, in the first display process,
switching from a first representation depth to a second representation depth;
maintaining the representation depth of the image after switching to the second representation depth;
The second representation depth is a depth between the first representation depth and the virtual image distance or a depth equal to the virtual image distance.
A display control device (30) according to claim 1, characterized in that: wherein the second representation depth is set differently depending on the first representation depth of the image;
A display control device (30) according to claim 2 or 3, characterized in that: The processor
further executing a second display process for changing from the second representation depth to the first representation depth when a predetermined second condition is satisfied after the first display process;
A display control device (30) according to any one of claims 2 to 4, characterized in that: executing the first display process when the depth between the first expression depth and the virtual image distance is longer than a predetermined distance;
A display control device (30) according to any one of claims 2 to 5, characterized in that: The processor
a first content;
displaying a second content having the first representation depth that is longer in depth from the virtual image distance than the first content;
In the first condition, the first predetermined time of the second content is shorter than the first predetermined time of the first content.
A display control device (30) according to any one of claims 2 to 6, characterized in that: The processor
before the first display processing, changing the size of the image while changing the depth of expression of the image to perceive a change in perspective;
After the first display processing, changing the size of the image while maintaining the depth of expression of the image causes a change in perspective to be perceived.
A display control device (30) according to any one of claims 2 to 7, characterized in that: A display control device according to any one of claims 1 to 8;
a display device 50 (50) for emitting display light;
A head-up display device (20) comprising a relay optical system (80) for directing the display light from the display (50) to a projection target. A display control method for a head-up display device provided in a vehicle and configured to allow an observer to visually recognize a virtual image of the image by projecting the image onto a projection target member, the method comprising:
generating an image that expresses a sense of depth by providing parallax to both eyes of the observer;
Determining a predetermined first condition including at least the elapse of a first predetermined time since the image was displayed;
A first display process that brings the representation depth of the image closer to the virtual image distance when the first condition is satisfied,
A display control method characterized by:

Images