JP2023046339A - Display control device, head-up display apparatus and display control method - Google Patents

Abstract

To reduce a sense of discomfort in visual recognition of a virtual image.SOLUTION: A processor displays a first virtual image V1 which includes a plurality of first image elements G10 having a gap D10 therebetween and represents the perspective. The plurality of first image elements G10 represent the same kind of information and have higher visibility than prescribed visibility. In a case where it is detected, estimated or predicted that the attitude variation of a movable body satisfies a prescribed condition, the processor executes first display control processing of reducing the number of gaps D10 between the first image elements G10 included in the first virtual image V1 and higher than the prescribed visibility, or shortening the gap D10.SELECTED DRAWING: Figure 9

Description

The present disclosure relates to a display control device, a head-up display device, and a head-up display device that are used in a mobile object such as a vehicle and superimpose an image on the foreground of the mobile object (actual view in the forward direction of the mobile object seen from the vehicle occupant). It relates to a display control method and the like.

A head-up display (HUD) device is an augmented reality that adds and emphasizes information to a real scene or a real object existing in the real scene by superimposing an image (virtual object) on the scenery in front of the vehicle. (AR: Augmented Reality), it can contribute to safe and comfortable vehicle operation by accurately providing desired information while minimizing the movement of the user's line of sight when driving the vehicle.

In particular, the head-up display device described in Patent Literature 1 corrects the display position of an image (virtual object) based on vibration information of a vehicle, thereby displaying an image (virtual object) of a real scene or a real object existing in the real scene. The image (virtual object) and the real scene are harmonized by suppressing the displacement of the positional relationship.

Further, the head-up display device described in Patent Literature 2 displays a plurality of image elements (virtual objects) arranged along a road surface and expressed with different senses of distance in perspective, thereby displaying an image (virtual object). and harmonize the actual scenery.

JP 2017-013590 A WO2020/009219

However, if multiple image elements (virtual objects) with different senses of distance cannot be individually and satisfactorily corrected based on changes in vehicle posture, the image (virtual object) and the real scene (real object) will not harmonize. It is assumed that the observer will feel uncomfortable.

Moreover, even if we can successfully correct images individually for multiple image elements (virtual objects) with different senses of distance based on changes in the posture of the vehicle, the difference between the real object and the virtual object will still affect the viewer. It is also assumed that it gives a feeling of discomfort.

In addition, even if image correction is not performed based on changes in the posture of the vehicle, multiple image elements (virtual objects) with different senses of distance are directly displaced from the actual scene (real objects) based on changes in the posture of the vehicle. It is also assumed that the feeling is lost and the observer feels uncomfortable.

A summary of certain embodiments disclosed herein follows. It should be understood that these aspects are presented only to provide the reader with an overview of these particular embodiments and are not intended to limit the scope of this disclosure. Indeed, the present disclosure may encompass combinations of the embodiments described below and various aspects not described below.

An overview of the present disclosure relates to providing a display control device, a head-up display device, a display control method, and the like that display an image with high visibility. More specifically, it also relates to reducing discomfort in viewing a virtual image.

Therefore, the display control device, the head-up display device, the display control method, and the like described in this specification employ the following means in order to solve the above problems. The present embodiment includes a plurality of first image elements having gaps therebetween to display a first virtual image representing perspective, and wherein said first image elements between said first image elements having a visibility greater than a predetermined visibility. The gist of this is to execute a first display control process that reduces the number of gaps or shortens the gaps.

Therefore, the display control device in the first embodiment described in this specification is a display control device that controls a head-up display device that displays an image superimposed on the foreground in front of a moving object, and includes one or more processors: and the processor displays a first virtual image representing perspective, including a plurality of first image elements having gaps therebetween, the plurality of first image elements conveying the same kind of information and when it is detected, estimated, or predicted that the posture change of the moving object satisfies a predetermined condition, the visibility of the first image element included in the first virtual image is higher than the predetermined visibility. Reduce the number of gaps between In this case, each of the multiple image elements (virtual objects) deviates from the actual scene (real objects) due to changes in the vehicle's posture, which makes it difficult to see some of them, reducing the sense of discomfort felt by the observer. have advantages.

In any of the display control devices in another preferred embodiment, the processor reduces a portion of the plurality of first image elements prior to the first display control process below a predetermined visibility. In this embodiment, for example, a first virtual image representing perspective is displayed including a plurality (eg, three) of first image elements having higher than a predetermined visibility. In this case, there are two gaps between the three first image elements that are higher than the predetermined visibility. If the posture variation of the moving body satisfies a predetermined condition, some (for example, one) first image elements are made lower than a predetermined visibility. Thus, the number of first image elements having a visibility higher than the predetermined visibility is two, and the gap between two first image elements having a visibility higher than the predetermined visibility is one. That is, the number of gaps between the first image elements higher than the predetermined visibility included in the first virtual image is reduced. When the posture of the moving body changes, the number of first image elements having visibility higher than a predetermined value decreases, so that each of the plurality of image elements (virtual objects) deviates from the actual scene (real object) based on the change in the posture of the vehicle. This has the advantage that it is possible to reduce the sense of incongruity given to an observer by making it difficult to visually recognize a part of the image. In any one of the display control devices in this embodiment, the processor determines that at least one of the first image elements having visibility higher than a predetermined visibility after the first display control process is the first image element before the first display control process. A first display control process is performed so that the image becomes larger than all the plurality of first image elements included in the virtual image. For example, a first virtual image is displayed that includes a plurality (eg, three) of first image elements that are higher than a predetermined visibility and expresses perspective. When the posture variation of the moving body satisfies a predetermined condition, some (for example, one) first image elements are made lower in visibility than the predetermined visibility, and at least the remaining first image elements higher in visibility than the predetermined visibility are set. increase one. This has the advantage of facilitating the transmission of information indicated by the first virtual image while making some of the plurality of image elements (virtual objects) that deviate from the actual scene (real object) less visible. Further, by enlarging the first image element, it is possible to make it difficult for the first image element and the actual scene (real object) to be displaced.

In any display control device in another preferred embodiment, the processor hides a portion of the plurality of first image elements prior to the first display control process. In this embodiment, a first virtual image representing perspective is displayed including a plurality (eg, three) of first image elements having greater than a predetermined visibility. In this case, there are two gaps between the three first image elements that are higher than the predetermined visibility. If the posture change of the moving object satisfies a predetermined condition, some (for example, one) first image elements are hidden. Thus, the number of first image elements having visibility higher than the predetermined visibility is two, and the gap between two first image elements having visibility higher than the predetermined visibility is one. That is, the number of gaps between the first image elements higher than the predetermined visibility included in the first virtual image is reduced. When the posture of the moving body changes, the number of first image elements having visibility higher than a predetermined value decreases, so that each of the plurality of image elements (virtual objects) deviates from the actual scene (real object) based on the change in the posture of the vehicle. This has the advantage that it is possible to reduce the sense of incongruity given to an observer by making it impossible to visually recognize a part of the distorted image. In any one of the display control devices in this embodiment, the processor determines that at least one of the first image elements having visibility higher than a predetermined visibility after the first display control process is the first image element before the first display control process. A first display control process is executed so as to enlarge the entire virtual image. For example, a first virtual image is displayed that includes a plurality (eg, three) of first image elements that are higher than a predetermined visibility and expresses perspective. When the posture variation of the moving body satisfies a predetermined condition, some (for example, one) first image elements are hidden, and at least one of the remaining first image elements with higher visibility than the predetermined visibility is displayed as the first image element. 1st display control processing is performed so that it may enlarge so that it may cover the whole 1st virtual image before display control processing of 1. FIG. This has the advantage of facilitating transmission of information indicated by the first virtual image while making some of the plurality of image elements (virtual objects) that deviate from the actual scene (real object) invisible. Further, by enlarging the first image element, it is possible to make it difficult for the first image element and the actual scene (real object) to be displaced.

In another embodiment, a display control device is a display control device that controls a head-up display device that displays an image superimposed on a foreground in front of a moving body, and has one or more processors, wherein the processors are: Displaying a first virtual image representing perspective, the plurality of first image elements representing the same kind of information and having a gap therebetween, the plurality of first image elements representing the same kind of information, and If it is detected, estimated, or predicted that the posture variation of the moving body is high and satisfies a predetermined condition, the first display control process is executed to shorten the gap included in the first virtual image. In this case, even if individual vibration correction between multiple image elements (virtual objects) is not performed or if individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple image elements (virtual objects) By shortening the gap between multiple image elements (virtual objects), the difference in sense of distance in perspective between multiple image elements (virtual objects) is reduced. This has the advantage that it is possible to reduce the sense of incongruity caused by deviation from the real scene (real object) based on the above.

In any one of the display control devices in this embodiment, the processor determines that at least one of the first image elements having visibility higher than a predetermined visibility after the first display control process is the first image element before the first display control process. A first display control process is performed so that the image becomes larger than all the plurality of first image elements included in the virtual image. For example, a first virtual image is displayed that includes a plurality (eg, three) of first image elements that are higher than a predetermined visibility and expresses perspective. When the posture variation of the moving body satisfies a predetermined condition, a first display control process is executed to enlarge a part (for example, one) of the first image elements so as to reduce the gap between the image elements. In this case, even if individual vibration correction between multiple image elements (virtual objects) is not performed or if individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple image elements (virtual objects) By shortening the gap between multiple image elements (virtual objects), the difference in sense of distance in perspective between multiple image elements (virtual objects) is reduced. This has the advantage that it is possible to reduce the sense of incongruity caused by deviation from the real scene (real object) based on the above. Further, by enlarging the first image element, it is possible to make it difficult for the first image element and the actual scene (real object) to be displaced.

In any one of the display control devices according to another preferred embodiment, the processor shortens the display distance of the first virtual image when it is detected, estimated, or predicted that the posture change of the moving body satisfies the predetermined condition. is further executed. According to this, it is possible to suppress the amount of change in the image (virtual object) due to the change in the attitude of the vehicle, and by extension, it is possible to suppress the deviation between the image (virtual object) and the actual scene (real object). have advantages.

In any one of the display control devices according to another preferred embodiment, the processor continuously or stepwise shortens the display distance of the first virtual image as time elapses in the second display control processing. According to this, there is an advantage that it is possible to reduce the annoyance of an instantaneous change in the display distance.

In any of the display control devices in another preferred embodiment, the processor moves the plurality of image elements closer together such that a gap between the plurality of first image elements before the first display control process is shortened. , a first display control process for shortening the gap included in the first virtual image. In this case, even if individual vibration correction between multiple image elements (virtual objects) is not performed or if individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple image elements (virtual objects) By shortening the gap between multiple image elements (virtual objects), the difference in sense of distance in perspective between multiple image elements (virtual objects) is reduced. This has the advantage that it is possible to reduce the sense of incongruity caused by deviation from the real scene (real object) based on the above.

In any of the display control devices in another preferred embodiment, the processor comprises a first display controller that glues the plurality of image elements such that gaps between the plurality of first image elements before the first display control process are eliminated. Execute display control processing. In this case, even if individual vibration correction between multiple image elements (virtual objects) is not performed or if individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple image elements (virtual objects) By shortening the gap between multiple image elements (virtual objects), the difference in sense of distance in perspective between multiple image elements (virtual objects) is reduced. This has the advantage that it is possible to reduce the sense of incongruity caused by deviation from the real scene (real object) based on the above.

In any of the display control devices in another preferred embodiment, the processor displays second image elements arranged to fill gaps between the plurality of first image elements before the first display control process. A first display control process is executed. In this case, even if individual vibration correction between multiple image elements (virtual objects) is not performed or if individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple image elements (virtual objects) By filling the gap between the image elements (virtual objects) with the second image element, it becomes difficult for the observer to perceive the difference in perspective between the image elements (virtual objects). This has the advantage that it is possible to reduce the sense of incongruity caused by each of the virtual objects) deviating from the real scene (real object) based on the change in posture of the vehicle.

In any display controller in this embodiment, the first image element has higher visibility than the second image element. Visual attention is directed to the first image element that presents information while reducing discomfort caused by each of a plurality of image elements (virtual objects) deviating from the actual scene (real object) based on changes in vehicle posture. can be made easier.

The head-up display device according to the embodiments described in this specification includes any display control device according to some embodiments, a light modulation element that emits display light, and the display light from the light modulation element to be projected. and a relay optical system directed to the part. Again, the same advantages as above are assumed.

In the display control method in the embodiments described herein, displaying a first virtual image representing a perspective including a plurality of first image elements having gaps therebetween; The first image element represents the same type of information, is higher than a predetermined visibility, and is included in the first virtual image when it is detected, estimated, or predicted that the posture change of the moving body satisfies a predetermined condition. and performing a first display control process that reduces the number of gaps or shortens the gaps between the first image elements that are higher than the predetermined visibility.

FIG. 1 is a diagram showing an application example of a vehicle display system to a vehicle. FIG. 2 is a diagram showing the configuration of the head-up display device. FIG. 3 is a block diagram of a vehicle display system of some embodiments. FIG. 4 is a flow diagram illustrating a method of performing a first display control process, according to some embodiments. FIG. 5 is a diagram showing an example of a foreground visually recognized by an observer and an image (virtual image) displayed superimposed on the foreground while the host vehicle is running. In FIG. 6, the left diagram shows the virtual image before the first display processing, and the right diagram shows the virtual image after the first display processing. In FIG. 7, the left diagram shows the virtual image before the first display processing, and the right diagram shows the virtual image after the first display processing. In FIG. 8, the left diagram shows the virtual image before the first display processing, and the right diagram shows the virtual image after the first display processing. In FIG. 9, the left diagram shows the virtual image before the first display processing, and the right diagram shows the virtual image after the first display processing. In FIG. 10, the left diagram shows the virtual image before the first display processing, and the right diagram shows the virtual image after the first display processing. In FIG. 11, the left diagram shows the virtual image before the first display processing, and the right diagram shows the virtual image after the first display processing. In FIG. 12, the left diagram shows the virtual image before the first display processing, and the right diagram shows the virtual image after the first display processing. FIG. 13 is a diagram for explaining the second display control process, in which the horizontal axis indicates time and the vertical axis indicates display distance.

1-13 below provide a description of the configuration and operation of an exemplary vehicular display system. In addition, the present invention is not limited by the following embodiments (including the contents of the drawings). Of course, modifications (including deletion of constituent elements) can be added to the following embodiments. In addition, in the following description, descriptions of known technical matters are omitted as appropriate in order to facilitate understanding of the present invention.

Please refer to FIG. FIG. 1 is a diagram showing an example of the configuration of a vehicle virtual image display system. In FIG. 1, the left-right direction of a vehicle (an example of a moving body) 1 (in other words, the width direction of the vehicle 1) is the X axis (the positive direction of the X axis is the left direction when the vehicle 1 faces forward). ), and the vertical direction (in other words, the height direction of the vehicle 1) along a line segment that is perpendicular to the left-right direction and is perpendicular to the ground or a surface corresponding to the ground (here, the road surface 6) is the Y-axis (Y-axis is the upward direction), and the front-rear direction along a line segment perpendicular to each of the left-right direction and the up-down direction is the Z-axis (the positive direction of the Z-axis is the straight-ahead direction of the vehicle 1). This point also applies to other drawings.

As illustrated, a vehicle display system 10 provided in a vehicle (self-vehicle) 1 displays the positions and line-of-sight directions of a left eye 700L and a right eye 700R of an observer (typically, a driver sitting in the driver's seat of the vehicle 1). An eye position detection unit (line-of-sight detection unit) 409 for detecting a pupil (or face) to be detected, an exterior sensor 411 configured by a camera (for example, a stereo camera) for imaging the front (broadly speaking, the surroundings) of the vehicle 1, a head It has an up display device (hereinafter also referred to as HUD device) 20 and a display control device 30 that controls the HUD device 20 . Note that the eye position detection unit (line of sight detection unit) 409 and the vehicle exterior sensor 411 may be omitted.

FIG. 2 is a diagram showing one aspect of the configuration of the head-up display device. The HUD device 20 is installed, for example, in a dashboard (reference numeral 5 in FIG. 1). The HUD device 20 accommodates a stereoscopic image display device (image display device) 40, a relay optical system 80, and the image display device 40 and the relay optical system 80, and transmits display light K from the image display device 40 from the inside to the outside. and a housing 22 having a light exit window 21 capable of emitting light toward. Note that the image display device 40 is not limited to a stereoscopic image display device that displays 3D images, and may display 2D images.

The image display device 40 is assumed here to be a parallax 3D display device. This stereoscopic display device (parallax type 3D display device) 40 is a display device 50 which is a naked-eye stereoscopic display device using a multi-viewpoint image display system capable of controlling depth representation by visually recognizing left-viewpoint images and right-viewpoint images, and , and a light source unit 60 that functions as a backlight.

The display 50 has a light modulation element 51 that modulates the illumination light from the light source unit 60 to generate an image, and, for example, a lenticular lens or a parallax barrier (parallax barrier). Left eye display light such as light rays K11, K12 and K13 (reference symbol K10 in FIG. 1) and right eye display light such as light rays K21, K22 and K23 (in FIG. 1). symbol K20) and an optical layer (an example of a light beam splitting section) 52. The optical layer 52 includes optical filters such as lenticular lenses, parallax barriers, lens arrays, and microlens arrays. In the embodiments, the optical layer 52 is not limited to the optical filter described above, and includes all types of optical layers arranged on the front or rear surface of the light modulation element 51 . However, this is an example and is not limited.

Further, in the image display device 40, instead of or in addition to the optical layer (an example of the light beam separating section) 52, the light source unit 60 is configured with a directional backlight unit (an example of the light separating section). , left-eye display light such as light rays K11, K12, and K13 (reference symbol K10 in FIG. 1), and right-eye display light such as light rays K21, K22, and K23, etc. (reference symbol K20 in FIG. 1). , may be emitted. Specifically, for example, the display control device 30, which will be described later, causes the light modulation element 51 to display a left-viewpoint image when the directional backlight unit emits illumination light directed toward the left eye 700L. When the left-eye display light K10 such as K11, K12, and K13 is directed toward the viewer's left eye 700L, and the directional backlight unit emits illumination light toward the right eye 700R, the right viewpoint image is displayed on the light modulation element 51. By displaying, right-eye display light K20 such as right-eye light rays K21, K22, and K23 is directed to the left eye 700L of the observer. However, this is an example and is not limited.

The display control device 30, which will be described later, performs, for example, image rendering processing (graphic processing), display device driving processing, and the like, so that the display light K10 for the left eye of the left viewpoint image V10 and the display light K10 for the left eye and the display light K10 for the right eye 700R of the observer's left eye 700L are displayed. By directing the display light K20 for the right eye of the right viewpoint image V20 and adjusting the left viewpoint image V10 and the right viewpoint image V20, the aspect of the perception image FU displayed by the HUD device 20 (perceived by the observer) is controlled. be able to. Note that the display control device 30, which will be described later, controls the display (display device 50) so as to reproduce a light field that (approximately) reproduces light rays output in various directions from a point in a fixed space. may

The relay optical system 80 has curved mirrors (concave mirrors, etc.) 81 and 82 that reflect the light from the image display device 40 and project the image display light K10 and K20 onto the windshield (projection target member) 2 . However, it may further include other optical members (refractive optical members such as lenses, diffractive optical members such as holograms, reflective optical members, or combinations thereof).

In FIG. 1, the image display device 40 of the HUD device 20 displays images with parallax (parallax images) for the left and right eyes. Each parallax image is displayed as V10 and V20 formed on a virtual image display surface (virtual image formation surface) VS, as shown in FIG. The focus of each eye of the observer (person) is adjusted so as to match the position of the virtual image display area VS. Note that the position of the virtual image display area VS is referred to as an "adjustment position (or imaging position)", and a predetermined reference position (for example, the center 205 of the eyebox 200 of the HUD device 20, the observer's viewpoint position, or , a specific position of the vehicle 1, etc.) to the virtual image display area VS is referred to as an adjustment distance (imaging distance).

However, in reality, since the human brain fuses each image (virtual image), the human is at a position farther back than the adjustment position (for example, due to the convergence angle between the left viewpoint image V10 and the right viewpoint image V20). It is a fixed position, and the position perceived as being farther away from the observer as the angle of convergence decreases) is recognized as a perceptual image (here, an arrowhead figure for navigation) FU is displayed. do. Note that the perceptual image FU may be referred to as a "stereoscopic virtual image", and may also be referred to as a "stereoscopic image" when the "image" is taken in a broad sense to include virtual images. It may also be referred to as a "stereoscopic image", "3D display", or the like.

FIG. 3 is a block diagram of a vehicular virtual image display system, according to some embodiments. The display controller 30 comprises one or more I/O interfaces 31 , one or more processors 33 , one or more display control processing circuits 35 and one or more memories 37 . FIG. 3 is only one embodiment and the illustrated components may be combined into fewer components or there may be additional components. For example, display control processing circuitry 35 (eg, a graphics processing unit) may be included in one or more processors 33 .

As shown, processor 33 and display control processing circuitry 35 are operatively coupled with memory 37 . More specifically, the processor 33 and the display control processing circuit 35 execute a program stored in the memory 37 to generate and/or transmit image data, for example, the vehicle display system 10 (image A display device 40) can be controlled. Processor 33 and/or display control processing circuitry 35 may include at least one general purpose microprocessor (e.g., central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA). ), or any combination thereof. Memory 37 includes any type of magnetic media such as hard disks, any type of optical media such as CDs and DVDs, any type of semiconductor memory such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

As shown, processor 33 is operatively coupled with I/O interface 31 . The I/O interface 31 communicates (CAN communication). The communication standard adopted by the I/O interface 31 is not limited to CAN. : MOST is a registered trademark), a wired communication interface such as UART or USB, or a personal area network (PAN) such as a Bluetooth network, a local network such as an 802.11x Wi-Fi network. It includes an in-vehicle communication (internal communication) interface, which is a short-range wireless communication interface within several tens of meters such as an area network (LAN). In addition, the I / O interface 31 is a wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e base (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, An external communication (external communication) interface such as a wide area communication network (for example, Internet communication network) may be included according to a cellular communication standard such as 5G.

As shown, the processor 33 is interoperably coupled with the I/O interface 31 to communicate with various other electronic devices and the like connected to the vehicle display system 10 (I/O interface 31). can be given and received. The I/O interface 31 includes, for example, a vehicle ECU 401 , a road information database 403 , a vehicle position detector 405 , an operation detector 407 , an eye position detector 409 , an external sensor 411 , a brightness detector 413 , an attitude detector 415 . , a personal digital assistant 417, and an external communication device 419, etc. are operatively coupled. Note that the I/O interface 31 may include a function of processing (converting, calculating, and analyzing) information received from other electronic devices or the like connected to the vehicle display system 10 .

Image display device 40 is operatively coupled to processor 33 and display control processing circuitry 35 . Accordingly, the image displayed by light modulating element 51 may be based on image data received from processor 33 and/or display control processing circuitry 35 . The processor 33 and the display control processing circuit 35 control the image displayed by the light modulation element 51 based on information acquired from the I/O interface 31 .

The software components stored in the memory 37 include a posture change detection module 502, a posture change estimation module 504, a posture change prediction module 506, a display parameter setting module 512, a graphic module 514, a deviation amount calculation module 522, and a correction amount calculation module. 524.

FIG. 4 is a flow diagram illustrating a method S100 of performing a first display control process, according to some embodiments. The method S100 is performed in the image display device 40 (light modulation element 51) and the display control device 30 that controls this image display device 40 (light modulation element 51). Some acts in method S100 are optionally combined, some acts are optionally reordered, and some acts are optionally omitted.

The display control device 30 (processor 33) generates image data based on the information acquired from the I/O interface 31, and outputs it to the image display device 40 (light modulation element 51), thereby superimposing the image data on the viewer's foreground. to display the first virtual image V1 (virtual image) V1 (step S110).

In step S<b>130 , the display control device 30 (processor 33 ) acquires information about the posture change of the moving object via the I/O interface 31 . The display control device 30 (processor 33) acquires information (posture variation information) indicating the posture variation of the moving object from the posture detection unit 415 (S132). The orientation detection unit 415 includes, for example, one or more of a gyro sensor, an acceleration sensor, a height sensor, and the like. That is, the posture variation information includes the angular velocity, acceleration, height of the moving object, vehicle posture (pitch angle, roll angle, etc.), frequency of change in the vehicle posture (vibration frequency), and the like.

The display control device 30 (processor 33) executes the posture change detection module 502 to determine whether the posture change of the moving body satisfies a predetermined condition based on the posture change information (S150). Based on the posture variation information, the posture change detection module 502 detects that there is posture variation, that there is posture variation of a predetermined magnitude, that there is posture variation at a predetermined speed, and that there is posture variation at a predetermined frequency. It includes various software components for performing various operations related to detecting a presence, etc. That is, the attitude change detection module 502 includes determination thresholds, table data, arithmetic expressions, etc. for determining whether or not the attitude change of the moving object satisfies a predetermined condition from various information acquired from the I/O interface 31 . obtain.

In some embodiments, the display control device 30 (processor 33) uses information (not limited to) from the vehicle ECU 401, the vehicle exterior sensor 411, etc. (not limited to these) to estimate the attitude change of the moving object ( posture change estimation information) is acquired (S134).

The display control device 30 (processor 33) executes the posture change estimation module 504 to determine whether or not the posture change of the moving body is estimated to satisfy a predetermined condition based on the posture change estimation information (S150). . Posture change estimation module 504 determines, based on the posture change estimation information, that there is posture change, that there is posture change of a predetermined magnitude, that there is posture change at a predetermined speed, and that there is posture change at a predetermined frequency. includes various software components for performing various operations related to estimating that there is a That is, the attitude change estimation module 504 includes determination threshold values, table data, arithmetic expressions, etc. for estimating whether or not the attitude change of the moving body satisfies a predetermined condition from various information acquired from the I/O interface 31 . obtain. The attitude change estimation module 504 determines whether or not it is estimated that the attitude change of the moving object satisfies a predetermined condition, according to the rate of change in speed (acceleration) acquired from the vehicle ECU 401 . In addition, the attitude change estimation module 504 determines whether or not it is estimated that the attitude change of the moving object satisfies a predetermined condition according to the change in the position (coordinates) of the real object outside the vehicle acquired from the outside sensor 411 . Note that the method of estimating the attitude change of the moving object is not limited to these, and various known methods can be applied.

In some embodiments, the display control device 30 (processor 33) can predict the attitude change of the moving body from (but not limited to) the road information database 403, the external communication device 419, etc. via the I/O interface 31. information (posture change prediction information) is acquired (S136).

The display control device 30 (processor 33) executes the posture change prediction module 506 to determine whether the posture change of the moving body is predicted to satisfy a predetermined condition based on the posture change prediction information (S150). . The posture change prediction module 506 determines, based on the posture change prediction information, that there is a posture change, that there is a posture change of a predetermined magnitude, that there is a posture change at a predetermined speed, and that there is a posture change at a predetermined frequency. includes various software components for performing various operations related to predicting that there will be, and the like. The posture change prediction module 506 predicts that the posture change of the moving body will satisfy a predetermined condition in the near future, according to the road surface position information obtained from the road information database 403 and the posture change is predicted to satisfy the predetermined condition. determine whether In addition, the posture change estimation module 504 calculates the posture of the moving object according to the position information of the preceding vehicle, which is acquired from the external communication device 419 through inter-vehicle communication with the preceding vehicle traveling in front of the moving object. A determination is made as to whether the variation is predicted to satisfy a predetermined condition. Note that the method of predicting the attitude change of the moving object is not limited to these, and various known methods can be applied.

In step S<b>150 , the display control device 30 (processor 33 ), based on the posture variation information (the posture variation estimation information and the posture variation prediction information) acquired via the I/O interface 31 , determines whether the posture variation of the moving body is It is determined whether it is detected, estimated, or predicted that a predetermined condition is satisfied. If it is determined that the posture change of the moving body satisfies the predetermined condition, the process proceeds to step S170. Note that the display control device 30 may acquire the determination results from an external device (a device different from the display control device 30) without having the function of performing these determinations. Therefore, pose change detection module 502, pose change estimation module 504, and pose change prediction module 506 may be omitted.

In step S170, the display control device 30 (processor 33) uses the display parameter setting module 512 to perform at least the first display control process (step S180), and further performs the second display control process (step S190). good too.

Display parameter setting module 512 includes various software components for performing various operations related to setting display parameters of a virtual image to be displayed based on various information and commands obtained from I/O interface 31. including. That is, the display parameter setting module 512 can include table data, arithmetic expressions, etc. for specifying display parameters from various information acquired from the I/O interface 31 .

Display parameters include the type, placement (positional coordinates, angle), size, viewing distance (in the case of 3D), visual effects (e.g., brightness, transparency, saturation, contrast, or other visual effects) of the displayed image. properties), including parameters for changing Specifically, for example, the display parameters include: (1) parameters for arranging an image so as to have a predetermined positional relationship with a real object positioned outside the vehicle 1 when viewed from the eye position 700 (image arrangement control; and/or parameters for controlling the actuators 28 and 29.), (2) parameters for changing the size and placement of the image (for controlling the size and placement of the image (3) a parameter for switching display or non-display of an image (a parameter for controlling the display 50); Parameters for pre-distorting an image to reduce distortion (parameters for controlling the display 50 to pre-distort an image displayed on the display 50, also called warping parameters), ( 5) parameters for controlling visibility such as image transmittance, brightness, and brightness (parameters for controlling the display device 50 for controlling image transmittance, brightness, brightness, etc., and/or image transmittance, brightness , parameters for controlling the light source unit 60 for controlling brightness, etc.); parameters.), etc. However, the display parameters set (selected) by the display parameter setting module 512 are not limited to these.

The display control device 30 (processor 33) of the present embodiment performs first display control processing (step S180) by the display parameter setting module 512 when detecting, estimating, or predicting that the posture change of the moving object satisfies a predetermined condition. ) to reduce the number of gaps or shorten the gaps between image elements contained in the image that are higher than a given visibility.

FIG. 5 is a diagram showing the foreground of the vehicle and the first image before the first display control process displayed by the head-up display device 20, which is visually recognized when the observer faces forward. The first virtual image V1 before the first display control process includes a plurality of (three in the example of FIG. 5) first image elements having gaps D10 (D11 and D12 in the example of FIG. 5) therebetween. Perspective is expressed by displaying G10 (G11, G12, and G13 in the example of FIG. 5) side by side while gradually changing the size in the vertical direction (Y-axis direction) as viewed from the observer. In the example of FIG. 5, the first image elements G10 are arranged in order of G11, G12, and G13 in the direction from the bottom to the top as seen from the observer (positive Y-axis direction). It has two gaps D10 consisting of a gap D11 located in between and a gap D12 located between G12 and G13. Here, all of the first image elements G11, G12, and G13 have visibility higher than the first visibility. Here, visibility is, for example, luminance or transmittance. That is, each of the first image elements G11, G12, G13 has luminance higher than the first luminance and/or transmittance lower than the first transmittance. The first image elements G11, G12, and G13 are all figures that guide the route, and represent the same kind of information.

6 to 12, the left figures show the virtual images before the first display processing, and the right figures show the virtual images after the first display processing. FIG. 13 is a diagram for explaining the second display control process, in which the horizontal axis indicates time and the vertical axis indicates display distance.

(First embodiment)
In the display control device 30 according to some embodiments, the processor 33 executes the first display control process (step S182) to reduce the visibility of the first image contained in the first virtual image V1 higher than the predetermined visibility. Reduce the number of gaps D10 between image elements. In any display control device, the processor 33 makes part of the plurality of first image elements G10 prior to the first display control process lower than a predetermined visibility, as shown in FIG. In the example of FIG. 6, a first virtual image V1 including a plurality (for example, three) of first image elements G11, G12, and G13 higher than a predetermined first visibility and expressing perspective is displayed ( Fig. 6 left). In this case, there are two gaps D11 and D12 between the three first image elements G10 with visibility higher than the predetermined visibility. When the posture variation of the moving object satisfies a predetermined condition, some (for example, two) first image elements G11 and G13 are made lower than the first visibility, as shown in the right diagram of FIG. As a result, the number of first image elements higher than the first visibility is one, and the number of first image elements higher than the first visibility is only one (G12). The gap between the first image element with higher gender will be zero (it could be one instead of zero). That is, the number of gaps between the first image elements G10 higher than the first visibility included in the first virtual image V1 is reduced. When the posture of the moving object changes, the number of first image elements G10 higher than the first visibility is reduced, so that each of the plurality of image elements (virtual objects) G10 changes to the actual scene (real object) based on the vehicle posture change. ) makes it difficult to visually recognize a part of the image, which has the advantage of reducing the sense of discomfort given to the observer.

(Second embodiment)
In any display control device in another preferred embodiment, the processor 33 hides part of the plurality of first image elements G10 before the first display control process. In this embodiment, the processor 33 hides some (for example, two) first image elements G11 and G13 by executing the first display control process (step S182). Thus, the number of first image elements G10 with higher than first visibility is one (G12) and the gap D10 between the first image elements with higher than first visibility is zero. That is, the number of gaps D10 between the first image elements G10 higher than the first visibility included in the first virtual image V1 is reduced. When the posture of the moving object changes, the number of first image elements having higher visibility than the first visibility is reduced. This has the advantage that a part of the misalignment cannot be visually recognized, and the discomfort given to the observer can be reduced.

(Third embodiment)
In the display control device 30 according to either the first or second embodiment, the processor 33, in the first display control process (step S182), sets the visibility higher than the first visibility after the first display control process. At least one of the 1 image elements G12 is enlarged. In the example shown in FIG. 7, the processor 33 executes the first display control process (step S182) to set some (for example, two) of the first image elements G11 and G13 to the first visibility. lower and at least one of the remaining first visibility higher than first image elements G12 is enlarged. This has the advantage of facilitating transmission of information indicated by the first virtual image V1 while making it difficult to visually recognize some of the plurality of image elements (virtual objects) that deviate from the actual scene (real object). . Further, by enlarging the first image element, it is possible to make it difficult for the first image element and the actual scene (real object) to be displaced.

(Fourth embodiment)
In the display control device 30 according to either the first or second embodiment, the processor 33 determines that at least one of the first image elements higher than the first visibility after the first display control process is the first The first display control process is executed so as to enlarge the entire first virtual image V1 before the display control process. In the example shown in FIG. 8, the processor 33 executes the first display control process (step S182) to hide a portion (for example, one) of the first image elements and hide the remaining first image elements. The first display control process is performed so that at least one of the first image elements whose visibility is higher than that of is enlarged to cover the entire first virtual image V1 before the first display control process. This has the advantage of facilitating transmission of information indicated by the first virtual image V1 while making some of the plurality of image elements (virtual objects) that deviate from the actual scene (real object) invisible. Further, by enlarging the first image element, it is possible to make it difficult for the first image element and the actual scene (real object) to be displaced.

(Fifth embodiment)
Moreover, in the display control device 30 in another embodiment, the processor 33 shortens the gap included in the first virtual image V1 by executing the first display control process (step S184). In this case, even if individual vibration correction between multiple image elements (virtual objects) is not performed or if individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple image elements (virtual objects) By shortening the gap between multiple image elements (virtual objects), the difference in sense of distance in perspective between multiple image elements (virtual objects) is reduced. This has the advantage that it is possible to reduce the sense of incongruity caused by deviation from the real scene (real object) based on the above.

(Sixth embodiment)
In any display control device 30 according to the fifth embodiment, the processor 33 brings the plurality of image elements closer together so that the gap between the plurality of first image elements before the first display control process is shortened. Thus, the first display control process for shortening the gap included in the first virtual image V1 is executed. In any of the display control devices 30, the processor 33 replaces the first image elements G11 and G13 with the first image elements G11 and G13 as shown in the right diagram of FIG. Each of the gap D11 between the first image elements G11 and G12 and the gap D12 between the first image elements G12 and G13 is shortened by bringing the G12 closer. According to this, even if individual vibration correction between multiple image elements (virtual objects) is not performed or individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple images By shortening the gap between the elements (virtual objects), the distance difference in perspective between the multiple image elements (virtual objects) is reduced. It is possible to reduce the sense of incongruity caused by deviation from the real scene (real object) based on the posture change.

(Seventh embodiment)
In any display control device 30 according to the fifth embodiment, the processor 33 glues the plurality of image elements such that the gaps between the plurality of first image elements before the first display control processing are eliminated. 1 display control processing is executed. In any of the display control devices 30, the processor 33 replaces the first image elements G11 and G13 with the first image elements G11 and G13 as shown in the right diagram of FIG. By adhering to G12, the gap D11 between the first image elements G11 and G12 and the gap D12 between the first image elements G12 and G13 disappear (substantially disappear). In this case, even if individual vibration correction between multiple image elements (virtual objects) is not performed or if individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple image elements (virtual objects) By shortening the gap between multiple image elements (virtual objects), the difference in sense of distance in perspective between multiple image elements (virtual objects) is reduced. This has the advantage that it is possible to reduce the sense of incongruity caused by deviation from the real scene (real object) based on the above.

(Eighth embodiment)
In any display control device 30 according to the fifth embodiment, the processor 33 arranges second image elements to fill gaps between the plurality of first image elements before the first display control process. is executed. In any of the display control devices 30, the processor 33 controls the gap D11 between the first image elements G11 and G12 and the gap D11 between the first image elements G11 and G12 as shown in the right diagram of FIG. A second image element G20 is added such that each gap D12 between the first image elements G12, G13 is eliminated. In this case, even if individual vibration correction between multiple image elements (virtual objects) is not performed or if individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple image elements (virtual objects) By filling the gap between the image elements (virtual objects) with the second image element, it becomes difficult for the observer to perceive the difference in perspective between the image elements (virtual objects). This has the advantage that it is possible to reduce the sense of incongruity caused by each of the virtual objects) deviating from the real scene (real object) based on the change in posture of the vehicle. Note that the second image element G20 may not be arranged so as to eliminate the gap between the first image elements, but may be arranged so as to partially fill the gap between the first image elements. It may be

(Ninth embodiment)
In any display control device 30 in the eighth embodiment, the first image element G10 has higher visibility than the second image element G20. Visual attention is directed to the first image element that presents information while reducing discomfort caused by each of a plurality of image elements (virtual objects) deviating from the actual scene (real object) based on changes in vehicle posture. can be made easier.

(Tenth embodiment)
In any one of the display control devices 30 according to the fifth embodiment, the processor 33 determines that at least one of the first image elements higher than the first visibility after the first display control process is determined by the first display control process. A first display control process is performed so that the image element becomes larger than the previous first image element. In the example shown in FIG. 12, a first virtual image V1 is displayed that includes a plurality of (for example, three) first image elements that have higher visibility than the first visibility and expresses perspective. When the posture variation of the moving object satisfies a predetermined condition, a first display control process is executed to enlarge at least a part (for example, all three) of the first image elements so as to reduce the gap between the image elements. do. In this case, even if individual vibration correction between multiple image elements (virtual objects) is not performed or if individual vibration correction does not work well (differences in sense of distance in perspective cannot be expressed well), multiple image elements (virtual objects) By shortening the gap between multiple image elements (virtual objects), the difference in sense of distance in perspective between multiple image elements (virtual objects) is reduced. This has the advantage that it is possible to reduce the sense of incongruity caused by deviation from the real scene (real object) based on the above. Further, by enlarging the first image element, it is possible to make it difficult for the first image element and the actual scene (real object) to be displaced.

(Eleventh embodiment)
In any one of the display control devices 30 according to the first to eleventh embodiments, the processor 33 displays the first virtual image V1 when detecting, estimating, or predicting that the posture variation of the moving object satisfies a predetermined condition. A second display control process (step S190) for shortening the distance is further executed. According to this, it is possible to suppress the amount of change in the image (virtual object) due to the change in the attitude of the vehicle, and by extension, it is possible to suppress the deviation between the image (virtual object) and the actual scene (real object). have advantages.

(Twelfth embodiment)
In any display control device 30 according to the eleventh embodiment, the processor 33 continuously or stepwise changes the display distance of the first virtual image V1 over time in the second display control process (step S190). shorten to In the example shown in FIG. 13, the processor 33 starts the second display control process (step S190) from time t1, and continuously shortens the display distance of the first virtual image V1 as time elapses (however, It may be a gradual change over time). According to this, there is an advantage that it is possible to reduce the annoyance of an instantaneous change in the display distance.

In the head-up display device 20 described in this specification, any one of the display control device 30 in some embodiments, the light modulation element 51 that emits display light, and the display light from the light modulation element 51 is received. and a relay optical system 80 directed to the projection unit 2 . Again, the same advantages as above are assumed.

In the display control method described herein, displaying a first virtual image V1 including a plurality of first image elements having gaps therebetween and expressing perspective; The image element G10 represents the same type of information, is higher than the first visibility, and is included in the first virtual image V1 when it is detected, estimated, or predicted that the posture change of the moving object satisfies a predetermined condition. and performing a first display control process to reduce the number of gaps D10 between the first image elements G10 having visibility greater than 1, or to shorten the gaps D10.

Refer to FIG. 3 again. The graphic module 514 performs display control processing such as rendering based on the display parameters set by the display parameter setting module 512 to generate image data, and implements various known software components for driving the image display device 40. include. That is, the graphic module 514, based on the display parameters set by the display parameter setting module 512, determines the type, arrangement (positional coordinates, angle), size, display distance (in the case of 3D), and visual effect of the image to be displayed. (eg, brightness, transparency, saturation, contrast, or other visual characteristics), various known software components may be included. The graphic module 514 stores the type of image (one example of display parameters), the position coordinates of the image (one example of display parameters), the angle of the image (pitch angle with the X direction as the axis, Y direction as the axis, yaw rate angle around the axis, rolling angle around the Z direction, etc., which are examples of display parameters), image size (one example of display parameters), image color (hue, saturation , brightness, etc.) to generate image data and drive the display 50 so as to be visually recognized by the observer.

A light source driving module (not shown) includes various known software components for performing driving the light source unit 24 . The light source driving module 516 may drive the light source units 24 based on the set display parameters.

Actuator drive module (not shown) includes various known software components for performing the actuation of first actuator 28 and/or second actuator 29 . The actuator driving module can drive the first actuator 28 and the second actuator 29 based on set display parameters.

The deviation amount calculation module 522 calculates the attitude (angle deviation amount) of the vehicle 1 based on the attitude change information acquired from the attitude detection unit 415 . For example, the deviation amount calculation module 522 calculates the angle (pitch angle) of the vehicle 1 about the pitch axis by integrating the angular velocities detected by the posture detection unit 415 . As a result, it is possible to calculate the deviation amount (angle) of the vehicle 1 in the direction of rotation about the Y-axis (pitch axis) shown in FIG. Although the pitch angle is calculated in this embodiment, the yaw angle or the roll angle may be calculated. For example, all angles around the X-axis, Y-axis and Z-axis may be calculated.

The correction amount calculation module 524 calculates the correction amount of the display position of the first virtual image V1 (plurality of first image elements G10) according to the attitude of the vehicle 1 (angle deviation amount). Specifically, the correction amount calculation module 524 converts the deviation amount of the angle (pitch angle) calculated by the deviation amount calculation module 522 into the number of pixels, and corrects the number of pixels corresponding to the deviation to the original number. Determine quantity. For example, the correction amount calculation module 524 determines a correction amount that restores the amount of pitch angle deviation. The correction amount calculation module 524 outputs the calculated correction amount to the graphic module 514 . In this embodiment, the correction amount in the pitch axis direction is calculated, but the correction amount in the yaw axis direction and the roll direction may be calculated. As for the roll angle, the correction amount is determined so as to restore the deviation amount of the roll angle without changing the angle.

The operations of the processing processes described above may be implemented by executing one or more functional modules of an information processing device such as a general purpose processor or an application specific chip. These modules, combinations of these modules, and/or combinations with known hardware that can replace their functions are all within the scope of protection of the present invention.

The functional blocks of vehicle display system 10 are optionally implemented in hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is noted that the functional blocks illustrated in FIG. 7 may optionally be combined or separated into two or more sub-blocks to implement the principles of the described embodiments. will be understood by those skilled in the art. Accordingly, the description herein optionally supports any possible combination or division of the functional blocks described herein.

Reference Signs List 1: vehicle 2: projection target 10: vehicle display system 20: head-up display device (HUD device)
21 : Light exit window 22 : Housing 24 : Light source unit 28 : First actuator 29 : Second actuator 30 : Display control device 31 : I/O interface 33 : Processor 35 : Display control processing circuit 37 : Memory 40 : Image display Device 50 : Display 51 : Light modulation element 52 : Optical layer 60 : Light source unit 80 : Relay optical system 90 : Virtual image optical system 200 : Eyebox 205 : Center 401 : Vehicle ECU
403 : road information database 405 : vehicle position detection unit 407 : operation detection unit 409 : eye position detection unit 411 : vehicle exterior sensor 413 : brightness detection unit 415 : attitude detection unit 417 : mobile information terminal 419 : external communication device 502 : Posture change detection module 504: Posture change estimation module 506: Posture change prediction module 512: Display parameter setting module 514: Graphic module 516: Light source drive module 522: Deviation amount calculation module 524: Correction amount calculation module 700: Eye position 700L: Left eye 700R: Right eye D10: Gap FU: Perceptual image G10: First image element G20: Second image element K: Display light K10: Display light (left eye display light)
K20: display light (display light for right eye)
V1: first virtual image V10: left viewpoint image V20: right viewpoint image VS: virtual image display area

Claims (11)

In a display control device that controls a head-up display device that displays an image superimposed on the foreground in front of a moving object,
having one or more processors,
The processor
displaying a first virtual image representing perspective, comprising a plurality of first image elements having gaps therebetween;
the plurality of first image elements represent the same type of information and have higher than predetermined visibility;
number of the gaps between the first image elements higher than the predetermined visibility included in the first virtual image when it is detected, estimated, or predicted that the posture variation of the moving body satisfies a predetermined condition; or perform a first display control process that shortens the gap;
A display control device characterized by: The processor
enlarging at least one of the first image elements higher than the predetermined visibility after the first display control process;
The display control device according to claim 1. The processor
At least one of the first image elements having visibility higher than the predetermined visibility after the first display control process is enlarged so as to cover the entire first virtual image before the first display control process. executing the first display control process in
The display control device according to claim 1. The processor
Further executing a second display control process for shortening a display distance of the first virtual image when it is detected, estimated, or predicted that the posture variation of the moving body satisfies the predetermined condition;
The display control device according to any one of claims 1 to 3. The processor, in the second display control process,
Continuously or stepwise shortening the display distance of the first virtual image as time elapses;
The display control device according to claim 4. The processor executes the first display control process of adhering the plurality of image elements such that the gaps between the plurality of first image elements before the first display control process are eliminated.
The display control device according to any one of claims 1 to 5. The processor executes the first display control process of displaying a second image element arranged to fill the gap between the plurality of first image elements before the first display control process. do,
The display control device according to claim 1. The second image element has higher visibility than the first image element,
The display control device according to claim 7. The processor executes the first display control process to make a part of the plurality of first image elements before the first display control process lower than the predetermined visibility or to hide them.
The display control device according to any one of claims 1 to 8. A display control device according to any one of claims 1 to 9;
a light modulation element that emits display light;
a relay optical system for directing the display light from the light modulation element to the projection target,
A head-up display device characterized by: In a display control device method for controlling a head-up display device that displays an image superimposed on the foreground in front of a moving object,
displaying a first virtual image representing perspective, comprising a plurality of first image elements having gaps therebetween;
the plurality of first image elements represent the same type of information and have higher than predetermined visibility;
number of the gaps between the first image elements higher than the predetermined visibility included in the first virtual image when it is detected, estimated, or predicted that the posture variation of the moving body satisfies a predetermined condition; or performing a first display control process that shortens the gap;
A display control method characterized by:

Images