JP2023074703A - Display control device, head-up display device and display control method - Google Patents

Abstract

To suppress variations in an impression of the depth feeling of an image that is presented even if a position of a display region of a virtual image and an eye position of an observer are different.SOLUTION: A processor executes object movement processing of, while moving a virtual object VO, gradually changing a size As at a prescribed change rate G following the movement of the virtual object VO, and changing the change rate G on the basis of at least one of the position in the height direction of a display region 100, the eye position in the height direction of an observer in an eye box 200, the pitching attitude of a vehicle and information that can estimate these.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to a display control device, a head-up display device, and a display control method that are used in a vehicle and superimpose an image on the foreground of the vehicle for visual recognition.

Japanese Patent Laid-Open No. 2004-100003 describes a method in which a virtual object that appears to actually exist in the foreground (actual scene) of a vehicle is represented as an image with a sense of perspective, augmented reality (AR) is generated, and a virtual object ( image) and real objects (roads, other vehicles, pedestrians, etc.) present in the actual scene, thereby reducing the visual load on the observer (driver of the own vehicle) and information A head-up display device is described for recognizing the

WO2018/216553

A head-up display device can typically display a virtual image of an image only in a limited area (virtual image display area) viewed by an observer. It is assumed that the position of the virtual image display area is fixed even if the eye level (eye position) of the driver of the vehicle changes. ) and the eye height (eye position) of the driver of the vehicle, the virtual image display area viewed from the observer and the real view area outside the vehicle that overlaps differ. Specifically, if the real scene area where the virtual image display area overlaps at the reference eye level is defined as the reference real image area, the real scene area where the virtual image display area overlaps when viewed from a position higher than the reference eye level is the distance from the observer. When viewed, it becomes an area below the reference real-view area (in terms of perspective, it becomes an actual-view area closer to the reference real-view area than the reference real-view area).

Conversely, the real scene area where the virtual image display area overlaps when viewed from a position lower than the reference eye level is an area above the reference real scene area as seen from the observer (in terms of perspective, it is farther than the reference real scene area). the actual view area on the side). At this time, the overlapping area between the virtual image display area viewed from a position lower than the reference eye height and the road surface (or the real object on the road surface) is reduced. The head-up display device displays the virtual object (image) in a small size when the virtual object (image) is to be recognized in the distance, and displays the virtual object (image) in a large size when the virtual object (image) is to be recognized in the vicinity. ) can emphasize the sense of depth. It is assumed that it is difficult to compare the relative size with the virtual object (image), and it becomes difficult to grasp the sense of depth of the virtual object (image). In other words, it is assumed that the virtual object (image) displayed in the virtual image display area having a small area overlapping the road surface (or the real object on the road surface) gives the observer a reduced sense of depth.

A summary of certain embodiments disclosed herein follows. It should be understood that these aspects are presented only to provide the reader with an overview of these particular embodiments and are not intended to limit the scope of this disclosure. Indeed, the present disclosure may encompass various aspects not described below.

The outline of the present disclosure is to suppress the deterioration of the sense of depth given to the observer, more specifically, the impression of the sense of depth of the image presented even if the position of the display area of the virtual image and the eye position of the observer are different. It also relates to suppressing the variation of .

Therefore, the display control device according to the first embodiment described in this specification is a display control device that controls an image display unit that displays a virtual image of an image within a display area that overlaps the foreground when viewed from an eyebox in a vehicle. comprises one or more processors, a memory, and one or more computer programs stored in the memory and configured to be executed by the one or more processors, wherein the processors are virtual Object movement processing that gradually changes the size at a predetermined change rate as the virtual object moves while moving the object, the position in the height direction of the display area, and the eye position in the eye box in the height direction of the observer. , the pitching attitude of the vehicle, and change rate control processing for changing the rate of change based on at least one of information that can estimate these. The first embodiment has the advantage of being able to adjust the sense of depth of the virtual object by estimating that the real scene area where the virtual object overlaps changes and adjusting the rate of change in size accompanying the movement of the virtual object. have.

Further, in the display control device according to the second embodiment, which can be subordinate to the first embodiment, the object movement processing moves the virtual object based on the progress of the vehicle, and adjusts the first change rate according to the movement of the virtual object. and gradually changing the size at a second change rate larger than the first change rate along with the movement of the virtual object while moving the virtual object based on the progress of the vehicle. a position of the display area (100) in the height direction, eye position in the eye box (200) of the observer in the height direction, pitching of the vehicle The second rate of change is greater than the rate of change of the size of the real object in the real world as the vehicle progresses. Secondly, in the embodiment, the size of the virtual object is changed while moving the virtual object based on the progress of the vehicle, so that the virtual object easily blends in with the real scene. Secondly, in the second embodiment, in addition to this, the virtual object is moved while changing its size at a rate of change greater than the rate of change of the size of the real object in the real world as the vehicle advances. This has the advantage that it is possible to emphasize the sense of depth of the virtual object even in a situation where the sense of depth of the virtual object is lowered.

Further, in the display control device according to the third embodiment, which can be subordinate to the first or second embodiment, the processor sets the brightness of the virtual object having a large rate of change to be lower than the brightness of the virtual object having a small rate of change. A virtual object with a large rate of change in size with movement emphasizes the sense of depth, but it is also assumed that the viewer's visual attention is excessively deprived. Thirdly, in the embodiment, the sense of depth of the virtual object is emphasized by increasing the rate of change in size that accompanies movement. In the third embodiment, in addition to this, the brightness of a virtual object whose size change rate is large due to movement is set lower than the brightness of a virtual object whose size change rate is small due to movement. can be suppressed from being excessively deprived.

Further, in the display control device according to the fourth embodiment, which can be subordinate to the first or second embodiment, the processor, in the object movement processing, controls the distant display mode and the movement of the virtual object below the distant display mode. , and a near display mode having a larger size than the distant display mode, and in the far display mode, the brightness of the virtual object with a large change rate and the brightness of the virtual object with a small change rate are set to be the same, and the near display mode is set. In this aspect, the brightness of a virtual object with a large rate of change is set lower than the brightness of a virtual object with a small rate of change. A virtual object with a large rate of change in size as it moves emphasizes the sense of depth, and is displayed large especially in the near display mode, and it is assumed that the visual attention of the observer is excessively deprived. In the fourth embodiment, the brightness of a virtual object having a large rate of change in size with movement is adjusted to the brightness of a virtual object with a large rate of change in size with movement, especially in a neighborhood display mode that tends to deprive the viewer of visual attention due to its large size. Since the brightness is set to be lower than that of a small virtual object, it is possible to effectively prevent the viewer's visual attention from being excessively deprived.

Further, in the display control device according to the fifth embodiment that can be subordinate to the first to fourth embodiments, the processor sets the maximum size of the virtual object in the object movement processing, Move the virtual object while maintaining the size of the virtual object at the maximum size so that the size of the object does not exceed the maximum size. is moved while changing gradually. In the fifth embodiment, by setting an upper limit (maximum size) to the size associated with movement, it is possible to prevent the virtual object from being displayed in an excessively large size and thereby excessively depriving the observer's visual attention. can be done.

In particular, in the display control device according to the fifth embodiment, in the object movement processing, the processor performs 1) a distant display mode, and a virtual object moving lower than the distant display mode and having a size larger than the distant display mode. In the distant display mode, the brightness of a virtual object with a large rate of change is set to be the same as that of a virtual object with a small rate of change. 2) setting the brightness of the object to be lower than the brightness of the virtual object whose rate of change is small; Move the virtual object while maintaining the maximum size so that the size of the virtual object does not exceed the maximum size. In this fifth embodiment, in a neighborhood display mode that is particularly large in size and tends to deprive the visual attention of the observer, the brightness of a virtual object whose size change rate with movement is large is adjusted to the size change rate with movement. is lower than the luminance of the virtual object with a small V, it is possible to effectively prevent the observer's visual attention from being excessively deprived. In addition to this, by setting an upper limit (maximum size) for the size associated with movement, it is possible to prevent the virtual object from being displayed in an excessively large size and thereby excessively depriving the observer's visual attention.

Further, in the display control device according to the sixth embodiment that can be subordinate to the first to fifth embodiments, the processor controls the position of the display area in the height direction, the eye position of the observer in the eye box in the height direction, the vehicle Based on the pitching posture of the pitcher and at least one of the information that can be estimated, calculate the ratio of the road surface in the display area when observed from a predetermined position of the eye position or the eye box, and determine that the ratio is small. If so, increase the rate of change. In the sixth embodiment, a situation in which the observer's sense of depth of the virtual object is estimated to decrease based on the ratio of the road surface in the display area when observed from a predetermined eye position or eyebox position. can be determined, and the perspective of the virtual object can be effectively emphasized.

Further, a head-up display device according to the seventh embodiment includes the display control device according to any one of the first to sixth embodiments, a display for displaying an image on a display surface, and an image displayed by the display. and one or more relay optical systems (25) for displaying a virtual image of an image in a display area overlapping the foreground when viewed from the eyebox by projecting the display light onto an external projection target.

Further, the display control method according to the eighth embodiment is a display control method for controlling an image display unit that displays a virtual image of an image within a display area that overlaps the foreground when viewed from an eyebox in the vehicle, and moves a virtual object. Also, object movement processing for gradually changing the size at a predetermined change rate as the virtual object moves, the position in the height direction of the display area, the eye position in the eye box in the height direction of the observer, and the pitching of the vehicle. a rate of change control process for changing the rate of change based on at least one of posture and information from which these can be estimated.

FIG. 1 is a diagram showing an application example of a vehicle display system. FIG. 2 is a diagram showing the configuration of the image display unit. FIG. 3 is a block diagram of a vehicle display system. FIG. 4 is a diagram showing an example of a virtual object displayed by the head-up display device. AA to AC in the upper part of FIG. 4 are diagrams showing display examples of the virtual object (first virtual object) when the display area (first display area) is arranged at a relatively low position. BA to BC in the lower part of FIG. 4 are hypothetical images when the display area (second display area) is arranged at a relatively high position (the second display area is higher than the first display area). FIG. 10 is a diagram showing a display example of an object (second virtual object); FIG. 5 is a flow diagram illustrating a method of performing an object movement process that incrementally changes the size of a virtual object while moving it, according to some embodiments. FIG. 6 is a graph showing several embodiments of size change characteristics accompanying movement of a virtual object, where the horizontal axis represents the representation depth virtually set for the virtual object, and the vertical axis represents only the representation depth VL. This is the size (viewing angle) of the virtual object viewed from a distant position. FIG. 6B is a graph illustrating some embodiments of size change characteristics with movement of a virtual object. FIG. 6C is a graph illustrating some embodiments of size change characteristics with movement of a virtual object. FIG. 6D is a graph illustrating some embodiments of size change characteristics with movement of a virtual object. FIG. 6E is a graph illustrating some embodiments of size change characteristics with movement of a virtual object. FIG. 6F is a graph illustrating some embodiments of size change characteristics with movement of a virtual object. FIG. 7A is a graph showing several embodiments of brightness characteristics accompanying movement of a virtual object, where the horizontal axis is the virtual depth of representation of the virtual object and the vertical axis is the brightness of the virtual object. . FIG. 7B is a graph illustrating some embodiments of luminance characteristics as virtual objects move. FIG. 7C is a graph illustrating some embodiments of luminance characteristics as virtual objects move. FIG. 7D is a graph illustrating some embodiments of luminance characteristics as virtual objects move.

1-7D below provide a description of this embodiment. In addition, the present invention is not limited by the following embodiments (including the contents of the drawings). Of course, modifications (including deletion of constituent elements) can be added to the following embodiments. In addition, in the following description, descriptions of known technical matters are omitted as appropriate in order to facilitate understanding of the present invention.

Please refer to FIG. The vehicle display system 10 of the present embodiment includes an image display unit 20 , a display control device 30 that controls the image display unit 20 , and electronic devices that are connected to the display control device 30 and will be described later.

The image display unit 20 in the vehicle display system 10 is a head-up display (HUD: Head-Up Display) device provided in the dashboard 5 of the vehicle 1 . The image display unit 20 emits the display light 40 toward the front windshield 2 (which is an example of the projected portion), and the front windshield 2 receives the display light 40 of the image M displayed by the image display unit 20 as an eye. Reflect back to box 200 . By arranging the eye 4 in the eye box 200, the observer can see the virtual image V of the image M displayed by the image display unit 20 at a position overlapping the foreground which is the real space viewed through the front windshield 2. can be visually recognized. In the drawings used in this embodiment, the left-right direction of the vehicle 1 is the X-axis direction (the left side of the vehicle 1 facing forward is the positive X-axis direction), and the vertical direction is the Y-axis direction (a vehicle running on a road surface). 1 is the positive direction of the Y-axis), and the longitudinal direction of the vehicle 1 is the direction of the Z-axis (the front of the vehicle 1 is the positive direction of the Z-axis).

The “eyebox” used in the description of the present embodiment means (1) an area in which at least a portion of the virtual image V of the image M is visible, and a portion of the virtual image V of the image M is not visible outside the area; ) a region where at least part of the virtual image V of the image M can be visually recognized with a predetermined luminance or more within the region, and the entire virtual image V of the image M is less than the predetermined luminance outside the region, or (3) the image display unit 20 is an area where at least part of the virtual image V can be stereoscopically viewed, and part of the virtual image V is not stereoscopically viewed outside the area. That is, when the observer places the eyes (both eyes) 4 outside the eyebox 200, the observer cannot see the entire virtual image V of the image M, and perceives that the visibility of the entire virtual image V of the image M is very low. or the virtual image V of the image M cannot be viewed stereoscopically. The predetermined brightness is, for example, about 1/50 of the brightness of the virtual image of the image M viewed at the center of the eyebox.

The display area 100 is a planar, curved, or partially curved area on which an image M generated inside the image display unit 20 is formed as a virtual image V, and is also called an imaging plane. The display area 100 is a position where a display surface (e.g., an exit surface of a liquid crystal display panel) 21a of a display device 21, which will be described later, of the image display unit 20 is formed as a virtual image. 20 (in other words, the display area 100 has a conjugate relationship with the display surface 21a of the display device 21, which will be described later). It can be said that it corresponds to an image displayed on a display surface 21a of the display unit 20, which will be described later. It is preferable that the visibility of the display area 100 itself is so low that it is not actually visible to the observer's eyes 4 or is difficult to be visually recognized. In the display area 100, an angle (tilt angle θt in FIG. 1) formed between the left-right direction (X-axis direction) of the vehicle 1 and the horizontal direction (XZ plane), the center 205 of the eyebox 200, and the display area 100 The angle formed by the line segment connecting the upper end 101 and the line segment connecting the center of the eyebox and the lower end 102 of the display area 100 is defined as the vertical angle of view of the display area 100, and the bisector of this vertical angle of view and the horizontal and the angle (vertical arrangement angle θv in FIG. 1) formed with the direction (XZ plane) are set.

The display area 100 of the present embodiment has a tilt angle θt of approximately 90 [degrees] so that the display area 100 generally faces the front (positive direction of the Z axis). However, the tilt angle θt is not limited to this, and can be changed within the range of 0≦θt<90 [degree]. In this case, for example, the tilt angle θt may be set to 60 [degrees], and the display area 100 may be arranged such that the upper area is farther from the lower area as viewed from the observer.

FIG. 2 is a diagram showing the configuration of the HUD device 20 of this embodiment. The HUD device 20 includes a display 21 having a display surface 21a that displays an image M, and a relay optical system 25 .

The display 21 in FIG. 2 is composed of a liquid crystal display panel 22 and a light source unit 24 . The display surface 21a is the surface of the liquid crystal display panel 22 on the viewing side, and emits the display light 40 of the image M. As shown in FIG. By setting the angle of the display surface 21a with respect to the optical axis 40p of the display light 40 directed from the center of the display surface 21a to the eyebox 200 (the center of the eyebox 200) via the relay optical system 25 and the projection target portion, the display area 100 angles (including the tilt angle θt) can be set.

The relay optical system 25 is arranged on the optical path of the display light 40 emitted from the display device 21 (the light from the display device 21 toward the eyebox 200 ). consists of one or more optical elements that project onto the front windshield 2 of the The relay optical system 25 of FIG. 2 includes one concave first mirror 26 and one planar second mirror 27 .

The first mirror 26 has, for example, a free-form surface shape with positive optical power. In other words, the first mirror 26 may have a curved shape with different optical power for each region. can be different. Specifically, the first image light 41, the second image light 42, and the third image light 43 (see FIG. 2) traveling from each region of the display surface 21a toward the eyebox 200 are added by the relay optical system 25. The optical power may be different.

The second mirror 27 is, for example, a flat mirror, but is not limited to this, and may be a curved surface having optical power. That is, the relay optical system 25 is added according to the area (optical path) through which the display light 40 passes by synthesizing a plurality of mirrors (for example, the first mirror 26 and the second mirror 27 of this embodiment). Different optical powers may be used. Note that the second mirror 27 may be omitted. That is, the display light 40 emitted from the display device 21 may be reflected by the first mirror 26 to the projection target (front windshield) 2 .

Also, in this embodiment, the relay optics 25 includes two mirrors, but is not limited to this, and additionally or alternatively, one or more refractive optics such as lenses. It may comprise a member, a diffractive optical member such as a hologram, a reflective optical member, or a combination thereof.

In addition, the relay optical system 25 of the present embodiment has a function of setting the distance to the display area 100 and a virtual image obtained by enlarging the image displayed on the display surface 21a with this curved surface shape (an example of optical power). In addition to this, it may have a function of suppressing (correcting) the distortion of the virtual image that may occur due to the curved shape of the front windshield 2 .

Also, the relay optical system 25 may be attached with actuators 28 and 29 controlled by the display control device 30 and may be rotatable.

The liquid crystal display panel 22 receives light from the light source unit 24 and emits spatially modulated display light 40 toward the relay optical system 25 (second mirror 27). The liquid crystal display panel 22 has, for example, a rectangular shape whose short side is the direction in which the pixels are arranged corresponding to the vertical direction (Y-axis direction) of the virtual image V viewed by the observer. An observer visually recognizes the light transmitted through the liquid crystal display panel 22 through the virtual image optical system 90 . The virtual image optical system 90 is a combination of the relay optical system 25 and the front windshield 2 shown in FIG.

The light source unit 24 is composed of a light source (not shown) and an illumination optical system (not shown).

A light source (not shown) is, for example, a plurality of chip-type LEDs, and emits illumination light to a liquid crystal display panel (an example of a spatial light modulation element) 22 . The light source unit 24 is composed of four light sources, for example, and is arranged in a line along the long side of the liquid crystal display panel 22 . The light source unit 24 emits illumination light toward the liquid crystal display panel 22 under the control of the display control device 30 . The configuration of the light source unit 24 and the arrangement of the light sources are not limited to this.

The illumination optical system (not shown) includes, for example, one or more lenses (not shown) arranged in the emission direction of the illumination light from the light source unit 24, and a diffuser lens (not shown) arranged in the emission direction of the one or more lenses. a plate (not shown);

The display device 21 may be a self-luminous display or a projection display that projects an image onto a screen. In this case, the display surface 21a is the screen of a projection display.

Further, the display 21 may be attached with an actuator (not shown) including a motor controlled by the display control device 30 so that the display surface 21a can be moved and/or rotated.

The relay optical system 25 has two rotation axes (first rotation axis AX1 and second rotation axis AX2) for moving the eyebox 200 in the vertical direction (Y-axis direction). When the HUD device 20 is attached to the vehicle 1, the first rotation axis AX1 and the second rotation axis AX2 are not perpendicular to the left-right direction (X-axis direction) of the vehicle 1 (in other words, the YZ plane and the second rotation axis AX2). parallel). Specifically, the angle between the first rotation axis AX1 and the second rotation axis AX2 and the left-right direction (X-axis direction) of the vehicle 1 is set to be less than 45 degrees. It is set to less than 20 [degree].

The HUD device 20 includes a first actuator 28 that rotates the first mirror 26 about a first rotation axis AX1, and a second actuator 29 that rotates the first mirror 26 about a second rotation axis AX2. In other words, the HUD device 20 rotates one relay optical system 25 about two axes (first rotation axis AX1 and second rotation axis AX2). Note that the first actuator 28 and the second actuator 29 may be configured as a single integrated biaxial actuator.

Also, the HUD device 20 in another embodiment rotates the two relay optical systems 25 on two axes (first rotation axis AX1 and second rotation axis AX2). For example, the HUD device 20 includes a first actuator 28 that rotates a first mirror 26 about a first rotation axis AX1 and a second actuator 29 that rotates a second mirror 27 about a second rotation axis AX2. You can

The amount of vertical movement of the eyebox 200 is relatively large due to the rotation of the first rotation axis AX1, and the amount of vertical movement of the display area 100 is relatively large due to the rotation of the second rotation axis AX2. If so, the arrangement of the first rotation axis AX1 and the second rotation axis AX2 is not limited to these. Actuator actuation may also include movement in addition to or instead of rotation.

Also, the HUD device 20 in other embodiments may not drive the relay optical system 25 . In other words, the HUD device 20 may not have an actuator that rotates and/or rotates the relay optics 25 . The HUD device 20 of this embodiment may include a wide eyebox 200 to cover the range of eye heights of the drivers expected to use the vehicle 1 .

Under the control of the display control device 30, which will be described later, the image display unit 20 has a virtual object VO at a position superimposed on the foreground, which is a real space (real scene) viewed through the front windshield 2 of the vehicle 1. The road surface 310 of the driving lane, the fork 330, road signs, obstacles (pedestrians 320, bicycles, motorcycles, other vehicles, etc.), and the vicinity of real objects 300 such as features (buildings, bridges, etc.), real objects 300 , or at a position set based on the real object 300, visual augmented reality (AR) can be viewed by an observer (typically, sitting in the driver's seat of the vehicle 1). It can also be made to perceive by the observer). In the description of this embodiment, an image whose displayed position can be changed according to the position of the real object 300 existing in the real scene is defined as an AR image. An image to be set is defined as a non-AR image. Examples of AR images are described below.

FIG. 3 is a block diagram of a vehicle display system 10, according to some embodiments. The display controller 30 comprises one or more I/O interfaces 31 , one or more processors 33 , one or more object movement processing circuits 35 and one or more memories 37 . Various functional blocks depicted in FIG. 3 may be implemented in hardware, software, or a combination of both. FIG. 3 is only one embodiment and the illustrated components may be combined into fewer components or there may be additional components. For example, object movement processing circuitry 35 (eg, a graphics processing unit) may be included in one or more processors 33 .

As shown, processor 33 and object movement processing circuitry 35 are operatively coupled with memory 37 . More specifically, the processor 33 and the object movement processing circuit 35 execute programs stored in the memory 37 to generate and/or transmit image data, for example, to display the vehicle display system 10 (image data). The display unit 20) can be operated. Processor 33 and/or object movement processing circuitry 35 may include at least one general purpose microprocessor (e.g., central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA). ), or any combination thereof. Memory 37 includes any type of magnetic media such as hard disks, any type of optical media such as CDs and DVDs, any type of semiconductor memory such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

As shown, processor 33 is operatively coupled with I/O interface 31 . The I/O interface 31 communicates with, for example, a vehicle ECU 401 (described later) provided in the vehicle or other electronic devices (reference numerals 403 to 417 described later) according to CAN (Controller Area Network) standards (also known as CAN communication). ). The communication standard adopted by the I/O interface 31 is not limited to CAN. : MOST is a registered trademark), a wired communication interface such as UART or USB, or a personal area network (PAN) such as a Bluetooth network, a local network such as an 802.11x Wi-Fi network. It includes an in-vehicle communication (internal communication) interface, which is a short-range wireless communication interface within several tens of meters such as an area network (LAN). In addition, the I / O interface 31 is a wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e base (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, An external communication (external communication) interface such as a wide area communication network (for example, Internet communication network) may be included according to a cellular communication standard such as 5G.

As shown, the processor 33 is interoperably coupled with the I/O interface 31 to communicate with various other electronic devices and the like connected to the vehicle display system 10 (I/O interface 31). can be given and received. The I/O interface 31 includes, for example, a vehicle ECU 401, a road information database 403, a vehicle position detector 405, a vehicle exterior sensor 407, an operation detector 409, an eye position detector 411, a gaze direction detector (not shown), a mobile An information terminal 415, an external communication device 417, etc. are operably connected. Note that the I/O interface 31 may include a function of processing (converting, calculating, and analyzing) information received from other electronic devices or the like connected to the vehicle display system 10 .

Display 21 is operatively linked to processor 33 and object movement processing circuitry 35 . Accordingly, the image displayed by image display 20 may be based on image data received from processor 33 and/or object movement processing circuitry 35 . The processor 33 and object movement processing circuit 35 control the image displayed by the image display section 20 based on the information obtained from the I/O interface 31 .

The vehicle ECU 401 detects the state of the vehicle 1 (e.g., travel distance, vehicle speed, accelerator pedal opening, brake pedal opening, engine throttle opening, injector fuel injection amount, engine speed, etc.) from sensors and switches provided in the vehicle 1. , motor rotation speed, steering angle, shift position, drive mode, various warning states, attitude (including roll angle and/or pitch angle), vehicle vibration (including magnitude, frequency, and/or frequency )), etc., and collects and manages (may also include control) the states of the vehicle 1. ) can be output to the processor 33 of the display controller 30 . The vehicle ECU 401 simply transmits a numerical value detected by a sensor or the like (for example, the pitching angle is 3 [degrees] in the forward tilting direction) to the processor 33, or alternatively, transmits the numerical value detected by the sensor. Determination results based on one or more states of the vehicle 1 including (for example, the vehicle 1 satisfies a predetermined forward lean condition), or/and analysis results (for example, the degree of brake pedal opening information that the vehicle is leaning forward due to braking) may be sent to the processor 33 . For example, the vehicle ECU 401 may output to the display control device 30 a signal indicating that the vehicle 1 satisfies a predetermined condition stored in advance in a memory (not shown) of the vehicle ECU 401 . Note that the I/O interface 31 may acquire the above-described information from sensors and switches provided in the vehicle 1 without using the vehicle ECU 401 .

In addition, the vehicle ECU 401 may output to the display control device 30 an instruction signal that instructs an image to be displayed by the vehicle display system 10. At this time, the coordinates, size, type, display mode of the image, the need for notification of the image, and so on. Necessity-related information that serves as a basis for determining the degree and/or the necessity of notification may be added to the instruction signal and transmitted.

The road information database 403 is included in a navigation device (not shown) provided in the vehicle 1 or an external server connected to the vehicle 1 via an external communication interface (I/O interface 31). Based on the position of the vehicle 1 acquired from the unit 405, road information (lanes, white lines, stop lines, crosswalks, road width, number of lanes, intersections, curves, forks, traffic regulations, etc.), presence/absence, position (including distance to vehicle 1), direction, shape, type of feature information (buildings, bridges, rivers, etc.) , detailed information, etc. may be read and sent to the processor 33 . The road information database 403 may also calculate an appropriate route (navigation information) from the departure point to the destination and output to the processor 33 a signal indicating the navigation information or image data indicating the route.

The vehicle position detection unit 405 is a GNSS (global navigation satellite system) or the like provided in the vehicle 1, detects the current position and direction of the vehicle 1, and transmits a signal indicating the detection result via the processor 33. , or directly to the road information database 403, a portable information terminal 415 and/or an external communication device 417, which will be described later. The road information database 403, the portable information terminal 415, and/or the external communication device 417 acquire the position information of the vehicle 1 from the vehicle position detection unit 405 continuously, intermittently, or at each predetermined event. , information about the surroundings of the vehicle 1 may be selected/generated and output to the processor 33 .

The vehicle exterior sensor 407 detects the real object 300 existing around the vehicle 1 (front, side, and rear). Real objects 300 detected by the vehicle exterior sensor 407 include, for example, obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), road surface 310 of driving lanes described later, lane markings, roadside objects, and/or features (buildings etc.). Exterior sensors include, for example, radar sensors such as millimeter wave radar, ultrasonic radar, and laser radar, cameras, or a detection unit consisting of a combination thereof, and processing detection data from the one or more detection units ( data fusion) and a processing device. A conventional well-known technique is applied to object detection by these radar sensors and camera sensors. By object detection by these sensors, the presence or absence of a real object in the three-dimensional space, and if the real object exists, the position of the real object (relative distance from the vehicle 1, the traveling direction of the vehicle 1 in the front-back direction) horizontal position, vertical position, etc.), size (horizontal direction (horizontal direction), height direction (vertical direction), etc.), movement direction (horizontal direction (horizontal direction), depth direction (front-rear direction)), moving speed (lateral direction (left-right direction), depth direction (front-rear direction)), and/or type, etc. may be detected. One or a plurality of sensors outside the vehicle 407 detects a real object in front of the vehicle 1 in each sensor detection period, and detects real object information (presence or absence of a real object, existence of a real object), which is an example of real object information. If so, information such as the position, size and/or type of each real object) can be output to the processor 33 . Note that these pieces of real object information may be transmitted to the processor 33 via another device (for example, the vehicle ECU 401). Also, when using a camera as a sensor, an infrared camera or a near-infrared camera is desirable so that a real object can be detected even when the surroundings are dark, such as at night. Moreover, when using a camera as a sensor, a stereo camera that can acquire distance and the like by parallax is desirable.

The operation detection unit 409 is, for example, a CID (Center Information Display) of the vehicle 1, a hardware switch provided on an instrument panel, or a software switch combining an image and a touch sensor. It outputs to the processor 33 operation information based on the operation by the passenger (the user sitting in the driver's seat and/or the user sitting in the front passenger seat). For example, the operation detection unit 409 sets the display area setting information based on the operation of moving the display area 100, the eye box setting information based on the operation of moving the eye box 200, and the observer's eye position 4 by the user's operation. Information based on the operation (an example of eye position information) and the like are output to the processor 33 .

The eye position detection unit 411 may include a camera such as an infrared camera that detects the positions of the eyes of an observer sitting in the driver's seat of the vehicle 1 , and may output the captured image to the processor 33 . The processor 33 acquires a captured image (an example of information that can estimate the positions of the eyes) from the eye position detection unit 411, and analyzes the captured image to specify the positions of the eyes of the observer. The eye position detection unit 411 may analyze the image captured by the camera and output to the processor 33 a signal indicating the position of the observer's eyes, which is the analysis result. In addition, the method of acquiring the position of the eyes of the observer of the vehicle 1 or the information that can estimate the position of the observer's eyes is not limited to these, and a known eye position detection (estimation) technique is used. may be obtained by The processor 33 adjusts at least the position of the image based on the position of the observer's eyes, thereby superimposing the image superimposed on the desired position of the foreground (position having a specific positional relationship with the real object). The detected observer (observer) may visually recognize it.

The line-of-sight direction detection unit (not shown) may include an infrared camera or a visible light camera that captures the face of an observer sitting in the driver's seat of the vehicle 1 , and may output the captured image to the processor 33 . The processor 33 acquires a captured image (an example of information that can estimate the line-of-sight direction) from a line-of-sight direction detection unit (not shown), and analyzes the captured image to ) can be specified. Note that the line-of-sight direction detection unit (not shown) may analyze the captured image from the camera and output to the processor 33 a signal indicating the line-of-sight direction (and/or the gaze position) of the observer, which is the analysis result. . Note that the method of acquiring information that can estimate the line-of-sight direction of the observer of the vehicle 1 is not limited to these, and includes an EOG (Electro-oculogram) method, a corneal reflection method, a scleral reflection method, and Purkinje image detection. may be obtained using other known gaze direction detection (estimation) techniques such as method, search coil method, infrared fundus camera method.

Vehicle posture detection unit 413 includes one or more of, for example, a gyro sensor, an acceleration sensor, and a height sensor, detects posture change information, and outputs the information to processor 33 . The posture change information includes the angular velocity, acceleration, height of the moving body, vehicle posture (pitch angle, roll angle, etc.), frequency of change in the vehicle posture (vibration frequency), and the like.

The personal digital assistant 415 is a smart phone, a laptop computer, a smartwatch, or other information device that can be carried by an observer (or another occupant of the vehicle 1). By pairing with the mobile information terminal 415, the I/O interface 31 can communicate with the mobile information terminal 415, and the data recorded in the mobile information terminal 415 (or the server through the mobile information terminal) can be transmitted. to get The mobile information terminal 415 has, for example, the same functions as the road information database 403 and the own vehicle position detection unit 405 described above, acquires the road information (an example of real object related information), and transmits it to the processor 33. good too. The mobile information terminal 415 may also acquire commercial information (an example of real object related information) related to commercial facilities near the vehicle 1 and transmit it to the processor 33 . The mobile information terminal 415 transmits schedule information of the owner of the mobile information terminal 415 (for example, an observer), incoming call information at the mobile information terminal 415, mail reception information, etc. to the processor 33, and the processor 33 and the object Movement processing circuitry 35 may generate and/or transmit image data for these.

The external communication device 417 is a communication device that exchanges information with the vehicle 1, for example, other vehicles connected to the vehicle 1 by vehicle-to-vehicle communication (V2V: vehicle-to-vehicle communication), pedestrian-to-vehicle communication (V2P: vehicle-to-pedestrian ) connected by pedestrians (mobile information terminals carried by pedestrians), network communication equipment connected by road-to-vehicle communication (V2I: Vehicle To roadside Infrastructure), broadly speaking, communication with vehicle 1 (V2X : Vehicle To Everything). The external communication device 417 acquires, for example, the positions of pedestrians, bicycles, motorcycles, other vehicles (preceding vehicles, etc.), road surfaces, lane markings, roadside objects, and/or features (buildings, etc.), and sends them to the processor 33. may be output. In addition, the external communication device 417 has the same function as the vehicle position detection unit 405 described above, may acquire the position information of the vehicle 1 and transmit it to the processor 33, and furthermore may store the information of the road information database 403 described above. It may also have a function to acquire the road information (an example of real object related information) and transmit it to the processor 33 . Information acquired from the external communication device 417 is not limited to the above.

Software components stored in memory 37 include determination module 502 , object movement processing module 504 , eye position detection module 512 , vehicle orientation detection module 514 , display area setting module 516 , and overlapping area calculation module 518 .

FIG. 4 is a diagram showing an example of a virtual object VO displayed by the head-up display device 20. As shown in FIG. AA to AC in the upper part of FIG. 4 are diagrams showing display examples of the virtual object VO (first virtual object V10) when the display area 100 (first display area 110) is arranged at a relatively low position. be. 4AA (V11), FIG. 4AB (V12), and FIG. 4AC (V13) in the order of FIG. 4AA (V11), FIG. 4AB (V12), and FIG. The size is gradually increased while moving toward the side (Z-axis negative direction). The rate of change in size accompanying the movement of the first virtual object V10 at this time (hereinafter also referred to as the rate of change in size) G is assumed to be the first rate of change G10.

In BA to BC in the lower part of FIG. 4, the display area 100 (second display area 120) is arranged at a relatively high position (the second display area 120 is higher than the first display area 110). FIG. 10 is a diagram showing a display example of a virtual object VO (second virtual object V20) when 4BA (V21), FIG. 4BB (V22), and FIG. 4BC (V23), the processor 33 moves the virtual object VO (V20) from the front side (positive direction of the Z-axis) to the rear side in the traveling direction of the vehicle over time. The size is gradually increased while moving toward the side (Z-axis negative direction). The rate of change in size accompanying the movement of the second virtual object V20 at this time (hereinafter also referred to as the rate of change in size) G is assumed to be the second rate of change in size G2. The processor 33 sets the second size change rate G2 to be greater than the first change rate G10. That is, the processor 33 increases the size change rate G in a state where the virtual object VO can be viewed at a relatively high position with respect to the foreground, thereby emphasizing the sense of depth resulting from the size change. Reference character PP indicates an area (overlapping area) where the display area 100 and the driving lane overlap. The second overlapping area PP2 in the second display area 120 arranged at a relatively high position is narrower than the first overlapping area PP1 in the first display area 110 arranged at a relatively low position. The overlapping area PP will be described later.

FIG. 5 is a flow diagram illustrating a method S100 of performing an object movement process that incrementally changes the size of a virtual object while moving it, according to some embodiments. The method S100 is performed in an image display unit 20 including a display and a display control device 30 that controls the image display unit 20. FIG. Some acts in method S100 are optionally combined, some acts are optionally reordered, and some acts are optionally omitted.

By executing the determination module 502, the processor 33 determines whether or not a predetermined condition for increasing the rate of change G in size accompanying movement of the virtual object VO is satisfied (step S110). The determination module 502 in some embodiments determines whether the virtual object VO is in the foreground based on the position of the display area 100 (S120), the observer's eye position (S130), the pose of the vehicle 1 (S140), or a combination thereof. On the other hand, when it is estimated to be visually recognized at a relatively high position, it is determined that the predetermined condition is satisfied.

The determination module 502 determines whether the state of the relay optical system satisfies a predetermined condition (S122).

The display area setting module 516 sets the amount of rotation (angle) of the first actuator 28 and the amount of rotation (angle) of the second actuator 29 based on the input information about the eye position 4 of the observer and setting information. , the state (angle, position) of the relay optical system 25 can be controlled. The display area setting module 516 determines the rotation amount (angle) of the first actuator 28 and the second actuator 29 based on the eye position information detected by the eye position detection module 512 and the eye position estimation information estimated by the eye position detection module 512 . includes various software components for performing various operations related to setting the amount of rotation (angle) of the . That is, the display area setting module 516 uses the eye position or information that can estimate the eye position to determine the amount of rotation (angle) about the first rotation axis AX1 and the rotation amount (angle) about the second rotation axis AX2. Table data for setting the amount of rotation (angle), an arithmetic expression, and the like may be included. The position (height) of the display area 100 can be determined by the amount of rotation (angle) of the actuator. Therefore, the rotation amount (angle) of the actuator from which the state of the relay optical system 25 can be estimated is an example of information from which the position of the display area 100 can be estimated. Based on this, when it is estimated that the virtual object VO is visually recognized at a relatively high position with respect to the foreground, it may be determined that the predetermined condition is satisfied. The determination module 502 can include thresholds, table data, arithmetic expressions, etc. for determining whether or not a predetermined condition is satisfied based on the amount of rotation (angle) of the actuator.

In addition, the display area setting module 516 determines the amount of rotation (angle) about the first rotation axis AX1 and the rotation amount (angle) about the second rotation axis AX2 based on an operation by the operation detection unit 409 or an instruction from the vehicle ECU 401. A rotation amount (angle) may be set. For example, the display area setting module 516 (1) position information of the observer's favorite eyebox (an example of eyebox position setting information) acquired from the observer identification unit (not shown), position information of the favorite display area (display an example of area setting information), (2) display area setting information based on an operation to move the display area 100 based on the user's operation, acquired from the operation detection unit 409 provided in the vehicle 1; (3) display area setting information indicating the position of the display area 100 determined by the vehicle ECU 401 and eye box setting information indicating the position of the eye box 200 acquired from the vehicle ECU 401; setting the rotation amount (angle) of the first actuator 28 about the rotation axis AX1 and the rotation amount (angle) of the second actuator 29 about the second rotation axis AX2. It contains various software components for performing various operations. Therefore, the information based on the operation by the operation detection unit 409 and the instruction from the vehicle ECU 401 is an example of information that can estimate the position of the display area 100, and the determination module 502 determines the position of the virtual object VO based on this information. is estimated to be viewed at a relatively high position with respect to the foreground, it may be determined that a predetermined condition is satisfied.

The determination module 502 determines whether or not the usage area of the display satisfies a predetermined condition (S124).

Further, the display area setting module 516 may change the area to be used of the display surface 21a of the display 21 based on the input information about the eye position 4 of the observer and setting information. That is, the display area setting module 516 can also change the position of the display area 100 used for displaying the virtual image V by changing the area used for displaying the image on the display surface 21 a of the display device 21 . Therefore, the information indicating the area used for displaying the image on the display surface 21a of the display 21 is an example of information that enables estimation of the position of the display area 100, and the determination module 502 indicates the use area of the display 21. Based on the information, when it is estimated that the virtual object VO is viewed at a relatively high position with respect to the foreground, it may be determined that the predetermined condition is satisfied. The determination module 502 can include threshold values, table data, arithmetic expressions, etc. for determining whether or not a predetermined condition is satisfied based on information indicating the use area of the display device 21 .

The eye position detection module 512 detects the eye position of the observer of the vehicle 1 . The eye position detection module 512 determines where the height of the observer's eyes is in height regions provided in a plurality of stages, detects the height of the observer's eyes (position in the Y-axis direction), Detection of the eye height (Y-axis position) and depth direction position (Z-axis direction position) of the observer, and / or detection of the observer's eye position (X, Y, Z-axis position), includes various software components for performing various operations related to the The eye position detection module 512, for example, acquires the position of the observer's eyes from the eye position detection unit 411, or obtains information from the eye position detection unit 411 that can estimate the position of the observer's eyes, including the height of the observer's eyes. and estimate the eye position, including the eye height of the observer. Information from which the eye position can be estimated may be, for example, the position of the driver's seat of the vehicle 1, the position of the observer's face, the height of the sitting height, the value input by the observer through an operation unit (not shown), and the like. When the eye position is high, the virtual object VO is superimposed on a relatively low position in the foreground of the observer, and when the eye position is low, the virtual object VO is superimposed on a relatively high position in the foreground of the observer and is visually recognized. It will be done. Therefore, the information indicating the position (height) of the eyes of the observer of the vehicle 1 or the information capable of estimating the eye position is an example of the information capable of estimating the relative position (height) of the overlap of the display area 100 with respect to the foreground. and the determination module 502 determines that the virtual object VO is viewed at a relatively high position with respect to the foreground based on information indicating the position (height) of the observer's eyes or information from which the eye positions can be estimated. If it is estimated, it may be determined that a predetermined condition is satisfied (S130). Based on information indicating the position (height) of the eyes of the observer or information capable of estimating the eye positions, the determination module 502 uses threshold values, table data, arithmetic expressions, etc. for determining whether or not a predetermined condition is satisfied. can include

A vehicle attitude detection module 514 is mounted on the vehicle 1 and detects the attitude of the vehicle 1 . The vehicle attitude detection module 514 determines where the attitude of the vehicle 1 is in the attitude area provided in a plurality of stages, detects the angle (pitching angle, rolling angle) of the vehicle 1 in the earth coordinate system, and detects the angle of the vehicle 1 with respect to the road surface. It includes various software components for performing various operations related to detection of angles (pitch angle, roll angle) and/or detection of the height of the vehicle 1 with respect to the road surface (Y-axis position). The vehicle attitude detection module 514 analyzes, for example, a three-axis acceleration sensor (not shown) provided in the vehicle 1 and the three-axis acceleration detected by the three-axis acceleration sensor to determine the position of the vehicle 1 with respect to the horizontal plane. A pitching angle (vehicle attitude) is estimated, and vehicle attitude information including information about the pitching angle of the vehicle 1 is output to the processor 33 . The vehicle posture detection module 514 may be configured by a height sensor (not shown) arranged near the suspension of the vehicle 1 other than the three-axis acceleration sensor described above. At this time, the vehicle attitude detection module 514 estimates the pitching angle of the vehicle 1 as described above by analyzing the height of the vehicle 1 from the ground detected by the height sensor, and obtains information about the pitching angle of the vehicle 1. to the processor 33. The method by which the vehicle attitude detection module 514 obtains the pitching angle of the vehicle 1 is not limited to the method described above, and the pitching angle of the vehicle 1 may be obtained using a known sensor or analysis method. When the vehicle 1 leans forward (pitch down), the virtual object VO is superimposed on a relatively low position in the foreground of the observer, and when the vehicle 1 leans backward (pitch up), the virtual object VO , will be superimposed on a relatively high position in the foreground of the observer. Therefore, the information indicating the attitude (pitching angle) of the vehicle 1 is an example of information capable of estimating the relative position (height) where the display area 100 overlaps with the foreground. pitching angle), if it is estimated that the virtual object VO is viewed at a relatively high position with respect to the foreground, it may be determined that a predetermined condition is satisfied (S140). The determination module 502 may include thresholds, table data, arithmetic expressions, etc. for determining whether or not a predetermined condition is satisfied based on information indicating the attitude (pitching angle) of the vehicle 1 .

The overlapping area calculation module 518 calculates the area of the overlapping area PP between the display area 100 and the driving lane. The overlapping area calculation module 518 calculates the area of the overlapping area PP between the display area 100 and the driving lane based on the position of the driving lane detected by the sensor 407 outside the vehicle and the set position of the display area 100 set by the display area setting module 516. Calculate In the example of FIG. 4AC, the first display area 110 is located relatively low, so the overlap area PP1 is wide. On the other hand, in the example of FIG. 4BC, the second display area 120 is positioned relatively high, so the overlapping area PP2 is narrower than in the example of FIG. 4AC. Therefore, the information indicating the area of the superimposed region PP is an example of information capable of estimating the relative position (height) where the display region 100 overlaps the foreground. Based on this, when it is estimated that the virtual object VO is viewed at a relatively high position with respect to the foreground, it may be determined that a predetermined condition is satisfied (not shown). The determination module 502 may include thresholds, table data, arithmetic expressions, etc. for determining whether or not a predetermined condition is satisfied based on information indicating the area of the superimposed region PP.

Next, in step S150, the processor 33 executes the object movement processing module 504 to gradually change the size of the virtual object VO while moving it (object movement processing). If the object movement processing module 504 determines in step S110 that the predetermined condition is satisfied, the object movement processing module 504 increases the rate of change G of the size accompanying the movement of the virtual object VO (sets it to the second rate of change G20). Processing is executed (step S160). On the other hand, if it is determined in step S110 that the predetermined condition is not satisfied, the object movement processing module 504 reduces the rate of change G of the size accompanying the movement of the virtual object VO (to the first rate of change G10 (G10<G20)). setting) is executed (not shown).

6A to 6F are graphs showing several embodiments of the size change characteristic Q accompanying the movement of the virtual object VO, where the horizontal axis is the representation depth VL that is virtually set for the virtual object VO, The vertical axis is the size (viewing angle) As of the virtual object VO viewed from a position separated by the representation depth VL. Here, the size (visual angle) As of the virtual object VO is the angle formed by the visual target (virtual object VO) viewed from the viewpoint, and is also called visual angular or angular size. Assuming that the virtual size of the virtual object VO is BS, and the size (viewing angle) As of the virtual object VO viewed from a position distant by the representation depth VL, the relational expression As=arctan (BS/VL) is obtained. That is, at a position where the representation depth VL is deep, the rate of change of the size (visual angle) As with respect to the change of the representation depth VL is small, and at a position with the shallow representation depth VL, the rate of change of the size (visual angle) As with respect to the change of the representation depth VL is growing.

In the embodiment shown in FIG. 6A, the second rate of change G21 (G20) is greater than the first rate of change G11 (G10) over the representation depth VL. In other words, the slope (second change rate G21) of the second change characteristic Q21 (Q20) at each representation depth VL is equal to the slope (first change rate G21) of the first change characteristic Q11 (Q10) at each representation depth VL (first is greater than the rate of change G11).

In the embodiment shown in FIG. 6B, the second rate of change G22 (G20) is greater than the first rate of change G12 (G10) at a fraction of the representational depth VL. Specifically, the second rate of change G22 (G20) is greater than the first rate of change G12 (G10) only in a portion (neighboring region) where the depth of expression VL is shallow, and in the other portion of the depth of expression VL, The second rate of change G22 (G20) is equal to the first rate of change G12 (G10). In other words, in the region near the expression depth VL, the slope of the second change characteristic Q22 (Q20) (second change rate G22) is the same as the slope of the first change characteristic Q12 (Q10) (first change rate G12 ), the slope of the second change characteristic Q22 (Q20) (the second rate of change G22) is the same as the slope of the first change characteristic Q12 (Q10) (the first change rate G12).

In the embodiment shown in FIG. 6C, the second rate of change G23 (G20) is greater than the rate of change G30 of the size of real objects in the real world as the vehicle travels. In other words, the slope (second rate of change G23) of the second change characteristic Q23 (Q20) at each representation depth VL is the slope ( larger than the change rate G30) over the whole. As a result, even in a situation where the display position of the virtual object VO is separated from the road surface and the sense of depth is reduced, the sense of depth of the virtual object can be further emphasized. Also, at this time, the first rate of change G13 (G10) is equal to the rate of change G30 of the real object over the entire representation depth VL. In other words, the slope (first change rate G13) of the first change characteristic Q13 (Q10) at each representation depth VL is the slope of the real object change characteristic Q30 at each representation depth VL (real object change rate G30).

In the embodiment shown in FIG. 6D, the first rate of change G14 (G10) is less than the rate of change G30 of the size of the real object in the real world as the vehicle travels, and the second rate of change G24 (G20) is: It is larger than the change rate G30 of the size of the real object in the real world as the vehicle travels. In other words, the slope (first change rate G14) of the first change characteristic Q14 (Q10) at each representation depth VL is the slope ( The slope (second rate of change G24) of the second change characteristic Q24 (Q20) at each expression depth VL is smaller than the rate of change G30) over the entire area, and is the size change characteristic of the real object in the real world. It is larger over the whole than the slope (change rate G30) at each expression depth VL of Q30.

In the embodiment shown in FIG. 6E, the first rate of change G15 (G10) is less than the rate of change G30 of the size of the real object in the real world as the vehicle travels, and the second rate of change G25 (G20) is: It is equal to the rate of change G30 of the size of the real object in the real world as the vehicle travels. In other words, the slope (first change rate G15) of the first change characteristic Q15 (Q10) at each representation depth VL is the slope ( The slope (second rate of change G25) of the second change characteristic Q25 (Q20) at each expression depth VL is smaller than the rate of change G30) over the entire area, and is the size change characteristic of the real object in the real world. The slope (change rate G30) of Q30 at each expression depth VL is equal throughout.

In the embodiment shown in FIG. 6F, the second change characteristic Q26 (Q20) moves the virtual object VO while maintaining the size (visual angle) As of Asm so as not to exceed the maximum size Asm. According to this, by setting an upper limit (maximum size) to the size associated with movement, it is possible to prevent the virtual object from being displayed in an excessively large size and excessively depriving the observer's visual attention. .

7A to 7D are graphs showing several embodiments of the brightness characteristic R accompanying the movement of the virtual object VO, where the horizontal axis is the representation depth VL that is virtually set for the virtual object VO, and the vertical axis is is the luminance of the virtual object VO.

In the embodiment shown in FIG. 7A, the second luminance characteristic R21 (R20) in the second object movement process is set lower than the first luminance characteristic R11 (R10) in the first object movement process. According to this, the brightness of the virtual object for which the second object-moving process is executed, the rate of size change of which increases with movement, is set lower than the brightness of the virtual object of which the rate of size change, which accompanies movement, is small. It is possible to prevent the observer's visual attention from being excessively deprived.

In the embodiment shown in FIG. 7B, the second luminance characteristic R22 (R20) in the second object movement process is set gradually lower as the representation depth VL becomes shallower. According to this, as the size (visual angle) increases as the virtual object moves, the brightness of the virtual object is gradually lowered, so that it is possible to prevent the observer's visual attention from being excessively deprived. .

In the implementation shown in FIG. 7C, the second luminance characteristic R23 (R20) in the second object movement process is set lower than the first luminance characteristic R13 (R10) only in a portion where the representation depth VL is shallow. According to this, in a neighborhood display mode that is particularly large in size and tends to deprive the visual attention of the observer, the brightness of a virtual object with a large size change rate accompanying movement is adjusted to the brightness of a virtual object with a small size change rate accompanying movement. Since the brightness is lower than the brightness of the object, it is possible to effectively prevent the viewer's visual attention from being excessively deprived.

In the embodiment shown in FIG. 7D, the second luminance characteristic R23 (R20) in the second object movement process is lower than the first luminance characteristic R13 (R10) only in a portion where the expression depth VL is shallow, and the expression depth VL is It is set gradually lower as the depth becomes shallower. According to this, in a neighborhood display mode that is particularly large in size and tends to deprive the visual attention of the observer, the brightness of a virtual object with a large size change rate accompanying movement is adjusted to the brightness of a virtual object with a small size change rate accompanying movement. Since the brightness is lower than the brightness of the object, it is possible to effectively prevent the viewer's visual attention from being excessively deprived.

Again, refer to FIG. The object moving process in step S162 in FIG. 5 is a process of moving the virtual object VO based on the progress of the vehicle 1, and gradually changing the size (visual angle) at a first rate of change G10 as the virtual object VO moves. 1 and the progress of the vehicle 1, while moving the virtual object VO, the size (visual angle) is gradually changed at a second rate of change G20 larger than the first rate of change G10 as the virtual object VO moves. and a second object movement process that changes the position of the display area 100 in the height direction, the eye position 4 in the height direction of the observer in the eye box 200, the pitching posture of the vehicle 1, and these at least Switching based on at least one of the estimable information, the second rate of change G20 is greater than the rate of change G30 of the size of the real object in the real world as the vehicle 1 travels, as shown in FIGS. 6C and 6D. is also big. In this embodiment, the size of the virtual object is changed while moving the virtual object based on the progress of the vehicle, so that the virtual object easily blends in with the real scene. In this embodiment, in addition to this, the virtual object is moved while changing its size at a rate of change greater than the rate of change of the size of the real object in the real world as the vehicle travels. This has the advantage that it is possible to emphasize the sense of depth of the virtual object even in a situation where the sense of depth of the virtual object is lowered.

In step S164 of FIG. 5, the processor 33, as shown in FIGS. 7A to 7D, adjusts the brightness of the virtual object VO, which has a large rate of change G in the second object moving process, to the rate of change G in the first object moving process. is set lower than the brightness of the virtual object VO with a small value. A virtual object with a large rate of change in size with movement emphasizes the sense of depth, but it is also assumed that the viewer's visual attention is excessively deprived. In this embodiment, the sense of depth of the virtual object is emphasized by increasing the rate of change in size accompanying movement. In this embodiment, in addition to this, since the brightness of a virtual object whose size change rate is large due to movement is set lower than the brightness of a virtual object whose size change rate is small due to movement, the observer's visual attention is excessive. can be suppressed.

In step S166 of FIG. 5, the processor 33, as shown in FIGS. 7C and 7D, sets the distant display mode (for example, FIGS. 4AA and 4BA) in the second object movement process and the far display mode in the movement of the virtual object. A near display mode (for example, FIGS. 4AC and 4BC) that is lower than the display mode and larger in size than the distant display mode is set. is set to be the same as the brightness of a virtual object with a small change rate, and in the neighborhood display mode, the brightness of a virtual object with a large change rate is set lower than the brightness of a virtual object with a small change rate. A virtual object with a large rate of change in size as it moves emphasizes the sense of depth, and is displayed large especially in the near display mode, and it is assumed that the visual attention of the observer is excessively deprived. In the present embodiment, in a neighborhood display mode that is particularly large in size and tends to deprive the viewer of visual attention, the brightness of a virtual object that has a large rate of change in size due to movement is adjusted to the brightness of a virtual object that has a small rate of change in size due to movement. Since the brightness is lower than the brightness of the object, it is possible to effectively prevent the viewer's visual attention from being excessively deprived.

In step S168 of FIG. 5, the processor 33 sets the maximum size Asm to the virtual object VO in the object movement process, and sets the size Asm of the virtual object VO in part of the movement of the virtual object VO, as shown in FIG. 6F. The virtual object VO is moved while maintaining the size As of the virtual object VO at the maximum size Asm so that the size As of the virtual object VO does not exceed the maximum size Asm. , the size As of the virtual object VO is gradually changed. In this embodiment, by setting an upper limit (maximum size) for the size accompanying movement, it is possible to prevent the virtual object from being displayed in an excessively large size and thereby excessively depriving the observer's visual attention. .

Also, in some embodiments of step S168 of FIG. 5, the processor 33, in the object movement process, performs 1) the far display mode and the movement of the virtual object from the far display mode, as shown in FIG. 7C or 7D. and a near display mode having a size larger than that of the distant display mode. In the neighborhood display mode, the brightness of a virtual object with a large change rate in the second object moving process is set to the brightness of a virtual object with a small change rate in the first object moving process. 2) set a maximum size for the virtual object such that the size of the virtual object is maximized for at least a portion of the movement of the virtual object displayed in the near view mode; Move the virtual object while maintaining the maximum size so that it does not exceed the size. In the present embodiment, in a neighborhood display mode that is particularly large in size and tends to deprive the viewer of visual attention, the brightness of a virtual object that has a large rate of change in size due to movement is adjusted to the brightness of a virtual object that has a small rate of change in size due to movement. Since the brightness is lower than the brightness of the object, it is possible to effectively prevent the viewer's visual attention from being excessively deprived. In addition to this, by setting an upper limit (maximum size) for the size associated with movement, it is possible to prevent the virtual object from being displayed in an excessively large size and thereby excessively depriving the observer's visual attention.

The operations of the processing processes described above may be implemented by executing one or more functional modules of an information processing device such as a general purpose processor or an application specific chip. These modules, combinations of these modules, and/or combinations with known hardware that can replace their functions are all within the scope of protection of the present invention.

The functional blocks of vehicle display system 10 are optionally implemented in hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is noted that the functional blocks illustrated in FIG. 3 may optionally be combined or separated into two or more sub-blocks to implement the principles of the described embodiments. will be understood by those skilled in the art. Accordingly, the description herein optionally supports any possible combination or division of the functional blocks described herein.

1: Vehicle 2: Front windshield 4: Eye (eye position)
10: Vehicle display system 20: Head-up display device (image display unit)
21: Display 21a: Display surface 22: Liquid crystal display panel 24: Light source unit 25: Relay optical system 26: First mirror 27: Second mirror 28: First actuator 29: Second actuator 30: Display control device 31: I /O interface 33 : Processor 35 : Object movement processing circuit 37 : Memory 40 : Display light 40p : Optical axis 41 : First image light 42 : Second image light 43 : Third image light 90 : Virtual image optical system 100 : Display area 101: upper end 102: lower end 110: first display area 120: second display area 200: eyebox 205: center 300: real object 310: road surface 320: pedestrian 330: branch road 401: vehicle ECU
403 : road information database 405 : vehicle position detection unit 407 : vehicle exterior sensor 409 : operation detection unit 411 : eye position detection unit 413 : vehicle posture detection unit 415 : mobile information terminal 417 : external communication device 502 : determination module 504 : object Movement processing module 512: Eye position detection module 514: Vehicle orientation detection module 516: Display area setting module 518: Overlapping area calculation module As: Size Asm: Maximum size G10: First rate of change G20: Second rate of change G30: Change rate G: Change rate M: Image PP: Superimposed area Q: Change characteristic Q10: First change characteristic Q20: Second change characteristic R: Luminance characteristic R10: First luminance characteristic R20: Second luminance characteristic V : Virtual image V10 : First virtual object V20 : Second virtual object VL : Expression depth VO : Virtual object

Claims (8)

In a display control device (30) that controls an image display unit (20) that displays a virtual image (V) of an image in a display area (100) that overlaps the foreground when viewed from an eyebox (200) in a vehicle,
one or more processors (33);
a memory (37);
one or more computer programs stored in said memory (37) and configured to be executed by said one or more processors (33);
Said processor (33)
While moving the virtual object (VO), gradually changing the size (As) at a predetermined rate of change (G) as the virtual object (VO) moves,
Based on at least one of the height direction position of the display area (100), the height direction eye position of the observer in the eye box (200), the pitching posture of the vehicle, and information capable of estimating these and executing an object movement process that changes the rate of change by
A display control device characterized by: The object moving process includes:
A first object whose size (As) is gradually changed at a first rate of change (G10) in accordance with the movement of the virtual object (VO) while moving the virtual object (VO) based on the progress of the vehicle. movement processing;
While moving the virtual object (VO) based on the progress of the vehicle, the size ( A second object movement process for gradually changing As), the position of the display area (100) in the height direction, the eye position of the observer in the eye box (200) in the height direction, Switching based on at least one of the pitching attitude of the vehicle and information that can estimate these,
The second rate of change (G20) is greater than the rate of change (G30) of the size (As) of real objects in the real world as the vehicle travels.
The display control device according to claim 1, characterized by: Said processor (33)
setting the luminance (R20) of the virtual object with a large rate of change lower than the luminance (R10) of the virtual object with a small rate of change;
3. The display control device according to claim 1, wherein: The processor (33), in the object movement process,
a distant display mode;
setting a near display mode that is lower than the distant display mode in the movement of the virtual object (VO) and has a larger size (As) than the distant display mode,
In the distant display mode, setting the luminance (R20) of the virtual object having a large rate of change and the luminance (R10) of the virtual object having a small rate of change to be the same,
In the neighborhood display mode, the brightness (R20) of the virtual object having a large rate of change is set lower than the brightness (R10) of the virtual object having a small rate of change.
3. The display control device according to claim 1, wherein: The processor (33), in the object movement process,
setting a maximum size (As) for the virtual object;
moving while maintaining the size (As) of the virtual object at the maximum size (Asm) so that the size (As) of the virtual object does not become larger than the maximum size (Asm) in part of the movement of the virtual object let
In the other part of the movement of the virtual object, moving the virtual object while gradually changing the size (As) of the virtual object within a range where the size (As) of the virtual object is smaller than the maximum size (Asm);
5. The display control device according to any one of claims 1 to 4, characterized in that: Said processor (33)
Based on at least one of the height direction position of the display area (100), the height direction eye position of the observer in the eye box (200), the pitching posture of the vehicle, and information capable of estimating these calculating the ratio (PP) of the road surface in the display area (100) when observed from the eye position or a predetermined position of the eye box (200),
If the ratio is determined to be small, increase the rate of change (G);
5. The display control device according to any one of claims 1 to 4, characterized in that: A display control device (30) according to any one of claims 1 to 6;
a display (21) for displaying an image on the display surface;
By projecting the display light of the image displayed by the display device (21) onto an external projection target part, a virtual image of the image ( and one or more relay optics (25) for displaying V). In a display control method for controlling an image display unit (20) that displays a virtual image (V) of an image in a display area (100) that overlaps the foreground when viewed from an eye box (200) in a vehicle,
While moving the virtual object (VO), gradually changing the size (As) at a predetermined rate of change as the virtual object (VO) moves;
Based on at least one of the height direction position of the display area (100), the height direction eye position of the observer in the eye box (200), the pitching posture of the vehicle, and information capable of estimating these changing the rate of change by
A display control method characterized by:

Images