JP2024017515A - Display control device, head-up display device and display control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress attitude variations of a vehicle from deteriorating a feeling of virtual reality of an image (a virtual object).

SOLUTION: An HUD device executes first position adjustment processing for dynamically changing a position of a virtual image in accordance with attitude variation information, in order to suppress a relative positional shift between the virtual image and a road surface caused by attitude variations of a vehicle; when a state where either V10 (V20) of first contents and second contents are arranged in a first area of a virtual image display area varies into a state where the either is arranged out of the first area, after the first position adjustment processing is executed, executes second position adjustment processing for suppressing an adjustment of a position of the virtual image with respect to the attitude variations of the vehicle more than the first position adjustment processing to the either V10 (V20) of the virtual image, and at the same time executes the second position adjustment processing to the other V20 (V10) of the virtual image as well.

SELECTED DRAWING: Figure 5

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present disclosure relates to a display control device, a head-up display device, and a head-up display device that are used in a moving object such as a vehicle and superimpose an image on the foreground of the moving object (actual view in the forward direction of the moving object as seen from a vehicle occupant). Related to display control methods, etc.

A Head Up Display (HUD) device displays an image (virtual object) superimposed on the scenery in front of the vehicle, creating an augmented reality (augmented reality) that adds and emphasizes information to the real scene or real objects existing in the real scene. By expressing AR (Augmented Reality) and accurately providing desired information while suppressing the line of sight movement of the user driving the vehicle, it can contribute to safe and comfortable vehicle operation.

The head-up display device described in Patent Document 1 displays an image (virtual object) at a display reference position, and suppresses a relative positional shift between the image (virtual object) and the real scene due to changes in the posture of the vehicle. The image is shifted according to changes in the vehicle's attitude. As a result, even if the vehicle's attitude changes, the relative positional relationship between the image and the actual scene is maintained (positional deviation between the image and the actual scene is suppressed), so it is possible to further harmonize the image with the actual scene. (Virtual reality can be enhanced).

The area of the actual scene where the images were originally visually recognized as overlapping before the attitude change of the vehicle is set as the superimposed actual scene area. In a head-up display device, if the virtual image display area deviates to the extent that it does not overlap the superimposed real view area due to a large attitude change of the vehicle, when the image position is adjusted to match the large attitude change, a part of the image that is being adjusted will be Or the entire image no longer fits within the virtual image display area (cannot be displayed). If part or all of the image is cut off due to the posture change (hereinafter, this phenomenon will also be referred to as "cut off"), the virtual reality of the virtual image will be impaired, and it is assumed that the viewer will feel uncomfortable.

In Patent Document 1, in order to prevent such an image from being cut off, when the posture change is large, adjustment of the image position is suppressed so that a part of the image (characteristic part) falls within the display area. A display device is disclosed.

International Publication 2020/208883

However, if you adjust the image position so that multiple contents (for example, image A and image B) are not cut off, for example, if you adjust the position in response to large vibrations that are not suppressed, image A will be cut off. The suppressed position adjustment is performed, and the unsuppressed position adjustment is performed on the image B that cannot be completely seen even if the unsuppressed position adjustment is performed. If content that undergoes different position adjustment processing for vibrations is mixed up in this way, it is expected that the viewer will feel uncomfortable.

In addition, because the amount of positional adjustment for vibration differs greatly for each image, not only virtual images with suppressed positional adjustment, but also virtual images with unsuppressed positional adjustment are in harmony with the real scene. It is expected that the impression will fade (the sense of virtual reality will deteriorate).

Furthermore, it is expected that the impression that the virtual image is in harmony with the real scene will fade (the sense of virtual reality will deteriorate) due to the timing at which the mode of position adjustment processing for vibration changes for each image. .

A summary of certain embodiments disclosed herein is provided below. It is to be understood that these aspects are presented solely to provide the reader with an overview of these particular embodiments and do not limit the scope of this disclosure. Indeed, the present disclosure may encompass combinations of the embodiments described below and various aspects not described below.

The summary of the present disclosure relates to reducing discomfort in viewing virtual images. More specifically, it also relates to suppressing a decrease in the virtual reality of an image (virtual object) due to changes in the posture of the vehicle.

Therefore, the display control device, head-up display device, display control method, etc. described in this specification employ the following means to solve the above problems. In this embodiment, in order to suppress the relative positional deviation between the virtual image and the road surface due to the attitude change of the vehicle, a first position adjustment process is executed to dynamically change the position of the virtual image in accordance with the attitude change information. When the first position adjustment process is executed, either the first content or the second content changes from being placed within the first area of the virtual image display area to being placed outside the first area, A second position adjustment process is executed for one of the virtual images to suppress adjustment of the position of the virtual image in response to changes in the vehicle attitude more than the first position adjustment process, and at the same time, a second position adjustment process is performed for the other virtual image. Its gist is to carry out the following.

Therefore, the display control device in the first embodiment described in this specification is mounted on a vehicle, has a display section that displays an image on a display surface, and directs the light of the image toward the projected section. A display control device that controls a head-up display device that causes a virtual image including a first content and a second content to be viewed in an overlapping manner in a virtual image display area that overlaps with a road surface in front of the vehicle,
The one or more processors are:
acquiring attitude change information indicating attitude change of the vehicle;
1) In order to suppress relative positional deviation between the virtual image and the road surface due to attitude change of the vehicle, a first position adjustment process is executed to dynamically change the position of the virtual image in accordance with the attitude change information. ,
2) When the first position adjustment process is executed, either the first content or the second content changes from being placed within the first area of the virtual image display area to being placed outside the first area. When the virtual image changes to a state in which the virtual image changes to a state where The second position adjustment process is also performed on the other virtual image. In this embodiment, if the change in vehicle attitude is small, the position of the virtual image is adjusted based on the first position adjustment amount that changes dynamically in accordance with the change in vehicle attitude, thereby distinguishing between the virtual image due to the change in vehicle attitude and the real scene. (first position adjustment process). Further, if the attitude change of the vehicle is large, a second position adjustment process is executed in which the amount of position adjustment is smaller than the first position adjustment process that changes dynamically in accordance with the attitude change. Therefore, even if the posture fluctuation becomes large, the amount of position adjustment is suppressed, so that it is possible to prevent the virtual image from being positioned outside the virtual image display area (second position adjustment process).

Furthermore, in the present embodiment, it is assumed that if the posture of the vehicle changes significantly and one of the contents (for example, the first content) undergoes the first position adjustment process, it will be cut off from the predetermined first area. In this case, the second image adjustment process in which the position adjustment for posture fluctuation is suppressed is performed not only for the one content that is expected to be cut off, but also for the other content (for example, the second content). is executed. According to this, it is assumed that there is an advantage that the amount of positional adjustment for vibration can be prevented from being greatly different from one image to another. Furthermore, an advantage is also envisaged in that the timing at which the mode of position adjustment processing for vibrations is switched is prevented from being different for each image.

Further, in the display control device according to the second embodiment that is subordinate to the first embodiment, the first content is arranged so as to be visually recognized above the second content,
The processor includes:
When the vehicle tilts backward and the second content, which is offset when the first position adjustment process is executed, is located below the first area, the position of the second content is changed to the first area. and/or the vehicle leans forward; If the first content, which is offset when the first position adjustment process is executed, is placed above the first area, the first content may be positioned at an upper peripheral edge within the first area. and, at the same time, execute the second position adjustment process of fixing the position of the second content at a predetermined position below the upper peripheral edge within the first area. In the present embodiment, it is possible to strongly suppress the deviation between the real scene and the virtual image caused by posture fluctuations (first position adjustment process) until one content reaches the periphery of a predetermined area. In the adjustment process, if one content is placed outside the periphery of a predetermined area, one content is fixedly displayed on the periphery of the predetermined area, and at the same time, the other content is also switched to fixed display.

First, a case where the vehicle tilts backward will be explained. When the vehicle tilts backward, the virtual image whose position is not adjusted will be perceived as being overlapped at a foreground position higher than the foreground position where the foreground was previously overlapped when viewed from the observer. The first position adjustment process of this embodiment offsets the virtual image downward with respect to the backward tilt of the vehicle. When the rearward tilt of the vehicle increases, the virtual image (second content) placed below reaches the lower peripheral edge of the first area (first position adjustment process). Even if the vehicle tilts further back, the virtual image (second content) placed below is fixed to the lower peripheral edge of the first area (second position adjustment process). When the backward tilt of the vehicle increases and the mode of the virtual image (second content) placed below is switched from the first position adjustment process to the second position adjustment process, at the same time, the virtual image that has not reached the lower peripheral edge part (First content) is also switched to a second position adjustment process that suppresses the position adjustment of the virtual image in response to a change in the attitude of the vehicle, rather than the first position adjustment process.

Next, a case where the vehicle leans forward will be explained. When the vehicle leans forward, the virtual image whose position is not adjusted will be perceived as being overlapped at a lower foreground position than the foreground position where the foreground was previously overlapped when viewed from the observer. The first position adjustment process of this embodiment offsets the virtual image upward with respect to the forward tilt of the vehicle. When the forward tilt of the vehicle increases, the virtual image (first content) placed above reaches the upper peripheral edge of the first area (first position adjustment process). Even if the forward tilt of the vehicle becomes larger than this, the virtual image (first content) placed above is fixed to the upper peripheral edge of the first area (second position adjustment process). When the forward tilt of the vehicle increases and the mode of the virtual image (first content) placed above is switched from the second position adjustment process to the second position adjustment process, at the same time, the virtual image that has not reached the upper peripheral portion (Second content) mode is also switched to a second position adjustment process that suppresses the position adjustment of the virtual image in response to a change in the attitude of the vehicle, rather than the first position adjustment process. According to this, when the vehicle tilts backward, the shift between the real scene and the virtual image due to attitude change is strongly suppressed until the second content reaches the lower peripheral edge of the predetermined area (the first position adjustment processing), the advantage is that it is possible to prevent the virtual image from being cut off. Further, it is assumed that there is an advantage that the amount of position adjustment for vibrations can be prevented from greatly differing from image to image. Furthermore, an advantage is also envisaged in that the timing at which the mode of position adjustment processing for vibrations is switched is prevented from being different for each image.

In a display control device in a third embodiment that may be subordinate to the first or second embodiment, the processor:
1) performing the first position adjustment process on the first content and the second content in response to a posture change in a first direction in which the vehicle tilts forward or backward from a reference posture;
When the first position adjustment process is executed, the one of the virtual images to be offset changes from being placed within the first area of the virtual image display area to being placed outside the first area. , performing the second position adjustment process on the first content and the second content;
2) performing the first position adjustment process on the one of the virtual images in response to a change in the attitude of the vehicle from the reference attitude in a second direction opposite to the first direction;
When the first position adjustment process is executed, the one of the virtual images to be offset changes from being placed within the first area of the virtual image display area to being placed outside the first area. , executing the second position adjustment process on the one of the virtual images. According to this, the first position adjustment process is performed on the first content and the second content in response to a posture change in the first direction (for example, forward leaning), and the first position adjustment process causes one of the contents ( For example, if the first content) is about to be cut off, the second position adjustment process is executed for the first content and the second content, but conversely, depending on the posture change in the second direction (for example, backward tilt), Then, if the first position adjustment process is executed on the first content and the second content, and one of the contents (for example, the second content) is about to be cut off due to the first position adjustment process, the first position adjustment process is performed on the first content and the second content. However, the first position adjustment process continues for one content (for example, the first content). Then, in response to a larger posture change in the second direction (for example, backward leaning), if one content (for example, the first content) is about to be cut off by the first position adjustment process, the other content (for example, The second position adjustment process is executed not only for the second content (second content) but also for one content (for example, the first content). According to the present embodiment, for example, either the first content or the second content may be placed outside the predetermined first area even in the first position adjustment process for a relatively small posture change in the second direction. Even in the case of a relatively small positional change in the second direction, since the first position adjustment process is not performed in accordance with the posture change in the second direction, even if the posture change in the second direction is relatively small, the second content is adjusted to both the first and second contents. Another possible advantage is that it is possible to prevent the position adjustment process from being executed.

In a display control device in a modification of the third embodiment that may be subordinate to the first or second embodiment, the processor:
1) performing the first position adjustment process on the first content and the second content in response to a posture change in a first direction in which the vehicle tilts forward or backward from a reference posture;
When the first position adjustment process is executed, the one of the virtual images to be offset changes from being placed within the first area of the virtual image display area to being placed outside the first area. , performing the second position adjustment process on the first content and the second content;
2) executing the first position adjustment process on the one of the virtual images in accordance with a change in the attitude of the vehicle from the reference attitude in a second direction opposite to the first direction; performing the second position adjustment process on the other;
When the first position adjustment process is executed, the one of the virtual images to be offset changes from being placed within the first area of the virtual image display area to being placed outside the first area. , executing the second position adjustment process on the one of the virtual images. According to this, the first position adjustment process is performed on the first content and the second content in response to a posture change in the first direction (for example, forward leaning), and the first position adjustment process adjusts one of the contents. If the content is about to be completely cut off, the second position adjustment process is executed for the first content and the second content, but conversely, depending on the posture change in the second direction (for example, backward tilting), one content (for example, , the first content) (but not the other content (e.g., the second content)), and if one of the contents (e.g., the first content) is cut off due to the first position adjustment processing, If this happens, the second position adjustment process is performed on one of the contents (for example, the first content). According to the present embodiment, for example, either the first content or the second content may be placed outside the predetermined first area even in the first position adjustment process for a relatively small posture change in the second direction. Even in the case of a relatively small positional change in the second direction, since the first position adjustment process is not performed in accordance with the posture change in the second direction, even if the posture change in the second direction is relatively small, the second content is adjusted to both the first and second contents. Another possible advantage is that it is possible to prevent the position adjustment process from being executed.

In the display control device in the fourth embodiment that may be subordinate to the first to third embodiments, the processor:
When the first position adjustment process is executed in response to a change in attitude of the vehicle in a predetermined direction such as tilting forward or tilting backward, the first content is arranged in the first area of the virtual image display area. When changing to a state where the virtual image is placed outside the first area, a second position adjustment for the first content that suppresses adjustment of the position of the virtual image in response to a change in the attitude of the vehicle more than the first position adjustment process. and simultaneously performing the second position adjustment process on the second content,
When the first position adjustment process is performed so that the arrangement of the first content and/or the second content is adjusted in accordance with the posture change in the predetermined direction, the second content is displayed in the virtual image more than the first content. When the arrangement is changed from being placed within the first area of the area to being likely to be outside the first area,
When the first position adjustment process is executed in accordance with the attitude change of the vehicle in the predetermined direction, the second content changes from being placed in the first area of the virtual image display area to being placed in the first area. When changing to a state where the virtual image is placed outside, performing a second position adjustment process on the second content that suppresses adjustment of the position of the virtual image in response to a change in the attitude of the vehicle more than the first position adjustment process, At the same time, the second position adjustment process is also performed on the first content. In the present embodiment, when the first content is placed in a position where it is easy to see off through the first position adjustment process according to the posture change in a predetermined direction, when the first content is likely to be off, the first content and the second content are 2 position adjustment processing is executed, the arrangement of the first content and/or the second content is changed, and the second content is cut off from the first content by the first position adjustment processing according to the posture change in a predetermined direction. When the arrangement is such that the second content is likely to be cut off, the second position adjustment process is performed on the first content and the second content. That is, an advantage is also envisaged in that even if the arrangement of the content changes, the virtual image is prevented from being cut off.

In the display control device according to the fifth embodiment, which may be subordinate to the first to fourth embodiments, the processor:
When the first position adjustment process is executed in response to a change in attitude of the vehicle in a predetermined direction such as tilting forward or tilting backward, the first content is arranged in the first area of the virtual image display area. When changing to a state where the virtual image is placed outside the first area, a second position adjustment for the first content that suppresses adjustment of the position of the virtual image in response to a change in the attitude of the vehicle more than the first position adjustment process. and simultaneously performing the second position adjustment process on the second content,
When the first position adjustment process is performed so that the arrangement of the first content and/or the second content is adjusted in accordance with the posture change in the predetermined direction, the second content is displayed in the virtual image more than the first content. When the arrangement is changed from being placed within the first area of the area to being likely to be outside the first area,
When the first position adjustment process is executed in accordance with the attitude change of the vehicle in the predetermined direction, the second content changes from being placed in the first area of the virtual image display area to being placed in the first area. When changing to a state where the virtual image is placed outside, performing a second position adjustment process on the second content that suppresses adjustment of the position of the virtual image in response to a change in the attitude of the vehicle more than the first position adjustment process, At the same time, the second position adjustment process is also performed on the first content. In the present embodiment, when the first content is placed in a position where it is easy to see off through the first position adjustment process according to the posture change in a predetermined direction, when the first content is likely to be off, the first content and the second content are 2 position adjustment processing is executed, the size of the first content and/or the second content is changed, and the second content is cut off from the first content by the first position adjustment processing according to the posture change in a predetermined direction. When the arrangement is such that the second content is likely to be cut off, the second position adjustment process is performed on the first content and the second content. That is, an advantage is also envisaged in that even if the arrangement of the content changes, the virtual image is prevented from being cut off.

In the display control device according to the sixth embodiment that may be subordinate to the first to fifth embodiments, the processor may, in the first position adjustment process,
The amount of position adjustment of the first content with respect to a change in the attitude of the vehicle is made larger than the amount of position adjustment of the second content with respect to a change in the attitude of the vehicle. According to the present embodiment, for a plurality of contents that have different position adjustment amounts in response to attitude changes, while strongly reducing the positional deviation between the virtual image and the real scene due to the attitude change of the virtual vehicle, when one content is likely to be cut off, , since it is possible to simultaneously switch from the first position adjustment process to the second image adjustment process in which the position adjustment for posture changes is suppressed more than the first position adjustment process for both the first and second contents, the posture change for each image can be changed. It is also assumed that there is an advantage of suppressing the difference in the position adjustment amount between the two positions.

In the display control device in the seventh embodiment that is subordinate to the sixth embodiment,
The target position of the first content is arranged farther than the target position of the second content. In this embodiment, in a model coordinate system based on a predetermined position of the vehicle (for example, the center of the eye box of a HUD device), the target position of the first content that is virtually arranged is set from the target position of the second content. make it far away This makes it possible to express differences in perspective for each content, and is also expected to have the advantage of suppressing deterioration in the virtual reality of the image (virtual object) due to changes in the posture of the vehicle.

A head-up display device according to a ninth embodiment that can be subordinated to the first to eighth embodiments includes a display unit that is mounted on a vehicle and displays an image on a display surface;
a relay optical system that directs display light from the display section to a projected section;
one or more processors;
memory and
one or more computer programs stored in the memory and configured to be executed by the one or more processors, superimposing a virtual image in a virtual image display area that overlaps with a road surface in front of the vehicle; A head-up display device that allows the user to visually check the
The processor includes:
acquiring attitude change information indicating attitude change of the vehicle;
In order to suppress a relative positional shift between the virtual image and the road surface due to a change in the attitude of the vehicle, a first position adjustment amount that dynamically changes according to the attitude change information is set;
1) when the attitude of the vehicle changes from a reference state to a first direction of forward or backward tilting, performing a first image adjustment process of adjusting the position of the virtual image based on the first position adjustment amount;
2) When the attitude of the vehicle changes from the reference state to a second direction opposite to the first direction, such as forward or backward leaning, the position of the virtual image with respect to the change in the attitude of the vehicle is determined by the first image adjustment process. A second image adjustment process is executed to suppress the adjustment of .

A display control method according to a tenth embodiment, which may be subordinate to the first to eighth embodiments, includes a display unit that is mounted on a vehicle and displays an image on a display surface, and directs light of the image toward a projected unit. A display control method for controlling a head-up display device that causes a virtual image to be superimposed and viewed in a virtual image display area that overlaps with a road surface in front of the vehicle,
acquiring attitude change information indicating attitude change of the vehicle;
setting a first position adjustment amount that dynamically changes in accordance with the attitude change information in order to suppress relative positional deviation between the virtual image and the road surface due to attitude change of the vehicle;
1) a first image adjustment process of adjusting the position of the virtual image based on the first position adjustment amount when the attitude of the vehicle changes from a reference state to a first direction of tilting forward or tilting backward;
2) When the attitude of the vehicle changes from the reference state to a second direction opposite to the first direction, such as forward or backward leaning, the position of the virtual image with respect to the change in the attitude of the vehicle is determined by the first image adjustment process. and a second image adjustment process for suppressing the adjustment.

FIG. 1 is a diagram showing an example in which the vehicle display system of this embodiment is applied to a vehicle. FIG. 2 is a diagram showing the configuration of a head-up display device. FIG. 3 is a block diagram of a vehicle display system of some embodiments. FIG. 4 is a diagram illustrating a model space in which content and virtual viewpoints are arranged, according to some embodiments. FIG. 5 is a diagram illustrating an example of a foreground that is visually recognized by an observer and an image (virtual image) that is displayed superimposed on the foreground while the vehicle is running. FIG. 6 is a diagram showing the virtual image before the position adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and virtual image that are visible when the viewer faces forward. shows. FIG. 7A is a diagram showing the virtual image after the first image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and the foreground that are visible when the observer faces forward. A virtual image is shown. 7B is a diagram showing the virtual image after the second image adjustment process, the left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and the virtual image that are visible when the viewer faces forward. and FIG. 8A is a diagram showing the virtual image after the first image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visually recognized when the viewer faces forward. A virtual image is shown. FIG. 8B is a diagram showing the virtual image after the second image adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visually recognized when the observer looks forward. A virtual image is shown. FIG. 9 is a diagram illustrating image adjustment processing for posture variation, and shows an example in which the first image adjustment processing and the second image adjustment processing are switched depending on the magnitude of posture variation (or position adjustment amount). FIG. 10 is a diagram illustrating image adjustment processing in some embodiments, in which the position adjustment amount of the first content and the position of the second content change depending on the magnitude of the posture change (or the position adjustment amount). Indicates the amount of adjustment. FIG. 11 is a diagram showing the position of the virtual image of the first content and the position of the virtual image of the second content, which are adjusted according to changes in the posture of the vehicle. FIG. 12 is a diagram illustrating image adjustment processing in some embodiments, in which the position adjustment amount of the first content and the position of the second content change depending on the magnitude of posture fluctuation (or position adjustment amount). Indicates the amount of adjustment. FIG. 13 is a diagram showing the position of the virtual image of the first content and the position of the virtual image of the second content, which are adjusted according to changes in the posture of the vehicle. FIG. 14 is a diagram showing the position of the virtual image of the first content and the position of the virtual image of the second content, which are adjusted according to changes in the posture of the vehicle. FIG. 15 is a diagram showing the virtual image before posture change. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and virtual image that are visible when the observer faces forward. show. FIG. 16A is a diagram showing the virtual image after size adjustment processing when the virtual image display area is shifted downward from the real scene due to forward tilting. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the relationship between the virtual image and the virtual plane. , shows the foreground and virtual image that are visible when the viewer faces forward. FIG. 16B is a diagram showing the virtual image after size adjustment processing when the virtual image display area is shifted upward from the real scene due to backward tilting. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the relationship between the virtual image and the virtual plane. , shows the foreground and virtual image that are visible when the viewer faces forward. FIG. 17 is a diagram illustrating the first image adjustment process in which the position is adjusted without performing the size adjustment process when the vehicle 1 tilts backward, and the left figure shows the first image before the attitude change. The virtual image and the foreground before the adjustment process are shown, and the right figure shows the virtual image and the foreground after the attitude change and after the first image adjustment process. FIG. 18 is a diagram illustrating the second image adjustment process in which the size adjustment process is performed without position adjustment when the vehicle 1 tilts backward. The left diagram shows the image before the attitude change and before the size adjustment. The figure on the right shows the virtual image and foreground after changing the pose and adjusting the size. FIG. 19 is a diagram illustrating the second image adjustment process in which the position adjustment and size adjustment process were performed when the vehicle 1 tilted backward, and the left diagram shows the second image adjustment process before the attitude change. The right figure shows the virtual image and foreground after the posture change and after the second image adjustment process. FIG. 20 is a diagram showing a flow of display control in some embodiments. FIG. 21 is a diagram for explaining the limit of the position adjustment amount. FIG. 22 is a diagram for explaining the limit of the position adjustment amount.

1-22, a description of the configuration and operation of an exemplary vehicle display system is provided below. Note that the present invention is not limited to the following embodiments (including the contents of the drawings). Of course, changes (including deletion of components) can be made to the embodiments described below. Further, in the following description, description of known technical matters will be omitted as appropriate in order to facilitate understanding of the present invention.

Please refer to FIG. FIG. 1 is a diagram showing an example of the configuration of a virtual image display system for a vehicle. In FIG. 1, the left and right direction (in other words, the width direction of the vehicle 1) of the vehicle (an example of a moving object) 1 is the X axis (the positive direction of the X axis is the left direction when the vehicle 1 faces forward). ), and the vertical direction (in other words, the height direction of the vehicle 1) along a line segment perpendicular to the left-right direction and perpendicular to the ground or a surface corresponding to the ground (road surface 6 in this case) is the Y-axis (Y-axis The positive direction is the upward direction.), and the front-rear direction along the line segment orthogonal to each of the left-right direction and the up-down direction is the Z-axis (the positive direction of the Z-axis is the straight direction of the vehicle 1). This point also applies to other drawings.

As shown in the figure, a vehicle display system 10 provided in a vehicle (moving body) 1 displays the positions and line-of-sight directions of a left eye 700L and a right eye 700R of an observer (typically a driver seated in the driver's seat of the vehicle 1). An external sensor 411 consisting of an eye position detection unit (line of sight detection unit) 409 for detecting pupils (or faces), a camera (for example, a stereo camera) that captures an image of the front of the vehicle 1 (in a broad sense, the surroundings), and the vehicle; 1, a head-up display device (hereinafter also referred to as a HUD device) 20, and a display control device 30 that controls the HUD device 20. Note that the eye position detection unit (line of sight detection unit) 409 and the vehicle external sensor 411 may be omitted.

FIG. 2 is a diagram showing one aspect of the configuration of the head-up display device 20. The HUD device 20 is installed, for example, in a dashboard (reference numeral 5 in FIG. 1). This HUD device 20 houses an image display device (display section) 40, a relay optical system 80, and the image display device 40 and the relay optical system 80, and directs display light K from the image display device 40 from the inside to the outside. The casing 22 has a light exit window 21 through which light can be emitted.

The image display device (display unit) 40 is here a parallax type 3D display device. This stereoscopic display device (parallax type 3D display device) 40 includes a display device 50 which is an autostereoscopic display device using a multi-view image display method that can control depth expression by visually recognizing a left viewpoint image and a right viewpoint image. , and a light source unit 60 that functions as a backlight. Note that the image display device (display unit) 40 is not limited to a stereoscopic image display device that displays 3D images, but may also display 2D images.

The display device 50 has a display surface 50a that optically modulates the illumination light from the light source unit 60 to generate an image M on the display surface 50a, and includes, for example, a lenticular lens or a parallax barrier. The light emitted from 50a is divided into left-eye display lights (represented by reference numeral K10 in FIG. 1) such as left-eye light rays K11, K12, and K13, and right-eye display light (reference numeral K10 in FIG. 1) such as light rays K21, K22, and K23 for the right eye. 1 K20) and an optical layer (an example of a light beam separation unit) 52. Optical layer 52 includes optical filters such as lenticular lenses, parallax barriers, lens arrays, and microlens arrays. In the embodiment, the optical layer 52 is not limited to the optical filter described above, but includes all types of optical layers disposed on the front or rear surface of the display surface 50a. However, this is an example and is not limited.

Furthermore, the image display device 40 has the light source unit 60 configured with a directional backlight unit (an example of a light beam separating section) instead of or in addition to the optical layer (an example of a light beam separating section). Emit left-eye display light such as light rays K11, K12, and K13 for the left eye (represented by reference numeral K10 in FIG. 1), and right-eye display light such as light rays K21, K22, and K23 for the right eye (represented by reference numeral K20 in FIG. 1). You may let them. Specifically, for example, when the directional backlight unit irradiates illumination light toward the left eye 700L, the display control device 30, which will be described later, displays a left viewpoint image on the display surface 50a, so that the left eye light ray K11 , K12, K13, etc., are directed toward the left eye 700L of the observer, and when the directional backlight unit irradiates illumination light toward the right eye 700R, a right viewpoint image is displayed on the display surface 50a. As a result, the right-eye display light K20, such as the right-eye light beams K21, K22, and K23, is directed toward the left eye 700L of the viewer. However, this is an example and is not limited.

The display control device 30, which will be described later, performs, for example, image rendering processing (graphic processing), display drive processing, etc., to provide the left eye display light K10, which is the source of the left viewpoint image V1, to the left eye 700L of the observer, and By directing the source right-eye display light K20 toward the right eye 700R and adjusting the left viewpoint image V1 and the right viewpoint image V2, the aspect of the content FU displayed by the HUD device 20 (perceived by the observer) is controlled. can do. Note that the display control device 30, which will be described later, controls the display (display device 50) so as to reproduce a light field that (approximately) reproduces the light rays output in various directions from points existing in a certain space. You can.

The relay optical system 80 includes curved mirrors (concave mirrors, etc.) 81 and 82 that reflect light from the image display device 40 and project image display lights K10 and K20 onto the windshield (projection target section) 2. However, it may further include other optical members (which may include a refractive optical member such as a lens, a diffractive optical member such as a hologram, a reflective optical member, or a combination thereof).

In FIG. 1, the image display device 40 of the HUD device 20 displays images with parallax (parallax images) for each of the left and right eyes. As shown in FIG. 1, each parallax image is displayed as V00 focused on the virtual image display area (virtual image plane) VS. The focus of each eye of the observer (person) is adjusted to match the position of the virtual image display area VS. Note that the position of the virtual image display area VS is referred to as an "adjustment position (or imaging position)" and may also be a predetermined reference position (for example, the center 205 of the eyebox 200 of the HUD device 20, the observer's viewpoint position, or , a specific position of the vehicle 1, etc.) to the virtual image display area VS is referred to as an adjustment distance (imaging distance).

Further, the virtual image display area VS is a virtual (apparent) surface set in the real space in front of the occupant (a viewer such as a driver), corresponding to the display surface 50a of the display 50. Further, the virtual image display area VS includes, for example, an elevational surface VS1 perpendicular to the road surface 6, an inclined surface VS2 inclined with respect to the road surface 6, a road surface superimposed surface VS3 superimposed on the road surface 6, and a side closer to the occupant (viewer). There are surfaces (not shown) that are elevations (including pseudo-elevations) and whose far side is an inclined surface. In display using surfaces other than the vertical surface VS1, the display distance of the virtual image varies depending on the display position on the virtual image display area, and therefore depth can be expressed.

FIG. 3 is a block diagram of a virtual image display system for a vehicle, according to some embodiments. Display control device 30 includes one or more I/O interfaces 31, one or more processors 33, one or more image processing circuits 35, and one or more memories 37. FIG. 3 is only one embodiment, and the illustrated components may be combined into fewer components or there may be additional components. For example, image processing circuitry 35 (eg, a graphics processing unit) may be included in one or more processors 33.

As shown, processor 33 and image processing circuitry 35 are operably coupled to memory 37 . More specifically, the processor 33 and the image processing circuit 35 execute programs stored in the memory 37 to generate and/or transmit image data, for example, in the vehicle display system 10 (image display system 35). control of the device 40). Processor 33 and/or image processing circuit 35 may include at least one general purpose microprocessor (e.g., central processing unit (CPU)), at least one application specific integrated circuit (ASIC), and at least one field programmable gate array (FPGA). , or any combination thereof. Memory 37 includes any type of magnetic media such as hard disks, any type of optical media such as CDs and DVDs, any type of semiconductor memory such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

As shown, processor 33 is operably coupled to I/O interface 31 . The I/O interface 31 communicates with, for example, a vehicle ECU 401 (described later) and/or other electronic devices (numerals 403 to 419 described later) provided in the vehicle according to the CAN (Controller Area Network) standard. communication). Note that the communication standard adopted by the I/O interface 31 is not limited to CAN, and includes, for example, CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), and MOST (Media Oriented Systems Transport). Wired communication interfaces such as: MOST (registered trademark), UART, or USB, or local communication interfaces such as personal area networks (PAN), e.g. Bluetooth networks, 802.11x Wi-Fi networks, etc. It includes an in-vehicle communication (internal communication) interface that is a short-range wireless communication interface within several tens of meters such as an area network (LAN). Further, the I/O interface 31 is a wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e base (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, It may also include an external communication interface such as a wide area communication network (for example, the Internet communication network) according to cellular communication standards such as 5G.

As shown in the figure, the processor 33 is interoperably connected to the I/O interface 31, so that it can communicate information with various other electronic devices connected to the vehicle display system 10 (I/O interface 31). It becomes possible to give and receive. The I/O interface 31 includes, for example, a vehicle ECU 401, a road information database 403, an own vehicle position detection section 405, an operation detection section 407, an eye position detection section 409, an external sensor 411, a brightness detection section 413, and an attitude detection section 415. , a mobile information terminal 417, an external communication device 419, and the like are operably connected. Note that the I/O interface 31 may include a function to process (convert, calculate, analyze) information received from other electronic devices connected to the vehicle display system 10.

Image display device 40 is operably coupled to processor 33 and image processing circuitry 35 . Accordingly, in some embodiments, the image displayed by display surface 50a may be based on image data received from processor 33 and/or image processing circuitry 35. The processor 33 and the image processing circuit 35 control (adjust) the image displayed on the display surface 50a based on information obtained from the I/O interface 31.

The software components stored in memory 37 include a drawing module 510 and an image adjustment module 520 (position adjustment module 522, depression angle adjustment module 524, size adjustment module 526).

The drawing module 510 forms an image M based on information (navigation information, vehicle information, etc.) acquired by the display control device 30, and temporarily stores the formed image M in a buffer (not shown). The image M displayed on the display 50 is visually recognized by the observer 700 as a virtual image V10 (virtual image V20). At this time, the virtual image V10 represents the first content FU1, and the virtual image V20 represents the second content FU2.

FIG. 4 is a diagram illustrating a model space in which content FU and virtual viewpoints VP are arranged. In FIG. 4, the model coordinate system of the virtual viewpoint VP has the depth direction as the Z1-axis direction (corresponding to the longitudinal direction Z of the vehicle 1), and the left-right direction as the X1-axis direction (corresponding to the width direction X of the vehicle 1). It is assumed that the vertical direction is the Y1 axis direction (corresponding to the vertical direction Y of the vehicle 1). The drawing module 510 calculates each vertex data of the content FU to be drawn in each drawing frame. In this case, a model space for each content FU is constructed. Then, data for each vertex to be drawn is calculated on the "model coordinate system (local coordinate system)" for each virtual object. The drawing module 510 converts the content FU drawn in the model coordinate system into a two-dimensional image by projecting it onto a predetermined projection plane (virtual image display area VS to be described later) based on the virtual viewpoint VP, and converts the content FU into a two-dimensional image. Let be image M. Note that the drawing module 510 may arrange each content FU arranged in the "model coordinate system (local coordinate system)" in the space of the "world coordinate system." That is, the vertex data of each content FU to be drawn calculated on the "model coordinate system" may be placed on the "world coordinate system." Note that some or all of the content FU does not need to be placed on the "world coordinate system."

The observer 700 visually recognizes the virtual image V10 formed (imaged) in the virtual image display area VS via the projection target section 2, and perceives that the content FU is located at the predetermined target position MP. For example, when the content FU1 is an arrow that guides a course, the arrow of the virtual image V11 is arranged so that the content FU1 is viewed from the virtual viewpoint VP as if it were placed at a predetermined target position MP1 in the real scene. It is displayed in the virtual image display area VS. That is, when an image (virtual image V10 here) obtained by projectively transforming the content FU into the virtual image display area VS is displayed based on the virtual viewpoint VP, the image is projected from the same position as the virtual viewpoint VP (for example, the center 205 of the eyebox 200). When viewed by the observer 700, as shown in FIG. 5, the content FU can be perceived as being placed at a predetermined target position MP as seen from the virtual viewpoint VP.

Normally, the virtual plane 100 on which the target position MP1 on which the content FU1 is arranged (set) is arranged is made to match the height of the surface of the foreground (road surface 6) (that is, the set height is set to 0 m). ). However, this is an example and is not limited. In another example, the target position MP1 (virtual plane 100) may be set at a position higher than the height of the foreground (road surface 6) surface (i.e., the set height may be set to 0.5 m or 1 m). good). In other examples, the target position MP1 (virtual plane 100) may be set at a position lower than the height of the surface of the foreground (road surface 6) (that is, the set height may be set to -1 m or -2 m). ). The depression angle β is the angle (looking down angle) between the horizontal direction (Z1-X1 plane) and the content FU1 (target position MP1) when viewed from a predetermined virtual viewpoint VP.

Further, a plurality of content FUs may be arranged in the model space. For example, the second content FU2 indicating the speed of the vehicle 1 may be placed at the target position MP2 closer to the virtual viewpoint VP than the first content FU1. With this arrangement, parallax can be created between the first content FU1 and the second content FU2 by movement of the virtual viewpoint VP (here, movement according to the attitude change of the vehicle 1). The positions of the virtual image V10 (first content FU1) and virtual image V20 (second content FU2) displayed in the virtual image display area VS are adjusted as the virtual viewpoint VP moves (here, movement according to the attitude change of the vehicle 1). However, the amount of position adjustment of the virtual image V11 (first content FU1) is set to be larger than the amount of position adjustment of the virtual image V20 (second content FU2).

The image adjustment module 520 (position adjustment module 522, depression angle adjustment module 524, size adjustment module 526) in FIG. A process for adjusting the angle of depression (position adjustment process), a process for adjusting the depression angle (depression angle adjustment process), and a process for adjusting the size (size adjustment process) are executed.

FIG. 6 is a diagram showing the virtual image before the position adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and virtual image that are visible when the viewer faces forward. shows. In FIG. 6, it is assumed that the vehicle attitude AT10 is AT11. The content FU is arranged so that the content FU appears to overlap the target position MP1 (first area 110) of the virtual plane 100 by the viewer. The content FU has a predetermined size and is arranged so as to overlap the first area 110 of the virtual plane 100. In FIG. 6, the size of the content FU is the first length L10 in the depth direction of the first area 110. The arbitrary virtual plane 100 is a virtual plane on which the content FU is arranged, and is set, for example, parallel to the front, rear, left, and right directions of the vehicle 1 (it may generally coincide with the road surface 6). The drawing module 510 displays an image M, which is the source of the virtual image V11, on the display device 50 so as to display an image (here, the virtual image V11) obtained by projectively transforming the content FU into the virtual image display area VS1 based on the virtual viewpoint VP1. display. Here, the distance along the virtual plane 100 from the virtual viewpoint VP1 to the first region 110 is set as D0, and the distance (height) from the virtual plane 100 to the virtual viewpoint VP1 is set as h0.

FIG. 7A is a diagram showing the virtual image after the first position adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visually recognized when the viewer faces forward. A virtual image is shown. In FIG. 7A, it is assumed that vehicle attitude AT10 is AT12 in which vehicle 1 is tilted forward by pitching angle α12, which is relatively smaller than vehicle attitude AT11 before the first position adjustment process. Here, the term "forward tilt" refers to the fact that the front of the vehicle 1 is lowered (in other words, the rear of the vehicle 1 is raised) based on the vehicle attitude AT11 shown in FIG. 6. When the vehicle 1 leans forward from the state shown in FIG. 6 to the state shown in FIG. ) by B12. When the first position adjustment process is not performed, the virtual image V11 shown in FIG. 6 is shifted downward to position G2 in the right diagram of FIG. The image is shifted by the image shift amount B12. That is, in the content FU viewed from the observer's viewpoint, the virtual image V11 is shifted from the desired target position MP1. By executing the first position adjustment process, the processor 33 in some embodiments moves the virtual image position G2 upward (in the Y-axis positive direction) so as to suppress (cancel) the image shift amount B12 due to posture fluctuation. The first position adjustment amount C12 (C10) is adjusted (the virtual image V12 whose position has been adjusted is displayed). Preferably, the first position adjustment amount C12 (C10) is made equal to the image shift amount B12 (B10) due to the attitude change (C10=B10), so that the virtual image V10 after the attitude change is adjusted to the first area 110 (target position MP1). According to this, the image shift amount B10 due to the posture change is canceled out by the first position adjustment amount C10, and is not recognized by the viewer. However, the first position adjustment amount C10 only needs to be able to reduce the image shift amount B10 due to posture variation, and may be smaller than the image shift amount B10. According to this, it is possible to suppress a positional shift of the virtual image due to a change in the vehicle attitude.

FIG. 7B is a diagram showing the virtual image after the second position adjustment process. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground that is visible when the observer faces forward. A virtual image is shown. In FIG. 7B, it is assumed that the vehicle attitude AT10 is an AT13 in which the vehicle 1 is tilted forward by a relatively larger pitching angle α13 (>α12) than the vehicle attitude AT11 before the second position adjustment process. When the vehicle 1 leans forward from the state shown in FIG. 6 to the state shown in FIG. ) by B13. If the position adjustment is not performed, the virtual image V11 shown in FIG. 6 will be shifted by the image shift amount B13 to the position G3 in the right diagram of FIG. 7B due to the image shift amount B13 due to the attitude change of the virtual image display area VS. . In order to offset the downward image shift amount B13 due to this attitude change, it is sufficient to move the virtual image V10 upward by the image shift amount B13, but if the virtual image V10 is moved upward by the image shift amount B13, It will go outside the virtual image display area VS13.

In some embodiments, when the attitude change of the vehicle 1 is large (an example of a predetermined condition described later), the processor 33 executes the second position adjustment process to obtain a downward image of the virtual image display area VS due to the attitude change. With respect to the shift amount B13, the virtual image position G3 is adjusted upward (in the Y-axis positive direction) by a second position adjustment amount C23 (C20) smaller than the first position adjustment amount C13 (C10) in the first position adjustment process ( Display the virtual image V13 whose position has been adjusted). By making the second position adjustment amount C23 (C20) smaller than the image shift amount B13 (B10) due to attitude change (C20<B10), the second position adjustment amount C23 (C20) is set to be smaller than the image shift amount B13 (B10) due to posture fluctuation, so that the first area 110 (target position MP1) is smaller than the first position adjustment process. The virtual image V13 can be maintained within the virtual image display area VS13, although the virtual image V13 is largely shifted to the second area 120 (a position different from the target position MP1) which is closer to the target position MP1. According to this, the image shift amount B13 (B10) due to the posture change is not canceled out by the second position adjustment amount C23 (C20), and therefore is recognized by the observer.

The content FU is arranged with a predetermined angular relationship with respect to the road surface 6. Specifically, for example, the content FU is arranged so as to be visually recognized parallel to the road surface 6. However, when the pitching angle α of the vehicle 1 changes, the angular relationship between the content FU and the road surface 6 changes. Specifically, when the virtual image V10 is displayed parallel to the road surface 6, the parallel relationship between the virtual image V10 and the road surface 6 is shifted by the pitching angle α due to the pitching angle α. The deviation in the angular relationship between the virtual image V10 and the road surface 6 due to the attitude change (pitching) of the vehicle 1 can be corrected by adjusting the angle β (looking down angle (depression angle) of the virtual image V10) about the left-right direction of the virtual image V10. Can be adjusted.

In some embodiments, the processor 33 may adjust the depression angle β of the virtual image V10 (content FU) in response to changes in the attitude of the vehicle 1 in the second position adjustment process. The processor 33 dynamically increases the depression angle β in accordance with the increase in the pitching angle α in the forward tilting direction. Conversely, the processor 33 dynamically decreases the depression angle β in accordance with the increase in the pitching angle α in the backward tilting direction.

The processor 33 in some embodiments may also adjust the depression angle β of the virtual image V10 (content FU) with respect to the attitude change of the vehicle 1 when executing the first position adjustment process (first depression angle adjustment process). Similarly to the execution of the second position adjustment process, the processor 33 dynamically increases the depression angle β in accordance with the increase in the pitching angle α in the forward tilting direction. Conversely, the processor 33 dynamically decreases the depression angle β in accordance with the increase in the pitching angle α in the backward tilting direction.

The processor 33 in some embodiments may adjust the depression angle β by the same angle as the pitching angle change amount α when executing the first position adjustment process (an example of the first depression angle adjustment process). Specifically, when the vehicle 1 leans forward from the state shown in FIG. 6 to the state shown in FIG. 7A, the amount of change α in the pitching angle is α12. In the virtual viewpoint VP1 shown in FIG. 6, the angle of the virtual viewpoint VP2 with respect to the virtual plane 100 is changed by α12 based on the change amount α12 of the pitching angle of forward tilt shown in FIG. 7A. As a result, the depression angle β12 looking down on the content FU located at the target position MP1 with respect to the virtual viewpoint VP2 becomes larger than the depression angle β11 by α12. Furthermore, the processor 33 in some embodiments may adjust the depression angle β by an angle obtained by multiplying the pitching angle change amount α by a predetermined coefficient (an example of the first depression angle adjustment process).

Further, the processor 33 in some embodiments may adjust the depression angle β by the same angle as the pitching angle change amount α when executing the second position adjustment process (an example of the first depression angle adjustment process). Specifically, when the vehicle 1 leans forward from the state shown in FIG. 6 to the state shown in FIG. 7B, the amount of change α in the pitching angle is α13. The virtual viewpoint VP1 shown in FIG. 6 moves to the virtual viewpoint VP3 position according to the amount of change α13 in the forward pitching angle shown in FIG. 7B, and the virtual viewpoint VP3 angle with respect to the virtual plane 100 changes by α13. In accordance with this, the processor 33 may increase the depression angle β of the content FU expressed by the virtual image V13 by the amount of change α13 in the forward pitching angle. Furthermore, the processor 33 in some embodiments may adjust the depression angle β by an angle obtained by multiplying the pitching angle change amount α by a predetermined coefficient (an example of the first depression angle adjustment process).

In some preferred embodiments, the processor 33 determines the amount of adjustment of the angle of depression β (depression angle adjustment amount E20) with respect to the amount of change α of the pitching angle in the depression angle adjustment process (second depression angle adjustment process) executed together with the second position adjustment process. may be made larger than the adjustment amount of the depression angle β (depression angle adjustment amount E10) with respect to the change amount α of the pitching angle in the depression angle adjustment process (first depression angle adjustment process) executed together with the first position adjustment process. The virtual image V13 that has undergone the second position adjustment process is visually recognized to be shifted from the target position MP1 compared to the virtual image V12 that has undergone the first position adjustment process. Specifically, when tilting forward, the virtual image V13 that has been subjected to the second position adjustment process is closer to the real scene (with respect to the target position MP1 (110)) that is closer to the observer than the virtual image V12 that has been subjected to the first position adjustment process. It is shifted to a position 120 overlapping the road surface 6) and is visually recognized. When the content FU is placed near the viewer along the virtual plane 100 parallel to the road surface 6, the angle of depression β looking down on the content FU with respect to the virtual viewpoint VP becomes large. Conversely, when the content FU is placed far from the viewer, the angle of depression β when looking down on the content FU with respect to the virtual viewpoint VP becomes small. Therefore, in some more preferred embodiments, the processor 33 adjusts the amount of adjustment of the depression angle β (depression angle adjustment amount) of the content FU in the depression angle adjustment process (second depression angle adjustment process) executed together with the second position adjustment process to the vehicle The depression angle adjustment is a combination of the depression angle adjustment amount by changing the pitching angle α of 1 and the depression angle adjustment amount by shifting the content FU in the perspective direction of the virtual plane 100 (as an example, set to roughly match the road surface 6). It may also be a quantity. The depression angle β13 in FIG. 7B is the depression angle adjustment amount due to the content FU shifting the depression angle β11 in FIG. It is calculated as being corrected by the depression angle adjustment amount that is the sum of the depression angle adjustment amount due to the change in the pitching angle α of the vehicle 1.

FIG. 8A is a diagram showing the virtual image after executing the first position adjustment process and the first depression angle adjustment process, the left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the relationship between the virtual image and the virtual plane when the observer is facing forward. The foreground and virtual image that are visually recognized are shown. In FIG. 8A, it is assumed that the vehicle attitude AT10 is an AT14 in which the vehicle 1 is tilted backward by a relatively smaller pitching angle α14 than the vehicle attitude AT11 before the first position adjustment process. Here, the term "backward tilt" refers to the fact that the front of the vehicle 1 is raised (in other words, the rear of the vehicle 1 is lowered) based on the vehicle attitude AT11 shown in FIG. 6 . When the vehicle 1 tilts backward from the state shown in FIG. 6 to the state shown in FIG. 8A, the virtual image display area VS shifts upward (in the Y-axis positive direction) by B14 relative to the real scene (road surface) 6 due to attitude change. When the first position adjustment process is not performed, the virtual image V11 shown in FIG. 6 is moved upward to position G4 in the right diagram of FIG. The image is shifted by the image shift amount B14. That is, the virtual image V11 shifts from the target position MP1 where the content FU is desired to be placed. By executing the first position adjustment process, the processor 33 in some embodiments moves the virtual image position G4 downward (in the Y-axis negative direction) so as to suppress (cancel) the image shift amount B14 due to posture fluctuation. The first position adjustment amount C14 (C10) is adjusted (the virtual image V14 whose position has been adjusted is displayed). Preferably, the first position adjustment amount C14 (C10) is made equal to the image shift amount B14 (B10) due to the posture change (C10=B10), thereby reducing the virtual image V11 that would be displayed at the position G4 after the posture change. , can be maintained in the first region 110 (target position MP1). According to this, the image shift amount B14 (B10) due to the posture change is canceled out by the first position adjustment amount C14 (C10) and is not recognized by the observer. However, the first position adjustment amount C10 only needs to be able to reduce the image shift amount B10 due to posture variation, and may be smaller than the image shift amount B10. According to this, it is possible to suppress positional deviations due to changes in vehicle posture.

In some embodiments, the processor 33 may adjust the depression angle β by the same angle as the amount of change α in the pitching angle of the vehicle 1 in the backward tilting direction in the first depression angle adjustment process. Specifically, when the vehicle 1 tilts backward from the state shown in FIG. 6 to the state shown in FIG. 8A, the amount of change α in the pitching angle is α14. In the virtual viewpoint VP1 shown in FIG. 6, the angle of the virtual viewpoint VP4 with respect to the virtual plane 100 changes by α14 due to the amount of change α14 in the backward pitching angle shown in FIG. 8A. As a result, the depression angle β14 looking down on the content FU located at the target position MP1 with reference to the virtual viewpoint VP4 becomes smaller than the depression angle β11 by α14. Further, the processor 33 in some embodiments may adjust the depression angle β by an angle obtained by multiplying the pitching angle change amount α by a predetermined coefficient.

FIG. 8B is a diagram showing the virtual image after executing the second position adjustment process and the second depression angle adjustment process, the left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the relationship between the virtual image and the virtual plane when the observer faces forward. It shows the foreground and virtual image that are visually recognized. In FIG. 8B, it is assumed that the vehicle attitude AT10 is an AT15 in which the vehicle 1 is tilted backward by a relatively larger pitching angle α15 (α14>) than the vehicle attitude AT11 before the second position adjustment process. When the vehicle 1 tilts backward from the state shown in FIG. 6 to the state shown in FIG. 8B, the virtual image display area VS shifts upward (in the Y-axis positive direction) by B15 relative to the real scene (road surface) 6 due to attitude change. If the position adjustment is not performed, the virtual image V11 shown in FIG. 6 will be shifted by the image shift amount B15 to position G5 in the right diagram of FIG. 8B due to the image shift amount B15 due to the attitude change of the virtual image display area VS. . In order to offset the upward image shift amount B15 due to this posture change, the virtual image that would be displayed at position G5 should be moved downward by the image shift amount B15, but the virtual image that would be displayed at position G5 If the image is moved downward by the image shift amount B15, it will move out of the virtual image display area VS15.

In some embodiments, when the posture change is large (an example of a predetermined condition described later), the processor 33 executes the second position adjustment process to increase the upward image shift amount B15 of the virtual image display area VS due to the posture change. , the virtual image position G5 is adjusted downward (in the Y-axis positive direction) by a second position adjustment amount C25 (C20) smaller than the first position adjustment amount C15 (C10) in the first position adjustment process (position adjusted display virtual image V15). By making the second position adjustment amount C25 (C20) smaller than the image shift amount B15 (B10) due to posture change (C20<B10), the second position adjustment amount C25 (C20) is set to be smaller than the image shift amount B15 (B10) due to posture change, so that the first area 110 (target position MP1) is smaller than the first position adjustment process. The virtual image V15 can be maintained within the virtual image display area VS15, although the virtual image V15 is largely shifted to the fifth area 150 (a position different from the target position MP1) on the far side. According to this, the image shift amount B10 due to the posture change is not canceled out by the second position adjustment amount C20, and therefore is recognized by the observer. The processor 33 in this embodiment dynamically reduces the depression angle β in accordance with the increase in the pitching angle α in the backward tilting direction.

In some preferred embodiments, the processor 33 converts the adjustment amount of the depression angle β (depression angle adjustment amount E20) with respect to the change amount α of the pitching angle in the second depression angle adjustment process executed together with the second position adjustment process into the first position adjustment process. The amount of adjustment of the angle of depression β with respect to the amount of change α of the pitching angle in the first angle of depression adjustment process that is executed together with the process (the amount of depression angle adjustment E10) may be made larger. The virtual image V15 that has been subjected to the second position adjustment process is visually recognized to be more shifted from the target position MP1 than the virtual image V13 that has been subjected to the first position adjustment process. Specifically, when tilting backward, the virtual image V15 that has been subjected to the second position adjustment process is closer to the real scene (road surface) on the far side of the observer with respect to the target position MP1 (110) than the virtual image V14 that has been subjected to the first position adjustment process. 6) and is visually recognized as being shifted to a position 150 overlapping with 6). When the content FU is placed away from the viewer along the virtual plane 100 parallel to the road surface 6, the depression angle β of the content FU with respect to the virtual viewpoint VP becomes small. Therefore, the depression angle β15 in FIG. 8B is the depression angle adjustment amount due to the content FU shifting the depression angle β11 in FIG. and the depression angle adjustment amount due to a change in the pitching angle α of the vehicle 1 may be corrected by the depression angle adjustment amount.

In some embodiments, the processor 33 performs a first image adjustment that adjusts the position of the virtual image based on the first position adjustment amount C10 when the attitude of the vehicle 1 changes from the reference state to a first direction of forward or backward tilting. When the process (S151 described below) is executed and the attitude of the vehicle 1 changes from the reference state to a second direction opposite to the first direction of tilting forward or backward, the first image adjustment process (S151 described below) A second image adjustment process (S152 to be described later) is executed to suppress adjustment of the position of the virtual image with respect to changes in the posture of the vehicle 1.

FIG. 9 is a diagram illustrating image adjustment processing for posture variation, and shows an example in which the first image adjustment processing and the second image adjustment processing are switched depending on the magnitude of posture variation (or position adjustment amount). From time t11 to t12, the pitching angle α is a forward tilt smaller than a preset posture threshold αTd. The position adjustment amount C adjusts the virtual image display area VS in accordance with the posture change (forward tilting of the pitching angle α) so as to strongly suppress (preferably cancel) the downward shift of the virtual image caused by the posture change. The first position adjustment amount C10 is set to dynamically adjust the position of the virtual image in the upward direction. Thereby, the positional shift of the virtual image is preferably canceled out (the position of the virtual image is maintained at the target position MP1). The depression angle adjustment amount E is determined by adjusting the attitude change (forward tilt of the pitching angle α) so as to suppress (preferably cancel) the depression angle deviation between the real scene (road surface 6) and the virtual image caused by the attitude change (change in the pitching angle α). ) is set to the first depression angle adjustment amount E10 that dynamically increases the depression angle β of the virtual image. If the positional shift of the virtual image is offset by the first position adjustment amount C10, the virtual image will not shift from the target position MP1 and no size adjustment is necessary, so the size adjustment amount F will be zero.

From time t12 to t13, the pitching angle α is a forward tilt greater than a preset posture threshold αTd. The position adjustment amount C is smaller than the first position adjustment amount C10, which strongly suppresses (preferably cancels out) the downward shift of the virtual image caused by the attitude change, and is smaller than the first position adjustment amount C10, which strongly suppresses (preferably cancels out) the downward shift of the virtual image caused by the attitude change, and is The second position adjustment amount C20 is set to fix the position of the virtual image within the virtual image display area VS regardless of the position of the virtual image. This causes a downward shift of the virtual image due to attitude change (forward tilting of the pitching angle α) (in other words, due to forward tilting, the real view area where the virtual image overlaps shifts to the vicinity of the viewer). The depression angle adjustment amount E is the depression angle adjustment amount that corrects the deviation in depression angle between the real scene (road surface 6) and the virtual image caused by posture fluctuations (changes in the pitching angle α), and the depression angle that expresses that the virtual image has shifted to the vicinity of the observer. The second depression angle adjustment amount E20 is set to dynamically increase the depression angle β of the virtual image in accordance with the attitude change (forward tilting of the pitching angle α). Here, the second depression angle adjustment amount E20 is larger than the first depression angle adjustment amount E10 because the depression angle adjustment amount representing that the virtual image has shifted to the vicinity of the viewer is also added. The size adjustment amount F is set to dynamically increase the size of the virtual image in accordance with posture fluctuations (changes in pitching angle α) in order to express that the virtual image has shifted to the vicinity of the observer.

From time t13 to t14, the pitching angle α is a forward tilt smaller than a preset posture threshold αTd, and from time t14 to t15, the pitching angle α is a backward tilt smaller than a preset posture threshold αTu. The position adjustment amount C is set to adjust the position adjustment amount (forward tilt of the pitching angle (backward tilt from t14 to t15)) is set to a first position adjustment amount C10 that dynamically adjusts the position of the virtual image in the virtual image display area VS upward (downward from t14 to t15). Thereby, the positional shift of the virtual image is preferably canceled out (the position of the virtual image is maintained at the target position MP1). The depression angle adjustment amount E is determined by adjusting the attitude change (forward tilt of the pitching angle α) so as to suppress (preferably cancel) the depression angle deviation between the real scene (road surface 6) and the virtual image caused by the attitude change (change in the pitching angle α). (backward tilt from t14 to t15)) is set to a first depression angle adjustment amount E10 that dynamically increases the depression angle β of the virtual image (decreases it from t14 to t15). If the positional shift of the virtual image is offset by the first position adjustment amount C10, the virtual image will not shift from the target position MP1 and no size adjustment is necessary, so the size adjustment amount F will be zero.

From time t15 to t16, the pitching angle α is a backward tilt that is greater than a preset posture threshold αTu. The position adjustment amount C is smaller than the first position adjustment amount C10, which strongly suppresses (preferably cancels out) the upward shift of the virtual image caused by the attitude change, and is smaller than the first position adjustment amount C10, which strongly suppresses (preferably cancels) the upward shift of the virtual image caused by the attitude change, and is The second position adjustment amount C20 is set to fix the position of the virtual image within the virtual image display area VS regardless of the position of the virtual image. As a result, an upward shift of the virtual image occurs due to the attitude change (backward tilting of the pitching angle α) (in other words, the actual view area where the virtual image overlaps is shifted to a distance from the viewer due to the backward tilt). The depression angle adjustment amount E is the depression angle adjustment amount that corrects the deviation in depression angle between the real scene (road surface 6) and the virtual image caused by posture fluctuations (changes in the pitching angle α), and the depression angle that expresses that the virtual image has shifted to a distance from the viewer. The second depression angle adjustment amount E20 is set by adding the adjustment amount and dynamically increasing the depression angle β of the virtual image in accordance with posture fluctuations (backward tilting of the pitching angle α). Here, the second depression angle adjustment amount E20 is larger than the first depression angle adjustment amount E10 because it also includes the depression angle adjustment amount that expresses that the virtual image has shifted to the far side of the viewer. The size adjustment amount F is set to a size adjustment amount that dynamically reduces the size of the virtual image together with the attitude change (change in the pitching angle α) in order to express that the virtual image has shifted to a distance from the viewer. The image processing from time t16 to t19 is the same as the image processing from time t11 to t12 and from t13 to t15, and a description thereof will be omitted.

FIG. 10 is a diagram illustrating image adjustment processing in some embodiments, and shows the position adjustment amount Ca of the first content FU1 and the second content that change depending on the magnitude of posture fluctuation (or position adjustment amount). The position adjustment amount Cb of FU2 is shown. From time t21 to t22, the pitching angle α is a relatively small forward tilt. Here, the first position adjustment amount C10a of the first content FU1, which is set according to the posture change (forward tilt of the pitching angle α), is smaller than the preset upper limit CTu1 of the position adjustment amount. Further, the first position adjustment amount C10b of the second content FU2, which is set according to the posture change (forward tilt of the pitching angle α), is also smaller than the preset upper limit CTu2 of the position adjustment amount. Therefore, the position adjustment amount Ca of the first content FU1 dynamically adjusts the position of the virtual image V10 (first content FU1) in the virtual image display area VS upward in accordance with the posture change (forward tilt of the pitching angle α). The first position adjustment amount C10a is set. Thereby, the positional shift of the virtual image V10 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP1).

The position adjustment amount Cb of the second content FU2 dynamically adjusts the position of the virtual image V20 (second content FU2) in the virtual image display area VS gradually upward in accordance with the posture change (forward tilt of pitching angle α). The first position adjustment amount C10b is set. Thereby, the positional shift of the virtual image V20 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP2).

The target position MP1 of the first content FU1 may be set at a position closer to and further away from the viewer than the target position MP2 of the second content FU2. Therefore, as shown in FIG. ) may be larger than the position adjustment amount Cb of the second content FU2.

The depression angle adjustment amount E of the virtual image V10 (first content FU1) is set so as to suppress (preferably cancel out) the depression angle deviation between the real scene (road surface 6) and the virtual image caused by attitude fluctuation (change in pitching angle α). The first depression angle adjustment amount E10 is set to dynamically increase the depression angle β of the virtual image in accordance with the attitude change (forward tilt of the pitching angle α). If the positional deviation of the virtual image is offset by the first positional adjustment amount C10a, the size adjustment amount F becomes zero because the virtual image will not deviate from the target position MP1 and no size adjustment is necessary.

From time t22 to t23, the pitching angle α is a relatively large forward tilt. Here, the first position adjustment amount C10a of the first content FU1, which is set according to the posture change (forward tilt of pitching angle α), is larger than the preset upper limit CTu1 of the position adjustment amount. Then, the position adjustment amount Ca of the first content FU1 is the first position adjustment amount C10a that strongly suppresses (preferably cancels out) the downward shift of the virtual image V10 (first content FU1) caused by the posture change. It is set to a second position adjustment amount C20a that is smaller and fixes the position of the virtual image V10 in the virtual image display area VS to the upper peripheral edge VTu (see FIG. 21) regardless of posture fluctuations (forward tilting of the pitching angle α). This causes a downward shift of the virtual image due to attitude change (forward tilting of the pitching angle α) (in other words, due to forward tilting, the real view area where the virtual image overlaps shifts to the vicinity of the viewer).

On the other hand, the first position adjustment amount C10b of the second content FU2, which is set according to the posture change (forward tilt of the pitching angle α), is smaller than the preset upward position adjustment amount limit CTu2, but the virtual image At the same time that the position adjustment amount Ca of V10 (first content FU1) is switched to the second position adjustment amount C20a (switched from the first position adjustment process to the second position adjustment process), the position adjustment of the second content FU2 is performed. The amount Cb is also smaller than the first position adjustment amount C10b, which strongly suppresses (preferably cancels out) the downward shift of the virtual image V20 (second content FU2) caused by the attitude change, and The second position adjustment amount C20b is set to fix the position of the virtual image V20 within the virtual image display area VS regardless of the forward tilt of the virtual image V20. This causes a downward shift of the virtual image due to attitude change (forward tilting of the pitching angle α) (in other words, due to forward tilting, the real view area where the virtual image overlaps shifts to the vicinity of the viewer).

The depression angle adjustment amount E of the virtual image V10 (first content FU1) is the depression angle adjustment amount that corrects the depression angle deviation between the real scene (road surface 6) and the virtual image caused by attitude change (change in pitching angle α), and the depression angle adjustment amount that corrects the depression angle deviation between the virtual image and the virtual image. A second depression angle adjustment amount E20 is set, which dynamically increases the depression angle β of the virtual image in accordance with the posture change (forward tilting of the pitching angle α) by adding the depression angle adjustment amount representing a shift to the vicinity. Here, the second depression angle adjustment amount E20 is larger than the first depression angle adjustment amount E10 because the depression angle adjustment amount representing that the virtual image has shifted to the vicinity of the viewer is also added. The size adjustment amount F is set to dynamically increase the size of the virtual image in accordance with posture fluctuations (changes in pitching angle α) in order to express that the virtual image has shifted to the vicinity of the observer. In the present embodiment, the first position adjustment process is switched to the second position adjustment process based on the comparison result between the position adjustment amount Ca of the first content FU1 and the threshold value CTu1, but the pitching angle corresponding to the threshold value CTu1 is A threshold value αtd1 of α may be set, and the first position adjustment process may be switched to the second position adjustment process based on the comparison result between the pitching angle α and the threshold value αtd1.

From time t23 to t24, the pitching angle α is a forward tilt smaller than a preset posture threshold αTd. The first position adjustment amount Ca of the first content FU1 is set such that the posture change (forward tilt of the pitching angle α) strongly suppresses (preferably cancels out) the downward shift of the virtual image caused by the posture change. The first position adjustment amount C10a is set to dynamically adjust the position of the virtual image in the virtual image display area VS upward in accordance with the above. Thereby, the positional shift of the virtual image is preferably canceled out (the position of the virtual image is maintained at the target position MP1). The depression angle adjustment amount E is determined by adjusting the attitude change (forward tilt of the pitching angle α) so as to suppress (preferably cancel) the depression angle deviation between the real scene (road surface 6) and the virtual image caused by the attitude change (change in the pitching angle α). The depression angle β of the virtual image is dynamically increased (reduced from t24 to t25) in accordance with the angle of depression (rearward tilt from t24 to t25)). If the positional shift of the virtual image is offset by the first position adjustment amount C10a, the virtual image will not shift from the target position MP1 and no size adjustment is necessary, so the size adjustment amount F will be zero.

The position adjustment amount Cb of the second content FU2 dynamically adjusts the position of the virtual image V20 (second content FU2) in the virtual image display area VS gradually downward in accordance with the posture change (forward tilt of pitching angle α). The first position adjustment amount C10b is set. Thereby, the positional shift of the virtual image V20 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP2).

From time t24 to t25, the pitching angle α is a relatively small forward tilt. Here, the first position adjustment amount C10a of the first content FU1, which is set according to the posture change (forward tilt of pitching angle α), is smaller than the preset limit CTd1 of the downward position adjustment amount. Therefore, the position adjustment amount Ca of the first content FU1 dynamically adjusts the position of the virtual image V10 (the first content FU1) in the virtual image display area VS downward in accordance with the attitude change (the backward tilt of the pitching angle α). The first position adjustment amount C10a is set. Thereby, the positional shift of the virtual image V10 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP1).

The depression angle adjustment amount E of the virtual image V10 (first content FU1) is set so as to suppress (preferably cancel out) the depression angle deviation between the real scene (road surface 6) and the virtual image caused by attitude fluctuation (change in pitching angle α). The first depression angle adjustment amount E10 is set to dynamically reduce the depression angle β of the virtual image in accordance with posture fluctuations (backward tilting of the pitching angle α). If the positional deviation of the virtual image is offset by the first positional adjustment amount C10a, the size adjustment amount F becomes zero because the virtual image will not deviate from the target position MP1 and no size adjustment is necessary.

The first position adjustment amount C10b of the second content FU2, which is set according to the posture change (forward tilt of pitching angle α), is smaller than the preset limit CTd2 of the downward position adjustment amount. Therefore, the position adjustment amount Cb of the second content FU2 also dynamically adjusts the position of the virtual image V10 (first content FU1) in the virtual image display area VS downward in accordance with the attitude change (backward tilting of the pitching angle α). The first position adjustment amount C10b is set. Thereby, the positional shift of the virtual image V10 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP1).

From time t25 to t26, the pitching angle α is tilted backwards more than from time t25 to t26. Here, the first position adjustment amount C10b of the second content FU2, which is set according to the posture change (the backward tilt of the pitching angle α), is larger than the preset limit CTd2 of the downward position adjustment amount. Then, the position adjustment amount Cb of the second content FU2 is the first position adjustment amount C10b that strongly suppresses (preferably cancels out) the upward shift of the virtual image V20 (second content FU2) caused by the posture change. It is set to a second position adjustment amount C20b that is smaller and fixes the position of the virtual image V20 in the virtual image display area VS to the lower peripheral portion VTd (see FIG. 21) regardless of posture fluctuations (rearward tilting of the pitching angle α). As a result, an upward shift of the virtual image occurs due to the attitude change (backward tilting of the pitching angle α) (in other words, the actual view area where the virtual image overlaps is shifted to a distance from the viewer due to the backward tilt).

On the other hand, the first position adjustment amount C10a of the first content FU1, which is set according to the posture change (the backward tilt of the pitching angle α), is smaller than the preset downward position adjustment amount limit CTd1, but the virtual image At the same time as the position adjustment amount Cb of V20 (second content FU2) is switched to the second position adjustment amount C20b (switched from the first position adjustment process to the second position adjustment process), the position adjustment of the first content FU1 is performed. The amount Ca is also smaller than the first position adjustment amount C10a, which strongly suppresses (preferably cancels out) the upward shift of the virtual image V10 (first content FU1) caused by the attitude change, and The second position adjustment amount C20a is set to fix the position of the virtual image V20 within the virtual image display area VS regardless of the backward tilt of the virtual image V20. This causes an upward shift of the virtual image due to the attitude change (backward tilting of the pitching angle α) (in other words, due to the backward tilt, the real view area where the virtual image overlaps is shifted to the far side of the viewer).

The depression angle adjustment amount E of the virtual image V10 (first content FU1) is the depression angle adjustment amount that corrects the depression angle deviation between the real scene (road surface 6) and the virtual image caused by attitude change (change in pitching angle α), and the depression angle adjustment amount that corrects the depression angle deviation between the virtual image and the virtual image. A second depression angle adjustment amount E20 that dynamically reduces the depression angle β of the virtual image in accordance with the attitude change (backwards tilt of the pitching angle α) is set by adding the depression angle adjustment amount expressing the shift to the vicinity. Here, the second depression angle adjustment amount E20 is larger than the first depression angle adjustment amount E10 because it also includes the depression angle adjustment amount that expresses that the real view area on which the virtual image overlaps has shifted to the far side of the viewer. The size adjustment amount F is set to dynamically reduce the size of the virtual image in accordance with attitude changes (changes in the pitching angle α) in order to express that the real-view region on which the virtual image overlaps has shifted to the far side of the observer. Ru. In the present embodiment, the first position adjustment process is switched to the second position adjustment process based on the comparison result between the position adjustment amount Cb of the second content FU2 and the threshold value CTd2, but the pitching angle corresponding to the threshold value CTd2 is A threshold value αtu2 of α may be set, and the first position adjustment process may be switched to the second position adjustment process based on the comparison result between the pitching angle α and the threshold value αtu2. The image processing from time t26 to t29 is the same as the image processing from time t21 to t22 and from t23 to t25, so a description thereof will be omitted.

The depression angle adjustment amount E of the first content FU1 is changed depending on the change in the attitude of the vehicle, but the depression angle adjustment amount Eb of the second content FU2 may be constant regardless of the change in the attitude of the vehicle.

FIG. 11 is a diagram showing the position of the virtual image V10 of the first content FU1 and the position of the virtual image V20 of the second content FU2, which are adjusted according to changes in the posture of the vehicle. In FIG. 11(c), the attitude of the vehicle is the reference attitude α0. The virtual image V10 is displayed at the reference position PO of the first region VT (virtual image display region VS as an example), and is displayed so as to overlap the target position MP1. The virtual image V20 is displayed at the reference position PO of the virtual image display area VS (or the first area VT), and is displayed so as to overlap the target position MP2. The reference attitude α0 is the pitching angle α of the vehicle stored in the memory 37 in advance, and is typically a state in which the pitching angle of the vehicle is zero (parallel to the road surface 6). Note that the reference posture α0 may be variable. Specifically, if the pitching angle α of the vehicle does not change much for a predetermined period of time or more, the stable pitching angle α at that time may be set (updated) as the reference attitude α0 and stored in the memory 37.

FIG. 11(d) shows how the virtual image is displayed when the vehicle leans forward slightly. The virtual image display area VS shifts downward with respect to the real scene when the vehicle leans forward slightly. According to the present embodiment, the virtual image V10 is preferably shifted upward from the reference position PO by the first position adjustment amount C10a so that it continues to be displayed at the target position MP1. Further, the virtual image V20 is preferably shifted upward from the reference position PO by the first position adjustment amount C10b so that it continues to be displayed at the target position MP2.

FIG. 11(e) shows how a virtual image is displayed when the vehicle leans forward significantly. When the vehicle leans forward significantly, the virtual image display area VS shifts significantly downward with respect to the real scene. According to the present embodiment, the virtual image V10 is preferably shifted to an upper peripheral portion VTu of the first display area VT by the second position adjustment amount C20a so that it continues to be displayed within the first display area VT. Fixed display. Further, the virtual image V20 is fixedly displayed at a position shifted by the second position adjustment amount C20b at the same time as the virtual image V10 is fixedly displayed by the second position adjustment amount C20a. The virtual image V20 is fixedly displayed in accordance with the virtual image V20, regardless of whether it has reached the upper peripheral edge VTu, and is typically fixedly displayed below the upper peripheral edge VTu.

FIG. 11(b) shows how a virtual image is displayed when the vehicle tilts slightly backward. The virtual image display area VS shifts upward relative to the real scene when the vehicle tilts slightly backward. According to this embodiment, the virtual image V10 is preferably shifted below the reference position PO by the first position adjustment amount C10a so that it continues to be displayed at the target position MP1. Further, the virtual image V20 is preferably shifted downward from the reference position PO by the first position adjustment amount C10b so that it continues to be displayed at the target position MP2.

FIG. 11(a) shows how a virtual image is displayed when the vehicle tilts significantly backward. When the vehicle tilts significantly backward, the virtual image display area VS shifts significantly upward relative to the real scene. According to the present embodiment, the virtual image V20 is preferably shifted to a lower peripheral portion VTd of the first display area VT by the second position adjustment amount C20b so as to continue to be displayed within the first display area VT. Fixed display. Further, the virtual image V10 is fixedly displayed at a position shifted by the second position adjustment amount C20a at the same time as the virtual image V20 is fixedly displayed by the second position adjustment amount C20b. The virtual image V10 is fixedly displayed in accordance with the virtual image V20, regardless of whether it has reached the lower peripheral edge VTd, and is typically fixedly displayed above the lower peripheral edge VTd.

FIG. 12 is a diagram illustrating image adjustment processing in some embodiments, and shows the position adjustment amount Ca of the first content FU1 and the second content that change depending on the magnitude of posture fluctuation (or position adjustment amount). The position adjustment amount Cb of FU2 is shown. From time t31 to t32, the pitching angle α is a relatively small forward tilt. Here, the first position adjustment amount C10a of the first content FU1, which is set according to the posture change (forward tilt of the pitching angle α), is smaller than the preset upper limit CTu1 of the position adjustment amount. Further, the first position adjustment amount C10b of the second content FU2, which is set according to the posture change (forward tilt of the pitching angle α), is also smaller than the preset upper limit CTu2 of the position adjustment amount. Therefore, the position adjustment amount Ca of the first content FU1 dynamically adjusts the position of the virtual image V10 (first content FU1) in the virtual image display area VS upward in accordance with the posture change (forward tilt of the pitching angle α). The first position adjustment amount C10a is set. Thereby, the positional shift of the virtual image V10 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP1).

The position adjustment amount Cb of the second content FU2 dynamically adjusts the position of the virtual image V20 (second content FU2) in the virtual image display area VS gradually upward in accordance with the posture change (forward tilt of pitching angle α). The first position adjustment amount C10b is set. Thereby, the positional shift of the virtual image V20 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP2).

The target position MP1 of the first content FU1 may be set closer to the viewer and further away than the target position MP2 of the second content FU2. Therefore, the processor 33 (position adjustment module 522) calculates the position adjustment amount Ca of the first content FU1 with respect to the attitude change (pitching angle α) of the vehicle 1, as shown in FIG. ) may be larger than the position adjustment amount Cb of the second content FU2.

The depression angle adjustment amount E of the virtual image V10 (first content FU1) is set so as to suppress (preferably cancel out) the depression angle deviation between the real scene (road surface 6) and the virtual image caused by attitude variation (change in pitching angle α). The first depression angle adjustment amount E10 is set to dynamically increase the depression angle β of the virtual image in accordance with the attitude change (forward tilt of the pitching angle α). The size adjustment amount F becomes zero because if the positional deviation of the virtual image is offset by the first position adjustment amount C10a, the virtual image will not deviate from the target position MP1 and no size adjustment is necessary.

From time t32 to t33, the pitching angle α is a relatively large forward tilt. Here, the first position adjustment amount C10a of the first content FU1, which is set according to the posture change (forward tilt of pitching angle α), is larger than the preset upper limit CTu1 of the position adjustment amount. Then, the position adjustment amount Ca of the first content FU1 is the first position adjustment amount C10a that strongly suppresses (preferably cancels out) the downward shift of the virtual image V10 (first content FU1) caused by the posture change. It is set to a second position adjustment amount C20a that is smaller and fixes the position of the virtual image V10 in the virtual image display area VS to the upper peripheral edge VTu (see FIG. 21) regardless of posture fluctuations (forward tilting of the pitching angle α). This causes a downward shift of the virtual image due to attitude change (forward tilting of the pitching angle α) (in other words, due to forward tilting, the real view area where the virtual image overlaps shifts to the vicinity of the viewer).

On the other hand, the first position adjustment amount C10b of the second content FU2, which is set according to the posture change (forward tilt of the pitching angle α), is smaller than the preset upward position adjustment amount limit CTu2, but the virtual image At the same time that the position adjustment amount Ca of V10 (first content FU1) is switched to the second position adjustment amount C20a (switched from the first position adjustment process to the second position adjustment process), the position adjustment of the second content FU2 is performed. The amount Cb is also smaller than the first position adjustment amount C10b, which strongly suppresses (preferably cancels out) the downward shift of the virtual image V20 (second content FU2) caused by the attitude change, and The second position adjustment amount C20b is set to fix the position of the virtual image V20 within the virtual image display area VS regardless of the forward tilt of the virtual image V20. This causes a downward shift of the virtual image due to attitude change (forward tilting of the pitching angle α) (in other words, due to forward tilting, the real view area where the virtual image overlaps shifts to the vicinity of the viewer).

The depression angle adjustment amount E of the virtual image V10 (first content FU1) is the depression angle adjustment amount that corrects the depression angle deviation between the real scene (road surface 6) and the virtual image caused by attitude fluctuation (change in pitching angle α), and the depression angle adjustment amount that corrects the depression angle deviation between the virtual image and the virtual image. A second depression angle adjustment amount E20 is set, which dynamically increases the depression angle β of the virtual image in accordance with the posture change (forward tilting of the pitching angle α) by adding the depression angle adjustment amount representing a shift to the vicinity. Here, the second depression angle adjustment amount E20 is larger than the first depression angle adjustment amount E10 because the depression angle adjustment amount representing that the virtual image has shifted to the vicinity of the viewer is also added. The size adjustment amount F is set to dynamically increase the size of the virtual image in accordance with posture fluctuations (changes in pitching angle α) in order to express that the virtual image has shifted to the vicinity of the observer. In the present embodiment, the first position adjustment process is switched to the second position adjustment process based on the comparison result between the position adjustment amount Ca of the first content FU1 and the threshold value CTu1, but the pitching angle corresponding to the threshold value CTu1 is A threshold value αtd1 of α may be set, and the first position adjustment process may be switched to the second position adjustment process based on the comparison result between the pitching angle α and the threshold value αtd1.

During time t35 to t36, the pitching angle α is a larger backward tilt (an example of attitude change in the second direction) than during time t34 to t35. Here, the first position adjustment amount C10b of the second content FU2, which is set according to the posture change (the backward tilt of the pitching angle α), is larger than the preset limit CTd2 of the downward position adjustment amount. Then, the position adjustment amount Cb of the second content FU2 is the first position adjustment amount C10b that strongly suppresses (preferably cancels out) the upward shift of the virtual image V20 (second content FU2) caused by the posture change. It is set to a second position adjustment amount C20b that is smaller and fixes the position of the virtual image V20 in the virtual image display area VS to the lower peripheral edge VTd (see FIG. 21) regardless of posture fluctuations (rearward tilting of the pitching angle α). As a result, an upward shift of the virtual image occurs due to the attitude change (backward tilting of the pitching angle α) (in other words, the actual view area where the virtual image overlaps is shifted to a distance from the viewer due to the backward tilt).

On the other hand, the first position adjustment amount C10a of the first content FU1, which is set according to the posture change in the second direction (the backward tilt of pitching angle α), is smaller than the preset limit CTd1 of the downward position adjustment amount. Become. Therefore, the position adjustment amount Ca of the first content FU1 dynamically adjusts the position of the virtual image V10 (the first content FU1) in the virtual image display area VS downward in accordance with the attitude change (the backward tilt of the pitching angle α). The first position adjustment amount C10a is set. Thereby, the positional shift of the virtual image V10 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP1).

From time t36 to t37, the pitching angle α is tilted backwards more than from time t35 to t36. Here, the first position adjustment amount C10b of the second content FU2, which is set according to the posture change (the backward tilt of the pitching angle α), is equal to the preset downward position adjustment amount following time t35 to t36. Since it is larger than the limit CTd2, it is set to the second position adjustment amount C20b.

On the other hand, the first position adjustment amount C10a of the first content FU1, which is set according to the posture change (the backward tilt of the pitching angle α), is larger than the preset upper limit CTd1 of the position adjustment amount. Then, the position adjustment amount Ca of the first content FU1 is the first position adjustment amount C10a that strongly suppresses (preferably cancels out) the upward shift of the virtual image V10 (first content FU1) caused by the posture change. It is set to a second position adjustment amount C20a that is smaller and fixes the position of the virtual image V10 in the virtual image display area VS to the lower peripheral edge VTd (see FIG. 21) regardless of posture fluctuations (rearward tilting of the pitching angle α). This causes an upward shift of the virtual image due to the attitude change (backward tilting of the pitching angle α) (in other words, due to the backward tilt, the real view area where the virtual image overlaps is shifted to the far side of the viewer).

From time t37 to t38, the pitching angle α is smaller backward than from time t36 to t37. Here, the first position adjustment amount C10b of the second content FU2, which is set according to the posture change (the backward tilt of the pitching angle α), is the same as the preset downward position adjustment amount following time t36 to t37. Since it is larger than the limit CTd2, it is set to the second position adjustment amount C20b. As a result, from time t36 to t37, an upward shift of the virtual image occurs due to attitude change (backward tilting of the pitching angle α) (in other words, the actual view area where the virtual image overlaps shifts to the far side of the viewer due to the backward tilt). .

On the other hand, the first position adjustment amount C10a of the first content FU1, which is set in accordance with the posture change (the backward tilt of the pitching angle α), is smaller than the preset limit CTd1 of the downward position adjustment amount. Therefore, the position adjustment amount Ca of the first content FU1 dynamically adjusts the position of the virtual image V10 (the first content FU1) in the virtual image display area VS downward in accordance with the attitude change (the backward tilt of the pitching angle α). The first position adjustment amount C10a is set (the second position adjustment process is switched to the first position adjustment process). Thereby, the positional shift of the virtual image V10 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP1). In the present embodiment, the first position adjustment process is switched to the second position adjustment process based on the comparison result between the position adjustment amount Ca of the first content FU1 and the threshold value CTd1, but the pitching angle corresponding to the threshold value CTd1 is A threshold value αtu1 of α may be set, and the first position adjustment process may be switched to the second position adjustment process based on the comparison result between the pitching angle α and the threshold value αtu1.

During time t38 to t40, the first position adjustment amount C10a of the first content FU1 does not exceed the position adjustment amount limits CTu1, CTd1, and the first position adjustment amount C10b of the second content FU2 also exceeds the position adjustment amount limits. In order not to exceed CTu2 and CTd2, the first content FU1 and the second content FU2 are set to the first position adjustment amount C10a and the first position adjustment amount C10b, respectively.

FIG. 13 is a diagram illustrating image adjustment processing in some embodiments, and shows the position adjustment amount Ca of the first content FU1 and the second content that change depending on the magnitude of posture fluctuation (or position adjustment amount). The position adjustment amount Cb of FU2 is shown. In some embodiments, the processor 33 performs a first position adjustment process on the first content FU1 and the second content FU2 in response to 1) a posture change in a first direction in which the vehicle tilts forward or backward from a reference posture; When the first position adjustment process is executed (times t41 to t44), one of the virtual images to be offset is changed from being placed within the first area VT of the virtual image display area VS to being placed outside the first area VT. If the state changes to the state where When the first position adjustment process is executed on one of the virtual images in response to the attitude change in the second direction, and the first position adjustment process is executed without executing the position adjustment process on the other virtual image (times t44 to t49), When one of the virtual images to be offset changes from being placed within the first area VT of the virtual image display area VS to being placed outside the first area VT, a second position adjustment process is applied to one of the virtual images. (time t46 to t47).

FIG. 14 is a diagram showing the position of the virtual image V10 of the first content FU1 and the position of the virtual image V20 of the second content FU2, which are adjusted according to changes in the posture of the vehicle. In FIG. 14(c), the attitude of the vehicle is the reference attitude. The virtual image V10 is displayed at the reference position PO of the first region VT (virtual image display region VS as an example), and is displayed so as to overlap the target position MP1. The virtual image V20 is displayed at the reference position PO of the virtual image display area VS (or the first area VT), and is displayed so as to overlap the target position MP2.

FIG. 14(d) shows how the virtual image is displayed when the vehicle leans forward slightly. The virtual image display area VS shifts downward with respect to the real scene when the vehicle leans forward slightly. According to the present embodiment, the virtual image V10 is preferably shifted upward from the reference position PO by the first position adjustment amount C10a so that it continues to be displayed at the target position MP1. Further, the virtual image V20 is preferably shifted upward from the reference position PO by the first position adjustment amount C10b so that it continues to be displayed at the target position MP2.

FIG. 14(e) shows how the virtual image is displayed when the vehicle leans forward significantly. When the vehicle leans forward significantly, the virtual image display area VS shifts significantly downward with respect to the real scene. According to the present embodiment, the virtual image V10 is preferably shifted to an upper peripheral portion VTu of the first display area VT by the second position adjustment amount C20a so that it continues to be displayed within the first display area VT. Fixed display. Further, the virtual image V20 is fixedly displayed at a position shifted by the second position adjustment amount C20b at the same time as the virtual image V10 is fixedly displayed by the second position adjustment amount C20a. The virtual image V20 is fixedly displayed in accordance with the virtual image V20, regardless of whether it has reached the upper peripheral edge part VTu, and is typically fixedly displayed below the upper peripheral edge part VTu.

FIG. 14(b) shows a virtual image display mode when the vehicle tilts slightly backward (posture change in the second direction). The virtual image display area VS shifts upward relative to the real scene when the vehicle tilts slightly backward. According to this embodiment, the virtual image V10 is preferably shifted below the reference position PO by the first position adjustment amount C10a so that it continues to be displayed at the target position MP1. Further, the virtual image V20 is not adjusted in position with respect to the attitude change in the second direction, and is displayed fixedly at the reference position PO.

FIG. 14(a) shows how a virtual image is displayed when the vehicle tilts significantly backward. The virtual image display area VS shifts significantly upward relative to the real scene when the vehicle tilts significantly backward. According to the present embodiment, the virtual image V20 is not adjusted in position with respect to the attitude change in the second direction, but is displayed fixedly at the reference position PO, and is typically displayed fixedly above the lower peripheral portion VTd. . Further, the virtual image V10 is preferably fixedly displayed at a position shifted to the lower peripheral portion VTd of the first display region VT so as to continue to be displayed within the first display region VT by the second position adjustment amount C20a. . According to the present embodiment, the positions of the virtual images V10 and V20 are dynamically adjusted in response to a small posture change in the first direction (for example, forward leaning), and In conjunction with the switching of the virtual image V10 of the first content FU1 from the first position adjustment process to the second position adjustment process, the virtual image V20 of the second content FU2 also switches from the first position adjustment process to the second position adjustment process. On the other hand, in response to a small posture change in the second direction (for example, backward tilting), the virtual image V20 does not perform dynamic position adjustment (fixed display at the reference position PO), and the virtual image V10 dynamically adjusts the position and , the virtual image V10 of the first content FU1 switches from the first position adjustment process to the second position adjustment process, and the virtual image V20 of the second content FU2 is fixedly displayed at the reference position PO. maintain.

The size adjustment process will be explained below. FIG. 15 is a diagram showing the virtual image before posture change. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the foreground and virtual image that are visible when the observer faces forward. show. In FIG. 15, it is assumed that the vehicle attitude AT10 is AT16. When viewing the first virtual image V11 from the virtual viewpoint VP6, the area that overlaps with any virtual plane 100 parallel to the road surface 6 is defined as a first area 110, and the length of the first area 110 in the depth direction is defined as a first length L10. . The arbitrary virtual plane 100 is a virtual plane on which the content FU is arranged, and is set parallel to the front, rear, left, and right directions of the vehicle 1 (it may generally coincide with the road surface 6). The drawing module 510 displays an image M, which is the source of the virtual image V16, on the display device 50 so as to display an image obtained by projectively transforming the content FU onto the virtual image display area VS16 (here, the virtual image V16) based on the virtual viewpoint VP6. display. Here, the distance from the virtual viewpoint VP6 to the first region 110 along the virtual plane 100 is set as D0, and the distance from the virtual plane 100 to the virtual viewpoint VP1 is set as h0.

FIG. 16A is a diagram showing the virtual image after size adjustment processing when the virtual image display area is shifted downward from the real scene due to forward tilting. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the relationship between the virtual image and the virtual plane. , shows the foreground and virtual image that are visible when the viewer faces forward. In FIG. 16A, it is assumed that the vehicle attitude AT10 is AT17 in which the vehicle 1 is tilted more forward than the vehicle attitude AT16 shown in FIG. In this case, the position of the virtual plane 100 on which the virtual image V17 overlaps is shifted to the second region 120 closer to the viewer than the first region 110 (target position MP1) in FIG. 15. That is, as shown in the right diagram of FIG. 16A, the virtual image V17 is visually recognized to overlap with a region (second region 120) of the road surface 6 that is closer to the vehicle 1 (observer) than the first region 110. In this manner, when it is estimated that the virtual image will be visible overlapping the closer foreground, the processor 33 increases the size of the virtual image. That is, based on the forward tilt of the vehicle 1, the virtual image moves from the reference position (for example, the center 205 of the eye box 200) to the second area 120 that is closer to the reference position than the first area 110 (target position MP1) in the virtual plane 100. When it is determined that the position where the virtual images overlap changes, the virtual image V17 is displayed at the position overlapping the second area 120, and the second length L20 in the depth direction of the second area 120 where the virtual image V17 overlaps is the first. It is made larger than the virtual image V16 shown in FIG. 15 so that the length is the same as L10 (L20=L10). According to this, it is possible to express a sense of perspective as if a virtual image (virtual object) having a predetermined size had moved along the road surface.

FIG. 16B is a diagram showing the virtual image after size adjustment processing when the virtual image display area is shifted upward from the real scene due to backward tilting. The left diagram shows the relationship between the virtual image and the virtual plane, and the right diagram shows the relationship between the virtual image and the virtual plane. , shows the foreground and virtual image that are visible when the viewer faces forward. In FIG. 16B, it is assumed that the vehicle attitude AT10 is AT18 in which the vehicle 1 is tilted more rearward than the vehicle attitude AT16 shown in FIG. In this case, the position of the virtual plane 100 on which the virtual image V18 overlaps is shifted to the third region 130 that is farther from the viewer than the first region 110 (target position MP1) in FIG. 15. That is, as shown in the right diagram of FIG. 16B, the virtual image V18 is visually recognized to overlap with a region (third region 130) of the road surface 6 that is farther from the vehicle 1 (observer) than the first region 110. In this way, when it is estimated that the virtual image will be visible overlapping the closer foreground, the processor 33 reduces the size of the virtual image. That is, based on the backward tilt of the vehicle 1, when viewed from the reference position (for example, the center 205 of the eyebox 200), the virtual image moves to the third area 130 which is farther from the reference position than the first area 110 (target position MP1) in the virtual plane 100. When it is determined that the position where the virtual images overlap changes, the virtual image V18 is displayed at the position overlapping the third area 130, and the third length L30 in the depth direction of the third area 130 where the virtual image V18 overlaps is the first It is made smaller than the virtual image V16 shown in FIG. 15 so that the length is the same as L10 (L30=L10). According to this, it is possible to express a sense of perspective as if a virtual image (virtual object) having a predetermined size had moved along the road surface. However, in forward tilt, the virtual image may be made larger so that the second length L20 is longer than the first length L10 (L20>L10). Further, in the backward tilt, the virtual image may be made smaller so that the third length L30 is shorter than the first length L10 (L30<L10). The perspective of the virtual image (virtual object) according to some embodiments is more emphasized than the same perspective as in real space, such as when a virtual image (virtual object) having a predetermined size moves along a road surface while maintaining its size. be done. Therefore, it is possible to give the viewer a stronger impression of approaching or separating.

FIG. 17 is a diagram illustrating the first position adjustment process in which the position is adjusted without performing the size adjustment process when the vehicle 1 tilts backward, and the left diagram shows the first position adjustment process before the attitude change. The right figure shows the virtual image and foreground after the attitude change and after the first position adjustment process. In FIG. 17, the virtual image display area VS16 is shifted upward (in the Y-axis positive direction) by B16 due to the attitude change. If the position adjustment process is not performed, the first virtual image V31 (V10) will shift to the position G6 in the right diagram of FIG. 17 as the virtual image display area VS shifts due to backward tilting. By adjusting the position, the processor 33 in some embodiments moves the first virtual image V31 downward (Y-axis negative) so as to suppress (preferably cancel out) the upward image shift amount B16 due to backward tilting. direction) by the first position adjustment amount C16 (C10) (display V21c). Preferably, the first position adjustment amount C16 (C10) is made equal to the image shift amount B16 due to the attitude change (C16=B16), thereby maintaining the virtual image V21c after the attitude change at a position overlapping the target position MP1. be able to. According to this, the image shift due to the posture change is canceled out by the first position adjustment amount C10 and is not recognized by the observer. However, the first position adjustment amount C16 (C10) only needs to reduce the image shift amount B16 (B10) due to posture variation, and may be smaller than the image shift amount B16 (B10). According to this, the display shift based on the change in the vehicle attitude is suppressed, so that it is possible to prevent the viewer from feeling uncomfortable due to the change in the size of the virtual image due to the display shift.

FIG. 18 is a diagram illustrating the second position adjustment process in which the size adjustment process is performed without performing the position adjustment when the vehicle 1 tilts backward. The left diagram shows the virtual image before the attitude change and before the size adjustment. The figure on the right shows the virtual image and foreground after changing the pose and adjusting the size. In FIG. 18, the virtual image display area VS is shifted upward (in the Y-axis positive direction) by B17 due to backward tilting. If the position adjustment is not performed, the first virtual image V32 also shifts upward (in the Y-axis positive direction) by B17, as shown in the right diagram of FIG. 18, as the virtual image display area VS shifts due to the attitude change. As a result, the virtual image V22a is shifted to a position where it overlaps a foreground region that is farther from the viewer than the first region 110 before the attitude change. In some embodiments, when it is estimated that the virtual image will be seen overlapping the more distant foreground, the processor 33 adjusts the size of the displayed virtual image V22a (right diagram in FIG. 18) by adjusting the posture change. It is made smaller than the previous virtual image V22 (left diagram in FIG. 18). Even if the attitude of the vehicle changes, the size of the virtual image changes, thereby emphasizing the perspective of the virtual image in real space and giving the viewer the impression that the virtual image is receding.

FIG. 19 is a diagram illustrating the second position adjustment process in which the position adjustment and size adjustment process are performed when the vehicle 1 tilts backward. The left diagram shows the state before the attitude change and before the second position adjustment process. The virtual image and the foreground are shown, and the right figure shows the virtual image and the foreground after the attitude change and after the second position adjustment process. In FIG. 19, the virtual image display area VS is shifted upward (in the Y-axis positive direction) by B18 due to the attitude change. If the second position adjustment process is not performed, the virtual image V22 shifts to the position G8 in the right diagram of FIG. 19 due to the shift of the virtual image display area VS due to the attitude change. By adjusting the position, the processor 33 in some embodiments moves the virtual image V21 downward by a second position adjustment amount smaller than the image shift amount B18 so as to suppress the upward image shift amount B18 due to backward tilting. Adjust only C28 (display virtual image V23c). Then, when it is estimated that the virtual image will be visually recognized as overlapping the more distant foreground, the processor 33 of the present embodiment adjusts the size of the displayed virtual image V23c (right figure in FIG. 19) to is made smaller than the virtual image V23 (left diagram in FIG. 19). Thereby, it is possible to reduce the image shift caused by the change in the posture of the vehicle, and to reduce the feeling of unnatural size due to the image shift.

The position adjustment process shown in FIG. 20 is performed by the processor 33 executing the image adjustment module 520 stored in the memory 37. In step S110, the image adjustment module 520 acquires the drawing data generated by the drawing module 510. In step S120, the image adjustment module 520 extracts the virtual image V10 (the original of the virtual image V10) from the information indicating the display position of the image M before performing the position adjustment process (information indicating the reference position PO) included in the acquired drawing data. Set the limit CT of the position adjustment amount of image M). Specifically, for example, the image adjustment module 520 sets a position adjustment range VT, and sets a limit CT of the position adjustment amount C based on the position adjustment range VT and the reference position PO.

FIG. 21 is a diagram for explaining the limit CT of the position adjustment amount. If the lower end of the position adjustment range VT is set close to the reference position PO of the virtual image V10, the limit CTd of the downward position adjustment amount of the virtual image V10 is set short. On the other hand, if the upper end of the position adjustment range VT is set far away from the reference position PO of the virtual image V10, the limit CTu of the upward position adjustment amount of the virtual image V10 is set longer. The image adjustment module 520 may set the position adjustment range VT individually for each of the plurality of virtual images V10 displayed in the virtual image display area VS, or may set it commonly for all the virtual images V10. The position adjustment range VT may be the virtual image display area VS (the entire virtual image display area VS may be set as the position adjustment range VT). The image adjustment module 520 has table data (not shown) that associates the attitude change (pitch angle α) of the vehicle 1 with the position adjustment amount C of the virtual image V10, and based on the table data, the image adjustment module 520 adjusts the position of the virtual image V10. The limit (attitude threshold) of the backward pitching angle αTu that is assumed when the position adjustment amount C reaches the downward position adjustment amount limit CTd, and the position adjustment amount C of the virtual image V10 is the upward position adjustment amount A limit (posture threshold value) αTd of the pitching angle of forward tilt that is assumed to reach the limit CTu may be set.

When the virtual image V10 or the virtual image V20 is shifted downward within the position adjustment range VT, the virtual image V10 or the virtual image V20 is dynamically shifted according to the posture change until it reaches the limit CTd of the downward position adjustment amount. For larger posture fluctuations, the display is fixed at a position adjusted by the downward position adjustment amount limit CTd (lower peripheral edge VTd). Conversely, when the virtual image V10 or the virtual image V20 is shifted upward within the position adjustment range VT, the virtual image V10 or the virtual image V20 is dynamically shifted according to the posture change until the upper position adjustment amount limit CTu is reached. However, for larger posture fluctuations, the display is fixed at the position adjusted by the upward position adjustment amount limit CTu (upper peripheral edge VTu).

FIG. 22 is a diagram for explaining the limit CT of the position adjustment amount. A plurality of position adjustment ranges VT may be provided within the virtual image display area VS for each content FU. If either the first position adjustment amount C10a of the first content FU1 or the first position adjustment amount C10b of the second content FU2 reaches the position adjustment amount limit CTu (or CTd), the content that has reached the limit In addition, the other content may also be switched from the first position adjustment process to the second position adjustment process at the same time.

Next, in step S130 in FIG. 20, the image adjustment module 520 acquires information indicating the attitude change of the vehicle 1 (attitude change information) from the attitude detection unit 415. Posture detection unit 415 includes one or more sensors such as a gyro sensor, an acceleration sensor, and a height sensor. The attitude detection unit 415 calculates the vehicle attitude such as pitching angle and roll angle, the frequency of change in the vehicle attitude, etc. as attitude change information from the sensor values such as the angular velocity, acceleration, and height of the moving object, and displays the information in the display control device. It may also be output to 30. That is, the attitude change information may include a frequency of change in the vehicle attitude (vibration frequency), etc. in addition to the vehicle attitude (pitch angle, roll angle, etc.). Part or all of the function for the posture detection unit 415 to calculate the posture change information may be provided in the display control device 30.

In step S140, the image adjustment module 520 calculates a first position adjustment amount C10 used in a first image adjustment process S151, which will be described later. First, the image adjustment module 520 calculates the amount of attitude variation (angular deviation amount) of the vehicle 1 based on the attitude variation information acquired from the attitude detection unit 415. For example, the image adjustment module 520 calculates the angle (pitch angle) α of the vehicle 1 around the pitch axis by performing an integral calculation on the angular velocity detected by the attitude detection unit 415. Thereby, the amount of deviation (angle) of the vehicle 1 in the rotational direction about the Y axis (the pitch axis) shown in FIG. 1 can be calculated. Note that in this embodiment, the pitching angle is calculated, but the yaw angle or the roll angle may also be calculated. For example, all angles around the X, Y, and Z axes may be calculated. However, part or all of the function of calculating the amount of posture variation (angular deviation amount) in the image adjustment module 520 is provided by a device other than the display control device 30 that can communicate with the display control device 30, and The device 30 may receive information indicating the amount of posture variation (angular deviation amount) of the vehicle 1 from the other device via the I/O interface 31. That is, some display control devices 30 may omit the function of calculating the posture variation amount (angular shift amount) in the image adjustment module 520.

In step S140, the image adjustment module 520 further includes a first position adjustment amount C10a for adjusting the display position of the virtual image V10 of the first content FU1 based on the amount of posture variation (angular deviation amount) of the vehicle 1; and calculates a first position adjustment amount C10b for adjusting the display position of the virtual image V"0 of the second content FU2. Specifically, the first position adjustment amount C10a (first position adjustment amount C10b) is ( The amount of adjustment is such that the amount of deviation (pitching angle) is converted into the number of pixels and the number of pixels corresponding to the deviation (image shift amount B10 due to posture change) is returned to the original value.Preferably, the image adjustment module 520 In order to restore the positional shift of the virtual image V10 of the first content FU1 due to the attitude change of the vehicle 1, a position adjustment amount (first position adjustment amount C10a) in the opposite direction that is equal to the image shift amount B10 due to the attitude change is calculated. In order to restore the positional shift of the virtual image V20 of the second content FU2 due to the attitude change of the vehicle 1, the image adjustment module 520 adjusts the position adjustment amount in the opposite direction (first position adjustment amount) equal to the image shift amount B10 due to the attitude change. C10b) is calculated.

In step S150 of some embodiments, the image adjustment module 520 generates image data by performing image adjustments on the drawing data obtained in step S110. In S150, the image adjustment module 520 rearranges the pixels of the drawing data as pixels of the image data based on the position adjustment amount C, the depression angle adjustment amount E, and the size adjustment amount F. In S160, the image adjustment module 520 outputs the image data generated (adjusted) in S150 to the display device 50.

In step S150 of some embodiments, the image adjustment module 520 adjusts the position adjustment amount limit CTu( or CTd), the first position adjustment amount C10a (C10b) is set to dynamically adjust the position of the virtual image V10 (V20) in the virtual image display area VS in accordance with the attitude change (pitch angle α) of the vehicle. (first position adjustment process). Thereby, the positional shift of the virtual image V20 is strongly suppressed and preferably canceled out (the position of the virtual image is maintained at the target position MP1 (MP2)). On the other hand, the image adjustment module 520 determines whether either the first position adjustment amount C10a of the first content FU1 or the first position adjustment amount C10b of the second content FU2 reaches the position adjustment amount limit CTu (or CTd). , not only one content that has reached the limit but also the other content are simultaneously switched from the first position adjustment process to the second position adjustment process.

In the first image adjustment process S151, the image adjustment module 520 adjusts the virtual image V10 to The position of (the image M that is the source of the virtual image V10) is adjusted. Preferably, the first position adjustment amount C10 is set so as to offset the positional shift of the image caused by the posture change.

In the first image adjustment process S151 in some embodiments, in addition to the position adjustment, the image adjustment module 520 adjusts the first depression angle adjustment amount E10 (depression angle adjustment amount E) in accordance with the attitude change acquired in step S130. may be dynamically changed, and the depression angle of the virtual image V10 (the image M that is the source of the virtual image V10) may be adjusted based on the first depression angle adjustment amount E10 (the depression angle adjustment amount E).

In the first image adjustment process S151 in some embodiments, the first position adjustment amount C10 is not set to offset the image position shift caused by the posture change (in other words, if the image position shift caused by the posture change is canceled out), the first position adjustment amount C10 is In addition to the position adjustment, the image adjustment module 520 dynamically changes the size adjustment amount F according to the posture change acquired in step S130, and based on this size adjustment amount F, the image adjustment module 520 adjusts the virtual image V10 ( The size of the image M) that is the source of the virtual image V10 may be adjusted.

In the second image adjustment process S152, the image adjustment module 520 dynamically changes the second depression angle adjustment amount E20 (depression angle adjustment amount E) according to the attitude change acquired in step S130, and this second depression angle adjustment amount The depression angle of the virtual image V10 (the image M that is the source of the virtual image V10) is adjusted based on E20 (the depression angle adjustment amount E). The depression angle adjustment amount E20 in the second image adjustment process S152 is a depression angle adjustment amount that corrects the depression angle deviation between the real scene (road surface 6) and the virtual image caused by attitude variation (change in pitching angle α). Preferably, the depression angle adjustment amount E20 in the second image adjustment process S152 is a depression angle adjustment amount that corrects the depression angle deviation between the real scene (road surface 6) and the virtual image caused by attitude variation (change in pitching angle α), An angle of depression adjustment amount may be added to express a shift to the vicinity (or a shift to a far distance) due to a change in the angle α).

Further, in the second image adjustment process S152 in some embodiments, the image adjustment module 520 dynamically changes the second position adjustment amount C20 (position adjustment amount C) in accordance with the posture change acquired in step S130. (position adjustment process in second image adjustment process S152). The image adjustment module 520 may include table data, arithmetic expressions, etc. for setting the second position adjustment amount C20 (position adjustment amount C) from the attitude change acquired in step S130. In the second image adjustment process S152 in some embodiments, the image adjustment module 520, in addition to the depression angle adjustment, adjusts the second position adjustment amount C20 (second The position of the virtual image V10 (the image M that is the source of the virtual image V10) may be adjusted based on the one-position adjustment amount C10.

In the position adjustment process in the second image adjustment process S152, the image adjustment module 520 calculates a second position adjustment amount C20 that is dynamically changed based on the amount of posture variation of the vehicle 1. For example, the second position adjustment amount C20 is obtained by multiplying the first position adjustment amount C10, which is determined based on the amount of posture variation of the vehicle 1, by a coefficient smaller than 1. Further, the position adjustment module 522 of some embodiments may read out the second position adjustment amount C20 stored in the memory 37, which is not based on the amount of posture variation of the vehicle 1. In this embodiment, the adjustment amount in the pitch axis direction is calculated, but the adjustment amount in the yaw axis direction and the roll direction may also be calculated. Regarding the roll angle, an adjustment amount is determined so as to return the deviation amount of the roll angle to the original value while leaving the angle as it is. However, part or all of the function of calculating the second position adjustment amount C20 of the image adjustment module 520 is provided by a device other than the display control device 30 that can communicate with the display control device 30, and the display control device 30 is , a display parameter (second position adjustment amount C20) for adjusting the position of the virtual image may be input from the other device via the I/O interface 31.

In addition to the position adjustment, in the second image adjustment process S152 in some embodiments, the image adjustment module 520 dynamically changes the size adjustment amount F in accordance with the posture change acquired in step S130, Based on this size adjustment amount F, the size adjustment process of the virtual image V10 (the image M that is the source of the virtual image V10) may be performed. The image adjustment module 520 may include table data, arithmetic expressions, etc. for setting the size adjustment amount F from the posture change obtained in step S130.

When the predetermined condition is satisfied, the display control device 30 (processor 33) of the present embodiment executes the second position adjustment process (step S152) to adjust the virtual image according to the deviation of the virtual image due to the attitude change of the moving body. Adjust the depression angle β of V10. An example of calculating the depression angle β of the virtual image V10 according to the attitude change (angular deviation amount) of the vehicle 1 using an arithmetic expression is shown below. By changing the position and angle of the viewpoint VP according to changes in the posture of the vehicle 1 and rendering according to the content (virtual object) FU seen from the virtual viewpoint VP, the position and angle of the image M that is the source of the virtual image V10 are changed. (depression angle β) and size may be changed.

The image adjustment module 520 (depression angle adjustment module 524) calculates the depression angle β of the virtual image V10 (content FU) according to the attitude change (angular deviation amount) of the vehicle 1.

First, as shown in FIGS. 6 to 8B, the angle (pitching angle) around the pitch axis of the vehicle 1 is set to α (a negative value when tilting forward and a positive value when tilting backward), and from the virtual viewpoint VP to the virtual plane. 100 (the height h when the angular deviation amount α is zero is h0), and the depth direction from the virtual viewpoint VP to the near end of the area on the virtual plane 100 where the virtual image V10 appears to overlap. The distance Z is defined as D (the distance D when the angular deviation amount α is zero is D0).

The image adjustment module 520 (depression angle adjustment module 524) may calculate the distance D according to the pitching angle α using Equation 1 below (but is not limited to this).

Further, the image adjustment module 520 (depression angle adjustment module 524) may calculate the depression angle β according to the distance D using Equation 2 below (but is not limited to this).

Note that the image adjustment module 520 (depression angle adjustment module 524) is not limited to the above calculation method as long as it can adjust the depression angle β of the virtual image V10 according to the attitude (angular deviation amount) of the vehicle 1. For example, the image adjustment module 520 (depression angle adjustment module 524) stores in the memory 37 in advance table data that associates the correction coefficient of the depression angle β of the virtual image V10 with the posture fluctuation (angular deviation amount) of the vehicle 1, The correction coefficient may be read out based on the input information (signal) indicating the attitude change (angular deviation amount) of the vehicle 1, and the depression angle β of the content FU (virtual image V10) may be adjusted.

Furthermore, the image adjustment module 520 (depression angle adjustment module 524) may calculate the depression angle adjustment amount Δβ based on the distance D to the content (virtual object) FU on the virtual plane 100 using the following formula 3 (this (but not limited to).

More preferably, the image adjustment module 520 (depression angle adjustment module 524) calculates the depression angle adjustment amount Δβ using the following formula 4, taking into account that the tilt angle θt of the virtual image display area VS is tilted by the angular deviation amount α. (but not limited to)

Further, the display control device 30 (processor 33) of some embodiments adjusts the size of the virtual image V10 in accordance with a shift in the virtual image due to a change in the posture of the moving object. An example of calculating the size of the virtual image V10 according to the attitude change (angular deviation amount) of the vehicle 1 using an arithmetic expression is shown below. By changing the position and angle of VP according to changes in the posture of the vehicle 1, by rendering according to the content (virtual object) FU seen from the virtual viewpoint VP, the position of the image M that is the source of the virtual image V10, The angle (depression angle β) and size may be changed.

The image adjustment module 520 (size adjustment module 526) calculates the size of the virtual image V10 (content FU) according to the attitude change (angular deviation amount) of the vehicle 1. Here, the size of the virtual image V10 is assumed to be the vertical angle (angle of view) of the virtual image V10 viewed from the reference position. The angle of view is the angle between a line connecting the upper end of the virtual image V10 from a predetermined reference position (for example, the center 205 of the eyebox 200) and a line connecting the lower end of the virtual image V10 from the predetermined reference position, It corresponds to the size of the virtual image V10 in the vertical direction (Y-axis direction) as seen from the observer.

The image adjustment module 520 (size adjustment module 526) sets the size according to the distance D, where L is the length in the depth direction Z of the area on the virtual plane 100 where the virtual images V10 appear to overlap from the virtual viewpoint VP1. It may be calculated using Formula 5 (but is not limited to this).

Note that the image adjustment module 520 (size adjustment module 526) is not limited to the above calculation method as long as it can adjust the size of the virtual image V10 according to the attitude (angular deviation amount) of the vehicle 1. For example, the image adjustment module 520 (size adjustment module 526) stores table data in advance in the memory 37 in which the adjustment coefficient for the size of the virtual image V10 is associated with the attitude (angular deviation amount) of the vehicle 1, and inputs the The size of the virtual image V10 may be adjusted by reading out the adjustment coefficient based on information (signal) indicating the attitude (angular deviation amount) of the vehicle 1. However, a part or all of the functions of the image adjustment module 520 (size adjustment module 526) are provided in a separate device from the display control device 30 that can communicate with the display control device 30, and the display control device 30 is Display parameters for adjusting the size of the virtual image may be input from the device via the I/O interface 31. That is, some display control devices 30 may omit the image adjustment module 520 (size adjustment module 526).

The head-up display device 20 described in this specification includes one of the display control devices 30 in some embodiments, a display surface 50a that emits display light, and a display surface 50a that emits display light from the display surface 50a. 2. In this case as well, advantages similar to those described above are envisaged.

The operations of the processing processes described above may be performed by executing one or more functional modules of an information processing device, such as a general-purpose processor or an application-specific chip. All these modules, combinations of these modules, and/or combinations with known hardware that can replace their functions fall within the scope of protection of the present invention.

The functional blocks of vehicle display system 10 are optionally implemented in hardware, software, or a combination of hardware and software to implement the principles of the various embodiments described. It is noted that the functional blocks described in FIG. 3 may optionally be combined or one functional block separated into two or more sub-blocks to implement the principles of the described embodiments. It will be understood by those skilled in the art. Accordingly, the description herein optionally supports any possible combination or division of the functional blocks described herein.

1: Vehicle 2: Projection area 6: Road surface 10: Vehicle display system 20: Head-up display device (HUD device)
30: Display control device 31: I/O interface 33: Processor 35: Image processing circuit 37: Memory 40: Image display device 50: Display device 50a: Display surface 200: Eye box 205: Center 510: Drawing module 520: Image adjustment Module 522: Position adjustment module 524: Depression adjustment module 526: Size adjustment module 700: Observer C: Position adjustment amount C10: First position adjustment amount C10a: First position adjustment amount C10b: First position adjustment amount C20a: Second Position adjustment amount C20b: Second position adjustment amount CT: Limit Ca: First content position adjustment amount Cb: Second content position adjustment amount D: Distance E: Depression angle adjustment amount E10: First depression angle adjustment amount E20: Second Depression angle adjustment amount F: Size adjustment amount FU1: First content FU2: Second content MP: Target position MP1: Target position MP2: Target position PO: Reference position VP: Virtual viewpoint VS: Virtual image display area VT: Position adjustment range VTd: Lower peripheral edge VTu: Upper peripheral edge α: Pitching angle α0: Reference posture β: Depression angle θt: Tilt angle

Claims (10)

It has a display unit that is mounted on a vehicle and displays an image on a display surface, and by directing the light of the image toward the projected unit, a first content and a second content are displayed in a virtual image display area that overlaps with the road surface in front of the vehicle. A display control device that controls a head-up display device that allows virtual images including content to be viewed in an overlapping manner,
The one or more processors are:
acquiring attitude change information indicating attitude change of the vehicle;
1) In order to suppress relative positional deviation between the virtual image and the road surface due to attitude change of the vehicle, a first position adjustment process is executed to dynamically change the position of the virtual image in accordance with the attitude change information. ,
2) When the first position adjustment process is executed, either the first content or the second content changes from being placed within the first area of the virtual image display area to being placed outside the first area. When the virtual image changes to a state in which the virtual image changes to a state where performing the second position adjustment process on the other virtual image as well;
A display control device characterized by: The first content is arranged so as to be visible above the second content,
The processor includes:
When the vehicle tilts backward and the second content, which is offset when the first position adjustment process is executed, is located below the first area, the position of the second content is changed to the first area. and/or the vehicle leans forward; If the first content, which is offset when the first position adjustment process is executed, is placed above the first area, the first content may be positioned at an upper peripheral edge within the first area. fixing the second content, and simultaneously performing the second position adjustment process of fixing the position of the second content at a predetermined position below the upper peripheral edge in the first area;
The display control device according to claim 1, characterized in that: The processor includes:
1) performing the first position adjustment process on the first content and the second content in response to a posture change in a first direction in which the vehicle tilts forward or backward from a reference posture;
When the first position adjustment process is executed, the one of the virtual images to be offset changes from being placed within the first area of the virtual image display area to being placed outside the first area. , performing the second position adjustment process on the first content and the second content;
2) performing the first position adjustment process on the one of the virtual images in response to a change in the attitude of the vehicle from the reference attitude in a second direction opposite to the first direction;
When the first position adjustment process is executed, the one of the virtual images to be offset changes from being placed within the first area of the virtual image display area to being placed outside the first area. , performing the second position adjustment process on the one of the virtual images;
The display control device according to claim 1, characterized in that: The processor includes:
When the first position adjustment process is executed in response to a posture change in a first direction in which the vehicle tilts forward or backward from a reference posture, the first content is placed within the first area of the virtual image display area. When changing from a state in which the virtual image is placed outside the first area to a state in which the virtual image is placed outside the first area, the adjustment of the position of the virtual image in response to a change in the attitude of the vehicle is suppressed for the first content rather than in the first position adjustment process. performing a second position adjustment process, and simultaneously performing the second position adjustment process on the second content;
When the arrangement of the first content and/or the second content is changed,
When the vehicle executes the first position adjustment process in accordance with the attitude change in the predetermined direction from the reference attitude, the second content changes from being arranged in the first area of the virtual image display area to When the virtual image changes to a state where it is placed outside the first area, a second position adjustment process for the second content suppresses adjustment of the position of the virtual image in response to a change in the attitude of the vehicle more than the first position adjustment process. and simultaneously performing the second position adjustment process on the first content.
The display control device according to claim 1, characterized in that: The processor includes:
When the first position adjustment process is executed in response to a posture change in a predetermined direction in which the vehicle tilts forward or backward from a reference posture, the first content is placed within the first area of the virtual image display area. When changing from a state in which the virtual image is placed outside the first area to a state in which the virtual image is placed outside the first area, the adjustment of the position of the virtual image in response to a change in the attitude of the vehicle is suppressed for the first content rather than in the first position adjustment process. performing a second position adjustment process, and simultaneously performing the second position adjustment process on the second content;
When the size of the first content and/or the second content is changed,
When the vehicle executes the first position adjustment process in accordance with the attitude change in the predetermined direction from the reference attitude, the second content changes from being arranged in the first area of the virtual image display area to When the virtual image changes to a state where it is placed outside the first area, a second position adjustment process for the second content suppresses adjustment of the position of the virtual image in response to a change in the attitude of the vehicle more than the first position adjustment process. and simultaneously performing the second position adjustment process on the first content.
The display control device according to claim 1, characterized in that: In the first position adjustment process, the processor:
making the amount of position adjustment of the first content with respect to the attitude change of the vehicle larger than the amount of position adjustment of the second content with respect to the attitude change of the vehicle;
The display control device according to claim 1, characterized in that: The processor includes:
arranging a target position of the first content farther than a target position of the second content;
7. The display control device according to claim 6. The processor includes:
Based on the attitude change information, setting an angle of depression adjustment amount that suppresses a deviation in a relative oblique angle (hereinafter referred to as an angle of depression) between the virtual image and the road surface due to a change in the attitude of the vehicle;
adjusting the depression angle of the first content based on the depression angle adjustment amount, and not adjusting the depression angle of the second content based on the depression angle adjustment amount;
The display control device according to claim 1, characterized in that: a display unit installed in a vehicle and displaying an image on a display surface;
a relay optical system that directs display light from the display section to a projected section;
one or more processors;
memory and
one or more computer programs stored in the memory and configured to be executed by the one or more processors; first content in a virtual image display area overlapping with a road surface in front of the vehicle; and a head-up display device that allows virtual images including second content to be viewed in an overlapping manner,
The one or more processors are:
acquiring attitude change information indicating attitude change of the vehicle;
1) In order to suppress relative positional deviation between the virtual image and the road surface due to attitude change of the vehicle, a first position adjustment process is executed to dynamically change the position of the virtual image in accordance with the attitude change information. ,
2) When the first position adjustment process is executed, either the first content or the second content changes from being placed within the first area of the virtual image display area to being placed outside the first area. When the virtual image changes to a state in which the virtual image changes to a state where performing the second position adjustment process on the other virtual image as well;
A head-up display device characterized by: It has a display unit that is mounted on a vehicle and displays an image on a display surface, and by directing the light of the image toward the projected unit, a first content and a second content are displayed in a virtual image display area that overlaps with the road surface in front of the vehicle. A display control method for controlling a head-up display device that allows virtual images including content to be viewed in an overlapping manner, the method comprising:
acquiring attitude change information indicating attitude change of the vehicle;
1) In order to suppress relative positional deviation between the virtual image and the road surface due to attitude change of the vehicle, a first position adjustment process is executed to dynamically change the position of the virtual image in accordance with the attitude change information. And,
2) When the first position adjustment process is executed, either the first content or the second content changes from being placed within the first area of the virtual image display area to being placed outside the first area. When the virtual image changes to a state in which the virtual image changes to a state where performing the second position adjustment process on the other virtual image as well;
A display control method characterized by:

Images