JP7494646B2 - Head-up display device, display control device, and control method for head-up display device - Google Patents

Description

The present invention relates to a head-up display (HUD) device that projects (projects) image display light onto a projection target such as a vehicle windshield or combiner to display a virtual image in front of the viewer, a display control device, and a control method for a head-up display device.

A head-up display (HUD) device that displays (draws) content superimposed on the real scene is believed to minimize eye movement and convey information such as the direction of travel and warnings to the driver (visitor) without loss, thereby providing safe driving.

To achieve this, a system is needed that can sense the outside world and display images that match actual objects, as well as a system that can change the display position in accordance with changes in the positional relationship with the object that occur when the driver moves their viewpoint. To make such corrections, it is necessary to accurately capture changes in the driver and the outside world and correct the display without delay.

However, for example, in systems such as on-road (road surface overlay) HUDs and 3D-HUDs, image distortion tends to be large, and when the viewpoint is moved, the displayed content is likely to shift away from the overlay object.

This will require systems that detect the driver's viewpoint position with higher accuracy and less delay, as well as systems to correct the display position. In this case, it cannot be denied that the burden of the correction process on the system will be heavy. Furthermore, there are limits to the correction process, and it cannot be said that there will be no cases where delays or position shifts occur, reducing the recognition of the presented information.

One possible solution to this problem is to implement viewpoint-tracking warping, which updates warping parameters according to the viewpoint position of the driver or other person. Note that warping is an image correction method in which the image projected in a HUD device is distorted in advance so that it has characteristics opposite to the distortion of a virtual image caused by the optical system, the curved shape of the windshield, etc. Warping processing in a HUD device is described, for example, in Patent Document 1.

JP 2015-87619 A

The inventors have investigated implementing viewpoint-tracking warping control, which updates warping parameters according to the viewpoint position of a viewer (which can be broadly interpreted as a driver, operator, crew member, etc.), and have recognized the following new problems.

When the viewpoint position moves and the warping parameters are updated, the appearance of the virtual image from the viewer's perspective (the appearance of the virtual image, the impression given by the virtual image, etc.) may change discontinuously when the parameters are switched, which may cause the viewer to feel uncomfortable or reduce the viewer's ability to recognize information (depth perception).

One of the objectives of the present invention is to prevent the viewer from feeling uncomfortable when the appearance of the image changes discontinuously due to the update of the warping parameters when performing viewpoint-tracking warping processing in a HUD device.

Other objects of the present invention will become apparent to those skilled in the art by reference to the following exemplary aspects and best modes, as well as the accompanying drawings.

Below, we will provide examples of embodiments according to the present invention to facilitate an understanding of the overview of the present invention.

In a first aspect, a head-up display device includes:
A head-up display (HUD) device that projects an image onto a projection target, thereby allowing a viewer to view a virtual image of the image,
an image generating unit that generates the image;
A display unit for displaying the image;
an optical system including an optical member that reflects display light of the image and projects the image onto the projection target;
a control unit that performs a warping control according to a viewpoint position of a viewer in an eyebox, and updates a warping parameter according to the viewpoint position of the viewer in the eyebox, and uses the warping parameter to perform a warping control according to a viewpoint position tracking warping control for distorting an image to be displayed on the display unit in advance so that the image has a distortion characteristic opposite to a distortion characteristic of a virtual image of the image;
having
When the inside of the eyebox is divided into a plurality of divided regions, each divided region is a viewing zone, and the position of the viewer's gaze point is detected in units of the viewing zone,
The control unit is
When it becomes necessary to update the warping parameters due to the movement of the viewpoint, a transition period of a predetermined time is set, and a first warping parameter is set, and the value of the parameter is changed to the value of the first warping parameter;
During the transition period, the first warping parameter is maintained regardless of the movement of the viewpoint, or the second warping parameter is set as a new warping parameter in response to the movement of the viewpoint while reducing the frequency of parameter updates or maintaining the same frequency of parameter updates as compared with a case where the warping parameter is updated every time a viewpoint position is detected in units of the viewing zone.
A first change process including the control
When it becomes necessary to update the warping parameters associated with the movement of the viewpoint after the transition period has elapsed, a third warping parameter corresponding to the viewing zone after the transition period has elapsed is set, and the value of the warping parameter is changed to become the value of the third warping parameter, or is changed gradually, linearly or nonlinearly, with respect to time.
The second change process including the above control is executed.

In the first aspect, when a movement of the viewpoint occurs, the movement continues for a certain period of time, and in many cases the movement of the viewpoint eventually converges (stops). This is taken into consideration, and a transition period of the movement of the viewpoint (simply referred to as a transition period) is set when the first warping parameter is newly set.

During this transition period, the first warping parameter is maintained regardless of the movement of the viewpoint, or the first change process is executed, which includes control to set the second warping parameter as a new warping parameter in response to the movement of the viewpoint while reducing the frequency of parameter updates or maintaining the same (in other words, the same) parameter update frequency compared to when the warping parameter is updated each time the viewpoint position is detected in units of the viewing zone.

While keeping the detection sensitivity of the viewpoint position high by setting the viewing zone finely, during the transition period, the first parameter is maintained, or the number of parameter updates is reduced to avoid frequent parameter updates compared to when the parameters are updated for each viewing zone (however, it is acceptable to set the same parameter update frequency), and the second parameter is set little by little in response to the movement of the viewing zone. This prevents the viewer from feeling uncomfortable. Here, setting the second parameter little by little can be seen as a process (tracking process) in which the parameter value is changed to follow the current situation little by little while watching the state of the viewing zone movement (in other words, while observing the progress of the viewing zone movement), and is brought closer to the value of the third parameter corresponding to the viewing zone detected after the transition period has elapsed. In addition, the first warping parameter may be maintained regardless of the viewing zone movement, but in this case, the parameter value also changes over time to become the value of the first warping parameter. Because the first warping parameter is a warping parameter updated for the first viewpoint movement, the change in the parameter value is likely to follow subsequent viewpoint movements, albeit slowly, and therefore may be included in the above-mentioned tracking process.

When the viewpoint position is detected after the transition period has elapsed, a third warping parameter corresponding to the detected viewing zone is set, and a second change process is executed, which includes control to change the parameter value (which may be intended to be a quick change) or to change it gradually over time. This second change process makes it possible for the parameter value to arrive at (reach) the value of the third parameter within a realistically acceptable time range.

This makes it possible to change the warping parameter value of the visual field after the viewpoint movement has converged within an acceptable delay time while maintaining high accuracy in detecting the viewpoint position and suppressing the sense of discomfort caused by discontinuous changes in the parameter value. In other words, even if the viewpoint moves over a certain range, effective measures are taken, and degradation in the display quality of the image (virtual image) displayed by the HUD device can be suppressed.

In addition, in the first change process and the second change process, the characteristics of the changes can basically be set separately and independently (however, they can also be linked or related). For example, the parameters can be changed linearly or nonlinearly. Also, the change rates of the first change process and the second change process can be flexibly designed to be large and small, small and large, or the same. This makes it possible, for example, to fine-tune the performance of the viewpoint position tracking warping or change the characteristics to suit the current situation, thereby improving the performance of the HUD device.

In a second aspect dependent on the first aspect,
A first rate of change of the parameter value with respect to time in the first change process may be set to be smaller than a second rate of change of the parameter value with respect to time in the second change process.

In the second aspect, the rate of change of the first change process (first rate of change) is set to be smaller than the rate of change of the second change process (second rate of change).

In one preferred embodiment, in the first change process during the transition period, from the viewpoint of observing the movement of the viewpoint position, or from the viewpoint of not causing discomfort to the viewer during the movement of the viewpoint, the rate of change of the parameter value over time is suppressed to a certain extent and changed little by little. In other words, the parameter value is not changed too suddenly during the transition period. Therefore, the rate of change of the parameter value in the first change process (first rate of change) tends to be set relatively small.

On the other hand, after the transition period has elapsed and the viewpoint movement has converged, it is preferable to ensure (guarantee) the tracking performance of the parameter change in the viewpoint tracking warping by making the parameter value reach the value of the parameter (third warping parameter) corresponding to the detected viewing area relatively quickly. Therefore, in this case, it is preferable to set the rate of change of the parameter value in the second change process (second rate of change) to be greater than the first rate of change. In other words, the second change process may be quickly switched to the parameter (third warping parameter) corresponding to the detected viewing area after the transition period has elapsed.

This makes it possible to suppress the decrease in visibility that accompanies discontinuous changes in warping parameter values, while ensuring (ensuring) the tracking performance of parameter changes in viewpoint-tracking warping.

In a third aspect dependent on the first or second aspect,
The control unit is
During the transition period, when a viewpoint moves while the first change process is being executed, and the parameter value is changed to a warping parameter value corresponding to a new viewing zone in response to the viewpoint movement,
The third change process may be executed so as to result in a rate of change of the parameter value that is greater than the rate of change of the parameter value in the first change process.

In the third aspect, during the transition period, if necessary, a third change process can be performed in which the rate of change of the parameter value is set to be larger than that of the first change process. As described above, during the transition period, the rate of change of the parameter value is set to be slightly smaller to suppress discontinuous changes.

However, for example, if the viewpoint moves through multiple viewing zones during the transition period, and the amount of change (fluctuation) in the warping parameter value increases accordingly, it is conceivable that if only the first change process is performed, the warping will not be able to keep up with the change in viewpoint position, resulting in significantly delayed processing. In this case, there is no choice but to respond by suddenly changing the parameter value after the transition period has elapsed, but this increases the likelihood that the appearance of the displayed virtual image will change discontinuously, causing discomfort to the viewer.

Therefore, before the transition period expires, the change in the parameter value is increased more than in the first change process to improve the tracking of the parameter value. By doing this, when the transition period expires, the parameter value will have changed quickly in response to the movement of the viewpoint, and the amount of change in the parameter value after the transition period has elapsed can be reduced. This makes it possible to suppress the occurrence of discomfort, etc.

In a fourth aspect dependent from the third aspect,
The control unit is
When the third change process is executed,
Instead of the second change process after the transition period has elapsed, a fourth change process in which a rate of change of a parameter value is greater than the rate of change in the third change process may be executed.

In the fourth aspect, after the transition period has elapsed, instead of the second change process in the first aspect, a fourth change process is performed in which the rate of change of the parameter value is set to be larger. As explained above, in the third aspect, the third change process is performed before the end of the transition period, improving the tracking of the parameter value and suppressing the amount of change in the parameter value after the transition period has elapsed.

However, even if the third change process is performed, if the amount of change in the parameter value after the transition period has elapsed is relatively large, the second change process described in the first aspect may take a long time for the parameter value to reach the target value, and the tracking performance of viewpoint tracking warping may deteriorate.

Therefore, instead of the second change process, a fourth change process with a larger change rate of the parameter value is executed to shorten the time required for the parameter value to reach the target value and suppress the deterioration of tracking performance. Therefore, there is no problem that the tracking performance of the viewpoint tracking warping is deteriorated and the display quality of the corrected image (virtual image) is deteriorated.

In a fifth aspect dependent on any one of the first to fourth aspects,
The control unit is
When irregular viewpoint movement, a decrease in accuracy of viewpoint position detection, or a temporary loss of viewpoint position is detected during execution of the first change process within the transition period, the first change process may be stopped and the warping parameters at the time of stopping may be maintained.

In the fifth aspect, if circumstances arise during the transition period that make it impossible to perform highly accurate viewpoint-tracking warping, the first change process is stopped (aborted) and measures are taken to maintain the warping parameters at the time of stopping. The parameter values are maintained at the parameter values at the time of stopping for the maintained warping parameters, for example. However, modifications are also possible, such as selecting (adopting) an appropriate other value rather than the parameter value at the time of stopping, and maintaining the parameter value at that value.

This prevents continued viewpoint-tracking warping from causing problems such as the image (virtual image) becoming unstable and appearing unnatural.

In a sixth aspect dependent on any one of the first to fifth aspects,
The head-up display device includes:
When adjusting the position of the eyebox in accordance with the height position of the viewpoint of the viewer, the optical member may not be moved, and a reflection position of the display light of the image on the optical member may be changed.

In a sixth aspect, when the HUD device that implements the above control adjusts the height position of the eyebox in accordance with the height position of the viewer's eyes (point of view), the optical member that projects light onto the projection target is not rotated, for example, by an actuator, but is changed by changing the reflection position of the light on the optical member.

Recently, HUD devices have tended to be developed on the premise that virtual images will be displayed over a fairly wide area in front of the vehicle, for example, which inevitably results in larger devices. Naturally, the optical components also become larger. If these optical components are rotated using an actuator or the like, the resulting error can actually reduce the accuracy of controlling the height position of the eyebox. To prevent this, the position at which the light rays are reflected by the optical components is changed.

In such large optical components, the reflective surface is optimally designed as a free-form surface to prevent distortion of the virtual image as much as possible. However, as mentioned above, there are cases where distortion inevitably becomes apparent, for example, when the viewer's viewpoint is located around the eyebox. Therefore, in such cases, by implementing control such as reducing the sensitivity of the update of the warping parameters, it is possible to prevent discomfort caused by changes in appearance due to distortion of the virtual image, and the above control can be effectively utilized to improve visibility.

In a seventh aspect dependent on any one of the first to sixth aspects,
The head-up display device includes:
A virtual image display surface corresponding to the image display surface of the display unit is disposed so as to be superimposed on a road surface in front of the vehicle.
Or,
The virtual image display surface may be arranged at an angle to the road surface so that the distance between the near end, which is the end closest to the vehicle, and the road surface is small, and the distance between the far end, which is the end farther from the vehicle, and the road surface is large.

In the seventh aspect, in the HUD device, a virtual image display surface (corresponding to a display surface such as a screen as a display unit) located in front of the vehicle is provided so as to be superimposed on the road surface or inclined with respect to the road surface. The former is sometimes called a road surface superimposed HUD, and the latter is sometimes called an inclined surface HUD.

These can display a variety of images, for example, within a range of 5 to 100 meters ahead of the vehicle, using a wide virtual image display surface that is superimposed on the road surface, or a wide virtual image display surface that is inclined relative to the road surface. It is preferable that the HUD device and eye box are enlarged to detect the viewpoint position with high accuracy over a wider range than before, and to perform image correction using appropriate warping parameters. In this case, for example, when the viewpoint is around the eye box, frequent updates of the warping parameters may cause a decrease in the visibility of the image (virtual image), and therefore the application of the control method of the present invention is effective.

In an eighth aspect, a display control device includes:
A display unit for displaying an image;
an optical system including an optical member that reflects display light of the image and projects the image onto the projection target; and a display control device that controls a head-up display (HUD) device that projects an image onto the projection target, thereby allowing a viewer to view a virtual image of the image,
one or more processors that perform a viewpoint-tracking warping control for updating a warping parameter in accordance with a viewpoint position of a viewer in an eyebox and pre-distorting an image to be displayed on the display unit using the warping parameter so that the image has a distortion characteristic opposite to a distortion characteristic of a virtual image of the image;
When the inside of the eyebox is divided into a plurality of divided regions, each divided region is a viewing zone, and the position of the viewer's gaze point is detected in units of the viewing zone,
The processor,
When it becomes necessary to update the warping parameters due to the movement of the viewpoint, a transition period of a predetermined time is set, and a first warping parameter is set, and the value of the parameter is changed to the value of the first warping parameter;
During the transition period, the first warping parameter is maintained regardless of the movement of the viewpoint, or the frequency of parameter updates is reduced compared to a case where the warping parameter is updated every time the viewpoint position is detected in units of the viewing zone, and a second warping parameter is set little by little as a new warping parameter in response to the movement of the viewpoint.
A first change process including the control
When it becomes necessary to update the warping parameters associated with the movement of the viewpoint after the transition period has elapsed, a third warping parameter corresponding to the viewing zone after the transition period has elapsed is set, and the value of the warping parameter is changed to become the value of the third warping parameter, or is changed gradually, linearly or nonlinearly, with respect to time.
The second change process including the above control is executed.

In the eighth aspect, the same effect as the first aspect described above can be obtained. In other words, while maintaining high accuracy in detecting the viewpoint position and suppressing the sense of discomfort caused by discontinuous changes in the parameter values, it is possible to change the value of the warping parameter of the visual field after the viewpoint movement has converged within an allowable delay time. In other words, even if the viewpoint moves over a certain range, effective measures are taken, and deterioration in the display quality of the image (virtual image) displayed by the HUD device can be suppressed.

In a ninth aspect dependent on the eighth aspect,
The processor,
A first rate of change of the parameter value with respect to time in the first change process may be set to be smaller than a second rate of change of the parameter value with respect to time in the second change process.

The ninth aspect provides the same effect as the second aspect described above. In other words, it is possible to ensure (guarantee) the tracking performance of parameter changes in viewpoint-tracking warping while suppressing the decrease in visibility that accompanies discontinuous changes in warping parameter values.

In a tenth aspect dependent on the eighth or ninth aspect,
The processor,
During the transition period, when a viewpoint moves while the first change process is being executed, and the parameter value is changed to a warping parameter value corresponding to a new viewing zone in response to the viewpoint movement,
The third change process may be executed so as to result in a rate of change of the parameter value that is greater than the rate of change of the parameter value in the first change process.

According to the tenth aspect, it is possible to obtain the same effect as the third aspect described above. In other words, it is possible to reduce the amount of change in the parameter value after the transition period has elapsed, and it is possible to suppress the occurrence of discomfort, etc.

In an eleventh aspect dependent on the tenth aspect,
The processor,
When the third change process is executed,
Instead of the second change process after the transition period has elapsed, a fourth change process in which a rate of change of a parameter value is greater than the rate of change in the third change process may be executed.

In the eleventh aspect, the same effect as the fourth aspect can be obtained. In other words, instead of the second change process, a fourth change process with a larger change rate of the parameter value is executed, thereby shortening the time required for the parameter value to reach the target value and suppressing the deterioration of tracking performance. Therefore, there is no problem in that the tracking performance of the viewpoint tracking warping is deteriorated, resulting in a deterioration in the display quality of the corrected image (virtual image).

In a twelfth aspect dependent on any one of the eighth to eleventh aspects,
The processor,
When irregular viewpoint movement, a decrease in accuracy of viewpoint position detection, or a temporary loss of viewpoint position is detected during execution of the first change process within the transition period, the first change process may be stopped and the warping parameters at the time of stopping may be maintained.

In the twelfth aspect, the same effect as in the fifth aspect can be obtained. In other words, it is possible to prevent adverse effects such as the image (virtual image) becoming unstable and appearing unnatural due to the continuation of viewpoint tracking warping.

In a thirteenth aspect, a control method for a head-up display device includes:
an optical system including an optical member that reflects display light of the image and projects the image onto the projection target, and a head-up display (HUD) device that projects an image onto the projection target to allow a viewer to view a virtual image of the image, comprising:
updating the warping parameters according to a viewer's viewpoint position in the eyebox;
performing a viewpoint-position tracking warping control in which the image to be displayed on the display unit is distorted in advance using the warping parameters so that the image has a distortion characteristic opposite to that of a virtual image of the image;
When the inside of the eyebox is divided into a plurality of divided regions, each divided region is a viewing zone, and the position of the viewer's gaze point is detected in units of the viewing zone,
When it becomes necessary to update the warping parameters due to the movement of the viewpoint, a transition period of a predetermined time is set, and a first warping parameter is set, and the value of the parameter is changed to the value of the first warping parameter;
During the transition period, the first warping parameter is maintained regardless of the movement of the viewpoint, or the frequency of parameter updates is reduced compared to a case where the warping parameter is updated every time the viewpoint position is detected in units of the viewing zone, and a second warping parameter is set little by little as a new warping parameter in response to the movement of the viewpoint.
Executing a first change process including the control of
When it becomes necessary to update the warping parameters associated with the movement of the viewpoint after the transition period has elapsed, a third warping parameter corresponding to the viewing zone after the transition period has elapsed is set, and the value of the warping parameter is changed to become the value of the third warping parameter, or is changed gradually, linearly or nonlinearly, with respect to time.
Executing a second change process including the control of
including.

In the thirteenth aspect, the same effect as the first aspect described above can be obtained. In other words, while maintaining high accuracy in detecting the viewpoint position and suppressing the sense of discomfort caused by discontinuous changes in the parameter values, it is possible to change the warping parameter value of the visual field after the viewpoint movement has converged within an allowable delay time. In other words, even if viewpoint movement occurs over a certain range, effective measures are taken, and deterioration in the display quality of the image (virtual image) displayed by the HUD device can be suppressed.

In a fourteenth aspect dependent on the thirteenth aspect,
The control method for a head-up display device may further include setting a first rate of change of the parameter value with respect to time in the first change process to be smaller than a second rate of change of the parameter value with respect to time in the second change process.

The fourteenth aspect provides the same effect as the second aspect described above. In other words, it is possible to ensure (guarantee) the tracking performance of parameter changes in viewpoint tracking warping while suppressing the decrease in visibility that accompanies discontinuous changes in warping parameter values.

In a fifteenth aspect dependent on the thirteenth or fourteenth aspect,
The control method for a head-up display device includes, during the transition period, a movement of a viewpoint while the first change process is being performed, and when the parameter value is changed to a warping parameter value corresponding to a new viewing zone in response to the movement of the viewpoint,
The method may further include executing a third change process so as to result in a rate of change of the parameter value that is greater than a rate of change of the parameter value in the first change process.

In the fifteenth aspect, the same effect as the third aspect can be obtained. In other words, the amount of change in the parameter value after the transition period has elapsed can be reduced, and the occurrence of discomfort, etc. can be suppressed.

In a sixteenth aspect dependent on the fifteenth aspect,
In the control method for a head-up display device, when the third change process is executed,
The method may further include executing a fourth change process in place of the second change process after the transition period has elapsed, the fourth change process being a change process in which a rate of change of a parameter value is greater than a rate of change in the third change process.

In the sixteenth aspect, the same effect as the fourth aspect can be obtained. In other words, instead of the second change process, a fourth change process with a larger change rate of the parameter value is executed, thereby shortening the time required for the parameter value to reach the target value and suppressing the deterioration of tracking performance. Therefore, there is no problem in that the tracking performance of the viewpoint tracking warping is deteriorated, resulting in a deterioration in the display quality of the corrected image (virtual image).

In a seventeenth aspect dependent on any one of the thirteenth to sixteenth aspects,
The control method for a head-up display device may further include stopping the first change process and maintaining the warping parameters at the time when irregular viewpoint movement, a decrease in accuracy of viewpoint position detection, or a temporary loss of viewpoint position is detected while executing the first change process within the transition period.

In the seventeenth aspect, the same effect as in the fifth aspect can be obtained. In other words, it is possible to prevent adverse effects such as the image (virtual image) becoming unstable and appearing unnatural by continuing the viewpoint tracking warping.

Those skilled in the art will readily appreciate that the exemplified embodiments of the present invention may be further modified without departing from the spirit of the present invention.

FIG. 1(A) is a diagram for explaining an overview of the warping process and the manner in which a virtual image (and a virtual image display surface) displayed after the warping process is distorted, and FIG. 1(B) is a diagram showing an example of a virtual image viewed by a viewer through a windshield. FIG. 2A is a diagram for explaining an overview of the viewpoint position tracking warping process, and FIG. 2B is a diagram showing an example of the configuration of an eyebox the inside of which is divided into a plurality of viewing zones. 3A and 3B are diagrams showing how the values of the warping parameters change linearly or nonlinearly. FIG. 4(A) is a diagram showing an example of viewpoint movement in the eyebox (an example in which the viewpoint moves to a viewing zone separated by one viewing zone), and FIG. 4(B) is a diagram showing an example of control for gradually changing parameter values when switching warping parameters (an example of a first change process and a second change process). Figure 5(A) shows another example of viewpoint movement in the eyebox (an example in which the viewpoint moves to a viewing zone separated by two viewing zones), and Figure 5(B) shows another example of control for gradually changing parameter values when switching warping parameters (an example in which a first change process, a third change process, and a fourth change process are executed). FIG. 6(A) shows yet another example of viewpoint movement in the eyebox (an example in which the viewpoint moves irregularly), and FIG. 6(B) shows yet another example of control for gradually changing parameter values when switching warping parameters (an example in which the first change process, etc. is stopped). FIG. 7 is a diagram illustrating an example of a system configuration of a HUD device. FIG. 8 is a flowchart showing an example of a procedure for image correction processing by warping. FIG. 9(A) is a diagram showing an example of display by a road surface superimposed HUD, FIG. 9(B) is a diagram showing an example of display by an inclined surface HUD, and FIG. 9(C) is a diagram showing an example of the configuration of the main parts of a HUD device.

The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should note that the present invention is not unduly limited by the mode described below.

Please refer to Figure 1. Figure 1(A) is a diagram for explaining an overview of the warping process and the manner in which the virtual image (and the virtual image display surface) displayed after the warping process is distorted, and Figure 1(B) is a diagram showing an example of a virtual image viewed by a viewer through a windshield.

As shown in FIG. 1A, the HUD device 100 has a display unit (e.g., a light-transmitting screen) 101, a reflecting mirror 103, and a curved mirror (e.g., a concave mirror, the reflecting surface of which may be a free-form surface) 105 as an optical member that projects display light. An image displayed on the display surface 102 of the display unit 101 (such as a liquid crystal panel) 101 is projected onto the windshield 2 as a member to be projected via the reflecting mirror 103 and the curved mirror 105. In FIG. 1, the reference numeral 4 indicates a projection area. Note that the HUD device 100 may have multiple curved mirrors, and may include a refractive optical element such as a lens, a diffractive optical element, or other functional optical element in addition to or instead of a part (or all) of the mirror (reflective optical element) of this embodiment.

A portion of the image display light is reflected by the windshield 2 and enters the viewpoint (eye) A of the viewer or the like located inside (or on) a pre-set eye box EB (here, a rectangular shape of a specified area), and is focused in front of the vehicle 1, whereby a virtual image V is displayed on a virtual virtual image display surface PS corresponding to the display surface 102 of the display unit 101.

The image on the display unit 101 is distorted due to the influence of the shape of the curved mirror 105, the shape of the windshield 2, and the like. To offset this distortion, a distortion with the opposite characteristics to the distortion is applied to the image. This predistortion type image correction is referred to in this specification as warping processing, or warping image correction (or warping correction).

In the upper left of Figure 1(A), PS' indicated by a dashed line indicates the virtual image display surface without warping processing, and V' indicates the virtual image displayed on that virtual image display surface PS'.

Figure 1(B) shows an example of a virtual image V viewed by a viewer through the windshield 2. In Figure 1(B), the virtual image V has a rectangular outer shape, and is provided with a total of 25 reference points (reference pixel points) G(i,j) (where i and j are both variables that can take values from 1 to 5), for example, 5 vertically and 5 horizontally. A warping process is used to provide each reference point in the image (original image) with a distortion that has the opposite characteristics to the distortion that occurs in the virtual image V, and therefore the distortion provided in advance and the distortion that actually occurs are offset, and a virtual image V without any curvature as shown in Figure 1(B) is displayed. The number of reference points G(i,j) can be increased as appropriate by an interpolation process or the like. In Figure 1(B), reference number 7 indicates a steering wheel.

Next, let us refer to Figure 2. Figure 2(A) is a diagram for explaining an overview of the viewpoint position tracking warping process, and Figure 2(B) is a diagram showing an example of the configuration of an eyebox whose interior is divided into multiple viewing zones. In Figure 2, parts that are common to Figure 1 are given the same reference numerals (this also applies to the following figures).

As shown in FIG. 2(A), the eye box EB is divided into multiple (here, nine) viewing zones Z1 to Z9, and the position of the viewer's viewpoint A is detected for each of the viewing zones Z1 to Z9. Note that the eye box EB has a three-dimensional structure, but is shown in a two-dimensional form for the sake of explanation.

Image display light K is emitted from the projection optical system 118 of the HUD device 100, a portion of which is reflected by the windshield 2 and enters the viewer's viewpoint (eye) A. When viewpoint A is within the eye box EB, the viewer can see a virtual image of the image.

The HUD device 100 has a ROM 210 (see FIG. 7), which has an image conversion table 212 built in. The image conversion table 212 stores warping parameters WP that determine polynomials, multipliers, constants, etc. for image correction (warping image correction) using a digital filter, for example. The warping parameters WP are provided corresponding to each of the viewing zones Z1 to Z9 in the eye box EB. In FIG. 2(A), the warping parameters WP(Z1) to WP(Z9) corresponding to each viewing zone are shown. Note that in the figure, only the symbols WP(Z1), WP(Z4), and WP(Z7) are shown.

When viewpoint A moves, the position of viewpoint A among the multiple viewing zones Z1 to Z9 is detected. Then, one of the warping parameters WP(Z1) to WP(Z9) corresponding to the detected viewing zone is read from ROM 210 (updating the warping parameters), and warping processing is performed using the warping parameters.

Figure 2 (B) shows an eyebox EB with more viewing zones than the example in Figure 2 (A). The eyebox EB is divided into 6 viewing zones vertically and 10 horizontally, for a total of 60 viewing zones. Each viewing zone is displayed as Z(X,Y), with the coordinate positions in the X and Y directions as parameters. Each divided region is called a viewing zone. Each viewing zone has a rectangular outer shape. One warping parameter corresponds to each viewing zone, and viewpoint movement is detected (determined) for each viewing zone (one rectangular section).

In the example of FIG. 2(B), the 12 viewing zones (white areas) within the bold rectangle C are the central area (central region) ZA, and the area outside of that (shown with diagonal lines) is the peripheral area (peripheral region) ZB, and the two are recognized (detected) separately for the purpose of adjusting the sensitivity of the warping process according to the present invention.

Next, refer to FIG. 3. FIGS. 3(A) and (B) are diagrams showing linear and nonlinear changes in the value of the warping parameter. In the example of FIGS. 3(A) and (B), a switch (update) is made from the warping parameter wp12 before the movement to the warping parameter wp13 after the movement.

In other words, the warping parameters are updated from wp12 to wp13 according to the viewer's viewpoint position in the eyebox, and viewpoint-position tracking warping control is performed in which the warping parameters are used to pre-distort the image to be displayed on the display surface (reference number 102 in FIG. 1A) of the display unit (reference number 101 in FIG. 1A) so that the image has the opposite distortion characteristics to the virtual image of the image. This control is executed (performed) by the warping control unit (sometimes simply referred to as the control unit) 180 in FIG. 7 (details will be described later).

When switching from warping parameter wp12 to warping parameter wp13, the control unit 180 controls the parameter value so that it gradually changes linearly or nonlinearly with respect to time (in other words, on the time axis), as shown by characteristic lines Q1 and Q2 in FIG. 3(A).

In FIG. 3(A), the warping parameter switching occurs at time t1. The linear change is indicated by dashed characteristic line Q1, and the nonlinear change is indicated by dashed characteristic line Q2. Characteristic line Q2 has a characteristic that the change is steep in the initial period of warping parameter switching and the change becomes gradual in the latter period. The initial steep change prevents the switching from becoming slow, and the latter gradual change achieves a smooth transition from wp12 to wp13, reducing the visual discomfort at the time of transition to wp13.

By changing the warping parameter value linearly or nonlinearly, the warping parameter value changes gradually over time, and reaches the parameter value of wp13 a predetermined time after the start of the change. Therefore, compared to a sudden (step-like) change at time t1, the appearance of the displayed image (virtual image) does not change abruptly, reducing the visual discomfort felt by the viewer (driver, etc.).

Also, in Figure 3(B), another example of a nonlinear change in the warping parameter value is shown by the dashed characteristic lines Q3 and Q4. Characteristic line Q3 has a characteristic in which the change is gradual in the initial period of switching the warping parameter, and the change becomes steeper in the latter period. The steep change in the latter period prevents the switching from becoming slow, and the gradual change in the first period reduces the visual discomfort when the switching begins.

The change in characteristic line Q4 in the first half is similar to that in characteristic line Q3 (in other words, it is gradual at first and then steep), and the change in the second half is similar to that in characteristic line Q2 shown above (in other words, it is steep at first and then gradual). By making the change gradual at the beginning, the visual discomfort is reduced, and by becoming steep, the delay in the change is reduced, and then the steep change is continued, suppressing the delay, and by making the change gradual towards the end of the change, the visual discomfort is reduced.

Next, refer to FIG. 4. FIG. 4(A) shows an example of viewpoint movement in the eyebox (an example in which the viewpoint moves to a viewing zone separated by one viewing zone), and FIG. 4(B) shows an example of control for gradually changing parameter values when switching warping parameters (examples of a first change process and a second change process). In FIG. 4, the notations viewing zone (wp13) and viewing zone (wp14) are used, which indicate that these are viewing zones corresponding to warping parameters wp13 and wp14 (the same applies to FIG. 5 and FIG. 6 described below).

In FIG. 4(A), viewpoint A moves from a viewing zone corresponding to warping parameter wp12 in eyebox EB to a viewing zone corresponding to warping parameter wp14. Accordingly, the warping parameters are updated. When updating the warping parameters, as described above in FIG. 3, the parameter values change linearly or nonlinearly with respect to time.

When it is detected that viewpoint A has moved from the viewing zone corresponding to warping parameter wp12 to the viewing zone corresponding to wp13 (in other words, the viewing zone adjacent to the viewing zone before the viewpoint moved) (in other words, when the first detection of the viewpoint movement is made), a new warping parameter wp13 (referred to as the first warping parameter) is set.

In the example of FIG. 4, a transition period (simply called the transition period) Ta of the viewpoint movement is set at the same time for a predetermined time. Here, the length of the transition period (predetermined time) is set to, for example, about 5 seconds. However, this is just an example, and the time can be shortened or extended as appropriate.

Before the transition period Ta has elapsed, the first change process (the parameter value change process shown by the characteristic line Q10 in FIG. 4(B)) is executed, and after the transition period Ta has elapsed, the second change process (the parameter value change process shown by the characteristic line Q20 in FIG. 4(B)) is executed.

In the example of FIG. 4(B), the changes shown by characteristic lines Q10 and Q20 are linear, but they may be non-linear. Also, in FIG. 4(B), the rate of change of characteristic line Q20 is greater than the rate of change of characteristic line Q10, but this is just one example, and the opposite may also be true, or the rates may be the same. The rate of change of each characteristic line can be set separately and independently, ensuring freedom of design.

In the first change process, during the transition period Ta, the first warping parameter is maintained regardless of the movement of the viewpoint, or, compared to updating the warping parameter each time the viewpoint position is detected in units of the viewing zone, the frequency of parameter updates is reduced and the second warping parameter is gradually set as a new warping parameter in response to the movement of the viewpoint.

While keeping the detection sensitivity of the viewpoint position high by setting the viewing zone finely, during the transition period, the first parameter is maintained or, compared to updating the parameters for each viewing zone, the number of parameter updates is reduced to avoid frequent parameter updates (however, it is acceptable to set the same (or the same) parameter update frequency), and the second parameter is gradually set in response to the movement of the viewing zone. This prevents the viewer from feeling uncomfortable. Here, gradually setting the second parameter can be seen as a process (tracking process) in which the parameter value is changed to follow the current situation little by little while watching the state of the viewing zone movement (in other words, while observing the progress of the viewing zone movement), and is brought closer to the value of the third parameter corresponding to the viewing zone detected after the transition period has elapsed. In addition, the first warping parameter may be maintained regardless of the viewing zone movement, but in this case, the parameter value also changes over time to become the value of the first warping parameter. Because the first warping parameter is a warping parameter updated for the first viewpoint movement, the change in the parameter value is likely to follow subsequent viewpoint movements, albeit slowly, and therefore may be included in the above-mentioned tracking process.

In the second change process, when it becomes necessary to update the warping parameters associated with the viewpoint movement after the transition period Ta has elapsed, a third warping parameter (wp14 in FIG. 4B, wp15 in FIG. 5B described later) corresponding to the viewing zone after the transition period Ta has elapsed (the viewing zone detected after the transition period has elapsed) is set, and the parameter value is changed. This change in the parameter value may be intended to be a quick change (in other words, a steep change), or may be a gradual change over time. In other words, the second change process includes both control for implementing a quick change (including a steep change) and control for implementing a gentle change that gradually changes over time. These controls can be used appropriately or in combination. This second change process makes it possible for the parameter value to arrive at (reach) the value of the third parameter within a realistically acceptable time range.

In FIG. 4(B), the first change process is shown by characteristic line Q10. In FIG. 4(B), a transition period (a predetermined time) elapses at time t1. At this point, it is detected that viewpoint A has moved into the viewing zone (wp14). From time t1 to time t2, the second change process (shown by characteristic line Q20) is executed. After time t2, there is no movement of viewpoint A, so the parameter value of warping parameter wp14 is maintained (processing shown by characteristic line Q30).

In this way, in the example of FIG. 4, the viewing zone is set finely to maintain high detection sensitivity for the viewpoint position, while the update frequency of the warping parameters is reduced compared to when the update is performed for each viewing zone where the viewpoint moves, suppressing frequent updates and causing gradual changes in the parameter values. The value of the warping parameters is changed little by little to correspond to the direction of the viewpoint movement without giving the viewer a sense of discomfort. While observing the state of the viewpoint movement (in other words, while observing the progress of the viewpoint movement), the parameter value is changed to gradually follow the current situation, and a process (tracking process) is performed to bring the parameter value closer to the value of the third parameter (wp14) corresponding to the viewing zone after the transition period Ta has elapsed (viewing zone (wp14)).

After the transition period Ta has elapsed, the next viewing zone (viewing zone (wp14)) is detected at time t1, the third warping parameter wp14 is set, and the second change process is performed as shown by the characteristic line Q20.

During the first change process, the parameter value is brought closer to the value of the third warping parameter (wp14), and in the second change process, a quick (steep) change in the parameter value is also allowed if necessary. Therefore, in the second change process, it is possible for the parameter value to reach (reach) the value of the third parameter (wp14) within a realistically allowable time range (within the period from time t1 to t2 in FIG. 4(B)).

This makes it possible to change the warping parameter value of the visual field after the viewpoint movement has converged within an acceptable delay time while maintaining high accuracy in detecting the viewpoint position and suppressing the sense of discomfort caused by discontinuous changes in the parameter value. In other words, even if the viewpoint moves over a certain range, effective measures are taken, and degradation in the display quality of the image (virtual image V) displayed by the HUD device 100 can be suppressed.

As described above, the characteristics of the changes in the first change process and the second change process can basically be set separately and independently. However, they can also be linked or related. For example, the parameter change can be designed flexibly as a linear change, and the change rates in the first change process and the second change process can be large and small, small and large, or the same, or the changes can be nonlinear. This makes it possible, for example, to fine-tune the performance of the viewpoint position tracking warping, or to change the characteristics to suit the current situation. This contributes to improving the performance of the HUD device 100.

Also, as shown in FIG. 4(B), the first rate of change of the parameter value with respect to time in the first change process (the slope of characteristic line Q10) may be set to be smaller than the second rate of change of the parameter value with respect to time in the second change process (the slope of characteristic line Q20).

As a preferred example, in the first change process during the transition period Ta, from the viewpoint of observing the movement of the viewpoint position, or from the viewpoint of preventing the viewer from feeling uncomfortable during the movement of the viewpoint, it is preferable to suppress the rate of change of the parameter value over time to a certain extent and change it little by little. In other words, during the transition period Ta, the parameter value is not changed too suddenly. Therefore, the rate of change of the parameter value during the first change process (first rate of change: the slope of the characteristic line Q10) tends to be set relatively small.

On the other hand, after the transition period Ta has elapsed and the viewpoint movement has converged, it is preferable to ensure (guarantee) the tracking performance of the parameter change in the viewpoint tracking warping by making the parameter value reach the value of the parameter (third warping parameter) corresponding to the detected viewing zone relatively quickly. Therefore, in this case, it is preferable to set the rate of change of the parameter value in the second change process (second rate of change: the slope of the characteristic line Q20) to be greater than the first rate of change.

This makes it possible to ensure (guarantee) the tracking performance of parameter changes in viewpoint-tracking warping, while suppressing the decrease in visibility that accompanies discontinuous changes in warping parameter values. In other words, even if a large and irregular movement of the viewpoint occurs, it is possible to display a highly visible image with little distortion while suppressing unnatural behavior (such as jerky discontinuous changes or sudden changes) in the virtual image V displayed by the HUD device 100. This prevents a decrease in the recognizability of the information provided to the viewer by the HUD device 100.

Next, refer to FIG. 5. FIG. 5(A) shows another example of viewpoint movement in the eyebox (an example in which the viewpoint moves to a viewing zone separated by two viewing zones), and FIG. 5(B) shows another example of control that gradually changes parameter values when switching warping parameters (an example in which a first change process, a third change process, and a fourth change process are executed).

In the example of FIG. 5, during the transition period Ta, a third change process (shown by characteristic line Q22) is performed as necessary, in which the rate of change of the parameter value is set to be greater than that of the first change process (shown by characteristic line Q21).

As described above, during the transition period Ta, the rate of change of the parameter value is made slightly smaller to suppress discontinuous changes, and control is performed to bring the parameter value closer to the parameter value of the visual field that will be detected after the transition period Ta has elapsed.

However, with this control alone, the change in the parameter value after the transition period Ta may be quite large, which may be likely to cause discomfort to the viewer.

For example, during the transitional period Ta, the viewpoint A moves through multiple viewing zones (in FIG. 5A, viewing zone (wp12) to viewing zone (wp(15)), and the latest viewing zone detected during the transitional period Ta (viewing zone (wp14) in FIG. 5A and FIG. 5B)) is located away from the viewing zone at the time when the transitional period Ta was set (original viewing zone: viewing zone (wp12) in FIG. 5A and FIG. 5B), and there is a large difference in the corresponding parameter values. Therefore, taking into account the amount of change in the parameter value from the latest viewing zone to the viewing zone immediately before it (change amount ΔS1 in FIG. 5B), it is determined (estimated) that the amount of change in the difference from the parameter value of the latest viewing zone (viewing zone (wp14)) (change amount ΔS2 in FIG. 5B) will be significantly large. Note that this determination is made by the control unit 180 (see FIG. 7), for example, using a predetermined threshold value.

At this time, if the rate of change (slope) of characteristic line Q22 were the rate of change (slope) of the normal first change process, the change would be slow and it would be difficult to reduce the large difference ΔS2 described above. In this case, when the warping parameters are updated after the transition period Ta has elapsed, a large change in the parameter value would occur, which is likely to cause discomfort to the viewer.

Therefore, in the example of FIG. 5(B), the slope of the characteristic line Q22 is set steeper than in the first change process, and the third change process is executed.

In FIG. 5(B), the rate of change (slope) of characteristic line Q22 is set to about twice the rate of change (slope) in the first change process. In other words, before the end of the transition period Ta, the change in the parameter value is increased more than in the first change process, thereby improving the tracking ability of the parameter value.

By doing this, when the transitional period Ta expires, the large difference ΔS2 is reduced to ΔS3. In other words, the parameter value is closer to the value of parameter wp15 corresponding to the viewing zone (viewing zone (wp15)) during the transitional period Ta. Therefore, the amount of change in the parameter value after the transitional period Ta has elapsed (ΔS4 in FIG. 5B) is smaller than when only the first change process is executed during the transitional period Ta. This makes it possible to suppress the occurrence of discomfort, etc.

However, the amount of change in the parameter value ΔS4 in FIG. 5(B) is not so small. Therefore, in the example of FIG. 5(B), after the transition period Ta has elapsed, instead of the normal second change process, a fourth change process (shown by characteristic line Q23) is executed, which has a larger rate of change in the parameter value than the second change process. The slope of characteristic line Q23 is steeper (e.g., twice the rate of change) than the normal second change process. Therefore, the time it takes for the parameter wp15 to reach the parameter value after the transition period Ta has elapsed is the time from time t13 to t14, which is significantly shortened. In other words, the delay in the parameter tracking to the viewpoint falls within an acceptable range.

In this way, the fourth change process is executed instead of the second change process, shortening the time it takes for the parameter value to reach the target value and suppressing the degradation of tracking performance. Therefore, there is no particular problem with the tracking performance of viewpoint tracking warping deteriorating and reducing the effectiveness of preventing degradation in image quality.

Next, refer to FIG. 6. FIG. 6(A) shows yet another example of viewpoint movement in the eyebox (an example in which the viewpoint moves irregularly), and FIG. 6(B) shows yet another example of control that gradually changes parameter values when warping parameters are switched (an example in which the first change process, etc., is stopped).

In the example of Figure 6, if irregular viewpoint movement, a decrease in the accuracy of viewpoint position detection, or a temporary loss of viewpoint position is detected during the first change process during the transition period Ta, the first change process is stopped and control is executed to maintain the warping parameters at the time of stopping.

As shown in FIG. 6(A), viewpoint A moves quite irregularly. In FIG. 6(B), irregular viewpoint movement is detected at time t21. From this point on, the first change process (parameter update process based on a predetermined change rate) or the third change process is stopped until time t22 when the transition period Ta ends, and the warping parameter at the time of stopping (wp13 in FIG. 6(B)) is maintained.

The parameter value is maintained at the parameter value at the time of stopping for the maintained warping parameter wp13, for example. Note that it is also possible to select (adopt) an appropriate other value (e.g., an intermediate value of the warping parameter) instead of the parameter value at the time of stopping, and maintain the parameter value at that value.

This prevents continued viewpoint-tracking warping from causing problems such as the image (virtual image) becoming unstable and appearing unnatural.

In FIG. 6(B), the transition period Ta ends at time t22, and the viewing zone (wp14) is detected. This sets the warping parameter WP14, and the second or fourth change process is performed from time t22 to time t23. After time t23, there is no movement of the viewpoint A, so the parameter value is maintained at the value of wp14.

Next, reference is made to FIG. 7. FIG. 7 is a diagram showing an example of the system configuration of a HUD device. A viewpoint detection camera 110 is provided in the vehicle 1 (or the HUD device 100 itself) to detect the position of the viewer's viewpoint A (eyes, pupils). The vehicle 1 is also provided with an operation unit 130 to enable the viewer to set necessary information for the HUD device 100, and a vehicle ECU 140 capable of collecting various information about the vehicle 1.

The HUD device 100 also has a light source 112, a light projecting unit 114, a screen 116 (having a display surface 117) as a display unit, a projection optical system 118, a viewpoint position detection unit (viewpoint position determination unit) 120, a bus 150, a display control unit (display control device) 160, a ROM 210 incorporating an image conversion table 212, a VRAM 220 that stores image (original image) data 222 and temporarily stores image data 224 after warping processing, and an image output unit 230.

The viewpoint position detection unit 120 can also be provided separately from the HUD device 100. The viewpoint position detection unit 120 has a viewpoint coordinate detection unit 122 and a viewing zone detection unit 124 in the eye box.

The display control unit (display control device) 160 has an I/O interface 170, a warping control unit (sometimes simply called a control unit) 180 that implements warping control as shown in Figures 3 to 9, an image generation unit 192, and a warping processing unit 194. The warping control unit 180 has a viewpoint position information setting unit 182, a warping parameter change control unit 184 (having an elapsed time management unit 183), and a timer 185. The elapsed time management unit 183 uses the count of the timer 185 to detect the end of a transitional period, which is a predetermined time, etc.

The basic operation is as follows. Based on the viewing zone information (information indicating which viewing zone of the eyebox viewpoint A is in) sent from the viewing zone detection unit 120, the viewing zone information setting unit 182 of the warping control unit 180 sets the viewing zone before and after the viewing zone movement. The warping parameter change control unit 184, while referring to various information from the vehicle ECU 140 etc., determines the manner of change over time of the warping parameter value (rate of change, type of characteristic line indicating change, etc.) as previously shown in Figures 4(B), 5(B), and 6(B).

The warping parameter change control unit 184
(1) Warping parameters WP(Z1) to WP(Z9) corresponding to each viewing zone and one or more warping parameters that interpolate between them are stored in advance in the ROM 210, and the image is gradually changed by sequentially reading out these warping parameters.
(2) By using a linear or nonlinear characteristic line (calculation formula), one or more warping parameters that interpolate between the warping parameters WP(Z1) to WP(Z9) corresponding to each viewing zone are sequentially generated to gradually change the image;
(3) Execute various operations related to correcting the above processes (1) and (2) while referring to various information from the vehicle ECU 140, etc.
That is, the warping parameter change control unit 184 may include various software components such as commands, judgment values, table data, and arithmetic expressions for determining the manner of change of the warping parameter value over time (rate of change, type of characteristic line indicating change, etc.) based on viewing zone information sent from the viewpoint position detection unit 120 (information indicating which viewing zone of the eyebox viewpoint A is in).

The image generation unit 192 accesses the VRAM 220 to read the image data, generates an image (original image), and supplies it to the warping processing unit 194. The warping processing unit 194 accesses the ROM 210, reads the corresponding warping parameters from the image conversion table (see FIG. 2) 212, and performs warping processing on the image data (original image data) using the read warping parameters. The data after warping processing is temporarily returned to the image generation unit 192 and temporarily stored in the VRAM 220 as appropriate, and is supplied to the image output unit 230 at a predetermined timing. An image in a predetermined format is generated in the image output unit 230, and the image is supplied to, for example, the light source 112 or the light projector 114.

Next, refer to FIG. 8. FIG. 8 is a flowchart showing an example of the procedure for image correction processing by warping. In step S1, the viewpoint is monitored, and in step S2, it is determined whether or not there is an initial viewpoint movement (initial detection of the viewing zone of the viewpoint movement destination). If N, return to step S1, and if Y, proceed to step S3.

In step S3, a warping parameter (first warping parameter) corresponding to the destination viewing zone is set. At the same time, measurement of a transition period (a predetermined time) is started. The parameter value may change gradually with time, either linearly or nonlinearly, or may change quickly.

In step S4, it is determined whether there is further viewpoint movement. If the answer is N, step S4 continues, and if the answer is Y, the process proceeds to step S5. In step S5, it is determined whether the process is within a transition period (predetermined time). If the answer is N, the process proceeds to step S10 (the details of step S10 will be described later). If the answer is Y, the process proceeds to step S6.

In step S6, a first change process is executed. In the first change process, the first warping parameter may be maintained regardless of the movement of the viewpoint, or the second warping parameter may be set in response to the movement of the viewpoint less frequently than in the case of updating for each viewing zone, or at the same frequency. However, in order to improve tracking performance, a third change process may be executed in which the rate of change of the parameter value is increased.

In step S7, it is determined whether or not irregular movement of the viewpoint, or a decrease in the accuracy of the viewpoint position detection, or a temporary loss of the viewpoint position has been detected. If N, the process returns to step S4. If Y, the process proceeds to step S8. In step S8, the first or third change process is stopped, and the current warping parameters (in other words, the values at the time of stopping) are maintained. The parameter values are maintained, for example, at the values at the time of stopping.

In step S9, it is determined whether a transition period (a predetermined time) has elapsed. If N, step S9 continues, and if Y, proceed to step S10. In step S10, a second change process is executed. In the second change process, a third warping parameter corresponding to the visual field detected after the transition period has elapsed is set. The third warping parameter may be switched quickly. The parameter value may change quickly (abruptly), or may change gradually, linearly, or nonlinearly over time. In addition, when the difference between the current parameter value and the target parameter value is large, a fourth change process with a larger rate of change in the parameter value may be executed instead of the second change process.

Next, let us refer to Figure 9. Figure 9(A) shows an example of a display using a road surface overlay HUD, Figure 9(B) shows an example of a display using an inclined surface HUD, and Figure 9(C) shows an example of the configuration of the main parts of a HUD device.

Figure 9 (A) shows an example of virtual image display using a road surface overlay HUD (on-road HUD) in which a virtual virtual image display surface PS corresponding to the image display surface (reference number 102 in Figure 1 (A) or reference number 163 in Figure 9 (C)) of a display unit (reference number 101 in Figure 1 (A) or reference number 161 in Figure 9 (C)) is positioned so as to be overlaid on the road surface 41 in front of the vehicle 1.

Figure 9 (B) shows an example of virtual image display using an inclined surface HUD (inclined image surface HUD) in which the virtual image display surface PS is inclined, for example, at an angle θ (0 < θ ≦ 90°) with respect to the road surface 41 so that the distance between the near end, which is the end of the virtual image display surface PS closer to the vehicle 1, and the road surface 41 is small, and the distance between the far end, which is the end farther from the vehicle 1, and the road surface 41 is large.

These devices use a wide virtual image display surface PS that is superimposed on the road surface 41, or a wide virtual image display surface PS that is inclined relative to the road surface 41, and can perform various displays, for example, within a range of 5 to 100 m ahead of the vehicle 1. It is preferable that the HUD device and eye box EB are made larger, and the viewpoint position is detected with high accuracy over a wider range than before, and image correction is performed using appropriate warping parameters. In this case, for example, the viewpoint A moves more frequently, and visibility of the image (virtual image V) may decrease due to frequent updates of the warping parameters, and therefore the application of the control method of the present invention is effective.

Next, refer to FIG. 9(C). The HUD device 107 in FIG. 9(C) has a control unit 171, a light projecting unit 151, a screen 161 as a display unit having an image display surface 163, a reflecting mirror 133, a curved mirror (concave mirror, etc.) 131 whose reflecting surface is designed as a free-form surface, and an actuator 173 that drives the display unit 161.

In the example of FIG. 9(C), when the HUD device 107 adjusts the position of the eye box EB in accordance with the height position of the viewer's viewpoint A, it does not move the curved mirror 131, which is an optical member that projects the display light onto the windshield 2 (an actuator for the curved mirror 131 is not provided), but instead changes the reflection position of the display light 51 of the image on the optical member 131.

In other words, when adjusting the height position of the eye box EB according to the height position of the viewer's eyes (viewpoint A), the optical member 131 that projects light onto the projection target 2 is not rotated, for example, by an actuator, but is instead adjusted by changing the reflection position of the light on the optical member 131. Note that the height direction is the Y direction in the figure (the direction along the perpendicular to the road surface 41, and the direction away from the road surface 41 is the positive direction). Note that the X direction is the left-right direction of the vehicle 1, and the Z direction is the front-rear direction (or forward direction) of the vehicle 1.

Recently, HUD devices have tended to be developed on the premise that virtual images will be displayed over a fairly wide area in front of the vehicle, for example, which inevitably results in larger devices. Naturally, the optical member 131 also becomes larger. If this optical member 131 is rotated using an actuator or the like, the resulting error may actually reduce the accuracy of control of the height position of the eye box EB. To prevent this, the position at which the light rays are reflected by the reflective surface of the optical member (curved mirror, specifically, for example, a concave mirror) 131 is changed.

In such a large optical element 131, the reflective surface is optimally designed as a free-form surface to prevent distortion of the virtual image as much as possible. However, there are cases where distortion inevitably becomes apparent, for example, when the viewer's viewpoint A is located near the eye box EB.

In such a case, it is effective to control (adjust) the parameter change so that the change in parameter value accompanying the update of the warping parameters described above is made gradual to reduce the sense of discomfort, and the parameter value is gradually changed by performing fine viewpoint position detection in the viewpoint movement direction, ultimately arriving at (reaching) the parameter at the center of the viewing zone after a predetermined time (after the transition period has elapsed). In other words, the visibility of the virtual image V can be improved by effectively utilizing the above control.

As described above, according to the present invention, when implementing viewpoint-position tracking warping control that updates warping parameters in accordance with the viewpoint position of the viewer, it is possible to effectively prevent discontinuous changes in the appearance of the image that accompany the updating of the warping parameters, causing discomfort to the viewer.

The present invention can be used with both monocular HUD devices, in which display light for the same image is incident on each of the left and right eyes, and parallax HUD devices, in which images with parallax are incident on each of the left and right eyes.

In this specification, the term "vehicle" can be broadly interpreted as a vehicle. Navigation terms (such as signs) are also broadly interpreted, taking into account the perspective of navigation information in the broad sense that is useful for driving a vehicle. HUD devices are also intended to include those used as simulators (such as aircraft simulators, simulators as game devices, etc.).

The present invention is not limited to the exemplary embodiments described above, and a person skilled in the art could easily modify the exemplary embodiments described above to the extent that they fall within the scope of the claims.

1: vehicle (own vehicle), 2: projection target member (reflective translucent member, windshield, etc.), 4: projection area, 7: steering wheel, 100: HUD device, 110: viewpoint detection camera, 112: light source, 114: light projection unit, 101: display unit, 102: display surface (image display surface), 118: projection optical system, 120: viewpoint position detection unit (viewpoint position determination unit), 122: viewpoint coordinate detection unit, 124: visual zone detection unit in eye box, 130: operation unit, 131: curved mirror (concave mirror, etc.), 133: reflector (including reflector, correction mirror, etc.), 140: vehicle ECU, 150: bus, 151: light source unit, 116: display unit (screen, etc.), 117: display surface (image image display surface), 160: display control unit, 170: I/O interface, 180: display control unit (display control device: includes one or more processors), 182: viewpoint position information setting unit, 183: elapsed time management unit, 184: warping parameter change control unit, 185: timer, 192: image generation unit, 194: warping processing unit, 210: ROM, 212: image conversion table, 220: VRAM, 222: image (original image) data, 224: image data after warping processing, 230: image output unit, EB: eyebox, Z (Z1 to Z9, etc.)...visible range (divided area) of eyebox, WP: warping parameter, PS: virtual image display surface, V: virtual image.

Claims (14)

A head-up display (HUD) device that is mounted on a vehicle and projects an image onto a projection member, thereby allowing a viewer to view a virtual image of the image,
an image generating unit that generates the image;
A display unit for displaying the image;
an optical system including an optical member that reflects display light of the image and projects the image onto the projection target;
a control unit that performs a viewpoint-position tracking warping control in which a parameter value is updated according to a viewpoint position of a viewer in an eyebox, and an image to be displayed on the display unit is distorted in advance using the parameter value so that the image has a distortion characteristic opposite to a distortion characteristic of a virtual image of the image;
having
When the inside of the eyebox is divided into a plurality of divided regions, each divided region is a viewing zone, and the position of the viewer's gaze point is detected in units of the viewing zone,
The control unit is
When it becomes necessary to update the parameter value due to the movement of the viewpoint, a measurement of a transition period of a predetermined time is started , and a first warping parameter corresponding to the viewing zone in which the position of the viewer's viewpoint is detected is set, and the parameter value is changed to the value of the first warping parameter;
During the transition period, the first warping parameter is maintained regardless of a movement of a viewpoint, or a new second warping parameter is set in response to the movement of a viewpoint , and the parameter value is changed to a value of the second warping parameter.
A first change process including the control
When it becomes necessary to update the parameter value accompanying the movement of the viewpoint after the transition period has elapsed, a third warping parameter corresponding to the viewing zone after the transition period has elapsed is set, and the parameter value is changed to become the value of the third warping parameter, or is changed gradually linearly or nonlinearly with respect to time.
A second change process including the control is executed ,
a first rate of change of the parameter value with respect to time in the first change process is set to be smaller than a second rate of change of the parameter value with respect to time in the second change process;
A head-up display device. The control unit is
During the transition period, when a viewpoint moves while the first change process is being executed, and the parameter value is changed to the value of the second warping parameter corresponding to a new viewing zone in response to the viewpoint movement,
executing a third change process so that the rate of change of the parameter value with respect to time in the first change process is greater than the first rate of change of the parameter value with respect to time in the first change process;
The head-up display device according to claim 1 . The control unit is
When the third change process is executed,
executing a fourth change process in place of the second change process after the transition period has elapsed, the fourth change process being such that a rate of change of the parameter value is greater than the rate of change in the third change process;
The head-up display device according to claim 2 . The control unit is
when irregular movement of the viewpoint, a decrease in accuracy of detecting the viewpoint position, or a temporary loss of the viewpoint position is detected during execution of the first change process within the transition period, the first change process is stopped, and the parameter value at the time of stopping is maintained.
The head-up display device according to any one of claims 1 to 3 . The head-up display device includes:
When adjusting the position of an eye box in accordance with the height position of the viewpoint of the viewer, the optical member is not moved, and a reflection position of the display light of the image on the optical member is changed.
5. The head-up display device according to claim 1, wherein the head-up display device is a head-up display. The head-up display device includes:
A virtual image display surface corresponding to the image display surface of the display unit is disposed so as to be superimposed on a road surface in front of the vehicle.
Or,
the virtual image display surface is disposed at an oblique angle with respect to the road surface such that a distance between a near end portion, which is an end portion close to the vehicle, and the road surface is small, and a distance between a far end portion, which is an end portion farther from the vehicle, and the road surface is large.
6. The head-up display device according to claim 1, A display unit for displaying an image;
an optical system including an optical member that reflects display light of the image and projects the image onto a projection target; and a display control device that controls a head-up display (HUD) device that allows a viewer to view a virtual image of the image by projecting the image onto the projection target,
one or more processors that perform a viewpoint-tracking warping control that updates parameter values according to a viewpoint position of a viewer in an eyebox and uses the parameter values to pre-distort an image to be displayed on the display unit so that the image has a distortion characteristic opposite to a distortion characteristic of a virtual image of the image;
When the inside of the eyebox is divided into a plurality of divided regions, each divided region is a viewing zone, and the position of the viewer's gaze point is detected in units of the viewing zone,
The processor,
When it becomes necessary to update the parameter value due to the movement of the viewpoint, a measurement of a transition period of a predetermined time is started , and a first warping parameter corresponding to the viewing zone in which the position of the viewer's viewpoint is detected is set, and the parameter value is changed to the value of the first warping parameter;
During the transition period, the first warping parameter is maintained regardless of a movement of a viewpoint, or a new second warping parameter is set in response to the movement of a viewpoint , and the parameter value is changed to become the value of the second warping parameter.
A first change process including the control
When it becomes necessary to update the parameter value accompanying the movement of the viewpoint after the transition period has elapsed, a third warping parameter corresponding to the viewing zone after the transition period has elapsed is set, and the parameter value is changed to become the value of the third warping parameter, or is changed gradually linearly or nonlinearly with respect to time.
A second change process including the control is executed ,
a first rate of change of the parameter value with respect to time in the first change process is set to be smaller than a second rate of change of the parameter value with respect to time in the second change process;
A display control device comprising: The processor,
During the transition period, when a viewpoint moves while the first change process is being executed, and the parameter value is changed to the value of the second warping parameter corresponding to a new viewing zone in response to the viewpoint movement,
executing a third change process so that the rate of change of the parameter value with respect to time in the first change process is greater than the first rate of change of the parameter value with respect to time in the first change process;
8. The display control device according to claim 7 . The processor,
When the third change process is executed,
executing a fourth change process in place of the second change process after the transition period has elapsed, the fourth change process being such that a rate of change of the parameter value is greater than the rate of change in the third change process;
9. The display control device according to claim 8 . The processor,
when irregular movement of the viewpoint, a decrease in accuracy of detecting the viewpoint position, or a temporary loss of the viewpoint position is detected during execution of the first change process within the transition period, the first change process is stopped, and the parameter value at the time of stopping is maintained.
10. The display control device according to claim 7 , wherein the display control device is a display control device. A method for controlling a head-up display (HUD) device that includes a display unit that displays an image, and an optical system that includes an optical member that reflects display light of the image and projects it onto a projection target, and that allows a viewer to view a virtual image of the image by projecting the image onto the projection target, comprising:
updating the parameter values according to the viewer's viewpoint position in the eyebox;
performing a viewpoint position tracking warping control in which the image to be displayed on the display unit is distorted in advance using the parameter values so that the image has a distortion characteristic opposite to that of a virtual image of the image;
When the inside of the eyebox is divided into a plurality of divided regions, each divided region is a viewing zone, and the position of the viewer's gaze point is detected in units of the viewing zone,
When it becomes necessary to update the parameter value due to the movement of the viewpoint, a measurement of a transition period of a predetermined time is started , and a first warping parameter corresponding to the viewing zone in which the position of the viewer's viewpoint is detected is set, and the parameter value is changed so as to become the value of the first warping parameter;
During the transition period, the first warping parameter is maintained regardless of a movement of a viewpoint, or a new second warping parameter is set in response to the movement of a viewpoint , and the parameter value is changed to a value of the second warping parameter.
Executing a first change process including the control of
When it becomes necessary to update the parameter value accompanying the movement of the viewpoint after the transition period has elapsed, a third warping parameter corresponding to the viewing zone after the transition period has elapsed is set, and the parameter value is changed so as to become the value of the third warping parameter, or is changed gradually linearly or nonlinearly with respect to time.
Executing a second change process including the control of
setting a first rate of change of the parameter value with respect to time in the first change process to be smaller than a second rate of change of the parameter value with respect to time in the second change process;
A method for controlling a head-up display device, comprising: During the transition period, when a viewpoint moves while the first change process is being executed, and the parameter value is changed to the value of the second warping parameter corresponding to a new viewing zone in response to the viewpoint movement,
and executing a third change process so that the rate of change of the parameter value with respect to time is greater than the first rate of change of the parameter value with respect to time in the first change process.
The display control device according to claim 11 . When the third change process is executed,
and executing a fourth change process in place of the second change process after the transition period has elapsed, the fourth change process having a rate of change of the parameter value greater than the rate of change in the third change process.
The display control device according to claim 12 . and when irregular viewpoint movement, a decrease in accuracy of viewpoint position detection, or a temporary loss of viewpoint position is detected during execution of the first change process within the transition period, the method further includes stopping the first change process and maintaining the parameter value at the time of stopping the first change process.
The method for controlling a head-up display device according to any one of claims 11 to 13 .

Images