JP2023120466A - Display control apparatus, display apparatus, and display control method - Google Patents

Abstract

To display an image with reduced distortion not only to a primary viewer but also a secondary viewer, the image being visually recognized by the primary viewer and the secondary viewer.SOLUTION: A display control apparatus 159 includes an information acquisition unit 135 which acquires line-of-sight information, and a control unit 160 which changes a correction parameter to be used for correcting image data. The control unit 160 uses, when only a first occupant 7a views an image, a correction parameter suitable for the first occupant, uses, when only a second occupant 7b views the image, a correction parameter suitable for the second occupant, sets, when both first and second occupants views the image, a first region S1 and a second regions S2 in the image and uses a correction parameter WP-a suitable for the first occupant as a correction parameter corresponding to the first region and uses a correction parameter WP-b suitable for the second occupant as a correction parameter corresponding to the second region.SELECTED DRAWING: Figure 3

Description

The present invention relates to a display control device, a display device, a display control method, and the like.

It is conceivable that an image displayed by a display device mounted on a vehicle (vehicle display device) may be viewed by passengers other than the driver.

Regarding this point, for example, in [0004] of Patent Document 1, "... a fellow passenger is riding (sitting) on the front passenger seat in addition to the driver, and the passenger sitting on the front passenger seat If you want to visually recognize the display image consisting of vehicle information, etc., in addition to the above-mentioned head-up display device (that is, the head-up display device for the driver's seat), an instrument in front of the passenger sitting in the passenger seat It is conceivable to separately arrange another head-up display device for the front passenger seat inside the panel.”

As an example of processing for correcting the distortion of an image to be displayed, for example, Patent Document 2 discloses warping processing in a HUD device (correction processing in which distortion having characteristics opposite to the distortion caused by an optical system is applied to image data in advance). ) is described.

JP 2012-108470 A JP 2015-87619 A

It is conceivable that not only the main viewer (passenger facing the image) but also the secondary viewer (passenger looking at the image from an angle) can see the image displayed by the display device.

Conventionally, the distortion correction of the displayed image is performed assuming only the main viewer. For this reason, the image may become an image with large distortion for the secondary viewer.

For example, when an occupant who is a secondary viewer views a displayed image from an angle to obtain information, the image is an image assuming a case where the occupant who is the main viewer sees it from the front. Therefore, the occupants, who are secondary viewers, may feel that the image is considerably distorted, causing discomfort or annoyance, or making it difficult to confirm necessary information. .

The aforementioned Patent Documents 1 and 2 do not mention this problem, nor do they describe any countermeasures therefor.

One of the objects of the present invention is to provide a display with reduced distortion not only for the main viewer but also for the secondary viewer when the image can be viewed by the primary viewer and the secondary viewer. That is.

Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best mode, as well as the accompanying drawings.

In order to facilitate an understanding of the general outline of the invention, the following illustrates embodiments in accordance with the invention.

In a first aspect, the display control device comprises:
A display control device that controls image display of a display device that is mounted on a vehicle and allows an occupant of the vehicle to visually recognize an image,
an information acquisition unit that acquires line-of-sight direction information including at least one of a line-of-sight direction and a face orientation of the occupant;
a control unit having a function of changing a correction parameter used when correcting image data of the image so as to suppress distortion of the image viewed from the passenger based on the sight line direction information;
has
When the display device has the first occupant as the main viewer and the second occupant as the secondary viewer,
The control unit
when only the first occupant is visually recognizing the image, using a correction parameter suitable for the first occupant as the correction parameter;
when only the second occupant is viewing the image, using a correction parameter suitable for the second occupant as the correction parameter;
When the first and second occupants are visually recognizing the image,
dividing the image into a first region that prioritizes the first occupant and a second region that prioritizes the second occupant;
Using a correction parameter suitable for the first occupant as the correction parameter corresponding to the first region,
A correction parameter suitable for the second occupant is used as the correction parameter corresponding to the second region.

In the first aspect, when there is a passenger (in other words, a secondary viewer) who views the display image obliquely in the left-right direction (in other words, the width direction of the vehicle), the secondary viewer is also taken into consideration. Then, the correction parameters are selected such that the distortion of the image as seen by the secondary viewer is reduced. Therefore, it is possible to provide a display with reduced distortion not only to a main viewer (for example, a viewer facing the image) but also to a secondary viewer.

Even when there are a plurality of passengers in the vehicle, it is possible to ensure the quality of display images (at least the minimum required image quality) for all passengers (viewers). Optimal display can be performed for all viewers, and therefore discomfort and annoyance given to each viewer can be reduced, and each passenger can obtain necessary information.

For example, assuming that there are first and second occupants seated next to each other, the first occupant faces the second occupant and is obliquely viewing the image displayed in the horizontal direction. , the second occupant may be viewing the image displayed facing the first occupant from an oblique angle in the left-right direction (in the case of line-of-sight crossing).

In such a case, in the distortion correction processing for the image data of each image, a correction parameter suitable for the occupant viewing from the oblique direction (the occupant who is a follower viewer) is applied.

It is also conceivable that both the first and second occupants are viewing a common display image (in the case of concentration of gazes or overlapping of gazes).

For example, if the second occupant is looking at the image obliquely, in consideration of this point, a region (second region) in the display image that suppresses deterioration in visibility as seen from the second occupant is set. In other words, the display image is divided into a first region that gives priority to the first occupant and a second region that gives priority to the second occupant.

When correcting the distortion of the image data corresponding to the first region, a correction parameter suitable for the first occupant is applied, and when correcting the distortion of the image data corresponding to the second region, a correction parameter suitable for the second occupant is applied. Correction parameters are applied.

By this processing, the image quality of the display image is ensured for both the first passenger and the second passenger, discomfort and annoyance are reduced, and the recognizability of the image is improved.

In a second aspect dependent on the first aspect,
When the width direction of the vehicle is the left-right direction,
The control unit
When setting the first and second regions,
setting the second area to the left of the first area in the image when the second occupant is positioned to the left of the first occupant;
When the second occupant is positioned to the right of the first occupant, the second area may be set to the right of the first area in the image.

In the second aspect, for example, the second occupant is positioned to the left (left side) of the first occupant, and each of the first and second occupants faces the first occupant, for example. When viewing the (first image) together, the second region to which the correction parameters suitable for the second occupant are applied is located on the left side of the first region corresponding to the seating position of each occupant. Arranged (set). If the second occupant is seated on the right side of the first occupant, the second area is arranged (set) to the right of the first area.

For an occupant viewing the image from an oblique angle, a portion of the image closer to him/herself has a greater visual impact than a distant portion. In other words, the portion closer to itself can be said to be the main portion of the image. Therefore, when the occupant views the image obliquely from the left side, the area (second area) to which the correction parameters suitable for the occupant are applied also corresponds to the seating position of the occupant, and is the same as that of the first area. It is provided on the left side. As a result, it is possible to apply the optimum correction parameters to a range that has a large visual impact on the passenger viewing the image from an oblique angle, thereby effectively improving the display quality.

In a third aspect depending on the first aspect,
The control unit
The image may be magnified, and the magnified image may be divided into the first and second regions.

In the third mode, the image is enlarged and the first and second areas are set in the enlarged image. For example, when a secondary occupant (a secondary viewer) views the image from an oblique angle, a second region (preferably including a main range that is estimated to be viewed by the secondary occupant) is set in the image. When the area of the second area is small, or when it is difficult to secure the second area itself, it may be difficult for a follower passenger to visually recognize the area.

In particular, when the first passenger and the second passenger are far from each other in the left-right direction, such a situation is likely to occur.

Therefore, when setting the first and second regions, for example, the image is enlarged (for example, the image is enlarged in the horizontal direction at a fixed magnification (or a magnification that is appropriately set according to the viewing environment), or , by enlarging at a constant magnification in each of the horizontal and vertical directions (or a magnification that is appropriately set according to the viewing environment), etc.), the required area of the second region (at least the minimum area) can be ensured. Therefore, the second occupant can reliably see the image in which the distortion is corrected. Even if the image is relatively difficult to see, the visibility of the image is ensured, so the practicality of the display device is improved.

In a fourth aspect depending on any one of the first to third aspects,
The control unit
When it is detected that the second occupant does not visually recognize the second area in a state in which the first and second areas are provided, a predetermined period from the time of detection, the second area may be left without being erased.

In the fourth aspect, in a state where line-of-sight concentration (line-of-sight overlap) occurs for a common image, the line-of-sight direction (face line direction) of the passenger (second passenger as a secondary viewer) who was looking at the image from an angle direction) changes and line-of-sight concentration (line-of-sight overlap) is eliminated (in other words, when there is a change in the number of passengers viewing the image), a predetermined period from the timing when this is detected , maintains the display state before detection.

If the area (second area) to which the correction parameter suitable for the occupant of the secondary viewer is applied is immediately extinguished, for example, immediately after that, when the line of sight moves and returns to the state of concentration of the line of sight, the correction parameter is repeated, and the appearance of the image of the occupant (the first occupant as the main viewer) who is looking at the image from the front repeatedly changes sharply in a short period of time, causing discomfort and annoyance. may occur.

Therefore, the previous state is maintained (in other words, the correction parameter suitable for the occupant of the secondary viewer is continued to be used) until a predetermined period of time elapses. As a result, it is possible to suppress discomfort or the like of the passenger (the passenger as the main viewer) viewing the image.

In a fifth aspect depending on the fourth aspect,
The control unit
During the predetermined period, the area of the second region may be reduced stepwise or continuously.

In the fourth aspect, during the above-mentioned predetermined period, the correction parameter suitable for the second occupant who is the secondary viewer continues to be used, provided that the correction parameter is applied to the area (second area). area is reduced (reduced) step by step or continuously within the predetermined period. As a result, the area of the region (first region) to which the correction parameters suitable for the first occupant are applied gradually increases during the predetermined period of time, and sharp changes in the appearance of the image are suppressed. , discomfort and the like are also suppressed.

In a sixth aspect, the display device comprises
a display control device according to any one of the first to fifth aspects;
an image data correction unit that corrects the image data of the image using the correction parameter so as to suppress distortion of the image viewed from at least one of the first passenger and the second passenger;
a display that displays an image based on the corrected image data;
have

According to the sixth aspect, when there is an occupant viewing the image from an oblique direction (an occupant of a subordinate viewer), an image subjected to distortion correction suitable for the occupant can be displayed on the display unit, The uncomfortable feeling of the passenger can be reduced.

Accordingly, it is possible to realize a high-performance display device capable of displaying an image with little distortion to both the occupant of the main viewer and the occupant of the secondary viewer.

In a seventh aspect depending on the sixth aspect,
The display device
It further has an optical system for projecting display light of an image displayed on the display unit onto a projection target member provided in the vehicle, and has a function of allowing the first passenger and the second passenger to visually recognize the virtual image. It may be a head-up display device with
or
The display device may have a function of displaying a real image on the display unit as a display panel and allowing the first passenger and the second passenger to visually recognize the real image.

The seventh aspect clarifies that the display device may be a HUD device or a display device. It is possible to realize a high-performance HUD device or display device capable of displaying images with less distortion to both the first and second occupants.

As the display device, a liquid crystal device (for example, a TFT liquid crystal device) or a micro LED display device, which is a self-luminous display device, may be used.

In an eighth aspect, the display control method includes:
A display control method for controlling image display of a display device mounted on a vehicle and allowing an occupant of the vehicle to visually recognize an image,
a first step of acquiring line-of-sight direction information including at least one of a line-of-sight direction and a face orientation of the occupant;
a second step of acquiring a correction parameter to be used in correcting image data of the image so as to suppress distortion of the image viewed from the passenger based on the line-of-sight direction information;
a third step of correcting image data of said image using the obtained correction parameters;
In the second step,
When the occupant as the main viewer who sees the image from the front is the first occupant, and the occupant as the secondary viewer who sees the image from an oblique direction is the second occupant,
when only the first occupant is visually recognizing the image, obtaining a correction parameter suitable for the first occupant as the correction parameter;
when only the second occupant is visually recognizing the image, obtaining a correction parameter suitable for the second occupant as the correction parameter;
When the first and second occupants are visually recognizing the image,
dividing the image into a first region that prioritizes the first occupant and a second region that prioritizes the second occupant;
Acquiring a correction parameter suitable for the first occupant as a correction parameter corresponding to the first region;
A correction parameter suitable for the second occupant is obtained as the correction parameter corresponding to the second region.

According to the eighth aspect, when there is an occupant viewing the image from an oblique direction (a subordinate viewer occupant), an image subjected to distortion correction suitable for the occupant can be displayed on the display unit, The uncomfortable feeling of the passenger can be reduced.

As a result, it is possible to display an image with little distortion to both the occupant of the main viewer and the occupant of the secondary viewer.

Those skilled in the art will readily appreciate that the illustrated embodiments in accordance with the present invention may be further modified without departing from the spirit of the invention.

FIG. 1(A) is a display system in which two display devices (HUD devices) are arranged in a vehicle that seats two people in the front row, and shows a state in which the lines of sight of two passengers cross each other, and FIG. 1(B). [Fig. 3] is a diagram showing a state in which line-of-sight concentration occurs. FIG. 2A is a diagram showing a configuration example of a HUD device having a warping correction function, and FIG. 2B is a diagram showing the effects of warping correction processing. FIG. 3A is a diagram showing an example of setting warping parameters when line-of-sight intersection occurs, FIG. 3B is a diagram showing an example of setting warping parameters when line-of-sight intersection occurs, and FIG. ) to (F) show an example of setting regions (first and second regions) to which different warping parameters are applied in the enlarged image, and FIG. It is a figure which shows the main structural examples of the display apparatus containing. FIG. 4(A) is a diagram showing an example of a display system in which three display devices (HUD devices) are arranged in a vehicle with three seats in the front row, and FIG. 4(B) is one of the three display devices. FIG. 5 shows a variant in which the platform is used to display images to passengers seated in the rear seats; FIG. 5A is a diagram showing an example of setting correction parameters when line-of-sight crossing occurs in the display system of FIG. 4A, and FIG. FIG. 10 is a diagram illustrating an example of setting correction parameters when line-of-sight concentration occurs; FIGS. 6A to 6C are diagrams showing an example of display control when gaze concentration is eliminated. FIGS. 7A to 7E are diagrams showing another example of display control when gaze concentration is eliminated. FIG. 8 is a diagram illustrating a configuration example of a main part of a display device (HUD device) and a configuration example of a display system. FIG. 9 is a flowchart illustrating an example of a display control procedure.

The best mode described below is used for easy understanding of the invention. Accordingly, those skilled in the art should note that the present invention is not unduly limited by the embodiments described below.

(First embodiment)
Please refer to FIG. FIG. 1(A) is a display system in which two display devices (HUD devices) are arranged in a vehicle that seats two people in the front row, and shows a state in which the lines of sight of two passengers cross each other, and FIG. 1(B). [Fig. 3] is a diagram showing a state in which line-of-sight concentration occurs.

In FIG. 1, the X direction indicates the width direction (or left-right direction) of the vehicle 1, the Y direction indicates the height direction of the vehicle 1, and the Z direction indicates the front of the vehicle 1 (front-rear direction).

The vehicle 1 is a front-row two-seat passenger car with a steering wheel 8 on the right side. In the vehicle 1, a driver (passenger) 3a is seated in a driver's seat 6, and a fellow passenger (passenger) 3b is seated in a passenger's seat 9. As shown in FIG.

Both the driver 3a and the fellow passenger 3b are "occupants". Also, in the following description, the terms "first passenger, second passenger" may be used. For example, an occupant facing (including substantially facing) a displayed image (display image), in other words, an occupant viewing the image from the front is referred to as a "first occupant", and the image is viewed obliquely in the horizontal direction. The viewing occupant may be referred to as the "second occupant."

In addition, the meaning of "front" or "right facing" should be interpreted in a broad sense, and the range that can be regarded as "front" and the range that can be regarded as "right facing" may be defined, including some fluctuation range.

In addition, the "passenger" can also be referred to as a "viewer" who visually recognizes the image. A viewer who views the image from the front is referred to as a "main viewer", and a viewer who views the image from an oblique angle in the horizontal direction is referred to as a "secondary viewer".

The above-mentioned first passenger is the "passenger as the main viewer", and the second passenger is the "passenger as the secondary viewer".

In FIG. 1A, a driver 3a and a fellow passenger 3b are seated side by side in the left-right direction. A fellow passenger (passenger) 3b is located on the left side of the driver (passenger) 3a. No occupant is present in the rear seat 11 .

Two display devices (here, HUD devices) 100a and 100b are arranged along the width direction (horizontal direction: X direction) of the vehicle 1 in the front portion, which is the portion on the windshield 2 side of the vehicle 1. . A HUD device 100a is installed (arranged) inside a dashboard and an instrument panel (both not shown) in front of the driver's seat 6 . A HUD device 100b is installed (arranged) inside the dashboard and instrument panel (both not shown) in front of the passenger seat 9 .

The display device is not limited to the HUD device, and a display device such as a liquid crystal device (for example, a TFT liquid crystal device) or a self-luminous display device such as a micro LED display device may be used.

Also, in FIG. 1A, a passenger placement detection camera 200 is provided to detect the placement of passengers in the vehicle. The image captured by the camera 200 is analyzed by an occupant placement detection unit (not shown in FIG. 1, reference numeral 202 in FIG. 8) to detect the occupant placement in the vehicle.

The HUD devices 100a and 100b project image display light onto the windshield 2 to allow the driver 7 and fellow passenger 10 to view the image (virtual image).

In FIG. 1A, a virtual image V1 is an image (virtual image) facing the driver 3a. In other words, regarding the image (virtual image) V1, the driver 3a is the passenger as the main viewer, and the fellow passenger 3b is the passenger as the secondary viewer.

Also, the virtual image V2 is an image (virtual image) facing the fellow passenger 3b. In other words, with respect to the image (virtual image) V2, the fellow passenger 3b is the passenger as the main viewer, and the driver 3a is the passenger as the secondary viewer.

The HUD devices 100a and 100b incorporate a display control device (processor: reference numeral 159 in FIG. 8, not shown in FIG. 1) having a function of changing correction parameters (warping parameters) for distortion correction.

As the image distortion correction, for example, a "pre-distortion correction process (warping correction process)" can be employed in which distortion having characteristics opposite to the distortion that occurs when the image is displayed is given to the image data in advance. In the following description, this may simply be referred to as "warping processing".

By appropriately switching the warping parameter as a correction parameter, it is possible to change the content of the distortion correction by the warping process. A specific example of display control in the display control device will be described later.

Further, in the vehicle 1, eye or face orientation detection cameras 300a and 300b are provided. The camera 300a captures the eyes (pupils) or face of the driver 3a. Based on the captured image, for example, a line-of-sight direction detection unit (not shown in FIG. 1, reference numeral 130 in FIG. 8) performs image processing, for example, changes in the position of the viewpoint in the eye box and changes in the direction of the face in the horizontal direction. can be detected to detect the line-of-sight direction (line-of-sight direction).

With the development of automatic driving technology, the driver 3a can relatively freely move his line of sight. In other words, the degree of freedom in the orientation of the line of sight is increased.

Therefore, the driver 3 can see, for example, an image (virtual image) V2 that faces the fellow passenger 10 and is displayed obliquely forward to the left (the line of sight direction in this situation is shown in FIG. ) with dashed arrows).

In addition, the fellow passenger 3b can also see an image (virtual image) V1 that faces the driver 3a and is displayed obliquely forward to the right (the direction of the line of sight in this situation is shown in FIG. 1A). indicated by the dashed arrow).

In FIG. 1A, the lines of sight of two passengers (a driver 3a and a fellow passenger 3b) who see an image (virtual image) obliquely ahead intersect. In the following description, this may be referred to as "line-of-sight crossing".

In addition, in FIG. 1A, solid line arrows are also shown for the driver 3a and the fellow passenger 3b. ) shows the line of sight when looking at V1 and V2.

Next, FIG. 1B is referred to. The arrangement of the display devices (in other words, the configuration of the display system) in FIG. 1B is the same as in FIG. 1A.

However, in FIG. 1B, both the driver 3a and the fellow passenger 3b are looking at the image (virtual image) V1. The line-of-sight direction of the driver 3a is indicated by a solid-line arrow, and the line-of-sight direction of the fellow passenger 3b is indicated by a broken-line arrow.

In the following description, a situation where a common image is viewed by a plurality of people at the same time may be referred to as "line-of-sight concentration (or line-of-sight overlap)".

When performing the warping process, consider whether or not there is a passenger looking at the image from an oblique angle. Specifically, consider the occurrence of "line of sight crossing" and "line of sight concentration", and set the appropriate warping parameters. need to be used (in other words, selected, determined, or obtained). This point will be described later.

Next, refer to FIG. FIG. 2A is a diagram showing a configuration example of a HUD device having a warping correction function, and FIG. 2B is a diagram showing the effects of warping correction processing.

The HUD device 100 is mounted on a vehicle (which can be interpreted in a broad sense) 1 . The HUD device 100 includes a display unit (for example, a light transmission screen, a liquid crystal display device, etc.) 101, a reflecting mirror (flat mirror) 103, and a curved mirror (for example, a concave mirror) as an optical member for projecting display light. The surface may be a free-form surface) 105, an image generation unit 150, a control unit 160, a ROM (storage device) 210 having an image conversion table 212 storing warping parameters WP as correction parameters, have

Note that the control unit 160 can be constructed as a functional block included in the display control device (processor: reference numeral 159 in FIG. 12). The controller 160 can selectively retrieve the warping parameters WP from the ROM 210 . In other words, for example, the control unit 160 uses the warping parameter WP has a function to change as appropriate.

An image displayed on the display unit 101 is projected onto the projectable area 5 of the windshield 2 as a projection target member via the reflecting mirror 103 and the curved mirror 105 . Reference numeral 4 indicates the projection area of the image.

In addition, in the HUD device 100, a plurality of curved mirrors may be provided. In addition to the mirror (reflecting optical element) of this embodiment, or in place of part (or all) of the mirror (reflecting optical element) of this embodiment, a refractive optical element such as a lens, a diffractive optical element, etc. A configuration including a functional optical element may be employed.

A part of the display light of the image is reflected by the windshield 2, and a preset eye box (actually, it is three-dimensional, but for convenience of explanation, it is drawn as a rectangle with a predetermined area in FIG. 2A) EB is incident on the viewpoint (eye) A of the driver or the like located inside (or on the EB), and is imaged in front of the vehicle 1 to display a virtual virtual image corresponding to the display surface 102 of the display unit 101 A virtual image V is displayed on the surface PS.

The image on the display unit 101 is distorted under the influence of the shape of the curved mirror 105, the shape of the windshield 2, and the like. In other words, distortion occurs due to optical members including the optical system of the HUD device 100 and the windshield 2 . In order to suppress the distortion, the image is given in advance distortion having characteristics opposite to the distortion (warping processing or warping image correction processing).

FIG. 2B shows an example of the virtual image V visually recognized by the passenger through the windshield 2 . In FIG. 2B, a virtual image V having a rectangular outer shape has, for example, 5 vertically and 5 horizontally, for a total of 25 reference points (reference pixel points or coordinate points) GD(i,j ) (where i and j are both variables that can take values from 1 to 5). Each reference point (each coordinate point) in the image (original image) is preliminarily given a distortion having characteristics opposite to the distortion occurring in the virtual image V due to the warping process. Therefore, the distortion that is given in advance and the distortion that actually occurs are offset, and ideally, a virtual image V without curvature is displayed as shown in FIG. 1(B). Note that the number of reference points GD(i, j) can be appropriately increased by interpolation processing or the like.

Next, refer to FIG. FIG. 3A is a diagram showing an example of setting warping parameters when line-of-sight intersection occurs, FIG. 3B is a diagram showing an example of setting warping parameters when line-of-sight intersection occurs, and FIG. ) to (F) show an example of setting regions (first and second regions) to which different warping parameters are applied in the enlarged image, and FIG. It is a figure which shows the main structural examples of the display apparatus containing.

In FIG. 3, it is assumed that there are two passengers 7a and 7b in the vehicle 1 (either one of them may be the driver, or both of them may be fellow passengers). Further, it is assumed that the displayed images include an image G100a facing the passenger 7a and an image G100b facing the passenger 7b. The images G100a and G100b may be virtual images displayed by the HUD device, or may be real images displayed by the display device.

In FIG. 3A, the occupant 7a is looking at an image (which may be a virtual image) G100b in the diagonally forward right direction, and the occupant 7b is looking at an image (which may be a virtual image) in the diagonally forward left direction G100a. , line-of-sight crossing occurs. References EBa and EBb denote eye boxes for the passengers 7a and 7b.

An eyebox is originally a virtual three-dimensional object, and a display device (display system) is designed on the premise that a displayed image can be seen when a person's eyes (viewpoint) are positioned within the eyebox. In this embodiment, the display images include a first image viewed from the front by the passenger and a second image viewed from an oblique direction by the passenger. In principle, the occupant can see the first and second images when the occupant's eyes are positioned within the eyebox.

However, the degree of distortion when viewing an image obliquely is expected to be greater than the degree of distortion when viewing an image from the front unless any measures are taken, resulting in an image that is difficult to see. If the distortion increases, it may be difficult to obtain sufficient information.

Therefore, in FIG. 3A, in consideration of the occupant as a secondary viewer who views the image from an oblique angle, correction parameters suitable for the occupant as the secondary viewer are used in the correction processing in the display control of the image. (warping parameters).

In FIG. 3A, for image G100a, only an occupant 7b who is a secondary viewer (an occupant viewing image G100a from an angle) exists. Therefore, the warping parameter WP-b suitable for the passenger 7b is used in the distortion correction process (warping process) in the display control of G100a.

Regarding the image G100b, there is only a passenger (an occupant viewing the image G100b from an angle) 7a who is a secondary viewer. Therefore, the warping parameter WP-a suitable for the passenger 7a is used in the warping process in the display control of G100b.

Here, "parameters suitable for occupants" will be described. An eye box (when the eye box is simplified and drawn on a plane, it is a virtual plane placed in front of the occupant) so as to correspond to the plurality of reference points described above with reference to FIG. ) is divided into a plurality of regions (unit regions: not shown).

Assuming that the occupant's viewpoint (right eye or left eye) is located in a certain unit area, a correction parameter (warping parameter) corresponding to that unit area is selected (viewpoint position follow-up warping processing). The warping parameters selected corresponding to this unit area can be the above-mentioned "parameters suitable for occupants".

However, it is an example and is not limited to this. Since there may be cases where the warping parameter is not set for each unit area of the eyebox, in such a case, a predetermined warping parameter (a parameter uniquely defined for the eyebox) corresponding to the occupant's eyebox is set to " parameter suitable for the occupant'.

Next, FIG. 3B will be referred to. In FIG. 3B, line-of-sight concentration occurs. Crew members 7a and 7b are both looking at image G100b. Therefore, when correcting the image data of the image G100b, it is necessary to apply two types of correction parameters, one suitable for the occupant 7a and one suitable for the occupant 7b.

Therefore, in the example of FIG. 2B, the image G100b is divided into a first region S1 in which the occupant 7b who is the main viewer (a viewer who sees the image G100b from the front) is prioritized, and a secondary viewer (the image G100b is viewed from the front). A second area S2 (indicated by diagonal lines in the figure) in which the occupant 7a, who is a viewer looking from an oblique direction, is prioritized. In other words, the first area S1 and the second area S2 are set (arranged) in the image G100b.

Corresponding to this division, the display image (original image) on the display surface 102 of the display unit 100 shown in FIG. , a warping parameter WP-b suitable for the passenger 7b is applied, and a warping parameter WP-a suitable for the passenger 7a is applied to the image data of the image region corresponding to the region S2.

Thus, when there is an occupant (in other words, a secondary viewer) who views the display image from an oblique angle in the left-right direction (in other words, in the width direction of the vehicle), the secondary viewer is also considered. Correction parameters are selected such that the distortion of the image as seen by the secondary viewer is reduced. Therefore, it is possible to provide a display with reduced distortion not only to a main viewer (for example, a viewer facing the image) but also to a secondary viewer.

Even when there are a plurality of passengers in the vehicle, it is possible to ensure the quality of display images (at least the minimum required image quality) for all passengers (viewers). Optimal display can be performed for all viewers, and therefore discomfort and annoyance given to each viewer can be reduced, and each passenger can obtain necessary information.

In other words, the image quality of the displayed image is ensured for both the occupant (first occupant) who views the image from the front and the occupant (second occupant) who views the image from an oblique direction. reduced, and image recognizability is improved.

Next, processing for enlarging an image when the line of sight is concentrated will be described. See FIGS. 3(C) and 3(D).

In FIG. 3C, a substantially square image G100b (the region of this image G100b is defined as S3) is enlarged in the left-right direction (the width direction of the vehicle) to form a horizontally long rectangular image G100b′. In image G100b', the previously described region segmentation is implemented. As a result, the enlarged image G100b' consists of the area S3 and the area S4.

A warping parameter WP-b suitable for the passenger 7b is applied to the region S3. A warping parameter WP-a suitable for the passenger 7a is applied to the region S4.

Although image enlargement processing is not essential, it is considered preferable in many cases. For example, in the example of FIG. 3C, the position of the face of the occupant 7a is near the right end of the eye box EBa, and even if the occupant 7a moves his/her eyes to the right side, the image G100b before enlargement is displayed. It can be difficult to see the whole. Similar difficulties can also be assumed when the distance between the occupant 7a and the occupant 7b in the left-right direction is large.

By enlarging the image as described above, if the eyes of the occupant 7a are in the eye box EBa, even if the image G100b' is obliquely viewed, the entire image G100b' (at least the main part) can be easily viewed. will be able to see In addition, the area of the image region S4 corrected by the parameters suitable for the passenger 7a can be increased, and the image quality seen from the passenger 7a is further improved.

In addition, when viewpoint concentration occurs and two types (broadly speaking, plural types) of correction parameters suitable for each passenger are applied, the image may be enlarged unconditionally, or some condition may be set. However, if the conditions are satisfied, the image may be enlarged.

For example, when it is difficult for a passenger viewing an image obliquely to see the image (at night, in bad weather, etc.), processing for enlarging the image may be performed.

In this way, when setting a plurality of regions to which different correction parameters are applied in one image, for example, the image is enlarged (for example, by a fixed magnification in the horizontal direction (or, depending on the viewing environment, set magnification), or by enlarging at a fixed magnification in each of the horizontal and vertical directions (or a magnification that is set as appropriate depending on the viewing environment), etc.), especially the image It is possible to reliably secure the required area (at least the minimum area) of the area to which the correction parameters suitable for the occupant viewed from an oblique direction are applied. Therefore, an occupant viewing the image from an oblique direction can reliably view the image in which the distortion has been corrected. Even if the image is relatively difficult to see, the visibility of the image is ensured, so the practicality of the display device is improved.

Next, FIG. 3D is referred to. In the example of FIG. 3D, a substantially square image G100b (the region of this image G100b is S3) is displayed in the left-right direction (vehicle width direction: lateral direction) and in the vehicle height direction (height direction). : in the vertical direction) to form an image G100b'' having an enlarged area and a similar outer shape, and the region segmentation described above is performed in the enlarged image G100b''. As a result, the magnified image G100b'' is composed of the area S3, the area S5, and the area S6.

In FIG. 3D, warping parameter WP-b suitable for passenger 7b is applied to regions S3 and S5, and warping parameter WP-a suitable for passenger 7a is applied to region 6. In FIG.

In addition, in the examples of FIGS. 3B, 3C, and 3D, the area (shaded area: second area) to which the correction parameters suitable for the occupant 7a viewing the image from an oblique direction is applied is It is set (arranged) on the left side of the area (white area: first area) to which correction parameters suitable for the occupant 7b viewing the image from the front are applied.

This corresponds to the position of the occupant 7b on the left side of the occupant 7a (in other words, the seating position of each occupant). When the occupant 7b is positioned on the right side of the occupant 7a, the second area is set (arranged) on the right side of the first area.

For an occupant viewing the image from an oblique angle, a portion of the image closer to him/herself has a greater visual impact than a distant portion. In other words, the portion closer to itself can be said to be the main portion of the image. Therefore, when the occupant views the image obliquely from the left side, the area (second area) to which the correction parameters suitable for the occupant are applied also corresponds to the seating position of the occupant, and is the same as that of the first area. It is provided on the left side. As a result, it is possible to apply the optimum correction parameters to a range that has a large visual impact on the passenger viewing the image from an oblique direction, thereby effectively improving the display quality.

In addition, the area of the area (hatched area: second area) to which the correction parameters suitable for the occupants viewing the image obliquely are applied, and the correction parameters suitable for the occupants viewing the image from the front are applied. The ratio of the area of the region (outlined region: first region) to the area can be appropriately changed according to the driving situation and the surrounding environment. For example, the ratio is normally 1:2, but if it is determined that the image is particularly difficult for the occupant looking at from an angle, the area of the second region is enlarged and the area ratio is set to 1:1. adaptive display control such as

Further, the outer shape of the first and second regions may be rectangular as in the examples of FIGS. 3(B) to 3(D). However, it is not limited to this. For example, as shown in FIGS. 3(E) and 3(F), by dividing the image into two diagonally, the outer shape of each area can be determined.

In FIG. 3(E), a horizontally long rectangular image is bisected by a diagonal line, the warping parameter WP-a is applied to the lower left area (hatched area) S8, and the upper right area (white area) ) The warping parameter WP-b is applied to S7.

In FIG. 3(F), a horizontally long rectangular image is bisected along the diagonal, and the warping parameter WP-a is applied to the upper left area (hatched area) S10, and the lower right area (outlined area) S10. The warping parameter WP-b is applied to the region S9.

Next, FIG. 3G is referred to. The configuration of the display device shown in FIG. 3G is similar (basically the same) as the configuration shown in the lower right of FIG. In FIG. 3G, the same reference numerals are given to the parts common to those in FIG.

However, FIG. 3G clearly shows that the control unit 160 is a component of the display control device 159 . FIG. 3G also clarifies that the display control device 159 includes an information acquisition unit 135 that acquires line-of-sight information (line-of-sight direction information). Based on the acquired line-of-sight information (line-of-sight direction information), the control unit 160 appropriately performs display control as shown in FIGS.

(Second embodiment)
Reference is now made to FIG. FIG. 4(A) is a diagram showing an example of a display system in which three display devices (HUD devices) are arranged in a vehicle with three seats in the front row, and FIG. 4(B) is one of the three display devices. FIG. 5 shows a variant in which the platform is used to display images to passengers seated in the rear seats; In addition, in the example of FIG. 4, the steering wheel 8 is located on the left side inside the vehicle.

In FIG. 4A, in the front portion of the vehicle 1 on the windshield 2 side, three display devices (here, HUD devices) 110a, 110b, and 110c are arranged in the width direction of the vehicle 1 (left-right direction: X direction).

A HUD device 110a is arranged at a position corresponding to the driver's seat 16 (a position facing the driver's seat), and a HUD device 110b is arranged at a position corresponding to the adjacent passenger's seat (center passenger's seat) 18, and the adjacent passenger's seat ( A HUD device 110c is arranged at a position corresponding to the passenger's seat 20 on the right end.

In addition, cameras 310a to 310c for detecting the direction of eyes or faces of each person are provided corresponding to the driver and each passenger (each passenger).

A driver 3e sits in the driver's seat 16, a fellow passenger (passenger) 3f sits in the passenger seat 18, and a fellow passenger (passenger) 3g sits in the passenger seat 20. - 特許庁No occupant is present in the rear seat 21 .

Further, the presence (seated) of the driver 3e, the presence (seated) of the fellow passengers 3f and 3g, and the absence of passengers in the rear seat 21 are detected by a camera (occupant placement detection). It can be determined by image analysis of an image captured by the camera 200 .

Each of the HUD devices (first to third HUD devices) 110a to 110c projects image display light onto the windshield 2 to allow the driver 3e and fellow passengers 3f and 3g to view the image (virtual image).

The HUD device 110a is a display device with the driver 3e as the main viewer and the fellow passengers 3f and 3g as secondary viewers.

The HUD device 110b is a display device for which the fellow passenger 3f is the main viewer and the driver 3e or the fellow passenger 3g is the secondary viewer. The HUD device 110c is a display device for which the fellow passenger 3g is the main viewer and the fellow passenger 3f or the driver 3e is the secondary viewer.

Information capable of specifying the main viewer of each HUD device is input and set in advance in each of the HUD devices 110a to 110c (the control unit 160 of the display control device thereof).

For example, among the three display devices 110a to 110c, for the display device 110a installed at the position corresponding to the driver 3e, for example, if "1" is input on the initial setting screen, the driver 3e is the main display device. It is set to be a viewer.

Similarly, for the display device 110b arranged at the position corresponding to the central passenger seat 18, if for example "2" is entered on the initial setting screen, it will be positioned next to the driver's seat 16 in the front row (in other words, , sitting in the middle passenger seat 18) is set to be the main viewer. Similarly, for the display device 110c, by inputting "3" on the initial setting screen, the main viewer is set to be the fellow passenger 19 seated in the passenger seat 20 on the far right in the front row.

After setting all the display devices in this way, if information such as the presence or absence of passengers in the vehicle 1, the location information of the passengers, and the line-of-sight information (line-of-sight direction information) is input to each of the display devices 110a to 110c, , the control unit 160 of each of the display devices 110a to 110c determines whether or not the correction parameters are changed, and automatically optimizes the correction parameters. Therefore, it is possible to realize a high-performance display system that is easy to use and can display an image with less distortion for both the driver and the passengers.

In FIG. 4B, two display devices (here, HUD devices) 100a and 100b are arranged in a passenger car with two seats in the front row (this point is the same as in FIGS. 1A and 1B). ), and a display device 100c is added between the driver's seat 6 and the passenger's seat 9. FIG. In addition, a camera 300c is added to detect the orientation of the eyes or face of the passenger 3c seated in the center of the rear seat 11. FIG.

In the above-described embodiment, the viewer of the image is an occupant sitting in the front row of the vehicle 1, but as in the example of FIG. A passenger (passenger) 3d seated at a position where the display by 100d can be seen may be added to the viewer. The display control operation is basically the same as in FIG. 4(A).

The example of FIG. 4A will be specifically described below. Please refer to FIG. FIG. 5A is a diagram showing an example of setting correction parameters when line-of-sight crossing occurs in the display system of FIG. 4A, and FIG. FIG. 10 is a diagram illustrating an example of setting correction parameters when line-of-sight concentration occurs;

First, FIG. 5A is referred to. In FIG. 5A, three eyeboxes EB3e, EB3f, and EB3g are set on the front side. Eye boxes EB3e, EB3f, and EB3g correspond to a driver 3e, a fellow passenger (passenger) 3f, and a fellow passenger (passenger) 3g, respectively.

In front of the vehicle 1, the virtual image display surface PS is divided into three in the left-right direction, and three rectangular image display areas are provided so as to correspond to the passengers 3e to 3g. Virtual images (images) V10 to V12 are displayed in each image display area.

The arrows in the drawing indicate the line-of-sight directions of the driver (passenger) 3e and fellow passengers (passengers) 3f and 3g. A driver (passenger) 3e sees a virtual image V10 in front, a fellow passenger (passenger) 3f in the center sees a virtual image V12 positioned at the right end from an angle, and a fellow passenger (passenger) 3g at the right end sees the virtual image V10 at the center. is viewed obliquely from the virtual image V11. Therefore, line-of-sight crossing occurs between fellow passengers (passengers) 3f and 3g.

Therefore, for the virtual images V11 and V12, display control similar to the example of FIG. 3A described above can be performed. In other words, when correcting the distortion of the image data corresponding to the central virtual image V11, the warping parameter WP-3g suitable for the rightmost fellow passenger (passenger) 3g is applied, and the distortion of the image data corresponding to the rightmost virtual image V12 is corrected. At this time, a warping parameter WP-3f suitable for the middle passenger (passenger) 3f is applied.

Next, FIG. 5B will be referred to. In FIG. 5B, the driver (passenger) 3e looks at the virtual image V10 at the left end, the fellow passenger (passenger) 3f at the center looks at the virtual image V12 at the right end from an angle, and the fellow passenger at the right end (Passenger) 3g is looking at the rightmost virtual image V12 from the front. Therefore, the line of sight is concentrated on the fellow passengers (occupants) 3f and 3g with respect to the virtual image V12.

In this case, no occupant sees the central virtual image V11. However, considering the normal situation in which each passenger faces the front, the central passenger 3f facing the virtual image V11 is the main viewer. Warping parameters suitable for the center occupant 3f are applied for the correction.

Further, for the virtual image V12 on the right end, display control similar to the examples of FIGS. 3B to 3F described above can be performed. In FIG. 5B, areas S10 and S11 are set in the virtual image V12. The area S11 is arranged on the left side of the area S10.

When correcting the distortion of the image data corresponding to the region S10, a warping parameter WP-3g suitable for a fellow passenger (passenger) 3g viewing the virtual image V12 from the front is applied. When correcting the distortion of the image data corresponding to the region S11, a warping parameter WP-3f suitable for a fellow passenger (passenger) 3f who sees the virtual image V12 from an oblique direction is applied.

In this way, even when there are a plurality of passengers in the vehicle, it is possible to ensure the quality of display images (at least the minimum required image quality) for all passengers (viewers). Optimal display can be performed for all viewers, and therefore discomfort and annoyance given to each viewer can be reduced, and each passenger can obtain necessary information.

In other words, the image quality of the displayed image is ensured for both the occupant (first occupant) who views the image from the front and the occupant (second occupant) who views the image from an oblique direction. reduced, and image recognizability is improved.

Next, refer to FIG. FIGS. 6A to 6C are diagrams showing an example of display control when gaze concentration is eliminated.

FIG. 6(A) is a reproduction of FIG. 5(C) shown above. Next, in FIG. 6B, the direction of the line of sight of the rightmost fellow passenger 3g is changed, and the line of sight concentration changes to a state in which each passenger sees an image (virtual image) in front of each passenger. In other words, there is no occupant looking at the image from an oblique direction, and the concentration of the line of sight is eliminated.
However, even if the change in the state is detected, the correction parameter of the display image is not changed immediately, and the state before the detection is maintained for a predetermined period. The display mode of the image (virtual image) in FIG. 6B is the same as in FIG. 6A, and there is no change.

This is because if the area (second area) S11 to which the correction parameters suitable for the occupant (the occupant as a follower viewer) who is viewing the image from an oblique direction is immediately erased, for example, immediately after that, When the occupant's line of sight moves and returns to the line of sight concentration state, the change (switching) of the correction parameters is repeated. This is because there is a possibility that the appearance of an image will repeatedly change abruptly in a short period of time, causing discomfort and annoyance.

When the predetermined period of time has passed and the state in which the line of sight concentration has been eliminated continues, the region S11 disappears as shown in FIG. is the only area to which correction parameters suitable for are applied.

In this way, in a state in which line-of-sight concentration (line-of-sight overlap) occurs for a common image, the line-of-sight direction (face orientation) of the passenger (second passenger as a subordinate viewer) who is obliquely viewing the image changes and the line-of-sight concentration (line-of-sight overlap) is eliminated (in other words, when the number of passengers viewing the image changes), detection is performed for a predetermined period from the timing when this is detected Keep the previous display state.

If the area (second area) to which the correction parameter suitable for the occupant of the secondary viewer is applied is immediately extinguished, for example, immediately after that, when the line of sight moves and returns to the state of concentration of the line of sight, the correction parameter is repeated, and the appearance of the image of the occupant (the first occupant as the main viewer) who is viewing the image repeatedly changes sharply in a short period of time, causing discomfort and annoyance. may occur.

Therefore, the previous state is maintained (in other words, the correction parameter suitable for the occupant of the secondary viewer is continued to be used) until a predetermined period of time elapses. As a result, it is possible to suppress discomfort, annoyance, and the like of the passenger (the first passenger as the main viewer) who is viewing the image.

Next, refer to FIG. FIGS. 7A to 7E are diagrams showing another example of display control when gaze concentration is eliminated. In the example of FIG. 7, the area of the region to which the correction parameter suitable for the occupant viewing the image from an angle is applied is reduced stepwise or continuously during the predetermined period.

FIG. 7(A) is a reproduction of FIG. 6(B) described above. As described above with reference to FIG. 6B, even when it is detected that the line-of-sight concentration state has been resolved (in other words, there is no occupant looking at the image obliquely), the image is not displayed for a predetermined period of time. The area (second area) to which the correction parameters suitable for the occupant who is obliquely watching remains.

Here, as shown in FIGS. 7B to 7D, the area of the region (second region) S11 is decreased stepwise or continuously over the predetermined period. . In FIG. 7C, the area of the region S11 is smaller than in FIG. 7B, and in FIG. 7D the area of the region S11 is even smaller.

Then, after a predetermined period of time has elapsed, the display state shown in FIG. 7(E) is reached. Note that FIG. 7(E) is the same as FIG. 6(C).

As described above, in the example of FIG. 7, the area (second area) to which the correction parameters suitable for the occupant who was viewing the image from an angle is applied during the predetermined period is changed stepwise or continuously. Decrease (shrink).

As a result, the area of the region (first region) to which the correction parameters suitable for the occupant (the first occupant as the main viewer) who is viewing the image from the front is applied gradually increases during the predetermined period continue. Therefore, abrupt changes in the appearance of the image are suppressed, and discomfort, annoyance, and the like are suppressed.

Reference is now made to FIG. FIG. 8 is a diagram illustrating a configuration example of a main part of a display device (HUD device) and a configuration example of a display system. Although the main structure of the display device was shown in FIGS. 2 and 3G, FIG. 8 shows a more detailed structure. Also, in the example of FIG. 8, three HUD devices are used as display devices (corresponding to the example of FIG. 4A).

In FIG. 8, the display system includes a communication unit 112, an ECU (electronic control unit) 120, eye or face direction detection cameras 310a to 10c, a line of sight direction detection unit 130, and an occupant placement detection camera (in-vehicle imaging camera). ) 200, an occupant placement detector 202 for detecting occupants (passengers including the driver), and first to third HUD devices 110a, 110b, and 110c. Each HUD device has the same configuration. The main configuration of the third HUD device 110c will be described below.

The third HUD device 110c includes a display unit 101, a display control unit 145, an image generation unit 150 (including an image data correction unit 155), and at least one processor (display control device) 159 (control unit 160). , the control unit 160 has a warping correction control unit 163), a database 170 storing navigation data, display image data, etc., and a ROM 210 as a storage device.

ROM 210 has an image conversion table 212 . The image conversion table 212 has warping parameters WP as correction parameters for distortion correction. As described above, the warping parameters WP include parameters for passengers viewing the image from the front (in other words, main parameters) and parameters for passengers viewing the image obliquely in the horizontal direction (in other words, sub-parameters). and are included.

The processor (display control device) 159 (the control unit 160 thereof) has a function of appropriately changing the warping parameters. The image data correction unit 155 corrects the image data of the display image using the warping parameter so as to suppress the distortion of the image seen by the passenger. An image based on the corrected image data is displayed on the display unit 101 .

According to this configuration, when there is an occupant viewing the image from an oblique angle, an appropriate correction parameter is selected in consideration of the occupant, and an image in which distortion is suppressed by the correction parameter is displayed on the display unit 101. be able to. Therefore, a sense of incongruity or the like is reduced, and the image display quality is improved. As a result, a high-performance display device and a display system capable of displaying an image with little distortion to all passengers are realized.

(Third Embodiment)
Reference is now made to FIG. FIG. 9 is a flowchart illustrating an example of a display control procedure.

In step S1, the presence or absence and arrangement of passengers (in other words, passengers including the driver and fellow passengers) are detected.

In step S2, the line-of-sight direction (line-of-sight direction) of each crew member (each passenger) is detected.

In step S3, it is determined whether or not there is an occupant viewing the image obliquely on each display device. When N, the process proceeds to step S4.

In step S4, the image is corrected using a warping parameter suitable for the passenger viewing the image from the front, and the corrected image is displayed.

When Y in step S3, the process proceeds to step S5. In step S5, a warping parameter is determined in consideration of an occupant viewing the image from an oblique direction, the image data is corrected, and the corrected image is displayed.
In other words, for an occupant viewing the image from an oblique angle, the image corrected by the warping parameters suitable for the occupant is displayed (presented).

In step S6, it is determined whether or not the state of viewing the image obliquely has been resolved. In other words, it is determined whether there is no occupant viewing the image from an oblique direction. If N, continue the determination process, and if Y, go to step S7.

In step S7, the warping parameters are updated to those suitable for the state after resolution of oblique viewing of the image, the image is corrected using the updated parameters, and the corrected image is displayed.

However, in step S7, when performing the process when the line-of-sight concentration is resolved, the process of maintaining the previous state for a predetermined period may be performed. Further, during the predetermined period, a process of gradually or continuously reducing the application area of the parameters suitable for the occupant viewing the image from an oblique angle may be performed.

In step S8, it is determined whether or not to end the image display. If N, end the image display control. If Y, the process proceeds to step S2.

As described above, according to the present invention, when an image can be visually recognized by a main viewer and a secondary viewer, the image can be displayed with reduced distortion not only for the primary viewer but also for the secondary viewer. can be provided.

The present invention can be used in either a monocular HUD device in which display light of the same image is incident on the left and right eyes, or a parallax HUD device in which an image with parallax is incident on the left and right eyes.

Also, as the display device, a liquid crystal display device, a micro LED device, and other various display devices can be used.

Moreover, in this specification, the term vehicle can be broadly interpreted as a vehicle. Also, terms related to navigation (such as signs) should be interpreted in a broad sense, taking into consideration the viewpoint of broad-sense navigation information that is useful for driving a vehicle, for example. HUD devices also include those used as simulators (for example, aircraft simulators, game device simulators, etc.).

The invention is not limited to the exemplary embodiments described above, and the person skilled in the art will easily be able to modify the exemplary embodiments described above to the extent that they fall within the scope of the claims. .

1... Vehicle (self-vehicle), 2... Projected member (reflecting translucent member, windshield, etc.), 3a... Occupant (driver), 3b... Fellow passenger (occupant), 3g- 3e... Occupant (including driver and fellow passenger), 4... Image projection area, 6... Driver's seat, 7a, 7b... Occupant (including driver and fellow passenger), 8. Steering wheel 9 Passenger seat 11 Rear seat 100 (100a to 100c) HUD device (display device in a broad sense) 110 (110a to 110c) HUD device ( In a broad sense, a display device), 101: display section (liquid crystal panel, etc.), 102: image display surface of the display section, 112: communication section, 120: ECU (electronic control unit), 130. Line-of-sight direction detection unit 135 Information acquisition unit 145 Display control unit 150 Image generation unit 155 Image data correction unit 159 Processor (display control device) , 160...control unit, 163...warping correction control unit, 170...database, 200...passenger placement detection camera, 202...passenger placement detection unit, 210...storage device (ROM) , 212... Image conversion table, 300 (300a to 300c), 310 (310a to 310c)... Eye or face orientation detection camera, WP... Warping parameter, G100a, G100b, G100b', G100b'' ... image (virtual image or real image), EB (EBa, EBb) ... eye box, V ... virtual image.

Claims (8)

A display control device that controls image display of a display device that is mounted on a vehicle and allows an occupant of the vehicle to visually recognize an image,
an information acquisition unit that acquires line-of-sight direction information including at least one of a line-of-sight direction and a face orientation of the occupant;
a control unit having a function of changing a correction parameter used when correcting image data of the image so as to suppress distortion of the image viewed from the passenger based on the sight line direction information;
has
When the occupant as the main viewer who sees the image from the front is the first occupant, and the occupant as the secondary viewer who sees the image from an oblique direction is the second occupant,
The control unit
when only the first occupant is visually recognizing the image, using a correction parameter suitable for the first occupant as the correction parameter;
when only the second occupant is viewing the image, using a correction parameter suitable for the second occupant as the correction parameter;
When the first and second occupants are visually recognizing the image,
dividing the image into a first region that prioritizes the first occupant and a second region that prioritizes the second occupant;
Using a correction parameter suitable for the first occupant as the correction parameter corresponding to the first region,
Using a correction parameter suitable for the second occupant as the correction parameter corresponding to the second region;
Display controller. When the width direction of the vehicle is the left-right direction,
The control unit
When setting the first and second regions,
setting the second area to the left of the first area in the image when the second occupant is positioned to the left of the first occupant;
When the second occupant is positioned in the right direction with respect to the first occupant, the second area is set to the right of the first area in the image;
The display control device according to claim 1. The control unit
enlarging the image and dividing the enlarged image into the first and second regions;
The display control device according to claim 1. The control unit
When it is detected that the second occupant does not visually recognize the second area in a state in which the first and second areas are provided, a predetermined period from the time of detection, the second to remain without erasing the area of
The display control device according to any one of claims 1 to 3. The control unit
During the predetermined period, the area of the second region is reduced stepwise or continuously;
The display control device according to claim 4. A display control device according to any one of claims 1 to 5;
an image data correction unit that corrects the image data of the image using the correction parameter so as to suppress distortion of the image viewed from at least one of the first passenger and the second passenger;
a display that displays an image based on the corrected image data;
display device. The display device
It further has an optical system for projecting display light of an image displayed on the display unit onto a projection target member provided in the vehicle, and has a function of allowing the first passenger and the second passenger to visually recognize the virtual image. It is a head-up display device with
or
A display device having a function of displaying a real image on the display unit as a display panel and allowing the first passenger and the second passenger to visually recognize the real image,
The display device according to claim 6. A display control method for controlling image display of a display device mounted on a vehicle and allowing an occupant of the vehicle to visually recognize an image,
a first step of acquiring line-of-sight direction information including at least one of a line-of-sight direction and a face orientation of the occupant;
a second step of acquiring a correction parameter to be used in correcting image data of the image so as to suppress distortion of the image viewed from the passenger based on the line-of-sight direction information;
a third step of correcting image data of said image using the obtained correction parameters;
In the second step,
When a passenger as a main viewer who sees the image from the front is a first passenger, and a passenger who is a secondary viewer who sees the image from an oblique direction is a second passenger,
when only the first occupant is visually recognizing the image, obtaining a correction parameter suitable for the first occupant as the correction parameter;
when only the second occupant is visually recognizing the image, obtaining a correction parameter suitable for the second occupant as the correction parameter;
When the first and second occupants are visually recognizing the image,
dividing the image into a first region that prioritizes the first occupant and a second region that prioritizes the second occupant;
Acquiring a correction parameter suitable for the first occupant as a correction parameter corresponding to the first region;
Acquiring a correction parameter suitable for the second occupant as the correction parameter corresponding to the second region;
Display control method.

Images