JP2021056358A - Head-up display device - Google Patents

Abstract

To provide a head-up display device for displaying a parallax type stereoscopic image suppressed in crosstalk.SOLUTION: A display control part includes the steps of: acquiring a position of a left eye 4L and a position of a right eye 4R of a visualizer; alternately increasing the emission intensity of a left eye light source in which the image light is condensed in a left viewing area where the left eye is arranged and a right eye light source in which the image light is condensed in a right viewing area where the right eye is arranged; increasing the emission intensity of the left eye light source when a transmission display panel displays a left eye image; and increasing the emission intensity of the right eye light source when the transmission display panel displays a right eye image.SELECTED DRAWING: Figure 6A

Description

The present disclosure relates to a head-up display device used in a vehicle to perceive a stereoscopic image by binocular parallax.

A parallax barrier with multiple slits and a lenticular lens are used to optically separate the left and right parallax images, allowing the viewer to see the corresponding parallax images for the left and right eyes, enabling stereoscopic display. A stereoscopic display device is known.

Japanese Unexamined Patent Publication No. 2014-50062

However, in the conventional stereoscopic display device, crosstalk or the like may occur depending on the light source that irradiates the lenticular lens with light.

A summary of the particular embodiments disclosed herein is presented below. It should be understood that these aspects are presented solely to provide the reader with an overview of these particular embodiments and do not limit the scope of this disclosure. In fact, the present disclosure may include various aspects not described below.

The outline of the present disclosure relates to providing a head-up display device for visually recognizing a stereoscopic image by binocular parallax with suppressed crosstalk.

Therefore, in the head-up display device of the embodiment described in the present specification, the light source group in which a plurality of light sources are arranged and the illumination light from the light source group are spatially light-modulated to form an image for the left eye and an image for the right eye. A transmissive display panel that emits the image light of the above, a condensing optical system that condenses the image light from the transmissive display panel into a viewing area shifted according to the position of the light source of the light source group, and the left eye of the viewer. The position and the position of the right eye are acquired or estimated, and the image light is focused on the left visual area where the left eye is located and the image light is focused on the right visual area where the right eye is located. When the light emission intensity of the right eye light source is increased alternately and the transmission display panel displays the image for the left eye, the emission intensity of the light source for the left eye is increased, and when the transmission display panel displays the image for the right eye, it is for the right eye. It is composed of a display control unit that increases the emission intensity of the light source.

It is a figure which shows the application example of the head-up display device to the vehicle which concerns on some Embodiments, and is the figure which shows the virtual position where a stereoscopic image is perceived. It is a figure which shows the example of the image displayed by the head-up display device which concerns on some embodiments. It is a figure which shows the example of the image displayed by the head-up display device which concerns on some embodiments. It is a figure which shows the application example of the head-up display device to the vehicle which concerns on some Embodiments, and is the figure which shows the virtual position where a stereoscopic image is perceived. It is a figure which shows the structure of the head-up display device which concerns on some embodiments. It is a figure which shows the optical path seen from the 2nd direction from a light source group to a visual area which concerns on some embodiments. It is a figure which shows the optical path seen from the 1st direction orthogonal to the 2nd direction from a light source group to a visual area, which concerns on some embodiments. It is a figure explaining the difference in the diffusion angle of the diffusion plate which has diffusion anisotropy, the figure on the left shows the diffusion angle by a diffusion plate in the long side direction of a liquid crystal panel, and the figure on the right is a figure which shows the diffusion angle by a diffusion plate, and the figure on the right is a short side of a liquid crystal panel. It is a figure which shows the diffusion angle by the diffusion plate in a direction. It is a block diagram of the display system for a vehicle which concerns on some embodiments. 6 is a timing chart showing the relationship between the image displayed by the liquid crystal panel and the driving state of each light source in the arrangement of the eyes of the viewer shown in FIG. 6A. 6 is a timing chart showing a modified example of the relationship between the image displayed by the liquid crystal panel and the driving state of each light source in the arrangement of the eyes of the viewer shown in FIG. 6A. 6 is a timing chart showing a modified example of the relationship between the image displayed by the liquid crystal panel and the driving state of each light source in the arrangement of the eyes of the viewer shown in FIG. 6A. 6 is a timing chart showing a modified example of the relationship between the image displayed by the liquid crystal panel and the driving state of each light source in the arrangement of the eyes of the viewer shown in FIG. 6A. It is a flowchart which shows the process when the HUD apparatus displays a stereoscopic image which concerns on some Embodiments. It is a figure which shows the mode in which the image for a left eye and / or the image for a right eye is displayed in a plurality of view areas in a head-up display device of some embodiments. It is a figure which shows another aspect in which the image for a left eye and / or the image for a right eye is displayed in a plurality of view areas in some embodiments of a head-up display device. It is a figure which shows another aspect in which the image for a left eye and / or the image for a right eye is displayed in a plurality of view areas in some embodiments of a head-up display device. It is a figure which shows another aspect in which the image for a left eye and / or the image for a right eye is displayed in a plurality of view areas in some embodiments of a head-up display device.

Hereinafter, FIGS. 1 and 4 provide an explanation of a virtual position of a perceived stereoscopic image. 2 and 3 provide a description of a stereoscopic image visually recognized by a viewer. 5 and 8 provide a description of the configuration of an exemplary vehicle display system and head-up display device. 6A and 6B provide a description of the optical path from the light source group to the view area. 9A, 9B, 9C, 9D, 10, 11A, 11B, 11C, and 11D provide a description of the processing of the display control unit of the present embodiment. The present invention is not limited to the following embodiments (including the contents of the drawings). Of course, changes (including deletion of components) can be made to the following embodiments. Further, in the following description, in order to facilitate understanding of the present invention, description of known technical matters will be omitted as appropriate.

See FIG. The vehicle display system 10 includes a HUD device 20, a display control device 30 that controls the HUD device 20, and a viewpoint detection camera 40 (eye position detection unit 411). In the description of the present embodiment, the left-right direction when the viewer 4 (driver) seated in the driver's seat of the vehicle 1 faces the front of the vehicle 1 is the X-axis (the left direction is the X-axis positive direction), and the top and bottom. The direction is the Y-axis (the upward direction is the Y-axis positive direction), and the front-rear direction is the Z-axis (the front direction is the Z-axis positive direction).

The HUD device 20 in the vehicle display system 10 is a head-up display (HUD: Head-Up Display) device provided in the dashboard 5 of the vehicle 1. The HUD device 20 emits the image light 50 toward the front windshield 2 (an example of the projected portion), and the image light 50 reflected by the front windshield 2 generates a viewing area E in a predetermined region, which will be described later. To do. By arranging the eye position in the viewing area E, the viewer 4 can visually recognize the image displayed by the HUD device 20 as a stereoscopic image. Here, the viewing area E is defined as a region where the image can be visually recognized as a stereoscopic image, but the region is not limited to this, and the entire image displayed by the HUD device 20 can be visually recognized. This is an area where the image displayed by the HUD device 20 is visually recognized in a desired state, such as an area where the distortion of the image displayed by the HUD device 20 visually recognized by a person is within a predetermined threshold value. The HUD device 20 visually recognizes a stereoscopic image (virtual image) on the front side (Z-axis positive direction) of the front windshield 2 (an example of the projected portion).

The virtual image display area 100 is a plane, a curved surface, or a partially curved surface region, and is also called an image plane. The virtual image display area 100 is a position where a virtual image of the display surface of the 3D display 21 described later (the exit surface of the liquid crystal panel 22 described later) of the HUD device 20 is formed. That is, the virtual image display area 100 is the HUD device. Corresponding to the display surface described later of 20 (in other words, the virtual image display area 100 has a conjugate relationship with the display surface of the 3D display 21 described later), the virtual image visually recognized in the virtual image display area 100 is a HUD device. It can be said that it corresponds to the image displayed on the display surface described later in 20. It is preferable that the virtual image display area 100 itself has low visibility to the extent that it is not actually visible to the viewer 4 or is difficult to see. The 3D display 21 can realize a stereoscopic display with a sense of depth with respect to the display surface by displaying a parallax image having a parallax corresponding to the left and right eyes. That is, the HUD device 20 causes the viewer 4 to perceive the stereoscopic image 200 based on the virtual image display area 100 by displaying the parallax image corresponding to the eye points (left and right eyes) arranged in the view area E. be able to. As a result, the viewer 4 perceives the stereoscopic image 200 having a sense of depth in the Z-axis direction that overlaps the foreground 300, which is the real space that is visually recognized through the front windshield 2.

In the vehicle display system 10 (HUD device 20) of the present embodiment, the virtual image display area 100 is tilted from an angle perpendicular to the road surface 310 (an example of a real object in the foreground 300) about the left-right direction (X-axis direction). Is placed. In other words, a first tilt angle θt1 (0 ≦ θt1 <90 [degree]) is provided between the road surface 310 about the left-right direction (X-axis direction) and the virtual image display area 100. The first tilt angle θt1 is, for example, 65 [degree], but is not limited thereto.

Further, the vehicle display system 10 (HUD device 20) of the present embodiment may display a stereoscopic image 200 to be perceived by the viewer 4 along the road surface 310. Specifically, the stereoscopic image 200 is located at a position overlapping the upper area 101 of the virtual image display area 100 when viewed from the viewer 4, and at a position away from the viewer with reference to the upper area 101 of the virtual image display area 100. The perceived distant stereoscopic image 201 is located at a position overlapping the lower region 102 below the upper region 101 (in the negative direction of the Y axis) when viewed from the viewer 4, and is closer to the viewer than the virtual image display region 100. It is an integral or intermittent stereo image including a near stereo image 202 perceived by the user.

In the vehicle display system 10 (HUD device 20) of the present embodiment, the stereoscopic image 200 has a first tilt angle θt1 between the stereoscopic image 200 and the road surface 310 (an example of the foreground 300) about the left-right direction (X-axis direction). A smaller second tilt angle θt2 (0 ≦ θt2 <θt1 <90 [image]) is provided. The second tilt angle θt2 is, for example, 7 [degree], but is not limited to this.

2 and 3 are views showing a display example of the stereoscopic image 200 displayed by the HUD device 20. The stereoscopic image 200 of FIG. 2 is an intermittent stereoscopic image, which is composed of six (plural) arrow-shaped images (also referred to as route images) showing a guide route, and overlaps the road surface 310 in the foreground 300 of the vehicle 1. At the positions, the reference numerals 211, 212, 213, 214, 215, and 216 are arranged in this order from the side of the guide path closest to the vehicle 1. The guide route indicates a right turn at the intersection after going straight, that is, the stereoscopic images 211, 212, and 213 showing the guide route to the intersection overlap the road surface 310 of the traveling lane of the vehicle 1 as seen from the viewer 4. The stereoscopic images 214, 215, and 216, which indicate the straight-ahead direction (Z-axis positive direction) toward the front intersection and show the guidance path ahead of the intersection, are the road surface 310 of the branch road in the right turn direction when viewed from the viewer 4. It is arranged so as to rise from, and indicates the right direction (X-axis negative direction). The stereoscopic images 211-216 are arranged so as to be visually recognized along the road surface 310, that is, the parallax image generation module 616 described later forms the virtual object indicated by the stereoscopic images 211-216 with the road surface 310. The arrangement (angle) is set so that the angle becomes 7 [image] (in other words, it is substantially parallel to the road surface 310).

The stereoscopic image 200 of FIG. 3 is an integral stereoscopic image, which is composed of one (singular) arrow-shaped image (also referred to as a route image) showing a guide route, and overlaps the road surface 310 in the foreground 300 of the vehicle 1. At the position, one path image extending from the vicinity to the distance is arranged. The guidance route shows a right turn at an intersection after going straight, that is, a part of the stereoscopic image 217 showing the guidance route to the intersection overlaps the road surface 310 of the traveling lane of the vehicle 1 as seen from the viewer 4, and is ahead. A part of the stereoscopic image 217 that indicates the straight direction (Z-axis positive direction) toward the intersection and shows the guidance path beyond the intersection is along the road surface 310 of the branch road in the right turn direction when viewed from the viewer 4. It is arranged so as to indicate the right direction (X-axis negative direction). The stereoscopic image 216 is arranged so as to be visually recognized along the road surface 310, that is, the parallax image generation module 616 described later has an angle of 7 [] with the virtual object shown by the stereoscopic image 217 with the road surface 310. The arrangement (angle) is set so as to be [image] (in other words, substantially parallel to the road surface 310).

The stereoscopic image 200 is not arranged along the road surface 310, but may be one that rises up and is perceived with respect to the road surface 310. The stereoscopic image 220 of FIG. 3 is perceived by getting up with respect to the road surface 310, and shows the speed “40 km / h” of the vehicle 1. FIG. 4 is a diagram schematically showing the position where the stereoscopic image 220 of FIG. 3 is perceived, and the stereoscopic image 220 of FIG. 3 is a road surface 310 (foreground 300 of the foreground 300) centered in the left-right direction (X-axis direction). It may be arranged so as to have a third tilt angle θt3 (0 ≦ θt2 <θ3 ≦ 90 [image]) larger than the second tilt angle θt2. The third tilt angle θt3 is, for example, 90 [degree], but is not limited thereto. That is, the vehicle display system 10 (HUD device 20) of the present embodiment perceives along the road surface 310 (feels the depth in the Z-axis direction) stereoscopic images 211, 212, 213, 214, 215, 216, and It is possible to display a perspective image 210 such as 217 and a non-perspective image 220 that is perceived as rising from the road surface 310 (it is difficult to feel the depth in the Z-axis direction).

FIG. 5 is a diagram showing the configuration of the HUD device 20 of the present embodiment. The HUD device 20 includes a 3D display 21 having a display surface for displaying an image, and a relay optical system 25.

The 3D display 21 of FIG. 5 is composed of a liquid crystal panel (display surface) 22 and a backlight module 24. The display surface is from an angle perpendicular to the optical axis 50p of the image light 50 from the display surface toward the viewing area E (center of the viewing area E) via the relay optical system 25 described later and the projected portion. It is tilted so that the virtual image display area 100 can be tilted with respect to the road surface 310. Further description of the 3D display 21 will be described later. The 3D display 21 may be attached with an actuator (not shown) such as a motor controlled by the display control device 30, and may be movable and / or rotatable on the display surface.

The relay optical system 25 is an image light 50 from the 3D display 21 between the 3D display 21 and the front windshield 2 (an example of the projected portion) (light from the 3D display 21 toward the view area E). It is composed of one or more optical members arranged on the optical path of the above and projecting the image light 50 from the 3D display 21 onto the front windshield 2 outside the HUD device 20. The relay optical system 25 of FIG. 5 includes one concave first mirror 26 and one flat second mirror 28.

The first mirror 26 has, for example, a free curved surface shape having positive optical power. In other words, the first mirror 26 may have a curved surface shape in which the optical power differs for each region, that is, the optical power applied to the image light 50 according to the region (optical path) through which the image light 50 passes. different. Specifically, the first image light 51, the second image light 52, and the third image light 53 (see FIG. 5) heading from each region of the display surface to the viewing area E are added by the relay optical system 25. The optical power may be different.

The second mirror 28 may be a curved surface having optical power instead of a flat surface. That is, the relay optical system 25 is added according to the region (optical path) through which the image light 50 passes by synthesizing a plurality of mirrors (for example, the first mirror 26 and the second mirror 28 of the present embodiment). The optical power may be different.

In the present embodiment, the relay optical system 25 includes two mirrors, but the present invention is not limited to this, and one or more refractive optics such as a lens may be added or substituted to these. It may include a member, a diffractive optical member such as a hologram, a catoptric member, or a combination thereof.

Further, the relay optical system 25 of the present embodiment has a function of setting the distance to the virtual image display area 100 by the curved surface shape (an example of optical power), and a virtual image obtained by enlarging the image displayed on the display surface. It has a function of generating (stereoscopic image 200), but in addition to this, it may have a function of suppressing (correcting) distortion of a virtual image that may occur due to the curved shape of the front windshield 2.

Further, the relay optical system 25 may be movable and / or rotatable to which an actuator (not shown) such as a motor controlled by the display control device 30 is attached.

The liquid crystal panel 22 receives illumination light from the backlight module 24, and emits the image light 50 obtained by spatially light-modulating the illumination light toward the relay optical system 25 (second mirror 28). The liquid crystal panel 22 has a short side along the first direction α in which pixels corresponding to the vertical direction (Y-axis direction) of the stereoscopic image 200 viewed from the viewer 4 are arranged, and a first direction orthogonal to the first direction α. It is a rectangular shape having a long side along the two directions β. The viewer 4 visually recognizes the transmitted light of the liquid crystal panel 22 via the virtual image optical system 90. The virtual image optical system 90 is a combination of the relay optical system 25 shown in FIG. 5 and the front windshield 2. Further, the condensing optical system 95 is a combination of the relay optical system 25 and the illumination optical system 70 shown in FIG. FIG. 6A shows the optical path when viewed from the short side side (second direction β) of the liquid crystal panel 22, and FIG. 6B shows the optical path when viewed from the long side side (first direction α) of the liquid crystal panel 22. Shown.

See FIG. 6A. The backlight module 24 includes a light source group 60 and an illumination optical system 70.

The light source group 60 is, for example, a plurality of chip-type LEDs, and emits illumination light to the liquid crystal panel 22. The light source group 60 is composed of, for example, four light sources 61, 62, 63, 64, and is arranged in a row along the second direction β. The brightness of the light source group 60 is selectively adjusted according to the position of the eyes of the viewer 4 under the control of the display control device 30. In short, it is estimated that the viewer 4 has no eyes by reducing the output intensity of the light source that emits the illumination light toward the position where the viewer 4 is estimated to have no eyes (including turning off the lights). The light intensity of the image light 50 toward the position is reduced. This will be described later.

The illumination optical system 70 has a first lens 71 arranged in the emission direction of the illumination light of the light source group 60, a second lens 73 arranged in the emission direction of the first lens 71, and an emission direction of the second lens 73. It is composed of an arranged diffuser plate 75.

The illumination optical system 70 refracts the illumination light from the light source group 60, passes through the liquid crystal panel 22, passes through the virtual image optical system 90, and has a parallax image (left eye image and right eye image) in the visual range E described later. ) Is imaged. Specifically, as shown in FIG. 6A, the illumination optical system 70 refracts the illumination light from the first light source 61, passes through the liquid crystal panel 22, passes through the virtual image optical system 90, and has a viewing range E1. An image for the left eye is formed on the image. Further, the illumination optical system 70 refracts the illumination light from the third light source 63, passes through the liquid crystal panel 22, passes through the virtual image optical system 90, and forms an image for the right eye in the visual range E3. By changing the number of light sources included in the light source group 60, the number of viewing areas E in which the illumination light from the light source (image light from the liquid crystal panel 22) is imaged can be changed.

The first lens 71 has a convex shape so that at least one of the incident surface (light source group 60 side) and / or the exit surface (second lens 73 side) has a positive optical power. The first lens 71 of the present embodiment has a convex toroidal shape in which the curvature centered on the first direction α and the curvature centered on the second direction β are different only on the incident surface, but the present invention is limited to this. The entrance surface and / or the exit surface may be a toroidal shape, a cylindrical shape, a spherical shape, a free curved surface shape, or a combination thereof.

The first lens 71 deflects the emitted light from each light source group 60 so that the illumination lights from the plurality of light source groups 60 are superimposed at the position of the diffuser plate 75. That is, the refractive power of the first lens 71 is set so that each of the plurality of light source groups 60 illuminates the entire surface of the liquid crystal panel 22 (the entire area where an image can be displayed) when viewed from the first direction α. Has been done. For example, as shown in FIG. 6A, both the illumination light from the first light source 61 and the illumination light from the third light source 63 illuminate the entire area of the liquid crystal panel 22. The same applies to the other light sources 62 and 64. Therefore, the image of the entire liquid crystal panel 22 can be visually recognized from the plurality of viewing areas E1, 402, 403, and 404, and the brightness of the three-dimensional image 200 that can be visually recognized within each viewing area E1, 402, 403, and 404 is uniform. Become.

The second lens 73 has a function of deflecting the emitted light of the first lens 71 in a desired direction. Only the incident surface of the second lens 73 of the present embodiment has a convex shape centered only on the first direction α. The second lens 73 of the present embodiment is not limited to this, and the incident surface and / or the exit surface is a toroidal shape, a cylindrical shape, a spherical shape, a free curved surface shape, or a combination thereof. May be good. The second lens 73 is not always necessary and may be omitted.

See FIG. 6B. The refractive force (optical power) of the first lens 71 and the second lens 73 is the light distribution angle (first direction α) of the illumination light in the short side direction (first direction α) of the image light emitted from the exit surface of the liquid crystal panel 22. (See FIG. 6B) is set to be narrower than the light distribution angle (see FIG. 6A) of the illumination light in the long side direction (second direction β).

The diffuser plate 75 diffuses the light deflected by the first lens 71 and the second lens 73 and emits the light to the liquid crystal panel 22. As a result, it is possible to reduce the luminance unevenness in the image light 50 generated by the plurality of light source groups 60 visually recognized in the viewing range E. The diffuser plate 75 may be an optical member having a function of diffusing light. For example, the surface thereof is composed of a bead member, a fine uneven structure, or a rough surface. Further, a dot sheet or a transparent milky white sheet may be used.

Further, in some embodiments, the diffusion plate 75 may have diffusion anisotropy. Diffusion anisotropy means that the angle (diffusion angle) at which the passing (transmitted) illumination light is diffused differs depending on the direction. That is, when the circular incident light flux passes through the diffusion plate 75 having diffusion anisotropy, an elliptical light flux is emitted. The diffusion plate 75 having diffusion anisotropy is, for example, formed by arranging a plurality of minute protrusions having different aspect ratios in parallel on the surface along a certain direction, and forming a mold or forming a plurality of minute protrusions. It is produced by stretching the provided transparent plate in a certain direction. However, the configuration and manufacturing method of the diffusion plate 75 having diffusion anisotropy are not limited to these.

FIG. 7 is a diagram for explaining the difference in the diffusion angle of the diffusion plate 75 having diffusion anisotropy, and the left figure shows the diffusion angle by the diffusion plate 75 in the long side direction (second direction β) of the liquid crystal panel 22. The figure on the right shows the diffusion angle by the diffusion plate 75 in the short side direction (first direction α) of the liquid crystal panel 22. The diffusion characteristics of the diffusion plate 75 of some embodiments are set so that the diffusion angle of the second direction β is smaller (narrower) than the diffusion angle of the first direction α. In other words, the diffusion angle of the diffuser plate 75 is set narrow in the direction in which the light distribution angle of the refractive power (optical power) of the first lens 71 and the second lens 73 is wide (second direction β), and the first lens 71 In the direction in which the light distribution angle of the refractive power (optical power) of the second lens 73 is narrow (first direction α), the diffusion angle of the diffuser plate 75 is set wide. As a result, in the second direction β, the illumination light from each of the light sources 61, 62, 63, 64 has a wide light distribution angle set by the first lens 71 and the second lens 73, so that the entire liquid crystal panel 22 is covered. Since the diffusion angle is set narrow by the diffuser plate 75 that can illuminate and has diffusion anisotropy, the diffusion angle of the image light emitted from each pixel of the liquid crystal panel 22 is also narrowed. Therefore, it is possible to suppress crosstalk generated by the image light from each pixel of the liquid crystal panel 22 heading toward a wide region (viewing area E).

The illumination optical system 70 may include a lenticular lens in place of or in addition to the first lens 71 and the second lens 73. As an example, the lenticular lens has a plurality of cylindrical lenses extending in each of the first directions α. The plurality of cylindrical lenses are arranged side by side from one end to the other end along the second direction β of the illumination optical system 70. In other words, the direction in which the cylindrical lens extends is the vertical direction (Y-axis direction) when the viewer 4 faces forward (Z-axis positive direction). A plurality of cylindrical lenses extending in the vertical direction in this way are arranged side by side periodically. As a result, the illumination optical system 70 may condense the image light of the image for the right eye and the image for the left eye into different viewing areas. The pitch of the plurality of wrench curry lenses is preferably larger than the pitch of the light source group 60. The direction in which the cylindrical lens extends is not parallel to the first direction α, but may be arranged at an angle (however, it is not parallel to the second direction β).

FIG. 8 is a block diagram of the vehicle display system 10 according to some embodiments. The vehicle display system 10 includes a HUD device 20 and a display control device 30 that controls the HUD device 20. The display control device 30 includes one or more I / O interfaces 32, one or more processors 34, one or more memories 36, and one or more image processing circuits 38. The various functional blocks shown in FIG. 8 may consist of hardware, software, or a combination of both. FIG. 8 is only one embodiment of the embodiment, and the illustrated components may be combined with a smaller number of components, or there may be additional components. For example, the image processing circuit 38 (eg, graphic processing unit) may be included in one or more processors 34.

As shown, the processor 34 and the image processing circuit 38 are operably connected to the memory 36. More specifically, the processor 34 and the image processing circuit 38 operate the vehicle display system 10 by executing a program stored in the memory 36, for example, generating and / or transmitting image data. be able to. The processor 34 and / and the image processing circuit 38 include at least one general purpose microprocessor (eg, central processing unit (CPU)), at least one application specific integrated circuit (ASIC), and at least one field programmable gate array (FPGA). , Or any combination thereof. The memory 36 includes any type of magnetic medium such as a hard disk, any type of optical medium such as a CD and DVD, any type of semiconductor memory such as a volatile memory, and a non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVROM.

As shown, the processor 34 is operably linked to the I / O interface 32. For example, the I / O interface 32 uses the vehicle display system 10 as a personal area network (PAN) such as a Bluetooth (registered trademark) network, or a local area network (LAN) such as an 802.1x Wi-Fi (registered trademark) network. , LTE®, 4G, or 5G cellular networks can include wireless communication interfaces that connect to wide area networks (WAN). The I / O interface 32 can also include a wired communication interface such as, for example, a USB port, a serial port, a parallel port, an OBDII, and / or any other suitable wired communication port.

As shown in the figure, the processor 34 is interoperably connected to the I / O interface 32 to provide information with various other electronic devices and the like connected to the vehicle display system 10 (I / O interface 32). Can be exchanged. The I / O interface 32 includes, for example, a vehicle ECU 501 provided in the vehicle 1, a road information database 503, a vehicle position detection unit 505, an external sensor 507, a line-of-sight direction detection unit 509, an eye position detection unit 511, and a mobile information terminal. The 513, the external communication connection device 520, and the like are operably connected. The HUD device 20 is operably connected to the processor 34 and the image processing circuit 38. Therefore, the image displayed by the HUD device 20 may be based on the image data received from the processor 34 and / and the image processing circuit 38. The processor 34 and the image processing circuit 38 control the image displayed by the HUD device 20 based on the information obtained from the I / O interface 32. The I / O interface 32 may include a function of processing (converting, calculating, analyzing) information received from another electronic device or the like connected to the vehicle display system 10.

The vehicle 1 is described in various states (for example, mileage, vehicle speed, accelerator pedal opening, engine throttle opening, injector fuel injection amount, engine speed, motor speed, steering angle, shift position, drive mode, etc.). It includes a vehicle ECU 501 that detects (warning state) and the like. The vehicle ECU 501 controls each part of the vehicle 1, and can, for example, transmit vehicle speed information indicating the current vehicle speed of the vehicle 1 to the processor 34. In addition to simply transmitting the data detected by the sensor to the processor 34, the vehicle ECU 501 can transmit the determination result and / or the analysis result of the data detected by the sensor to the processor 34. For example, information indicating whether the vehicle 1 is traveling at a low speed or stopped may be transmitted to the processor 34.

Further, the vehicle ECU 501 may transmit an instruction signal instructing the stereoscopic image 200 displayed by the vehicle display system 10 to the I / O interface 32, and at this time, it is necessary to notify the coordinates of the stereoscopic image 200 and the stereoscopic image 200. The degree and / and the necessity-related information that is the basis for determining the notification necessity may be added to the instruction signal and transmitted.

The vehicle 1 may include a road information database 503 including a navigation system or the like. The road information database 503 is based on the position of the vehicle 1 acquired from the own vehicle position detection unit 505, which will be described later, and is based on the road information (lane, white line, stop line, pedestrian crossing) on which the vehicle 1 which is an example of the actual object-related information travels. , Road width, number of lanes, intersections, curves, pedestrian crossings, traffic regulations, etc.), feature information (buildings, bridges, rivers, etc.), presence / absence, position (including distance to vehicle 1), direction (vehicle 1) The direction), shape, type, detailed information, etc. may be read out and transmitted to the processor 34. Further, the road information database 503 may calculate an appropriate route from the departure point to the destination and transmit it to the processor 34 as navigation information.

The vehicle 1 may include a vehicle position detection unit 505 made of GNSS (Global Navigation Satellite System) or the like. The road information database 503, the mobile information terminal 513 described later, and / and the vehicle external communication connection device 520 acquire the position information of the vehicle 1 from the own vehicle position detection unit 505 continuously, intermittently, or at a predetermined event. The information around the vehicle 1 can be selected and / and generated and transmitted to the processor 34.

The vehicle 1 may include one or more out-of-vehicle sensors 507 that detect real objects present around the vehicle 1 (front, side, and rear). The actual object detected by the vehicle exterior sensor 507 may include, for example, a pedestrian, a bicycle, a motorcycle, another vehicle, a road surface 310, a lane marking, a roadside object, or / and a feature (building, etc.). Examples of the vehicle exterior sensor include a radar sensor such as a millimeter wave radar, an ultrasonic radar, and a laser radar, and a camera sensor composed of a camera and an image processing device, which may be composed of a combination of both a radar sensor and a camera sensor. It may be composed of only one of them. Conventional well-known methods are applied to object detection by these radar sensors and camera sensors. By object detection by these sensors, the presence or absence of a real object in the three-dimensional space, and if the real object exists, the position of the real object (relative distance from the vehicle 1 and the traveling direction of the vehicle 1 in the front-rear direction). (Horizontal position, vertical position, etc.), size (horizontal direction (horizontal direction), height direction (vertical direction), etc.), movement direction (horizontal direction (horizontal direction), depth) Direction (front-back direction)), movement speed (horizontal direction (left-right direction), depth direction (front-back direction)), or / and type may be detected. One or more external sensors 507 detect a real object in front of the vehicle 1 for each detection cycle of each sensor, and the real object-related information (presence / absence of a real object, real object) which is an example of the real object-related information. If an object exists, information such as the position, size, or / and type of each real object) can be transmitted to the processor 34. Note that these real object-related information may be transmitted to the processor 34 via another device (for example, vehicle ECU 501). Further, when using a camera as a sensor, an infrared camera or a near-infrared camera is desirable so that a real object can be detected even when the surroundings are dark such as at night. When using a camera as a sensor, a stereo camera that can acquire a distance or the like by parallax is desirable.

The vehicle 1 may include a line-of-sight direction detection unit 509 including an infrared camera or the like that captures the face of the viewer 4 to detect the gaze direction of the viewer 4 (hereinafter, also referred to as “line-of-sight direction”). The processor 34 acquires an image (an example of information capable of estimating the line-of-sight direction) captured by the infrared camera, and analyzes the captured image to identify the line-of-sight direction of the viewer 4. The processor 34 may acquire the line-of-sight direction of the viewer 4 specified by the line-of-sight direction detection unit 509 (or another analysis unit) from the image captured by the infrared camera from the I / O interface 32. Further, the method of acquiring the information capable of estimating the line-of-sight direction of the viewer 4 of the vehicle 1 or the line-of-sight direction of the viewer 4 is not limited to these, and is not limited to these, and is the EOG (Electro-occlera) method and the corneal reflex method. , The scleral reflex method, the Purkinje image detection method, the search coil method, the infrared fundus camera method, and other known line-of-sight detection (estimation) techniques may be used.

The vehicle 1 may include an eye position detection unit 511 including an infrared camera or the like that detects the eye position of the viewer 4. The processor 34 acquires an image (an example of information capable of estimating the eye position) captured by the infrared camera, and can identify the eye position of the viewer 4 by analyzing the captured image. The processor 34 may acquire information on the eye position of the viewer 4 identified from the image captured by the infrared camera from the I / O interface 32. The method of acquiring information capable of estimating the eye position of the viewer 4 of the vehicle 1 or the eye position of the viewer 4 is not limited to these, and is a known eye position detection (estimation) technique. May be obtained using. The processor 34 adjusts at least the position of the stereoscopic image 200 based on the position of the eyes of the viewer 4, so that the stereoscopic image 200 superimposed on the desired position of the foreground 300 is superimposed on the desired position of the foreground 300. It may be visually recognized in 4).

The personal digital assistant 513 is a smartphone, a laptop computer, a smart watch, or other information device that can be carried by the viewer 4 (or another occupant of the vehicle 1). The I / O interface 32 can communicate with the mobile information terminal 513 and acquires the data recorded in the mobile information terminal 513 (or the server through the mobile information terminal). The mobile information terminal 513 has, for example, the same functions as the road information database 503 and the vehicle position detection unit 505 described above, acquires the road information (an example of information related to an actual object), and transmits the road information to the processor 34. May be good. Further, the mobile information terminal 513 may acquire commercial information (an example of information related to a real object) related to a commercial facility in the vicinity of the vehicle 1 and transmit it to the processor 34. The mobile information terminal 513 transmits the schedule information of the owner (for example, the viewer 4) of the mobile information terminal 513, the incoming information on the mobile information terminal 513, the received information of the mail, etc. to the processor 34, and the processor 34 and The image processor 38 may generate and / or transmit image data relating to these.

The out-of-vehicle communication connection device 520 is a communication device that exchanges information with the vehicle 1. For example, another vehicle or pedestrian-to-vehicle communication (V2P: Vehicle To) connected to the vehicle 1 by vehicle-to-vehicle communication (V2V: Vehicle To Vehicle). It is a network communication device connected by a pedestrian (a mobile information terminal carried by a pedestrian) and a vehicle-to-vehicle communication (V2I) connected by a Pedestrian), and in a broad sense, it is a communication device with a vehicle 1 (V2I). V2X: Includes everything connected by Vehicle To Everything). The out-of-vehicle communication connection device 520 acquires, for example, the positions of pedestrians, bicycles, motorcycles, other vehicles (preceding vehicles, etc.), road surfaces, lane markings, roadside objects, and / and features (buildings, etc.), and the processor 34. May be sent to. Further, the vehicle external communication connection device 520 has the same function as the own vehicle position detection unit 505 described above, and may acquire the position information of the vehicle 1 and transmit it to the processor 34, and further, the road information database 503 described above. The road information (an example of information related to a real object) may be acquired and transmitted to the processor 34. The information acquired from the external communication connection device 520 is not limited to the above.

The software components stored in the memory 36 are the real object-related information detection module 602, the real object position setting module 604, the eye position detection module 606, the notification necessity determination module 608, the image position determination module 610, and the image size determination module 612. , Graphic module 614, parallax image generation module 616, display control module 618, and light source control module 620.

The real object-related information detection module 602 acquires information including at least the position of the real object existing in front of the vehicle 1 (also referred to as real object-related information). The real object-related information detection module 602, for example, visually recognizes the position of the real object existing in the foreground 300 of the vehicle 1 (the viewing direction (forward) of the vehicle 1 from the viewer 4 in the driver's seat of the vehicle 1) from the vehicle exterior sensor 507. It is the position in the height direction (vertical direction), the horizontal direction (horizontal direction), and the position in the depth direction (front direction) may be added to these positions), and the size of the actual object (height direction). , Lateral size), relative speed to vehicle 1 (including relative moving direction), and information (an example of real object-related information) may be acquired. Further, the real object-related information detection module 602 uses the external communication connection device 520 to control the position, relative speed, type of the real object (other vehicle), the lighting state of the direction indicator of the real object (other vehicle), and the steering angle operation. Information (an example of the actual object and related information) indicating the state of, and / and the planned progress route and progress schedule by the driving support system may be acquired.

Further, the real object-related information detection module 602 is located at the position of the lane marking 311 (see FIGS. 2 and 3) on the left side of the traveling lane 310 (see FIGS. 2 and 3) of the vehicle 1 and the position on the right side of the vehicle outside sensor 507. The position of the lane marking 312 (see FIGS. 3 and 3) may be acquired, and the region (traveling lane 310) between the left and right lane markings 311, 312 may be recognized.

The real object position setting module 604 acquires an observation position indicating the current position of the real object from the road information database 503, the vehicle exterior sensor 507, the mobile information terminal 513, or the vehicle exterior communication connection device 520 via the I / O interface 32. Or, the observation position of the real object obtained by fusing these two or more observation positions is acquired, and the position of the real object (also referred to as a specific position) is set based on the acquired observation position. The image position determination module 610, which will be described later, determines the position of the stereoscopic image 200 based on the specific position of the real object set by the real object position setting module 604.

The real object position setting module 604 sets a specific position of the real object based on the observed position of the real object acquired immediately before, and at least one or more acquired in the past including the observed position of the real object acquired immediately before. It is possible to set a specific position of the real object based on the predicted position of the real object at a predetermined time predicted based on the observed position of the real object of. That is, by executing the real object position setting module 604 and the image position determination module 610 described later, the processor 34 determines the specific position of the stereoscopic image (AR image) 200 based on the observation position of the real object acquired immediately before. A stereoscopic image (AR image) 200 predicted based on the first position setting process to be set and the observation position of one or more real objects acquired in the past including the observation position of the real object acquired at least immediately before. It is possible to execute the second position setting process for setting the position of the stereoscopic image (AR image) 200 based on the predicted position of the real object in the display update cycle.

In the first stereoscopic image (AR image) 200, for example, the real object position setting module 604 sets the specific position of the real object, which is the reference of the display position, in the first position setting process and the second position setting process. And are calculated by using them properly in chronological order, and in the second stereoscopic image (AR image) 200, even if the specific position of the real object that is the reference of the display position is calculated by the second position setting process. Good. The real object positioning module 604 uses a prediction algorithm such as the least squares method, a Kalman filter, an α-β filter, or a particle filter, and uses one or more observation positions in the past to obtain the next value. May be predicted.

The eye position detection module 606 detects the positions of the left and right eyes of the viewer 4 of the vehicle 1. The eye position detection module 606 determines where in the plurality of viewing areas E the eyes of the viewer 4 are located, the positions of the eyes of the viewer 4 in the left-right direction (X-axis direction), and / and the depth direction (Z). Includes various software components for performing various actions related to (axial) position detection, visual 4 eye position (X, Y, Z axis position) detection. The eye position detection module 606 obtains, for example, the eye position of the viewer 4 from the eye position detection unit 411, or receives information from the eye position detection unit 411 that can estimate the eye position of the viewer 4. The positions of the left and right eyes of the viewer 4 are estimated. Information that can estimate the positions of the left and right eyes is, for example, the position of either the left or right eye, the position of the driver's seat of the vehicle 1, the position of the face of the viewer 4, the height of the sitting height, and the information not shown by the viewer 4. It may be an input value in the operation unit.

The notification necessity determination module 608 determines whether each stereoscopic image 200 displayed by the vehicle display system 10 is the content to be notified to the viewer 4. The notification necessity determination module 608 may acquire information from various other electronic devices connected to the I / O interface 32 and calculate the notification necessity. Further, the electronic device connected to the I / O interface 32 in FIG. 8 transmits information to the vehicle ECU 501, and the notification necessity determination module 608 detects (acquires) the notification necessity determined by the vehicle ECU 501 based on the received information. ) May. The "notification necessity" is, for example, the degree of danger derived from the degree of seriousness that can occur, the degree of urgency derived from the length of the reaction time required to take a reaction action, the vehicle 1 or the viewer 4 (or the viewer 4). It can be determined by the effectiveness derived from the situation of the other occupants of the vehicle 1), or a combination thereof (the index of the necessity of notification is not limited to these). That is, the notification necessity determination module 608 may determine whether to notify the viewer 4 and may select not to display a part or all of the stereoscopic image 200. The vehicle display system 10 does not have to have a function of estimating (calculating) the necessity of notification, and a part or all of the function of estimating the necessity of notification is a display of the vehicle display system 10. It may be provided separately from the control device 30.

The image position determination module 610 sets the stereoscopic image 200 at the determination position (observation position or predicted position) of the real object set by the real object position setting module 604 so that the stereoscopic image 200 is visually recognized in a specific positional relationship with the real object. Based on this, the coordinates of the stereoscopic image 200 (including at least the left-right direction (X-axis direction) and the up-down direction (Y-axis direction) when the viewer 4 looks forward from the driver's seat of the vehicle 1) are determined. In addition to this, the image position determination module 610 is based on the determination position of the real object set by the real object position setting module 604, before and after the viewer 4 sees the direction of the virtual image display area 100 from the driver's seat of the vehicle 1. The direction (Z-axis direction) may be determined. The image position determination module 610 adjusts the position of the stereoscopic image 200 based on the eye position of the viewer 4 detected by the eye position detection unit 511. For example, the image position determination module 610 determines the positions of the stereoscopic image 200 in the horizontal direction and the vertical direction so that the center of the stereoscopic image 200 is visually recognized so as to overlap the center of the real object. The "specific positional relationship" can be adjusted depending on the situation of the real object or the vehicle 1, the type of the real object, the type of the displayed image, and the like.

The image sizing module 612 may change the size of the stereoscopic image (AR image) 200 according to the position and / and size of the corresponding real object. For example, the image size determination module 612 can reduce the size of the stereoscopic image (AR image) 200 if the position of the corresponding real object is far away. Further, the image size determination module 612 can increase the size of the stereoscopic image (AR image) 200 if the size of the associated real object is large.

Further, the image size determination module 612 was detected by the type, number, or / and notification necessity determination module 608 of the real object that displays the stereoscopic image 200 detected by the real object-related information detection module 602 in association with each other. The size of the stereoscopic image 200 can be determined based on the magnitude of the (estimated) notification need.

Even if the image size determination module 612 has a function of predicting and calculating the size of the stereoscopic image (AR image) 200 to be displayed in the current display update cycle based on the size of the actual object a predetermined number of times in the past. Good. As a first method, the image sizing module 612 tracks the pixels of a real object between two past images captured by a camera (an example of an external sensor 507), for example using the Lucas-Kanade method. Then, the size of the real object in the current display update cycle may be predicted, and the size of the AR image may be determined according to the predicted size of the real object. As a second method, the rate of change in the size of the real object is obtained based on the change in the size of the real object between the two captured images in the past, and the size of the AR image is determined according to the rate of change in the size of the real object. You may. The method of estimating the size change of the real object from the viewpoint that changes in time series is not limited to the above, and is known to include, for example, an optical flow estimation algorithm such as the Horn-Schunkk method, the Boxon-Buxton, and the Black-Jepson method. You may use the method of.

The graphic module 614 describes the visual effects (eg, brightness, transparency, saturation, contrast, or other visual characteristics), size, display position, and distance (viewer 4 to stereoscopic image 200) of the stereoscopic image 200 to be displayed. Includes various known software components for changing the distance). The graphic module 614 has coordinates set by the image position determination module 610 (horizontal direction (X-axis direction) and vertical direction (Y-axis) when the viewer 4 sees the direction of the virtual image display area 100 from the driver's seat of the vehicle 1. The image data of the stereoscopic image 200 is generated and stored in the memory 36 so that the viewer 4 can see the image size set by the image size determination module 612 (including at least the direction). The graphic module 614 has an AR image on the HUD device 20 that changes the display position according to the position of the real object existing in the foreground 300, and / or a non-AR image that does not change the display position according to the position of the real object. Images containing, can be displayed. A part or all of the functions of the graphic module 614 may be provided separately from the display control device 30 of the vehicle display system 10.

The parallax image generation module 616 reads out the image data generated by the graphic module 614 and the positions of the left eye 4L and the right eye 4R of the viewer 4 acquired by the eye position detection module 606 from the memory 36, and is based on these. Then, the image data for the right eye and the image data for the left eye are generated. Specifically, the parallax image generation module 616 uses the parallax between the right eye and the left eye so that the stereoscopic image 200 is visually recognized at the display position included in the read image data when viewed from the eye position of the viewer 4. Then, the image data for the right eye and the image data for the left eye for controlling the liquid crystal panel 22 are generated. A part or all of the functions of the parallax image generation module 616 may be provided separately from the display control device 30 of the vehicle display system 10.

The display control module 618 sequentially switches between the right eye image and the left eye image and displays them on the liquid crystal panel 22 based on the right eye image data and the left eye image data generated by the parallax image generation module 616. The display control module 618 switches between a right-eye image and a left-eye image at a frequency of, for example, 120 Hz or higher. As a result, both the right-eye image and the left-eye image are visually recognized at a frequency of 60 Hz or higher, which reduces discomfort with respect to the image.

The light source control module 620 is based on the position of the left eye 4L and the position of the right eye 4R of the viewer 4 acquired by the eye position detection module 606, and the light source for the left eye toward the viewing range E where the position of the left eye 4L exists, and the light source for the right eye 4R. A light source for the right eye toward the viewing area E where the position exists is selected from the light source group 60, and the display control module 618 turns on the light source for the left eye as the image for the left eye is displayed on the liquid crystal panel 22. , The right eye light source is turned on in accordance with displaying the right eye image on the liquid crystal panel 22. Explaining this with reference to FIG. 6A, the light source control module 620 directs the image light 50 (illumination light) to the visual range E1 in which the left eye 4L is arranged in accordance with displaying the image for the left eye on the liquid crystal panel 22. The light source 61 is made to emit light, all the lights other than the first light source 61 are turned off (including reducing the brightness to the extent that the image is difficult to see in the corresponding viewing area E), and then the image for the right eye is displayed on the liquid crystal panel 22. The third light source 63 that directs the image light 50 (illumination light) to the viewing area E3 where the right eye 4R is arranged is turned off in accordance with the switching and display, and the lights other than the third light source 63 are turned off (corresponding viewing area E). It also includes reducing the brightness to the extent that the image is difficult to see.)

The light source control module 620 has the image light 50 (or the position of the left and right eyes of the viewer 4 (or the viewing area E where the left and right eyes of the viewer 4 are arranged) and the image light 50 (or the position of the eyes of the viewer 4) in the memory 36. Table data associated with each of the light sources 61 to 64 of the light source group 60 to which the illumination light is directed is stored in advance, and by reading this table data, the positions of the left and right eyes of the viewer 4 (or the viewer 4). Light sources 61 to 64 corresponding to the visual range E) in which the left and right eyes of the above are arranged are set. The positions of the left and right eyes of the viewer 4 for setting the light sources 61 to 64 include at least the coordinates in the left-right direction (X-axis direction), and preferably include the coordinates in the depth direction (Z-axis direction) (height). It may include the coordinates of the direction (Y-axis direction).

FIG. 9A is a timing chart showing the relationship between the image displayed by the liquid crystal panel 22 and the driving state of each of the light sources 61 to 64 in the arrangement of the eyes of the viewer shown in FIG. 6A. In the state shown in FIG. 6A, the left eye 4L of the viewer 4 is in the viewing area E1, the first light source 61 corresponds (the first light source 61 is the light source for the left eye), and the right eye 4R is in the viewing area E3. Yes, the third light source 63 corresponds (the third light source 63 is the light source for the right eye). Between time t11 and time t12 (the same applies between time t13 and time t14), the liquid crystal panel 22 displays an image for the left eye, and only the first light source 61, which is the light source for the left eye, has high brightness in the light source group 60. Light is emitted, and the other light sources 62, 63, and 64 are set to low brightness (including turning off). As a result, in the viewing areas E2, 403, 404 corresponding to the light sources 62, 63, 64, the image (image for the left eye) displayed on the liquid crystal panel 22 becomes difficult to see (not visible). Therefore, since it is difficult (not incident) that the image light of the left eye image is incident on the right eye 4R of the viewer 4 arranged in the view area E3, crosstalk is caused in which the image light of the left eye image is incident on the right eye 4R. It can be made less likely to occur. Next, between the time t12 and the time t13 (the same applies between the time t14 and the time t15), the liquid crystal panel 22 displays the image for the right eye, and only the third light source 63, which is the light source for the right eye, is displayed in the light source group 60. Light is emitted with high brightness, and the other light sources 61, 62, 64 are set to low brightness (including turning off). As a result, in the viewing areas E1, 402, 404 corresponding to the light sources 61, 62, 64, the image (image for the right eye) displayed on the liquid crystal panel 22 becomes difficult to see (not visible). Therefore, the image light of the image for the right eye is less likely to be incident (not incident) on the left eye 4L of the viewer 4 arranged in the view area E1, so that crosstalk is caused in which the image light of the image for the right eye is incident on the left eye 4L. It can be made less likely to occur.

FIG. 9B is a timing chart showing a modified example of the relationship between the image displayed by the liquid crystal panel 22 and the driving state of each of the light sources 61 to 64 in the arrangement of the eyes of the viewer shown in FIG. 6A. _{The period T DISP} between the time t21 and the time t22 _{(the same applies to the period T DISP} between the time t23 and the time t24), the liquid crystal panel 22 displays the image for the left eye, and is the light source for the left eye in the light source group 60. only during the period _{T DISP} shorter periods of time _{T LITE} the left-eye image only the first light source 61 is displayed, light is emitted at a high luminance, and sets the light source 62, 63, 64 otherwise a low luminance (including off) .. _{Subsequently, the period T DISP} between the time t22 and the time t23 _{(the same applies to the period T DISP} between the time t14 and the time t15), the liquid crystal panel 22 displays the image for the right eye, and the light source group 60 for the right eye. The third light source 63, which is a light source, is made to emit light with high brightness only during a period T _{LITE shorter than the period T DISP} , and the other light sources 61, 62, 64 are set to low brightness (including turning off). Therefore, all the light sources are turned off (that is, black is displayed) before and after the time t22, t23, and t24 when the liquid crystal panel 22 switches between the left-eye image and the right-eye image. As a result, when the display timing of the liquid crystal panel 22 and the lighting timing of the light sources 61, 62, 63, 64 deviate from each other, it is possible to prevent the light of the unintended image from heading to the unintended view range E. In the end, crosstalk can be made less likely to occur.

FIG. 9C is a timing chart showing a modified example of the relationship between the image displayed by the liquid crystal panel 22 and the driving state of each of the light sources 61 to 64 in the arrangement of the eyes of the viewer shown in FIG. 6A. In the example of FIG. 9C, the left eye image or the right eye image is not displayed while the liquid crystal panel 22 switches between the left eye image and the right eye image (time t31 to t32, time t33 to t34, time t35 to t36). During this black display, the left-eye light source and the right-eye light source are switched on and off (that is, the display is black). This also prevents the light of the unintended image from heading to the unintended view range E when the display timing of the liquid crystal panel 22 and the lighting timing of the light sources 61, 62, 63, 64 deviate from each other. It can be done, and by extension, crosstalk can be made less likely to occur.

FIG. 9D is a timing chart showing a modified example of the relationship between the image displayed by the liquid crystal panel 22 and the driving state of each of the light sources 61 to 64 in the arrangement of the eyes of the viewer shown in FIG. 6A. In the example of FIG. 9D, the black image that does not display the left eye image or the right eye image while the liquid crystal panel 22 switches between the left eye image and the right eye image (time t41 to t42, time t43 to t44, time t45 to t46). The display is displayed, and during this black display, all the light sources are turned off (that is, the black display is displayed). This also prevents the light of the unintended image from heading to the unintended view range E when the display timing of the liquid crystal panel 22 and the lighting timing of the light sources 61, 62, 63, 64 deviate from each other. It can be done, and by extension, crosstalk can be made less likely to occur.

FIG. 10 is a flowchart showing a process when the HUD device 20 displays a stereoscopic image. In step S1, the processor 34 detects (or estimates) the position of the left eye 4L and the position of the right eye 4R of the viewer 4 by executing the eye position detection module 606.

In step S2, the processor 34 executes the parallax image generation module 616 based on the image data generated by the graphic module 614 and the positions of the left eye 4L and the right eye 4R of the viewer 4 acquired in step S1. , The image data for the left eye and the image data for the right eye are generated.

In step S3, the processor 34 sets the light source for the left eye and the light source for the right eye based on the positions of the left eye 4L and the right eye 4R of the viewer 4 acquired in step S1 by executing the light source control module 620. (Step S3). The order of steps S2 and S3 may be reversed, or they may be executed at the same time.

In step S4, the processor 34 and the image processing circuit 38 alternately display the left-eye image and the right-eye image on the liquid crystal panel 22, and in synchronization with the display, the left-eye light source and the right-eye light source are alternately turned on and off. ..

The operation of the processing process described above can be carried out by executing one or more functional modules of an information processing device such as a general-purpose processor or a chip for a specific purpose. All of these modules, combinations of these modules, and / and combinations with known hardware that can replace their functionality are within the scope of the protection of the present invention.

The functional blocks of the vehicle display system 10 are optionally performed by hardware, software, or a combination of hardware and software to implement the principles of the various embodiments described. The functional blocks described in FIG. 8 may be optionally combined or one functional block separated into two or more subblocks in order to implement the principles of the embodiments described. It will be understood by those skilled in the art. Accordingly, the description herein optionally supports any possible combination or division of functional blocks described herein.

In the vehicle display system 10 of the present embodiment, some or all the functions of the modules 602 to 620 are provided separately from the display control device 30 of the vehicle display system 10 (for example, the vehicle ECU 501). May be good.

In the above embodiment, of the plurality of viewing areas E, one is the left visual area EL on which the image for the left eye is displayed, and one is the right visual area ER on which the image for the right eye is displayed. , But not limited to this, the image for the left eye and / or the image for the right eye may be displayed in a plurality of view areas E. When the left eye image and / or the right eye image is displayed in a plurality of view areas E, the image is also displayed in one or more view areas E adjacent to the left view area EL (or ER) in which the eyes are present. Is preferably displayed. In this way, by displaying the image in one or a plurality of view areas E adjacent to the left view area EL (or ER) in which the eyes exist, the position of the eyes of the viewer 4 can be moved. Even when the view area E is straddled, since the desired image is already displayed in the view area E, there is an advantage that the viewer 4 can visually recognize the desired image without delay.

11A to 11D are diagrams showing a mode in which the left-eye image and / or the right-eye image in the first display mode are displayed in a plurality of viewing areas E. In these figures, the reference numeral EL indicates the viewing area E in which the left eye 4L is located, and the reference numeral ER indicates the viewing area E in which the right eye 4R is located. Although the left-eye image and the right-eye image are shown to be displayed at the same time in FIGS. 11A to 11D, the left-eye image shown in FIGS. 11A to 11D and the left-eye image and FIGS. 11A to 11D show. The described right-eye image may be displayed in a time-division manner. That is, one frame may include one or more fields for displaying the left eye image and one or more fields for displaying the right eye image. In FIGS. 11A to 11D, the reference numerals ELI1 and ERI1 indicate the view areas where the images are displayed between the left eye 4L and the right eye 4R, and the reference numerals ELO1, ELO2, ELO3, ... (The numbers in the code are in ascending order from the side closer to the right visual area ER), and the codes ERO1, ERO2, ERO3, ... (The numbers in the code are in ascending order from the side closer to the right visual area ER) are the left eye 4L and the right eye 4R. Indicates the viewshed in which the image is displayed, not between.

See FIG. 11A. In some embodiments, the display control device 30 comprises a left-viewed area EL (ER) in which the eyes are present, left-viewed areas ELI1, ELO1 (ERI1, ERO1) adjacent to the left-viewed area EL (ER), and an image for the left eye (image for the right eye). May be displayed. That is, the display control device 30 has a viewing area E5 (EL) in which the left eye 4L exists, a viewing area E6 (ELI1) inside the viewing area E5 (EL) in which the left eye 4L exists (right eye 4R side), and a left eye 4L. The left eye image is displayed in the view area E4 (ELO1) outside the view area E5 (EL) where the right eye 4R exists (the side opposite to the right eye 4R), and the view area E8 (ER) in which the right eye 4R exists. The view E7 (ERI1) inside the view area E8 (ER) where the right eye 4R exists (on the left eye 4L side) and the view outside the view area E8 (ER) where the right eye 4R exists (opposite the left eye 4L side). The image for the right eye is displayed in the area E9 (ERO1).

Next, refer to FIG. 11B. In some embodiments, the display control device 30 may display a left-eye image or a right-eye image in all view areas E. Specifically, the image for the left eye is displayed in the viewing areas E1 to E6 on the left eye 4L side, and the image for the right eye is displayed in the viewing areas E7 to E12 on the right eye 4R side with the boundary between the left eye 4L and the right eye 4R. It may be displayed.

Further, as shown in FIG. 11C, in some embodiments, the display control device 30 displays the left-viewed area ELO1 (E4) on which the left-eye area image based on the left-viewed area EL (E5) where the left-eye 4L is located is displayed. ), The arrangement of ELO2 (E3) and the arrangement of the left visual area ELI1 (E7) and ELO1 (E9) on which the right eye image based on the left visual area EL (E8) where the right eye 4R is located are different. You may let me.

Further, as shown in FIG. 11D, in some embodiments, the display control device 30 determines the number (2) of the left visual area EL (E5) and ELO1 (E4) on which the left eye image is displayed, and the right eye image. The number (3) of the right visual area ER (E8), ERI1 (E7), and ERO1 (E8) in which is displayed may be different.

1 ... Vehicle, 2 ... Front windshield, 4 ... Viewer, 4L ... Left eye, 4R ... Right eye, 5 ... Dashboard, 10 ... Vehicle display system, 20 ... HUD device (head-up display device), 21 ... 3D display Instrument, 22 ... LCD panel, 24 ... Backlight module, 25 ... Relay optical system, 26 ... First mirror, 28 ... Second mirror, 30 ... Display control device, 32 ... I / O interface, 34 ... Processor, 36 ... Memory, 38 ... image processing circuit, 40 ... viewpoint detection camera, 50 ... image light, 50p ... optical axis, 51 ... first image light, 52 ... second image light, 53 ... third image light, 60 ... light source group, 70 ... Illumination optical system, 71 ... 1st lens, 73 ... 2nd lens, 75 ... Diffusing plate, 90 ... Virtual image optical system, 95 ... Condensing optical system, 100 ... Virtual image display area, 200 ... Stereoscopic image, 201 ... Far Stereoscopic image, 202 ... Near stereoscopic image, 310 ... Road surface, 411 ... Eye position detection unit, 501 ... Vehicle ECU, 503 ... Road information database, 505 ... Own vehicle position detection unit, 507 ... External sensor, 509 ... Line-of-sight direction detection unit 511 ... Eye position detection unit, 513 ... Mobile information terminal, 520 ... External communication connection device, 602 ... Real object related information detection module, 604 ... Real object position setting module, 606 ... Eye position detection module, 608 ... Notification necessity Judgment module, 610 ... Image position determination module, 612 ... Image size determination module, 614 ... Graphic module, 616 ... Disparity image generation module, 618 ... Display control module, 620 ... Light source control module, E ... Visibility.

Claims (4)

A group of light sources (60) in which a plurality of light sources are arranged, and
A liquid crystal panel (22) that spatially modulates the illumination light from the light source group (60) and emits the image light of the left eye image and the right eye image.
A condensing optical system (95) for condensing the image light from the liquid crystal panel (22) into a viewing area (E) shifted according to the position of the light source in the light source group (60).
Acquire or estimate the position of the left eye and the position of the right eye of the viewer,
The image light is focused on the left eye light source (60L) in which the image light is focused in the left visual region (EL) where the left eye is arranged, and in the right visual region (ER) where the right eye is arranged. The light source for the right eye (60R) and the light source for the right eye (60R) are alternately increased in emission intensity.
When the liquid crystal panel (22) displays the image for the left eye, the emission intensity of the light source for the left eye (60L) is increased.
When the liquid crystal panel (22) displays the image for the right eye, it is composed of a display control unit (30) that increases the emission intensity of the light source for the right eye (60R).
Head-up display device. The display control unit (30)
The light source at which the image light is focused in the viewing area (E) without the left and right eyes of the viewer is turned off.
The head-up display device according to claim 1. The condensing optical system (95)
A lens (71) having positive optical power arranged on the optical path of the illumination light between the light source group (60) and the liquid crystal panel (22).
Includes a concave mirror (26) having positive optical power arranged on the optical path of the image light between the liquid crystal panel (22) and the view area (E).
The head-up display device according to claim 1. The lens (71) is a lenticular lens.
The head-up display device according to claim 3.

Images