WO2023228752A1

WO2023228752A1 - Image display device

Info

Publication number: WO2023228752A1
Application number: PCT/JP2023/017619
Authority: WO
Inventors: 凌一竹内
Original assignee: 京セラ株式会社
Priority date: 2022-05-27
Filing date: 2023-05-10
Publication date: 2023-11-30
Also published as: JP2023174351A

Abstract

Provided is an image display device that can avoid a state of incorrect perception by causing an observer to perceive a shielding portion as being transparent. The image display device comprises: an image data processing unit for generating a first image that is viewed by one of the left eye and the right eye of an observer and is generated in a range corresponding to a shielding portion obstructing the view of the observer and a second image that is viewed by the other of the left eye and the right eye of the observer and is generated in the range corresponding to the shielding portion, the first image being generated on the basis of first image data obtained by capturing an image around the observer by means of a first imaging unit; an image display unit that displays the first image and the second image in the shielding portion; and a sightline detection unit that detects the focus position of the observer. The second image has a parallax with respect to the first image, and the image display unit displays, in the shielding portion, an image having no parallax instead of the first image and the second image on the basis of the focus position of the sightline of the observer.

Description

image display device

Cross-reference of related applications

This application claims priority to Japanese Patent Application No. 2022-087156 (filed on May 27, 2022), and the entire disclosure of that application is incorporated herein by reference.

The present disclosure relates to an image display device. The present disclosure particularly relates to an image display device that can display captured images of the surroundings of a vehicle on a shielding part, thereby making the driver perceive that the scenery outside the vehicle is connected.

Conventionally, a technique is known in which graphics superimposed on a real environment are displayed in a vehicle or the like (for example, see Patent Document 1). Further, there is a known transparency technology that utilizes a reflex projection technology to allow an observer to perceive an image as if the object is transparent. In this transparency technique, a computer creates a left-eye image and a right-eye image that are perceived by the observer's left and right eyes, respectively, based on images captured by a camera installed outside the vehicle, for example. The left eye image and the right eye image are projected onto a shielding part that blocks the line of sight, such as a vehicle pillar or dashboard, and the driver, who is an observer, perceives the image of the scenery outside the vehicle as a 3D image, which becomes a blind spot. It is perceived as if parts of it are transparent and connected to the scenery of the outside world.

Japanese Patent Application Publication No. 2015-194473

An image display device according to an embodiment of the present disclosure includes:
A first imaging unit captures an image of the surroundings of the observer, and blocks the observer's field of view as seen by one of the observer's left eye and right eye based on the first image data obtained by capturing the image. an image data processing unit that generates a first image in a range corresponding to the shielding part and a second image in a range corresponding to the shielding part, which is seen by the other of the observer's left eye and right eye;
an image display unit that displays the first image and the second image on the shielding unit;
a line of sight detection unit that detects the focus position of the observer;
the second image has parallax with respect to the first image,
The image display section displays a parallax-free image, which is an image without parallax, on the shielding section, instead of the first image and the second image, based on the focus position of the observer's line of sight.

An image display device according to an embodiment of the present disclosure includes:
an imaging unit that captures an image of an observer and the surroundings of the observer and outputs image data obtained by capturing the image;
an image display unit that causes the observer to perceive a parallax image reflecting the parallax between the left eye and the right eye of the observer based on the image data in a shielding unit that blocks the observer's field of view;
a line-of-sight detection unit that detects the focus position of the observer's line of sight;
The image display section allows the observer to perceive a parallax-free image that does not reflect parallax between the left eye and right eye of the observer when the focus position of the observer's line of sight is on the shielding section.

FIG. 1 is a plan view schematically showing an example of an image display device according to an embodiment of the present disclosure. FIG. 2 is a partially enlarged sectional view showing the configuration of the screen and the diffuser plate. FIG. 3 is a plan view schematically showing a vehicle equipped with an image display device. FIG. 4 is a side view schematically showing a vehicle equipped with an image display device. FIG. 5 is a block diagram showing the configuration of the image display device. FIG. 6 is a flowchart for explaining the operation of the control section. FIG. 7 is a diagram showing a state where no parallax image is displayed on the shielding section. FIG. 8 is a diagram showing a state in which a parallax image is displayed on the shielding part and can be perceived correctly. FIG. 9 is a diagram illustrating a state in which a parallax image is displayed on a shielding part and cannot be perceived correctly. FIG. 10 is a diagram for explaining the relationship between the observer's line of sight, the convergence angle, and the shielding part. FIG. 11 is a flowchart of a process for generating a first image and a second image or an image without parallax. FIG. 12 is a diagram illustrating an image without parallax.

An image display device according to an embodiment of the present disclosure will be described below with reference to the drawings. In each figure, the same or corresponding parts are given the same reference numerals. In the following description of the embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

FIG. 1 is a plan view schematically showing an image display device 1 according to the present embodiment. FIG. 2 is a partially enlarged sectional view showing the configuration of the retroreflective screen 11 and the diffuser plate 16 provided in the image display device 1. As shown in FIG. FIG. 3 is a plan view schematically showing a vehicle 2 on which the image display device 1 is mounted. FIG. 4 is a side view schematically showing a vehicle 2 on which the image display device 1 is mounted.

The image display device 1 includes an imaging section, an image data processing device 33 (see FIG. 5) including a first image processing section 8 and a second image processing section 9, an image display section, and a line of sight detection section. The image data processing device 33 is sometimes referred to as an image data processing section.

The imaging unit images the observer and the surroundings of the observer, and outputs the image data obtained by capturing the image. In this embodiment, the imaging unit includes a front exterior camera 3a, a rear exterior camera 3b, and an interior camera 6. The front exterior camera 3a and the rear exterior camera 3b capture images of the surrounding scenery of the vehicle 2. Hereinafter, the external cameras including the front external camera 3a and the rear external camera 3b may be referred to as a first imaging section. Further, image data obtained by imaging the surroundings of the observer by the first imaging unit may be referred to as first image data. The in-vehicle camera 6 images the driver 5. The image data of the in-vehicle camera 6 detects the positions of the left eye EL and right eye ER of the driver 5 who is an observer seated in the driver's seat 4 of the vehicle 2, and calculates the positions of the left eye EL and right eye ER and the position of the pupil. It is used to detect the line of sight of the driver 5 from the position. In the following, the in-vehicle camera 6 may be referred to as a second imaging section. Moreover, the image data obtained by photographing the observer by the second imaging unit may be referred to as second image data. Here, the imaging unit may include only one of the front exterior camera 3a and the rear exterior camera 3b.

The first image processing section 8 of the image data processing section applies an image to the shielding section 7, which is seen by one of the left eye EL and the right eye ER, based on the image data output from the front exterior camera 3a and the rear exterior camera 3b. A first image of the corresponding range is generated. Here, the shielding part 7 is an object that blocks the observer's line of sight, and is an object that blocks the observer's field of view when looking outside the vehicle from the left eye EL and right eye ER. For example, the first image may be an image in the range corresponding to the shielding part 7 that is seen by the right eye ER.

The second image processing section 9 of the image data processing section applies an image to the shielding section 7, which is seen by the other of the left eye EL and the right eye ER, based on the image data output from the front exterior camera 3a and the rear exterior camera 3b. A second image of the corresponding range is generated. For example, the image of the range corresponding to the shielding part 7, which is seen by the left eye LR, may be set as the second image. The second image has parallax with respect to the first image. Similarly, the first image has parallax with respect to the second image. Therefore, even if the first image and the second image contain the same object, since the viewpoints from which the object is viewed are different, the position and shape of the object in the images differ depending on the parallax. Here, the range corresponding to the shielding part 7 that can be seen by the left eye EL refers to the range that would be visible by the left eye EL if the shielding part 7 did not exist. Similarly, the range corresponding to the shielding part 7 that can be seen by the right eye RL refers to the range that would be visible by the right eye RL if the shielding part 7 did not exist.

The image display section displays the first image and the second image on the shielding section 7. The image display unit may display a parallax image, which is an image in which the first image and the second image are mixed, on the shielding unit 7. The image display unit displays the parallax image, thereby allowing the viewer to perceive an image that reflects the parallax. In this embodiment, the image display section includes a display device 10. Here, as the shielding part 7 in the vehicle 2, a dashboard, a door, a pillar, a back seat 23, etc. can be enumerated. By displaying the image taken by the camera 3 outside the vehicle and subjected to image processing on the shielding portion 7, the driver 5, who is an observer, can perceive that the image is connected to the scenery outside the vehicle.

The line of sight detection unit detects the focus position of the observer. The line of sight detection unit may detect the line of sight of the observer based on image data of the observer. In this embodiment, image data captured by the observer is output from the in-vehicle camera 6. Although details will be described later, in this embodiment, the line of sight detection unit also detects the convergence angle of the observer. The image data processing unit determines that the observer is unable to correctly perceive the parallax image in which the first image and the second image are mixed when the detected line of sight and convergence angle of the observer satisfy a certain condition. judge. Then, the image data processing section generates a parallax-free image, and the image display section displays the parallax-free image on the shielding section 7. A parallax-free image is an image that does not reflect the viewer's parallax. Here, the convergence angle is the angle at which the left and right lines of sight intersect from the eyeball.

As shown in FIGS. 1 and 2, the display device 10 includes a retroreflective screen 11 and a diffuser plate 16 provided close to the retroreflective screen 11. The diffuser plate 16 may be attached and laminated to the surface of the retroreflective screen 11 on the side facing the viewer. The display device 10 includes a dashboard display device 10a provided on the dashboard, a right side pillar display device 10b provided on the right side pillar, a left side pillar display device 10c provided on the left side pillar, and a rear seat display device 10a. The backsheet display device 10d may include a backsheet display device 10d provided with 22 backsheets 23.

The display device 10 also includes, as a projection section, a first projection section that projects the first image onto the retroreflective screen 11 and a second projection section that projects the second image onto the retroreflective screen 11. Ru. For example, the projection section (right side pillar projection section 12) of the right side pillar display device 10b attached to the right side pillar includes a first projection section 12R that projects a first image and a second projection section 12L that projects a second image. It consists of: Here, the

display devices

10a, 10b, 10c, and 10d have flexibility, and are bonded to each shielding portion 7 with adhesive or the like in a state of being flexibly curved according to the ups and downs of each shielding portion 7. . Since each projection section has the same configuration, the right side pillar projection section 12 will be explained in detail as an example.

The first projection unit 12R includes a liquid crystal display device 13R that displays a first image, and a first projection lens 14R that projects the image light of the first image emitted from the liquid crystal display device 13R onto the retroreflective screen 11. You may do so. The second projection unit 12L includes a liquid crystal display device 13L that displays a second image, and a second projection lens 14L that projects the image light of the second image emitted from the liquid crystal display device 13L onto the retroreflective screen 11. You may do so. Each of the liquid

crystal display devices

13R and 13L may include a transmissive liquid crystal display element and a backlight device that emits light to the back surface of the liquid crystal display element. Here, an LED light emitting display device may be used instead of the liquid crystal display device. Each

projection lens

14R, 14L may be configured by a combination of a plurality of lenses, respectively, so that the first image and the second image are formed on the retroreflective screen 11 with parallax. Here, although the driver 5 is illustrated as the observer, the observer may also be a fellow passenger sitting in the front passenger seat. The first projection unit 12R may be arranged, for example, on the right side of the headrest so that its exit pupil is at the same height as and near the right eye ER of the observer. Similarly, the second projection section 12L may be arranged, for example, on the left side of the headrest so that its exit pupil is at the same height as and near the left eye EL of the observer.

The backseat projection section 24 and the dashboard projection section 25 are also configured similarly to the right side pillar projection section 12, and the backseat projection section 24 displays an image corresponding to the range covered by the backseat 23 on the backseat display device 10d. to project. Further, the dashboard projection unit 25 projects an image corresponding to the range covered by the dashboard onto the dashboard display device 10a.

The dashboard projection unit 25 may be attached to the center of the ceiling of the vehicle 2, for example. The back seat projection section 24 may be attached, for example, to the upper part of the backrest seat of the driver's seat 4.

The retroreflective screen 11 has retroreflectivity and reflects incident light in the direction of incidence. The image light of the first image and the image light of the second image emitted from the first projection lens 14R and the second projection lens 14L are directed to the first projection lens 14R and the second projection lens 14L by the retroreflective screen 11. reflected. Therefore, the image light of the first image and the image light of the second image, which are overlapped (mixed) on the retroreflective screen 11, are perceived separately at the observer's position. Furthermore, in this embodiment, a diffuser plate 16 is disposed on the viewer-side surface of the retroreflective screen 11. The diffusing plate 16 has a diffusing ability to direct the light reflected by the retroreflective screen 11 to both eyes of the viewer. For example, when the dashboard projection unit 25 is located above the observer, the dashboard display device 10a may use a diffusion plate 16 with a large diffusion capacity in the vertical direction and a small diffusion capacity in the horizontal direction. Here, in order to suppress image confusion such as an image for the right eye ER entering the left eye EL as much as possible, and to allow the viewer to perceive a clear stereoscopic image, the diffusion power in the left and right direction is higher than the diffusion power in the vertical direction. It is preferable that it is also small. The diffuser plate 16 may be, for example, a holographic optical element bonded onto the reflective surface of the retroreflective screen 11.

As shown in FIG. 2, the retroreflective screen 11 may have a configuration in which a plurality of minute glass beads 11a having a diameter of, for example, 20 μm or more and 100 μm or less are arranged on a reflective film 11b. The image light projected onto the retroreflective screen 11 enters the glass beads 11a, is refracted on the surface of the glass beads 11a, reaches the back surface on the reflective film 11b side, and is reflected by the reflective film 11b. The light reflected by the reflective film 11b is refracted again at the back surface of the glass bead 11a, reaches the surface of the glass bead 11a, is separated from the incident path of the incident light by a minute distance less than the diameter of the glass bead 11a, and is reflected by the incident light. Since the light travels along a parallel optical path, retroreflection is achieved.

As described above, the image light of the first image for the right eye ER and the image light of the second image for the left eye EL, which overlap on the retroreflective screen 11, are separated at the observer's position, The light enters the right eye ER and the left eye EL separately. The driver 5 who is an observer can perceive a three-dimensional image from the mixed image of the image light of the first image and the image light of the second image. A mixed image of the image light of the first image and the image light of the second image reflecting the parallax of the observer is called a parallax image.

Here, in order to make the scenery outside the vehicle seen through the windshield and rear window glass appear to be connected to the image displayed on the shielding part 7, calculations regarding the reflexive projection are performed using a coordinate system based on the vehicle body. may be executed. For example, a coordinate system in which the vehicle length direction is the X axis, the vehicle width direction is the Y axis, and the vehicle height direction is the Z axis is used, and the positions of the left eye EL and right eye ER of the driver 5 are determined by the coordinates in this coordinate system. may be defined as The positions of both eyes of the driver 5 are determined in a coordinate system based on the vehicle body, and calculations regarding reflexive projection are performed, thereby ensuring continuity between the scenery outside the vehicle and the image displayed on the shielding part 7. can have. Furthermore, by detecting the positions of both eyes of the driver 5, it is possible to flexibly follow differences in body shape and posture of the driver 5 and display an image.

Further, the front exterior camera 3a installed at the front of the vehicle 2 and the rear exterior camera 3b installed at the rear of the vehicle 2 may be cameras equipped with fisheye lenses. By using a fisheye lens, it is possible to image the scenery outside the vehicle over a wide range of solid angles. Here, the number of cameras 3 outside the vehicle is not limited, and may be one, for example, or three or more. Moreover, the installation location of the vehicle exterior camera 3 is not limited as long as it is possible to image the exterior of the vehicle. That is, the vehicle exterior camera 3 may be installed outside the vehicle or may be installed inside the vehicle. Further, the in-vehicle camera 6 is installed at a position where it can image the driver 5, such as a position adjacent to a room mirror. The camera outside the vehicle 3 and the camera inside the vehicle 6 may be, for example, CCD cameras, but are not limited to a specific type of camera. Here, a line-of-sight detection unit is configured including the in-vehicle camera 6, the line-of-sight recognition device 31, and the in-vehicle camera control device 37.

FIG. 5 is a block diagram showing the configuration of the image display device 1. Image data of images captured by the front exterior camera 3 a and the rear exterior camera 3 b are sent to the exterior camera control device 35 . The vehicle exterior camera control device 35 constitutes a part of the image display device 1 . The vehicle exterior camera control device 35 performs necessary signal processing (analog-to-digital conversion, for example) on the image data and outputs it to the image data processing device 33.

Furthermore, in this embodiment, the image display device 1 includes a seating sensor 36 that detects whether or not the driver 5 is seated. The seating sensor 36 is provided in the driver's seat 4. The seating sensor 36 may be constituted by a known load sensor or limit switch.

When the driver 5 is seated in the driver's seat 4, the seat sensor 36 installed in the driver's seat 4 detects that the driver 5 is seated. The detection result of the seating sensor 36 is sent to the line-of-sight recognition device 31, and the line-of-sight detection unit starts measuring the positions of the driver's 5 eyes. The line of sight recognition device 31 extracts the positions of the left eye EL and right eye ER and the pupil position of the driver 5 from the captured image of the in-vehicle camera 6 through image recognition processing, and calculates the convergence angle and line of sight of the driver 5. A known method may be used to calculate the vergence angle and line of sight. The line of sight recognition device 31 may treat the eyeball as a sphere and calculate the convergence angle and line of sight using the deviation of the position of the pupil from the reference position (eyeball angle). The calculation result of the line of sight recognition device 31 is output to the image data processing device 33. The image data processing device 33 executes image processing to make the shielding portion 7 appear transparent, and the display control device 39 controls the display device 10 based on the result of the image processing.

Furthermore, in the image display device 1, a control unit 50 is configured including an external camera control device 35, an image data processing device 33, an in-vehicle camera control device 37, a line of sight recognition device 31, and a display control device 39. The control unit 50 controls the image display device 1 . The control unit 50 is realized by a processor such as an electronic control unit (ECU) as a hardware resource, and a computer-readable program as a software resource. Control unit 50 may include one or more processors. The processor may include a general-purpose processor that loads a specific program and executes a specific function, and a dedicated processor that is specialized for specific processing. The dedicated processor may include an application specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include an FPGA (Field-Programmable Gate Array). The control unit 50 may be either an SoC (System-on-a-Chip) or an SiP (System In-a-Package) in which one or more processors cooperate. The control unit 50 includes a storage unit, and may store various information or programs for operating each component of the image display device 1 in the storage unit. The storage unit may be composed of, for example, a semiconductor memory.

FIG. 6 is a flowchart for explaining the operation of the control unit 50. When the driver 5 is seated in the driver's seat 4 and starts driving the vehicle 2 by pressing a start button or the like, the seating sensor 36 detects that the driver 5 is seated in the driver's seat 4 . Further, the vehicle exterior camera control device 35 operates each

vehicle exterior camera

3a, 3b to image the surroundings of the vehicle 2 (step S1). The in-vehicle camera control device 37 operates the line-of-sight recognition device 31 and the in-vehicle camera 6, and the in-vehicle camera 6 starts capturing an image. The line of sight recognition device 31 detects the line of sight of the driver 5 based on the image captured by the in-vehicle camera 6 (step S2). The image data processing device 33 cuts out the image to be projected onto the shielding part 7 from the image captured by the camera 3 outside the vehicle, and generates a first image and a second image (step S3). In step S3, if the observer's line of sight and the convergence angle satisfy a certain condition, a parallax-free image, which is an image without parallax, is generated instead of the first image and the second image. Details of the parallax-free image will be described later. Moreover, when the first image and the second image are generated, the display control device 39 causes the shielding unit 7 to display the first image and the second image (step S4). The display control device 39 causes the right side pillar display device 10b attached to the right side pillar, for example, to display an image that is originally blocked by the right side pillar and is not visible to the driver 5. Here, if a parallax-free image is generated in step S3, the parallax-free image is displayed in step S4.

Here, the image display device 1 according to the present embodiment is in a state where it cannot correctly perceive an image in which the first image and the second image are mixed when the observer's line of sight and the convergence angle satisfy certain conditions. It is determined that the image without parallax is displayed on the shielding unit 7. In order for the image of the scenery projected on pillars, dashboards, etc. to be correctly perceived as connected to the scenery of the outside world, the observer's focus must be on the scenery, that is, the viewer must be able to look into the distance. is necessary. However, when looking at structures inside a car, such as pillars and dashboards, the observer may end up focusing on the structures themselves, and may not be able to correctly perceive the projected image. In the image display device 1 according to the present embodiment, when the observer's line of sight is within the range of the shielding unit 7 and the convergence angle of the observer detected by the line of sight detection unit is equal to or greater than the threshold, It is determined that the object cannot be perceived correctly. Then, when the driver 5 who is an observer cannot see a double image, the display of the mixed image of the first image and the second image is stopped and the image without parallax is displayed in a state where the driver 5 cannot perceive it correctly. (misidentification of objects) and can help improve driving comfort. That is, the image display device 1 according to the present embodiment allows the viewer to perceive the shielding portion 7 as being transparent, thereby avoiding a situation where the viewer cannot perceive it correctly.

Here, such display processing of a parallax-free image is executed by the image data processing device 33 regardless of whether the display target is the left or right side pillar. Further, display processing of a non-parallax image may be performed with the dashboard, door, and back seat 23 as display targets. It is particularly suitable for use in displaying pillars because objects that were conventionally in blind spots can be perceived as if they were made transparent without being misrecognized.

Hereinafter, the display process of a parallax-free image will be explained in detail with reference to the drawings. FIG. 7 is a diagram showing a state in which no parallax images are displayed on the shielding unit 7. When a parallax image (an image in which the first image and the second image are mixed) is not displayed on the right side pillar display device 10b, only the building visible through the windshield 42 and the right window glass 46 is visible. FIG. 8 is a diagram showing a state in which a parallax image is displayed on the shielding unit 7 and can be perceived correctly by the observer. When the parallax image is displayed and the observer focuses to look far away (that is, outside the vehicle), the observer can see the object 44 on the right side pillar display device 10b of the right side pillar 40. Perceive images. On the other hand, FIG. 9 is a diagram showing a state in which a parallax image is displayed on the shielding unit 7 and the observer cannot perceive it correctly. When a parallax image is displayed and the viewer focuses on right side pillar 40 itself or in front of it, object 44 is perceived as a double image. In other words, the actual scene of the outside world is not connected, and the image displayed on the right side pillar display device 10b cannot be perceived as a continuous transparent image. Therefore, from the viewpoint of driving comfort, when the image display device 1 detects that the observer is unable to perceive correctly based on the focus position of the line of sight, the image display device 1 displays the first image and the second image instead of the first image and the second image. , display a parallax-free image.

In step S3 of FIG. 6, the image data processing device 33 determines whether the observer is in a state where the observer cannot perceive correctly based on the detected line of sight and convergence angle. FIG. 10 is a diagram for explaining the relationship between the observer's line of sight, the convergence angle, and the shielding part 7. The line of sight and the convergence angle are calculated by the line of sight recognition device 31 based on the image captured by the in-vehicle camera 6, as described above. The convergence angle is indicated by θ in FIG. Further, the display surface is a surface on which the first image and the second image of the shielding section 7 are displayed, as shown in FIG. The image data processing device 33 determines whether the observer's line of sight is within the range of the shielding part 7. The range of the shielding part 7 is the width on the display surface. If the line of sight is within the range sandwiched between the left end 7A and right end 7B of the shielding part 7, it is determined that the observer is looking at the shielding part 7. Here, it is determined whether the line of sight of the right eye ER and the line of sight of the left eye EL are within the range of the shielding part 7, and when at least one line of sight is not within the range of the shielding part 7, the observer It may be determined that the person is not looking at the shielding part 7. Further, as another example, one of the line of sight of the right eye ER and the line of sight of the left eye EL may be used as a representative to determine whether the line of sight is within the range of the shielding part 7. When the observer is looking at the shielding part 7, the image data processing device 33 next determines whether the observer is looking into the distance. When the convergence angle of the observer is equal to or greater than the threshold, the image data processing device 33 determines that the observer is looking at the display surface or something in front of the display surface and is unable to correctly perceive the image based on the parallax image. . When the observer is in a state where he or she cannot perceive correctly, the image data processing device 33 generates a parallax-free image. Further, the image display unit displays a parallax-free image on the shielding unit 7. Here, the above threshold value is determined based on the positional relationship between the positions of both eyes of the observer and the display surface of the shielding section 7. The threshold value may be determined based on an image captured by the in-vehicle camera 6, for example, when the vehicle 2 starts driving.

FIG. 11 is a flowchart for explaining details of the process (corresponding to step S3 in FIG. 6) of generating the first image and the second image or the non-parallax image. The image data processing device 33 generates a parallax-free image when the detected line of sight of the observer is within the range of the shielding unit 7 (YES in step S11) and the convergence angle is greater than or equal to the threshold (YES in step S12). is generated (step S13). If the detected line of sight of the observer is not within the range of the shielding unit 7 (NO in step S11), or if the convergence angle is smaller than the threshold (NO in step S12), the image data processing device 33 processes the first image. and generates a second image (step S14). The image display unit displays the generated image (“parallax-free image” or “first image and second image”) on the shielding unit 7.

Here, the parallax-free image may be colorless and transparent, or may be a monochrome uniform image. The single color may be an achromatic color such as white or gray. The parallax-free image may be an image outside the vehicle. The parallax-free image may be an image in a range corresponding to the shielding part 7 that is seen by either the right eye ER or the left eye LR. The parallax-free image may be an image in a range corresponding to the shielding part 7 that is seen from a virtual viewpoint between the right eye ER and the left eye LR. From the viewpoint of alerting, the non-parallax image may include text information or content. The text information may be, for example, a warning urging the user to look far away, a notification urging the user to focus on the outside of the vehicle, or a notification urging the user not to focus on the shielding portion 7 . When displaying an image without parallax, the first image and the second image may be displayed on the shielding unit 7 as the same image. For example, as shown in FIG. 12, a parallax-free image including a warning "Check the front outside the vehicle" is displayed on the shielding unit 7, so that the driver 5 who is an observer tries to focus on a distant object. After that, when the parallax image is displayed on the shielding unit 7, the driver 5 is expected to correctly perceive the image based on the parallax image.

As described above, the image display device 1 according to the present embodiment, with the above configuration, allows the viewer to perceive the shielding part as being transparent, thereby avoiding a situation where the viewer cannot perceive it correctly.

Although the embodiments according to the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various changes or modifications based on the present disclosure. It should therefore be noted that these variations or modifications are included within the scope of this disclosure. For example, functions included in each component or each process can be rearranged to avoid logical contradictions, and multiple components or processes can be combined or divided into one. It is. Although the embodiments according to the present disclosure have been described with a focus on the apparatus, the embodiments according to the present disclosure can also be realized as a method including steps executed by each component of the apparatus. Embodiments according to the present disclosure can also be realized as a method, a program, or a storage medium on which a program is recorded, which is executed by a processor included in an apparatus. It is to be understood that these are also encompassed within the scope of the present disclosure.

1 Image display device 2 Vehicle 3 External camera 4 Driver's seat 5 Driver 6 In-vehicle camera 7 Shielding section 8 First image processing section 9 Second image processing section 10 Display device 11 Retroreflective screen 11a Glass beads 11b Reflective film 12 Projection section 12L Second projection section 12R

First projection section

13L, 13R Liquid crystal display device 14L Second projection lens 14R First projection lens 16 Diffusion plate 22 Rear seat 23 Back seat 24 Back seat projection section 25 Dashboard projection section 31 Line of sight recognition device 33 Image data processing device 35 External camera control device 36 Seating sensor 37 In-vehicle camera control device 39 Display control device 40 Right side pillar 42 Windshield 44 Object 46 Right window glass 50 Control unit EL Left eye ER Right eye

Claims

A first imaging unit images the surroundings of the observer, and based on the first image data obtained by the imaging, a shield that blocks the observer's field of view is seen by one of the observer's left eye and right eye. an image data processing unit that generates a first image in a range corresponding to the shielding part, and a second image in a range corresponding to the shielding part, which is seen by the other of the observer's left eye and right eye;
an image display unit that displays the first image and the second image on the shielding unit;
a line of sight detection unit that detects the focus position of the observer;
the second image has parallax with respect to the first image,
The image display unit displays a parallax-free image, which is an image without parallax, on the shielding unit, instead of the first image and the second image, based on the focus position of the observer's line of sight. Image display device.
The image display unit displays a parallax-free image, which is an image without parallax, in place of the first image and the second image when the focus position of the observer's line of sight is on the shielding unit. The image display device according to claim 1, wherein the image display device displays on the shielding section.
The line of sight detection unit detects the line of sight of the observer based on second image data obtained by photographing the observer by a second imaging unit,
The image display section displays the parallax-free image when the observer's line of sight is within the range of the shielding section and the convergence angle of the observer detected by the line of sight detection section is greater than or equal to a threshold. The image display device according to claim 1, wherein the image display device displays on the shielding section.
The image display device according to any one of claims 1 to 3, wherein the parallax-free image includes text information.
The image display device according to claim 4, wherein the text information is a warning urging you to look into the distance.
The image display device according to any one of claims 1 to 3, which is mounted on a vehicle and wherein the shielding portion is a pillar.
The image display section includes:
a retroreflective screen provided in the shielding part;
a first projection unit that projects the first image onto the retroreflective screen;
The image display device according to any one of claims 1 to 3, further comprising a second projection unit that projects the second image onto the retroreflective screen.
The image display according to claim 7, wherein when displaying the parallax-free image on the image display section, the first projection section and the second projection section project the same image onto the retroreflective screen. Device.
The image display section includes:
The image display device according to claim 7, further comprising a diffuser plate provided adjacent to the retroreflective screen.
an imaging unit that captures an image of an observer and the surroundings of the observer and outputs image data obtained by capturing the image;
an image display unit that causes the observer to perceive a parallax image reflecting the parallax between the left eye and the right eye of the observer based on the image data in a shielding unit that blocks the observer's field of view;
a line-of-sight detection unit that detects the focus position of the observer's line of sight;
The image display unit is configured to display an image that causes the observer to perceive a parallax-free image that does not reflect parallax between the left eye and right eye of the observer when the focus position of the observer's line of sight is on the shielding unit. Display device.