JP2024036167A - aerial display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an aerial display device with which it is possible to improve display quality.

SOLUTION: The aerial display device comprises: a display element 20 that displays an image; an optical element 40 that is located so as to receive light from the display element 20, and that reflects light from the display element 20 to the side opposite the display element 20, forming an aerial image in the air; a sensing element 50 that forms a detection region in an aerial region that overlaps the aerial image and detects an object in the detection region; and a correction unit that calculates the position of an observer and corrects the position of the image of the display element 20 on the basis of the position of the observer.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an aerial display device.

Aerial display devices that can display images and videos as aerial images are being researched and are expected to serve as a new human-machine interface. An aerial display device includes, for example, a dihedral corner reflector array in which dihedral corner reflectors are arranged in an array, and reflects light emitted from a display surface of a display element to form a real image in the air. The display method using a dihedral corner reflector array has no aberration and can display a real image (aerial image) at a plane-symmetrical position.

Patent Document 1 discloses an optical element in which a transparent square prism protruding from the surface of a transparent flat plate is used as a dihedral corner reflector, and a plurality of square prisms are arranged in an array on a plane. Further, Patent Document 2 discloses that each of the first and second light control panels is formed by vertically arranging a plurality of plane light reflection parts inside a transparent flat plate, and the first and second light control panels are arranged on each other's planes. An optical element is disclosed in which light reflecting portions are arranged so as to be orthogonal to each other. The optical elements of Patent Documents 1 and 2 reflect the light emitted from the display element twice on orthogonal reflecting surfaces to generate an aerial image.

The display devices using the optical elements of Patent Documents 1 and 2 can recognize an aerial image by observing from an oblique direction of the optical element, and can recognize a good aerial image by observing from the normal direction of the optical element. It's difficult to do.

Japanese Patent Application Publication No. 2011-191404 Japanese Patent Application Publication No. 2011-175297

The present invention provides an aerial display device that can improve display quality.

According to a first aspect of the present invention, there is provided a display element that displays an image, and a display element that is arranged to receive light from the display element, and that reflects the light from the display element to a side opposite to the display element and sends it into the air. an optical element that forms an aerial image; a sensing element that forms a detection area in a spatial region that overlaps with the aerial image; and a sensing element that detects an object within the detection area; An aerial display device is provided, including a correction section that corrects the position of the image of the display element based on the position.

According to a second aspect of the present invention, the aerial display device according to the first aspect, wherein the correction unit calculates a shift amount of the image of the display element based on a horizontal distance and a vertical distance to the observer. is provided.

According to a third aspect of the present invention, there is provided the aerial display device according to the first aspect, wherein the correction unit displays the image on a straight line passing through the observer and the detection area of the display element. .

According to the fourth aspect of the present invention, the correction unit further includes a distance image sensor that images the observer and measures a distance to the observer, and the correction unit adjusts the observation distance based on the output of the distance image sensor. An aerial display device according to a first aspect is provided that calculates the position of a person.

According to the fifth aspect of the present invention, the recognition unit further includes a recognition unit that recognizes the face of the observer based on the imaging data acquired by the distance image sensor, and the correction unit adjusts the recognition result of the recognition unit. There is provided an aerial display device according to a fourth aspect, which calculates the position of the observer based on the above.

According to a sixth aspect of the present invention, the sensing element according to the first aspect includes a light emitting part that emits light toward the detection area, and a light receiving part that receives reflected light reflected by the target object. Such an aerial display device is provided.

According to the seventh aspect of the present invention, the optical elements are provided under a planar base material and the base material, each extending in a first direction, and arranged in a second direction orthogonal to the first direction. and a plurality of optical elements, each of the plurality of optical elements having an incident surface and a reflection surface that are inclined with respect to the normal direction of the base material and touch each other, the aerial display device according to the first aspect. is provided.

According to an eighth aspect of the present invention, the first aspect further includes an alignment control element that is disposed between the display element and the optical element and transmits an oblique light component of the light from the display element. An aerial display device according to the present invention is provided.

According to a ninth aspect of the present invention, the alignment control element includes a plurality of transparent members and a plurality of light blocking members arranged alternately, and the plurality of light blocking members are inclined with respect to a normal line of the alignment control element. An aerial display device according to an eighth aspect is provided.

According to a tenth aspect of the present invention, there is provided the aerial display device according to the first aspect, wherein the display element and the optical element are arranged parallel to each other.

According to an eleventh aspect of the present invention, the present invention further includes a lighting element that emits light, and the display element is arranged to receive light from the lighting element, and is composed of a liquid crystal display element. Such an aerial display device is provided.

According to the present invention, it is possible to provide an aerial display device that can improve display quality.

FIG. 1 is a perspective view of an aerial display device according to an embodiment of the present invention. FIG. 2 is a side view of the aerial display device shown in FIG. 1 in the XZ plane. FIG. 3 is a perspective view illustrating the appearance of the aerial display device. FIG. 4 is a schematic plan view of the display element shown in FIG. 1. FIG. 5A is a plan view of the orientation control element shown in FIG. 1. FIG. 5B is a cross-sectional view of the alignment control element taken along line AA' in FIG. 5A. FIG. 6 is a perspective view of the optical element shown in FIG. 1. FIG. 7 is a schematic plan view of an image sensor included in the distance image sensor. FIG. 8 is a block diagram of the aerial display device. FIG. 9 is a perspective view illustrating how light is reflected in the optical element. FIG. 10 is a side view of the XZ plane illustrating how light is reflected in the optical element. FIG. 11 is a side view of the YZ plane illustrating how light is reflected in the optical element. FIG. 12 is a diagram illustrating the angle conditions of the incident surface and the reflective surface of the optical element. FIG. 13 is a flowchart illustrating the overall operation of the aerial display device. FIG. 14 is a schematic diagram illustrating the operation of recognizing the observer's face. FIG. 15 is a perspective view illustrating the operation of the aerial display device when an observer views the aerial display device from the front. FIG. 16 is a perspective view illustrating the operation of the aerial display device when an observer views the aerial display device from an angle. FIG. 17 is a perspective view illustrating how an observer touches an aerial image. FIG. 18 is a side view of the YZ plane explaining the image correction operation. FIG. 19 is a plan view of the XZ plane for explaining the image correction operation. FIG. 20 is a schematic diagram illustrating the relationship between a subject and an image captured by an image sensor. FIG. 21 is a schematic diagram illustrating the relationship between the distance to the reference object and the height of the image. FIG. 22 is a schematic diagram illustrating the height of the image at the subject distance d1 in FIG. 21. FIG. 23 is a schematic diagram illustrating the height of the image at the subject distance d2 in FIG. 21. FIG. 24 is a schematic diagram illustrating the height of the image at the subject distance dn in FIG. 21. FIG. 25 is a diagram illustrating a method of calculating the image height. FIG. 26 is a flowchart illustrating the operation of calculating the observer's position in step S103. FIG. 27 is a diagram illustrating the operation of the image correction unit when an observer views the aerial display device from the front. FIG. 28 is a diagram illustrating the operation of the image correction unit when an observer views the aerial display device from above in the Y direction. FIG. 29 is a diagram illustrating the operation of the image correction unit when an observer views the aerial display device from below in the Y direction.

Hereinafter, embodiments will be described with reference to the drawings. However, the drawings are schematic or conceptual, and the dimensions, proportions, etc. of each drawing are not necessarily the same as those in reality. Further, even when the same parts are shown in two drawings, the relationships and ratios of the dimensions may be different. In particular, some of the embodiments shown below illustrate devices and methods for embodying the technical idea of the present invention, and the technical idea of the present invention is is not specified. In the following description, elements having the same functions and configurations are denoted by the same reference numerals, and redundant description will be omitted.

[1] Configuration of aerial display device 1 FIG. 1 is a perspective view of an aerial display device 1 according to an embodiment of the present invention. In FIG. 1, the X direction is a direction along one side of the aerial display device 1, the Y direction is a direction perpendicular to the X direction in a horizontal plane, and the Z direction is a direction perpendicular to the XY plane ( (also called the normal direction). FIG. 2 is a side view of the aerial display device 1 shown in FIG. 1 in the XZ plane. In FIG. 2, illustration of the sensing element 50 and distance image sensor 60 is omitted. FIG. 3 is a perspective view illustrating the appearance of the aerial display device 1.

The aerial display device 1 is a device that displays images (including moving images). The aerial display device 1 displays an aerial image in the air above its own light exit surface. The light exit surface of the aerial display device 1 means the upper surface of the member disposed in the uppermost layer among the plurality of members constituting the aerial display device 1. An aerial image is a real image formed in the air.

The aerial display device 1 includes a lighting element (also referred to as a backlight) 10, a display element 20, an orientation control element 30, an optical element 40, a sensing element 50, a distance image sensor 60, and a housing 70. The lighting element 10, the display element 20, the orientation control element 30, and the optical element 40 are arranged in this order along the Z direction and parallel to each other. The lighting element 10, the display element 20, the orientation control element 30, and the optical element 40 are fixed at a desired position with a fixing member (not shown) with a desired spacing between them.

The lighting element 10 emits illumination light and emits this illumination light toward the display element 20. The lighting element 10 includes a light source section 11, a light guide plate 12, and a reflective sheet 13. The lighting element 10 is, for example, a side light type lighting element. The lighting element 10 constitutes a surface light source. The lighting element 10 may be configured so that the light intensity peaks in an oblique direction at an angle θ ₁ , which will be described later.

The light source section 11 is arranged so as to face the side surface of the light guide plate 12. The light source section 11 emits light toward the side surface of the light guide plate 12. The light source section 11 includes a plurality of light emitting elements made of, for example, white LEDs (Light Emitting Diodes). The light guide plate 12 guides the illumination light from the light source section 11 and emits the illumination light from its upper surface. The reflective sheet 13 reflects the illumination light emitted from the bottom surface of the light guide plate 12 toward the light guide plate 12 again. The lighting element 10 may include a member (including a prism sheet and a diffusion sheet) that improves optical characteristics on the upper surface of the light guide plate 12.

The display element 20 is a transmissive display element. The display element 20 is composed of, for example, a liquid crystal display element. The drive mode of the display element 20 is not particularly limited, and a TN (Twisted Nematic) mode, a VA (Vertical Alignment) mode, a homogeneous mode, or the like can be used. The display element 20 receives illumination light emitted from the illumination element 10. The display element 20 transmits the illumination light from the illumination element 10 and performs light modulation. The display element 20 then displays a desired image on its screen. The detailed configuration of the display element 20 will be described later.

The alignment control element 30 has a function of reducing unnecessary light. Unnecessary light is a light component that does not contribute to generating an aerial image, and includes a light component that passes through the optical element 40 in the normal direction. The orientation control element 30 is configured to transmit light components within a predetermined angular range centering on an oblique direction at an angle θ ₁ with respect to the normal direction, and to block light components outside the above angular range. The area of the alignment control element 30 is set to be approximately the same as the area of the display element 20. The detailed configuration of the alignment control element 30 will be described later.

The optical element 40 reflects light incident from the bottom side toward the top side. Further, the optical element 40 reflects incident light obliquely incident from the bottom side, for example, in the front direction (normal direction). The area of the optical element 40 is set to be larger than the area of the display element 20. The detailed configuration of the optical element 40 will be described later. The optical element 40 forms an aerial image 2 in the air. The aerial image 2 is parallel to the element surface of the optical element 40 and is a two-dimensional image. The element surface refers to a virtual plane in which the optical element 40 extends in the in-plane direction. Element plane has the same meaning as in-plane. The same meaning applies to the element surfaces of other elements. The observer 3 who is in front of the optical element 40 can visually recognize the aerial image 2.

The sensing element 50 forms a detection region 51 in a spatial region that includes part or all of the aerial image 2 generated by the aerial display device 1 . The sensing element 50 detects a target (object) present in the detection area 51. The sensing element 50 emits infrared light to the detection area 51 and detects the reflected light reflected by the object. The sensing element 50 includes a light emitting section that emits infrared light toward the detection region 51 and a light receiving section (sensor) that detects the reflected light from the object. The sensing element 50 is composed of, for example, a line sensor in which a plurality of light emitting elements and a plurality of light receiving elements are alternately arranged in a line. A line sensor can scan space in a line using infrared light, and can scan a two-dimensional space consisting of the direction in which multiple light emitting elements are lined up and the direction in which the light travels. be. The direction of the infrared light emitted by the sensing element 50 can be set as appropriate, and for example, the infrared light may be emitted obliquely upward.

The distance image sensor 60 has a function of measuring the distance from itself to a target object and a function of capturing an image of the target object. The distance image sensor 60 is configured as a TOF (time of light) type distance image sensor. The TOF method is a method that measures the distance to an object based on the time it takes for light emitted from a light-emitting element to irradiate the object and for the reflected light reflected by the object to be detected by a light-receiving element. . Further, the distance image sensor 60 includes an image sensor and images a subject. The distance image sensor 60 includes a light emitting element that emits a laser beam (for example, infrared light) for distance measurement, and an image sensor that captures an image. The image sensor is composed of a CMOS (complementary metal oxide semiconductor) sensor or a CCD (charge coupled device) sensor. The image sensor is also used as a TOF type light receiving element.

Note that instead of the distance image sensor 60, a distance sensor that measures the distance from itself to the target object and an image sensor that captures an image of the target object may be provided separately. In the case of this embodiment, the distance sensor is configured to include a light emitting section that emits laser light (for example, infrared light) and a light receiving section that receives reflected light reflected by the target object.

The housing 70 accommodates the illumination element 10 , the display element 20 , the orientation control element 30 , the optical element 40 , the sensing element 50 , and the distance image sensor 60 . The housing 70 has an opening at the top that exposes the optical element 40 and an opening that exposes the distance image sensor 60. Furthermore, the sensing element 50 is attached to one side of the housing 70.

Note that the sensing element 50 is not housed in the housing 70 and may be placed at any position outside the housing 70. The sensing element 50 can be placed at an optimal location for sensing operation. Further, the distance image sensor 60 may not be housed in the housing 70, but may be placed at an arbitrary position outside the housing 70. The distance image sensor 60 can be placed at the optimal location for the detection operation.

[1-1] Configuration of display element 20 FIG. 4 is a schematic plan view of the display element 20 shown in FIG. 1.

The display element 20 includes a display area 21, a frame 22, and a case 23. The display area 21 is an area where images are displayed. The frame 22 is a peripheral area around the display area 21, and is an area where peripheral circuits that drive pixels are arranged. The frame 22 is shielded from light by a black light shielding layer (black matrix), and is visually recognized as black by an observer. The case 23 fixes the substrate constituting the display element 20 from the periphery.

The display area 21 is provided with a plurality of pixels Dp arranged in a matrix. A plurality of pixels arranged in a matrix is also called a pixel array. Each of the plurality of pixels Dp is the minimum unit of color information of an image displayed by the display element 20. The pixel Dp is composed of, for example, a red sub-pixel R, a green sub-pixel G, and a blue sub-pixel B, which respectively correspond to the three primary colors of light, red (R), green (G), and blue (B). Note that the area of the pixel Dp is sufficiently smaller than the illustrated area.

Let the number of columns of the pixel array be "md" and the number of rows be "nd". The number of pixels of the display element 20 is "md*nd". The number of pixels of the display element 20 can be set arbitrarily.

[1-2] Configuration of alignment control element 30 FIG. 5A is a plan view of the alignment control element 30 shown in FIG. 1. FIG. 5B is a cross-sectional view of the alignment control element 30 taken along line AA' in FIG. 5A.

The base material 31 is configured to be planar in the XY plane and has a rectangular parallelepiped shape. The base material 31 transmits light.

A plurality of transparent members 33 are provided on the base material 31, each extending in the Y direction and aligned in the X direction. Furthermore, a plurality of light shielding members 34 are provided on the base material 31, each extending in the Y direction and lined up in the X direction. The plurality of transparent members 33 and the plurality of light shielding members 34 are arranged alternately so that adjacent ones are in contact with each other.

A base material 32 is provided on the plurality of transparent members 33 and the plurality of light shielding members 34 . The base material 32 is configured to be planar in the XY plane and has a rectangular parallelepiped shape. The base material 32 transmits light.

The transparent member 33 extends in an oblique direction at an angle θ ₁ with respect to the normal direction of the base material 31 in the XZ plane. The transparent member 33 is a parallelogram whose side surfaces are inclined by an angle θ ₁ in the XZ plane. The transparent member 33 transmits light.

The light shielding member 34 extends in an oblique direction at an angle θ ₁ with respect to the normal direction of the base material 31 in the XZ plane. The light shielding member 34 is a parallelogram whose side surfaces are inclined by an angle θ ₁ in the XZ plane. The light blocking member 34 blocks light. The thickness of the light shielding member 34 is set to be thinner than the thickness of the transparent member 33.

Two adjacent light shielding members 34 are arranged so that their ends slightly overlap in the Z direction.

As the base materials 31 and 32 and the transparent member 33, glass or transparent resin (including acrylic resin) is used. As the light shielding member 34, for example, resin mixed with black dye or pigment is used.

Note that the alignment control element 30 may be configured by omitting one or both of the base materials 31 and 32. If the plurality of transparent members 33 and the plurality of light shielding members 34 are arranged alternately, the function of the alignment control element 30 can be realized.

The alignment control element 30 configured in this manner can transmit display light such that the light intensity in an oblique direction at an angle θ ₁ with respect to the normal direction reaches a peak. For example, the alignment control element 30 is configured to block light components outside the range of 30°±30° with respect to the normal direction. Preferably, the alignment control element 30 is configured to block light components outside the range of 30°±20° with respect to the normal direction.

Note that, as a modification, the orientation control element 30 may be arranged between the lighting element 10 and the display element 20. Furthermore, the aerial display device 1 may be configured without the orientation control element 30.

[1-3] Configuration of optical element 40 FIG. 6 is a perspective view of the optical element 40 shown in FIG. 1. FIG. 6 also shows an enlarged view of a part of the optical element 40. The enlarged view in FIG. 6 is a side view in the XZ plane.

The optical element 40 includes a base material 41 and a plurality of optical elements 42. The base material 41 is configured to be planar in the XY plane and has a rectangular parallelepiped shape.

A plurality of optical elements 42 are provided on the bottom surface of the base material 41. Each of the plurality of optical elements 42 is composed of a triangular prism. The optical element 42 is arranged so that three side surfaces of a triangular prism are parallel to the XY plane, and one side surface is in contact with the base material 41. The plurality of optical elements 42 each extend in the Y direction and are arranged side by side in the X direction. In other words, the plurality of optical elements 42 have a sawtooth shape in the XZ plane.

Each of the plurality of optical elements 42 has an incident surface 43 and a reflective surface 44. When viewed from the Y direction, the left side surface is the incident surface 43, and the right side surface is the reflective surface 44. The incident surface 43 is a surface onto which light from the display element 20 is incident. The reflective surface 44 is a surface that reflects light that has entered the entrance surface 43 from the outside inside the optical element 42 . The incident surface 43 and the reflective surface 44 have an angle θ _p .

The base material 41 and the optical element 42 are made of transparent material. The optical element 42 is formed integrally with the base material 41, for example, from the same transparent material as the base material 41. The base material 41 and the optical element 42 may be formed separately, and the optical element 42 may be adhered to the base material 41 using a transparent adhesive. As the transparent material constituting the base material 41 and the optical element 42, glass or transparent resin (including acrylic resin) is used.

The optical element 40 configured in this manner reflects incident light internally to form a real image in the air. Further, the optical element 40 forms an aerial image at a position in front of the element surface.

[1-4] Configuration of distance image sensor 60 The distance image sensor 60 includes an image sensor 61. FIG. 7 is a schematic plan view of the image sensor 61 included in the distance image sensor 60.

The imaging device 61 includes a substrate 62, an imaging region 63, and a peripheral region 64. The imaging area 63 is an area that converts light into electrical signals. The peripheral area 64 is an area around the imaging area 63, and is an area where peripheral circuits that drive pixels are arranged. The peripheral area 64 is shielded from light by a black light shielding layer. A circuit is provided on the substrate 62.

The imaging area 63 is provided with a plurality of pixels Pp arranged in a matrix. A plurality of pixels arranged in a matrix is also called a pixel array. Each of the plurality of pixels Pp is the smallest unit of area that converts light into an electrical signal. Note that the area of the pixel Pp is sufficiently smaller than the illustrated area.

Let the number of columns of the pixel array be "mi" and the number of rows be "ni". The number of pixels of the image sensor 61 is "mi*ni". The number of pixels of the image sensor 61 can be set arbitrarily. "C" in FIG. 7 is the center of the image sensor 61 (referred to as the center of the image sensor). The center "C" of the image sensor is the intersection of a line passing through the edge of the pixel in the "mi/2" column from the left and a line passing through the edge of the pixel in the "ni/2" column from the top.

[1-5] Block configuration of aerial display device 1 FIG. 8 is a block diagram of the aerial display device 1. The aerial display device 1 includes a control section 80, a storage section 81, an input/output interface (input/output IF) 82, a display section 83, a sensing element 50, a distance image sensor 60, and an input section 84. The control section 80, the storage section 81, and the input/output interface 82 are connected to each other via a bus 85.

The input/output interface 82 is connected to the display section 83, the sensing element 50, the distance image sensor 60, and the input section 84. The input/output interface 82 performs interface processing on each of the display section 83, sensing element 50, distance image sensor 60, and input section 84 according to a predetermined standard.

The display section 83 includes a lighting element 10 and a display element 20. The display section 83 displays images.

The control unit 80 includes one or more processors such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The control unit 80 implements various functions by executing programs stored in the storage unit 81. The control section 80 includes a display processing section 80A, an information processing section 80B, a detection position calculation section 80C, a face recognition section 80D, and an image correction section 80E.

The display processing unit 80A controls the operation of the display unit 83 (specifically, the lighting element 10 and the display element 20). The display processing unit 80A controls turning on and off of the lighting element 10. The display processing unit 80A transmits an image signal to the display element 20 and causes the display element 20 to display an image.

The information processing unit 80B generates an image to be displayed by the aerial display device 1. The information processing section 80B can use the image data stored in the storage section 81. The information processing unit 80B may acquire image data from outside using a communication function (not shown).

The detected position calculation unit 80C controls the operation of the sensing element 50. The detection position calculation section 80C controls the light emitting section included in the sensing element 50 to emit infrared light, and forms a detection region 51 made of infrared light in a predetermined spatial region. The detection position calculation unit 80C calculates the position of the finger 3A of the observer 3 based on a plurality of detection signals sent from the light receiving unit included in the sensing element 50.

The face recognition unit 80D acquires image data from the image sensor 61. The face recognition unit 80D extracts a plurality of feature points such as eyes, nose, mouth, and outline using the image data acquired from the image sensor 61, and recognizes the face based on the positional relationship of the plurality of feature points. . Furthermore, the face recognition unit 80D recognizes the position of the eyes based on the positional relationship of the plurality of feature points.

The image correction unit 80E controls the operation of the distance image sensor 60. The image correction unit 80E calculates the position of the observer 3 based on the information sent from the distance image sensor 60. Furthermore, the image correction unit 80E corrects the position of the image displayed on the display element 20 based on information regarding the position of the observer 3.

The storage unit 81 includes nonvolatile storage devices such as ROM (Read Only Memory), HDD (Hard Disk Drive), and SSD (Solid State Drive), and volatile storage devices such as RAM (Random Access Memory) and registers. including. The storage unit 81 stores programs executed by the control unit 80. The storage unit 81 stores various data necessary for control of the control unit 80. Furthermore, the storage unit 81 stores data of images displayed by the aerial display device 1.

The input unit 84 includes, for example, a touch panel and buttons, and receives information input by the user. The information processing section 80B can select an image to be displayed on the display section 83 based on the information received by the input section 84.

[2] Operation of the aerial display device 1 Next, the operation of the aerial display device 1 configured as described above will be described.

[2-1] Display operation of aerial image 2 First, the display operation of aerial image 2 will be explained.
The arrows in FIG. 2 indicate optical paths. As shown in FIG. 2, light emitted from an arbitrary point "o" on the display element 20 enters the alignment control element 30. Of the light emitted from the display element 20 , a light component at an angle θ ₁ (including a light component within a predetermined angular range centered on the angle θ ₁ ) is transmitted through the alignment control element 30 . The light transmitted through the alignment control element 30 enters the optical element 40. The optical element 40 reflects the incident light to the side opposite to the orientation control element 30, and forms an aerial image 2 in the air.

FIG. 9 is a perspective view illustrating how light is reflected in the optical element 40. FIG. 10 is a side view of the XZ plane illustrating how light is reflected in the optical element 40. FIG. 10 is a diagram of the optical element 40 viewed with both eyes of the observer 3 (that is, a line connecting both eyes) parallel to the X direction. FIG. 11 is a side view of the YZ plane illustrating how light is reflected in the optical element 40. FIG. 11 is a diagram of the optical element 40 viewed with both eyes of the observer 3 parallel to the Y direction.

Light emitted from an arbitrary point "o" on the display element 20 enters the incident surface 43 of the optical element 40 and reaches the reflective surface 44 . The light that reaches the reflective surface 44 at an angle larger than the critical angle with respect to the normal direction of the reflective surface 44 is totally reflected on the reflective surface 44 and is reflected on the opposite side of the optical element 40 from the side where the optical element 42 is formed. is emitted from the plane of The critical angle is the minimum angle of incidence beyond which total internal reflection occurs. The critical angle is the angle of the plane of incidence with respect to the normal.

In the XZ plane of FIG. 10, the light emitted from point "o" is totally reflected by the reflective surface 44 of the optical element 42, and the light is imaged in the air to generate an aerial image.

In the YZ plane of FIG. 11, the light emitted from point "o" is not reflected by the reflective surface 44 of the optical element 42, and the light does not form an image in the air, so it does not contribute to the generation of an aerial image.

That is, the condition under which the observer 3 can visually recognize the aerial image is that both eyes of the observer 3 are parallel to the X direction or close to it (for example, ±10 degrees with respect to the X direction). Further, when the observer 3 moves his/her viewpoint along the Y direction with both eyes parallel to the X direction or in a state close to it, the aerial image can always be recognized.

FIG. 12 is a diagram illustrating the angle conditions of the incident surface 43 and the reflective surface 44 in the optical element 40.

Let the angle of the incident surface 43 with respect to the Z direction (direction perpendicular to the element surface) be θ ₂ , the angle of the reflective surface 44 with respect to the Z direction be θ ₃ , and the angle between the incident surface 43 and the reflective surface 44 be θ _p . The angle θ _p is expressed by the following equation (1).
θ _p = θ ₂ + θ ₃ ...(1)

The light emitted from the orientation control element 30 at an angle θ ₁ enters the incident surface 43 . Let the refractive index of the material of the optical element 40 be n _p and the refractive index of air be 1. Let the incident angle at the incident surface 43 be θ ₄ and the refraction angle be θ ₅ . Let the incident angle on the reflecting surface 44 be θ ₆ and the reflection angle be θ ₇ (=θ ₆ ). Let the incident angle at the upper surface of the optical element 40 be θ ₈ and the refraction angle be θ ₉ . The refraction angle θ ₉ is the exit angle. The output angle θ ₉ is expressed by the following equation (2).
θ ₉ = sin ^-1 (n _p *sin (sin ^-1 ((1/n _p ) * sin (90° - (θ ₁ + θ ₂ ))) + θ ₂ +2θ ₃ -90°)) ... (2 )

The critical angle at the reflective surface 44 is expressed by the following equation (3).
Critical angle <θ ₆ (=θ ₇ )
Critical angle = sin ^-1 (1/n _p )...(3)

That is, the incident angle θ ₆ on the reflective surface 44 is set to be larger than the critical angle on the reflective surface 44 . In other words, the angle θ ₃ of the reflective surface 44 is set such that the angle of incidence of light incident on the reflective surface 44 is greater than the critical angle.

Further, the light incident on the entrance surface 43 is set so as not to be totally reflected on the entrance surface 43. That is, the angle θ ₂ of the incident surface 43 is set such that the incident angle of light incident on the incident surface 43 is smaller than the critical angle.

The angle between the element surface of the optical element 40 and the surface of the aerial image 2 and the distance between the element surface of the optical element 40 and the surface of the aerial image 2 are determined by the angle θ ₁ of the light incident on the optical element 40 and the angle between the element surface of the optical element 40 and the surface of the aerial image 2. Adjustment is possible by optimally setting the refractive index, the angle θ ₂ of the incident surface 43 of the optical element 40, and the angle θ ₃ of the reflective surface 44 of the optical element 40.

[2-2] Flow of overall operation Next, the flow of overall operation in the aerial display device 1 will be explained. FIG. 13 is a flowchart illustrating the overall operation of the aerial display device 1.

The image sensor 61 images the observer 3 and generates image data 65 (step S100).

Subsequently, the face recognition unit 80D obtains image data 65 from the image sensor 61. The face recognition unit 80D recognizes the face of the observer 3 based on the image data 65 acquired from the image sensor 61 (step S101).

Subsequently, the distance image sensor 60 measures the distance to the observer 3 (step S102). Specifically, the distance image sensor 60 emits a laser beam (for example, infrared light) for distance measurement, and receives reflected light reflected by an object. The distance image sensor 60 then measures the distance to the observer 3 based on the round trip time of the laser beam.

Subsequently, the image correction unit 80E calculates the position of the observer 3 (step S103). Specifically, the image correction unit 80E calculates the distance L from the light exit surface of the aerial display device 1 to the observer 3, and the height H from the center of the distance image sensor 60 to the eyes of the observer 3. The distance L is also called the horizontal distance, and the height H is also called the vertical distance.

Subsequently, the image correction unit 80E corrects the position of the image displayed on the display element 20 based on the position of the observer 3 (step S104).

Subsequently, the display element 20 displays the image at the position corrected by the image correction unit 80E (step S105). The optical element 40 reflects the light from the display element 20 and forms an aerial image 2 in the air.

Subsequently, the sensing element 50 forms a detection region 51 made of infrared light in a predetermined spatial region (step S106).

Subsequently, the detection position calculation unit 80C determines whether the observer 3 has touched the aerial image 2, that is, the touch operation by the observer 3, based on the detection result of the sensing element 50 (step S107).

If a touch operation is not detected (step S107=No), the control unit 80 repeats the operations from step S100 onwards.

When a touch operation is detected (step S107=Yes), the detection position calculation unit 80C determines whether the position touched by the observer 3 (touch position) is within a predetermined operation area of the detection area 51. is determined (step S108).

If the touch position is not within the operation area (step S108=No), the control unit 80 repeats the operations from step S107 onwards.

If the touch position is within the operation area (step S108=Yes), the display element 20 displays an image linked to the touched information (step S109).

Subsequently, the control unit 80 monitors the termination operation by the observer 3 (step S110). The termination operation is performed using the input unit 84, for example. If the end operation is not detected (step S110=No), the control unit 80 repeats the operations from step S107 onwards. If an end operation is detected (step S110=Yes), the control unit 80 ends the process.

[2-3] Face Recognition Operation Next, the operation of recognizing the face of the observer 3 (face recognition operation) will be described. FIG. 14 is a schematic diagram illustrating the operation of recognizing the face of the observer 3.

The image sensor 61 images the observer 3 and generates image data 65 . The face recognition unit 80D acquires image data 65 from the image sensor 61. The face recognition unit 80D uses the image data 65 acquired from the image sensor 61 to extract a plurality of feature points such as eyes, nose, mouth, and outline, and recognizes the face based on the positional relationship of the plurality of feature points. do. In FIG. 16, a rectangular frame is recognized as a face. Furthermore, the face recognition unit 80D recognizes the position of the eyes based on the positional relationship of the plurality of feature points.

Based on the result of face recognition by the face recognition unit 80D, the positional relationship between the aerial display device 1 and the observer 3 is determined.

[2-4] Outline of Image Correction Operation Next, an outline of the operation (image correction operation) for correcting the position of an image displayed on the display element 20 will be described.

FIG. 15 is a perspective view illustrating the operation of the aerial display device 1 when the observer 3 views the aerial display device 1 from the front. In FIG. 15, a button is shown as an example of the aerial image 2.

The display element 20 displays a button image 24 near the center of its screen. The optical element 40 reflects the light from the display element 20 and forms an aerial image 2 in the air.

The sensing element 50 forms a detection area 51 that includes a spatial area in which the aerial image 2 is displayed. The detection area 51 is formed using infrared light. When the finger 3A of the observer 3 touches the aerial image 2, the sensing element 50 detects the finger 3A of the observer 3 within the detection area 51. The detection position calculation unit 80C calculates the position of the finger 3A of the observer 3 based on the plurality of detection signals sent from the sensing element 50. Thereby, the aerial display device 1 can display the aerial image 2 toward the observer 3 and detect that the observer 3 has touched the aerial image 2.

When the observer 3 views the aerial display device 1 diagonally from above with the display element 20 displaying the image 24 in the center, the aerial image 2 is visually recognized as shifted upward from the detection area 51 . In this case, it may not be possible to detect that the finger 3A of the observer 3 touches the aerial image 2. When the observer 3 views the aerial display device 1 diagonally from below with the display element 20 displaying the image 24 in the center, the aerial image 2 is visually recognized as shifted downward from the detection area 51 . In this case as well, there is a possibility that it may not be possible to detect that the finger 3A of the observer 3 touches the aerial image 2.

FIG. 16 is a perspective view illustrating the operation of the aerial display device 1 when the observer 3 views the aerial display device 1 from an angle. The definition of up and down in the explanation of FIG. 16 is based on the aerial display device 1 arranged in the state shown in FIG.

The sensing element 50 forms a detection region 51 in a fixed spatial region. That is, the sensing element 50 always forms the detection region 51 at the same position regardless of the position of the observer 3.

If it is determined that the observer 3 is viewing the aerial display device 1 from diagonally above, the display element 20 displays the image 24 by moving it below the center (the position of the image in FIG. 15). Thereby, the aerial display device 1 can display the aerial image 2 so as to overlap the detection area 51. Further, the aerial display device 1 can detect that the finger 3A of the observer 3 touches the aerial image 2.

If it is determined that the observer 3 is viewing the aerial display device 1 from diagonally below, the display element 20 moves and displays the image 24 above the center (the position of the image in FIG. 15). Thereby, the aerial display device 1 can display the aerial image 2 so as to overlap the detection area 51. Further, the aerial display device 1 can detect that the finger 3A of the observer 3 touches the aerial image 2.

[2-5] Details of Image Correction Operation Next, detailed operations for correcting the position of the image displayed on the display element 20 will be described.

FIG. 17 is a perspective view illustrating how the observer 3 touches the aerial image 2.

The center of the detection region 51 formed by the sensing element 50 is defined as "Sc". The distance from the light emission surface of the aerial display device 1 to the observer 3 is “L”, the distance from the light emission surface of the aerial display device 1 to the detection area 51 is “Ls”, and the distance from the detection area 51 to the observer 3 Let be “Le”. The light exit surface of the aerial display device 1 has the same meaning as the top surface of the aerial display device 1. The distance Ls is a design value determined according to the specifications of the aerial display device 1. The distance L is expressed as "L=Ls+Le". The distance L is calculated based on distance information measured by the distance image sensor 60. A specific method for calculating the distance L will be described later.

FIG. 18 is a side view of the YZ plane explaining the image correction operation. FIG. 18 shows the aerial display device 1 viewed diagonally from above by the observer 3.

Based on the center of the distance image sensor 60 (specifically, the center of the image sensor 61), the height of the eyes of the observer 3 is "H", the height of the center Sc of the detection area 51 is "Hs", The height from the center Sc of the detection area 51 to the eyes of the observer 3 is defined as "He". The height Hs is a design value determined according to the specifications of the aerial display device 1. The height H is expressed as "H=Hs+He". The height H is calculated based on the image data captured by the distance image sensor 60 and the result of face recognition by the face recognition unit 80D. A specific method for calculating the height H will be described later.

FIG. 19 is a plan view of the XZ plane for explaining the image correction operation. FIG. 19 is a top view of the aerial display device 1 of FIG. 18.

Let the distance in the X direction from the center of the distance image sensor 60 to the center Sc of the detection area 51 be "Xd", and the oblique distance from the center of the distance image sensor 60 to the observer 3 be "Lf". The distance Xd is a design value determined according to the specifications of the aerial display device 1. The oblique distance Lf is calculated based on distance information measured by the distance image sensor 60.
The distance L is expressed by the following equation (4).
L=√((Lf ² -Xd ² )) ...(4)

The amount of image shift on the display element 20 is set to "Hi". The shift amount Hi is expressed by the following equation (5).
Hi:He=Ls:Le
Hi=(He*Ls)/Le=((HHs)*Le)/(L-Le)...(5)

The image correction unit 80E performs an operation of correcting the image position in step S104 using equation (5).

(Calculation of height H of observer 3)
Next, a method for calculating the eye height H of the observer 3 will be explained.

FIG. 20 is a schematic diagram illustrating the relationship between the subject 3B and the image 68 captured by the image sensor 61.

The distance image sensor 60 includes a lens 66. The lens 66 is arranged above the image sensor 61 and fixed by a casing (not shown).

The image sensor 61 images the subject 3B through a lens 66. The image sensor 61 is illuminated with an image 68 of the subject 3B.

The height of the object 3B is "Oa'", the distance from the object 3B to the center of the lens 66 (referred to as the object distance) is "fa", and the height of the image 68 (referred to as the image height) is "Ob". ”, and the distance from the center of the lens 66 to the image sensor 61 is “fb”. These numerical values have a relationship of "fa:fb=Oa':Ob". The subject distance fa corresponds to the distance L to the observer 3 described above.

FIG. 21 is a schematic diagram illustrating the relationship between the distance to the reference subject 3B and the height of the image 68. The distance image sensor 60 includes a housing 67 that houses an image sensor 61 and a lens 66. The housing 67 supports the lens 66. The central surface of the lens 66 is also referred to as the main surface (or top surface) of the distance image sensor 60. FIG. 21 shows a case where the distances between the reference subject 3B and the center of the lens 66 are d1, d2, and dn (d1<d2<dn). A perpendicular line passing through the center of the image sensor 61 is written as "Cz". The height of the reference subject 3B from the perpendicular Cz is defined as "Oa", and "Oa" is referred to as the reference subject height. “Oa” in FIG. 21 is half of “Oa′” in FIG. 20. Even if the height of the object and the height of the image are each halved, the aforementioned relationship of the ratio between the object and the image is satisfied.

FIG. 22 is a schematic diagram illustrating the height of the image 68-1 at the subject distance d1 in FIG. 21. A horizontal line passing through the center C of the image sensor 61 is written as "Cx".

The image height can be converted into a pixel line position of the image sensor 61. Once the number of rows and columns in the pixel array of the image sensor 61 is determined, the pixel line position (the number of pixel rows from the center C) corresponding to the image height is calculated. That is, the image height corresponds to the pixel line position. The number of pixel lines from the horizontal line Cx to the top of the image 68-1 in the image sensor 61 is assumed to be "n1".

FIG. 23 is a schematic diagram illustrating the height of the image 68-2 at the subject distance d2 in FIG. 21. The number of pixel lines at the subject distance d2 is "n2".

FIG. 24 is a schematic diagram illustrating the height of the image 68-n at the subject distance dn in FIG. 21. The number of pixel lines at the subject distance dn is "nn".

It can be seen from FIGS. 21 to 24 that as the subject distance increases, the number of pixel lines decreases (the image height decreases). The relationships shown in FIGS. 21 to 24 are calculated in advance as reference data Df, and this reference data Df is stored in the storage unit 81.

FIG. 25 is a diagram illustrating a method of calculating the image height. The horizontal axis in FIG. 25 represents the subject distance, and the vertical axis represents the image height. The image height on the vertical axis corresponds to the number of pixel lines of the image sensor 61. The straight line shown in FIG. 25 is a plot of the reference data Df.

A black circle in FIG. 25 is an example of an image acquired by the image sensor 61. It is assumed that the object distance of the acquired image is "fe" and the image height is "Oe". A reference image height Or is calculated from reference data Df corresponding to the subject distance fe. The height H is calculated from the value of “Oe/Or”. The height H is expressed by the following equation (6). The reference object height Oa is stored in the storage unit 81 together with the reference data Df.
H=Oa・(Oe/Or)...(6)

If the height of the subject 3B is the position of the eyes of the observer 3 calculated by the face recognition unit 80D, the height H of the eyes of the observer 3 is calculated using equation (6).

Note that the height H is not limited to the eye level of the observer 3. The face recognition unit 80D may calculate the approximate position of the observer's 3 face. In this case, the height H is the height of the face of the observer 3.

(Operation to calculate the position of observer 3)
FIG. 26 is a flowchart illustrating the operation of calculating the position of the observer 3 in step S103.

The image correction unit 80E calculates the image height Oe based on the imaging data 65 acquired in step S100 (step S200).

Subsequently, the image correction unit 80E calculates the parameter “Oe/Or” based on the reference data Df stored in the storage unit 81. Subsequently, the image correction unit 80E calculates the eye height H of the observer 3 using equation (6) (step S201).

Subsequently, the image correction unit 80E calculates the distance L to the observer 3 using equation (4) (step S202).

In this way, the position of the observer 3, that is, the height H of the eyes of the observer 3, and the distance L to the observer 3 are calculated. Thereafter, in step S104, an operation is performed to correct the position of the image displayed on the display element 20.

[2-6] Specific operations of image correction section 80E Next, specific operations of image correction section 80E will be described. FIG. 27 is a diagram illustrating the operation of the image correction unit 80E when the observer 3 views the aerial display device 1 from the front. The position of the observer 3 (for example, the position of the eyes of the observer 3) is calculated by the distance image sensor 60 and the image correction unit 80E, as described above.

If it is determined that the observer 3 is looking at the aerial display device 1 from the front, the image correction unit 80E does not correct the position of the image 24 on the display element 20, and the display element 20 is placed in the normal position, for example, An image 24 is displayed in the center of the screen. Thereby, the observer 3 visually recognizes the aerial image 2 in a spatial region that overlaps with the detection region 51. Therefore, when the observer 3 touches the aerial image 2 with the finger 3A, the sensing element 50 can detect the touch operation.

FIG. 28 is a diagram illustrating the operation of the image correction unit 80E when the observer 3 views the aerial display device 1 from above in the Y direction. If it is determined that the observer 3 is viewing the aerial display device 1 from above in the Y direction, the image correction unit 80E shifts the image 24 of the display element 20 downward in the Y direction. The image correction unit 80E corrects the image position so that the image 24 is shifted by the shift amount Hi. That is, the image correction unit 80E displays the image 24 on a straight line passing through the observer 3 and the detection area 51 on the display element 20. For example, the image correction unit 80E causes the display element 20 to display the image 24 so that the center of the image 24 is on a straight line passing through the eyes of the observer 3 and the center of the detection area 51. Thereby, the observer 3 visually recognizes the aerial image 2 in a spatial region that overlaps with the detection region 51.

FIG. 29 is a diagram illustrating the operation of the image correction unit 80E when the observer 3 views the aerial display device 1 from below in the Y direction. When the observer 3 is viewing the aerial display device 1 from below in the Y direction, the image correction unit 80E shifts the image 24 of the display element 20 upward in the Y direction. The image correction unit 80E corrects the image position so that the image 24 is shifted by the shift amount Hi. That is, the image correction unit 80E displays the image 24 on a straight line passing through the observer 3 and the detection area 51 on the display element 20. For example, the image correction unit 80E causes the display element 20 to display the image 24 so that the center of the image 24 is on a straight line passing through the eyes of the observer 3 and the center of the detection area 51. Thereby, the observer 3 visually recognizes the aerial image 2 in a spatial region that overlaps with the detection region 51.

[3] Effects of Embodiment According to the embodiment, the aerial display device 1 can display the aerial image 2 in the air by reflecting the light emitted from the display element 20 with the optical element 40. Further, the aerial display device 1 can display the aerial image 2 parallel to the element surface of the optical element 40 in the front direction thereof. Furthermore, it is possible to realize an aerial display device 1 that can improve display quality.

Further, when the observer 3 views the optical element 40 with both eyes parallel to the X direction (that is, the direction in which the plurality of optical elements 42 are lined up) or in a state close to it, the observer 3 visually recognizes the aerial image. be able to. Further, when the observer 3 moves his/her viewpoint along the Y direction with both eyes parallel to the X direction or in a state close to it, the aerial image can always be visually recognized. Moreover, a wider viewing angle can be achieved when both eyes of the observer 3 are parallel to the X direction or in a state close to it.

Furthermore, the position of the image on the display element 20 is corrected depending on the position where the observer 3 views the aerial display device 1 . Thereby, as seen from the observer 3, the detection region 51 formed by the sensing element 50 and the aerial image 2 can be overlapped. Therefore, the operation of touching the aerial image 2 by the observer 3 can be detected more accurately.

Furthermore, the position of the eyes of the observer 3 can be calculated, and the position of the image on the display element 20 can be corrected according to the position of the eyes of the observer 3. Thereby, the operation of touching the aerial image 2 by the observer 3 can be detected more accurately.

Further, a plurality of elements constituting the aerial display device 1 can be arranged in parallel. Thereby, it is possible to realize an aerial display device 1 that can be downsized in the Z direction.

[4] Modification The above embodiment describes an example in which the image on the display element 20 is shifted along the Y direction when the observer 3 moves in the Y direction. Based on the same principle of operation, when the observer 3 moves in the X direction, the image on the display element 20 is controlled to shift in the X direction. The operation of shifting the image of the display element 20 in the X direction corresponds to the operation of shifting the image in the Y direction described in the embodiment, with the X and Y directions interchanged.

Furthermore, when it is detected that the observer 3 has moved in the Y direction, the image on the display element 20 may be always shifted by a constant amount in the opposite direction to the direction in which the observer 3 has moved. This makes it possible to further simplify the control for correcting the position of the image.

In the embodiment described above, the position of the image on the display element 20 is corrected. As a modification, the detection area 51 of the sensing element 50 may be moved depending on the position of the observer 3.

In the embodiment described above, the optical element 40 having a sawtooth shape is used as an imaging element. However, the present embodiment is not limited thereto, and can also be applied to an aerial display device including other types of imaging elements capable of forming an aerial image in the air.

In the embodiment described above, the display element 20 and the optical element 40 are arranged in parallel. However, the present invention is not limited to this, and the display element 20 may be arranged diagonally with respect to the optical element 40. The angle between the display element 20 and the optical element 40 is set in a range greater than 0 degrees and less than 45 degrees. In this modification, the orientation control element 30 can be omitted.

In the embodiment described above, the left side surface of the optical element 42 is defined as the incident surface 43, and the right side surface is defined as the reflective surface 44. However, the present invention is not limited to this, and the incident surface 43 and the reflective surface 44 may be configured in reverse. In this case, the left and right functions of the aerial display device 1 described in the embodiment are also reversed.

Although the above embodiment has been described using a liquid crystal display element as an example of the display element 20, the display element 20 is not limited to this. The display element 20 can also be a self-luminous organic EL (electroluminescence) display element, a micro LED (light emitting diode) display element, or the like. A micro LED display element is a display element that uses LEDs to emit light for each of R (red), G (green), and B (blue) that constitute a pixel. When using the self-luminous display element 20, the lighting element 10 is not necessary.

The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the invention at the implementation stage. Moreover, each embodiment may be implemented in combination as appropriate, and in that case, the combined effect can be obtained. Furthermore, the embodiments described above include various inventions, and various inventions can be extracted by combinations selected from the plurality of constituent features disclosed. For example, if a problem can be solved and an effect can be obtained even if some constituent features are deleted from all the constituent features shown in the embodiment, the configuration from which these constituent features are deleted can be extracted as an invention.

DESCRIPTION OF SYMBOLS 1... Aerial display device, 2... Aerial image, 3... Observer, 10... Illumination element, 11... Light source part, 12... Light guide plate, 13... Reflection sheet, 20... Display element, 21... Display area, 22... Frame, 23... Case, 24... Image, 30... Orientation control element, 31, 32... Base material, 33... Transparent member, 34... Light shielding member, 40... Optical element, 41... Base material, 42... Optical element, 43... Incident surface , 44...Reflecting surface, 50...Sensing element, 51...Detection area, 60...Distance image sensor, 61...Imaging element, 62...Substrate, 63...Imaging area, 64...Peripheral area, 65...Imaging data, 66...Lens, 67... Housing, 68... Image, 70... Housing, 80... Control unit, 80A... Display processing unit, 80B... Information processing unit, 80C... Detection position calculation unit, 80D... Face recognition unit, 80E... Image correction unit, 81... Storage section, 82... Input/output interface, 83... Display section, 84... Input section, 85... Bus.

Claims (11)

a display element that displays an image;
an optical element that is arranged to receive light from the display element, reflects the light from the display element to a side opposite to the display element, and forms an aerial image in the air;
a sensing element that forms a detection region in a spatial region that overlaps with the aerial image and detects an object within the detection region;
a correction unit that calculates the position of an observer and corrects the position of the image on the display element based on the position of the observer;
An aerial display device comprising: The aerial display device according to claim 1, wherein the correction unit calculates a shift amount of the image of the display element based on a horizontal distance and a vertical distance to the observer. The aerial display device according to claim 1, wherein the correction unit displays the image on a straight line passing through the observer and the detection area among the display elements. further comprising a distance image sensor that images the observer and measures the distance to the observer,
The aerial display device according to claim 1, wherein the correction unit calculates the position of the observer based on the output of the distance image sensor. further comprising a recognition unit that recognizes the observer's face based on the imaging data acquired by the distance image sensor,
The aerial display device according to claim 4, wherein the correction unit calculates the position of the observer based on the recognition result of the recognition unit. The aerial display device according to claim 1, wherein the sensing element includes a light emitting section that emits light toward the detection area, and a light receiving section that receives reflected light reflected by the target object. The optical element includes a planar base material and a plurality of optical elements provided under the base material, each extending in a first direction and arranged in a second direction orthogonal to the first direction,
The aerial display device according to claim 1, wherein each of the plurality of optical elements has an incident surface and a reflective surface that are inclined with respect to the normal direction of the base material and that touch each other. The aerial display device according to claim 1, further comprising an orientation control element that is disposed between the display element and the optical element and transmits an oblique light component of the light from the display element. The alignment control element includes a plurality of transparent members and a plurality of light shielding members arranged alternately,
The aerial display device according to claim 8, wherein the plurality of light shielding members are inclined with respect to a normal line of the orientation control element. The aerial display device according to claim 1, wherein the display element and the optical element are arranged parallel to each other. Further comprising a lighting element that emits light,
The aerial display device according to claim 1, wherein the display element is arranged to receive light from the illumination element and is composed of a liquid crystal display element.