JP2024101393A - Aerial display device and display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an aerial display device allowed to detect a touch operation to an aerial image by an observer further correctly.

SOLUTION: An aerial display device comprises a display element 20 that displays an image, an optical element 40 that is arranged to receive light from the display element 20 so as to reflect the light from the display element 20 oppositely to the display element 20 and forms an aerial image in the air, a sensing element 50 that forms, above the optical element 40, a first detecting area parallel to the display element 20 and a second detecting area different in level from the first detecting area and detects an object being present in the first and second detecting areas, and a determination section 60C that calculates a first detection position in the first detecting area and a second detection position in the second detecting area and determines whether or not the aerial image is touched based on the first and second detection positions.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an aerial display device and a display device.

Aerial display devices capable of displaying images and videos as aerial images are being researched, and are expected to become a new human-machine interface. For example, an aerial display device may be equipped with a two-sided corner reflector array in which two-sided corner reflectors are arranged in an array, and it reflects light emitted from the display surface of a display element to form a real image in the air. The display method using a two-sided corner reflector array is aberration-free and can display a real image (aerial image) in a plane-symmetrical position.

Patent Document 1 discloses an optical element in which a transparent rectangular prism protruding from the surface of a transparent flat plate is used as a dihedral corner reflector, and multiple rectangular prisms are arranged in an array on a flat surface. Patent Document 2 discloses an optical element in which each of the first and second light control panels is formed by arranging multiple planar light reflecting sections vertically inside a transparent flat plate, and the first and second light control panels are arranged so that the planar light reflecting sections are orthogonal to each other. The optical elements of Patent Documents 1 and 2 reflect light emitted from a display element twice at orthogonal reflecting surfaces to generate an aerial image.

The observer can touch the aerial image displayed by the aerial display device without touching the device. The sensing element of the aerial display device detects objects present in the area of the aerial image and recognizes that the observer has touched the aerial image. The sensing element is required to detect the observer's touch operation on the aerial image with high accuracy.

JP 2011-191404 A JP 2011-175297 A

The present invention provides an aerial display device and a display device that can more accurately detect touch operations on an aerial image by an observer.

According to a first aspect of the present invention, there is provided an aerial display device comprising: a display element for displaying an image; an optical element arranged to receive light from the display element and reflecting the light from the display element to the opposite side of the display element to form an aerial image in the air; a sensing element that forms a first detection area parallel to the display element and a second detection area at a different level from the first detection area above the optical element and detects objects present in the first and second detection areas; and a determination unit that calculates a first detection position in the first detection area and a second detection position in the second detection area and determines whether the aerial image has been touched based on the first and second detection positions.

According to a second aspect of the present invention, there is provided an aerial display device according to the first aspect, in which the determination unit determines the direction of movement of the object based on the first and second detection positions, and determines that the aerial image has been touched when the image is on an extension line of the direction of movement.

According to a third aspect of the present invention, there is provided an aerial display device according to the first aspect, in which the sensing element is of a capacitive type.

According to a fourth aspect of the present invention, there is provided an aerial display device according to the third aspect, in which the sensing element includes a plurality of first electrodes, a plurality of second electrodes arranged opposite the plurality of first electrodes, a drive circuit that supplies drive signals to the plurality of first electrodes, and a detection circuit that detects the voltages of the plurality of second electrodes, and the determination unit compares the detection result by the detection circuit with a threshold value and determines the position of the target object according to the comparison result.

According to a fifth aspect of the present invention, there is provided an aerial display device according to the first aspect, in which the sensing elements include a first sensing element that forms the first detection area and a second sensing element that forms the second detection area, and each of the first sensing element and the second sensing element includes a light-emitting section that emits light toward the detection area and a light-receiving section that receives the reflected light reflected by the object.

According to a sixth aspect of the present invention, there is provided an aerial display device according to the first aspect, in which the optical element includes a planar substrate and a plurality of optical elements disposed below the substrate, each extending in a first direction and aligned in a second direction perpendicular to the first direction, each of the plurality of optical elements being inclined with respect to the normal direction of the substrate and having an entrance surface and a reflection surface that are in contact with each other.

According to a seventh aspect of the present invention, there is provided an aerial display device according to the first aspect, further comprising a light control element disposed between the display element and the optical element, which transmits oblique light components of the light from the display element.

According to an eighth aspect of the present invention, there is provided an aerial display device according to the seventh aspect, in which the light control element includes a plurality of transparent members and a plurality of light blocking members arranged alternately, and the plurality of light blocking members are inclined with respect to the normal to the light control element.

According to a ninth aspect of the present invention, there is provided an aerial display device according to the first aspect, in which the display element and the optical element are arranged parallel to each other.

According to a tenth aspect of the present invention, there is provided a display device comprising: a display element for displaying an image; a sensing element that forms a first detection area parallel to the display element above the display element and a second detection area at a different level from the first detection area, and detects an object present in the first and second detection areas; and a determination unit that calculates a first detection position in the first detection area and a second detection position in the second detection area, and determines whether the image has been touched based on the first and second detection positions.

According to an eleventh aspect of the present invention, there is provided a display device according to the tenth aspect, in which the determination unit determines the direction of movement of the object based on the first and second detection positions, and determines that the image has been touched when the image is present on an extension line of the direction of movement.

According to a twelfth aspect of the present invention, there is provided a display device according to the tenth aspect, in which the sensing element is a capacitive type.

According to a thirteenth aspect of the present invention, there is provided a display device according to the twelfth aspect, in which the sensing element includes a plurality of first electrodes, a plurality of second electrodes arranged opposite the plurality of first electrodes, a drive circuit that supplies drive signals to the plurality of first electrodes, and a detection circuit that detects the voltages of the plurality of second electrodes, and the determination unit compares the detection result by the detection circuit with a threshold value and determines the position of the object according to the comparison result.

According to a 14th aspect of the present invention, there is provided a display device according to the 10th aspect, in which the sensing element includes a first sensing element that forms the first detection area and a second sensing element that forms the second detection area, and each of the first sensing element and the second sensing element includes a light-emitting section that emits light toward the detection area and a light-receiving section that receives the reflected light reflected by the object.

The present invention provides an aerial display device and a display device that can more accurately detect touch operations on an aerial image by an observer.

FIG. 1 is a perspective view of an aerial display device according to a first embodiment of the present invention. FIG. 2 is a side view of the aerial display device shown in FIG. 1 in the XZ plane. FIG. 3A is a plan view of the light control element shown in FIG. FIG. 3B is a cross-sectional view of the light control element taken along line AA' in FIG. 3A. FIG. 4 is a perspective view of the optical element shown in FIG. FIG. 5A is a perspective view of the sensing element shown in FIG. FIG. 5B is a plan view of the sensing element shown in FIG. 5A. FIG. 5C is a cross-sectional view of the sensing element taken along line BB' shown in FIG. 5B. FIG. 6 is a block diagram of an aerial display device. FIG. 7 is a perspective view illustrating the state of light reflection in an optical element. FIG. 8 is a side view of the XZ plane for explaining how light is reflected in the optical element. FIG. 9 is a side view of the YZ plane for explaining how light is reflected in the optical element. FIG. 10 is a diagram for explaining the angular conditions of the incident surface and the reflecting surface in the optical element. FIG. 11 is a schematic diagram illustrating the operation of the sensing element. FIG. 12 is a schematic diagram illustrating a plurality of detection regions of a sensing element. FIG. 13 is a diagram illustrating the relationship between a plurality of detection regions in a sensing element and voltage. FIG. 14 is a schematic diagram illustrating an aerial image displayed by the aerial display device. FIG. 15 is a schematic diagram illustrating how the position of an aerial image changes depending on the position of the observer's viewpoint. FIG. 16 is a schematic diagram illustrating the detection operation of the aerial display device. FIG. 17 is a schematic diagram illustrating the detection operation when an observer views the aerial display device from an oblique direction. FIG. 18 is a schematic diagram for explaining a method for calculating the position of an image in the Z coordinate. FIG. 19 is a schematic diagram for explaining the virtual display surface with the Z coordinate corrected. FIG. 20 is a schematic diagram illustrating an operation for determining a touch operation by a viewer. FIG. 21 is a flowchart explaining the overall operation of the aerial display device. FIG. 22 is a perspective view of a display device according to a second embodiment of the present invention. FIG. 23 is a schematic diagram illustrating the detection operation of the display device. FIG. 24 is a schematic diagram illustrating a detection operation when an observer views the display device from an oblique direction. FIG. 25 is a flowchart illustrating the overall operation of the display device. FIG. 26 is a perspective view of a display device according to a third embodiment of the present invention. FIG. 27 is a diagram illustrating the configuration of a sensing element. FIG. 28 is a flowchart explaining the overall operation of the aerial display device according to the fourth embodiment of the present invention. FIG. 29 is a schematic diagram illustrating an operation of determining a touch operation by a viewer.

The following describes the embodiments with reference to the drawings. However, the drawings are schematic or conceptual, and the dimensions and ratios of each drawing are not necessarily the same as those in reality. Even when the same parts are shown in different drawings, the dimensional relationships and ratios of each may be different. In particular, the following embodiments are examples of devices and methods for embodying the technical idea of the present invention, and the shape, structure, arrangement, etc. of the components do not specify the technical idea of the present invention. In the following description, elements having the same function and configuration are given the same reference numerals, and duplicate descriptions are omitted.

[1] First embodiment [1-1] Configuration of the aerial display device 1 Fig. 1 is a perspective view of the aerial display device 1 according to the first embodiment of the present invention. In Fig. 1, the X direction is a direction along one side of the aerial display device 1, the Y direction is a direction perpendicular to the X direction in a horizontal plane, and the Z direction is a direction perpendicular to the XY plane (also called the normal direction). Fig. 2 is a side view of the aerial display device 1 shown in Fig. 1 in the XZ plane.

The aerial display device 1 is a device that displays images (including videos). The aerial display device 1 displays an aerial image in the air above its own light emission surface. The light emission surface of the aerial display device 1 refers to the upper surface of the component that is arranged in the uppermost layer among the multiple components that make up the aerial display device 1 and are arranged on the optical path. An aerial image is a real image that is formed in the air.

The aerial display device 1 includes an illumination element (also called a backlight) 10, a display element 20, a light control element 30, an optical element 40, and a sensing element 50. The illumination element 10, the display element 20, the light control element 30, the optical element 40, and the sensing element 50 are arranged in this order along the Z direction and parallel to one another. The illumination element 10, the display element 20, the light control element 30, the optical element 40, and the sensing element 50 are fixed at specific positions by a fixing member (not shown) with a specific distance between them. The illumination element 10, the display element 20, the light control element 30, the optical element 40, and the sensing element 50 are housed in a housing (not shown).

The lighting element 10 emits illumination light and outputs the illumination light toward the display element 20. The lighting element 10 includes a light source unit 11, a light guide plate 12, and a reflective sheet 13. The lighting element 10 is, for example, a side light type lighting element. The lighting element 10 constitutes a surface light source. The lighting element 10 may be configured so that the light intensity reaches a peak in an oblique direction at an angle _θ1 , which will be described later.

The light source unit 11 is disposed to face the side of the light guide plate 12. The light source unit 11 emits light toward the side of the light guide plate 12. The light source unit 11 includes a plurality of light-emitting elements, for example, white LEDs (Light Emitting Diodes). The light guide plate 12 guides the illumination light from the light source unit 11 and emits the illumination light from its upper surface. The reflection sheet 13 reflects the illumination light emitted from the bottom surface of the light guide plate 12 back toward the light guide plate 12. The lighting element 10 may be provided with a member (including a prism sheet and a diffusion sheet) that improves optical characteristics on the upper surface of the light guide plate 12.

The display element 20 is a transmissive display element. The display element 20 is composed of, for example, a liquid crystal display element. The driving mode of the display element 20 is not particularly limited, and the TN (Twisted Nematic) mode, VA (Vertical Alignment) mode, homogeneous mode, or the like can be used. The display element 20 receives illumination light emitted from the illumination element 10. The display element 20 transmits the illumination light from the illumination element 10 and performs optical modulation. Then, the display element 20 displays a specific image on its own screen.

The light control element 30 has a function of reducing unnecessary light. The unnecessary light is a light component that does not contribute to generating an aerial image, and includes a light component that transmits through the optical element 40 in the normal direction. The light control element 30 is configured to transmit light components in a predetermined angular range centered on an oblique direction at an angle θ ₁ with respect to the normal direction, and to block light components outside the above-mentioned angular range. The area of the light control element 30 is set to be equal to or larger than the area of the display element 20. A detailed configuration of the light control element 30 will be described later.

The optical element 40 reflects light incident from the bottom side to the top side. The optical element 40 also reflects light incident obliquely from the bottom side, for example, in the front direction (normal direction). The area of the optical element 40 is set to be equal to or larger than the area of the display element 20. The detailed configuration of the optical element 40 will be described later. The optical element 40 forms an aerial image 2 in the air. The aerial image 2 is parallel to the element surface of the optical element 40 and is a two-dimensional image. The element surface refers to a virtual plane on which the optical element 40 extends in the in-plane direction. The element surface has the same meaning as in-plane. The element surfaces of the other elements have the same meaning. An observer 3 in front of the optical element 40 can view the aerial image 2.

The sensing element 50 is configured to be able to detect an object without contact. The sensing element 50 forms a detection area in a two-dimensional spatial region that includes part or all of the aerial image 2 generated by the aerial display device 1. The sensing element 50 detects an object (body) present in the detection area and detects the position of the object. In this embodiment, a capacitive sensing element is used as an example. Although the sensing element 50 is illustrated as being separated from the optical element 50 in FIG. 1, for example, the sensing element 50 is disposed close to the upper surface of the optical element 50. The area of the sensing element 50 is set to be equal to or larger than the area of the display element 20. The detailed configuration of the sensing element 50 will be described later.

[1-1-1] Configuration of the Light Control Element 30 Fig. 3A is a plan view of the light control element 30 shown in Fig. 1. Fig. 3B is a cross-sectional view of the light control element 30 taken along the line AA' in Fig. 3A.

The substrate 31 is configured to be planar in the XY plane and has a rectangular parallelepiped shape. The substrate 31 transmits light.

On the base material 31, a plurality of transparent members 33 are provided, each extending in the Y direction and aligned in the X direction. Also, on the base material 31, a plurality of light-shielding members 34 are provided, each extending in the Y direction and aligned in the X direction. The plurality of transparent members 33 and the plurality of light-shielding members 34 are alternately arranged such that adjacent ones are in contact with each other.

A substrate 32 is provided on the plurality of transparent members 33 and the plurality of light-shielding members 34. The substrate 32 is configured to be planar in the XY plane and has a rectangular parallelepiped shape. The substrate 32 transmits light.

The transparent member 33 extends in an oblique direction at an angle θ ₁ with respect to the normal direction of the base material 31 in the XZ plane. The transparent member 33 is a parallelogram with a side surface inclined by the angle θ ₁ in the XZ plane. The transparent member 33 transmits light.

The light blocking member 34 extends in an oblique direction at an angle θ ₁ with respect to the normal direction of the base material 31 in the XZ plane. The light blocking member 34 is a parallelogram with side surfaces inclined by an angle θ ₁ in the XZ plane. The light blocking member 34 blocks light. The thickness of the light blocking member 34 is set to be thinner than the thickness of the transparent member 33.

Two adjacent light blocking members 34 are positioned so that their ends slightly overlap in the Z direction.

Glass or transparent resin (including acrylic resin) is used for the base materials 31 and 32 and the transparent member 33. For example, resin mixed with black dye or pigment is used for the light blocking member 34.

The light control element 30 may be constructed by omitting one or both of the base materials 31 and 32. The function of the light control element 30 can be realized if multiple transparent members 33 and multiple light blocking members 34 are arranged alternately.

The light control element 30 configured in this manner can transmit the display light so that the light intensity in the oblique direction at an angle θ ₁ with respect to the normal direction becomes a peak. For example, the light control element 30 is configured to block light components other than the range of 30°±30° with respect to the normal direction. Desirably, the light control element 30 is configured to block light components other than the range of 30°±20° with respect to the normal direction.

As a modified example, the light control element 30 may be disposed between the lighting element 10 and the display element 20. Also, the light control element 30 may be omitted when constructing the aerial display device 1.

[1-1-2] Configuration of the Optical Element 40 Fig. 4 is a perspective view of the optical element 40 shown in Fig. 1. Fig. 4 also shows an enlarged view of a portion of the optical element 40. The enlarged view in Fig. 4 is a side view in the XZ plane.

The optical element 40 includes a substrate 41 and a plurality of optical elements 42. The substrate 41 is configured to be planar in the XY plane and has a rectangular parallelepiped shape.

A plurality of optical elements 42 are provided on the bottom surface of the substrate 41. Each of the plurality of optical elements 42 is formed as a triangular prism. The optical element 42 is arranged so that three side surfaces of the triangular prism are parallel to the XY plane, and one side surface is in contact with the substrate 41. The plurality of optical elements 42 each extend in the Y direction and are arranged side by side in the X direction. In other words, the plurality of optical elements 42 have a sawtooth shape in the XZ plane.

Each of the multiple optical elements 42 has an incident surface 43 and a reflective surface 44. When viewed from the Y direction, the left side surface is the incident surface 43, and the right side surface is the reflective surface 44. The incident surface 43 is a surface onto which light from the display element 20 is incident. The reflective surface 44 is a surface that reflects light that has been incident on the incident surface 43 from the outside, within the optical element 42. The incident surface 43 and the reflective surface 44 form an angle θ _p .

The substrate 41 and the optical element 42 are made of a transparent material. The optical element 42 is, for example, formed integrally with the substrate 41 using the same transparent material as the substrate 41. The substrate 41 and the optical element 42 may be formed separately, and the optical element 42 may be adhered to the substrate 41 using a transparent adhesive. The transparent material constituting the substrate 41 and the optical element 42 may be glass or a transparent resin (including an acrylic resin).

The optical element 40 configured in this way internally reflects incident light and forms a real image in the air. The optical element 40 also forms an aerial image 2 at a position in front of the element surface.

[1-1-3] Configuration of sensing element 50 Fig. 5A is a perspective view of the sensing element 50 shown in Fig. 1. Fig. 5B is a plan view of the sensing element 50 shown in Fig. 5A. Fig. 5B also shows a circuit configuration of the sensing element 50. Fig. 5C is a cross-sectional view of the sensing element 50 taken along line BB' shown in Fig. 5B.

The sensing element 50 includes two substrates 51 and 52, a plurality of electrodes 53 and 54, and an insulating layer 55. The substrates 51 and 52 are made of transparent and insulating substrates, and are made of glass substrates or plastic substrates.

A plurality of electrodes 53 are provided on the substrate 51. The planar shape of each of the plurality of electrodes 53 is, for example, a square. The electrodes 53 are arranged so that one diagonal line is parallel to the X direction. The plurality of electrodes 53 are arranged in a matrix. One row of the electrodes 53 aligned in the Y direction is electrically connected via a plurality of connection electrodes extending in the Y direction. One row of the electrodes 53 aligned in the Y direction is connected to a signal line TL via an extraction electrode extending in the Y direction. The electrodes 53 function as transmission electrodes.

A plurality of electrodes 54 are provided on the substrate 52. Each of the plurality of electrodes 54 has a planar shape, for example, a square. The electrodes 54 are arranged so that one diagonal line is parallel to the X direction. The plurality of electrodes 54 are arranged in a matrix. One row of the electrodes 54 aligned in the X direction are electrically connected via a plurality of connection electrodes extending in the X direction. One row of the electrodes 54 aligned in the X direction are connected to a signal line RL via an extraction electrode extending in the X direction. The electrodes 54 function as receiving electrodes.

The multiple electrodes 53 and the multiple electrodes 54 are arranged so as not to overlap in a plan view. The substrates 51 and 52 are arranged so that the multiple electrodes 53 and the multiple electrodes 54 face each other. The electrodes 53 and 54 are made of transparent electrodes, for example, made of ITO (indium tin oxide).

An insulating layer 55 is provided between the multiple electrodes 53 and the multiple electrodes 54. The insulating layer 55 is made of a transparent insulating material. The insulating layer 55 may be made of multiple layers.

The sensing element 50 further includes a drive circuit 56 and a detection circuit 57. The drive circuit 56 is connected to the multiple signal lines TL. The drive circuit 56 supplies a drive signal having a pulse waveform to each of the multiple signal lines TL. The drive circuit 56 also generates the drive signal so that pulses are sequentially applied to the multiple signal lines TL.

The detection circuit 57 is connected to multiple signal lines RL. The detection circuit 57 detects the voltage values of the multiple signal lines RL.

[1-1-4] Block configuration of the aerial display device 1 Figure 6 is a block diagram of the aerial display device 1. The aerial display device 1 includes a control unit 60, a storage unit 61, an input/output interface (input/output IF) 62, a display unit 63, a sensing element 50, and an input unit 64. The control unit 60, the storage unit 61, and the input/output interface 62 are connected to each other via a bus 65.

The input/output interface 62 is connected to the display unit 63, the sensing element 50, and the input unit 64. The input/output interface 62 performs interface processing for each of the display unit 63, the sensing element 50, and the input unit 64 in accordance with a predetermined standard.

The display unit 63 includes an illumination element 10 and a display element 20. The display unit 63 displays an image.

The sensing element 50 forms multiple detection regions above. The sensing element 50 detects objects present in the multiple detection regions based on changes in the capacitance between the electrodes 53 and 54.

The control unit 60 is composed of one or more processors, such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 60 realizes various functions by executing programs stored in the memory unit 61. The control unit 60 includes a display processing unit 60A, an image processing unit 60B, and a detection position determination unit 60C.

The display processing unit 60A controls the operation of the display unit 63 (specifically, the lighting element 10 and the display element 20). The display processing unit 60A controls the on and off of the lighting element 10. The display processing unit 60A transmits an image signal to the display element 20, causing the display element 20 to display an image.

The image processing unit 60B generates an image to be displayed by the aerial display device 1. The image processing unit 60B can use image data stored in the memory unit 61. The image processing unit 60B may also acquire image data from outside using a communication function (not shown).

The detection position determination unit 60C controls the operation of the sensing element 50. Based on the detection results of the sensing element 50, the detection position determination unit 60C calculates the detection position where the object is detected for each of the multiple detection areas. Based on the multiple detection positions, the detection position determination unit 60C determines the direction (vector) in which the object moves. The detection position determination unit 60C calculates the position of the intersection between the extension line of the direction in which the object moves and the display surface of the display element 20. The detection position determination unit 60C determines whether the calculated position on the display surface is included in the area of the image of the display element 20.

The storage unit 61 includes non-volatile storage devices such as a ROM (Read Only Memory), a HDD (Hard Disk Drive), and a SSD (Solid State Drive), and volatile storage devices such as a RAM (Random Access Memory) and a register. The storage unit 61 stores the programs executed by the control unit 60. The storage unit 61 stores various data necessary for the control of the control unit 60. The storage unit 61 stores data for images to be displayed by the aerial display device 1.

The input unit 64 includes, for example, a touch panel or buttons, and receives information input by the user. The image processing unit 60B can select an image to be displayed on the display unit 63 based on the information received by the input unit 64.

[1-2] Operation Next, the operation of the aerial display device 1 configured as described above will be described.

[1-2-1] Basic operation of the aerial display device 1 First, the basic operation of the aerial display device 1 will be described.

The arrows in Fig. 2 indicate optical paths. As shown in Fig. 2, light emitted from an arbitrary point "o" of the display element 20 is incident on the light control element 30. Of the light emitted from the display element 20, a light component at an angle _θ1 (including a light component within a predetermined angle range centered on the angle _θ1 ) is transmitted through the light control element 30. The light transmitted through the light control element 30 is incident on the optical element 40. The optical element 40 reflects the incident light to the opposite side to the light control element 30, and forms an aerial image 2 in the air.

Figure 7 is a perspective view illustrating how light is reflected in the optical element 40. Figure 8 is a side view of the XZ plane illustrating how light is reflected in the optical element 40. Figure 8 is a view of the optical element 40 when both eyes of the observer 3 (i.e., the line connecting both eyes) are parallel to the X direction. Figure 9 is a side view of the YZ plane illustrating how light is reflected in the optical element 40. Figure 9 is a view of the optical element 40 when both eyes of the observer 3 are parallel to the Y direction.

Light emitted from an arbitrary point "o" of the display element 20 enters the incident surface 43 of the optical element 40 and reaches the reflecting surface 44. Light that reaches the reflecting surface 44 at an angle greater than the critical angle with respect to the normal direction of the reflecting surface 44 is totally reflected by the reflecting surface 44 and is emitted from the flat surface opposite the side on which the optical element 40's optical elements 42 are formed. The critical angle is the smallest incident angle above which total reflection occurs. The critical angle is the angle with respect to the normal to the incident surface.

In the XZ plane of FIG. 8, the light emitted from point "o" is totally reflected by the reflecting surface 44 of the optical element 42, and the light is focused in the air to generate an aerial image.

In the YZ plane of FIG. 9, the light emitted from point "o" is not reflected by the reflecting surface 44 of the optical element 42, and the light does not form an image in the air, so it does not contribute to the generation of an aerial image.

In other words, the condition for observer 3 to be able to see the aerial image is that both eyes of observer 3 are parallel to the X direction or close to it (for example, ±10 degrees with respect to the X direction). Also, if observer 3's eyes are parallel to the X direction or close to it and the viewpoint is moved along the Y direction, observer 3 can always recognize the aerial image.

Figure 10 is a diagram explaining the angular conditions of the incident surface 43 and the reflecting surface 44 of the optical element 40.

The angle of the incident surface 43 with respect to the Z direction (the direction perpendicular to the element surface) is θ ₂ , the angle of the reflecting surface 44 with respect to the Z direction is θ ₃ , and the angle between the incident surface 43 and the reflecting surface 44 is θ _p . The angle θ _p is expressed by the following formula (1).
θ _p = θ ₂ + θ ₃ ...(1)

Light emitted from the light control element 30 at an angle θ ₁ is incident on the incident surface 43. The refractive index of the material of the optical element 40 is n _p , and the refractive index of air is 1. The incident angle at the incident surface 43 is θ ₄ , and the refraction angle is θ _5. The incident angle at the reflecting surface 44 is θ ₆ , and the reflection angle is θ ₇ (=θ ₆ ). The incident angle at the upper surface of the optical element 40 is θ ₈ , and the refraction angle is θ _9. The refraction angle θ ₉ is the exit angle. The exit angle θ ₉ is expressed by the following equation (2).
θ ₉ = sin ^-1 (n _p *sin (sin ^-1 ((1/n _p ) * sin (90° - (θ ₁ + θ ₂ ))) + θ ₂ +2θ ₃ -90°)) ... (2)

The critical angle at the reflecting surface 44 is expressed by the following formula (3).
Critical angle <θ ₆ (=θ ₇ )
Critical angle = sin ^-1 (1/n _p )...(3)

That is, the angle of incidence _θ6 on the reflecting surface 44 is set to be larger than the critical angle on the reflecting surface 44. In other words, the angle _θ3 of the reflecting surface 44 is set so that the angle of incidence of light incident on the reflecting surface 44 is larger than the critical angle.

Moreover, the angle θ2 of the incident surface 43 is set so that the light incident on the incident surface 43 is not totally reflected by the incident surface 43. In other words, the angle _θ2 of the incident surface 43 is set so that the angle of incidence of the light incident on the incident surface 43 is smaller than the critical angle.

The angle between the element surface of the optical element 40 and the surface of the aerial image 2, and the distance between the element surface of the optical element 40 and the surface of the aerial image 2 can be adjusted by optimally setting the angle θ ₁ of light incident on the optical element 40, the refractive index of the optical element 40, the angle θ ₂ of the incident surface 43 of the optical element 40, and the angle θ ₃ of the reflecting surface 44 of the optical element 40.

[1-2-2] Operation of Sensing Element 50 Next, the operation of the sensing element 50 will be described.

Figure 11 is a schematic diagram illustrating the operation of the sensing element 50.

The drive circuit 56 supplies a drive signal to the electrode 53 via the signal line TL. The drive signal consists of a pulse waveform having a predetermined period. The electrodes 53 and 54 are capacitively coupled, and an electric field is generated between them. The detection circuit 57 is configured to be able to detect the voltage value of the electrode 54.

The finger 3A of the observer 3 has a ground voltage GND. When an object (e.g., the finger 3A of the observer 3) approaches the sensing element 50, the object and the electrode 53 are capacitively coupled. In this case, the capacitance value between the electrodes 53 and 54 changes. Specifically, when the object and the electrode 53 are capacitively coupled, the capacitance value between the electrodes 53 and 54 decreases. When the capacitance value between the electrodes 53 and 54 changes, the voltage of the electrode 54 changes. The detection circuit 57 detects the voltage value of the electrode 54.

Figure 12 is a schematic diagram illustrating multiple detection regions of the sensing element 50.

The sensing element 50 can form multiple detection regions based on the detected voltage value. The detection region here is a two-dimensional region that the sensing element 50 can detect, and is a region parallel to the top surface of the sensing element 50. The multiple detection regions have different heights (levels) from the top surface of the sensing element 50. In FIG. 12, four detection regions DE1 to DE4 are shown as an example.

Figure 13 is a diagram explaining the relationship between multiple detection areas and voltage in the sensing element 50. In Figure 13, when an object is detected, it is indicated as "touch on," and when an object is not detected, it is indicated as "touch off."

The detection circuit 57 detects the voltages of the multiple signal lines RL. The multiple voltages detected by the detection circuit 57 are sent to the detection position determination unit 60C. The detection position determination unit 60C compares the voltage value detected by the detection circuit 57 with thresholds Th1 to Th4. If the voltage value is equal to or greater than the threshold Th1 and less than the threshold Th2, the detection position determination unit 60C determines that an object is present in the detection area DE1. If the voltage value is equal to or greater than the threshold Th2 and less than the threshold Th3, the detection position determination unit 60C determines that an object is present in the detection area DE2. If the voltage value is equal to or greater than the threshold Th3 and less than the threshold Th4, the detection position determination unit 60C determines that an object is present in the detection area DE3. If the voltage value is equal to or greater than the threshold Th4, the detection position determination unit 60C determines that an object is present in the detection area DE4.

In this way, the sensing element 50 can detect objects at multiple levels in the air above it.

[1-2-3] Detection operation of the aerial display device 1 Next, the detection operation of the aerial display device 1 will be described.

Figure 14 is a schematic diagram illustrating an aerial image 2 displayed by an aerial display device 1. The aerial display device 1 includes a housing 4. The housing 4 houses an illumination element 10, a display element 20, a light control element 30, an optical element 40, and a sensing element 50. The housing 4 has an opening at the top that exposes the sensing element 50. In Figure 14, the aerial display device 1 is positioned so that the vertical direction is the Y direction.

The aerial display device 1 displays an aerial image 2 in the air. In FIG. 14, a push button is shown as an example of the aerial image 2. An observer 3 presses the push button with his/her finger 3A. The sensing element 50 forms a detection area in a two-dimensional space that includes the aerial image 2, and detects the finger 3A when the observer 3 presses the push button with his/her finger 3A.

Figure 15 is a schematic diagram that explains how the position of the aerial image 2 changes depending on the position of the viewpoint of the observer 3.

As a result of the structure of the optical element 40, when the observer 3 moves his/her viewpoint along the Y direction, the position of the aerial image 2 changes to follow the viewpoint of the observer 3. When the observer 3's viewpoint moves upward on the drawing, the aerial image 2 also moves upward, and when the observer 3's viewpoint moves downward on the drawing, the aerial image 2 also moves downward. In other words, the aerial image 2 is always located directly in front of the observer 3.

Figure 16 is a schematic diagram explaining the detection operation of the aerial display device 1. Figure 17 is a schematic diagram explaining the detection operation when an observer 3 views the aerial display device 1 from an oblique direction.

The display element 20 displays an image 21 on its screen. The image 21 is, for example, a push button. The aerial display device 1 displays an aerial image 2 in a parallel two-dimensional spatial region on the upper surface of the display element 20.

The sensing element 50 forms multiple detection regions in a two-dimensional spatial region parallel to the aerial image 2. In this embodiment, an example will be described in which the sensing element 50 forms two detection regions. The sensing element 50 forms detection region DE1 and detection region DE2 in order from the side furthest from the sensing element 50. The detection region DE2 is set to be in the same two-dimensional spatial region as the aerial image 2, i.e., to overlap with the aerial image 2.

The observer 3 touches the aerial image 2 with his/her finger 3A. The finger 3A of the observer 3 passes through the detection area DE1 and the detection area DE2. The sensing element 50 detects an object in each of the detection area DE1 and the detection area DE2.

The upper surface of the sensing element 50 is the reference plane of the coordinates. In this embodiment, the coordinates define the position of an object in space, and have the same meaning as the position of the object. The coordinates of the image 21 are (x, y, z). The coordinates of the detection position of the object in the detection area DE1 are ( _x1 , _y1 , _z1 ), and the coordinates of the detection position of the object in the detection area DE2 are ( _x2 , _y2 , _z2 ). As shown in FIG. 16, the detection positions of the object in the detection areas DE1 and DE2 differ depending on the position of the viewpoint of the observer 3.

From the equation of a straight line in space, the following formula (4) holds.
(x-x ₁ )/(x ₂ -x ₁ )=(y-y ₁ )/(y ₂ -y ₁ )=(zz-z ₁ )/(z ₂ -z ₁ )...(4)

It is assumed that when the push button is pressed, the deviation in the X direction is small. That is, the following equation (5) is satisfied in the X coordinate.
x ₁ = x ₂ (=x) ... (5)

From equation (4), the following equation (6) holds in the Y coordinate.
y = y ₁ + (z - z ₁ ) * (y ₂ - _{y 1} )/(z ₂ - z ₁ ) ... (6)

The detection position determination unit 60C determines the position of the object based on the voltage value detected by the sensing element 50. The detection position by the detection position determination unit 60C includes coordinates in the detection areas DE1 and DE2. _z1 is the distance from the reference plane to the detection area DE1, and _z2 is the distance from the reference plane to the detection area DE2. _z1 and _z2 are predetermined design values.

Figure 18 is a schematic diagram explaining a method for calculating the position of the image 21 in the Z coordinate. In Figure 18, the display surface is the display surface of the display element 20, the reference surface is the upper surface of the sensing element 50, and the aerial image surface is the surface on which the aerial image 2 is displayed.

As described above, light emitted from the display element 20 at an angle _θ1 is reflected by the optical element 40 to the front of the viewer 3. The distance between the display element 20 and the optical element 40 is taken as l. The reference surface of the optical element 40 when defining the distance l is, for example, the upper surface of the optical element 40. The distance l is a predetermined design value.

In the Z coordinate, the following equation (7) holds.
z=l/sinθ ₁ ...(7)

Figure 19 is a schematic diagram explaining a virtual display surface with a corrected Z coordinate. The virtual display surface is a virtual display surface obtained by correcting the position of the display surface of the display element 20, assuming that light from the display element 20 travels in a straight line. The virtual display surface is obtained by correcting the position of the display surface using equation (7). The light of the image displayed on the virtual display surface virtually travels in the Z direction and is observed by the observer 3.

The coordinates (x, y) on the virtual display surface can be calculated from equations (5) to (7). The coordinate calculation methods described above do not take into account the optical effects (refractive index, thickness, etc.) of the light control element 30, the optical element 40, and the sensing element 50, but these can be added and corrected as appropriate.

Figure 20 is a schematic diagram illustrating the operation of determining a touch operation by an observer 3. Figure 20 shows coordinates on a virtual display surface 22 of a display element 20. The display element 20 displays an image 21 on its screen.

The upper left corner of the virtual display surface 22 is defined as coordinates (0,0), and the lower right corner of the virtual display surface 22 is defined as coordinates (m,n). The upper left corner of the image 21 is defined as coordinates ( _m1 , _n1 ), and the lower right corner of the image 21 is defined as coordinates ( _m2 , _n2 ).

The detection position determination unit 60C calculates the positions ( _x1 , _y1 , _z1 ) and ( _x2 , _y2 , _z2 ) of the object in the detection areas DE1 and DE2 based on the detection results of the sensing element 50. Furthermore, the detection position determination unit 60C calculates the position (x, y) on the virtual display surface 22 based on the positions ( _x1 , _y1 , _z1 ) and ( _x2 , _y2 , _z2 ).

In other words, the detection position determination unit 60C determines the movement direction (vector) in which the object moves based on the positions ( _x1 , _y1 , _z1 ) and ( _x2 , _y2 , _z2 ). The detection position determination unit 60C calculates the position (x, y) which is the intersection of an extension line of the movement direction of the object and the virtual display surface 22. The detection position determination unit 60C calculates the extension line so as to trace in reverse the optical path of light that forms the aerial image 2 visually recognized by the observer 3. In other words, the detection position determination unit 60C calculates the extension line so as to trace in reverse the optical path of light that is obliquely emitted from the display element 20 and reflected by the optical element 40.

The detected position determination unit 60C determines whether or not the calculated position (x, y) is included in the area of the image 21. The area of the image 21 is a rectangular area defined by coordinates ( _m1 , _n1 ) and coordinates ( _m2 , _n2 ). When the calculated position (x, y) is included in the area of the image 21, the detected position determination unit 60C determines that the aerial image 2 has been selected, that is, that the observer 3 has touched the aerial image 2. On the other hand, when the calculated position (x, y) is not included in the area of the image 21, the detected position determination unit 60C determines that the aerial image 2 has not been selected, that is, that the observer 3 has not touched the aerial image 2.

[1-2-4] Flow of Overall Operation Next, we will explain the flow of the overall operation of the aerial display device 1. Figure 21 is a flowchart explaining the overall operation of the aerial display device 1.

The control unit 60 displays the aerial image 2 (step S100). The display processing unit 60A displays an image on the screen of the display element 20. The optical element 40 reflects light from the display element 20 and forms the aerial image 2 in the air.

Then, the sensing element 50 performs a sensing operation (step S101). The driving circuit 56 of the sensing element 50 supplies driving signals to the multiple electrodes 53 via the multiple signal lines TL. The detection circuit 57 of the sensing element 50 detects the voltages of the multiple electrodes 54 via the multiple signal lines RL. The sensing element 50 then forms detection regions DE1 and DE2 by comparing the detected voltage with multiple thresholds.

Next, the detection position determination unit 60C determines whether or not an object has been detected based on the signal from the sensing element 50 (step S102).

When an object is detected (S102 = Yes), the detection position determination unit 60C calculates the position of the object (step S103) based on the detection result of the sensing element 50. That is, the detection position determination unit 60C calculates the position ( _x1 , _y1 , _z1 ) in the detection area DE1 and the position ( _x2 , _y2 , _z2 ) in the detection area DE2.

Next, the detection position determination unit 60C calculates the position of the object on the virtual display surface of the display element 20 (step S104).

Next, the detection position determination unit 60C determines whether the calculated object position is included in the area of the image 21 (step S105).

If the position of the object is included in the area of the image 21 (S105 = Yes), the detection position determination unit 60C determines that the aerial image 2 has been touched by the observer 3 (step S106).

The control unit 60 then performs an operation according to the touch operation of the observer 3.

[1-3] Effects of the First Embodiment In the first embodiment, the sensing element 50 forms a plurality of detection areas (including detection areas DE1 and DE2) parallel to the display element 20 above the optical element 40, and detects an object present in the plurality of detection areas. The detection position determination unit 60C calculates the movement direction (travel direction) of the object based on the detection result of the sensing element 50. The detection position determination unit 60C calculates the position where an extension of the calculated movement direction intersects with the screen (display surface) of the display element 20. Then, the detection position determination unit 60C determines that the observer 3 has touched the aerial image when the position where the extension intersects with the screen is included in the area of the image of the display element 20.

Therefore, according to the first embodiment, even if the position of the aerial image 2 changes depending on the position from which the observer 3 views the aerial display device 1, it is possible to determine the object present in the area of the aerial image 2. In addition, it is possible to realize an aerial display device 1 that can more accurately detect the touch operation on the aerial image 2 by the observer 3.

The aerial display device 1 can display an aerial image 2 in the air by reflecting light emitted from the display element 20 by the optical element 40. The aerial display device 1 can display the aerial image 2 in the front direction, parallel to the element surface of the optical element 40. It is also possible to realize an aerial display device 1 that can improve display quality.

When the observer 3 looks at the optical element 40 with both eyes parallel to or close to the X direction (i.e., the direction in which the multiple optical elements 42 are arranged), the observer 3 can see an aerial image. When the observer 3 moves the viewpoint along the Y direction with both eyes parallel to or close to the X direction, the observer 3 can always see an aerial image. When the observer 3's eyes are parallel to or close to the X direction, a wider viewing angle can be achieved.

In addition, the multiple elements that make up the aerial display device 1 can be arranged in parallel. This makes it possible to realize an aerial display device 1 that can be made smaller in size in the Z direction.

[2] Second embodiment The second embodiment is an example of a display device 70 including a sensing element 50. The display device 70 is a general display device that displays an image on its own screen.

Figure 22 is a perspective view of a display device 70 according to a second embodiment of the present invention. The display device 70 includes an illumination element 10, a display element 20, and a sensing element 50. The illumination element 10, the display element 20, and the sensing element 50 are arranged parallel to one another. The illumination element 10, the display element 20, and the sensing element 50 are fixed at specific positions by a fixing member (not shown) so as to leave a specific distance between them. The illumination element 10, the display element 20, and the sensing element 50 are housed in a housing (not shown). The configurations of the illumination element 10, the display element 20, and the sensing element 50 are the same as those of the first embodiment.

Figure 23 is a schematic diagram illustrating the detection operation of the display device 70. Figure 24 is a schematic diagram illustrating the detection operation when the observer 3 views the display device 70 from an oblique direction. The operation of the sensing element 50 is the same as in the first embodiment.

The display element 20 displays an image 21 on its screen. The image 21 is, for example, a push button. The observer 3 visually recognizes the image 21 displayed on the screen of the display element 20.

The sensing element 50 forms multiple detection regions in a two-dimensional spatial region above and parallel to the display element 20. In this embodiment, an example will be described in which the sensing element 50 forms two detection regions. The sensing element 50 forms detection region DE1 and detection region DE2 in order from the side furthest from the sensing element 50. The sensing element 50 detects an object in each of the detection region DE1 and the detection region DE2.

The observer 3 operates the image 21 with his/her finger 3A. The finger 3A of the observer 3 passes through the detection area DE1 and the detection area DE2. The sensing element 50 detects an object in each of the detection area DE1 and the detection area DE2.

The upper surface of the sensing element 50 is the reference plane for the coordinates. The coordinates of the image 21 are (x, y, z). The coordinates of the detection position of the object in the detection area DE1 are ( _x1 , _y1 , _z1 ), and the coordinates of the detection position of the object in the detection area DE2 are ( _x2 , _y2 , _z2 ). As shown in Fig. 23, the detection positions of the object in the detection areas DE1 and DE2 differ depending on the position of the viewpoint of the observer 3.

As in the first embodiment, the above formula (4) holds true from the equation of a straight line in space. The above formulas (5) and (6) hold true in the X and Y coordinates.

In the Z coordinate, z is the distance between the reference surface and the display element 20, _z1 is the distance between the reference surface and the detection area DE1, and _z2 is the distance between the reference surface and the detection area DE2. z, _z1 , and _z2 are predetermined design values.

From the formulas (5) and (6), the detection position determination unit 60C calculates the coordinates (x, y) on the display element 20. In other words, the detection position determination unit 60C determines the movement direction (vector) in which the object moves based on the positions ( _x1 , _y1 , _z1 ) and ( _x2 , _y2 , _z2 ). The detection position determination unit 60C calculates the position (x, y) which is the intersection point between the extension line of the movement direction of the object and the display surface of the display element 20. The coordinate calculation methods described above do not take into account the optical influence (refractive index, thickness, etc.) of the sensing element 50, but may be appropriately added and corrected.

Next, the overall operation flow of the display device 70 will be described. FIG. 25 is a flowchart explaining the overall operation of the display device 70. The block diagram of the display device 70 is the same as that of the first embodiment.

The display processing unit 60A displays an image on the screen of the display element 20 (step S200). Then, the sensing element 50 performs a sensing operation (step S201). The sensing element 50 forms detection areas DE1 and DE2.

Next, the detection position determination unit 60C determines whether or not an object has been detected based on the signal from the sensing element 50 (step S202).

If an object is detected (S202 = Yes), the detection position determination unit 60C calculates the position of the object (step S203). That is, the detection position determination unit 60C calculates the position ( _x1 , _y1 , _z1 ) in the detection area DE1 and the position ( _x2 , _y2 , _z2 ) in the detection area DE2.

Next, the detection position determination unit 60C calculates the position of the object on the screen (display surface) of the display element 20 (step S204).

Next, the detection position determination unit 60C determines whether the calculated object position is included in the area of the image 21 (step S205).

If the position of the object is included in the area of image 21 (S205 = Yes), the detection position determination unit 60C determines that image 21 has been touched by the observer 3 (step S206).

The control unit 60 then performs an operation according to the touch operation of the observer 3.

According to the second embodiment, the sensing element 50 can be applied to a general display device 70 that displays an image on its own screen. In addition, the display device 70 can detect an object in a non-contact manner.

[3] Third Embodiment The third embodiment is another configuration example of the sensing element 50.

Figure 26 is a perspective view of a display device 70 according to a third embodiment of the present invention. The display device 70 includes multiple sensing elements 50. In this embodiment, two sensing elements 50-1 and 50-2 will be used as an example.

The sensing elements 50-1 and 50-2 are disposed above the display element 20. The sensing elements 50-1 and 50-2 are disposed outside the area occupied by the display element 20 in a plan view. For example, the sensing elements 50-1 and 50-2 are disposed on one side of the display device 70 in the X direction. The sensing element 50-1 is disposed above the sensing element 50-2. The lighting element 10, the display element 20, and the sensing elements 50-1 and 50-2 are fixed to specific positions by fixing members not shown. The lighting element 10, the display element 20, and the sensing elements 50-1 and 50-2 are housed in a housing not shown. The sensing elements 50-1 and 50-2 may be provided separately from the housing in which the lighting element 10 and the display element 20 are housed.

The sensing elements 50-1 and 50-2 form a detection area DE1 and a detection area DE2, respectively. The detection area DE1 and the detection area DE2 are set parallel to the display element 20. The detection area DE1 is formed above the detection area DE2.

The sensing element 50-1 detects an object present in the detection area DE1. The sensing element 50-1 emits light into the detection area DE1 and detects the reflected light reflected by the object. The light emitted by the sensing element 50-1 is preferably light other than visible light, for example infrared light. The sensing element 50-1 includes a light emitting unit that emits infrared light toward the detection area DE1, and a light receiving unit (sensor) that detects the reflected light reflected by the object. The sensing element 50-1 is composed of, for example, a line sensor in which multiple light emitting elements and multiple light receiving elements are alternately arranged in a row. The line sensor can scan space in a line using infrared light, and can scan a two-dimensional space consisting of the direction in which multiple light emitting elements are arranged and the direction in which light travels.

Similar to sensing element 50-1, sensing element 50-2 detects an object present in detection area DE2.

Figure 27 is a diagram explaining the configuration of sensing element 50-1. Figure 27(a) is a plan view, and Figure 27(b) is a side view. The configuration of sensing element 50-2 is the same as that in Figure 27.

The sensing element 50-1 comprises a substrate 80, a plurality of light-emitting elements 81, a plurality of light-receiving elements 82, a plurality of lenses 83, and a case 84. The number of light-emitting elements 81, light-receiving elements 82, and lenses 83 shown in FIG. 27 is an example. The plurality of light-emitting elements 81 are collectively referred to as the light-emitting section, and the plurality of light-receiving elements 82 are collectively referred to as the light-receiving section.

The multiple light-emitting elements 81 and the multiple light-receiving elements 82 are mounted on a substrate 80. The substrate 80 includes multiple wiring layers (not shown). The multiple light-emitting elements 81 and the multiple light-receiving elements 82 are arranged alternately.

Each of the multiple light-emitting elements 81 emits laser light. Laser light is, for example, infrared light that does not stimulate the user's vision and is not affected by visible light. The light-emitting elements 81 are, for example, composed of laser diodes.

Each of the multiple light receiving elements 82 detects the laser light (infrared light) reflected by the target object. The light receiving elements 82 are composed of, for example, a photodiode.

The multiple lenses 83 are provided corresponding to the multiple light-emitting elements 81 and are arranged above the multiple light-emitting elements 81. The multiple lenses 83 are arranged side by side in one direction. Each of the multiple lenses 83 is composed of, for example, a plano-convex lens. The multiple lenses 83 have the function of focusing the laser light emitted from the multiple light-emitting elements 81 in the forward direction. The multiple lenses 83 also have the function of focusing the laser light reflected by an object.

The case 84 houses the substrate 80, a plurality of light-emitting elements 81, a plurality of light-receiving elements 82, and a plurality of lenses 83. The case 84 has an opening at the top that exposes the lenses 83. In Figure 27, only the external shape of the case 84 is shown in a simplified form.

The detection operation of the display device 70 is the same as in the second embodiment. In the third embodiment, the same effect as in the second embodiment can be obtained. The sensing element of the third embodiment can be applied to the first embodiment.

[4] Fourth Embodiment In the fourth embodiment, when the observer 3 touches an area other than the aerial image 2, a selection position is presented by a cursor.

Figure 28 is a flowchart explaining the overall operation of the aerial display device 1 according to the fourth embodiment of the present invention. The configuration of the aerial display device 1 is the same as that of the first embodiment.

The operations in steps S300 to S306 are the same as those in steps S100 to S106 in the first embodiment.

If the position of the object is not included in the area of the image 21 (S305 = No), the detection position determination unit 60C displays the cursor 23 at the selected position on the virtual display surface 22 (step S307).

Figure 29 is a schematic diagram illustrating the operation of determining a touch operation by an observer 3. Figure 29 shows coordinates on the virtual display surface 22 of the display element 20. The display element 20 displays an image 21 on its screen.

When the position selected by the observer 3 is not included in the area of the image 21, the detection position determination unit 60C displays a cursor 23 at the position selected by the observer 3 on the virtual display surface 22 of the display element 20. By visually recognizing the cursor 23, the observer 3 can recognize that he or she is touching an area other than the aerial image 2. This allows the finger 3A of the observer 3 to be guided to the aerial image 2.

The fourth embodiment can also be applied to the display device 70 of the second embodiment.

As a modified example, the observer 3 may be notified of the non-selected state by using sound. The detection position determination unit 60C outputs a specific sound when the position selected by the observer 3 is not included in the area of the image 21. In this modified example, the aerial display device 1 is equipped with a speaker capable of outputting sound. This allows the observer 3 to recognize that the aerial image 2 is not being touched.

[5] Modifications In the above embodiment, the sensing element 50 is of a capacitive type. However, the present invention is not limited to this, and other types of sensing elements capable of detecting objects in space can be used.

In the above embodiment, the display element 20 and the optical element 40 are arranged parallel to each other. However, this is not limited to the above, and the display element 20 may be arranged diagonally relative to the optical element 40. The angle between the display element 20 and the optical element 40 is set in the range of more than 0 degrees and less than 45 degrees. In this modified example, the light control element 30 can be omitted.

In the above embodiment, the left side of the optical element 42 is defined as the incident surface 43, and the right side is defined as the reflective surface 44. However, this is not limited to this, and the incident surface 43 and the reflective surface 44 may be configured in reverse. In this case, the action of the aerial display device 1 described in the embodiment is also reversed.

In the above embodiment, a liquid crystal display element is described as an example of the display element 20, but the present invention is not limited to this. The display element 20 may be a self-luminous organic EL (electroluminescence) display element or a micro LED (Light Emitting Diode) display element. A micro LED display element is a display element that uses LEDs to emit the R (red), G (green), and B (blue) that make up a pixel. When a self-luminous display element 20 is used, the illumination element 10 is not necessary.

The present invention is not limited to the above-described embodiments, and can be modified in various ways in the implementation stage without departing from the gist of the invention. The embodiments may also be implemented in appropriate combination, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combinations selected from the multiple constituent elements disclosed. For example, if the problem can be solved and an effect can be obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiments, the configuration from which these constituent elements are deleted can be extracted as an invention.

1...Aerial display device, 2...Aerial image, 3...Observer, 4...Housing, 10...Illumination element, 11...Light source unit, 12...Light guide plate, 13...Reflective sheet, 20...Display element, 21...Image, 22...Virtual display surface, 23...Cursor, 30...Light control element, 31...Substrate, 32...Substrate, 33...Transparent member, 34...Light blocking member, 40...Optical element, 41...Substrate, 42...Optical element, 43...Incident surface, 44...Reflective surface, 50...Sensing element, 51...substrate, 52...substrate, 53...electrode, 54...electrode, 55...insulating layer, 56...drive circuit, 57...detection circuit, 60...control unit, 60A...display processing unit, 60B...image processing unit, 60C...detection position determination unit, 61...memory unit, 62...input/output interface, 63...display unit, 64...input unit, 65...bus, 70...display device, 80...substrate, 81...light-emitting element, 82...light-receiving element, 83...lens, 84...case.

Claims (14)

A display element for displaying an image;
an optical element that is arranged to receive light from the display element, and that reflects the light from the display element to an opposite side to the display element to form an aerial image in the air;
a sensing element that forms a first detection area parallel to the display element and a second detection area at a different level from the first detection area above the optical element, and detects an object present in the first and second detection areas;
a determination unit that calculates a first detection position in the first detection area and a second detection position in the second detection area, and determines whether or not the aerial image has been touched based on the first and second detection positions. The aerial display device according to claim 1 , wherein the determination unit determines a direction of movement of the object based on the first and second detection positions, and determines that the aerial image has been touched when the image is present on an extension line of the direction of movement. The aerial display device according to claim 1 , wherein the sensing element is of a capacitive type. the sensing element includes a plurality of first electrodes, a plurality of second electrodes arranged opposite to the plurality of first electrodes, a drive circuit that supplies drive signals to the plurality of first electrodes, and a detection circuit that detects voltages of the plurality of second electrodes;
The aerial display device according to claim 3 , wherein the determination unit compares the detection result by the detection circuit with a threshold value, and determines the position of the object based on the comparison result. the sensing element includes a first sensing element forming the first sensing area and a second sensing element forming the second sensing area;
The aerial display device according to claim 1 , wherein each of the first sensing element and the second sensing element includes an emitter that emits light toward the detection area and a light receiver that receives reflected light reflected by the object. The optical element includes a planar substrate and a plurality of optical elements provided under the substrate, each of the optical elements extending in a first direction and aligned in a second direction perpendicular to the first direction;
The aerial display device according to claim 1 , wherein each of the optical elements has an incident surface and a reflecting surface that are inclined with respect to a normal direction of the base material and are in contact with each other. The aerial display device according to claim 1 , further comprising a light control element disposed between the display element and the optical element, which transmits oblique light components of the light from the display element. The light control element includes a plurality of transparent members and a plurality of light blocking members arranged alternately,
The aerial display device according to claim 7 , wherein the plurality of light-shielding members are inclined with respect to a normal to the light control element. The aerial display device according to claim 1 , wherein the display element and the optical element are arranged parallel to each other. A display element for displaying an image;
a sensing element that forms a first detection area parallel to the display element and a second detection area at a different level from the first detection area above the display element, and detects an object present in the first and second detection areas;
a determination unit that calculates a first detection position in the first detection area and a second detection position in the second detection area, and determines whether or not the image has been touched based on the first and second detection positions. The display device according to claim 10 , wherein the determination unit determines a moving direction in which the object moves based on the first and second detection positions, and determines that the image has been touched when the image is present on an extension line of the moving direction. The display device according to claim 10 , wherein the sensing element is of a capacitive type. the sensing element includes a plurality of first electrodes, a plurality of second electrodes arranged opposite to the plurality of first electrodes, a drive circuit that supplies drive signals to the plurality of first electrodes, and a detection circuit that detects voltages of the plurality of second electrodes;
The display device according to claim 12 , wherein the determination section compares a detection result by the detection circuit with a threshold value, and determines a position of the object in accordance with a comparison result. the sensing element includes a first sensing element forming the first sensing area and a second sensing element forming the second sensing area;
The display device according to claim 10 , wherein each of the first sensing element and the second sensing element includes a light-emitting portion that emits light toward the detection area and a light-receiving portion that receives light reflected by the object.