JP2022100561A - Spatial floating image display apparatus - Google Patents

Abstract

To provide a suitable spatial floating image display apparatus and the like with high convenience for a user.SOLUTION: A spatial floating image display apparatus 1000 includes: a light source and an image source (display device 1); a retroreflective member 2 that forms a spatial floating image of predetermined directivity on the basis of image light; a housing 60 that houses the display device and the retroreflective member; a drive mechanism 62 that drives the housing 60 to move; a controller (control board 61) that controls display of a spatial floating image 3 formed at a predetermined position in a space outside the housing 60; a first sensor (sensor 31) for detecting operation to the spatial floating image 3 by a user who visually recognizes and operates the spatial floating image 3; and a second sensor (camera 32) for detecting a position of a face UF of the user to the spatial floating image 3. The controller adjusts the state of the spatial floating image 3 according to the position of the face UF of the user by moving the housing 60 using the drive mechanism 62 according to the position of the face UF of the user to the spatial floating image 3.SELECTED DRAWING: Figure 26

Description

The present invention relates to a technique such as a space floating image display device.

As an example of the prior art relating to the space floating image display device and the like, Japanese Patent Application Laid-Open No. 2019-128722 (Patent Document 1) can be mentioned. Patent Document 1 describes a technique for reducing false positives on the operation surface of a displayed spatial image.

Japanese Unexamined Patent Publication No. 2019-128722

The spatial floating image display device of the prior art example has room for improvement in terms of visibility and ease of operation of the spatial floating image by the user.

An object of the present invention is to provide a suitable space floating image display device or the like that is highly convenient for the user.

In order to solve the above problems, for example, the configuration described in the claims is adopted. The spatial floating image display device according to the embodiment of the present invention forms a light source, an image source that emits image light based on the light from the light source, and a spatial floating image having a predetermined directionality based on the image light. A retroreflection member, a housing that houses the light source, the image source, and the retroreflection member, a drive mechanism that moves the housing or components inside the housing, and a space outside the housing. A controller for controlling the display of the spatial floating image formed at a predetermined position, a first sensor for detecting an operation on the spatial floating image by a user who visually recognizes and operates the spatial floating image, and the above. The controller comprises a second sensor for detecting the position of the user's face or head with respect to the spatial floating image, and the controller may use the housing or the housing or the housing according to the position of the user's face or head with respect to the spatial floating image. The components in the housing are moved by the drive mechanism.

According to the present invention, it is possible to provide a suitable space floating image display device or the like that is highly convenient for the user. Issues, configurations, effects, etc. other than the above are shown in [Modes for Carrying Out the Invention].

It is a figure which shows an example of the usage form of the space floating image display device which concerns on one Embodiment of this invention. It is a figure which shows an example of the main part structure and the retroreflection part structure of the space floating image display device which concerns on one Embodiment of this invention. It is a figure which shows an example of the installation method of a space floating image display device. It is a figure which shows another example of the installation method of a space floating image display device. It is a figure which shows the configuration example of the space floating image display device. It is a figure which shows the other example of the main part composition of the space floating image display device which concerns on one Embodiment of this invention. It is explanatory drawing for demonstrating the function of the sensing apparatus used in the space floating image display apparatus. It is explanatory drawing of the principle of 3D image display used in a space floating image display apparatus. It is explanatory drawing of the measurement system which evaluated the characteristic of a reflective polarizing plate. It is a characteristic diagram which shows the transmittance characteristic with respect to the ray incident angle of a reflective polarizing plate transmission axis. It is a characteristic diagram which shows the transmittance characteristic with respect to the ray incident angle of a reflective polarizing plate reflection axis. It is a characteristic diagram which shows the transmittance characteristic with respect to the ray incident angle of a reflective polarizing plate transmission axis. It is a characteristic diagram which shows the transmittance characteristic with respect to the ray incident angle of a reflective polarizing plate reflection axis. It is sectional drawing which shows an example of the specific structure of a light source device. It is sectional drawing which shows an example of the specific structure of a light source device. It is sectional drawing which shows an example of the specific structure of a light source device. It is a layout drawing which shows the main part of the space floating image display device which concerns on one Embodiment of this invention. It is sectional drawing which shows the structure of the display device which concerns on one Embodiment of this invention. It is sectional drawing which shows an example of the specific structure of a light source device. It is sectional drawing which shows an example of the specific structure of a light source device. It is sectional drawing which shows an example of the specific structure of a light source device. It is explanatory drawing for demonstrating the light source diffusion characteristic of a display device. It is explanatory drawing for demonstrating the diffusion characteristic of a display device. It is explanatory drawing for demonstrating the diffusion characteristic of a display device. It is explanatory drawing for demonstrating the generation principle of a ghost image. It is explanatory drawing for demonstrating the generation principle of a ghost image in the prior art. It is sectional drawing which shows the structure of the display device which concerns on one Embodiment of this invention. It is explanatory drawing which shows the structure of the space floating image display apparatus of Embodiment 1. FIG. It is explanatory drawing which shows the display example of the space floating image in Embodiment 1. It is explanatory drawing which shows the structure in the case of the vertical installation in Embodiment 1. It is explanatory drawing which shows the structure which was seen from the top in Embodiment 1. FIG. FIG. 1 is an explanatory diagram for determining the position of a face by a camera in the first embodiment. In Embodiment 1, it is explanatory drawing about the rotational movement of a housing. It is a modification of Embodiment 1 and is explanatory drawing about the translation | translation of a housing. FIG. 5 is an explanatory diagram for determining the position of a face in the depth direction in a modified example of the first embodiment. It is a modification of Embodiment 1 and is explanatory drawing about the position of the face in the camera image. It is a modification of Embodiment 1 and is explanatory drawing about the reading of a code. It is explanatory drawing which shows the structure of the space floating image display apparatus of Embodiment 2. It is explanatory drawing which shows the display example of the space floating image in Embodiment 2. It is explanatory drawing which shows the processing flow in Embodiment 2. It is explanatory drawing about the determination (the 1) of the characteristic of an operation in Embodiment 2. FIG. 2 is an explanatory diagram for determining operation characteristics (No. 2) in the second embodiment. FIG. 2 is an explanatory diagram for determining the operation characteristics (No. 3) in the second embodiment. It is a modification of Embodiment 2 and is explanatory drawing about the reading of a code.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. It should be noted that the present invention is not limited to the description of the examples, and various modifications and modifications by those skilled in the art can be made within the scope of the technical idea disclosed in the present specification. In the drawings, those having the same function may be given the same reference numerals, and the repeated description may be omitted. In the drawings, the representation of each component may not represent the actual position, size, shape, range, etc. to facilitate understanding of the invention, and the invention is the position disclosed in the drawings. Etc. are not always limited. In the description, when explaining the processing by the program, the program, the function, the processing unit, etc. may be mainly described, but the main body as the hardware for them is the processor or the controller composed of the processor or the like. And so on. The apparatus executes processing according to the program read out on the memory by the processor while appropriately using resources such as the memory and the communication interface. As a result, a predetermined function, a processing unit, and the like are realized. The processor is composed of, for example, a semiconductor device such as a CPU or a GPU. A processor is composed of a device or a circuit capable of performing a predetermined operation. The processing is not limited to software program processing, but can be implemented by a dedicated circuit. FPGA, ASIC, etc. can be applied to the dedicated circuit. The program may be pre-installed as data in the target device, or may be distributed and installed as data from the program source to the target device. The program source may be a program distribution server on a communication network or a non-transient computer-readable storage medium. The program may be composed of a plurality of program modules. In addition, expressions such as identification information, identifiers, IDs, names, and numbers can be replaced with each other.

In the following embodiment, it is possible to transmit an image by image light from an image emitting source through a transparent member that partitions a space such as glass, and display it as a space floating image outside the transparent member. Regarding the space floating image display device.

According to the following embodiment, for example, a space floating image display device suitable for an ATM of a bank, a ticket vending machine at a station, a digital signage, or the like can be realized. For example, at present, a touch panel is usually used in ATMs of banks, ticket vending machines of stations, etc., but a transparent glass surface or a light-transmitting plate material is used on the glass surface or the light-transmitting plate material. High-resolution video information can be displayed in a floating state. At this time, by making the divergence angle of the emitted image light small, that is, making it a sharp angle, and further aligning it with a specific polarization, only the regular reflected light is efficiently reflected to the retroreflective member, so that the light utilization efficiency is improved. It is expensive, and it is possible to suppress ghost images generated in addition to the main spatial floating image, which has been a problem in the conventional retroreflection method, and it is possible to obtain a clear spatial floating image. Further, the device including the light source of the present embodiment can provide a new and highly usable spatial floating image display device (spatial floating image display system) capable of significantly reducing power consumption. Further, for example, it is possible to provide a so-called unidirectional space floating image display device capable of displaying a so-called unidirectional space floating image that can be visually recognized inside and / or outside the vehicle.

On the other hand, in the conventional technique, as shown in FIGS. 23 and 24, an organic EL panel or a liquid crystal panel is combined with the retroreflective member 151 as a high-resolution color display image source 150. In the conventional technique, the image light is diffused at a wide angle. Therefore, in addition to the reflected light normally reflected by the retroreflective member 151, the ghost image 301 and the ghost image 301 are generated by the image light obliquely incident on the retroreflective member 2a as shown in FIG. 302 was generated and the image quality of the floating image in space was impaired. Further, as shown in FIG. 23, a plurality of first ghost images 301, second ghost images 302, and the like are generated in addition to the normal space floating image 300. For this reason, other than the observer, the floating image in the same space, which is a ghost image, is monitored, which poses a big problem in terms of security.

<Example 1 of space floating image display device>
FIG. 1 is a diagram showing an example of a usage mode of the space floating image display device according to an embodiment of the present invention, and is a diagram showing an overall configuration of the space floating image display device according to the present embodiment. The specific configuration of the spatial floating image display device will be described in detail with reference to FIG. 2, etc., but light having a narrow directional characteristic and a specific polarization is emitted from the display device 1 as an image light beam and is retroreflected. Once incident on the member 2, it is retroreflected and transmitted through the transparent member 100 (glass or the like) to form an aerial image (spatial floating image 3) which is a real image on the outside of the glass surface.

Further, in a store or the like, a space is partitioned by a show window (also referred to as "wind glass") 105 which is a translucent member such as glass. According to the spatial floating image display device of the present embodiment, it is possible to display the floating image in one direction with respect to the outside and / or the inside of the store (space) through the transparent member.

In FIG. 1A, the inside of the wind glass 105 (inside the store) is oriented in the depth direction, and the outside (for example, a sidewalk) is shown to be in front. On the other hand, it is also possible to reflect the specific polarization by providing the wind glass 105 with a means for reflecting the specific polarization, and to form an aerial image at a desired position in the store.

FIG. 1B is a schematic block diagram showing the configuration of the above-mentioned space floating image display device 1000 (image display device). The spatial floating image display device 1000 includes a video display unit that displays an original image of an aerial image, a video control unit that converts an input video according to the resolution of the panel, and a video signal receiving unit that receives a video signal. Includes. The video signal receiver supports wired input signals such as HDMI (High-Definition Multimedia Interface) input and wireless input signals such as Wi-Fi (registered trademark, Wireless Fidelity) to receive video. It also functions independently as a display device, and can also display video information from tablets, smartphones, and the like. Furthermore, if a stick PC or the like is connected, it is possible to have capabilities such as calculation processing and video analysis processing.

FIG. 2 is a diagram showing an example of a main part configuration and a retroreflective part configuration of the spatial floating image display device according to the embodiment of the present invention. The configuration of the space floating image display device will be described more specifically with reference to FIG. As shown in FIG. 2A, a display device 1 is provided in an oblique direction of a transparent member 100 such as glass to diverge video light having a specific polarization in a narrow angle. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light having a specific polarization having a diffusion characteristic with a narrow angle.

The image light of the specific polarization from the display device 1 is a polarization separation member 101 having a film provided on the transparent member 100 for selectively reflecting the image light of the specific polarization (in the figure, the polarization separation member 101 is in the form of a sheet). It is formed on the surface of the transparent member 100 and is reflected by the transparent member 100), and is incident on the retroreflective member 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflective member. The image light is polarized and converted from a specific polarization to the other polarization by being passed through the λ / 4 plate 21 twice, when it is incident on the retroreflective member and when it is emitted. Here, since the polarization separating member 101 that selectively reflects the image light of the specific polarization has the property of transmitting the polarization of the other polarization that has been polarized, the image light of the specific polarization after the polarization conversion can be obtained. It passes through the polarization separating member 101. The image light transmitted through the polarization separating member 101 forms a spatial floating image 3 which is a real image on the outside of the transparent member 100.

The light forming the spatial floating image 3 is a set of light rays that converge from the retroreflective member 2 to the optical image of the spatial floating image 3, and these rays travel straight even after passing through the optical image of the spatial floating image 3. .. Therefore, the spatial floating image 3 is an image having high directivity, unlike the diffused image light formed on the screen by a general projector or the like. Therefore, in the configuration of FIG. 2, when the user visually recognizes from the direction of the arrow A, the space floating image 3 is visually recognized as a bright image. However, when another person visually recognizes from the direction of the arrow B, the space floating image 3 cannot be visually recognized as an image at all. This characteristic is very suitable for use in a system that displays a video that requires high security or a video that is highly confidential and that is desired to be kept secret from the person facing the user.

Depending on the performance of the retroreflective member 2, the polarization axes of the reflected image light may be uneven. In this case, a part of the video light whose polarization axes are not aligned is reflected by the above-mentioned polarization separation member 101 and returns to the display device 1. This light may be re-reflected on the image display surface of the liquid crystal display panel 11 constituting the display device 1 to generate a ghost image and deteriorate the image quality of the spatial floating image. Therefore, in this embodiment, the absorption type polarizing plate 12 is provided on the image display surface of the display device 1. The image light emitted from the display device 1 is transmitted through the absorption type polarizing plate 12, and the reflected light returned from the polarization separating member 101 is absorbed by the absorption type polarizing plate 12, so that the rereflection can be suppressed. This makes it possible to prevent deterioration of image quality due to the ghost image of the spatial floating image.

The above-mentioned polarization separating member 101 may be formed of, for example, a reflective polarizing plate or a metal multilayer film that reflects a specific polarization.

Next, as a typical retroreflective member 2 in FIG. 2B, the surface shape of the retroreflective member manufactured by Nippon Carbite Industries Co., Ltd. used in this study is shown. The light rays incident on the inside of the regularly arranged hexagonal prisms are reflected by the wall surface and the bottom surface of the hexagonal prisms and emitted as retroreflected light in the direction corresponding to the incident light, and are a real image based on the image displayed on the display device 1. Display a floating image in a certain space. The resolution of this spatial floating image largely depends on the outer shape (diameter) D and pitch P of the retroreflective portion 2 of the retroreflective member 2 shown in FIG. 2B, in addition to the resolution of the liquid crystal display panel 11. For example, when a 7-inch WUXGA (1920 x 1200 pixels) liquid crystal display panel is used, even if one pixel (1 triplet) is about 80 μm, for example, the diameter D of the retroreflective part is 240 μm and the pitch P is 300 μm. If so, one pixel of the spatial floating image is equivalent to 300 μm. Therefore, the effective resolution of the spatial floating image is reduced to about 1/3. Therefore, in order to make the resolution of the spatial floating image equal to the resolution of the display device 1, it is desired that the diameter and pitch of the retroreflective portion be close to one pixel of the liquid crystal display panel. On the other hand, in order to suppress the occurrence of moire due to the pixels of the retroreflective member and the liquid crystal display panel, it is advisable to design by excluding each pitch ratio from an integral multiple of one pixel. Further, the shape may be arranged so that neither side of the retroreflective portion overlaps with any one side of one pixel of the liquid crystal display panel.

On the other hand, in order to manufacture the retroreflective member at a low price, it is preferable to mold it by using the roll press method. Specifically, it is a method of aligning the recursive parts and shaping them on the film. The inverted shape of the shape to be shaped is formed on the roll surface, and the UV curable resin is applied on the base material for fixing to make the space between the rolls. By passing it through, a required shape is formed and cured by irradiating with ultraviolet rays to obtain a retroreflective member 2 having a desired shape.

<< How to install a floating image display device >>
Next, a method of installing the space floating image display device will be described. The installation method of the space floating image display device can be freely changed according to the usage pattern. FIG. 3A is a diagram showing an example of an installation method of a space floating image display device. The space floating image display device shown in FIG. 3A is installed horizontally so that the surface on the side where the space floating image 3 is formed faces upward. That is, in FIG. 3A, the space floating image display device is installed so that the transparent member 100 faces upward, and the space floating image 3 is formed above the space floating image display device.

FIG. 3B is a diagram showing another example of the installation method of the space floating image display device. The space floating image display device shown in FIG. 3B is installed vertically so that the surface on which the space floating image 3 is formed faces sideways (direction of the user 230). That is, in FIG. 3B, the space floating image display device is installed so that the transparent member 100 faces sideways, and the space floating image 3 is formed on the side of the space floating image display device (direction of the user 230). To.

<< Configuration of space floating image display device >>
Next, the configuration of the space floating image display device 1000 will be described. FIG. 3C is a block diagram showing an example of the internal configuration of the space floating image display device 1000. The space floating image display device 1000 includes a retroreflective unit 1101, an image display unit 1102, a light guide body 1104, a light source 1105, a power supply 1106, an operation input unit 1107, a non-volatile memory 1108, a memory 1109, a control unit 1110, and a video signal input. Unit 1131, audio signal input unit 1133, communication unit 1132, aerial operation detection sensor 1351 (first sensor described later), aerial operation detection unit 1350, audio output unit 1140, video control unit 1160, storage unit 1170, image pickup unit 1180 ( It is equipped with a second sensor), which will be described later. The control unit 1110 corresponds to the controller of the space floating image display device 1000.

Each component of the space floating image display device 1000 is housed in the housing 1190. The imaging unit 1180 and the aerial operation detection sensor 1351 shown in FIG. 3C may be provided on the outside of the housing 1190.

The retroreflective unit 1101 of FIG. 3C corresponds to the retroreflective member 2 of FIG. The retroreflective unit 1101 retroreflects the light modulated by the image display unit 1102. Of the light reflected from the retroreflective unit 1101, the light output to the outside of the spatial floating image information device 1000 forms the spatial floating image 3.

The image display unit 1102 of FIG. 3C corresponds to the liquid crystal display panel 11 of FIG. The light source 1105 of FIG. 3C corresponds to the light source device 13 of FIG. The image display unit 1102, the light guide body 1104, and the light source 1105 of FIG. 3C correspond to the display device 1 of FIG. The liquid crystal display panel 11 or the image display unit 1102 corresponds to an image source that generates image light.

The video display unit 1102 is a display unit that modulates the transmitted light to generate an image based on the image signal input by the control by the image control unit 1160 described later. As the image display unit 1102, for example, a transmissive liquid crystal panel is used. Further, as the image display unit 1102, for example, a reflective liquid crystal panel of a type that modulates the reflected light, a DMD (Digital Micromirror Device: registered trademark) panel, or the like may be used.

The light source 1105 generates light for the image display unit 1102, and is a solid-state light source such as an LED light source and a laser light source. The power supply 1106 converts an AC current input from the outside into a DC current and supplies electric power to the light source 1105. Further, the power supply 1106 supplies a required DC current to each part in the space floating image display device 1000.

The light guide body 1104 guides the light generated by the light source 1105 and irradiates the image display unit 1102. A combination of the light guide body 1104 and the light source 1105 can also be referred to as a backlight of the image display unit 1102. Various methods can be considered for the combination of the light guide body 1104 and the light source 1105. A specific configuration example of the combination of the light guide body 1104 and the light source 1105 will be described in detail later.

The aerial operation detection sensor 1351 is a sensor that detects the operation of the space floating image 3 by the finger of the user 230. The aerial operation detection sensor 1351 senses, for example, a range that overlaps with the entire display range of the space floating image 3. The aerial operation detection sensor 1351 may sense only a range that overlaps with at least a part of the display range of the space floating image 3.

Specific examples of the aerial operation detection sensor 1351 include a distance sensor using invisible light such as infrared rays, an invisible light laser, and ultrasonic waves. Further, the aerial operation detection sensor 1351 may be configured so that a plurality of sensors can be combined to detect the coordinates of the two-dimensional plane. Further, the aerial operation detection sensor 1351 may be composed of a ToF (Time of Light) type LiDAR (Light Detection and Ringing) or an image sensor.

The aerial operation detection sensor 1351 may be capable of sensing for detecting a touch operation or the like on an object displayed as a space floating image 3 by a user with a finger. Such sensing can be performed using existing techniques.

The aerial operation detection unit 1350 acquires a sensing signal from the aerial operation detection sensor 1351, and based on the sensing signal, the presence or absence of contact with the object of the spatial floating image 3 by the finger of the user 230, and the contact between the finger of the user 230 and the object. The position (contact position) is calculated. The aerial operation detection unit 1350 is composed of, for example, a circuit such as an FPGA (Field Programmable Gate Array). Further, some functions of the aerial operation detection unit 1350 may be realized by software, for example, by a spatial operation detection program executed by the control unit 1110.

The aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be configured to be built in the space floating image display device 1000, but may be provided outside separately from the space floating image display device 1000. When provided separately from the space floating image display device 1000, the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 provide information to the space floating image display device 1000 via a wired or wireless communication connection path or a video signal transmission path. It is configured to be able to transmit signals.

Further, the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be provided separately. This makes it possible to construct a system in which only the air operation detection function can be added as an option, with the space floating image display device 1000 having no air operation detection function as the main body. Further, only the aerial operation detection sensor 1351 may be a separate body, and the aerial operation detection unit 1350 may be built in the space floating image display device 1000. When it is desired to arrange the aerial operation detection sensor 1351 more freely with respect to the installation position of the space floating image display device 1000, there is an advantage in the configuration in which only the aerial operation detection sensor 1351 is separated.

The image pickup unit 1180 is a camera having an image sensor, and images the space near the space floating image 3 and / or the face, arms, fingers, and the like of the user 230. A plurality of image pickup units 1180 may be provided. By using a plurality of image pickup units 1180 or by using an image pickup unit with a depth sensor, it is possible to assist the aerial operation detection unit 1350 when the user 230 detects the touch operation of the spatial floating image 3.

For example, the aerial operation detection sensor 1351 is configured as an object intrusion sensor that detects the presence or absence of intrusion of an object into the intrusion detection plane for a plane (intrusion detection plane) including the display surface of the spatial floating image 3. In the case, the aerial operation detection sensor provides information such as how far an object that has not entered the intrusion detection plane (for example, the user's finger) is from the intrusion detection plane, or how close the object is to the intrusion detection plane. It may not be detected by 1351.

In such a case, the distance between the object and the intrusion detection plane can be calculated by using information such as the depth calculation information of the object based on the captured images of the plurality of imaging units 1180 and the depth information of the object by the depth sensor. .. Then, these information and various information such as the distance between the object and the intrusion detection plane are used for various display controls for the space floating image 3.

Further, instead of using the aerial operation detection sensor 1351, the aerial operation detection unit 1350 may detect the touch operation of the space floating image 3 by the user 230 based on the image captured by the image pickup unit 1180.

Further, the image pickup unit 1180 may take an image of the face of the user 230 who operates the space floating image 3, and the control unit 1110 may perform the identification process of the user 230. Further, in order to determine whether or not another person is standing around or behind the user 230 who operates the space floating image 3 and another person is looking into the operation of the user 230 with respect to the space floating image 3, the imaging unit 1180 may be used. The range including the user 230 who operates the space floating image 3 and the peripheral area of the user 230 may be imaged.

The operation input unit 1107 is, for example, an operation button or a light receiving unit of a remote controller, and inputs a signal for an operation different from the aerial operation (touch operation) by the user 230. Apart from the above-mentioned user 230 who touch-operates the space floating image 3, the operation input unit 1107 may be used, for example, for the administrator to operate the space floating image display device 1000.

The video signal input unit 1131 connects an external video output device and inputs video data. The voice signal input unit 1133 connects an external voice output device and inputs voice data. The voice output unit 1140 can output voice based on the voice data input to the voice signal input unit 1133. Further, the voice output unit 1140 may output a built-in operation sound or an error warning sound.

The non-volatile memory 1108 stores various data used in the space floating image display device 1000. The data stored in the non-volatile memory 1108 includes, for example, data for various operations displayed on the spatial floating image 3, display icons, object data for user operation, layout information, and the like. The memory 1109 stores video data to be displayed as the space floating video 3, control data of the device, and the like.

The control unit 1110 controls the operation of each connected unit. Further, the control unit 1110 may perform arithmetic processing based on the information acquired from each unit in the space floating image display device 1000 in cooperation with the program stored in the memory 1109. The communication unit 1132 communicates with an external device, an external server, or the like via a wired or wireless interface. Various data such as video data, image data, and audio data are transmitted and received by communication via the communication unit 1132.

The storage unit 1170 is a storage device that records various data such as video data, image data, audio data, and various information. For example, various information such as various data such as video data, image data, and audio data may be recorded in the storage unit 1170 in advance at the time of product shipment. Further, the storage unit 1170 may record various information such as various data such as video data, image data, audio data, etc. acquired from an external device, an external server, or the like via the communication unit 1132.

The video data, image data, and the like recorded in the storage unit 1170 are output as a spatial floating image 3 via the image display unit 1102 and the retroreflective unit 1101. Video data, image data, and the like, such as display icons and objects for the user to operate, which are displayed as the spatial floating video 3, are also recorded in the storage unit 1170.

Layout information such as display icons and objects displayed as the spatial floating image 3 and information on various metadata related to the objects are also recorded in the storage unit 1170. The audio data recorded in the storage unit 1170 is output as audio from, for example, the audio output unit 1140.

The video control unit 1160 performs various controls related to the video signal input to the video display unit 1102. The video control unit 1160 is a video such as which video signal is input to the video display unit 1102 among the video signal stored in the memory 1109 and the video signal (video data) input to the video signal input unit 1131. Controls switching, etc.

Further, the video control unit 1160 generates a superimposed video signal in which the video signal stored in the memory 1109 and the video signal input from the video signal input unit 1131 are superimposed, and the superimposed video signal is input to the video display unit 1102. Therefore, control may be performed to form the composite image as the spatial floating image 3.

Further, the video control unit 1160 may perform control to perform image processing on a video signal input from the video signal input unit 1131, a video signal stored in the memory 1109, and the like. Image processing includes, for example, scaling processing for enlarging, reducing, and transforming an image, bright adjustment processing for changing brightness, contrast adjustment processing for changing the contrast curve of an image, and decomposition of an image into light components for each component. There is a Retinex process that changes the weighting of.

Further, the video control unit 1160 may perform special effect video processing or the like for assisting the user 230 in the air operation (touch operation) with respect to the video signal input to the video display unit 1102. The special effect video processing is performed based on, for example, the detection result of the touch operation of the user 230 by the aerial operation detection unit 1350 and the image captured by the user 230 by the image pickup unit 1180.

As described above, the space floating image display device 1000 is equipped with various functions. However, the space floating image display device 1000 does not have to have all of these functions, and may have any configuration as long as it has a function of forming the space floating image 3.

<Example 2 of space floating image display device>
FIG. 4 is a diagram showing another example of the configuration of the main part of the space floating image display device according to the embodiment of the present invention. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light having a specific polarization having a narrow angle diffusion characteristic. For example, a small screen size of about 5 inches to a large screen size of more than 80 inches is provided. It is composed of a liquid crystal display panel, and a polarization separating member 101 such as a reflective polarizing plate is provided on the surface of the folded mirror 22, and the image light from the liquid crystal display panel 11 is reflected toward the retroreflective member 2. The image light of the specific polarization from the display device 1 is reflected by a film (adhesive to the sheet 101 in the figure) that selectively reflects the image light of the specific polarization provided on the transparent member 100, and is reflected by the retroreflective member 2. Incident to.

A λ / 4 plate 21 is provided on the light incident surface of the retroreflective member 2, and the polarization is converted by passing the video light twice, and the specific polarization is converted to the other polarization, so that the polarization separation member 101 is transmitted. The space floating image 3 which is a real image is displayed on the outside of the transparent member 100. An absorption-type polarizing plate is provided on the external light incident surface of the transparent member 100. In the polarization separating member 101 described above, the polarization axes become uneven due to retroreflection, so that a part of the image light is reflected and returns to the display device 1. This light is reflected again on the image display surface of the liquid crystal display panel 11 constituting the display device 1, generates a ghost image, and significantly deteriorates the image quality of the spatial floating image. Therefore, in this embodiment, an absorbent polarizing plate 12 is provided on the image display surface of the display device 1, the image light is transmitted, and the above-mentioned reflected light is absorbed to prevent deterioration of the image quality due to the ghost image of the spatial floating image. Further, in order to reduce the deterioration of image quality due to sunlight or illumination light outside the set, it is preferable to provide an absorption type polarizing plate on the surface of the transparent member 100. The polarization separating member 101 is formed of a reflective polarizing plate or a metal multilayer film that reflects a specific polarization.

Next, the sensor 44 having a TOF function is shown in FIG. 5 so as to sense the relationship between the distance and the position of the object and the sensor 44 with respect to the spatial floating image obtained by the above-mentioned spatial floating image display device. By arranging them in a plurality of layers, it is possible to detect not only the coordinates in the plane direction of the object but also the coordinates in the depth direction, the moving direction of the object, and the moving speed. In order to read the two-dimensional distance and position, a plurality of combinations of the ultraviolet light emitting part and the light receiving part are arranged linearly, the light from the light emitting point is irradiated to the object, and the reflected light is received by the light receiving part. The distance to the object becomes clear by the product of the difference between the time when light is emitted and the time when light is received and the speed of light. Further, the coordinates on the plane are a plurality of light emitting parts and light receiving parts, and can be read from the coordinates at the part where the difference between the light emitting time and the light receiving time is the smallest. From the above, it is possible to obtain three-dimensional coordinate information by combining the coordinates of an object in a plane (two-dimensional) and a plurality of the above-mentioned sensors.

Further, a method of obtaining a three-dimensional floating image of space as the above-mentioned floating image of space image will be described with reference to FIG. FIG. 6 is an explanatory diagram of the principle of three-dimensional image display used in the space floating image display device. A horizontal lenticular lens is arranged so as to match the pixels of the image display screen of the liquid crystal display panel 11 of the display device 1 shown in FIG. As a result, as shown in FIG. 6, in order to display the motion deviation from the three directions P1, P2, and P3 in the horizontal direction of the screen, the image from the three directions is regarded as one block for every three pixels, and one pixel. Image information from three directions is displayed for each, and the emission direction of light is controlled by the action of the corresponding lenticular lens (indicated by a vertical line in FIG. 6) to separate and emit light in three directions. As a result, a three-dimensional image with three parallax can be displayed.

<Reflective polarizing plate>
In the spatial floating image display device of this embodiment, the polarization splitting member 101 is used to improve the contrast performance that determines the image quality of the image as compared with a general half mirror. The characteristics of the reflective polarizing plate will be described as an example of the polarization separating member 101 of this embodiment. FIG. 7 is an explanatory diagram of a measurement system for evaluating the characteristics of the reflective polarizing plate. The transmission characteristics and reflection characteristics with respect to the incident angle of light rays from the direction perpendicular to the polarization axis of the reflective polarizing plate of FIG. 7 are shown as V-AOI in FIGS. 8 and 9, respectively. Similarly, the transmission characteristics and the reflection characteristics with respect to the incident angle of light rays from the horizontal direction with respect to the polarization axis of the reflective polarizing plate are shown as H-AOI in FIGS. 10 and 11, respectively.

As shown in FIGS. 8 and 9, the reflective polarizing plate having a grid structure has reduced characteristics for light from a direction perpendicular to the polarization axis. Therefore, the specifications along the polarization axis are desirable, and the light source of the present embodiment capable of emitting the image light emitted from the liquid crystal display panel at a narrow angle is an ideal light source. Similarly, the characteristics in the horizontal direction also deteriorate with respect to light from an angle. In consideration of the above characteristics, a configuration example of this embodiment will be described below in which a light source capable of emitting the image light emitted from the liquid crystal display panel at a narrower angle is used as the backlight of the liquid crystal display panel. This makes it possible to provide a high-contrast spatial floating image.

<Display device>
Next, the display device 1 of this embodiment will be described with reference to the drawings. The display device 1 of this embodiment includes a light source device 13 constituting the light source together with the image display element 11 (liquid crystal display panel), and in FIG. 12, the light source device 13 is shown as a developed perspective view together with the liquid crystal display panel. ing.

As shown by the arrow 30 in FIG. 12, the liquid crystal display panel (image display element 11) has a diffusion characteristic with a narrow angle due to the light from the light source device 13 which is a backlight device, that is, directional (straightness). The retroreflective member 2 reflects the video light that is modulated according to the input video signal to obtain an illumination light beam that is strong and has characteristics similar to laser light with the planes of polarization aligned in one direction, and is transparent. A spatial floating image, which is a real image, is formed by passing through the member 100. (See FIG. 1). Further, in FIG. 12, a liquid crystal display panel 11 constituting the display device 1, an optical direction conversion panel 54 for controlling the directivity characteristic of the luminous flux emitted from the light source device 13, and an angle diffuser plate (if necessary). (Not shown). That is, polarizing plates are provided on both sides of the liquid crystal display panel 11, and the video light having a specific polarization is emitted by modulating the light intensity with the video signal (see the arrow 30 in FIG. 12). .. As a result, the desired image is projected as light having a specific polarization having high directivity (straightness) toward the retroreflective member 2 via the optical direction conversion panel 54, reflected by the retroreflective member 2, and then stored in the store. The spatial floating image 3 is formed by transmitting through the eyes of the observer outside the (space). A protective cover 50 (see FIGS. 13 and 14) may be provided on the surface of the above-mentioned optical direction conversion panel 54.

In this embodiment, the display device 1 including the light source device 13 and the liquid crystal display panel 11 is configured in order to improve the utilization efficiency of the luminous flux (arrow) 30 emitted from the light source device 13 and significantly reduce the power consumption. In the transparent sheet (arrow 30) provided on the surface of the transparent member 100 (wind glass 105 or the like) after the light (arrow 30) from the light source device 13 is projected toward the retroreflective member 2 and reflected by the retroreflective member 2. By (not shown), the directivity can be controlled so as to form a floating image at a desired position. Specifically, this transparent sheet controls the imaging position of a floating image while imparting high directivity by an optical component such as a Fresnel lens or a linear Fresnel lens. According to this, the image light from the display device 1 efficiently reaches the observer outside the show window 105 (for example, a sidewalk) with high directivity (straightness) like a laser beam. As a result, it is possible to display a high-quality floating image with high resolution and to significantly reduce the power consumption of the display device 1 including the LED element 201 of the light source device 13.

<Example 1 of display device>
FIG. 13 shows an example of a specific configuration of the display device 1. In FIG. 13, a liquid crystal display panel 11 and an optical direction conversion panel 54 are arranged on the light source device 13 of FIG. The light source device 13 is formed of, for example, plastic on the case shown in FIG. 12, and is configured by accommodating the LED element 201 and the light guide body 203 inside the case, and is configured on the end surface of the light guide body 203. As shown in FIG. 12 and the like, has a shape in which the cross-sectional area gradually increases toward the light receiving portion in order to convert the divergent light from each LED element 201 into a substantially parallel light beam. However, a lens shape is provided that has the effect of gradually reducing the divergence angle by totally reflecting the light a plurality of times when propagating inside. A liquid crystal display panel 11 constituting the display device 1 is attached to the upper surface thereof. Further, an LED element 201 which is a semiconductor light source and an LED board 202 on which the control circuit thereof is mounted are attached to one side surface (the left end face in this example) of the case of the light source device 13, and the outside of the LED board 202. A heat sink, which is a member for cooling the heat generated by the LED element and the control circuit, may be attached to the side surface.

Further, the frame (not shown) of the liquid crystal display panel attached to the upper surface of the case of the light source device 13 is electrically connected to the liquid crystal display panel 11 attached to the frame and further to the liquid crystal display panel. An FPC (Flexible Printed Circuits: Flexible Wiring Board) (not shown) or the like is attached and configured. That is, the liquid crystal display panel 11 which is a liquid crystal display element, together with the LED element 201 which is a solid light source, modulates the intensity of transmitted light based on a control signal from a control circuit (not shown) constituting an electronic device. Generates a display image by. At this time, since the generated video light has a narrow diffusion angle and only a specific polarization component, it is possible to obtain a new display device that is close to a surface-emitting laser image source driven by a video signal. At present, it is technically and safety-wise impossible to obtain a laser luminous flux having the same size as the image obtained by the display device 1 described above by using the laser device. Therefore, in this embodiment, for example, light close to the above-mentioned surface emission laser image light is obtained from a light flux from a general light source provided with an LED element.

Subsequently, the configuration of the optical system housed in the case of the light source device 13 will be described in detail with reference to FIG. 13 and FIG. Since FIGS. 13 and 14 are cross-sectional views, only one plurality of LED elements 201 constituting the light source are shown, and these are converted into substantially collimated light by the shape of the light receiving end surface 203a of the light guide body 203. .. Therefore, the light receiving portion on the end surface of the light guide body and the LED element are attached while maintaining a predetermined positional relationship. Each of the light guides 203 is made of a translucent resin such as acrylic. The LED light receiving surface at the end of the light guide has, for example, a conical convex outer peripheral surface obtained by rotating a paraboloid cross section, and at the top thereof, a convex portion (that is, a convex lens surface) is formed at the center thereof. ) Is formed, and a convex lens surface protruding outward (or a concave lens surface recessed inward may be used) is provided in the central portion of the flat surface portion (not shown). The external shape of the light receiving portion of the light receiving body to which the LED element 201 is attached has a radial surface shape forming a conical outer peripheral surface, and the light emitted from the LED element in the peripheral direction may be totally reflected inside the light receiving portion. It is set within a possible angle range, or a reflective surface is formed.

On the other hand, the LED element 201 is arranged at a predetermined position on the surface of the LED substrate 202, which is the circuit board thereof. The LED substrate 202 is fixed to the LED collimator (light receiving end surface 203a) by arranging and fixing the LED elements 201 on the surface thereof so as to be located at the center of the recess described above.

According to this configuration, the shape of the light receiving end surface 203a of the light guide body 203 makes it possible to take out the light radiated from the LED element 201 as substantially parallel light, and it is possible to improve the utilization efficiency of the generated light. Become.

As described above, the light source device 13 is configured by attaching a light source unit in which a plurality of LED elements 201 as a light source are arranged on a light receiving end surface 203a which is a light receiving portion provided on the end surface of the light guide body 203, and is configured from the LED element. The divergent light is guided as substantially parallel light by the lens shape of the light receiving end surface 203a of the light source end surface, and the inside of the light source 203 is guided (direction parallel to the drawing) as shown by the arrow, and the light source direction changing means 204 is used. The light is emitted toward the liquid crystal display panel 11 arranged substantially parallel to the light guide (direction perpendicular to the front from the drawing). By optimizing the distribution (density) of the light flux direction changing means according to the shape of the inside or the surface of the light guide body, the uniformity of the light flux incident on the liquid crystal display panel 11 can be controlled. The above-mentioned luminous flux direction changing means 204 arranges the light flux propagating in the light guide body substantially parallel to the light guide body by providing, for example, a portion having a different refractive index inside the light guide body and the shape of the surface of the light guide body. It emits light toward the liquid crystal display panel 11 (in the direction perpendicular to the front from the drawing). At this time, it is practical if the relative brightness ratio when comparing the brightness of the center of the screen and the peripheral portion of the screen with the liquid crystal display panel 11 facing the center of the screen and the viewpoint at the same position as the diagonal dimension of the screen is 20% or more. There is no problem, and if it exceeds 30%, the characteristics will be even better.

Note that FIG. 13 is a cross-sectional layout diagram for explaining the configuration of the light source of the present embodiment for polarization conversion and its operation in the light source device 13 including the light guide body 203 and the LED element 201 described above. In FIG. 13, the light source device 13 includes, for example, a light guide body 203 provided with a light beam direction changing means 204 on a surface or inside formed of plastic or the like, an LED element 201 as a light source, a reflection sheet 205, and a retardation plate 206. It is composed of a lenticular lens or the like, and a liquid crystal display panel 11 having a light source light incident surface and a video light emitting surface having a polarizing plate is attached to the upper surface thereof.

Further, a film or sheet-shaped reflective polarizing plate 49 is provided on the light incident surface (lower surface of the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and among the natural light beams 210 emitted from the LED element 201. The polarization (for example, P wave) 212 on one side is selectively reflected, reflected by the reflection sheet 205 provided on one surface (lower part of the figure) of the light guide 203, and directed toward the liquid crystal display panel 52 again. To. Therefore, a retardation plate (λ / 4 plate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49, and the light is reflected by the reflective sheet 205 and passed twice. Converts the reflected light beam from P-polarized light to S-polarized light to improve the efficiency of using the light source light as the image light. The image luminous flux whose light intensity is modulated by the image signal on the liquid crystal display panel 11 (arrow 213 in FIG. 13) is incident on the retroreflective member 2 and passes through the wind glass 105 after reflection as shown in FIG. It is possible to obtain a floating image of space, which is a real image, inside or outside the store (space).

FIG. 14 is a cross-sectional layout diagram for explaining the configuration and operation of the light source of the present embodiment for polarization conversion in the light source device 13 including the light guide body 203 and the LED element 201, similarly to FIG. Similarly, the light source device 13 also has a light guide body 203 provided with a light beam direction changing means 204 on the surface or inside formed of, for example, plastic, an LED element 201 as a light source, a reflection sheet 205, a retardation plate 206, and a lenticular lens. As an image display element, a liquid crystal display panel 11 having a light source light incident surface and an image light emitting surface having a polarizing plate is attached to the upper surface thereof.

Further, a film or sheet-shaped reflective polarizing plate 49 is provided on the light incident surface (lower surface of the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and one side of the natural light beam 210 emitted from the LED light source 201 is biased. The wave (for example, S wave) 211 is selectively reflected, reflected by the reflection sheet 205 provided on one surface (lower part of the figure) of the light guide body 203, and directed to the liquid crystal display panel 11 again. A retardation plate (λ / 4 plate) is provided between the light guide sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49, and the light is reflected by the reflection sheet 205 and reflected twice. The light beam is converted from S-polarized light to P-polarized light, and the utilization efficiency of the light source light as video light is improved. The image luminous flux whose light intensity is modulated by the image signal on the liquid crystal display panel 11 (arrow 214 in FIG. 14) enters the retroreflective member 2 and, as shown in FIG. 1, passes through the wind glass 105 after reflection and is stored in the store. A spatial floating image, which is a real image, can be obtained inside or outside the (space).

In the light source device shown in FIGS. 13 and 14, in addition to the action of the polarizing plate provided on the light incident surface of the corresponding liquid crystal display panel 11, the reflective polarizing plate reflects the polarization component on one side, which is theoretically obtained. The contrast ratio obtained is the product of the inverse of the cross transmittance of the reflective polarizing plate and the inverse of the cross transmittance obtained by the two polarizing plates attached to the liquid crystal display panel. As a result, high contrast performance can be obtained. In fact, it was confirmed by experiments that the contrast performance of the displayed image was improved by 10 times or more. As a result, a high-quality image comparable to that of the self-luminous organic EL was obtained.

<Example 2 of display device>
FIG. 15 shows another example of a specific configuration of the display device 1. The light source device 13 of FIG. 15 is similar to the light source device of FIG. 17 and the like. The light source device 13 is configured by accommodating an LED, a collimator, a synthetic diffusion block, a light guide, and the like in a case such as plastic, and a liquid crystal display panel 11 is attached to the upper surface thereof. Further, LED elements 14a and 14b, which are semiconductor light sources, and an LED board 102 on which the control circuit thereof is mounted are attached to one side surface of the case of the light source device 13, and LED elements are mounted on the outer surface of the LED board 102. A heat sink 103, which is a member for cooling the heat generated in the control circuit, is attached (see also FIGS. 17 and 18).

Further, the liquid crystal display panel 11 attached to the frame and the FPC 403 electrically connected to the liquid crystal display panel 11 (see FIG. 7) are attached to the liquid crystal display panel frame attached to the upper surface of the case. It is composed of. That is, the liquid crystal display panel 11 which is a liquid crystal display element has the intensity of transmitted light based on the control signal from the control circuit (not shown here) constituting the electronic device together with the LED elements 14a and 14b which are solid light sources. Is generated by modulating the display image.

<Example 3 of display device>
Subsequently, another example of a specific configuration of the display device 1 will be described with reference to FIG. The light source device of the display device 1 converts the divergent light beam of natural light (mixed with P-polarized light and S-polarized light) from the LED into a substantially parallel light beam by the LED collimator 18, and the liquid crystal display panel 11 by the reflective light guide body 304. Reflects towards. The reflected light is incident on the wave plate and the reflective polarizing plate 49 arranged between the liquid crystal display panel 11 and the reflective light guide 304. A specific polarization (for example, S polarization) is reflected by the reflection type polarizing plate, the phase is converted by the wavelength plate, and the polarization returns to the reflection surface, passes through the retardation plate again, and passes through the reflection type polarizing plate (for example, P bias). Wave) is converted.

As a result, the natural light from the LED is aligned with a specific polarization (for example, P polarization), is incident on the liquid crystal display panel 11, is brightly modulated according to the video signal, and displays the video on the panel surface. Similar to the above example, a plurality of LEDs constituting the light source are shown (however, only one is shown in FIG. 16 because of the vertical cross section), and these are mounted in a predetermined position with respect to the LED collimator 18. .. The LED collimator 18 is made of a translucent resin such as acrylic or glass, respectively. The LED collimator 18 has a conical convex outer peripheral surface obtained by rotating a parabolic cross section, and at the top thereof, has a concave portion having a convex portion (that is, a convex lens surface) formed in the central portion thereof. Further, the central portion of the flat surface portion has a convex lens surface protruding outward (or a concave lens surface recessed inward). The paraboloid surface forming the conical outer peripheral surface of the LED collimator 18 is set within an angle range at which light emitted from the LED in the peripheral direction can be totally reflected inside the paraboloid surface, or is a reflective surface. Is formed.

The above configuration is the same as that of the light source device of the display device shown in FIGS. 17, 18 and the like. Further, the light converted into substantially parallel light by the LED collimator 15 shown in FIG. 16 is reflected by the reflective light guide 304, and the light of a specific polarization is transmitted by the action of the reflective polarizing plate 49 and reflected. The polarized light of the above passes through the light guide 304 again and is reflected by the reflector 271 provided on the other surface of the light guide that does not come into contact with the liquid crystal display panel 11. At this time, the polarization is converted by passing twice through the retardation plate (λ / 4 plate) 270 arranged between the reflection plate 271 and the liquid crystal display panel 11, and the light guide body 304 is transmitted again to the opposite surface. It is transmitted through the provided reflective polarizing plate 49, and is incident on the liquid crystal display panel 11 with the polarization directions aligned. As a result, all the light from the light source can be used, so that the light utilization efficiency is doubled.

The light emitted from the liquid crystal display panel has the same diffusion characteristics in the conventional TV set in both the horizontal direction of the screen (displayed on the X-axis in FIG. 22 (a)) and the vertical direction of the screen (displayed on the Y-axis in FIG. 22 (b)). ing. On the other hand, as for the diffusion characteristic of the luminous flux emitted from the liquid crystal display panel of this embodiment, for example, as shown in Example 1 of FIG. 22, the viewing angle at which the brightness is 50% of the front view (angle 0 degree) is 13 degrees. By setting it to, it becomes 1/5 of the conventional 62 degrees. Similarly, the viewing angle in the vertical direction is uneven up and down, and the reflection angle of the reflective light guide and the area of the reflecting surface are optimized so that the upper viewing angle is suppressed to about 1/3 of the lower viewing angle. do. As a result, the amount of video light in the monitoring direction is significantly improved as compared with the conventional liquid crystal TV, and the brightness is 50 times or more.

Further, if the viewing angle characteristic shown in Example 2 of FIG. 22 is used, the brightness becomes 50% of the front view (angle 0 degree). By setting the viewing angle to 5 degrees, it becomes 1/12 of the conventional 62 degrees. .. Similarly, the viewing angle in the vertical direction is set to be even in the vertical direction, and the reflection angle of the reflective light guide and the area of the reflecting surface are optimized so that the viewing angle is suppressed to about 1/12 of the conventional one. As a result, the amount of video light in the monitoring direction is significantly improved as compared with the conventional liquid crystal TV, and the brightness is 100 times or more. As described above, by setting the viewing angle as the narrowing angle, the amount of light flux toward the monitoring direction can be concentrated, so that the efficiency of light utilization is greatly improved. As a result, even if a conventional liquid crystal display panel for TV is used, it is possible to realize a significant improvement in brightness with the same power consumption by controlling the light diffusion characteristics of the light source device, and the space floats toward bright outdoors. It can be a display device compatible with a video display device.

When using a large LCD panel, the light around the screen is directed inward so that the center of the screen faces the observer when the observer faces it, thereby improving the overall brightness of the screen. do. FIG. 20 shows the convergence angles of the long side and the short side of the panel when the distance L from the observer's panel and the panel size (screen ratio 16:10) are used as parameters. When monitoring the screen vertically, the convergence angle may be set according to the short side. For example, when the 22 "(inch) panel is used vertically and the monitoring distance is 0.8 m, the convergence angle is 10 degrees. Then, the image light from the screen 4 corner can be effectively directed to the observer.

Similarly, when monitoring with a vertical use of a 15 "(inch) panel, if the monitoring distance is 0.8 m and the convergence angle is 7 degrees, the image light from the screen 4 corners is effectively directed to the observer. As described above, depending on the size of the liquid crystal display panel and whether it is used vertically or horizontally, the screen brightness can be directed to the observer at the optimum position for monitoring the center of the screen. Can improve the overallness of.

As a basic configuration, as shown in FIG. 16, a light flux having a narrow-angle directional characteristic is incident on the liquid crystal display panel 11 by a light source device, and the luminance is modulated according to the video signal, so that the light flux is displayed on the screen of the liquid crystal display panel 11. The spatially floating image obtained by reflecting the generated image information by the retroreflective member is displayed outdoors or indoors via the transparent member 100.

<Example 1 of light source device>
Subsequently, the configuration of the optical system such as the light source device housed in the case will be described in detail together with FIG. 17 with reference to FIGS. 18 (a) and 18 (b). 17 and 18 show LEDs 14a, 14b constituting the light source, which are attached at predetermined positions with respect to the LED collimator 15. Each of the LED collimators 15 is made of a translucent resin such as acrylic. As shown in FIG. 18B, the LED collimator 15 has a conical convex outer peripheral surface 156 obtained by rotating a parabolic cross section, and at the top thereof, a convex portion (convex portion) at the center thereof. That is, it has a concave portion 153 forming a convex lens surface) 157. Further, the central portion of the flat surface portion has a convex lens surface protruding outward (or a concave lens surface recessed inward) 154. The parabolic surface 156 forming the conical outer peripheral surface of the LED collimator 15 is set within an angle range at which the light emitted from the LEDs 14a and 14b in the peripheral direction can be totally reflected inside the paraboloid surface 156. , A reflective surface is formed.

Further, the LEDs 14a and 14b are respectively arranged at predetermined positions on the surface of the LED substrate 102, which is the circuit board thereof. The LED substrate 102 is arranged and fixed to the LED collimator 15 so that the LEDs 14a or 14b on the surface thereof are respectively located at the center of the recess 153.

According to such a configuration, among the light radiated from the LED 14a or 14b by the LED collimator 15 described above, the light radiated upward (to the right in the figure) from the central portion thereof is the light of the LED collimator 15. The two convex lens surfaces 157 and 154 forming the outer shape are focused to form parallel light. Further, the light emitted from the other portion toward the peripheral direction is reflected by the paraboloid forming the conical outer peripheral surface of the LED collimator 15, and is similarly condensed into parallel light. In other words, according to the LED collimator 15 having a convex lens formed in the center thereof and a paraboloid formed in the peripheral portion thereof, almost all the light generated by the LEDs 14a or 14b can be taken out as parallel light. Therefore, it is possible to improve the utilization efficiency of the generated light.

A polarization conversion element 21 is provided on the light emitting side of the LED collimator 15. As is clear from FIG. 18, the polarization conversion element 21 has a columnar (hereinafter, parallelogram) translucent member having a parallelogram cross section and a columnar (hereinafter, triangular column) having a triangular cross section. ) Is combined with the translucent member, and a plurality of pieces are arranged in an array in parallel with the plane orthogonal to the optical axis of the parallel light from the LED collimator 15. Further, at the interface between the adjacent translucent members arranged in an array, a polarizing beam splitter (hereinafter abbreviated as "PBS film") 211 and a reflective film 212 are alternately provided. Further, a λ / 2 phase plate 213 is provided on the exit surface where the light incident on the polarization conversion element 21 and transmitted through the PBS film 211 is emitted.

Further, a rectangular synthetic diffusion block 16 shown in FIG. 18A is provided on the emission surface of the polarization conversion element 21. That is, the light emitted from the LED 14a or 14b becomes parallel light by the action of the LED collimator 15 and enters the synthetic diffusion block 16, diffused by the texture 161 on the exit side, and then reaches the light guide body 17.

The light guide body 17 is a member formed of a translucent resin such as acrylic into a rod shape having a substantially triangular cross section (see FIG. 18 (b)), and as is clear from FIG. 17, it is synthesized. A light guide body light incident portion (plane) 171 facing the emission surface of the diffusion block 16 via the first diffuser plate 18a, a light guide body light reflection portion (plane) 172 forming a slope, and a second diffusion. A light guide body light emitting portion (plane) 173 facing the liquid crystal display panel 11 which is a liquid crystal display element is provided via the plate 18b.

As shown in FIG. 17, which is a partially enlarged view of the light guide body light reflecting portion (plane) 172 of the light guide body 17, a large number of reflecting surfaces 172a and connecting surfaces 172b are alternately serrated. It is formed. Then, the reflecting surface 172a (a line segment rising to the right in the figure) forms αn (n: a natural number, for example, 1 to 130 in this example) with respect to the horizontal plane indicated by the alternate long and short dash line in the figure. As an example, here, αn is set to 43 degrees or less (however, 0 degrees or more).

The light incident part (plane) 171 of the light guide body is formed in a curved convex shape inclined toward the light source side. According to this, the parallel light from the emission surface of the synthetic diffusion block 16 is diffused and incident through the first diffusion plate 18a, and as is clear from the figure, the light incident part (plane) of the light guide body. It reaches the light guide body light reflecting portion (plane) 172 while being slightly bent (deflected) upward by 171 and is reflected here to reach the liquid crystal display panel 11 provided on the upper exit surface in the figure.

According to the display device 1 described in detail above, it is possible to further improve the light utilization efficiency and its uniform lighting characteristics, and at the same time, to manufacture the modular S polarized wave light source device in a small size and at low cost. It becomes. In the above description, the polarization conversion element 21 is attached after the LED collimator 15, but the present invention is not limited thereto, and the same can be applied by providing the polarization conversion element 21 in the optical path leading to the liquid crystal display panel 11. Action / effect can be obtained.

A large number of reflecting surfaces 172a and connecting surfaces 172b are alternately formed in a sawtooth shape on the light guide body light reflecting portion (surface) 172, and the illumination light beam is totally reflected on each reflecting surface 172a. Further upward, the light emitting portion (plane) 173 of the light guide body is provided with a narrowing angle diffuser plate to be incident on the optical direction conversion panel 54 that controls the directivity characteristics as a substantially parallel diffused light beam, and is incident from an oblique direction. It is incident on the liquid crystal display panel 11. In this embodiment, the light direction conversion panel 54 is provided between the light guide body emitting portion (plane) 173 and the liquid crystal display panel 11, but the same effect can be obtained by providing the light direction changing panel 54 on the emitting surface of the liquid crystal display panel 11.

<Example 2 of light source device>
FIG. 19 shows another example of the configuration of the optical system such as the light source device 13. Similar to the example shown in FIG. 18, a plurality of (two in this example) LEDs 14a and 14b constituting the light source are shown, and these are attached to the LED collimator 15 at a predetermined position. Each of the LED collimators 15 is made of a translucent resin such as acrylic. And, like the example shown in FIG. 18, this LED collimator 15 has a conical convex outer peripheral surface 156 obtained by rotating a parabolic cross section, and at the top thereof, a convex portion (that is, a convex portion) is formed in the center thereof. , Convex lens surface) It has a recess 153 forming 157. Further, the central portion of the flat surface portion has a convex lens surface protruding outward (or a concave lens surface recessed inward) 154. The paraboloid surface 156 forming the conical outer peripheral surface of the LED collimator 15 is set or reflected within an angle range within which the light emitted from the LED 14a in the peripheral direction can be totally reflected. A surface is formed.

Further, the LEDs 14a and 14b are respectively arranged at predetermined positions on the surface of the LED substrate 102, which is the circuit board thereof. The LED substrate 102 is arranged and fixed to the LED collimator 15 so that the LEDs 14a or 14b on the surface thereof are respectively located at the center of the recess 153.

According to such a configuration, among the light radiated from the LED 14a or 14b by the LED collimator 15 described above, the light radiated upward (to the right in the figure) from the central portion thereof is the light of the LED collimator 15. The two convex lens surfaces 157 and 154 forming the outer shape are focused to form parallel light. Further, the light emitted from the other portion toward the peripheral direction is reflected by the paraboloid forming the conical outer peripheral surface of the LED collimator 15, and is similarly condensed into parallel light. In other words, according to the LED collimator 15 having a convex lens formed in the center thereof and a paraboloid formed in the peripheral portion thereof, almost all the light generated by the LEDs 14a or 14b can be taken out as parallel light. Therefore, it is possible to improve the utilization efficiency of the generated light.

A light guide body 170 is provided on the light emitting side of the LED collimator 15 via the first diffuser plate 18a. The light guide body 170 is a member formed of a translucent resin such as acrylic into a rod shape having a substantially triangular cross section (see FIG. 19A), and as is clear from FIG. 19A. In addition, a light guide body light incident portion (plane) 171 facing the emission surface of the diffusion block 16 via the first diffuser plate 18a, a light guide body light reflection portion (plane) 172 forming a slope, and a reflection type. It includes a light guide body light emitting unit (plane) 173 facing the liquid crystal display panel 11 which is a liquid crystal display element via the polarizing plate 200.

If, for example, a reflective polarizing plate 200 having a property of reflecting P-polarized light (transmitting S-polarized light) is selected, the reflective polarizing plate 200 reflects P-polarized light among the natural light emitted from the LED as a light source, and FIG. 19 (b) shows. It passes through the λ / 4 plate 202 provided in the light guide body light reflecting unit 172 shown in the above, is reflected by the reflecting surface 201, and is converted into S-polarized light by passing through the λ / 4 plate 202 again to be converted into S-polarized light. All the light rays incident on the are unified to S polarization.

Similarly, if a reflective polarizing plate 200 having a characteristic of reflecting S-polarized light (transmitting P-polarized light) is selected, the S-polarized light among the natural light emitted from the LED as the light source is reflected, and FIG. 19 (b) shows. It passes through the λ / 4 plate 202 provided in the light guide body light reflecting unit 172 shown in the above, is reflected by the reflecting surface 201, and is converted into P-polarized light by passing through the λ / 4 plate 202 again to be converted into P-polarized light, and is converted into P-polarized light. All the light rays incident on the are unified to P polarization. Polarization conversion can be realized even with the above-mentioned configuration.

<Example 3 of light source device>
Another example of the configuration of an optical system such as a light source device will be described with reference to FIG. In the third example, as shown in FIG. 16, the divergent luminous flux of natural light (a mixture of P-polarized light and S-polarized light) from the LED 102 is converted into a substantially parallel luminous flux by the collimator lens 18, and the liquid crystal display panel by the reflective light guide 304. It reflects toward 11. The reflected light is incident on the reflective polarizing plate 206 arranged between the liquid crystal display panel 11 and the reflective light guide 304. A reflective plate 271 is arranged so that a specific polarization (for example, S polarization) is reflected by the reflective polarizing plate 206, passes through a surface connecting the reflective surfaces of the light guide 304, and faces the opposite surface of the light guide 304. It is reflected by and transmitted through the phase plate (λ / 4 wavelength plate) 270 twice to be polarized, transmitted through the light guide and the reflective polarizing plate, incident on the liquid crystal display panel 11, and modulated by video light. At this time, by matching the specific polarization and the polarization plane that has been polarized, the efficiency of light utilization is doubled, and the degree of polarization (extinguishing ratio) of the reflective polarizing plate is also added to the extinguishing ratio of the entire system. By using the light source device of this embodiment, the contrast ratio of the information display system is significantly improved.

As a result, the natural light from the LED is aligned with a specific polarization (for example, P polarization). Similar to the above example, a plurality of LEDs constituting the light source are provided (however, only one is shown in FIG. 16 because of the vertical cross section), and these are attached to the LED collimator 18 at a predetermined position. .. The LED collimator 18 is made of a translucent resin such as acrylic or glass, respectively. The LED collimator 18 has a conical convex outer peripheral surface obtained by rotating a parabolic cross section, and at the top thereof, has a concave portion having a convex portion (that is, a convex lens surface) formed in the central portion thereof. Further, the central portion of the flat surface portion has a convex lens surface protruding outward (or a concave lens surface recessed inward). The paraboloid surface forming the conical outer peripheral surface of the LED collimator 18 is set within an angle range at which the light emitted from the LED 18 in the peripheral direction can be totally reflected inside the paraboloid surface, or is a reflective surface. Is formed.

Further, the LEDs are arranged at predetermined positions on the surface of the LED substrate 102, which is the circuit board thereof. The LED substrate 102 is fixed to the LED collimator 18 by arranging and fixing LEDs on the surface thereof so as to be located at the center of the recess.

According to such a configuration, among the light radiated from the LED by the LED collimator 18, the light radiated from the central portion thereof is collected and parallel by the two convex lens surfaces forming the outer shape of the LED collimator 18. It becomes light. Further, the light emitted from the other portion toward the peripheral direction is reflected by the paraboloid forming the conical outer peripheral surface of the LED collimator 18, and is similarly condensed into parallel light. In other words, according to the LED collimator 18 in which a convex lens is formed in the central portion thereof and a paraboloid is formed in the peripheral portion thereof, almost all the light generated by the LED can be taken out as parallel light. It is possible to improve the utilization efficiency of the generated light.

<Example 4 of light source device>
Further, another example of the configuration of an optical system such as a light source device will be described with reference to FIG. 25. On the light emitting side of the LED collimator 18, two optical sheets 207 that convert the diffusion characteristics in the vertical direction and the horizontal direction (not shown in the front-rear direction of the figure) in the drawing are used, and two pieces of light from the LED collimator 18 are used. It is incident between the optical sheets 207 (diffuse sheets) of the above. When the optical sheet 207 is composed of one sheet, the vertical and horizontal diffusion characteristics are controlled by the fine shapes of the front surface and the back surface. Further, a plurality of diffusion sheets may be used to share the action. Depending on the front surface shape and back surface shape of the optical sheet 207, the diffusion angle of the light from the LED collimator 18 in the vertical direction of the screen is matched with the width of the vertical surface of the reflection surface of the diffusion sheet, and the horizontal direction is the light flux emitted from the liquid crystal display panel 11. It is preferable to optimally design the number of LEDs and the divergence angle from the LED substrate (optical element) 102 as design parameters so that the surface density becomes uniform. That is, the diffusion characteristics are controlled by the surface shapes of a plurality of diffusion sheets instead of the light guide. In this embodiment, the polarization conversion is performed in the same manner as in Example 3 of the light source device described above. On the other hand, a polarization conversion element 21 may be provided between the LED collimator 18 and the diffusion film 207 to perform polarization conversion, and then the light source light may be incident on the diffusion sheet 207.

If the above-mentioned reflective polarizing plate 206 has a characteristic of reflecting S-polarization (P-polarization is transmitted), S-polarization is reflected among the natural light emitted from the LED as a light source, and is shown in FIG. 25. It passes through the retardation plate 270, is reflected by the reflecting surface 271, and is converted into P-polarized light by passing through the retardation plate 270 again to be incident on the liquid crystal display panel 11. It is necessary to select the optimum value for the thickness of the retardation plate according to the angle of incidence of the light beam on the retardation plate, and the optimum value exists in the range of λ / 16 to λ / 4.

<Lenticular lens>
In order to control the diffusion distribution of the image light from the liquid crystal display panel 11, a lenticular lens is provided between the light source device 13 and the liquid crystal display panel 11 or on the surface of the liquid crystal display panel 11 to optimize the lens shape. This makes it possible to control the emission characteristics in one direction. Further, by arranging the microlens arrays in a matrix, it is possible to control the emission characteristics of the image luminous flux from the display device 1 in the X-axis and Y-axis directions, and as a result, the spatial floating image display device having desired diffusion characteristics. Can be obtained.

The operation of the lenticular lens will be described. By optimizing the lens shape of the lenticular lens, it is possible to efficiently obtain a spatial floating image by transmitting or reflecting the transparent member 100 emitted from the display device 1 described above. That is, for the image light from the display device 1, two lenticular lenses are combined, or a microlens array is arranged in a matrix to provide a sheet for controlling the diffusion characteristics, and the image is imaged in the X-axis and Y-axis directions. The brightness of light (relative brightness) can be controlled according to the reflection angle (0 degrees in the vertical direction). In this embodiment, as shown in FIG. 22B, the luminance characteristic in the vertical direction is steepered by such a lenticular lens as compared with the conventional case, and the directional characteristic in the vertical direction (positive / negative direction of the Y axis) is balanced. By increasing the brightness (relative brightness) of the light due to reflection and diffusion by changing, the diffusion angle is narrow (high straightness) and only a specific polarization component, like the image light from a surface-emitting laser image source. It is possible to suppress the ghost image generated by the retroreflective member when the conventional technique is used, and to efficiently control the spatial floating image due to the retroreflective to the observer's eye.

Further, by the above-mentioned light source device, the X-axis direction and the Y-axis direction with respect to the emission light diffusion characteristic characteristics (denoted as conventional in the figure) from the general liquid crystal display panel shown in FIGS. 22 (a) and 22 (b). By setting both of them to have a directional characteristic with a significantly narrow angle, it is possible to realize a display device that emits light of a specific polarization that emits an image light source that is almost parallel to a specific direction.

FIG. 21 shows an example of the characteristics of the lenticular lens used in this embodiment. In this example, in particular, the characteristic in the X direction (vertical direction) is shown. In the characteristic O, the peak in the light emission direction is at an angle of about 30 degrees upward from the vertical direction (0 degrees) and is vertically symmetrical. It shows the brightness characteristics. Further, the characteristics A and B in FIG. 21 further show an example of a characteristic in which the image light above the peak luminance is condensed at around 30 degrees to increase the luminance (relative luminance). Therefore, in these characteristics A and B, the brightness (relative brightness) of light is sharply reduced as compared with the characteristic O at an angle exceeding 30 degrees.

That is, according to the above-mentioned optical system including the lenticular lens, when the image light beam from the display device 1 is incident on the retroreflective member 2, the emission angle and viewing angle of the image light aligned to the narrow angle by the light source device 13 Can be controlled, and the degree of freedom in installing the retroreflective sheet (retroreflective member 2) can be greatly improved. As a result, the degree of freedom of the relationship between the image formation positions of the spatial floating image formed at a desired position by reflecting or transmitting the transparent member 100 can be greatly improved. As a result, it is possible to efficiently reach the eyes of the observer outdoors or indoors as light having a narrow diffusion angle (high straightness) and only a specific polarization component. According to this, even if the intensity (luminance) of the video light from the display device is reduced, the observer can accurately recognize the video light and obtain information. In other words, by reducing the output of the display device, it is possible to realize a space floating image display device with low power consumption.

(Embodiment 1)
Next, the space floating image display device of the first embodiment will be described with reference to FIGS. 26 and 26. The basic configuration of the first embodiment is the same as that of FIGS. 1 to 3C described above.

[Spatial floating image display device]
FIG. 26 shows the configuration of the space floating image display device 1000 of the first embodiment. The space floating image display device 1000 is arranged in an applicable space such as a bank or a hospital. FIG. 26 shows the case of horizontal placement as in FIG. 3A. The space floating image display device 1000 is arranged in a place where it is used, for example, as a part of an ATM device of a bank, it is housed in a housing of the ATM device, or is connected to the ATM device as an external unit. .. The transparent member 100 of the space floating image display device 1000 is arranged so as to be exposed to the outside. The space floating image display device 1000 is a device that forms and displays a space floating image 3 having high brightness and high directivity in the space by the above-mentioned mechanism. The user 230 facing the space floating image 3 at a standard position (the line-of-sight position of a person having a standard height who can see the image without stretching or bending) can see this space floating image from the eye UE at that position. 3 can be preferably visually recognized.

The user 230 is a person who uses and operates the space floating image display device 1000, and is schematically shown. The face, eyes, and fingers of the user 230 are represented by UF, UE, and UH, respectively. The space floating image display device 1000 detects the face UF of the user 230 by using the camera 32 corresponding to the above-mentioned image pickup unit 1180 (FIG. 3C), and displays the space floating image 3 according to the image pickup position of the face UF. Control to adjust. The space floating image display device 1000 similarly detects a touch operation by the finger UH on the space floating image 3 by using the sensor 31 corresponding to the above-mentioned aerial operation detection unit 1351 (FIG. 3C).

Here, for the sake of explanation, the coordinate system and direction of space are illustrated by (X, Y, Z). The X direction and the Y direction are two directions constituting the horizontal plane, and the Z direction is a vertical direction. The X direction is the left-right direction of the space floating image display device 1000 and the user 230, the Y direction is the front-back direction of the space floating image display device 1000 and the user 230, and the Z direction is the space floating image display device 1000 and the user. The vertical direction and the height direction of 230. Further, the coordinate system and the direction in the space floating image 3 are illustrated by (x, y, z). In the screen of the spatial floating image 3, the x direction is the horizontal direction, the y direction is the vertical direction, and the z direction is the depth / depth direction perpendicular to them. When operating on the surface of the space floating image 3, the finger UH (index finger in the illustrated example) moves in the z direction or the like.

The space floating image display device 1000 includes a housing 60, a control board 61, and a drive mechanism 62. The housing 60 (corresponding to the housing 1190 in FIG. 3C described above) houses the display device 1, the retroreflective member 2, the transparent member 100, and the polarization separating member 101 as the above-mentioned components. And they are fixed in a predetermined positional relationship. Further, in the housing 60, a sensor 31 which is a first sensor and a camera 32 which is a second sensor are installed. The display device 1 includes the above-mentioned light source device 13, a liquid crystal display panel 11, and an absorption-type polarizing plate 12. The emission surface of the retroreflective member 2 is provided with a λ / 4 plate 21. A polarization separating member 101 is provided on the back surface of the transparent member 100.

Each component in the housing 60 is connected to the control board 61 by a signal line or the like. Wireless communication may be applied between the components. The control board 61 corresponds to a portion on which each unit such as the control unit 1110 in FIG. 3C is mounted, and controls each component such as the display device 1. In this example, the control board 61 is provided inside the housing 60, but the present invention is not limited to this and can be mounted on the outside of the housing 60 or the like.

A drive mechanism 62 is arranged around the lower surface and the side surface except for the upper surface of the housing 60. The drive mechanism 62 includes, for example, a motor, a drive circuit board, and the like. The drive mechanism 62 drives the housing 60 so as to move, as will be described later, based on the control from the control board 61. By also having a housing in the drive mechanism 62, the housing 60 and the drive mechanism 62 may have a double housing.

A sensor 31, which is a first sensor, is arranged at a predetermined position on the upper surface of the housing 60 in a predetermined direction. The position and orientation of the sensor 31 are, for example, a position and orientation such that the detection axis (indicated by a broken line) is parallel to the plane of the space floating image 3 as shown in the figure. In this example, the position of the sensor 31 is a position on the front side close to the user 230 in the Y direction, and the direction of the sensor 31 is an obliquely upward direction toward the back side in the Y direction. In FIG. 26, the optical axis of the spatial floating image 3 (the axis corresponding to the optical axis of the reflected light from the retroreflective member 2), that is, the surface of the spatial floating image 3 and the position of the eye UE of the user 230 are perpendicular to each other. The line connecting to is indicated by a alternate long and short dash line. Further, the angle with respect to the horizontal plane of the surface of the space floating image 3 and the detection axis of the first sensor is shown by the angle a1.

The camera 32, which is the second sensor, is arranged at a predetermined position on the upper surface of the housing 60 in a predetermined direction. As shown in the figure, the position and orientation of the camera 32 assume that the user 230 who visually recognizes the spatial floating image 3 has a standard height and the position of the face UF is standard, and the optical axis is the standard. The position and orientation are such that the imaging range can cover the area of the face UF at a specific position. In FIG. 26, the optical axis of the camera 32 (second sensor) is indicated by a two-dot chain line, and the angle of view range (imaging range) is indicated by a broken line. The angle of the optical axis of the second sensor with respect to the horizontal plane is indicated by the angle a2. The angle of view of the second sensor in the Z direction is shown by the angle α. Further, the coordinate system of the camera 32 is shown by (CX, CY, CZ). CZ is the optical axis direction of the camera 32, and (CX, CY) is the horizontal direction and the vertical direction in the captured image of the camera 32.

The space floating image display device 1000 (control board 61) determines the position of the face UF (or head) of the user 230 in the space (X, Y, Z) from the image captured by the camera 32. This determination is possible, for example, using known image analysis and face recognition techniques, based on the detection of the face UF or head region and the positions of the eye UF, nose, mouth, etc. within that region. The space floating image display device 1000 may calculate, for example, the center point between the left and right eye UFs in the area of the face UF as the face position. In FIG. 26, the standard face position is shown as a point of position coordinates (X0, Y0, Z0) corresponding to the eye UF. The position of the face UF in the space and the position of the face UF in the image of the camera 32 have a predetermined correspondence relationship and can be converted.

As will be described later, the housing 60 can be rotated and moved by the drive mechanism 62, and the rotation shaft J1 shows an example of arrangement of the rotation shaft during the rotation movement. In the case of the rotation axis J1, the housing 60 is rotated in the YZ plane, so that the transparent member 100 on the upper surface is arranged so as to be tilted toward the front or the back in the Y direction with respect to the user 230. change. That is, with the rotational movement of the housing 60 and the transparent member 100, the display state of the space floating image 3 also changes so as to be tilted toward the front or the back in the Y direction with respect to the user 230.

In the first embodiment, the space floating image display device 1000 (control board 61) adjusts the state such as the position of the space floating image 3 by moving the entire housing 60 by using the drive mechanism 62. A plurality of components are fixed in the housing 60 in a predetermined positional relationship. Therefore, when the housing 60 is moved by rotation or the like by the drive mechanism 62, the plurality of components also move while maintaining a predetermined positional relationship. In the first embodiment, the case of rotational movement is shown as an example of the movement of the housing 60.

As the first sensor, the sensor 31 (aerial operation detection sensor 1351), various sensors such as a TOF type distance measuring sensor, an infrared sensor, and a stereo camera can be applied. The sensor 31 is positioned and oriented as shown in the figure so as to detect an operation by the finger UH on the space floating image 3, for example, a touch operation in which a finger is inserted in the z direction with respect to the surface of the space floating image 3 and a button or the like is pressed. Have been placed. The state such as the position of the finger UH in the z direction (depth direction) can be detected, for example, in the same manner as in FIG. 5 described above.

A camera such as a general CCD can be applied to the camera 32 (imaging unit 1180) which is the second sensor. The camera 32 is arranged at the position and orientation shown in the figure, the angle of view α, and the like so that the eye UE is located on the extension of the optical axis of the camera with respect to the face UF facing the standard position described above. Have been placed. The second sensor is not limited to the camera described above, and may be any sensor that can capture and detect a region including the user's face UF as a two-dimensional image.

[Spatial floating image]
FIG. 27 shows a display example of the spatial floating image 3 seen from the eye UE of the user 230 facing the standard position as shown in FIG. 26. The housing 60 and the drive mechanism 62 are actually housed in, for example, an ATM device and are not visible to the user 230. The example of the space floating image 3 is an example of a password input screen at an ATM.

The image content and the graphical user interface that can be displayed as the space floating image 3 by the space floating image display device 1000 are not limited to this example. As another example, a GUI such as a keyboard of a personal computer may be used, and it can be applied to various uses such as store advertisements, art videos, and confirmation screens for products and prices at unmanned cash registers.

[Vertical placement]
FIG. 28 shows an arrangement example in which the space floating image display device 1000 of FIG. 26 is used vertically in the same manner as in FIG. 3B. In this state, the transparent member 100 is arranged vertically (Z direction). The spatial floating image 3 is formed at a position where it is emitted outward from the transparent member 100 in an obliquely upward direction in the Y direction. The surface of the space floating image 3 is formed in a state of being inclined diagonally downward in the Y direction.

The sensor 31 is arranged on the front surface of the housing 60 (the surface where the transparent member 100 is located) at a position on the upper side in the Z direction so that the detection axis is parallel to the surface of the space floating image 3. The camera 32 is arranged on the front surface of the housing 60 at a position lower in the Z direction, with the optical axis oriented diagonally upward, so as to cover an image pickup range covering the face UF at a standard position. The spatial floating image display device 1000 can be applied to both the case of using it horizontally as shown in FIG. 26 and the case of using it vertically as shown in FIG. 28, and the mechanism and operation are substantially the same. be. Hereinafter, the case of horizontal placement as shown in FIG. 26 will be described as an example.

FIG. 29 shows an example of arrangement of each sensor on the XY plane when the space floating image display device 1000 is viewed from above. In this example, the sensor 31 is composed of two sensors 31R and 31L on the left and right in the X direction. The sensors 31R and 31L are located at the left and right positions (XR, XL) with respect to the center position XC of the housing 60 as the positions in the X direction. When the sensor 31 is used by another method, it may be installed at the center position XC or the like of the housing 60.

In this example, the camera 32 is one camera. In that case, the position of the camera 32 in the X direction is the center position XC of the housing 60. The angle of view of the camera 32 in the X direction is a range wide enough to cover, for example, one side in front of the housing 60, including the face UF in the standard position. The position, direction, and imaging range of the second sensor are not limited to this example, and can be taken at a position where a face UF (standard face UFA) can be photographed at an assumed standard position (X0, Y0, Z0). All you need is. In another example, the camera 32 may be two left and right cameras (32R, 32L). In that case, the positions of the two cameras (32R, 32L) are, for example, the left and right positions (XR, XL) in the figure.

As for the assumed standard positions (X0, Y0, Z0), the face UF faces the space floating image 3 as shown in FIGS. 26 and 29, and the optical axis of the space floating image 3 and the line-of-sight direction of the eye UE are shown. It is a position corresponding to a matching state like a one-dot chain line. Due to the characteristics such as high directivity of the spatial floating image 3, the spatial floating image 3 can be suitably visually recognized from the eye UE of the user 230 at this standard position, and the more deviated from this standard position, the more difficult it becomes to visually recognize the spatial floating image 3.

[Determination of face position by camera]
Next, FIG. 30 is a schematic explanatory view for determining the position of the face UF by the camera 32, which is the second sensor of FIG. 26. FIG. 30A shows the state of the YZ plane when the space floating image display device 1000 is viewed from the side (the drive mechanism 62 is not shown here) of the user 230, as in FIG. 26. As the position of the face UF, a case where the height position in the Z direction changes to three types of height positions (ZA, ZB, ZC) is shown. The position of the face UF in the space (X, Y, Z) changes according to the height, movement, and the like of the user 230. The position A of the face UF is the case where it is in the standard position (X0, Y0, Z0), and the height position is Z0 = ZA. The position B of the face UF is a position higher than the standard position, and the height position is ZB. The position C of the face UF is a position lower than the standard position, and the height position is ZC. ZC <ZA <ZB.

FIG. 30B shows the position of the face UF in the image 3001 of the angle of view α captured by the camera 32, which corresponds to the example of the positions A, B, and C of the face UF in (A). In the image 3001 of the camera 32 of (B), the face UF of the user 230 is in the center position (CX0) in the CX direction, and is in the position corresponding to the height positions ZA, ZB, and ZC in the CY direction. When the face UF is in the position A (height position ZA) corresponding to the standard position (X0, Y0, Z0), the face UF is shown near the center of the image 3001 as shown in the figure. When the face UF is in the position B or the position C, the face is reflected in the upper or lower position from the center of the image 3001 as shown in the figure.

Therefore, the control board 61 of the space floating image display device 1000 analyzes the contents of the image 3001 of the camera 32, and the user for the space floating image display device 1000 and the space floating image 3 in the space (X, Y, Z). The position of the face UF of 230 can be determined. As an example, the control board 61 calculates the distance from the center point of the image 3001 (CX0, CY0) (shown as the intersection of the alternate long and short dash lines) to the position of the face UF in the image as a standard position. May be good. The position of the face UF in the image 3001 is defined, for example, at the midpoint of both eyes. The control board 61 can calculate the position of the face UF in the space (X, Y, Z) by conversion from the position of the face UF in the image 3001.

In particular, the control board 61 determines the position coordinates including the height position of the face UF. Further, the control board 61 may discriminate the height position of the face UF by dividing it into a plurality of types of height positions (for example, height positions ZA, ZB, ZC, etc.). For example, the control board 62 is at least at a height position near the standard position (X0, Y0, Z0), a height position above the standard position by a predetermined distance or more, and a height position below the standard position by a predetermined distance or more. Three types of height positions with respect to the position may be discriminated. In this case, for example, the initial value of the position setting of the spatial floating image display device 1000 is set to a position suitable for the operator facing the standard position (X0, Y0, Z0), and then the position of the face is detected by the camera 32. Adjust to another position (for example, (X1, Y1, Z1)) that is easy to see, and after the series of operations is completed and the operator leaves the space floating image display device 1000, the position of the space floating image display device 1000 is changed again. Return to standard position. As the next operator, there are many people who face the standard position set from the average height, etc., so it is possible to prepare for smoother operation.

The control board 61 adjusts the position and orientation of the space floating image 3 based on the determined position of the face UF by moving the housing 60.

[Rotation movement of housing]
FIG. 31 is a schematic explanatory view of the rotational movement of the housing 60 in the space floating image display device 1000. FIG. 31A shows the housing 60 from the standard state as shown in FIG. 30 by the drive mechanism 62 in the state of the XX plane when the space floating image display device 1000 is similarly viewed from the side. An example of the state after the rotation is shown. The housing 60 is rotated about the rotation axis J1 extending in the X direction so as to be tilted toward the back side in the Y direction at an angle θ, for example, on the YZ plane. The surface AA indicates the upper surface of the housing 60 (the surface on which the transparent member 100 is located) in the standard position. The surface AB indicates the upper surface of the housing 60 after rotation at an angle θ. The space floating image 3A at the standard position is in the state of the space floating image 3B after rotation. The rotation axis J1 is not limited to this example, and the details of the rotation mechanism are not particularly limited.

The position UFA of the face UF shown by the broken line indicates the standard position. The face UFB shown by the solid line shows an example of a position above the standard position. When the face UFB is located at an upper position with respect to the standard space floating image 3A, it is difficult to visually recognize the contents of the space floating image 3A.

(B) of FIG. 31 shows the position of the face UF in the image 3101 of the camera 32 corresponding to the example of (A). In the image 3101, the face FB indicates a face region in the case of the face UFB at the upper position. The space floating image display device 1000 determines the position of the face FB (for example, the position (CXB, CYB)) from the image 3101, and determines the position of the face UFB in the space (for example, (XB, YB, ZB)) from the position. Determine. The control board 61 of the spatial floating image display device 1000 rotates and moves the housing 60 by the drive mechanism 62 in response to the determination of the position of the face UF, so that the face UF is centered in the image 3101 of the camera 32. It is controlled so that the image is captured in the vicinity of (CX0, CY0).

In this example, the region of the face FBR in the image 3101 shows a state in which the face is reflected near the center after the housing 60 is rotated at the angle θ. Along with this rotation, the space floating image 3A is changed to the state of the space floating image 3B after the rotation. That is, in this post-rotation state, the user 230 can suitably visually recognize the contents of the spatial floating image 3B from the corresponding eye UE of the face UFB located at the upper position in the Z direction. In this post-rotation state, the spatial relationship between the face UFB eye UE and the spatial floating image 3B is similar to the spatial relationship between the standard face UFA eye UE before rotation and the spatial floating image 3A. It is adjusted to be a relationship.

As described above, in the first embodiment, the space floating image 3 can be adjusted to a suitable state according to, for example, the difference in height of the user 230 facing the space floating image display device 1000, and the user 230 can adjust the space. The floating image 3 (for example, FIG. 27) can be suitably visually recognized and operated. As another example, even when the face UF is located at the lower position C (height position ZC) from the standard position as shown in FIG. 30, the same adjustment as described above is possible. In that case, the direction of rotational movement may be a clockwise direction opposite to the counterclockwise direction of the angle θ in FIG.

As described above, according to the first embodiment, the convenience of the user can be enhanced with respect to the visual recognition and operation of the spatial floating image 3. Since the space floating image display device 1000 automatically adjusts the space floating image 3 according to the state of the user 230, the user 230 is suitable for the space floating image 3 without requiring any troublesome operation or operation. Visualization and operation are possible.

[Modification example-translation]
FIG. 32 shows a modified example of the first embodiment. In this modification, a case of parallel movement as another method of movement of the housing 60, particularly a case of parallel movement in the height direction (Z direction) is shown. FIG. 32 shows the housing 60 from the standard position when the face UF of the user 230 is the face UFB at a position higher than the standard in the state of the YZ plane when the space floating image display device 1000 is viewed from the side. The case where it is translated upward in the Z direction is shown. The space floating image display device 1000 adjusts the state of the space floating image 3 by the same mechanism as in FIG. 31 based on the image captured by the camera 32. The control board 61 is translated by the drive mechanism 62 upward at a distance DZ. The upper surface AC indicates the position of the upper surface after being translated upward by a distance DZ from the position of the upper surface AA of the standard housing 60. The distance DZ is a distance calculated so that the position of the face UF is near the center in the image of the camera 32. Along with this translation, the space floating image 3A is changed to the state of the space floating image 3B. The user 230 can suitably visually recognize and operate the adjusted spatial floating image 3B from the position of the face UFB (height position ZB).

[Transformation example-Move component]
As another modification, although not shown, by moving the components in the housing 60 instead of the housing 60 itself, the same effect as when the housing itself is moved can be obtained. For example, by rotating and moving the display device 1 and the retroreflective member 2, which are the constituent elements in the housing 60, while maintaining the mutual positional relationship, the position on the space where the space floating image 3 is displayed can be determined, for example. , Can be adjusted to a higher position.

As a result of the above, the user 230 can more preferably visually recognize the adjusted spatial floating image 3B even from the position of the face UF located at a position shifted upward from the standard position.

[Modification example-control in the left-right direction]
The following is also possible as another modification. This modification has a function of adjusting the spatial floating image 3 according to the position of the face UF in the left-right direction (X direction) of the spatial floating image display device 1000 and the user 230. An example of making adjustments in the X direction is shown with reference to FIG. 29 described above. In the X direction, the face UF position XE is, for example, an example of a position offset to the right with respect to the standard central position XA. It is difficult for the user 230 to visually recognize the standard space floating image 3 (3A) from this position XE. The distance DX is the distance between the position XA and the position XE in the X direction. The spatial floating image display device 1000 determines the position XE of the face UF in the X direction and the distance DX of the deviation based on the image of the camera 32, and determines the position X of the face UF in the X direction and the like. The body 60 is moved to adjust the state of the spatial floating image 3 to be suitable. In this case, the movement of the housing 60 by the drive mechanism 62 includes, for example, translation in the X direction or rotation around the Z axis in the XY plane. As a result, the user 100 can preferably visually recognize the adjusted spatial floating image 3 even from a position shifted to some extent in the X direction.

[Transformation example-Control in the depth direction]
33 and 34 show other variants. This modification has a function of adjusting the spatial floating image 3 according to the position of the face UF in the depth direction and the Y direction of the user 230 of the spatial floating image display device 1000 and the user 230. 33 and 34 are used to show an example of making adjustments in the Y direction. It is assumed that the preset standard position of the face UF (face UFA) is the height position ZA in the Z direction, the position YA in the Y direction, and the position XA in FIG. 29 in the X direction. It is assumed that the standard space floating image 3 set according to the standard position is the space floating image 3A. The face UFD indicates a case where the face UF is in the position YD in the Y direction. The distance YD is the distance between the position YD of the face UFD and the position YA of the standard face UFA. It is difficult for the user 230 to visually recognize the standard space floating image 3 (3A) from this position YD.

The spatial floating image display device 1000 determines the position YD of the face UF in the Y direction, the deviation distance DY, and the like based on the image of the camera 32, and depending on the state of the determined face UF, the housing The body 60 is moved to adjust the state of the spatial floating image 3 to be suitable. In this case, the movement of the housing 60 by the drive mechanism 62 includes, for example, translation in the Y direction or rotation around the X axis on the YZ plane. As a result, the user 100 can preferably visually recognize the adjusted spatial floating image 3 even from a position shifted to some extent in the Y direction.

[Transformation example-Control of floating image in space]
Further, as a modification, the following is also possible. When the position of the face UF changes in the Z direction as shown in FIG. 30, when the user 230 moves, the position changes in the X direction as shown in FIG. 29, and when the position changes in the Y direction as shown in FIG. 33. In any case, the space floating image display device 1000 controls the start and end of the display and adjustment of the space floating image 3 as follows. Here, a case of changing in the Y direction as shown in FIG. 33 will be described as an example. In FIG. 33, for example, consider a case where the user 230 approaches the space floating image display device 1000 in the Y direction from a distance to the front. Position YF is an example of a distant position.

Based on the detection by the image of the camera 32, the space floating image display device 1000 does not display the space floating image 3 when there is no face UF of the user 230 in the vicinity at first, in other words, puts it in a standby state. Alternatively, the spatial floating image display device 1000 determines the position of the face UF in the Y direction from the size of the face in the image even when the face UF is shown in the image of the camera 32, and the position is determined. When it is equal to or more than the threshold value, it is determined that the user 230 has not come to the position in front of the space floating image display device 1000, and the space floating image 3 is not displayed. For example, the threshold HY is an example of a threshold for the distance in the Y direction, for example, a predetermined distance from the standard position YA. When the space floating image display device 1000 determines that the position of the face UF is within the threshold value HY, the space floating image display device 1000 starts displaying and adjusting the space floating image 3. For example, initially, the space floating image display device 1000 displays the space floating image 3 (3A) at a standard position. After that, the space floating image display device 1000 adjusts the state of the space floating image 3 according to the position of the face UF within the threshold value HY and the distance DY.

FIG. 34 is an example of the image 3401 of the camera 32 corresponding to the example of FIG. 33. When the face UF is far from the standard position like the position YF, the image 3401 of the camera 32 shows the area of the face FF whose size is relatively smaller than the area of the face FA when the face UF is in the standard position. .. Further, the region of the face FF is shown as a position below the height position of the face FA when it is in the standard position (for example, a position (CXF, CYF)).

The control board 61 determines the movement of the face UF of the user 230 in the Y direction by determining the size of the region of the face UF of the user 230 and its change based on the analysis of the image 3401 captured by the camera 32. .. The control board 61 determines whether or not the position of the face UF is within the threshold value HY, the distance DY from the standard position YA, and the like. At that time, for example, the control board 61 may detect the pixel region of the face or the head in the image 3401 and determine the size (in other words, the area) based on the number of pixels or the like. The position of the control board 61 may be determined in the Y direction by conversion from its size. By comparing the size of the control board 61 with the size threshold value, whether or not the position of the face UF is within the threshold value HY (for example, whether or not the face UF is near the standard position YA) and the like. May be judged.

For example, when the distance DY corresponding to the position of the face UF in the Y direction exceeds a predetermined threshold value, the control board 61 starts adjusting the spatial floating image 3 in the X direction, the Y direction, and the Z direction. If it falls within a predetermined threshold, the adjustment of the spatial floating image 3 in the X direction, the Y direction, and the Z direction may be started.

As described above, in the modified example, the convenience of using the space floating image 3 by the user 230 is enhanced by controlling on / off and the like regarding the display and adjustment of the space floating image 3 according to the state of the user 230. Regarding the detection of the position of the face UF of the user 230 in the Y direction, if a TOF type sensor, a stereo camera, or the like is applied as the second sensor, it is easier to measure the position in the Y direction.・ Because it can be made highly accurate, more efficient control is possible.

[Modification example-code reading]
The following is also possible as a modification of the first embodiment. In this modification, the space floating image display device 1000 is applied to an application using a code such as a QR code (registered trademark). For example, the QR code may be used in an ATM device, or the QR code may be used in an unmanned cash register in a store. In this modification, the space floating image display device 1000 has a function of reading a QR code in the space floating image 3 in addition to the function of adjusting the space floating image 3 according to the position of the face UF described above.

FIG. 35A shows a modified example in which the user 230 uses the QR code of the display screen of his / her smartphone SP on the surface of the space floating image 3 in the state of the XZ surface when the space floating image display device 1000 is viewed from the side. Shows how to perform the operation of holding the. It is assumed that the face UF of the user 230 is in the above-mentioned standard position (height position ZA or the like).

FIG. 35B shows a display example of the image 3501 that guides the person to hold the QR code as a display example of the space floating image 3 by the space floating image display device 1000. In the image 3501, the area 3502 displays a GUI such as a message that guides the user 230 to hold the QR code, such as "Please hold the QR code here." The area 3502 is an area in which the reading result is displayed when the QR code is read from the image of the camera 32 by the control board 61 of the space floating image display device 1000. The reading result is the content corresponding to the QR code.

The user 230 visually confirms the image 3501 of the space floating image 3 as shown in FIG. 35 (B) at a position in front of the space floating image display device 1000, and displays the image 3501 on the display screen of the smartphone SP according to the guide. Hold the QR code over the area 3502. At that time, the camera 32 captures a range of the angle of view α including the QR code held over in the area 3502 of the space floating image 3. The control board 61 reads a QR code from the image of the camera 32. That is, the control board 61 recognizes the QR code information from the QR code image. The control board 61 displays the reading result of the QR code in the area 3503 of the image 3501 of the space floating image 3.

As described above, in this modification, the camera 32, which is the second sensor for adjusting the spatial floating image 3 described above, can also be used for reading a code such as a QR code. As described above, the camera 32 is designed so that a range including a standard face UF and at least a part of the spatial floating image 3 can be captured at an angle of view α. Therefore, the camera 32 can image the QR code arranged so as to overlap the spatial floating image 3 with high accuracy, and can realize the reading of the QR code with high accuracy.

Further, the space floating image display device 1000 adjusts the state of the space floating image 3 according to the position of the face UF as described above based on the image of the camera 32 at the time of the operation of reading the QR code. .. As a result, by setting the state of the spatial floating image 3 suitable for the individual state of the user 230, the user 230 can more easily operate the QR code on the surface of the spatial floating image 3, and the spatial floating image 3 can be operated. The video display device 1000 can read the QR code with higher accuracy.

Further, in FIG. 35A, the QR code of the smartphone SP is held slightly toward the front side in the z direction with respect to the surface of the space floating image 3. When reading a code on the surface of the space floating image 3, the distance between the surface of the space floating image 3 and the surface of the code (the screen of the corresponding smartphone SP) can be set in advance as a determination condition for reading the code. .. As an example of the determination condition, the reading of the code may be started when the distance is within a predetermined threshold value by ± in the front-back direction with the surface of the space floating image 3 as 0.

In particular, there may be a case where the image or object of a code such as a QR code must be aligned with high accuracy with respect to the position of the surface of the spatial floating image 3. In that case, the space floating image display device 1000 also adjusts the space floating image 3 according to the position of the face UF described above, so that the user 230 can use the code with respect to the position of the surface of the space floating image 3. It becomes easy to align the image and the object of the above with high accuracy.

The above modification is not limited to the QR code, and can be applied to an application in which an ID card, an identity verification certificate, or the like is read at the position of the space floating image 3.

As another modification, the direction of the camera 32 and the imaging range may be as follows. In the configuration example shown in FIG. 26 and the like, the direction of the optical axis of the spatial floating image 3 is different from the direction of the optical axis of the camera 32. The optical axis of the camera 32 is designed to pass above the upper side of the space floating image 3 and capture a standard face UF or eye UE. Not limited to this design, it is possible. For example, the direction of the optical axis of the camera 32 may be parallel to the direction of the optical axis of the spatial floating image 3. In this case, the coordinate system of the spatial floating image 3 and the coordinate system of the image of the camera 32 can be made closer to each other. Further, the optical axis of the camera 32 may pass through the region of the space floating image 3.

(Embodiment 2)
The space floating image display device 1000 of the second embodiment will be described with reference to FIGS. 36 and later. In the second embodiment, an example of the application is an ATM device of a bank. The spatial floating image display device 1000 according to the second embodiment has a function of performing user authentication associated with the bank account of the user 230. The space floating image display device 1000 user-authenticates 230 individuals who use and operate the screen displayed as the space floating image 3 (for example, the password input screen of the bank account), and the space floating according to the user authentication result. Control the display of the screen of the image 3.

Further, in the second embodiment, the space floating image display device 1000 has a function of adjusting the sensitivity regarding the detection / determination of the operation by the finger UH on the screen of the space floating image 3. When the user 230 presses a button or the like on the screen of the spatial floating image 3, the operation is also in the air, so that the characteristics (in other words, the habit) such as how to move the finger UH are different for each user 230. It is assumed that it is difficult for some users 230 to detect and determine the aerial operation using the first sensor. Therefore, the space floating image display device 1000 of the second embodiment determines the operation characteristics of the space floating image 3 for each user 230 by using the first sensor, the second sensor, or the third sensor described later. Then, the space floating image display device 1000 of the second embodiment adjusts the sensitivity of the operation of the screen of the space floating image 3 for the individual according to the operation characteristics of the user 230 individual. The adjustment of the sensitivity is specifically for determining the operation of the space floating image 3 using the first sensor (and the above-mentioned aerial operation detection unit 1350 in FIG. 3C) so as to match the characteristics of the individual operation. It is to change / adjust the judgment conditions.

In the second embodiment, the sensitivity of the operation of the space floating image 3 including the user authentication can be adjusted by using the characteristics of the individual operation, so that the user 230 can more easily operate the space floating image 3 including the user authentication. Available.

[User Authentication]
FIG. 36 shows the state of the XX plane when the configuration of the space floating image display device 1000 or the like in the second embodiment is viewed from the side. The balloon shows an outline of the user authentication process mainly performed by the control board 61 (particularly, the control unit 1110 in FIG. 3C). In the example of the second embodiment, the control board 61 (particularly, the processor constituting the control unit 1110) performs the process related to the two-step user authentication as the process related to the user authentication. Of the two stages, the first user authentication is face authentication using an image of the face UF of the user 230 using the camera 32, which is the second sensor. The control board 61 associates the face image of the user 230 with the face image taken by the camera 32 with the registered user (that is, the bank account) based on the comparison with the pre-registered face image. Face recognition processing is performed to determine whether or not the user is a user. If the result of the first user authentication is successful (OK), the space floating image display device 1000 displays a predetermined space floating image 3 (described later). If the result of the first user authentication is failure (NG), the space floating image display device 1000 then transitions to the second user authentication.

The second user authentication is user authentication using the dedicated touch pen UP. The dedicated touch pen UP is possessed by the user 230, and information including the user ID of the individual user 230 is built in the memory. The user ID is an ID associated with the bank account of user 230. The user 230 operates the dedicated touch pen UP according to the guide on the screen of the space floating image 3 displayed by the space floating image display device 1000. The dedicated stylus UP also has a short-range wireless communication function (for example, Bluetooth (registered trademark)) as a communication function. The control board 61 of the space floating image display device 1000 communicates with the dedicated touch pen UP through the communication unit 1132 of FIG. 3C, and receives / acquires information including a user ID from the dedicated touch pen UP. The control board 61 is a registered user (that is, a user associated with a bank account) of the user 230 having the dedicated touch pen UP based on the comparison with the pre-registered user ID from the acquired user ID. Whether or not to perform user authentication processing with a dedicated stylus.

If the result of the second user authentication is successful (OK), the space floating image display device 1000 displays a predetermined space floating image 3 (described later). When the result of the second user authentication is failure (NG), the space floating image display device 1000 erases the space floating image 3 (in other words, the display ends) as a process corresponding to the comprehensive user authentication result failure. Alternatively, processing such as displaying the spatial floating image 3 indicating "authentication failure" is performed, and the display does not transition to the predetermined spatial floating image 3.

In this example, the display of the predetermined space floating image 3 is the display of the password input screen in the GUI of the application of the ATM device. On the screen of the predetermined space floating image 3, the space floating image display device 1000 accepts an operation by the user 230's finger UH or the dedicated touch pen SP. The space floating image display device 1000 detects and determines an operation by the finger UH or the dedicated touch pen SP on the screen of the space floating image 3 by using the sensor 31 which is the first sensor. For example, when detecting an operation by the dedicated touch pen SP, the position, the depth of entry, and the like with respect to the surface of the space floating image 3 are determined for the tip portion of the dedicated touch pen SP.

[Predetermined screen]
FIG. 37 shows, as an example of the screen of the predetermined space floating image 3, the state in which the password input screen is viewed from the eye UE of the user 230 facing the user. This screen is composed of a first stage screen 3701 and a second stage screen 3702. The space floating image 3 is a vertically long screen in this example. The space floating image display device 1000 first displays the screen 3701 of the first stage. On the screen 3701, a confirmation button 3703 as a GUI component for sensitivity adjustment and a guide message ("Please press the confirmation button above") are displayed at the center position. The user 230 performs an operation (touch operation in the air) of pressing the confirmation button 3703 with the finger UH or the dedicated stylus UP according to the guide. When the space floating image display device 1000 detects the operation of pressing the confirmation button 3703 using the first sensor or the third sensor, the space floating image display device 1000 may display an effect or the like indicating that the confirmation button 3703 has been pressed. .. As a result, the user 230 is fed back that the confirmation button 3703 is pressed in an easy-to-understand manner.

In response to the operation of the confirmation button 3703, the space floating image display device 1000 performs a sensitivity adjustment process, and then transitions to the display of the screen 3702 in the second stage. The screen 3702 of the second stage is the main body of the password input screen, and is provided with a number button 3704, a guide message ("Please press the password (4 digits)"), an input password confirmation field 3705, and the like. .. The user 230 sequentially presses the number button 3704 corresponding to the personal identification number with a finger UH or the like. The control board 61 detects the pressing of the number button 3704 based on the determination condition using the first sensor, and displays the input number for confirmation in the input password confirmation field 3705 according to the detection. This confirmation display may be an asterisk (*) or the like instead of the number itself, as in the publicly known technique. This example shows an example applied to the input of the personal identification number of the ATM, but the present invention is not limited to this, and can be similarly applied to various uses.

[Characteristic detection / sensitivity adjustment]
In FIG. 36, the space floating image display device 1000 of the second embodiment has a characteristic detection sensor 33 (in other words, a sensor for adjusting sensitivity) as a third sensor in addition to the same components as those of the first embodiment in the housing 60. ). Alternatively, the sensor 31 which is the first sensor described above may be used as a characteristic detection sensor, in other words, the first sensor and the third sensor may be integrated. Alternatively, the camera 32, which is the second sensor described above, may be used as a characteristic detection sensor, in other words, the second sensor and the third sensor may be integrated. The characteristic detection sensor 33 is a sensor for detecting the characteristics of the operation of the user 230 with respect to the spatial floating image 3. The characteristic detection sensor 33 may be installed at the same position as the sensor 31, which is the first sensor, as shown in FIG. 36, or may be installed at a different position. In this example, the characteristic detection sensor 33 is located at the same position as the first sensor, on the upper surface of the housing 60 in the Z direction, on the front side in the Y direction, and near the left and right positions XR and XL in FIG. 29 in the X direction. is set up. The direction of the detection axis of the characteristic detection sensor 33 is, for example, a direction parallel to the surface of the space floating image 3. In this example, as the characteristic detection sensor 33, two cameras on the left and right in the X direction are used as stereo cameras. By using this, it is possible to detect the position, movement, etc. of the finger UH or the like in a three-dimensional space (X, Y, Z). In another example, the third sensor may be provided as one sensor in the center position in the X direction. The third sensor may be a sensor that can detect the position, movement, etc. of the finger UH or the like in a three-dimensional space.

As the sensor 31 which is the first sensor and the camera 32 which is the second sensor, the same ones as described in the first embodiment can be applied. Not limited to this, the second sensor may apply a sensor different from the camera 32 applied in the first embodiment in consideration of user authentication. Further, when detecting the characteristics of the operation of the individual user, the second sensor may be used in combination with the third sensor. Further, as the third sensor, one TOF type distance measuring sensor may be used. Further, one or more sensors 32 which are the first sensors, one or more cameras 32 which are the second sensors, and any one or more sensors which are the third sensors (for example, a camera such as a CCD, or a TOF method). The function may be configured by combining the distance measuring sensor).

[Processing flow]
FIG. 38 shows the main processing flow of the space floating image display device 1000 according to the second embodiment, and includes steps S200 to S208. In step S200, the control board 61 performs the two-step user authentication shown in FIG. When the user authentication in step S200 is successful and the individual user 230 is identified, the control board 61 starts the characteristic detection / sensitivity adjustment process in step S201. In step S202, the control board 61 displays the screen 3701 of the first stage of FIG. 37. The user 230 performs an operation (touch operation in the air) of pressing the confirmation button 3702 with the finger UH or the dedicated touch pen SP.

In step S203, the control board 61 detects the operation of pressing the confirmation button 3702 using the third sensor. The control board 61 acquires an image from the third sensor. At this time, the space floating image display device 1000 may detect the initial movement of the finger UH or the like by using the first sensor or the second sensor. Then, the space floating image display device 1000 may start the detection by the third sensor in response to the detection of the initial motion.

In step S204, the control board 61 confirms whether or not the operation of pressing the confirmation button can be detected, and repeats the same until it can be detected. If it cannot be detected, the flow may be terminated, for example.

In step S205, the control board 61 determines the depth, position, direction / angle, area, speed, time, etc. of the finger UH when the confirmation button is pressed, based on the analysis of the image from the third sensor. By measuring the value of a predetermined parameter, the characteristics of the operation of the user 230 are determined. Specific examples will be described later.

In step S206, the control board 61 changes / updates the determination conditions at the time of detecting the operation of the space floating image 3 using the first sensor so as to match the characteristics of the operation of the individual user determined above. The control board 61 changes the determination condition according to the individual characteristics based on the determination condition set as a standard in advance. The control board 61 stores the determination condition in association with the individual user ID. In other words, this change in the determination condition corresponds to the adjustment of the sensitivity when operating the button or the like on the screen of the predetermined space floating image 3. Also, this corresponds to a calibration tailored to the individual user.

In step S207, the control board 61 shifts the display of the space floating image 3 to the screen 3702 of the second stage of FIG. 37. The control board 61 (particularly, the aerial operation detection unit 1350 of FIG. 3C) applies the above-mentioned changed / updated determination conditions when the user 230 detects / determines the operation of the screen 3702 of the space floating image 3.

After that, in step S208, the control board 61 ends the characteristic detection / sensitivity adjustment process at a predetermined opportunity, for example, in response to the success of the password input on the screen 3702 of the second stage of FIG. 37. After the end of the flow of FIG. 38, the determination condition is returned to the setting value of the standard determination condition corresponding to any user.

The space floating image display device 1000 stores the above-mentioned changed / updated determination conditions as information for each individual user for at least a certain period of time. In that case, the spatial floating image display device 1000 adjusts the sensitivity using the screen 3701 of the first stage when the same user 230 individual uses the same screen 3702 next time based on the user authentication (determination condition). (Update) may be omitted, and the saved determination condition may be read out and applied to the screen 3702 of the second stage. Alternatively, the space floating image display device 1000 may apply the change (sensitivity adjustment) of the above determination condition every time the screen 3702 is used, and may not save the determination condition after the change / update.

[Example of characteristic detection / sensitivity adjustment]
FIG. 39 is a schematic explanatory view of an example of sensitivity adjustment using the third sensor, and is a predetermined space by the finger UH of the user 230 in the state of the XX plane when the space floating image display device 1000 is viewed from the side. A state when operating the screen of the floating image 3 (the screen 3701 of the first stage in FIG. 37), particularly an example of the movement of the finger UH on the XX plane is shown.

The control board 61 (particularly, the processor of the control unit 1110 in FIG. 3C) uses, for example, a finger UH (the same applies to the dedicated touch pen UP) for the surface of the space floating image 3 from the two images captured by the two cameras which are the third sensors. ), The position, movement, and other states of the finger UH during the operation are determined, and the characteristics (habit) of the operation of the user 230 are determined from the state.

In the example of FIG. 39, as characteristics, in the coordinate system (x, y, z) of the spatial floating image 3, the characteristic of the depth of entry of the finger UH in the z direction, which is the depth direction, and the characteristic of the spatial floating image 3 A case is shown in which characteristics such as a deviation and orientation (angle from the axis in the z direction) of the finger UH with respect to the position of the target object 4001 (confirmation button 3703 in FIG. 37) in the center are determined. The confirmation button 3703, which is the target object 4001, is provided to measure the characteristics of the operation.

The control board 61 uses the detection information of the third sensor or the like to represent parameters such as depth, position, angle, and area that represent the operation characteristics of the user 230 individual (identified as an individual from the above-mentioned user authentication result). Calculate the value. For example, in FIG. 38, the depth in the z direction is a value corresponding to a distance dz from, for example, the position of the tip of the finger UH (for example, zc) with respect to the reference position z where the surface of the space floating image 3 is located. The position za corresponds to the case where the tip of the finger UH just touches the surface of the space floating image 3. The position zb is an example of a position on the front side of the position za of the spatial floating image 3, and corresponds to a case where the tip of the finger UH does not touch the surface. The position zc is an example of a position on the back side of the surface of the space floating image 3, and corresponds to a case where the tip of the finger UH has entered the back.

The control board 61 obtains the characteristics of the operation of the individual user 230 as a value, and then changes the determination conditions so as to match the characteristics. An example of the determination condition is a depth threshold value relating to the depth of entry of the finger UH in the z direction with respect to the surface of the space floating image 3. The control board 61 sets, for example, a threshold value (for example, a value close to the distance dz) among the determination conditions to be applied to the individual according to the characteristic of the determined depth (for example, the distance dz). For example, different threshold values are set for a user who touches the surface of the spatial floating image 3 shallowly and a user who touches the surface deeply.

When the user 230 operates the screen 3702, the control board 61 (particularly, the aerial operation detection unit 1350 of FIG. 3C) enters the target UH into the target object 4001 as a determination condition based on the detection information of the first sensor. When the depth of is equal to or greater than the depth threshold value (for example, the distance dz), it is determined that the operation of pressing the target object 4001 (touch operation in the air) has been performed.

By such a change of the determination condition, the determination of the operation of the space floating image 3 can be made a suitable determination suitable for the individual user 230. For the user 230, it is possible to obtain a feeling that the space floating image 3 is easy to operate.

The angle 3901 when the finger enters can be converted into a distance in the depth direction by a simple calculation, and can be reflected in the determination condition regarding the depth.

FIG. 40 shows, as an example of determining other characteristics, the target object 4001 (confirmation button 3703) in the space floating image 3 on the XY plane seen from above, where the user 230 is the tip of the index finger of the right hand as the finger UH. The state of the operation of pressing is shown. In this example, at this time, the user 230 moves his / her right hand as shown by the broken line as a locus, and enters the fingertip in the back in the z direction so as to press the target object 4001. Such an operation appears as a habit that differs for each user. In this example, the fingertip of the user 230 passes through a position (xf, yf) slightly shifted to the right with respect to the central target object 4001 on the xy plane of the spatial floating image 3. As a characteristic of such an operation, the space floating image display device 1000 has a position, a deviation distance, a direction (for example, right) and an angle of contact with the fingertip with respect to the position of the central target object 4001 on the surface of the space floating image 3. To judge. The angle is, for example, an angle 3901 with respect to the axis in the z direction in FIG. 39. When the finger UH is made to enter the target object 4001 on the surface of the space floating image 3 in the z direction, the position or direction deviation in the surface may be large as an individual characteristic, and the deviation with respect to the axis in the z direction may be large. The angle 3901 may be large.

Also, if it is another user 230, it may be operated with the left hand, it may be operated with another fingertip, it may be operated with two fingers, or it may be operated with the entire palm (shape such as goo or par). May do. The space floating image display device 1000 also determines such characteristics based on the third sensor.

FIG. 41 shows, as an example of determining other characteristics, the xy plane of the space floating image 3 viewed from the viewpoint of the user 230, and the position and area as characteristics when the finger UH contacts the surface of the space floating image 3. An example of determining is shown. Examples of the determination conditions include a threshold value for the deviation of the contact position with respect to the target object 4001 at the center position of the space floating image 3, a threshold value for the contact area, and the like. For example, the contact area 4101 shows an outline of the contact area of the finger UH when the user 230 presses the position slightly to the right of the target object 4001 with the index finger of the right hand as in FIG. 40. On the other hand, the contact area 4102 shows an outline of the contact area of the finger UH when another user 230 presses the position slightly to the left of the target object 4001 with the entire left hand. In this way, the contact area on the surface of the spatial floating image 3 differs depending on the individual. The control board 61 schematically calculates such a contact area, and changes the threshold value of the contact area under the determination conditions for each individual according to the calculated contact area. That is, the threshold value of the relatively small contact area is set for the user on the right hand side in the figure, and the threshold value of the relatively large contact area is set for the user on the left hand side.

As other characteristics and determination conditions, the speed of movement of the finger UH or the like may be used. The space floating image display device 1000 uses a third sensor to determine the speed of movement of the finger UH or the like with respect to the target object 4001 of the space floating image 3 based on the time-series detection data. Then, the space floating image display device 1000 updates the speed threshold value as one of the operation determination conditions according to the characteristics of the speed of the individual. For example, when the number button 3704 is pressed, if the speed is equal to or higher than the threshold value, it is determined that the number button is pressed.

As other characteristics and determination conditions, the contact time of the finger UH or the like may be used. The space floating image display device 1000 uses a third sensor to determine the time of contact of the finger UH or the like with the target object 4001 of the space floating image 3 based on the time-series detection data. Then, the space floating image display device 1000 updates the threshold value of the contact time as one of the operation determination conditions according to the characteristics of the contact time of the individual. For example, when the number button 3704 is pressed, if the contact state is maintained for a time equal to or longer than the threshold value, it is determined to be pressed.

As described above, by setting the determination conditions in consideration of various characteristics, high-precision detection and prevention of erroneous operation detection become possible. For example, in the case of an individual user who uses the index finger of the right hand shown in the figure, even if the entire hand unintentionally touches the surface of the space floating image 3 (for example, multiple number buttons), it is detected as a regular operation. It is not judged and erroneous operation can be prevented.

Further, as another control method, in the case of an individual who operates with the entire finger UH, such as a user who uses the entire left hand in the figure, the space floating image display device 1000 responds to the determination of the characteristics of the individual. As the determination condition, the determination condition for operation with one fingertip may be switched to the determination condition for operation with the entire finger. For example, the determination condition for the operation of the entire finger is to press the target object 4001 based on the depth threshold value or the like by using the portion of the entire finger UH that first contacts the surface of the space floating image 3. It is a condition to judge. In this case, even if the entire finger touches a plurality of buttons, it can be determined that one button that is partially touched first is pressed.

[Transformation example-single screen]
The following is also possible as a modification of the second embodiment. In FIG. 37, the space floating image display device 1000 omits the screen 3701 of the first stage, displays the screen 3702 from the beginning, and responds to the detection of the operation on the button (for example, the number button 3704) of the screen 3702 by the individual. The characteristics may be determined and the operation determination conditions may be updated immediately.

[Modification example-code reading]
The following is also possible as a modification of the second embodiment. For example, in the above-mentioned ATM device, at the time of the second user authentication, the user authentication using the camera 32, the QR code, or the like may be applied instead of the authentication using the dedicated touch pen UP. Alternatively, even when applied to other devices, user authentication using the camera 32, QR code, or the like may be applied in the same manner. As the user authentication mechanism, the same mechanism as that of the modification of the first embodiment (FIG. 35) can be applied.

FIG. 42 shows an example of user authentication in this modified example. FIG. 42 shows the QR code displayed on the display screen of the smartphone SP held by the user 230 on the finger UH in the state of the YZ plane when the space floating image display device 1000 or the like is viewed from the side of the space floating image 3. Shows how to hold it up to the surface.

The control board 61 guides the user 230 to hold the QR code at the time of the second user authentication in the ATM device, and the user 230 is a smartphone in the area of the spatial floating image 3. Hold the QR code of the SP so that it overlaps. The control board 61 acquires an image in which the QR code is captured by using the camera 32 which is the second sensor, and authenticates the individual user 230 based on the reading result of the QR code. The control board 61 displays the result of user authentication in the screen of the space floating image 3.

As described above, in this modification, the second sensor can be utilized to enable user authentication using the presentation of the code on the surface of the space floating image 3. Further, in this modification, by changing the determination condition for each individual to be applied in the operation of this user authentication in the same manner as described above, the user 230 can operate this user authentication (for example, the spatial floating image 3). The operation of aligning the cord with high accuracy on the surface of the surface) can be performed more easily.

Although the present invention has been specifically described above based on the embodiments, the present invention is not limited to the above-described embodiments and can be variously modified without departing from the gist. A form in which each of the above-described embodiments is combined is also possible. In each of the above-described embodiments, it is also possible to add / delete / replace the components except for the essential elements.

1000 ... Spatial floating image display device, 1 ... Display device, 2 ... Retroreflective member, 3 ... Spatial floating image, 11 ... Liquid crystal display panel, 12 ... Absorption type polarizing plate, 13 ... Light source device, 21 ... λ / 4 board, 100 ... Transparent member, 101 ... Polarizing separation member, 31 ... Sensor (first sensor), 32 ... Camera (second sensor), 60 ... Housing, 61 ... Control board, 62 ... Drive mechanism, 230 ... User, UF ... face, UE ... eyes, UH ... fingers.

Claims (15)

Light source and
An image source that emits image light based on the light from the light source,
A retroreflective member that forms a spatial floating image with a predetermined directivity based on the image light,
A housing that houses the light source, the image source, and the retroreflective member.
A drive mechanism for moving the housing or components within the housing, and
A controller that controls the display of the space floating image formed at a predetermined position in the space outside the housing, and
A first sensor for detecting an operation on the space floating image by a user who visually recognizes and operates the space floating image, and
A second sensor for detecting the position of the user's face or head with respect to the spatial floating image, and
Equipped with
The controller moves the housing or components within the housing by the drive mechanism according to the position of the user's face or head with respect to the spatial floating image.
Space floating image display device. In the space floating image display device according to claim 1,
The drive mechanism rotates and moves the housing or components within the housing.
Space floating image display device. In the space floating image display device according to claim 1,
The drive mechanism translates the housing or components within the housing.
Space floating image display device. In the space floating image display device according to claim 1,
The controller uses the second sensor to determine the position in the vertical direction as the current position of the face or head with respect to the standard position of the face or head with respect to the spatial floating image.
Space floating image display device. In the space floating image display device according to claim 1,
The controller uses the second sensor to determine the position in the left-right direction as the current position of the face or head with respect to the standard position of the face or head with respect to the spatial floating image.
Space floating image display device. In the space floating image display device according to claim 1,
The controller uses the second sensor to determine the position in the anteroposterior direction as the current position of the face or head with respect to the standard position of the face or head with respect to the spatial floating image.
Space floating image display device. In the space floating image display device according to claim 1,
The controller uses the second sensor to determine the difference in the current position of the face or head with respect to the standard position of the face or head with respect to the spatial floating image, and when the difference is within the threshold value, the controller. Start adjusting,
Space floating image display device. In the space floating image display device according to claim 1,
The controller reads the object using the second sensor in response to an operation of holding the predetermined object over the spatial floating image so as to overlap it.
Space floating image display device. Light source and
An image source that emits image light based on the light from the light source,
A retroreflective member that forms a spatial floating image with a predetermined directivity based on the image light,
A housing that houses the light source, the image source, and the retroreflective member.
A controller that controls the display of the space floating image formed at a predetermined position in the space outside the housing, and
A first sensor for detecting an operation on the space floating image by a user who visually recognizes and operates the space floating image, and
A sensor for detecting the characteristics of the operation by the user with respect to the space floating image, and
Equipped with
The controller determines the characteristics of the operation by the user for the space floating image based on the sensor, and changes the determination condition for the operation for the space floating image using the first sensor so as to match the characteristics. do,
Space floating image display device. In the space floating image display device according to claim 9,
The controller displays a screen including the target object for determining the characteristic as the space floating image, and determines the characteristic of the operation with respect to the target object.
Space floating image display device. In the space floating image display device according to claim 9,
As the characteristic, the controller determines the depth of entry of the user's finger or belongings in the direction perpendicular to the surface of the space floating image, and changes the threshold value regarding the depth as the determination condition.
Space floating image display device. In the space floating image display device according to claim 9,
As the characteristic, the controller determines the position of contact of the user's finger or belongings on the surface of the space floating image, and changes the threshold value for the position as the determination condition.
Space floating image display device. In the space floating image display device according to claim 9,
As the characteristic, the controller determines the area of contact of the user's finger or belongings on the surface of the space floating image, and changes the threshold value for the area as the determination condition.
Space floating image display device. In the space floating image display device according to claim 9,
A second sensor for detecting the position of the user's face or head with respect to the spatial floating image is provided.
The controller uses the second sensor to acquire an image of the user's face, and performs user authentication by face authentication based on the face image.
Space floating image display device. In the space floating image display device according to claim 9,
A second sensor for detecting the position of the user's face or head with respect to the spatial floating image is provided.
The controller reads the object using the second sensor in response to an operation of holding the predetermined object over the spatial floating image so as to overlap it.
Space floating image display device.

Images