JP2023007394A - Information processing system - Google Patents

Abstract

To provide an information processing system that displays an image of an operation screen including a plurality of operation buttons in an operation space as an aerial image, and identifies and processes operation information on the basis of non-contact operation for the aerial image.SOLUTION: An information processing system includes a controller, a display, a retroreflective material, and an optical sensor. The controller generates image information on an operation screen including a plurality of operation buttons. The display emits an optical signal into a housing according to the image information from the controller, and the retroreflective material transmits the optical signal incident from the display toward an operation space and forms an image in the operation space as an aerial image. The optical sensor detects a position of an object existing in the operation space on the basis of reflected light of light incident into the operation space. Also, the controller generates operation information for identifying an operation button operated on the basis of a detection result of the optical sensor and executes processing according to the generated operation information.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing system that processes information by user's operation.

In recent years, due to the corona crisis, there has been an increase in the need to avoid touching touch panels used by an unspecified number of users. It has been demanded.

An electronic device that can be operated by gestures without contact has been proposed (see Patent Document 1). This electronic device has a structure of an electronic device capable of processing based on a user's gesture operation detected by a proximity sensor.

JP 2018-50175 A

According to the conventional technology, it is possible to operate an electronic device using gestures without touching a display with a touch panel. It is difficult for the user to operate the arranged operation buttons. Moreover, in the prior art, in addition to the operation screen displayed on the display with the touch panel, no consideration is given to displaying an image of the non-contact operation screen as an aerial image in the operation space.

An object of the present invention is to display an image of an operation screen including a plurality of operation buttons as an aerial image in an operation space, and to identify and process operation information based on non-contact operations on the aerial image.

In order to achieve the above object, the present invention provides a controller for generating image information relating to an image of an operation screen including a plurality of operation buttons, and a controller which is configured in a substantially box shape and has an operation space formed by an opening in a part thereof. a housing arranged in the housing to input the image information from the controller, display an image according to the input image information on a screen, and emit an optical signal based on the image into the housing. and a display arranged in the housing to receive the optical signal from the display, transmit the incident optical signal toward the operation space, and use an image formed by the transmitted optical signal as an aerial image in the operation space. a retroreflecting material that forms an image inside the operating space; irradiating light toward the operating space; an optical sensor that detects the position of the object existing in the space, and the controller receives operation information specifying the operated operation button among the plurality of operation buttons based on the detection result of the optical sensor. It is characterized by generating and executing processing according to the generated operation information.

According to the present invention, an image of an operation screen including a plurality of operation buttons can be displayed as an aerial image in an operation space, and operation information based on non-contact operations on the aerial image can be identified and processed.

1 is a block diagram showing a configuration example of an information processing system according to Example 1 of the present invention; FIG. 1 is a perspective view showing an appearance of an information processing system according to Example 1 of the present invention; FIG. BRIEF DESCRIPTION OF THE DRAWINGS It is sectional drawing of the floating display module based on Example 1 of this invention, (a) is right side sectional drawing, (b) is front sectional drawing. 1A is a cross-sectional view of a floating display module according to a first embodiment of the present invention, FIG. FIG. 4 is a cross-sectional view when the bottom reflector is not curved; It is a block diagram which shows the structural example of the information processing system which concerns on Example 2 of this invention. FIG. 10 is a perspective view showing the appearance of an information processing system according to Example 2 of the present invention; FIG. 11 is a block diagram showing a configuration example of a large-sized operation terminal according to Example 3 of the present invention; 3A is an external perspective view of the large-sized operation terminal, and FIG. 3B is an enlarged portion of the operation section of the large-sized operation terminal. FIG. FIG. 12 is a block diagram showing a configuration example of a large-sized operation terminal according to Example 4 of the present invention; FIG. 12A is a diagram for explaining the relationship between the large-sized operation terminal according to the fifth embodiment of the present invention and the visual sense of the user, and FIG. FIG. 14 is an explanatory diagram for explaining the relationship between the positions of the operation buttons and the sensor reaction positions in the large-sized operation terminal according to the fifth embodiment of the present invention; FIG. 11 is an enlarged configuration diagram of an aerial display unit in the large-sized operation terminal according to Example 5 of the present invention; FIG. 5 is a diagram showing a display example of an aerial image applied to Examples 1 to 5 of the present invention; 4(a) and 4(b) are schematic cross-sectional views for explaining a bottom reflector of an aerial display module according to another embodiment of the present invention. FIG. (a) and (b) are front views for explaining a bottom reflector of a mid-air display module according to another embodiment of the present invention. (a) to (d) are diagrams showing changes in the screen when an operation button is touch-operated on a display example of an aerial image applied to Examples 1 to 6 of the present invention.

Preferred embodiments of the present invention will now be described in detail with reference to the drawings.

(1) Example 1
FIG. 1 is a block diagram showing a configuration example of an information processing system according to Embodiment 1 of the present invention. In FIG. 1, a small operation terminal 100 and a touchless device 200 are configured as, for example, an information processing system or an information processing device. The small-sized operation terminal 100 includes a control section 110 and an operation panel 140 including a display 120 and a touch panel 130 . The touchless device 200 is a device that transmits and receives information to and from the small operation terminal 100 , and includes a touchless device control box (BOX) 210 and an aerial display section 240 . The touchless device control box 210 includes a control board 230 and a touchless device control unit 220, and the aerial display unit 240 includes an aerial display 250 and an aerial display module 400 including an optical touch sensor 260 and a speaker 270. to be installed.

A display 120 is connected to the control unit 110 via a DVI (Digital Visual Interface) interface 111, and a touch panel 130 is connected via a USB (Universal Serial Bus) interface 112. 113 to the control board 230 in the touchless device control BOX 210 . The touchless device control unit 220 is connected to the control board 230 in the touchless device control box 210 via the USB interface 231 and the audio interface 232, and the speaker 270 of the floating display unit 240 is connected via the audio signal interface 233. , and connected to the aerial display 250 of the aerial display section 240 via the backlight control interface 234 . The touchless device control unit 220 is connected to the aerial display 250 of the aerial display unit 240 via an HDMI (High Definition Multimedia Interface) (registered trademark) interface 221, and is connected to the aerial display 250 via a USB interface 222. It is connected to the optical touch sensor 260 of the display unit 240 .

The control unit 110 is configured by, for example, a computer (not shown) including a processor, a storage device, and a communication device, and functions as a controller that performs integrated control of the touchless device 200 . The processor is configured using, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). A storage device is a device that stores computer programs and data, and includes, for example, ROM (Read Only Memory), RAM (Random Access Memory), and nonvolatile semiconductor memory.

The storage device includes, for example, an SSD (Solid State Drive), an IC card (Integrated Circuit Card), an SD (Secure Digital) memory card, and the like.

A communication device is a wired or wireless communication interface that realizes communication with another device via a communication means such as a LAN or the Internet. The communication device is, for example, a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Bus) module, or a serial communication module.

The operation panel 140 includes a display 120 that displays an image based on image information generated by the control unit 110 on its surface, and a touch panel 130 that transfers touch operation information corresponding to a touch operation on the display screen to the controller. The display 120 is composed of, for example, a liquid crystal display having a plurality of liquid crystal display elements or a liquid crystal display device, and displays various information on the screen. The touch panel 130 is, for example, a resistive film type touch panel in which a plurality of transparent electrode films are vertically divided and arranged on a screen panel. In the touch panel 130, when a portion of the upper transparent electrode film is pushed downward and contacts the lower transparent electrode film, a portion of the upper transparent electrode film and a portion of the lower transparent electrode film The position is detected at the timing of energization, and the detection signal is transferred to the control section 110 . At this time, when the user performs a touch operation on the touch panel, the touch panel 130 transfers touch operation information (information including characters, etc.) corresponding to the touch operation to the control unit 110 .

For example, the control unit 110 generates image information regarding an image of an operation screen including a plurality of operation buttons based on touch operation information input from the touch panel 130, and displays and generates the image of the operation screen on the display 120. Image information is transferred to the control board 230 of the touchless device control BOX 210 .

The control board 230 relays information and signals transmitted and received between the control unit 110 and the touchless device control unit 220, and relays information and signals transmitted and received between the control unit 110 and the floating display unit 240. When the image information transferred from the control unit 110 is received, the received image information is transferred to the touchless device control unit 220 .

The touchless device control section 220 is composed of, for example, a computer (not shown) having a processor, a storage device, and a communication device, and functions as an auxiliary controller that controls the floating display section 240 in an integrated manner. When receiving the image information from the control unit 110 , the touchless device control unit 220 transfers the image information to the aerial display 250 and displays an image according to the image information on the screen of the aerial display 250 .

The aerial display 250 is composed of, for example, a liquid crystal display (liquid crystal display device) having a plurality of liquid crystal display elements. The light is emitted toward an operation space (an operation space formed in the aerial display module 400) for the operation, and an aerial image is formed in the operation space. The aerial image formed within the operation space includes an image of an operation screen including a plurality of operation buttons.

The optical touch sensor 260 irradiates light into the operation space, receives the reflected light reflected by an object existing in the operation space, and determines the position of the object existing in the operation space based on the incident reflected light. is an optical sensor that detects Specifically, the optical touch sensor 260 has, for example, an infrared projector and an infrared image sensor, and is arranged around the operation space. At this time, infrared rays emitted from the image sensor into the operation space are reflected by an object (user's finger) existing in the operation space, and the reflected infrared rays enter the image sensor. For example, when a user's finger is inserted into the operation space, the optical touch sensor 260 detects the position (coordinates) of the finger, and outputs a detection signal corresponding to the detected position to the touchless device control unit 220 as a detection result. transfer to

The touchless device control unit 220 responds to the detection signal of the optical touch sensor 260, generates an audio signal indicating an operation sound, and transfers the generated audio signal to the speaker 270 via the control board 230. A detection signal indicating that the operation button has been operated is transferred to the control unit 110 via the control board 230 . As a result, the speaker 270 sounds an operation sound indicating that the operation button in the aerial image has been touched in the air by the user. Further, the control unit 110 can determine which position of the operation button in the aerial image has been operated by receiving a detection signal indicating that the operation button has been operated. At this time, the control unit 110 determines the detection signal from the optical touch sensor 260, and the operation information (indicating which operation button has been operated) according to the position of the object (finger) detected by the optical touch sensor 260. Based on the operation information), a process corresponding to the operation of the operation button, for example, a payment process is executed.

FIG. 2 is a perspective view showing the appearance of the information processing system according to Example 1 of the present invention. In FIG. 2, a display 120 and a touch panel 130 are mounted on the small operation terminal 100, a touchless device control BOX 210 is connected to the small operation terminal 100, and an aerial display section 240 is connected to the touchless device control BOX 210. be done. A control board 230 and a touchless device control unit 220 are mounted on the touchless device control box 210 . The aerial display unit 240 is equipped with an aerial display 250 , an optical touch sensor 260 and a speaker 270 .

At this time, by mounting the touchless device control BOX 210 and the aerial display unit 240 (aerial display module 400) separately, the installation location of the touchless device control BOX 210 becomes free, and the aerial display unit to be operated by the user becomes available. The outer shape of the display unit 240 can be reduced.

3A and 3B are cross-sectional views of the floating display module according to the first embodiment of the present invention, where (a) is a right cross-sectional view and (b) is a front cross-sectional view. In FIG. 3, the aerial display module 400 is composed of a housing including a front portion 400a, a rear portion 400b, a left side portion 400c, a right side portion 400d, a top portion 400e, and a bottom portion 400f, and is formed in a substantially rectangular parallelepiped box shape. ing. The left side portion 400c and the right side portion 400c are each formed with an inclined upper portion, and the aerial display module 400 as a whole is configured such that the front side is low and the rear side is high. That is, the upper surface portion 400e is formed as an inclined surface. A substantially rectangular opening (not shown) is formed in the upper surface portion 400e. The area of the opening is configured as an operation space for user's operation.

Inside the front part 400a, the aerial display 250 is arranged opposite to the rear part 400b. A speaker 270 and a retention section 440 are arranged from the top side to the bottom side of the rear surface portion 400b. The retaining portion 440 is detachably fixed to the rear surface portion 400b with a screw or the like. An optical touch sensor 260 is arranged around the opening of the upper surface portion 400e.

A substantially rectangular bottom reflector 410 is arranged above the bottom surface 400f. The bottom reflector 410 is arranged from the front surface portion 400a to the rear surface portion 400b, and the central portion thereof is curved upward. A substantially rectangular retroreflector 420 is arranged above the bottom reflector 410 . One end of the retroreflective material 420 is fixed to the front side of the upper surface portion 400e, and the other end is fixed to the bottom side of the rear surface portion 400b. there is For example, the retroreflector 420 is arranged at an angle of approximately 45 degrees with respect to the aerial display 250 arranged along the approximately vertical direction. An acrylic plate 430 formed in a substantially rectangular shape is arranged above the retroreflecting material 420 so as to be substantially parallel to the retroreflecting material 420 . One end of the acrylic plate 430 is fixed to the front side of the upper surface portion 400e, and the other end is fixed to the bottom side of the rear surface portion 400b. At this time, the acrylic plate 430 is disposed between the retroreflector 420 and the operation space in the housing, and serves as an auxiliary transmission plate that transmits the optical signal transmitted through the retroreflector 420 toward the operation space. Configured. The acrylic plate 430 is arranged for the purpose of preventing dust from adhering to the retroreflecting material 420 and preventing the finger of the user from contacting the retroreflecting material 420, as well as preventing reflection of illumination.

At this time, the acrylic plate 430 is anti-scratched for the purpose of anti-reflection and prevention of breakage and scratches. Dust or the like that has fallen onto the acrylic plate 430 slides down to the bottom of the acrylic plate 430 and accumulates in the retention portion 440 . The retention part 440 is detachably arranged at a position facing the end (lower part) of the acrylic plate 430 on the outside of the housing, and is configured as a container for storing objects that have fallen on the acrylic plate 430 . Dust and the like accumulated in the retention portion 440 is discharged to the outside of the floating display module 400 by periodically removing part of the retention portion 440 . By periodically performing the work of removing part of the retention portion 440, the inside of the retention portion 440 can be cleaned periodically. Inside the left side portion 400c and the right side portion 400d, side reflectors 450, which are formed in a substantially rectangular shape and reflect light from the operation space, are fixed respectively. Each of the side reflectors 450 arranged on the left and right side surfaces inside the housing is configured as a reflector for widening the viewing angle of the user.

When receiving the image information transferred from the control unit 110 , the aerial display 250 displays an image according to the received image information on the screen, and transmits an optical signal based on the displayed image to the retroreflection material 420 . emit toward At this time, part of the light emitted from the aerial display 250 is directed toward the bottom reflector 410 and reflected by the bottom reflector 410 as indicated by arrow A, and the rest of the light is indicated by arrows B and C. , after being reflected by the retroreflector 420 , the light travels toward the bottom reflector 410 and is reflected by the bottom reflector 410 . The light reflected by the bottom reflector 410 passes through the retroreflector 420 as indicated by arrow D, and forms an aerial image 500 in the operation space above the upper surface 400e. At this time, the aerial image 500 is displayed in the operation space as an image of an operation screen including a plurality of operation buttons.

The user can operate the operation buttons in a non-contact manner while looking at the operation screen and the operation buttons in the aerial image 500 displayed in the operation space. At this time, since the viewing angle with respect to the aerial image 500 displayed in the operation space is restricted by the respective side reflectors 450, it is possible to prevent peeping by persons other than the person himself/herself.

When the user operates the operation button in a non-contact manner, the optical touch sensor 260 at the position corresponding to the operation button detects that the operation button has been operated, and outputs a detection signal to the touchless device control unit 220 and the control board 230 . , to the control unit 110 . The control unit 110 determines the detection signal from the optical touch sensor 260, and processes according to the operation of the operation button. For example, the operation of the operation button instructs automatic cash payment processing or shopping settlement processing. If it is an operation, these processes are executed.

4A and 4B are cross-sectional views of the floating display module according to the first embodiment of the present invention, in which (a) is a cross-sectional view when the bottom reflector is convexly curved, (b) is a cross-sectional view when the bottom reflector is concavely curved; (c) is a cross-sectional view when the bottom reflector is not curved. In FIG. 4A, the central portion of the bottom reflector 410 is curved upward, and the bottom reflector 410 becomes a convexly curved plate, and part of the light from the aerial display 250 is directed to the rear surface 400b. When an aerial image 510 is formed inside and overlapped with the aerial image 500 is displayed in a partial space 400i of the operation space, as shown in FIG. , a concavely curved plate whose center portion is curved toward the bottom surface portion 400f can be used. When a concavely curved plate is used as the bottom reflector 410, part of the light from the aerial display 250 forms an aerial image 520 on the outside of the rear surface portion 400b, and an aerial image 520 overlaps the aerial image 500 in the operation space. Image 520 can be prevented from being displayed. Further, as shown in FIG. 4C, a flat plate shaped like a flat plate can be used as the bottom reflector 410 . As the bottom reflector 410, a flat plate shaped like a flat plate is used, and when this bottom reflector 410 is arranged at a substantially right angle to the aerial display 250, the light from the aerial display 250 is reflected in the operation space. It is possible to prevent the aerial image 500 from being formed in the operation space and overlapping the aerial image 500 from being displayed in the operation space. Even if a convex curved plate is used as the bottom reflector 410, if the aerial image 510 overlapping the aerial image 500 is not displayed in the space 400i, a convex curved plate can be used as the bottom reflector 410. can also be used.

In this embodiment, since the aerial image 500 including the operation screen and the operation buttons is displayed in the operation space, the user can touch the operation buttons in the aerial image 500 without touching the screen or the device. can be operated. When the user operates an operation button in the aerial image 500 displayed in the operation space, the optical touch sensor 260 detects that the operation button has been operated, and the control unit 110 outputs the detection result. transfer to When generating image information about an image of an operation screen including a plurality of operation buttons, control unit 110 associates and manages the position of each operation button belonging to the operation screen and the detection position of optical touch sensor 260 . At this time, when the optical touch sensor 260 detects an object inserted into the operation space, for example, the user's finger, the control unit 110 determines the detection signal from the optical touch sensor 260 and detects the optical touch. Based on the position of the object (finger) detected by the sensor 260, operation information (operation information indicating which operation button has been operated) for identifying the operated operation button among the plurality of operation buttons on the operation screen is generated. , executes processing according to the generated operation information, for example, processing of settlement by shopping. Further, after the control unit 110 transfers the image information to the touchless device control unit 220, the touchless device control unit 220 executes display processing of the aerial image 500 on the aerial display 250, Since the processing of the detection result of touch sensor 260 is executed, control unit 110 can continue information transmission/reception between display 120 and touch panel 130 even when an operation on aerial image 500 is started.

According to this embodiment, it is possible to identify and process information based on a non-contact operation on an aerial image including an operation screen and operation buttons. In addition, according to this embodiment, by mounting the touchless device control BOX 210 and the aerial display unit 240 (aerial display module 400) separately, the installation location of the touchless device control BOX 210 is free, and the user can The outer shape of the aerial display unit 240 (aerial display module 400) to be operated can be reduced. Moreover, when it is difficult to incorporate the touchless device 200 into the small operation terminal 100, the touchless device 200 can be indirectly controlled by the small operation terminal 100 by externally connecting the touchless device 200 to the small operation terminal 100. can do. Furthermore, since the user can perform touchless operations, it is possible to contribute to reducing the spread of viruses.

(2) Example 2
FIG. 5 is a block diagram showing a configuration example of an information processing system according to Embodiment 2 of the present invention. In this embodiment, as shown in FIG. 5, the control unit 110 of the small-sized operation terminal 100 is directly connected to the floating display unit 240 of the touchless device 200, and the touchless device control box 210 is removed from the touchless device 200. , and other configurations are the same as those of the first embodiment shown in FIG.

At this time, the optical touch sensor 260 of the aerial display unit 240 belonging to the touchless device 200 is connected to the control unit 110 of the small operation terminal 100 via the USB interface 113, and the aerial display 250 is connected to the HDMI interface 114. and a speaker 270 is connected via the audio signal interface 115 .

FIG. 6 is a perspective view showing the appearance of an information processing system according to a second embodiment of the invention. In FIG. 6, a display 120 and a touch panel 130 are mounted on a small operation terminal 100, and an aerial display section 240 is connected. The aerial display unit 240 is equipped with an aerial display 250 , an optical touch sensor 260 and a speaker 270 . At this time, the air display unit 240 is directly controlled by the control unit 110 built in the small operation terminal 100 .

According to the present embodiment, it is possible to obtain the same effect as that of the first embodiment, and it is possible to directly control the floating display section 240 of the touchless device 200 by the control section 110 of the small operation terminal 100. 1, the simplification of the configuration can be achieved.

(3) Example 3
FIG. 7 is a block diagram showing a configuration example of a large-sized operation terminal according to Embodiment 3 of the present invention. As shown in FIG. 7, this embodiment integrates the small-sized operation terminal 100 and the touchless device 200 shown in FIG. The functions of each part belonging to the operation terminal 300 are the same as those of the first embodiment shown in FIG. 1, and the same parts are denoted by the same reference numerals, and their explanations are omitted.

At this time, the control unit 310 has the same function as the control unit 110, the display 320 has the same function as the display 120 but has a different shape, and the touch panel 330 has a different shape from the touch panel 130. have the same function.

A display 320 is connected to the control unit 310 via a DVI interface 311 , a touch panel 330 is connected via a USB interface 312 , and a control board 230 is connected via a USB interface 313 . The control board 230 is connected to the touchless device control section 220 via a USB interface 231 and an audio interface 232, is connected to the speaker 270 of the floating display section 240 via an audio signal interface 233, and is connected to a backlight control interface 234. , the aerial display 250 of the aerial display unit 240 is connected. The touchless device control unit 220 is connected to the aerial display 250 of the aerial display unit 240 via the HDMI interface 221, and is connected to the optical touch sensor 260 of the aerial display unit 240 via the USB interface 222. .

8A and 8B are diagrams showing the appearance of a large-sized operation terminal according to the third embodiment of the present invention, in which (a) is an external perspective view of the large-sized operation terminal, and (b) is an enlarged view of the operation portion of the large-sized operation terminal. Department.

In FIG. 8, the large-sized operation terminal 300 is configured as, for example, an automatic teller machine, and an operation panel 340 and an aerial display section 240 are incorporated in an operation section 360 of the large-sized operation terminal 300 . The operation unit 360 is formed in a substantially flat plate shape, and a substantially rectangular display 320 and a touch panel 330 are arranged in a substantially central portion of the operation unit 360 . A touchless device control section 220 and a control board 230 are arranged in a region below the display 320 and the touch panel 330 . In the area on the right side of the display 320 and the touch panel 330, a substantially box-shaped aerial display section 240 is arranged. The aerial display unit 240 is equipped with an aerial display 250 , an optical touch sensor 260 and a speaker 270 .

In the large-sized operation terminal 300 as an automatic teller machine, a user can perform an automatic cash transaction by performing a touchless operation on the touchless device 200 without touching the touch panel 330. .

According to the present embodiment, the same effect as in the first embodiment can be obtained, and the user can perform a touchless operation on the touchless device 200 without touching the touch panel 330. Automated cash transactions can be performed.

(4) Example 4
FIG. 9 is a block diagram showing a configuration example of a large-sized operation terminal according to Embodiment 4 of the present invention. As shown in FIG. 9, this embodiment integrates the small-sized operation terminal 100 and the touchless device 200 shown in FIG. The functions of the units belonging to the operation terminal 300 are the same as those of the second embodiment shown in FIG. 5, and the same components are denoted by the same reference numerals, and description thereof will be omitted.

At this time, the control unit 310 has the same function as the control unit 110, the display 320 has a different shape from the display 120 but has the same function, and the touch panel 330 has a different shape from the touch panel 130. have the same function.

A display 320 is connected to the control unit 310 via a DVI interface 311 and a touch panel 330 is connected via a USB interface 312 . Further, the control unit 310 is connected to the optical touch sensor 260 of the aerial display unit 240 via the USB interface 313, and is connected to the aerial display 250 of the aerial display unit 240 via the HDMI interface 314. A speaker 270 of the floating display section 240 is connected via the signal interface 315 .

In the large-sized operation terminal 300 as an automatic teller machine, a user can perform an automatic cash transaction by performing a touchless operation on the touchless device 200 without touching the touch panel 330. .

According to this embodiment, the same effects as those of the second embodiment can be obtained. Automated cash transactions can be performed.

(5) Example 5
10A and 10B are diagrams for explaining the relationship between the large-sized operation terminal and the visual sense of the user according to the fifth embodiment of the present invention. It is a visual diagram. In FIG. 10, when displaying an aerial image 500 (not shown) in the operation space of the aerial display section 240 of the large-sized operation terminal 300, the surface of the aerial display section 240 (the surface of the aerial image 500) is tilted at an angle of 90°. When the eyes of the user 10 are positioned at the position P1 on the virtual vertical line, no parallax occurs between the aerial image 500 in the aerial display unit 240 and the line of sight of the user 10. Aerial image 500 can be viewed at the selected eye position.

On the other hand, when the eye position of the user 10 is separated leftward from the position P1 to the position P2 by a distance d1, a parallax θ is generated between the aerial image 500 and the line of sight of the user 10 . This parallax θ increases as the distance d1 increases. When a parallax θ occurs between the aerial image 500 and the line of sight of the user 10, the user 10 sees the aerial image 500 and when operating an operation button or the like in the aerial image 500, recognizes the operation button. It may become difficult to operate, and operability may decrease. This is because a deviation occurs between the position of the operation button recognized by the user 10 and the sensor reaction position to which the optical touch sensor 260 reacts.

FIG. 11 is an explanatory diagram for explaining the relationship between the positions of the operation buttons and the sensor reaction positions in the large-sized operation terminal according to the fifth embodiment of the present invention. In FIG. 11, the sensor reaction position of the optical touch sensor 260 exists on the straight line H1, and the position of the operation button recognized by the user is determined by the parallax θ generated between the aerial image 500 and the user's line of sight. When the position is on the straight line H2 above the straight line H1, the user moves the finger 12 at the penetration angle α with respect to the straight line H1, which is the sensor reaction position, in order to operate the portion of the operation button at the position V1. When the fingertip 12 is inserted into the operation space in the direction D1, the optical touch sensor 260 reacts when the fingertip 12 reaches the position V2 separated from the position V1 by ΔV. When a parallax θ occurs between the aerial image 500 and the line of sight of the user, the intrusion angle α becomes narrower, and a deviation occurs between the position of the operation button recognized by the user and the actual position of the operation button. The operability when the operator operates the operation buttons is deteriorated.

FIG. 12 is an enlarged configuration diagram of the floating display section 240 in the large-sized operation terminal according to the fifth embodiment of the present invention. In FIG. 12, the aerial display unit 240 is provided with a substantially rectangular displayable display area R1 having an area corresponding to the entire display area. A substantially rectangular aerial image displayable region R2 and a substantially rectangular operation button displayable region R3 are set in the middle of the displayable displayable region R1. The size of each region is set to have a relationship of R1>R2>R3.

At this time, the display area of the aerial image 500 when the image by the optical signal emitted from the aerial display unit 240 is formed as the aerial image 500 in the operation space in the aerial display module 400, and the display displayable area R1. is set to, for example, a one-to-one relationship. That is, the size of the image displayed on the aerial display unit 240 and the size of the aerial image 500 displayed in the operation space within the aerial display module 400 are set to have a one-to-one relationship.

Here, in this embodiment, considering the parallax θ occurring between the aerial image 500 and the line of sight of the user, the upper side of the aerial image displayable region R2 is located farther than the upper side of the operation button displayable region R3. The lower side of the aerial image displayable area R2 is enlarged lower than the lower side of the operation button displayable area R3 by a distance L2, and the right side of the aerial image displayable area R2 is the operation button. The displayable area R3 is enlarged to the right by a distance L3 from the right side of the displayable area R3. Furthermore, considering the parallax θ occurring between the aerial image 500 and the user's line of sight, the aerial image displayable region R2 is positioned relative to the virtual center line c1 passing through the center of the display displayable region R1. An imaginary center line c2 passing through the center is offset to the left by a distance s1, and an imaginary center line c3 passing through the center of the operation button displayable region R3 is offset to the left by a distance s2.

At this time, the control unit 110 sets the array range of the operation buttons belonging to the aerial image 500 displayed in the operation space to a range narrower than the sensor reaction range of the optical touch sensor 260, and displays the image including the operation screen and the operation buttons. The center of an aerial image displayable region R2 in which an aerial image can be displayed and the operation button displayable region R3 in which the operation buttons are arranged with respect to the center of the display displayable region R1 that is the display region of the entire operation space. are each offset to the left to generate the image.

According to this embodiment, the range (aerial image displayable region R2) expanded from the array range (operation button displayable region R3) of the operation buttons arranged in the aerial image 500 is detected by the sensor reaction of the optical sensor 260. In addition, by offsetting the center of the aerial image displayable region R2 and the operation button displayable region R3 to the left with respect to the center of the display displayable region R1, the aerial image 500 and the line of sight of the user are offset. Even if a parallax θ occurs between the two, it is possible to prevent the operation button from becoming difficult for the user to recognize when operating the operation button, and as a result, it is possible to improve the operability. The configuration of the aerial display unit 240 in this embodiment can be applied to those in the first to fourth embodiments.

FIG. 13 is a diagram showing a display example of an aerial image applied to Examples 1 to 5 of the present invention. In FIG. 13, aerial images 500a to 500e are sequentially displayed in the operation space of the aerial display unit 240 according to the user's operation. Each of the aerial images 500a to 500e displays, for example, information such as minimum characters and operation buttons necessary for operating the automatic cash transaction. Therefore, the user can smoothly carry out the operations related to the automatic cash transaction while viewing the aerial images 500a to 500e that are sequentially switched by the operation.

(6) Other Embodiments FIGS. 14 and 15 show configuration examples of an aerial display module according to a sixth embodiment of the present invention.

FIG. 14 shows an example of measures for vertically enlarging and displaying the aerial image 500 . As shown in FIG. 14(a), the longitudinal direction of the bottom reflector 410 is convexly curved from a state parallel or nearly parallel to the aerial image 500 toward the retroreflector 420 as shown in FIG. 14(b). This makes it possible to enlarge the aerial image 500 in the vertical direction.

Also, FIG. 15 shows an example of measures for enlarging and displaying the aerial image 500 in the horizontal direction. As shown in FIG. 15(a), the width direction of the bottom reflector 410 is convexly curved in the direction toward the retroreflector 420 as shown in FIG. By doing so, the aerial image 500 can be enlarged in the horizontal direction.

FIGS. 16A to 16D show screen changes when an operation button is touched on a display example of an aerial image 500 applied to Examples 1 to 6 of the present invention. When a button displayed in the aerial image 500 is touch-operated, the display color of the button is reversed, the button size of the button is enlarged, or an operation sound is generated as shown in FIGS. By ringing , it is possible to make it easier for the user to understand that the button has been touch-operated.

The optical touch sensor 260 included in the configurations of Examples 1 to 6 is a method of detecting reflection by a finger and the coordinates of the reflection by equipping a reflective sensor on either the left or right side of the frame of the opening. Alternatively, light emitting units and light receiving units of light receiving and emitting sensors are provided on the left, right, and in the vertical and horizontal directions within the frame of the opening, and are realized by a method of detecting the obstruction by the finger and the reflected coordinates.

It should be noted that the present invention is not limited to the embodiments described above, but includes various modifications and equivalent configurations within the scope of the appended claims. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and the present invention is not necessarily limited to those having all the described configurations.

In addition, each configuration, function, etc. described above may be realized by hardware, for example, by designing a part or all of them with an integrated circuit, and a processor interprets and executes a program that realizes each function. You may implement|achieve by software by doing.

Information such as programs, tables, files, etc. that realize each function is stored in storage devices such as memory, hard disk, SSD (Solid State Drive), or IC (Integrated Circuit) card, SD card, DVD (Digital Versatile Disc) recording Can be stored on media.

DESCRIPTION OF SYMBOLS 100... Small operation terminal, 110... Control part, 120... Display, 130... Touch panel, 140... Operation panel, 200... Touchless apparatus, 210... Touchless apparatus control BOX 220 Touchless device control unit 230 Control board 240 Air display unit 250 Air display 260 Optical touch sensor 270 Speaker 300 Large operation terminal 310 Large operation terminal control unit 320 Display 330 Touch panel 400 Mid-air display module 410 Bottom reflector 420 Retroreflecting material 430 Acrylic plate 440 Retention part 450 Side reflecting material 500 Aerial image

Claims (11)

a controller that generates image information about an image of an operation screen including a plurality of operation buttons;
a housing configured in a substantially box shape and having an operation space formed by an opening in a part of the housing;
a display arranged in the housing for inputting the image information from the controller, displaying an image according to the input image information on a screen, and emitting an optical signal based on the image into the housing;
The light signal is incident from the display arranged in the housing, the incident light signal is transmitted toward the operation space, and an image formed by the transmitted light signal is formed as an aerial image in the operation space. a retroreflective material that causes
Light is emitted into the operation space, the reflected light reflected by an object existing in the operation space is incident, and the position of the object existing within the operation space is detected based on the incident reflected light. an optical sensor for
The controller is
An information processing system, comprising: generating operation information specifying an operated operation button among the plurality of operation buttons based on a detection result of the optical sensor; and executing processing according to the generated operation information. The information processing system according to claim 1,
The controller is
generating the image by setting the array range of the operation buttons belonging to the aerial image displayed in the operation space to a range narrower than the sensor reaction range of the optical sensor, and displaying a display area showing the entire operation space; The image is generated by offsetting the centers of an aerial image displayable area in which the aerial image can be displayed and an operation button displayable area in which the operation buttons are arranged to the left of the center of the possible area. information processing system. The information processing system according to claim 1,
Further comprising a bottom reflector that is arranged on the bottom side of the housing and reflects light from the display and light from the retroreflector toward the retroreflector,
The retroreflective material is
arranged obliquely with respect to the screen of the display and arranged above the bottom reflector;
The bottom reflector is
Consists of a convexly curved plate whose central portion is curved upward, a concavely curved plate whose central portion is curved toward the bottom, or a flat plate whose entirety is flat. An information processing system characterized by a monkey. The information processing system according to claim 3,
An information processing system, wherein side reflectors for reflecting light from the operation space are arranged on left and right side surfaces inside the housing. The information processing system according to claim 1,
an auxiliary transmitting plate disposed between the retroreflecting material and the operation space in the housing and transmitting the optical signal transmitted through the retroreflecting material toward the operation space;
a retention section detachably disposed at a position facing the end of the auxiliary transmission plate outside the housing for storing fallen objects that have fallen onto the auxiliary transmission plate;
The retroreflective material is
arranged obliquely with respect to the screen of the display,
The auxiliary transmission plate is
An information processing system, wherein the end portion is positioned downward and substantially parallel to the retroreflective material. The information processing system according to claim 1,
an operation panel for displaying, on a display screen, an image based on the image information generated by the controller, and for transferring touch operation information corresponding to a touch operation on the display screen to the controller;
an auxiliary controller that transmits and receives information to and from the controller, the display, and the optical sensor;
The controller is
transferring the image information to the auxiliary controller in response to the touch operation information;
The auxiliary controller is
When the image information is received from the controller, the received image information is transferred to the display, and when the detection result is received from the optical sensor, the received detection result is transferred to the controller. information processing system. The information processing system according to claim 1,
an auxiliary controller that transmits and receives information to and from the controller, the display, and the optical sensor;
a speaker arranged in the housing and connected to the auxiliary controller;
The auxiliary controller is
When the image information is received from the controller, the received image information is transferred to the display, and when the detection result is received from the optical sensor, an audio signal indicating a touch operation sound based on the received detection result. and causing the speaker to sound with the generated audio signal. The information processing system according to claim 7,
a control box on which the auxiliary controller is mounted;
further comprising a display module equipped with the speaker, the display, and the optical sensor;
An information processing system, wherein the control box and the display module are mounted separately from each other. The information processing system according to claim 3,
A touchless device, wherein the longitudinal direction of the bottom reflector is convexly curved in a direction toward the retroreflector with respect to a state parallel or substantially parallel to the aerial image. The information processing system according to claim 3,
A touchless device, wherein the width direction of the bottom reflector is convexly curved in a direction toward the retroreflector with respect to a state parallel or substantially parallel to the aerial image. The information processing system according to claim 1,
A touchless device characterized in that when a button displayed in an aerial image is touch-operated, the display color of the button is reversed, the button size of the button is enlarged, and an operation sound is generated.

Images