WO2023079879A1

WO2023079879A1 - Aerial operation device and start-up method

Info

Publication number: WO2023079879A1
Application number: PCT/JP2022/036830
Authority: WO
Inventors: 仁中山; 正孝浜野
Original assignee: 株式会社村上開明堂
Priority date: 2021-11-04
Filing date: 2022-09-30
Publication date: 2023-05-11
Also published as: JPWO2023079879A1

Abstract

According to an embodiment of the present invention, an aerial operation device includes a display device radiating light to display information, an aerial image forming device reflecting light from the display device a plurality of times to form an aerial image and displaying an operation screen with buttons, and a sensor detecting an object F approaching any of the buttons. The display device has a sleep mode in which the operation screen is not displayed via the aerial image forming device and an operable mode in which the operation screen is displayed via the aerial image forming device. When the sensor detects the object, the display device switches from the sleep mode to the operable mode to display the operation screen in the air.

Description

Aerial operation device and start-up method

FIELD OF THE DISCLOSURE The present disclosure relates to aerial manipulators and launching methods.
This application claims priority based on Japanese Application No. 2021-180315 dated November 4, 2021, and incorporates all the content described in the Japanese Application.

Japanese Unexamined Patent Application Publication No. 2016-95635 describes an aerial touch panel for a surgical simulator display system. The aerial touch panel includes a display section, an aerial imaging panel as an imaging section, and an optical position input section. When the operator of the surgical simulator holds a hand over the operation screen displayed in the air, the position input unit calculates the positional relationship between the position of the hand and the operation screen. The position input unit transmits an operation input signal to the control unit when it is determined that the operator's hand is at the position of the operation screen. This allows the operator to operate the surgical simulator image without touching anything.

A settlement terminal device is described in JP-A-2016-51321. The settlement terminal device has a main body and a lid, and a touch panel is provided on the upper surface of the lid. When any screen relating to payment processing is displayed on the touch panel, the payment terminal device detects an input operation on the screen. If no input operation is detected on the screen for a certain period of time or longer, the touch panel shifts to sleep mode. The settlement terminal device cancels the sleep mode when the input operation is detected after shifting to the sleep mode.

Japanese Patent Application Laid-Open No. 2021-103493 describes an information processing system. The information processing system has an aerial imaging device, a control device for controlling a user's personal computer, and a sensor for detecting the position of an object such as a hand in the space where the aerial image is displayed. The sensor has first and second sensing areas, and the aerial imaging device forms first and second aerial images corresponding to the first and second sensing areas, respectively. When an object touches the first aerial image, the sensor detects the object reaching the first detection area and the controller puts the personal computer into sleep mode. When the object touches the first aerial image and then the second aerial image, the sensor detects the object reaching the second detection area, and the control device restarts the personal computer.

Japanese Patent Application Laid-Open No. 2019-128725 describes an aerial image forming system that enters a sleep state. An aerial image forming system includes an aerial image forming device that forms an aerial image, a human detection sensor that forms a detection plane including a position where the aerial image is to be formed, and a human detection sensor that detects the position of the person. and an image controller for determining the position for forming the aerial image. The image controller determines whether to form an aerial image. When the aerial imaging device forms an aerial image, the aerial imaging system wakes from sleep.

Japanese Patent Application Laid-Open No. 2019-197251 describes a display system equipped with a head-mounted display (HMD). The HMD includes a connection device and an image display section. The image display unit is connected to each of the personal computer, the image output device, and the power supply device via the connection device. The connection device has a control section and an operation section. The control unit has a sleep function for stopping the display of the image on the image display unit. When the sleep function is disabled, the HMD remains in normal operating mode.

JP 2016-95635 A JP 2016-51321 A JP 2021-103493 A JP 2019-128725 A JP 2019-197251 A

Each system described above has a sleep mode, and returns from sleep mode to operable mode. However, in an aerial operation device that forms an aerial image of an operation screen including switches in the air and operates a device on the operation screen, there is no established method for recovering from the sleep mode in an easy-to-understand manner. Therefore, an in-flight operation device that forms an image of an operation screen in the air is required to return from the sleep mode in an easy-to-understand manner.

An object of the present disclosure is to provide an in-flight operation device and a start-up method that can be recovered from sleep mode in an easy-to-understand manner.

An aerial operation device according to the present disclosure includes a display device that emits light to display information, an aerial imaging device that displays an operation screen with buttons as an aerial image by reflecting light from the display device multiple times, a sensor for detecting an object approaching the button. The display device has a sleep mode in which the operation screen is not displayed via the aerial imaging device, and an operable mode in which the operation screen is displayed via the aerial imaging device. The display device switches from the sleep mode to the operable mode and displays the operation screen in the air when the sensor detects the object.

In this aerial operation device, the display device emits light, and the aerial imaging device reflects the light from the display device multiple times to display an operation screen with buttons as an aerial image. The air manipulator has a sensor that detects an object approaching the button. The display device has a sleep mode in which the operation screen is not displayed and an operable mode in which the operation screen is displayed. The display device switches from the sleep mode to the operable mode and displays the operation screen in the air when the sensor detects the object. By displaying the operation screen in the air when an object is detected, the operation screen can be displayed in an easy-to-understand manner when returning from the sleep mode to the operable mode. That is, since the operation screen as an aerial image is displayed at a position on the nearer side than the aerial imaging device, the operation screen can be displayed in an easy-to-understand manner when the sleep mode is canceled. Furthermore, since it is not necessary to touch the device when canceling the sleep mode, the aerial operation device can be excellent in terms of hygiene.

The display device may display a sleep mode release button in the air when it is in sleep mode. In this case, the user can recognize that it is in sleep mode, and the sleep mode can be easily released by pressing the sleep mode release button.

The air control device may be equipped with multiple sensors. In this case, the display device displays the operation screen in the air when at least one of the plurality of sensors detects the object. Therefore, when any one of the plurality of sensors detects an object, the mode shifts to the operable mode and the operation screen is displayed in the air. Therefore, it is possible to return from the sleep mode in an easy-to-understand manner.

The display device may switch from the sleep mode to the operable mode and display the operation screen in the air when the plurality of sensors detect a predetermined motion of the target object. In this case, the sleep mode is released when the plurality of sensors detect a predetermined motion of the object. Since the sleep mode is not canceled unless the object performs a predetermined action, unintentional display of the operation screen is suppressed. Therefore, the security of the air control device can be enhanced.

A start-up method according to the present disclosure includes a display device that irradiates light to display information, an aerial imaging device that reflects light from the display device multiple times to display an operation screen with buttons as an aerial image, and a sensor for detecting an object approaching a button. The start-up method includes a step in which the display device enters a sleep mode in which the operation screen is not displayed via the aerial imaging device, and a step in which the display device switches from the sleep mode to the operable mode when the sensor detects an object. and displaying the operation screen in the air via the aerial imaging device.

In this startup method, an operation screen with buttons is displayed in the air as an aerial image by an aerial imaging device, and an object approaching the button trying to operate the button is detected by the sensor. The display device has a sleep mode in which the operation screen is not displayed and an operable mode in which the operation screen is displayed. When the sensor detects the object, the display device switches from the sleep mode to the operable mode, and the display device displays the operation screen in the air. Therefore, since the operation screen is displayed in the air when the object is detected, the operation screen can be displayed in an easy-to-understand manner when the sleep mode is switched to the operable mode. That is, since the operation screen as an aerial image is displayed at a position on the nearer side than the aerial imaging device, the operation screen can be displayed in an easy-to-understand manner when the sleep mode is canceled. Furthermore, since it is not necessary to touch the device when canceling the sleep mode, the start-up method can be excellent from a hygienic point of view.

According to the present disclosure, it is possible to return from sleep mode in an easy-to-understand manner.

1 is a perspective view showing an application example of an aerial operation device according to an embodiment; FIG. FIG. 4 is a perspective view showing a state in which the aerial operating device according to the embodiment displays a display surface with buttons; 1 is a vertical cross-sectional view showing an aerial operating device according to an embodiment; FIG. 3 is a block diagram showing functions of the air control device according to the embodiment; FIG. FIG. 4 is a diagram schematically showing a sensor and a press determination surface according to the embodiment; FIG. 4 is a diagram schematically showing an example of detection of an object by a sensor; 4 is a flow chart showing an example of steps of a start-up method according to an embodiment; FIG. 11 is a perspective view showing an aerial operation device according to a modification; FIG. 11 is a vertical cross-sectional view showing an aerial operating device according to a modification; FIG. 11 is a diagram schematically showing a sleep mode release button displayed by an air control device according to a modification;

Below, embodiments of the aerial operation device and the start-up method according to the present disclosure will be described with reference to the drawings. In the description of the drawings, the same reference numerals are given to the same or corresponding elements, and overlapping descriptions are omitted as appropriate. The drawings may be partially simplified or exaggerated for easy understanding, and the dimensional ratios and the like are not limited to those described in the drawings.

FIG. 1 is a perspective view showing an aerial operation device 1 as an example according to this embodiment. FIG. 2 is a perspective view showing a state in which the air control device 1 displays an operation screen 10 with buttons 11 in the air. For example, the aerial operation device 1 according to this embodiment is an operation input device for operating equipment.

The aerial operation device 1 is provided in the toilet T as an example, and the button 11 of the operation screen 10 is a switch for operating each part of the toilet T. An operation screen 10 with buttons 11 is a floating image displayed in the air. That is, the operation screen 10 with the button 11 is displayed as an aerial image. Therefore, the user of the aerial operation device 1 does not have to directly touch the buttons and the like, so that each part of the toilet T can be operated hygienically.

For example, the toilet T includes a toilet bowl B and a wall portion W adjacent to the toilet bowl B. For example, the wall portion W is provided on the right side as viewed from the user seated on the toilet bowl B. As shown in FIG. An exemplary wall portion W has the aerial operation device 1 and toilet paper E fixed thereto, and a washbasin M may be provided. The toilet bowl B has, for example, a washlet (registered trademark), and the washlet of the toilet T can be operated from the button 11 on the operation screen 10 . For example, the operation screen 10 has a plurality of buttons 11, and the user can operate each part of the washlet by operating the plurality of buttons 11. FIG.

The aerial operation device 1 is fixed to the wall W, for example. The aerial operation device 1 may be embedded in the wall W. Further, the aerial operation device 1 may be detachable from the wall W. An exemplary aerial operation device 1 includes a housing 2 and a protrusion 3 protruding from the bottom of the housing 2 . A plurality of sensors 12 for detecting an object F approaching the button 11 are embedded in the projecting portion 3 . In the present disclosure, "the sensor detects an object" means that the sensor irradiates the object with light and receives the light reflected from the object along with the irradiation of the light. .

In the present disclosure, "object" indicates an object that operates the button 11 to operate the device. For example, the "object" is the user's finger. However, the object is not limited to the finger, and may be a stick-shaped object such as a pen or something other than a stick-shaped object. The sensors 12 are provided corresponding to the buttons 11 , and each button 11 is displayed obliquely above each sensor 12 . As an example, the number of buttons 11 and sensors 12 is seven.

The operation screen 10 is displayed so as to be oriented obliquely upward in both the first direction D1 and the third direction D3. In this embodiment, the first direction D1 is the vertical direction of the aerial operation device 1, and the third direction D3 is the direction of projection from the wall portion W. As shown in FIG. By displaying the operation screen 10 so as to face diagonally upward, it is possible to make it easier for the user to visually recognize the operation screen 10 . For example, since the operation screen 10 is illuminated, the operation screen 10 can be easily viewed regardless of the brightness of the surroundings. The operation screen 10 has, for example, a shape elongated in the second direction D2. The second direction D2 is a direction that intersects (perpendicularly as an example) both the first direction D1 and the third direction D3.

As an example, the operation screen 10 is displayed in a rectangular shape. For example, the vertical length L1 of the operation screen 10 is shorter than the horizontal length L2 of the operation screen 10 . Assuming that the aspect ratio of length L1 and length L2 is 1:X (where X is a real number), the value of X is 2 or more and 10 or less, for example. The lower limit of the value of X may be three. The upper limit of the value of X may be 5, 6, 7, 8 or 9.

As an example, the above value of X may be 4 and the aspect ratio of length L1 and length L2 may be 1:4. The button 11 has, for example, a rectangular shape. As an example, button 11 may be square-shaped. The length of one side of the button 11 is, for example, 18 mm or more and 24 mm or less. However, the shape and size of the button 11 can be changed as appropriate.

The operation screen 10 may display items other than the plurality of buttons 11. For example, the operation screen 10 may include multiple buttons 11 and character information 13 . The character information 13 may include, for example, at least one of date information, time information, temperature information, temperature information, and atmospheric pressure information.

On the operation screen 10 , for example, a plurality of buttons 11 may be displayed horizontally on the lower side of the operation screen 10 and text information 13 may be displayed above the plurality of buttons 11 . The button 11 may be arranged so that the center 11b of the operation screen 10 is shifted from the center 10b. In the present embodiment, as an example, each of the plurality of buttons 11 is displayed at a position offset from a reference line V passing through the center 10b and extending in the longitudinal direction (second direction D2) of the operation screen 10. FIG.

FIG. 3 is a cross-sectional view schematically showing the internal structure of the aerial operation device 1. As shown in FIG. As shown in FIGS. 2 and 3, for example, the housing 2 has a box shape extending horizontally. As an example, the housing 2 has a front surface 2b, a pair of side surfaces 2c, an upper surface 2d, and a lower surface 2f. The front surface 2b extends in the first direction D1 and the second direction D2 when attached to the wall portion W. As shown in FIG. The side surface 2c extends in the first direction D1 and the third direction D3. The upper surface 2d extends in the second direction D2 and the third direction D3. The lower surface 2f faces the opposite side of the upper surface 2d.

The projecting portion 3 has, for example, a rectangular plate shape. As an example, the protruding portion 3 includes a main surface 3b that protrudes from the lower side of the front surface 2b and faces upward, an end surface 3c that faces the second direction D2 or the third direction D3, and a back surface that faces the opposite side of the main surface 3b. 3d. As described above, the sensor 12 is built in the projecting portion 3 . This makes it easier for the sensor 12 to detect the object F.

The aerial operation device 1 includes an aerial imaging device 14 that displays an operation screen 10 with buttons 11 , a display device 15 arranged inside the housing 2 , and a control section 20 . The aerial imaging device 14 and the display device 15 correspond to an aerial image display unit that displays the operation screen 10 as an aerial image in the air. The aerial imaging device 14 is, for example, a retroreflecting member (retroreflecting mirror) fixed to an opening 2g located inside the front surface 2b of the housing 2 .

The opening 2g is defined by, for example, a pair of first plate portions 2h forming the front surface 2b of the housing 2 and a pair of second plate portions 2j fixed to the inside of the housing 2 of the first plate portions 2h. be done. As an example, the first plate portion 2h and the second plate portion 2j are provided as a pair of upper and lower plates. The aerial imaging device 14 is fixed to the housing 2 while being sandwiched between the first plate portion 2h and the second plate portion 2j.

The aerial imaging device 14 has, for example, a rectangular shape with long sides extending in the second direction D2 and short sides extending in the first direction D1. Each of one long side and the other long side of the aerial imaging device 14 is sandwiched between the first plate portion 2h and the second plate portion 2j. The display device 15 is arranged obliquely with respect to the aerial imaging device 14 .

As an example, the display device 15 is a liquid crystal display (LCD). The display device 15 is arranged diagonally above the aerial imaging device 14 inside the housing 2 . The display device 15 has a screen for displaying images. The screen of the display device 15 irradiates the light C as an image obliquely downward toward the aerial imaging device 14, for example. The aerial image forming device 14 internally reflects the light C from the display device 15 a plurality of times (for example, twice), and the user of the aerial operation device 1 manipulates it into a space on the front side of the aerial image forming device 14 . A screen 10 is imaged.

The sensor 12 may be exposed on the main surface 3b of the projecting portion 3, for example. As an example, sensor 12 is a depth sensor. For example, the sensor 12 is provided on a virtual straight line extending from the button 11, that is, at a front position with respect to the button 11, which is an aerial image. The sensor 12 acquires distance image data including information on the position (two-dimensional position) of the object F on a plane perpendicular to the virtual straight line and information on the distance K from the sensor 12 to the object F. For example, the sensor 12 outputs the acquired distance image data to the control unit 20 at a predetermined cycle (for example, 1/30 second).

As a specific example, the sensor 12 irradiates light rays (or infrared rays) to each point on an object existing within an imaging area including the object F, and receives light rays reflected from each point on the object. Then, the sensor 12 measures the distance between the sensor 12 and each point on the object based on the received light beam, and outputs the measured distance for each pixel. The distance between the sensor 12 and each point on the object may be measured by, for example, the Light Coding method.

In the light coding method, the sensor 12 irradiates each point on an object existing within an imaging area including the object F with a light beam in a random dot pattern. Sensor 12 then measures the distance between sensor 12 and each point on the object by receiving rays reflected from each point on the object and detecting distortions in the pattern of the reflected rays. The sensor 12 detects two-dimensional position information of each point on the object and distance information from the sensor 12 to each point on the object as a plurality of pixels, and outputs the detected plurality of pixels to the control unit 20. do.

The control unit 20 can communicate with each of the sensor 12 and the display device 15. The control unit 20 includes, for example, a CPU (Central Processing Unit) that executes programs, a storage unit including ROM (Read Only Memory) and RAM (Random Access Memory), an input/output unit, and a driver. Each function of the control unit 20 is realized by operating the input/output unit under the control of the CPU to read or write data in the storage unit. The form and location of the control unit 20 are not particularly limited.

4 is a functional block diagram of the control unit 20. FIG. As shown in FIGS. 3 and 4, the control unit 20 includes, as functional components, an image output unit 21, an object detection unit 22, a recognition unit 23, a signal output unit 24, and a notification unit 25. have The image output unit 21 outputs to the display device 15 a control signal for image data of an image to be displayed on the display device 15 . The display device 15 can display various types of images based on image data control signals from the image output unit 21 .

The object detection unit 22 detects the object F based on the distance image data output from the sensor 12 . "The object detection unit detects the object" means that the distance image data is output from the sensor to the control unit as the sensor detects the object, and the object detection unit receives the distance image data. It indicates that an object is detected. When the target object F is detected, the target object detection unit 22 outputs position data indicating the position of the target object F to the recognition unit 23 . The recognition unit 23 recognizes that the button 11 has been pressed by the object F based on the position data output from the object detection unit 22 .

The notification unit 25 receives an operation signal from the recognition unit 23 when the recognition unit 23 recognizes that the button 11 has been pressed. The notification unit 25 is notification means for notifying the user of the air control device 1 that the button 11 has been operated when the recognition unit 23 recognizes that the button 11 has been operated. The notification unit 25 has, for example, an audio output unit 25b and a color change unit 25c.

As an example, the audio output unit 25b is a speaker, and outputs audio when an operation signal is received from the recognition unit 23. The user can understand that the button 11 has been operated by listening to the voice from the voice output unit 25b. The color changer 25 c generates a color change signal and outputs the color change signal to the display device 15 when receiving an operation signal from the recognition unit 23 , for example.

When the display device 15 receives the color change signal from the color change unit 25c, it changes the color of the button 11 pressed by the user, for example. By visually recognizing that the color of the button 11 has been changed, the user can recognize that the button 11 has been operated. Note that the display device 15 may change the color of portions other than the button 11 when receiving the color change signal from the color change section 25c.

By providing the notification unit 25 as described above, the user can understand that the button 11 has been operated. However, in the notification unit 25, at least one of the audio output unit 25b and the color change unit 25c may be omitted. The notification unit 25 may notify the user that the button 11 has been operated in a manner different from the voice output by the voice output unit 25b or the color change by the color change unit 25c.

The signal output unit 24 generates a control signal based on the pressing operation of the button 11 when the recognition unit 23 recognizes that the button 11 has been pressed. The signal output unit 24 transmits the generated control signal to the device 30 external to the air control device 1 , and transmits the control signal to the device 30 to operate the device 30 .

The device 30 may be, for example, a toilet bowl B with a washlet (see FIG. 1), each part of the washlet, or a washstand M. As an example, the device 30 may be a toilet bowl B, and the toilet bowl B may operate to clean the toilet bowl when receiving a control signal from the signal output unit 24 . Thus, in the exemplary aerial operation device 1, each part of the toilet T can be operated by operating the button 11 displayed as an aerial image. Therefore, since it is not necessary to directly (physically) press a button or the like for the operation of each part of the toilet T, the operation can be realized in a sanitary manner.

For example, the recognition unit 23 determines whether the distance K between the sensor 12 and the object F is equal to or less than the threshold Y1, and whether the distance K between the sensor 12 and the object F is equal to or less than the threshold Y2 which is smaller than the threshold Y1. or may be determined. In this case, the recognition unit 23 determines whether or not the object F has reached the first surface Z1 whose distance K from the sensor 12 is the threshold Y1, and whether or not the object F has reached the second surface Z2 whose distance K from the sensor 12 is the threshold Y2. It is determined whether or not the object F has reached the

Each of the first surface Z1 and the second surface Z2 is a virtual surface formed at a site at a constant distance from the sensor 12, and is a press determination surface for determining whether the button 11 has been pressed. As shown in FIGS. 3 and 5, each of the first surface Z1 and the second surface Z2 has, for example, a circular shape. Each of the first surface Z1 and the second surface Z2 is provided at a position near the button 11, for example. At least one of the first surface Z1 and the second surface Z2 may match the position of the button 11, or both the first surface Z1 and the second surface Z2 may not match the position of the button 11. good. The threshold value Y1 may be the same value as the threshold value Y2, and the first surface Z1 and the second surface Z2 may be the same surface. An example in which the threshold value Y1 is the same as the threshold value Y2 and the first surface Z1 and the second surface Z2 are the same press determination surface Z will be described below.

For example, in the plurality of sensors 12, the values of the threshold values Y are the same. However, the values of the threshold Y may differ from each other in the plurality of sensors 12 . For example, the threshold Y value for the sensor 12 corresponding to a particular button 11 may be smaller than the threshold Y value for the sensor 12 corresponding to another button 11 . In this case, the specific button 11 cannot be pressed unless the object F is moved closer to the sensor 12 than the other buttons 11 . Therefore, the possibility of erroneously operating the specific button 11 can be reduced. The specific button includes, for example, a button used in an emergency (emergency button).

Also, the value of the threshold Y in the sensor 12 corresponding to a specific button may be greater than the value of the threshold Y in the sensor 12 corresponding to another button 11. In this case, the specific button 11 can be pressed even if the object F is farther from the sensor 12 than the other buttons 11 . Therefore, the specific button 11 can be made easier to operate than the other buttons 11 .

As shown in FIGS. 3 and 4, the display device 15 includes a mode switching section 16 in this embodiment. The display device 15 has a sleep mode in which the operation screen 10 is not displayed and an operable mode in which the operation screen 10 is displayed. The mode switching unit 16 switches between sleep mode and operable mode.

For example, the mode switching unit 16 may switch to sleep mode when power is supplied to the air control device 1 (or when the power is turned on). Also, the mode switching unit 16 may set the operable mode when power is supplied to the air control device 1 . In the sleep mode, the light C may not be emitted from the display device 15 and nothing may be displayed. Also, in the sleep mode, the display device 15 may display only the sleep mode. Since the operation screen 10 is not displayed in the sleep mode, it is impossible to operate the equipment 30 with the aerial operation device 1 .

As an example in which only the sleep mode is displayed, for example, as shown in FIG. 10, the display device 15 may display only the sleep mode release button 10A in the air. The sleep mode release button 10A is a floating image displayed in the air. That is, the sleep mode release button 10A is displayed as an aerial image by the aerial imaging device 14, like the operation screen 10. FIG.

As an example, the display device 15 displays the sleep mode release button 10A as a rectangular area. For example, the area of the sleep mode release button 10A is smaller than the area of the operation screen 10 . However, the display mode of the sleep mode release button 10A can be changed as appropriate. The sleep mode release button 10A may be displayed as an aerial image that moves in the air.

The sleep mode is released by pressing the sleep mode release button 10A. Specifically, when the object F touches the sleep mode release button 10A and the sensor 12 detects the object F, the mode switching unit 16 switches from the sleep mode to the operable mode. By switching to the operable mode, the display device 15 displays the operation screen 10 via the aerial imaging device 14 .

When the button 11 is not pressed even after a certain period of time (for example, 3 minutes or more and 5 minutes or less) has passed while the display device 15 is in the operable mode (displaying the operation screen 10), the mode switching unit 16 You can switch to sleep mode. The mode switching unit 16 may switch to the operable mode when the sensor 12 detects the object F positioned on the depression determination surface Z while the display device 15 is in the sleep mode.

For example, when the object F reaches the press determination surface Z in the sleep mode, the sensor 12 outputs distance image data to the control unit 20, and the object detection unit 22 detects the object F. The object detection unit 22 outputs the position data of the object F to the recognition unit 23, and the recognition unit 23 detects that the object F has reached the depression determination surface Z based on the position data output from the object detection unit 22. to recognize

The image output unit 21 generates a control signal when the recognition unit 23 recognizes that the object F has reached the depression determination surface Z, and transmits the generated control signal to the display device 15 . When the display device 15 receives the control signal from the image output unit 21, the mode switching unit 16 switches from the sleep mode to the operable mode, and the display device 15 displays the operation screen 10 in the air. In this case, the user of the air control device 1 can return from the sleep mode to the operable mode by bringing the object F into contact with the depression determination surface Z. FIG.

However, the manner in which the sleep mode is returned to the operable mode is not limited to the manner described above. For example, as shown in the modified example of FIG. 6, the mode switching unit 16 may switch from the sleep mode to the operable mode when the plurality of sensors 12 detect the target F. As an example, the mode switching unit 16 may switch from the sleep mode to the operable mode when the two sensors 12 arranged along the second direction D2 detect the target F.

In the example of FIG. 6, one of the two sensors 12 detects the object F approaching the sensor 12, and the other of the two sensors 12 detects the object F moving along the second direction D2. When detected, the mode switching unit 16 switches from the sleep mode to the operable mode. In this case, by bringing the object F closer to the sensor 12 and moving the object F along the second direction D2, the sleep mode can be returned to the operable mode.

A modified example of the method for returning from sleep mode to operable mode has been described above. In this method, since the sleep mode is not released unless a special operation of moving the object F in the second direction D2 after the object F approaches the sensor 12, security can be improved.

However, the method of canceling sleep mode is not limited to the above example. For example, after the sensor 12 detects an object F approaching the sensor 12, the mode switching unit 16 may switch from the sleep mode to the operable mode when the sensor 12 detects an object F leaving the sensor 12. . Various methods can be employed for switching from the sleep mode to the operable mode.

Next, a startup method according to this embodiment will be described. FIG. 7 is a flow chart showing an example of the steps of the start-up method according to this embodiment. An example of a method for starting up the aerial operation device 1 will be described below with reference to FIG. First, the air control device 1 is powered on. For example, the aerial operation device 1 is activated while the display device 15 is in sleep mode (the step of entering sleep mode, step S1).

Next, for example, the control unit 20 determines whether or not the sensor 12 has detected the target object F (step of determining whether or not the sensor has detected the target object; step S2). The detection of the object F by the sensor 12 in step S2 may be different from the detection that the object F has reached the depression determination plane Z. FIG. For example, the sensor 12 detects whether or not the object F exists in a predetermined area farther from the sensor 12 than the press determination surface Z.

When the control unit 20 determines that the sensor 12 has not detected the object F in the predetermined area, the series of steps ends. On the other hand, when the control unit 20 determines that the sensor 12 has detected the object F in the predetermined area, the control unit determines whether or not the object F exists within the set area (for example, the depression determination surface Z). 20 determines (step S3).

In step S3, for example, when the control unit 20 determines that there is no object F on the depression determination surface Z, a series of steps is completed. Further, when the control unit 20 determines that the object F is present on the depression determination surface Z, the display device 15 displays the operation screen 10 in the air via the aerial imaging device 14 (displays the operation image in the air). step S4), and the mode switching unit 16 switches from the sleep mode to the operable mode (step of switching to the operable mode, step S5). A series of steps of the start-up method is completed through the above steps.

Next, the effects of the aerial operation device 1 and the startup method according to this embodiment will be described. In the aerial operation device 1 and the start-up method according to the present embodiment, the display device 15 irradiates the light C, and the aerial imaging device 14 reflects the light C from the display device 15 multiple times to form an aerial image with the button 11 attached. to display the operation screen 10 of . The aerial operating device 1 includes a sensor 12 that detects an object F approaching the button 11 . The display device 15 has a sleep mode in which the operation screen 10 is not displayed and an operable mode in which the operation screen 10 is displayed.

When the sensor 12 detects the object F, the display device 15 switches from sleep mode to operable mode and displays the operation screen 10 in the air. By displaying the operation screen 10 in the air when the object F is detected, the operation screen 10 can be displayed in an easy-to-understand manner when returning from the sleep mode to the operable mode. That is, since the operation screen 10 as an aerial image is displayed at a position on the nearer side than the aerial imaging device 14, the operation screen 10 can be displayed in an easy-to-understand manner when the sleep mode is canceled. Furthermore, since it is not necessary to touch the device when canceling the sleep mode, the aerial operation device 1 can be excellent in terms of hygiene.

The display device 15 may display the sleep mode release button 10A in the air when in sleep mode. In this case, the user can recognize that it is in sleep mode, and can easily release the sleep mode by pressing the sleep mode release button 10A.

The aerial operation device 1 according to this embodiment includes a plurality of sensors 12. Therefore, when at least one of the plurality of sensors 12 detects the object F, the display device 15 displays the operation screen 10 in the air. Therefore, when any one of the plurality of sensors 12 detects the object F, the mode shifts to the operable mode and the operation screen 10 is displayed in the air.

The display device 15 switches from the sleep mode to the operable mode when the plurality of sensors 12 detect a predetermined action of the object F (for example, an action of approaching the sensor 12 and then moving in the second direction D2). The operation screen 10 may be displayed in the air. In this case, the sleep mode is released when the plurality of sensors 12 detect a predetermined motion of the object F. FIG. Therefore, since the sleep mode is not canceled unless the object F performs a predetermined action, unintentional display of the operation screen 10 is suppressed. Therefore, the security of the air control device 1 can be enhanced.

Next, an aerial operation device 41 according to a modification will be described with reference to FIGS. 8 and 9. FIG. A part of the configuration of the air operation device 41 is the same as a part of the configuration of the air operation device 1 described above. In the following, explanations that overlap with the configuration of the aerial operation device 1 will be omitted as appropriate. FIG. 8 is a perspective view showing an aerial operation device 41 according to a modification. FIG. 9 is a vertical cross-sectional view schematically showing the internal structure of the aerial operation device 41. As shown in FIG. As shown in FIGS. 8 and 9, the aerial operation device 41 displays a display surface 50 having a plurality of buttons 51, which are numeric keys, in the air as an aerial image. The aerial operation device 41 has a columnar shape. The aerial operation device 41 extends in a first direction A1, a second direction A2 and a third direction A3.

The first direction A1 corresponds to the depth direction (front-rear direction) of the aerial operation device 41 . The second direction A2 corresponds to the width direction of the aerial operation device 41 . A third direction A3 corresponds to the height direction of the aerial operation device 41 . The length (height) of the air operation device 41 in the third direction A3 is the length (depth) of the air operation device 41 in the first direction A1 and the length (width) of the air operation device 41 in the second direction A2. longer than This makes it possible to further improve the visibility of the display surface 50 for the user.

The aerial operation device 41 includes a housing 42 that constitutes the outer surface of the aerial operation device 41 . The housing 42 has, for example, an upper surface 42b, a pair of side surfaces 42c, a front surface 42d, and a lower surface 42g. The upper surface 42b faces vertically upward, and the pair of side surfaces 42c are arranged along the second direction A2. The front surface 42d faces one side in the first direction A1, and the rear surface 42f faces the other side in the first direction A1. The lower surface 42g faces vertically downward. Further, the housing 42 has an inclined surface 42h located above the front surface 42d.

An opening through which the aerial imaging device 54 is exposed is formed in the inclined surface 42h. This opening has, for example, a rectangular shape. The inclined surface 42h is directed obliquely upward, for example. The display surface 50 imaged by the aerial imaging device 54 is displayed at a position closer to the user than the inclined surface 42h. Therefore, the visibility of the display surface 50 can be improved.

A button 51 on the display surface 50 is a button for operating an automated teller machine (ATM). A display surface 50 with a button 51 is a floating image displayed in the air. That is, the display surface 50 with the button 51 is displayed as an aerial image. Therefore, the user of the air control device 41 does not have to directly touch the touch panel or the like, and thus can sanitarily operate the automated teller machine.

For example, the display surface 50 has a plurality of buttons 51, and the user can operate the automatic teller machine by operating the plurality of buttons 51. A sensor 43 for detecting an object F approaching the button 51 is embedded in the housing 42, for example. The configuration of the sensor 43 is similar to that of the sensor 12 described above.

The aerial operation device 41 includes a plurality of sensors 43, for example. The sensors 43 are provided, for example, corresponding to the buttons 51 , and each button 51 is displayed obliquely above each sensor 43 . As an example, the number of buttons 51 is twelve and the number of sensors 43 is twelve or less. The display surface 50 is displayed at a position closer to the user than the aerial imaging device 54 and obliquely above the aerial imaging device 54 . This makes it easier for the user to visually recognize the display surface 50 .

For example, the display surface 50 may be illuminated, and in this case, the display surface 50 can be easily viewed regardless of the brightness of the surroundings. The display surface 50 has, for example, a shape that extends vertically. As an example, the display surface 50 is displayed in a rectangular shape, and the vertical length of the display surface 50 is longer than the horizontal length of the display surface 50 . However, the shape and size of the display surface 50 are not limited to the above examples and are not particularly limited. The display surface 50 may display items other than the button 51 . For example, the display surface 50 may include a plurality of buttons 51 and character information. Character information may include at least one of date information and time information, for example.

On the display surface 50, for example, a plurality of buttons 51 are displayed in a grid pattern. As an example, the plurality of buttons 51 are displayed as numeric keys. In this case, by pressing the button 51 of the air control device 41, it becomes possible to designate the withdrawal amount, the deposit amount, the transfer amount, or the personal identification number for the automated teller machine.

The shape of the button 51 is, for example, a square shape (a rectangular shape with rounded corners as an example). However, the shape of the button 51 may be circular, oval, or polygonal, and is not particularly limited. For example, the button 51 may include at least one of an enter button and a return button. In this case, the enter button corresponds to a button for deciding an operation, and the return button corresponds to a button for returning to the previous operation. However, the type of button 51 is not limited to the above examples and is not particularly limited.

The aerial operation device 41 includes an aerial imaging device 54 that displays a display surface 50 with buttons 51 , a display device 55 arranged inside the housing 42 , and a control section 60 . The aerial imaging device 54 and the display device 55 correspond to an aerial image display unit that displays the display surface 50 as an aerial image in the air. The aerial imaging device 54 is, for example, a retroreflective member (retroreflective mirror) fixed to an opening formed in the inclined surface 42 h of the housing 42 . The display device 55 is arranged obliquely with respect to the aerial imaging device 54 . The configurations of the aerial imaging device 54, the display device 55, and the control unit 60 are the same as the configurations of the aerial imaging device 14, the display device 15, and the control unit 20 described above.

The modification of the air control device 41 provided in the automated teller machine has been described above. The same effects as those of the air operation device 1 can be obtained from the air operation device 41 as well. The operation input device according to the present disclosure can be further modified with respect to the aerial operation device 41 according to the modified example. For example, in the modified example described above, the air control device 41 provided in the automated teller machine has been described. However, the aerial operation device according to the present disclosure may be provided in kitchens, bathrooms, hospitals, public institutions, stations, airports, and the like.

For example, in an aerial operation device installed in a kitchen, a display surface with buttons is displayed as an aerial image, making it possible to operate a cooking app without directly touching the display with a finger or the like with ingredients attached. That is, conventionally, in order to touch the display of a mobile terminal or the like, the user has to wash his/her hands before operating the cooking application displayed on the display. However, in the aerial operation device according to the present disclosure, the display surface with the buttons is displayed as an aerial image, so the cooking app can be operated by operating the buttons in the air. Therefore, since the cooking application can be operated without washing hands, the aerial operation device can be highly operable.

For example, in an aerial operation device provided in ATMs, hospitals, public institutions, stations, or airports, a display surface with buttons is displayed as an aerial image. Alternatively, it is possible to operate an airport reception machine or the like. Therefore, it is possible to eliminate the need to touch the shared display with a finger, so that the operation can be performed hygienically.

DESCRIPTION OF SYMBOLS 1... Air control device 2... Housing 2b... Front surface 2c... Side surface 2d... Top surface 2f... Bottom surface 2g... Opening 2h... First plate portion 2j... Second plate portion 3... Protruding portion, 3b...Main surface 3c...End surface 3d...Back surface 10...Operation screen 10b...Center 11...Button 11b...Center 12...Sensor 13...Character information 14...Aerial imaging device 15...Display device , 16... Mode switching unit 20... Control unit 21... Image output unit 22... Object detection unit 23... Recognition unit 24... Signal output unit 25... Notification unit 25b... Audio output unit 25c... Color Change part 30... Equipment 41... Air control device 42... Housing 42b... Upper surface 42c... Side surface 42d... Front surface 42f... Rear surface 42g... Lower surface 42h... Inclined surface 43... Sensor 50... Display Surface 51 Button 54 Aerial imaging device 55 Display device 60 Control unit A1 First direction A2 Second direction A3 Third direction B Toilet bowl C Light D1 ... first direction, D2 ... second direction, D3 ... third direction, E ... toilet paper, F ... object, K ... distance, M ... washstand, T ... toilet, V ... reference line, W ... wall, Y, Y1, Y2... Threshold value, Z... Pressing determination surface, Z1... First surface, Z2... Second surface.

Claims

a display device that displays information by irradiating light;
an aerial imaging device that reflects light from the display device multiple times to display an operation screen with buttons as an aerial image;
a sensor for detecting an object approaching the button;
with
The display device has a sleep mode in which the operation screen is not displayed via the aerial imaging device, and an operable mode in which the operation screen is displayed via the aerial imaging device,
When the sensor detects the object, the display device switches from the sleep mode to the operable mode and displays the operation screen in the air.
aerial manipulator.
The display device displays a sleep mode release button in the air when in the sleep mode.
2. An aerial operating device according to claim 1.
comprising a plurality of said sensors;
3. An aerial operation device according to claim 1 or 2.
The display device switches from the sleep mode to the operable mode and displays the operation screen in the air when the plurality of sensors detect a predetermined action of the object.
4. An aerial operating device according to claim 3.
a display device that displays information by irradiating light;
an aerial imaging device that reflects light from the display device multiple times to display an operation screen with buttons as an aerial image;
a sensor that detects an object approaching the button;
A method for starting up an aerial operation device comprising
a step in which the display device enters a sleep mode in which the operation screen is not displayed via the aerial imaging device;
a step of switching the display device from the sleep mode to an operable mode and displaying the operation screen in the air via the aerial imaging device when the sensor detects the object;
Start-up method with