JP2024029268A - Operation input device and calibration method - Google Patents

Abstract

The present invention provides an operation input device and a calibration method that can perform highly accurate calibration according to the characteristics of each of a plurality of users. In an operation input device according to an embodiment, a calculation unit is configured to store a first position, which is a position of an object relative to a predetermined position in a first logical area when a first button is pressed, and a first position when a second button is pressed. a second position that is the position of the object relative to a predetermined position in the second logical area when the third button is pressed; and a third position that is the position of the object relative to the predetermined position in the third logical area when the third button is pressed. , calculate the reference position of the object from . [Selection diagram] Figure 4

Description

The present disclosure relates to an operation input device and a calibration method.

Japanese Unexamined Patent Publication No. 2017-16516 describes an input device that performs a calibration operation. The input device includes a display section, a touch panel, a control section, and a processing section. The processing section includes a coordinate conversion section, a determination section, a display control section, a calibration processing section, a setting section, and an output section. The display unit is an operation screen for inputting operations by touch, and includes a plurality of button images. The touch panel is of a resistive type and is placed over the display section. The touch panel detects a user's touch position on the display unit.

The calibration processing unit detects a positional shift amount between a touch position and a corresponding position based on a user's touch input to the display unit. Then, the calibration processing section corrects a predetermined transformation rule that the coordinate transformation section refers to during coordinate transformation so as to offset the detected positional deviation amount.

The display screen has multiple button images. A specific button image among the plurality of button images includes a reference position. When the user touches a specific button image, the touch panel determines the detection position as the touch position for that touch. Then, the coordinate transformation unit coordinates transforms the detected position into a corresponding position by referring to a predetermined transformation rule. If the corresponding position is included in a specific button image, the determination unit determines that a touch input has been performed on the specific button.

Further, when a touch input is performed on a specific button image, the calibration processing section uses the touch input as a calibration operation to correct positional deviation. The calibration processing unit calculates a difference between a reference position and a corresponding position for a touch position on a specific button image. Then, the calibration processing section corrects a predetermined transformation rule that the coordinate transformation section refers to in coordinate transformation so as to cancel out the obtained difference. Therefore, the positional deviation between the reference position and the corresponding position is corrected. As a result, the positional deviation between the touch position and the corresponding position is corrected.

JP 2017-16516 Publication

By the way, in the case of an operation input device that performs operations by pressing an image with an object such as a finger, calibration is required, which is a process of matching the position of the button displayed on the image with the position pressed by the object such as a finger. It can be. Further, the operation input device may be used by multiple users. In this case, for example, the thickness of the finger or the angle of the finger relative to the button differs depending on the user, so the position at which the button is pressed may vary depending on the user.

If calibration is performed for each user, highly accurate operation can be achieved for multiple users, even if the button is pressed at a different position depending on the user. However, as described above, highly accurate calibration may not be achieved simply by correcting a predetermined transformation rule referred to in coordinate transformation so as to cancel out the determined difference. Therefore, it is required to perform highly accurate calibration in accordance with the characteristics of each of a plurality of users.

An object of the present disclosure is to provide an operation input device and a calibration method that can perform highly accurate calibration according to the characteristics of each of a plurality of users.

An operation input device according to the present disclosure includes a display device that displays information as an image by irradiating light, a sensor that detects the position of an object approaching the image, and an object that is detected from the position of the object detected by the sensor. a calculation unit for calculating a reference position of the object in the image, the display device displays a first button included in the first logical area for the sensor to detect the position of the object in the image; A second button included in a second logical area for detecting the position of the object is displayed in a location different from the first logical area, and the sensor is included in the third logical area for detecting the position of the object in the image. The third button is displayed in a location different from the first logical area and the second logical area, and the calculation unit calculates the position of the object relative to the predetermined position of the first logical area when the first button is pressed. a second position, which is the position of the object relative to a predetermined position in the second logical area when the second button is pressed, and the object relative to a predetermined position in the third logical area when the third button is pressed; The reference position of the object is calculated from the third position, which is the position of .

In this operation input device, a display device emits light to display information as an image, and a sensor detects the position of an object such as a finger approaching the image. The operation input device includes a calculation section, and the calculation section calculates a reference position of the object from the position of the object detected by the sensor. By the calculation unit calculating the reference position of the object in this manner, calibration can be performed using the pressed position of the object relative to the button displayed in the image as the reference position. Therefore, since calibration can be performed for each user who operates the object, highly accurate operation can be achieved for a plurality of users. Further, the display device displays a first button included in the first logical area, displays a second button included in the second logical area in a location different from the first logical area, and displays a second button included in the third logical area. The third button is displayed in a location different from the first logical area and the second logical area. The calculation unit calculates a first position of the object relative to a predetermined position in the first logical area, a second position of the object relative to the predetermined position in the second logical area, and a third position of the object relative to the predetermined position in the third logical area. , calculate the reference position of the object from . Since the calculation unit calculates the reference position from the first position, second position, and third position, which are the positions where the image is pressed by the object, it is possible to perform highly accurate calibration based on the positions of the three points. Therefore, highly accurate calibration can be performed in accordance with the characteristics of a plurality of users.

The aforementioned operation input device includes an aerial imaging device that displays an image by reflecting light from the display device multiple times, and the aerial imaging device may display the image as a virtual image in the air. In this case, highly accurate calibration can be performed even for images displayed as virtual images in the air. Therefore, it is possible to improve the operability of the display surface that is displayed as a virtual image.

The image displayed by the display device may be divided into a plurality of areas, including an area where the first button is displayed, an area where the second button is displayed, and an area where the third button is displayed. They may be different from each other. In this case, since the area where the first button is displayed, the area where the second button is displayed, and the area where the third button is displayed can be separated from each other, the calibration can be performed with higher accuracy from three positions. tion can be carried out.

The aforementioned operation input device may include a signal output section that outputs a control signal to a device external to the operation input device to control the operation of the device, and the signal output section outputs a control signal when the first button is pressed. A control signal may be output to the device each time the second button is pressed and when the third button is pressed. In this case, since the signal output section outputs a control signal to the device each time the first, second, and third buttons are pressed, calibration can be performed simultaneously with the user's operation of the device. Therefore, it contributes to further improvement in operability.

A calibration method according to the present disclosure is a calibration method for calculating a reference position of an object from a position of the object approaching an image displayed by a display device, wherein the display device is included as an image in a first logical area. calculating a first position that is the position of the object relative to a predetermined position in the first logical area when the first button is pressed by the object; displaying a second button included in the second logical area in a location different from the first logical area; and determining the position of the object relative to a predetermined position in the second logical area when the second button is pressed by the object. a step of calculating a certain second position; a step of causing the display device to display a third button included in the third logical area as an image in a location different from the first logical area and the second logical area; Calculating a third position, which is the position of the object with respect to a predetermined position in the third logical area when the button is pressed, and calculating a reference position of the object from the first position, second position, and third position. and a step of doing so.

In this calibration method, a display device displays a first button included in a first logical area, and a calculation unit calculates a first position of the object relative to a predetermined position in the first logical area. The display device displays a second button included in the second logical area, and the calculation unit calculates a second position of the object relative to a predetermined position in the second logical area. The display device displays a third button included in the third logical area, and the calculation unit calculates a third position of the object with respect to a predetermined position in the third logical area. Then, the calculation unit calculates a reference position of the object from the first position, second position, and third position of the object. Therefore, like the operation input device described above, the calculation unit calculates the reference position from the first position, second position, and third position, which are the positions where the image is pressed by the object, so the height is high based on the positions of the three points. Accuracy can be calibrated. As a result, highly accurate calibration can be performed in accordance with the characteristics of a plurality of users.

According to the present disclosure, highly accurate calibration can be performed in accordance with the characteristics of each of a plurality of users.

FIG. 1 is a perspective view schematically showing an operation input device according to an embodiment. FIG. 2 is a cross-sectional view schematically showing the internal structure of the operation input device according to the embodiment. FIG. 2 is a block diagram showing the functions of the operation input device according to the embodiment. It is a flowchart which shows the process of the calibration method concerning an embodiment. FIG. 3 is a diagram showing one step of the calibration method according to the embodiment. (a), (b), and (c) are diagrams showing logical areas and buttons used in the calibration method according to the embodiment. FIG. 3 is a diagram showing one step of the calibration method according to the embodiment. It is a figure showing one process of the calibration method concerning an embodiment. FIG. 3 is a diagram showing one step of the calibration method according to the embodiment. FIG. 3 is a diagram showing one step of the calibration method according to the embodiment. FIG. 3 is a diagram showing one step of the calibration method according to the embodiment.

Hereinafter, embodiments of an operation input device according to the present disclosure will be described with reference to the drawings. In the description of the drawings, the same or corresponding elements are given the same reference numerals, and redundant description will be omitted as appropriate. In addition, some parts of the drawings may be simplified or exaggerated for ease of understanding, and the dimensional ratios, angles, etc. are not limited to those shown in the drawings.

FIG. 1 is a perspective view showing an example of an operation input device 1 according to an embodiment. The operation input device 1 as an example is provided attached to an automated teller machine (ATM). The operation input device 1 displays an image 10, which is a display surface of a virtual image displayed in the air, toward the user. An image 10 is a device operation screen on the operation input device 1. The operation input device 1 as an example has a columnar shape. The operation input device 1 extends in a first direction D1, a second direction D2, and a third direction D3.

The first direction D1 corresponds to the depth direction (back and forth direction) of the operation input device 1, the second direction D2 corresponds to the width direction of the operation input device 1, and the third direction D3 corresponds to the height direction of the operation input device 1. Equivalent to. As an example, the length (height) of the operation input device 1 in the third direction D3 is the length (depth) of the operation input device 1 in the first direction D1, and the length (depth) of the operation input device 1 in the second direction D2. longer than (width). This makes it possible to further improve the visibility of the image 10 for the user. However, the relationship between the length of the operation input device 1 in the first direction D1, the length of the operation input device 1 in the second direction D2, and the length of the operation input device 1 in the third direction D3 is limited to the above example. There are no particular limitations.

Hereinafter, for convenience of explanation, the direction facing the user when viewed from the operation input device 1 will be referred to as the front or front, and the direction facing away from the user when viewed from the operation input device 1 will be referred to as the rear or rear side. It is sometimes called. However, these directions are for convenience of explanation and do not limit the arrangement of components.

FIG. 2 is a longitudinal sectional view showing an example of the internal structure of the operation input device 1. As shown in FIG. As shown in FIGS. 1 and 2, the operation input device 1 includes a housing 2 that forms the outer surface of the operation input device 1. The housing 2 has, for example, an upper surface 2b facing vertically upward, a pair of side surfaces 2c lined up along the second direction D2, a front surface 2d facing one side in the first direction D1, and a front surface 2d facing the other side in the first direction D1. It has a back surface 2f that faces toward the front, and a lower surface 2g that faces vertically downward. Furthermore, the housing 2 has an inclined surface 2h located above the front surface 2d.

For example, a rectangular opening through which an aerial imaging device 14, which will be described in detail later, is exposed is formed in the inclined surface 2h. For example, the inclined surface 2h is directed diagonally upward. Since the image 10 formed by the aerial imaging device 14 is displayed at a position closer to the user than the inclined surface 2h, the visibility of the image 10 can be improved.

The operation input device 1 displays an image 10 with a button 11 in the air. For example, button 11 in image 10 is a button for operating an automated teller machine. The image 10 with the button 11 is a floating image displayed so as to stand out in the air. That is, the image 10 with the button 11 is displayed as a virtual image in the air. Therefore, the user of the operation input device 1 is able to operate the automatic teller machine hygienically since he or she does not have to directly touch the touch panel or the like.

For example, a button 11 is displayed multiple times in the image 10, and the user can operate the automatic teller machine by operating the button 11 multiple times. As a specific example, by pressing the button 11, it becomes possible to specify the amount to be debited, the amount to be deposited, the amount to be transferred, or a password to the automated teller machine. For example, a sensor 3 for detecting an object F approaching the button 11 is embedded in the housing 2 . In the present disclosure, the "object" refers to something that operates a device such as an automated teller machine by operating the button 11, and indicates, for example, a user's finger.

The sensor 3 is provided corresponding to, for example, a button 11, and the button 11 is displayed diagonally above the sensor 3. The number of buttons 11 and sensors 3 may be singular or plural. The image 10 is displayed at a position closer to the user than the aerial imaging device 14, and is displayed diagonally above the aerial imaging device 14. Thereby, the image 10 can be displayed easily for the user to visually recognize.

For example, the image 10 may be displayed as glowing, in which case the image 10 can be easily viewed regardless of the surrounding brightness. For example, the image 10 has a shape that extends vertically. As an example, the image 10 is displayed in a rectangular shape, and the length of the image 10 in the vertical direction is longer than the length of the image 10 in the horizontal direction. However, the shape and size of the image 10 are not limited to the above example and are not particularly limited. The image 10 may display items other than the button 11. For example, the image 10 may include a plurality of buttons 11 and text information. The text information may include, for example, at least one of information regarding the operation of the button 11, date information, and time information. The button 11 has a rectangular shape, for example. However, the shape of the button 11 may be circular, oval, or polygonal, and is not particularly limited.

The operation input device 1 includes an aerial imaging device 14 that displays an image 10 with a button 11 as a virtual image, a display device 15 disposed inside a housing 2, and a control unit 20. The aerial imaging device 14 and the display device 15 correspond to a virtual image display unit that displays the image 10 as a virtual image in the air. The aerial imaging device 14 is, for example, a retroreflective member (retroreflector) fixed to an opening formed in the inclined surface 2h of the housing 2.

The display device 15 is arranged obliquely to the aerial imaging device 14 . As an example, display device 15 is a liquid crystal display (LCD). The display device 15 is arranged inside the housing 2 at a position behind the aerial imaging device 14 . Display device 15 has a screen that displays images. The screen of the display device 15, for example, irradiates light C as an image diagonally downward toward the aerial imaging device 14. The aerial imaging device 14 reflects the light C from the display device 15 multiple times (for example, twice) inside the aerial imaging device 14, and reflects the light C from the aerial imaging device 14 into a space in front of the aerial imaging device 14 when viewed from the user. Image 10 is formed.

For example, the sensor 3 may be exposed on the inclined surface 2h of the housing 2 so as to face obliquely upward. As an example, sensor 3 is a depth sensor. For example, the sensor 3 is provided on a virtual straight line extending from the button 11, that is, in a position in front of the button 11, which is a virtual image. The sensor 3 acquires distance image data including information on the position (two-dimensional position) of the object F in a plane perpendicular to the virtual straight line and information on the distance K from the sensor 3 to the object F.

For example, the sensor 3 outputs the acquired distance image data to the control unit 20 at a predetermined period (for example, 1/30 seconds). As a specific example, the sensor 3 irradiates a light beam (or infrared rays) to each point on an object existing in the photographing region including the target object F, and receives the light beam reflected from each point on the object. Then, the sensor 3 measures the distance between the sensor 3 and each point on the object based on the received light beam, and outputs the measured distance for each pixel. The distance between the sensor 3 and each point on the object may be measured by, for example, a light coding method.

In the Light Coding method, the sensor 3 irradiates each point on the object, including the object F, within the imaging area with a light beam in a random dot pattern. Then, the sensor 3 measures the distance between the sensor 3 and each point on the object by receiving the light beam reflected from each point on the object and detecting the distortion of the pattern of the reflected light beam. The sensor 3 detects information on the two-dimensional position of each point on the object and information on the distance from the sensor 3 to each point on the object as a plurality of pixels, and outputs the detected plurality of pixels to the control unit 20. do.

The control unit 20 is capable of communicating with each of the sensor 3 and the display device 15. The control unit 20 includes, for example, a CPU (Central Processing Unit) that executes a program, a storage unit including a ROM (Read Only Memory) and a RAM (Random Access Memory), an input/output unit, and a driver. Each function of the control unit 20 is realized by operating the input/output unit under the control of the CPU and reading or writing data in the storage unit. The form and location of the control unit 20 are not particularly limited.

FIG. 3 is a functional block diagram of the control unit 20. As shown in FIGS. 2 and 3, the control unit 20 includes an image output unit 21, an object detection unit 22, a recognition unit 23, a signal output unit 24, and a notification unit 25 as functional components. , and a calculating section 26. The image output unit 21 outputs to the display device 15 a control signal for image data of an image to be displayed on the display device 15. The display device 15 is capable of displaying various types of images based on image data control signals from the image output section 21.

The target object detection unit 22 detects the target object F based on the distance image data output from the sensor 3. When the target object F is detected, the target object detection unit 22 outputs position data indicating the position of the target object F to the recognition unit 23. The recognition unit 23 recognizes that the button 11 has been pressed by the object F based on the position data output from the object detection unit 22.

The recognition unit 23 determines whether the button 11 has been pressed by the object F, based on the position of the object F detected by the object detection unit 22. Specifically, the recognition unit 23 determines whether the distance K between the sensor 3 and the object F is less than or equal to the threshold value Y. When the recognition unit 23 determines that the distance K is less than or equal to the threshold value Y, the recognition unit 23 determines that the object F has reached the virtual press determination surface Z and the button 11 has been pressed.

The press determination surface Z is a virtual surface formed at a constant distance from the sensor 3. For example, the press determination surface Z is provided at a position close to the button 11 . The position of the press determination surface Z may coincide with the position of the button 11, or may be a position separated from the button 11 by a predetermined distance.

The notification unit 25 receives an operation signal from the recognition unit 23 when the recognition unit 23 recognizes that the button 11 has been pressed. The notification unit 25 is a notification unit that notifies the user of the operation input device 1 that the button 11 has been operated when the recognition unit 23 recognizes that the button 11 has been operated. The notification section 25 includes, for example, an audio output section 25b and a color change section 25c.

As an example, the audio output unit 25b is a speaker, and outputs audio when receiving an operation signal from the recognition unit 23. By hearing the audio from the audio output section 25b, the user can recognize that the button 11 has been operated. For example, the color change unit 25c generates a color change signal when receiving the operation signal from the recognition unit 23, and outputs the color change signal to the display device 15.

When the display device 15 receives the color change signal from the color change section 25c, it changes the color of the button 11 pressed by the user, for example. By visually recognizing that the color of the button 11 has been changed, the user can recognize that the button 11 has been operated. Note that the display device 15 may change the color of a portion other than the button 11 when receiving a color change signal from the color change section 25c.

By providing the above notification section 25, the user can know that the button 11 has been operated. However, in the notification section 25, at least one of the audio output section 25b and the color change section 25c may be omitted. Further, the notification unit 25 may notify the user that the button 11 has been operated in a manner different from the audio output by the audio output unit 25b or the color change by the color change unit 25c.

The signal output unit 24 generates a control signal based on the pressing operation of the button 11 when the recognition unit 23 recognizes that the button 11 has been pressed. The signal output unit 24 transmits the generated control signal to a device external to the operation input device 1 (for example, the above-mentioned automatic teller machine), and operates the device by transmitting the control signal to the device. If the device is an automated teller machine, the operation of withdrawing cash, depositing cash, transferring money, or specifying a password is performed.

The calculation unit 26 calculates the reference position of the object F. In the present disclosure, the "reference position of the target object" indicates the position of the target object at which the user tends to press (which part of the button the user tends to press), which is determined for each user of the operation input device. By the calculation unit 26 calculating the reference position of the object F, it becomes possible to correct individual differences in the position at which the button 11 is pressed among a plurality of users. That is, it becomes possible to calibrate the pressing position of the button 11 for each user.

An example of the calibration method according to this embodiment will be described below with reference to FIG. 4. FIG. 4 is a flowchart illustrating an example of each step of a method for calculating the reference position of the object F by calculation by the calculation unit 26. As an initial operation, first, as shown in FIG. 5, the control unit 20 causes the display device 15 to display the image 10 divided into areas A1, A2, A3, A4, A5, A6, A7, A8, and A9. In this way, the display device 15 divides the image 10 into nine areas and displays them. Note that the divided areas A1, A2, A3, A4, A5, A6, A7, A8, and A9 are not visible to the user.

Further, in the operation input device 1, when the calculation unit 26 calculates the reference position of the object F, the button 11 included in the logical area 12 is used as a GUI part, as shown in FIG. 6(a). The logical area 12 is an area for detecting the position of the object F in the image 10, and is formed to surround the button 11, for example.

As shown in FIG. 6B, for example, the logical area 12 has a rectangular shape, and a predetermined position 11b (a predetermined position PX1, a predetermined position PX2, a predetermined position PX2, and a predetermined position PX1, PX2, and (corresponding to position PX3). The predetermined position 11b indicates, for example, the center position of the button 11. Further, as shown in FIG. 6(c), the logical area 12 is not visible to the user of the operation input device.

Using the button 11 and logical area 12 described above, the control unit 20 causes the display device 15 to display the first button 11A included in the first logical area 12A (first Displaying a button, step S1). For example, the display device 15 displays the first button 11A included in the first logical area 12A in the area A8 along with the text information 13. At this time, the first logical area 12A is also arranged inside the area A8. Further, the first button 11A is, for example, an operation start button.

Subsequently, as shown in FIG. 9, when the first button 11A is pressed by the object F, the calculation unit calculates a first position P1 that is the position of the object F with respect to a predetermined position PX1 of the first logical area 12A. 26 calculates (step of calculating the first position, step S2). Note that even if the first button 11A is not pressed by the object F, if an area within the first logical area 12A is pressed, the calculation unit 26 performs calculation; If the outside is pressed, this pressing operation is treated as an invalid operation. As an example, the calculation unit 26 calculates a position lower right than the predetermined position PX1 as the first position P1 pressed by the object F, and the storage unit of the control unit 20 stores the first position P1.

Next, as shown in FIG. 10, the control unit 20 causes the display device 15 to display the second buttons 11B1 and 11B2 included in the second logical area 12B (a step of displaying the second buttons, step S3). As in the example of FIG. 10, the display device 15 may display a plurality of second buttons. For example, the display device 15 displays the second buttons 11B1 and 11B2 included in the second logical area 12B in the area A6 together with the text information 17. Note that the area in which the display device 15 displays the second buttons 11B1 and 11B2 may be any area other than the area A8 described above, but may be an area other than the area A8 (for example, area A1, area A4) that is not in the same row or column as area A8. , area A3 or area A6).

For example, when the second button 11B1 is pressed, the signal output unit 24 outputs a control signal corresponding to the second button 11B1 to a device external to the operation input device 1, and the device external to the operation input device 1 operates. be done. Further, when the second button 11B1 is pressed, the calculation unit 26 calculates a second position P2 that is the position of the object F with respect to the predetermined position PX2 of the second logical area 12B (step of calculating the second position, Step S4).

For example, the calculation unit 26 calculates the upper right position of the predetermined position PX2 as the second position P2 pressed by the object F, and the storage unit of the control unit 20 stores the second position P2. Further, even if the second buttons 11B1 and 11B2 are not pressed by the object F, if an area within the second logical area 12B is pressed, the calculation unit 26 performs calculation, but the second logical area 12B If the outside of the button is pressed, this pressing operation is treated as an invalid operation.

Subsequently, as shown in FIG. 11, the control unit 20 causes the display device 15 to display the third buttons 11C1 and 11C2 included in the third logical area 12C (a step of displaying the third button, step S5). For example, the display device 15 displays the third buttons 11C1 and 11C2 included in the third logical area 12C in the area A1 along with the text information 16. The area where the display device 15 displays the third buttons 11C1 and 11C2 may be any area other than the above-mentioned area A8 or area A6, but it must be an area that is not in the same row or column as area A8 or area A6. is more preferable.

For example, when the third button 11C2 is pressed, the signal output unit 24 outputs a control signal corresponding to the third button 11C2 to a device external to the operation input device 1, and the device external to the operation input device 1 operates. be done. The signal output unit 24 outputs a control signal corresponding to the third button 11C2 to a device external to the operation input device 1, and the device external to the operation input device 1 is operated.

Further, when the third button 11C2 is pressed, the calculation unit 26 calculates a third position P3 that is the position of the object F with respect to the predetermined position PX3 of the third logical area 12C (step of calculating the third position, Step S6). For example, the calculation unit 26 calculates the lower right position of the predetermined position PX3 as the third position P3 pressed by the object F, and the storage unit of the control unit 20 stores the third position P3. Note that even if the third buttons 11C1 and 11C2 are not pressed by the object F, if an area within the third logical area 12C is pressed, the calculation unit 26 performs calculation; If the outside of the button is pressed, this pressing operation is treated as an invalid operation.

Thereafter, the calculation unit 26 calculates the reference position of the object F from the first position P1, second position P2, and third position P3 stored in the storage unit of the control unit 20. As an example, the calculation unit 26 calculates the logical area from a first position P1 to the lower right of the predetermined position PX1, a second position P2 to the upper right of the predetermined position PX2, and a third position P3 to the lower right of the predetermined position PX3. The calculation is performed using the right side of the 12 predetermined position 11b as the reference position. Thereafter, a series of steps is completed, and, for example, the subsequent pressing operation of the button 11 by the object F is performed using the right side of the predetermined position 11b as a reference position.

Next, the effects obtained from the operation input device 1 and the calibration method according to this embodiment will be explained. In the operation input device 1 and the calibration method according to the present embodiment, the display device 15 emits light C to display information as an image 10, and the sensor 3 detects the position of an object F such as a finger approaching the image 10. To detect. The operation input device 1 includes a calculation section 26 , and the calculation section 26 calculates the reference position of the object F from the position of the object F detected by the sensor 3 . By the calculation unit 26 calculating the reference position of the object F in this manner, calibration can be performed using the pressed position of the object F with respect to the button 11 displayed on the image 10 as the reference position.

Therefore, since calibration can be performed for each user who operates the object F, highly accurate operations can be achieved for a plurality of users. Furthermore, the display device 15 displays the first button 11A included in the first logical area 12A, and displays the second buttons 11B1 and 11B2 included in the second logical area 12B in a location different from the first logical area 12A. , the third buttons 11C1 and 11C2 included in the third logical area 12C are displayed in a different location from the first logical area 12A and the second logical area 12B.

The calculation unit 26 calculates a first position P1 of the object F with respect to a predetermined position PX1 of the first logical area 12A, a second position P2 of the object F with respect to a predetermined position PX2 of the second logical area 12B, and a third logical area 12C. The reference position of the object F is calculated from the third position P3 of the object F with respect to the predetermined position PX3. The calculation unit 26 calculates the reference position from the first position P1, the second position P2, and the third position P3, which are the actual pressed positions of the image 10 by the object F, so it can be calculated with high precision based on the positions of the three points. Calibration can be performed. Therefore, highly accurate calibration can be performed in accordance with the characteristics of a plurality of users.

The operation input device 1 according to the present embodiment includes an aerial imaging device 14 that displays an image 10 by reflecting light C from a display device 15 multiple times, and the aerial imaging device 14 displays the image 10 as a virtual image in the air. may be displayed. Therefore, even the image 10 displayed as a virtual image in the air can be calibrated with high precision. Therefore, it is possible to improve the operability of the image 10, which is the display surface that is displayed as a virtual image to stand out.

The image 10 displayed by the display device 15 is divided into areas A1 to A9, which include an area A8 where the first button 11A is displayed, an area A6 where the second buttons 11B1 and 11B2 are displayed, and a third button. The areas A1 where 11C1 and 11C2 are displayed are different from each other. Therefore, the area A8 where the first button 11A is displayed, the area A6 where the second buttons 11B1 and 11B2 are displayed, and the area A1 where the third buttons 11C1 and 11C2 are displayed can be spaced apart from each other. Calibration can be performed with higher accuracy from the position of the point.

The operation input device 1 according to the present embodiment may include a signal output section 24 that outputs a control signal to a device external to the operation input device 1 to control the operation of the device. A control signal is output to the device when the button 11A is pressed, when the second buttons 11B1 and 11B2 are pressed, and when the third buttons 11C1 and 11C2 are pressed. Therefore, each time the first button 11A, the second buttons 11B1, 11B2, and the third buttons 11C1, 11C2 are pressed, the signal output unit 24 outputs a control signal to the device, so that the calibration can be performed simultaneously with the user's operation of the device. It can be performed. Therefore, it contributes to further improvement in operability.

The embodiments of the operation input device according to the present disclosure have been described above. However, the operation input device according to the present disclosure is not limited to the above-described embodiments, and may be modified or applied to other things without changing the gist of each claim. good. That is, the configuration, shape, size, number, material, and arrangement of each part of the operation input device can be changed as appropriate without changing the gist described above.

For example, in the above-described embodiment, an example was described in which the sensor 3 measures the distance between the sensor 3 and each point on the object using the Light Coding method, but the method is not limited to this method. For example, the sensor 3 may measure the distance between the sensor 3 and each point on the object using a TOF (Time Of Flight) method. In the TOF method, the sensor 3 calculates the flight time (delay time) of the light ray until it reaches the sensor 3 after being reflected at each point on the object, and from the calculated flight time and the speed of light, the sensor 3 Measure the distance between 3 and each point on the object. Even with such a configuration, the same effects as those of the embodiment described above can be achieved. Furthermore, the type of sensor is not limited to a depth sensor. That is, instead of the sensor 3, which is a depth sensor, an infrared sensor, an ultrasonic sensor, or the like may be provided, and the type of sensor can be changed as appropriate.

Moreover, in the above-mentioned embodiment, the operation input device 1 provided in an automated teller machine was illustrated. However, the aerial operating device according to the present disclosure may be installed in a kitchen, a toilet, a water area, a hospital, a public institution, a station, an airport, etc. For example, in an aerial operation device installed in a kitchen, a display screen with buttons is displayed as a virtual image in the air, making it possible to operate a cooking app without directly touching the display with a finger or the like covered with food. That is, in the past, in order to touch the display of a mobile terminal, etc., one had to wash one's hands before operating a cooking app displayed on the display, but with the aerial operation device according to the present disclosure, the display surface with buttons is It is displayed as a virtual image, so you can operate the cooking app by operating the buttons in the air. Therefore, since the cooking application can be operated without taking the trouble of washing hands, it is possible to provide an aerial operating device with high operability.

For example, in aerial operating devices installed at ATMs, hospitals, public institutions, stations, or airports, a display screen with buttons is displayed as a virtual image in the air, allowing users to operate ATMs or It is possible to operate reception machines at hospitals or airports. Therefore, since it is not necessary to touch the shared display with a finger, the operation can be performed hygienically.

Further, in the above-described embodiment, the display device 15 that divides the area into nine areas (divided into areas A1, A2, A3, A4, A5, A6, A7, A8, and A9) and displays the display device 15 has been described. However, the display device according to the present disclosure may display an image by dividing the area into two or more and eight or less, or ten or more. Furthermore, the display device may display the button four or more times, and the calculation unit may calculate the position of the object relative to the predetermined position four or more times. In this case, the precision of calibration can be further improved. Furthermore, in the embodiment described above, an example was described in which the aerial imaging device 14 and the display device 15 that display a virtual image display the image 10 in the air. However, the operation input device according to the present disclosure may include a display device that displays an image on a display instead of the aerial imaging device 14 and the display device 15. Then, the above-described calibration may be performed in an operation on the display screen of the display.

1... Operation input device, 2... Housing, 2b... Top surface, 2c... Side surface, 2d... Front surface, 2f... Back surface, 2g... Bottom surface, 2h... Inclined surface, 3... Sensor, 10... Image, 11... Button, 11A... First button, 11b...predetermined position, 11B1, 11B2...second button, 11C1, 11C2...third button, 12...logical area, 12A...first logical area, 12B...second logical area, 12C...third logical area , 13, 16, 17... Character information, 14... Aerial imaging device, 15... Display device, 20... Control section, 21... Image output section, 22... Target detection section, 23... Recognition section, 24... Signal output section , 25...Notification section, 25b...Audio output section, 25c...Color change section, 26...Calculation section, A1, A2, A3, A4, A5, A6, A7, A8, A9...Area, C...Light, D1...th 1 direction, D2...second direction, D3...third direction, F...object, K...distance, L...diagonal line, P1...first position, P2...second position, P3...third position, PX1, PX2, PX3...Predetermined position, Y...Threshold value, Z...Press determination surface.

Claims (5)

a display device that displays information as an image by irradiating light;
a sensor that detects the position of an object approaching the image;
a calculation unit that calculates a reference position of the object from the position of the object detected by the sensor;
Equipped with
The display device includes:
displaying a first button included in a first logical area for the sensor to detect the position of the object in the image;
displaying a second button included in a second logical area for the sensor to detect the position of the object in the image in a location different from the first logical area;
displaying a third button included in a third logical area for the sensor to detect the position of the object in the image in a location different from the first logical area and the second logical area;
The calculation unit includes:
a first position, which is the position of the object relative to a predetermined position in the first logical area when the first button is pressed, and a first position relative to a predetermined position in the second logical area when the second button is pressed; A reference position of the object is determined from a second position, which is the position of the object, and a third position, which is the position of the object with respect to a predetermined position of the third logical area when the third button is pressed. calculate,
Operation input device. an aerial imaging device that displays the image by reflecting light from the display device multiple times;
the aerial imaging device displays the image as a virtual image in the air;
The operation input device according to claim 1. The image displayed by the display device is divided into a plurality of regions,
the area where the first button is displayed, the area where the second button is displayed, and the area where the third button is displayed are different from each other;
The operation input device according to claim 1 or 2. comprising a signal output unit that outputs a control signal to a device external to the operation input device to control the operation of the device,
The signal output unit outputs a control signal to the device each time the first button is pressed, the second button is pressed, and the third button is pressed.
The operation input device according to any one of claims 1 to 3. A calibration method for calculating a reference position of an object from a position of the object approaching an image displayed by a display device, the method comprising:
a step in which the display device displays a first button included in the first logical area as the image;
calculating a first position that is the position of the object relative to a predetermined position of the first logical area when the first button is pressed by the object;
a step in which the display device displays a second button included in the second logical area as the image in a location different from the first logical area;
calculating a second position that is the position of the object relative to a predetermined position of the second logical area when the second button is pressed by the object;
a step in which the display device displays a third button included in the third logical area as the image in a location different from the first logical area and the second logical area;
calculating a third position that is the position of the object relative to a predetermined position of the third logical area when the third button is pressed by the object;
calculating a reference position of the object from the first position, the second position, and the third position;
Calibration method.

Images