WO2022215346A1

WO2022215346A1 - Electronic device

Info

Publication number: WO2022215346A1
Application number: PCT/JP2022/005227
Authority: WO
Inventors: 順以山口
Original assignee: シャープ株式会社
Priority date: 2021-04-08
Filing date: 2022-02-10
Publication date: 2022-10-13
Also published as: JPWO2022215346A1

Abstract

The electronic device according to the embodiment is capable of receiving information input from an operator in a non-contact manner, and comprises: a first detection unit that detects the inclination of the electronic device with respect to a predefined reference axis; a measurement unit that measures the distance to the operator; a second detection unit that detects the eye position of the operator; and an acquisition unit that acquires, on the basis of the inclination detected by the first detection unit, the distance measured by the measurement unit, and the eye position detected by the second detection unit, the position of a virtual operation space for recognizing information input operations performed by the operator in the non-contact manner.

Description

Electronics

　Embodiments according to the present invention relate to electronic devices. This application claims priority to Japanese Patent Application No. 2021-065575 filed in Japan on April 8, 2021, the contents of which are incorporated herein.

Cited Document 1 describes a technique that uses a small distance sensor to detect gestures of an operator in a space, and recognizes the movement as an operation for performing predetermined information processing.

JP 2017-058818 A

In the configuration of Cited Document 1, there is a problem that, when recognizing the operator's finger motion, there is a possibility that, for example, the character key type switching motion and the actual character selection motion may be erroneously recognized. ing.

However, with respect to character selection operations, etc., there is a risk of erroneous input occurring, for example, depending on the position of the information processing device and the operator, or the relationship between the touch panel of the information processing device and the operator's viewpoint. In addition, Cited Document 1 does not disclose anything about such problems.

According to one aspect of the present disclosure, it is possible to provide an electronic device capable of suppressing erroneous input.

An electronic device according to an aspect of the present disclosure is an electronic device capable of accepting non-contact information input by an operator, comprising: a first detection unit that detects an inclination of the electronic device with respect to a predetermined reference axis; a measurement unit that measures the distance between the operator, a second detection unit that detects the position of the operator's eyes, the inclination detected by the first detection unit, the distance measured by the measurement unit, and the second detection unit an obtaining unit that obtains a position in the virtual operation space for recognizing a non-contact information input operation by the operator based on the position of the eye detected in .

1 is an external view of a smartphone according to a first embodiment; FIG. 1 is an external view of a smartphone according to a first embodiment; FIG. Sectional drawing of the smart phone which concerns on 1st Embodiment. FIG. 2 is a hardware configuration diagram of a smartphone according to the first embodiment; FIG. 2 is a hardware configuration diagram of a smartphone according to the first embodiment; 3 is a functional block diagram of a CPU according to the first embodiment; FIG. 4 is a flowchart showing the operation of the smart phone according to the first embodiment; 6 is a flowchart showing the operation of the smartphone according to the modified example of the first embodiment; 4 is a conceptual diagram of a determination table according to the first embodiment; FIG. 4 is a conceptual diagram of an imaging region according to the first embodiment; FIG. FIG. 4 is a conceptual diagram showing the inclination of the smartphone according to the first embodiment; FIG. 4 is a conceptual diagram showing a method of determining the position of the virtual operation space in the smart phone according to the first embodiment; FIG. 4 is a conceptual diagram showing a method of determining the position of the virtual operation space in the smart phone according to the first embodiment; FIG. 4 is a conceptual diagram showing a method of determining the position of the virtual operation space in the smart phone according to the first embodiment; FIG. 10 is a hardware configuration diagram of a smartphone according to the second embodiment; FIG. 10 is a hardware configuration diagram of a smartphone according to the second embodiment; The functional block diagram of CPU which concerns on 2nd Embodiment. 8 is a flowchart showing the operation of the smart phone according to the second embodiment; FIG. 11 is a conceptual diagram showing a method of calibrating the virtual operation space in the smartphone according to the second embodiment; FIG. 11 is a conceptual diagram showing a method of calibrating the virtual operation space in the smartphone according to the second embodiment;

The present embodiment will be described below with reference to the drawings. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and overlapping descriptions are omitted. In addition, this embodiment described below does not unduly limit the content described in the claims. Moreover, not all the configurations described in the present embodiment are essential constituent elements of the present disclosure.

<First embodiment>
First, the electronic device according to the first embodiment will be described below by taking a smartphone as an example. FIG. 1 is an external perspective view of a smartphone according to this embodiment.

As shown in FIG. 1, the smartphone 10 according to the present embodiment includes, for example, a power button 11, a display 12, a camera 13, a ToF (Time of Flight) sensor 14, a speaker 15, a microphone 16, and a USB (Universal Serial Bus) terminal. 17.

The power button 11 is for powering on the smartphone 10 by an operation by the operator of the smartphone 10 . The display 12 has a touch panel function. The display 12 displays various application screens and accepts various information input by the operator such as character input and selection operation by the operator through the touch panel function. Information such as characters and numbers can be input by the operator without touching the display 12, the details of which will be described later. Note that FIG. 1 shows, as an example, a case where a numeric keypad is displayed on the display 12 as a display example for inputting information from the user. The camera 13 is an in-camera for imaging an operator or the like, for example. The ToF sensor 14 measures the distance to the object using infrared light, for example. Note that the camera 13 and the ToF sensor 14 may be integrated. The speaker 15 outputs voice during a call, and the microphone 16 receives the voice of the operator during a call. The USB terminal 17 is used for charging the smartphone 10 and transferring information.

Next, the concept of non-contact information input on the display 12 described above will be described with reference to FIGS. 2A and 2B. 2A is an external perspective view of the smartphone 10, similar to FIG. 1, and FIG. 2B is a cross-sectional view of the smartphone 10, showing an example in which the operator inputs numbers using the numeric keypad in a non-contact operation.

As shown in FIGS. 2A and 2B, a virtual operation space 19 is provided above the display 12 of the smartphone 10 (that is, the space between the display 12 and the operator). The virtual operation space 19 is provided at a position separated from the display 12 by a certain distance. This virtual operation space is provided at an appropriate position according to various conditions. This method will be described in detail later. Note that the virtual operation space 19 may or may not actually be displayed in the space so that the operator can recognize it, in other words, it may or may not actually be visible to the operator. may

FIG. 3A is a hardware configuration diagram showing the internal configuration of the smartphone 10 according to this embodiment. As illustrated, the smartphone 10 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a gyro sensor 104, a gyro data processing unit (first detection unit) 105, a ToF sensor 106 (ToF sensor 14 described in FIG. 1), ToF data processing unit (measurement unit) 107, camera 108 (camera 13 described in FIG. 1), imaging data processing unit (second detection unit) 109, position determination unit ( An acquisition unit) 110 and an input determination unit 111 are provided.

The CPU 101 controls the operation of the smartphone 10 as a whole. Various processors can be used as the CPU 101, and the processor is not necessarily limited to a CPU. The ROM 102 holds various programs and data such as a program 120 and a determination table 121 for operating the smartphone 10 . A RAM 103 functions as a work area for the CPU 101 and holds various programs and data. The gyro sensor 104 detects changes in the rotation angle and orientation of the smartphone 10 as angular velocity, and transfers the result to the gyro data processing unit 105 . The gyro data processing unit 105 calculates, for example, the angle of the smartphone 10 with respect to the horizontal axis based on the data obtained from the gyro sensor 104 . Note that the angle of the smartphone 10 is not necessarily calculated based on the horizontal axis, and may be a predetermined reference axis or reference plane. good. As described above, the ToF sensor 106 detects the distance to an object positioned on the display 12 side of the smartphone 10 and transfers the result to the ToF data processing unit 107 . The ToF data processing unit 107 calculates the distance to the face, for example, the eyes of the operator of the smartphone 10 based on the data obtained from the ToF sensor 106 . As described above, the camera 108 captures an image of an object located on the display 12 side of the smartphone 10 at a certain angle of view, and transfers the result to the captured data processing unit 109 . The captured data processing unit 109 detects the positions of the operator's eyes captured by the camera 108 and calculates the relative positional relationship between the smartphone 10 and the operator. Further, the captured data processing unit 109 detects the position and movement of the operator's finger captured by the camera 108 . The position determination unit 110 determines the position of the figure based on the angle obtained by the gyro data processing unit 105, the distance to the face obtained by the ToF data processing unit 107, the eye position obtained by the imaging data processing unit 109, and the like. Appropriate positions (coordinates) of the virtual operation space 19 described with reference to 2A and 2B are calculated. The input determination unit 111 determines whether the position of the operator's finger is within the virtual operation space 19 obtained by the position determination unit 110, and whether the motion of the operator's finger is within the virtual operation space 19. determine whether Then, when it is within the virtual operation space 19, it receives an input operation by the operator based on the operation of the finger.

Note that some hardware in FIG. 3A may be implemented by software. FIG. 3B is a hardware configuration diagram showing the internal configuration of smartphone 10 according to the modification of FIG. 3A, and FIG. 3C is a functional block diagram of CPU 101 when program 120 is executed.

3A, the gyro data processing unit 105, the ToF data processing unit 107, the position determination unit 110, and the input determination unit 111 described in FIG. 3A are implemented by the CPU 101 executing the program 120 instead of hardware. It may function as a processing unit. Note that the imaging data processing unit 109 is similar, but the imaging data processing unit 109 is realized by a processor (GPU (Graphical Processing Unit) 112 in the example of FIG. 3B) different from the CPU, as shown in FIG. 3B. may

4A is a flowchart showing the operation of the smartphone 10. FIG. A method of determining the position of the smartphone 10 in the virtual operation space 19 and a method of detecting an operator's motion in the virtual operation space 19 will be described below.

As shown, the gyro sensor 104 first acquires angular velocity data of the smartphone 10 (step S10). Next, the gyro data processing unit 105 detects, for example, the tilt θ1 of the smartphone 10 with respect to the horizontal axis based on the angular velocity data obtained in step S10, and stores it in the RAM 103 (step S11).

The ToF sensor 106 also acquires distance data to the object within the imaging range (step S12). Next, the ToF data processing unit 107 calculates the distance L to the operator's face, eg eyes, based on the distance data obtained in step S12, and stores it in the RAM 103 (step S13).

Furthermore, the camera 108 takes an image of the object within the angle of view θ2 (step S14). Based on the image data obtained in step S14, the imaging data processing unit 109 detects the operator's face, for example, the eyes, calculates the relative positional relationship between the smartphone 10 and the operator, and stores it in the RAM 103. (step S15).

The above steps S10, S12, and S14 are executed in parallel, for example. Then, based on θ1 obtained in step S11, the distance L to the operator's face obtained in step S13, and the relative positional relationship between the smartphone 10 and the operator's face obtained in step S15, , the position determining unit 110 calculates an appropriate position in the virtual manipulation space 19 and stores it in the RAM 103 (step S16). Calculation of the appropriate position of the virtual manipulation space 19 in step S16 can be performed using, for example, a determination table 121 prepared in advance. The determination table will be described later with reference to FIG.

Also, the camera 108 captures an image of the object within the angle of view θ2 (step S17). This operation may be performed in step S14. Subsequently, the imaging data processing unit 109 determines whether or not the operator's finger is detected in the imaging data obtained in step S17 (step S18).

When the operator's finger is detected in step S18 (step S19, YES), the ToF data processing unit 107 calculates the distance to the operator's finger (step S20). Then, the camera 108 transfers the imaging data obtained in step S17 (or S14) to the input determination unit 111, and the ToF data processing unit 107 transfers the calculation result in step S20 to the input determination unit 111 (step S21). . Of course, at this time, the calculation result obtained in step S<b>20 may be temporarily held in the RAM 103 and read from the RAM 103 by the input determination unit 111 .

Based on the appropriate positional information regarding the virtual operation space 19 obtained in step S16 and the positional information of the operator's finger obtained in step S21, the input determination unit 111 determines the positions and movements of the operator's fingers. It is determined whether or not it is within the operation space 19 (step S22). If it is determined in step S22 that the operator's finger is within the virtual operation space 19 (step S23, YES), the input determination unit 111 regards the detected finger movement of the operator as an operation command to the smartphone. to decide. Then, the CPU 101 performs processing based on the operation instruction (step S24).

Note that the ToF data processing unit 107 uses the imaging data acquired by the camera 108 to identify specific parts of the operator, such as the face and finer eyes. A distance may be calculated. An example of this case will be described with reference to FIG. 4B. FIG. 4B is a modification of the method described with reference to FIG. 4A, and relates to the case where the ToF data processing unit 107 measures the distance to a specific site using imaging data.

As shown in the figure, when the imaged data processing unit 109 detects, for example, eyes of the operator based on the imaged data, the imaged data processing unit 109 transmits data on the positions of the eyes to the ToF data processing unit 107 ( step S30). Then, the ToF data processing unit 107 calculates the distance to the eye based on the data regarding the position of the eye received from the imaging data processing unit 109 (step S31). In this way, the ToF data processing unit 107 may use imaging data. Note that the camera 108 may be a ToF three-dimensional image sensor, and for example, the ToF data processing unit 107 and imaging data processing unit 109 described with reference to FIG. 3A may be integrated. Any configuration is not limited as long as it can calculate the distance to a predetermined part of the operator, for example, the face, and specify the position of the predetermined part of the operator, for example, the eyes.

A specific example of the above operation will be described below together with the concept of the determination table 121. First, the determination table will be described with reference to FIG. FIG. 5 is a conceptual diagram of the determination table 121. As shown in FIG. As illustrated, the determination table 121 includes the position of the face (eg, the eyes in this example) in the imaging range of the imaging data obtained by the imaging data processing unit 109, the horizontal axis of the smartphone 10 obtained by the gyro data processing unit 105, for example. and the distance L to the operator's face (eyes in this example) obtained by the ToF data processing unit 107, information about the position (space coordinates) of the virtual operation space 19 is held.

As shown in FIG. 6A, the positions of the eyes in the imaging range are three regions A, B, and C in the height direction (for example, the longitudinal direction of the housing of the smartphone 10) in the imaging range 130 obtained by the camera 108. are categorized. For example, area A is the upper ⅓ area of the imaging range 130 , area B is the central ⅓ area of the imaging range 130 , and area C is the lower ⅓ area of the imaging range 130 . Note that FIG. 6A is only an example, and it is not necessarily divided into three in the height direction, it may be divided into two or four or more, and it is divided into a plurality of regions in the width direction instead of the height direction. Alternatively, it may be divided in both the height direction and the width direction.

The angle θ1 of the smartphone 10 is, for example, the angle of the smartphone 10 with respect to the horizontal axis, as shown in FIG. 6B. More specifically, it is the angle formed between the horizontal axis and the back surface of the smartphone 10 . In the example of FIG. 5, the inclination θ1 is classified into three ranges of 0° or more and less than 30°, 30° or more and less than 60°, and 60° or more and 90° or less. θ1=0° is a state in which the smartphone 10 is parallel to the horizontal axis, and θ1=90° is a state in which the smartphone 10 is upright with respect to the horizontal axis as shown in FIG. 6B. In the example of FIG. 5, the range of the angle θ1 is 0° or more and 90° or less, but the range may be narrower or wider than this. Also, the reference axis for the angle θ1 is not limited to the horizontal axis.

The distance L is the distance to the operator's face measured by the ToF sensor 106, and may be, for example, the distance to the eyes of the operator. In the example of FIG. 5, the distance L is classified into four ranges: 0 cm or more and less than 15 cm, 15 cm or more and less than 30 cm, 30 cm or more and less than 45 cm, and 45 cm or more.

The determination table 121 holds data on the position (spatial coordinates) of the virtual operation space 19 for each combination of conditions. For example, when the position of the eye is within the region A and the angle θ1 is within the range of 0°≦θ1<30°, the following position data is held according to the distance L.
・0cm≦L<15cm: “Aa-1”
・15cm≦L<30cm: “Aa-2”
・30cm≦L<45cm: “Aa-3”
・45 cm ≤ L: “A-a-4”
Further, when θ1 is within the range of 30°≦θ1<60°, the following position data is held according to the distance L.
・0cm≦L<15cm: “Ab-1”
・15cm≦L<30cm: “A-b-2”
・30cm≦L<45cm: “Ab-3”
・45 cm ≤ L: “A-b-4”
Further, when θ1 is within the range of 60°≦θ1≦90°, the following position data is held according to the distance L.
・0cm≦L<15cm: “Ac-1”
・15cm≦L<30cm: “Ac-2”
・30cm≦L<45cm: “Ac-3”
・45 cm ≤ L: “Ac-4”
The above is the same for regions B and C as well. Information indicating the position of the virtual operation space 19 is, for example, “Aa-1” indicated by double quotations in the above description. More specifically, when the display surface of the display 12 is a two-dimensional plane of the X axis and the Y axis, and the direction perpendicular to the display surface is the Z axis, "Aa-1" is the direction along the Z axis. Information such as the distance between the display 12 and the virtual operation space 19 in the virtual operation space 19, the X-axis coordinates and Y-axis coordinates of the operation surface by the operator in the virtual operation space 19, or the angle of the virtual operation space 19 with respect to the horizontal axis, for example. including. For example, the above position data includes the eyes of the operator, a predetermined position (for example, near the center) of the virtual operation space 19, and a predetermined position (for example, a numeric keypad) of the actual input screen displayed on the display 12. (near the center) is coordinate information that forms a straight line.

For example, when the numeric keypad 18 is displayed on the display 12 as shown in FIG. The coordinate information is such that it is aligned with the area for receiving the number "5" or "8" on the ten key 18 above. Although the example of FIG. 2A shows the case of the numeric keypad 18, a QWERTY layout keyboard may also be used. In this case, the operator's eyes, the area in the virtual manipulation space 19 that receives the keys "G", "H", "J", or "K", and the keys "G", "H" on the display 12 , “J”, or “K” may be selected so as to be aligned with the region. Also, the screen displayed on the display 12 may allow the operator to select two areas of "YES" and "NO", or select one area such as "OK". It may be something that causes Regardless of what kind of screen is displayed on the display 12, a specific area on the screen displayed on the display 12, an area corresponding to this specific area in the virtual operation space 19, and an operation The virtual operation space 19 may be provided so as to be aligned with the position of the person's eyes. In other words, when the operator taps a certain area in the virtual operation space 19, coordinate information that can realize a positional relationship that allows the operator to recognize that the area intended by the operator on the display screen of the display 12 has correctly received the input. If it is By using the determination table 121 in this way, the appropriate position of the virtual manipulation space 19 can be easily obtained in step S16.

Several specific examples of the method of determining the position of the virtual operation space 19 using the determination table 121 will be described first with reference to FIG. 7A. FIG. 7A is a cross-sectional view of the smartphone 10 according to this embodiment, showing the position of the virtual operation space 19A corresponding to the angle of the smartphone 10 with respect to the horizontal axis and the position of the operator's eyes.

In the example of FIG. 7A , the smartphone 10 has an angle θ1 of 30° with respect to the horizontal axis, the distance L measured by the ToF sensor 14 is 20 cm, and the operator's eye 200 is positioned at the position of the camera 13. The case is shown in the area B in the imaging range 130 . In this case, the position data "Bb-2" corresponds to the determination table 121 described with reference to FIG. Therefore, the position determination unit 110 determines the position of the virtual operation space 19A based on the position data "Bb-2", and the input determination unit 111 detects the operator's finger in the virtual operation space 19A. . It should be noted that the area of the operation surface in the virtual operation space 19 may be set larger than the display area of the numeric keypad 18 on the display 12 in order to facilitate key input by the operator.

The example of FIG. 7B shows a case where the position of the operator's eyes 200 has moved to area C in the imaging range 130 in FIG. 7A. In FIG. 7B, for reference, the virtual operation space 19A determined in the case of FIG. 7A is indicated by a dashed line.

As shown, the distance L measured by the ToF sensor 14 is 35 cm. In this case, the position data "Cb-3" corresponds to the determination table 121 described with reference to FIG. Therefore, the position determination unit 110 determines the position of the virtual operation space 19B based on the position data “Cb-3”, and the input determination unit 111 detects the operator's finger in the virtual operation space 19B. .

When the examples of FIGS. 7A and 7B are compared, if the virtual operation space 19A exists at the position shown in FIG. 7A regardless of the position of the operator's eyes shown in FIG. 7B, as shown in FIG. 7B When the operator moves his/her finger to the position of the virtual operation space 19A with the intention of tapping, for example, the central portion in the Y direction of the numeric keypad 18 on the display 12, the operator's line of sight is shifted compared to the case of FIG. 7A. Therefore, in the virtual operation space 19A, it is determined that the portion below the central portion has been tapped, which causes an erroneous input. Specifically, the smartphone 10 may recognize that the operator intended to select the number "4" on the numeric keypad, but actually selected the symbol "0" located below it. However, in this embodiment, the position of the virtual operation space 19 is changed according to the angle θ1, the distance L, and the position of the operator's eyes. In the example of FIG. 7B, a virtual operation space 19B is provided at a position corresponding to the deviation of the line of sight. This can reduce the occurrence of erroneous input by the operator.

Fig. 7C shows another example. As illustrated, the smartphone 10 has an angle θ1 of 60° with respect to the horizontal axis. At this time, for example, when the operator's eyes 200 are positioned in front of the numeric keypad 18 on the display 12, a virtual operation space 19C indicated by broken lines in FIG. 7C is provided. On the other hand, the position of the operator's eye 200 changed, the distance L measured by the ToF sensor 14 was 45 cm, and the position of the operator's eye 200 was in the area A in the imaging range 130 of the camera 13. do. In this case, the position data "Ac-4" corresponds to the determination table 121 described with reference to FIG. Therefore, the position determining section 110 determines the position of the virtual operation space 19D based on the position data "Ac-4". That is, since the operator looks down on the smartphone 10 from above rather than from the front, the virtual operation space 19D is also provided above the virtual operation space 19C along the Y direction according to this positional difference. In the example of FIG. 7C, the distance between the display 12 and the virtual operation space 19D and the distance between the display 12 and the virtual operation space 19C in the Z-axis direction also change. This is determined by the determination table 121, but is not limited as long as the virtual operation space 19 is provided at a position where input is easier based on the viewpoint of the operator. However, in the case of the smartphone 10, the virtual operation space is preferably positioned at a distance of, for example, 10 cm from the smartphone 10. Of course, the angle of the virtual operation space 19D with respect to the horizontal axis may also differ from that of the virtual operation space 19C. This also applies to the example of FIG. 7B.

Also in this example, if the operator taps the virtual operation space 19C, an erroneous input may occur. However, by changing the virtual operation space 19C to the virtual operation space 19D according to the change in the eye position of the operator, it is possible to suppress the occurrence of erroneous input.

<Second embodiment>
Next, an electronic device according to the second embodiment will be described. This embodiment adjusts the position of the virtual operation space 19 by focusing on individual differences in the operator's parallax (difference in visual acuity between the left eye and the right eye). Only points different from the first embodiment will be described below.

FIG. 8A is a hardware configuration diagram showing the internal configuration of the smartphone 10 according to this embodiment, and corresponds to FIG. 3A described in the first embodiment. The configuration according to the present embodiment differs from FIG. 3A described in the first embodiment in that the smartphone 10 further includes a calibration control unit (adjustment unit) 140 and calibration data 122 is held in the RAM 103. It is in.

The calibration control unit 140 calculates correction data for adjusting the position of the virtual operation space 19 determined by the method described in the first embodiment based on the parallax of the operator as described above. The calibration data 122 is adjustment data for the position of the virtual operational space 19 based on the correction data calculated by the calibration control section 140 . These will be described later in detail using specific examples.

Note that some hardware in FIG. 8A may be implemented by software, as in the first embodiment. 8B and 8C are respectively a hardware configuration diagram showing the internal configuration of smartphone 10 according to the modification of FIG. 3A and a functional block diagram of CPU 101 when program 120 is executed. As illustrated, the functions of the calibration control unit 140 may be realized by the CPU 101 when the program 120 is executed.

FIG. 9 is a flowchart showing operations of the smartphone 10 according to this embodiment. The following description will focus on the operation of adjusting the position of the virtual operation space 19 by the calibration control unit 140 . The operations of FIG. 9 may be performed, for example, when the smartphone 10 is powered on for the first time by the operator, or every time the power is turned on, and/or from a setting command within the smartphone.

As illustrated, the calibration control unit 140 notifies the operator of the start of calibration by displaying a message (for example, a message such as "The input surface will be adjusted now") on the display 12. (Step S30). Subsequently, the calibration control section 140 displays data for calibration on the display 12 (step S31). Further, the calibration control unit 140 prompts the operator to adjust the angle of the smartphone 10 and the position of the operator's face to appropriate positions (step S32). Specifically, for example, the operator's eye position is on a substantially vertical line from the center of the calibration data on the display 12, in other words, the operator's line of sight faces the center of the calibration data. Then, a message for the operator is displayed on the display 12 . Then, when the calibration control unit 140 determines that the angle of the smartphone 10 and the position of the operator's face are appropriate based on the detection results from the gyro sensor 104, the ToF sensor 106, and the camera 108, The calibration control unit 140 prompts the operator to tap a specific portion of the data for calibration in the virtual operation space (step S33). Also at this time, the calibration control unit 140 displays a message such as, for example, please tap a specific place on the display 12 .

In addition, in parallel with the processing of steps S30 to S33, steps S10 to S21 described with reference to FIGS. 4A and 4B in the first embodiment (in the case of the processing of FIG. Steps S30 and S31) are executed. Note that step S16 is completed at least before step S33 is executed.

After step S33, when the operator taps the virtual operation space 19 (steps S17 to S21), the calibration control unit 140 changes the position obtained in step S16 and the position actually tapped by the operator in the virtual operation space 19. Detect the difference from the determined position. Then, this is stored in the RAM 103 as the calibration data 122 (step S34). After that, the calibration control unit 140 notifies the operator of the completion of the calibration (step S35). After that, for example, the position determination unit 110 of the smartphone 10 corrects the virtual operational space 19 obtained in step S16 using the calibration data 122, and determines the position of the virtual operational space 19 (step S36).

A specific example of the calibration operation will be described with reference to FIGS. 10A and 10B. 10A and 10B are external perspective views of the smartphone 10 during execution of the calibration operation.

First, the calibration control unit 140 displays the start of calibration on the display 12, and then displays data 20 for calibration on the display 12 as shown in FIG. 10A. In the example of FIG. 10A, the letters "A" to "E" are displayed at the four corners and the center of the (3×3) display area, but the calibration data 20 is not limited to this example. do not have. Subsequently, after the calibration control unit 140 displays on the display 12 a message urging that the positions of the smartphone 10 and the operator are appropriate, in the example of FIG. A message 21 prompting the user is displayed on the display 12 . Based on this, the virtual operation space 19E accepts the operator's finger motion. Note that the area of the operation surface in the virtual operation space 19 may be set larger than the display area of the calibration data 122 on the display 12 in order to facilitate key input by the operator.

At this time, as shown in FIG. 10A, it is assumed that the operator's tap position is shifted to the left (X direction) of the actual "E" area in the virtual operation space 19E. Then, as shown in FIG. 10B, the calibration control unit 140 recognizes that the operator has a deviation between the actual virtual operation space 19E and the position recognized by the operator due to the parallax. Therefore, the calibration control unit 140 detects the amount of deviation δ between the actually tapped position and the current virtual operation space and the direction thereof. These data are stored in the RAM 103 as calibration data 122 . After that, when the smartphone 10 receives an input from the operator in the virtual operation space 19, the smartphone 10 sets the position of the virtual operation space 19 to a position shifted leftward by δ from the position obtained in step S16.

By the above calibration operation, erroneous input based on parallax etc. by the operator can be suppressed.

<Modifications, etc.>
As described above, according to the configuration according to the embodiment, erroneous input in the non-contact smartphone 10 can be suppressed. In addition, although the smart phone 10 was mentioned as an example in the said embodiment, it is widely applicable to other electronic devices. For example, it can be applied to tablet PCs, televisions, automatic ticket vending machines for trains and movies, automatic check-in machines at airports, and cash registers at restaurants and the like. In addition, when the positions of these electronic devices and their displays are fixed, the angle of the display is likely to be constant in many cases. It may be the case that angle θ1 need not be taken into account in determining the position of . However, even if the electronic device is fixed, the angle of the electronic device itself may change when, for example, it is mounted on a ship or an aircraft. preferably.

Further, in the above embodiment, the case where the operator's input is performed with a finger and the finger is detected by the camera 108 has been described as an example. However, input by the operator is not limited to a finger, and is not limited as long as it is a member capable of designating a specific area, such as a touch pen (stylus pen). Then, when the captured image data processing unit 109 detects these members in advance, it may be recognized that "a finger has been detected" in step S19 in FIGS. 4A and 4B.

Also, in step S14 in FIGS. 4A and 4B, the imaging data may contain a plurality of human faces. At this time, the captured data processing unit 109 may select one face using, for example, face authentication processing. That is, the imaging data processing unit 109 causes the RAM 103 to hold the face data of the operator previously photographed by the camera 13, and when a plurality of faces are recognized in step S14, the face data held in advance in the RAM 103 is stored in the RAM 103. Authentication processing may be performed using the face data, and the position of the authenticated face data may be calculated in step S15.

The operations described in the first and second embodiments can be implemented by executing the program 120, for example. For example, even if the program 120 is not held in the electronic device at the time of purchase by the user, the program 120 can be downloaded via the Internet or the like and held in the ROM 102 or the RAM 103, so that after the purchase of the electronic device, the above first and It is possible to realize the operation described in the second embodiment.

Although the present embodiment has been described in detail as above, those skilled in the art will easily understand that many modifications that do not substantially deviate from the novel matters and effects of the present embodiment are possible. . Accordingly, all such modifications are intended to be included within the scope of this disclosure. For example, a term described at least once in the specification or drawings together with a different broader or synonymous term can be replaced with the different term anywhere in the specification or drawings. All combinations of this embodiment and modifications are also included in the scope of the present disclosure. Also, the configuration and operation of the electronic devices described in the above embodiments are not limited to those described in the present embodiment, and various modifications are possible.

Claims

An electronic device capable of accepting contactless information input by an operator,
a first detection unit that detects the tilt of the electronic device with respect to a predetermined reference axis;
a measuring unit that measures the distance from the operator;
a second detection unit that detects the position of the operator's eyes;
Non-contact information input by the operator based on the inclination detected by the first detection unit, the distance measured by the measurement unit, and the position of the eye detected by the second detection unit An electronic device comprising: an acquisition unit that acquires a position in a virtual operation space for recognizing a motion.
further comprising a display capable of displaying an image for prompting the operator to input the information;
The virtual operation space is a space separated from the display, and when the information input operation of the operator is detected in the space, it is determined that the information has been input in the image displayed on the display. 2. The electronic device of claim 1, which is a space that can be determined.
The electronic device according to claim 2, wherein the information input action of the operator is detected by the position of the operator's finger in the virtual operation space.
The acquisition unit comprises: a first area for inputting first information in the image displayed on the display; a second area corresponding to input of the first information in the virtual operation space; 4. The electronic device according to claim 2, wherein the position of the virtual operation space is obtained so that the position of the eye is aligned with the position of the eye.
2. The acquisition unit acquires the position of the virtual operational space based on a table that holds a relationship between the tilt, the distance, the position of the eye, and the position of the virtual operational space. 5. The electronic device according to any one of 4.
prompting the operator to select a specific region in the virtual operating space acquired by the acquisition unit, and based on the relationship between the region selected by the operator's selection motion and the specific region; 6. The electronic device according to any one of claims 1 to 5, further comprising an adjustment unit that adjusts the position of said virtual operation space.