JP2015230496A - Electronic equipment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide electronic equipment capable of reducing improper operations by a user without changing a layout of a screen.SOLUTION: A process that a CPU of electronic equipment executes includes: a step S520 of detecting a hand of a user in an input image obtained from a camera; a step S522 of generating a hand image showing a shape of the detected hand of the user; a step S524 of displaying the hand image over an operation screen of a touch panel, and displaying the hand image on a touch panel associatively with a movement of the hand of the user; and a step S532 of executing an instruction corresponding to a touch operation at the position of the hand image on the touch panel.

Description

  The present disclosure relates to control of an electronic device, and more particularly to control of an electronic device having a camera.

  In recent years, various techniques have been developed to reduce user's erroneous operations on icons and the like. For example, Patent Document 1 discloses a car navigation system for reducing erroneous operations of icons displayed on a display unit. The car navigation system is intended to reduce erroneous operations by allowing the user to recognize where his / her finger is located on the screen. As a specific means for achieving this object, the car navigation system displays an image obtained by photographing a hand operating the operation unit with a camera from above on the display unit. At that time, the car navigation system changes the display position of the icon in accordance with the position of the finger displayed on the display unit.

JP 2007-286696 A

  However, every time the position of the user's hand changes, if the display position of the icon changes, the user needs to grasp the contents of the screen each time. On the contrary, there is a possibility that the user's erroneous operations increase. For this reason, it is desired to reduce the erroneous operation of the electronic device by a method different from the method disclosed in Patent Document 1.

  The present disclosure has been made to solve the above-described problems, and an object of the present disclosure is to provide an electronic device that can reduce a user's erroneous operation without changing the screen layout. is there.

  An electronic device according to an embodiment includes a touch panel, a camera provided on a surface opposite to the display screen of the touch panel, and a control device for controlling the electronic device. The control device includes a hand detection unit for detecting a user's hand from an input image obtained from the camera, a generation unit for generating a hand image having a shape of the user's hand detected by the hand detection unit, and a touch panel A display control unit for displaying a hand image on the operation screen and displaying the hand image on the touch panel in conjunction with the movement of the user's hand, and a command corresponding to the touch operation at the position of the hand image in the touch panel And an execution unit for executing.

  In one aspect, it is possible to reduce a user's erroneous operation on the electronic device without changing the screen layout.

  The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention taken in conjunction with the accompanying drawings.

It is the figure which showed typically the external appearance of the smart phone according to 1st Embodiment. It is a figure which shows a mode that the user is operating the smart phone according to 1st Embodiment. It is a block diagram which shows the main hardware constitutions of the smart phone according to 1st Embodiment. It is a block diagram which shows the function structure of the smart phone according to 1st Embodiment. It is a flowchart showing a part of process which the smart phone according to 1st Embodiment performs. It is a figure which shows a mode that the user is operating the smart phone according to 2nd Embodiment. It is a block diagram which shows the function structure of the smart phone according to 2nd Embodiment.

  Hereinafter, the present embodiment will be described with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  In addition, below, the detail of the smart phone which functions as an example of an electronic device is demonstrated, However, An electronic device is not limited to a smart phone. For example, the electronic device may include a device having a tablet terminal, a digital camera, an electronic dictionary, a PDA (Personal Digital Assistant), a game machine, and other cameras. Moreover, each embodiment described below may be selectively combined.

<Introduction>
First, in order to facilitate understanding of each embodiment described below, problems of recent smartphones will be described.

  In recent years, smartphones equipped with a touch panel have become widespread. Along with this, the display screen of the touch panel has become high-definition, and the display size of objects such as icons and links that the user can perform touch operations becomes very small compared to the size of the display screen of the touch panel. ing. Along with this, an erroneous operation occurs in which an object different from the object intended by the user is touched.

  As an example of a method for preventing such an erroneous operation, there is a method of enlarging and displaying a part of the display screen of the touch panel. However, in this method, the user has to perform an operation for enlarging the screen, and the operability of the touch panel is not good. In addition, as a method for preventing an erroneous operation of the touch panel, there is a method in which a user touches an object with a pen tip using a touch pen. However, with this method, the user needs to carry the touch pen separately, and the portability of the smartphone is impaired. The smartphone according to each embodiment described below can solve these problems.

<First Embodiment>
[Overview]
With reference to FIG. 1 and FIG. 2, the outline | summary of the smart phone 100 according to 1st Embodiment is demonstrated. FIG. 1 is a diagram schematically illustrating the appearance of the smartphone 100. FIG. 2 is a diagram illustrating a state in which the user is operating the smartphone 100.

  As shown in FIG. 1A and FIG. 1B, the smartphone 100 includes a touch panel 4 and a camera 5. The camera 5 is provided on the opposite surface of the display screen of the touch panel 4. On the touch panel 4, an object associated with the function of the smartphone 100 is displayed. The object includes an icon, a button, a link, other images associated with the function of the smartphone 100, and the like.

  As shown in FIG. 2A, when the user moves his / her hand in front of the camera 5, the smartphone 100 moves the user's hand in an image obtained from the camera 5 (hereinafter also referred to as “input image”). To detect. The smartphone 100 generates a hand image 30 having the shape of the detected user's hand. The hand image 30 may be an image itself obtained by cutting the user's hand from the input image, or may be a processed image having the same shape as the detected hand. The smartphone 100 displays the generated hand image 30 so as to overlap the operation screen 32. At this time, the smartphone 100 displays the hand image 30 so as to be smaller than the size of the actual hand of the user.

  The smartphone 100 sequentially generates the hand image 30 from the repeatedly obtained input image and updates the display of the hand image 30 to display the hand image 30 in conjunction with the movement of the user's hand. For this reason, the user can operate the hand image 30 in accordance with the movement of his / her hand. The user superimposes the hand image 30 on the position of the target object, and performs a predetermined operation on the smartphone 100, whereby the smartphone 100 is associated with the object displayed at the position of the hand image 30. Perform the function. That is, the smartphone 100 performs the same process (hereinafter, “touch process”) as that performed when a touch operation is detected by performing a user operation (hereinafter, also referred to as “alternative operation”) instead of the touch operation. (Also called). As an example of the alternative operation, as shown in FIG. 2B, there is an operation of bringing the thumb and index finger into contact with each other.

  Thus, even if the user operates the hand image 30 displayed in a size smaller than his / her hand as intended and the objects are displayed densely, the smartphone 100 It becomes possible to reduce erroneous operation of the touch panel 4 by the user.

[Hardware configuration]
With reference to FIG. 3, an example of a hardware configuration of smartphone 100 according to the first embodiment will be described. FIG. 3 is a block diagram illustrating a main hardware configuration of the smartphone 100. As shown in FIG. 3, the smartphone 100 includes a ROM (Read Only Memory) 1, a CPU (Central Processing Unit) 2, a RAM (Random Access Memory) 3, a touch panel 4, a camera 5, and a distance sensor 9. A network interface (I / F) 10 and a storage device 20.

  The ROM 1 stores an operating system (OS), an initial program (boot program) executed when the smartphone 100 is activated, and the like. The CPU 2 controls the operation of the smartphone 100 by executing various programs such as an operating system and a control program for the smartphone 100 stored in the ROM 1 and the storage device 20. The RAM 3 functions as a working memory for executing the program by the CPU 2 and temporarily stores various data necessary for executing the program.

  Touch panel 4 includes a monitor 4A and a touch sensor 4B. The touch sensor 4B is provided so as to overlap the monitor 4A. Thereby, the touch panel 4 can detect the touch position with respect to the monitor 4A. The touch position is detected as coordinates on the touch panel 4, for example. The touch panel 4 is realized by, for example, a resistance film method, a capacitance method, or other methods capable of detecting contact with the touch panel 4.

  The camera 5 is provided on the opposite surface of the display screen of the touch panel 4. The camera 5 includes a lens, a complementary metal-oxide semiconductor (CMOS) or other imaging element, and an autofocus mechanism. The camera 5 captures a subject included in the field of view and generates an input image in response to a user operation. The generated input image is output to the CPU 2.

  The distance sensor 9 detects a distance between the smartphone 100 and the user's hand (hereinafter also referred to as “user distance”). The distance sensor 9 is provided on the surface opposite to the display surface of the touch panel 4. That is, the distance sensor 9 is provided on the same surface as the surface on which the camera 5 is provided. The distance sensor 9 includes, for example, an ultrasonic sensor, an infrared sensor, and other sensors that can detect a user distance. In one aspect, the distance sensor 9 is built in the camera 5. As a technique for realizing this, there is an image plane phase difference AF (Auto Focus) technique. Distance sensor 9 is not an essential component in smartphone 100 according to the present embodiment. The distance sensor 9 is used for the smart phone 100A according to 2nd Embodiment demonstrated below, for example.

  The network I / F 10 transmits and receives data to and from other communication devices via the antenna 10A. Other communication devices are, for example, smart phones, personal computers, server devices, and other electronic devices having communication functions. Smartphone 100 may be configured to download a program for realizing various processes according to the present embodiment via antenna 10A.

  The storage device 20 includes, for example, a storage medium such as eMMC (Embedded MultiMediaCard). The eMMC includes a NAND flash memory and a control circuit. Storage device 20 stores a program and the like for realizing various processes according to the present embodiment. Further, the storage device 20 may store a program such as an operating system.

[Function configuration]
An example of the function of the smartphone 100 will be described with reference to FIG. FIG. 4 is a block diagram illustrating a functional configuration of the smartphone 100. The CPU 2 includes a hand detection unit 210, a generation unit 220, a display control unit 230, a contact detection unit 240, and an execution unit 250.

  The hand detection unit 210 detects the user's hand from the input image obtained from the camera 5. As a method for detecting the user's hand from the input image, for example, an image processing technique such as template matching is used. The template matching in the present embodiment is a method of acquiring the position of the hand in the input image by searching for the hand image 30 from the input image with reference to the hand image 30 registered in advance in the storage device 20 or the like. . As an example, the position of the user's hand in the input image is detected as a coordinate value in the input image. The hand detection unit 210 outputs the detected position of the user's hand to the generation unit 220.

  The generation unit 220 generates the hand image 30 having the shape of the user's hand detected by the hand detection unit 210. As an example, the generation unit 220 extracts an edge within a predetermined range from the hand position (for example, coordinates) in the input image detected by the hand detection unit 210, and cuts out the input image along the extracted edge portion. The hand image 30 is generated. The hand image 30 may be an image itself obtained by cutting the user's hand from the input image, or may be a processed image having the contour of the user's hand in the input image. The generation unit 220 outputs the generated hand image 30 to the display control unit 230 and the contact detection unit 240.

  In one aspect, the generation unit 220 sequentially generates the hand images 30 in the order in which the input images are obtained. Thereby, the production | generation part 220 can produce | generate the hand image 30 which has the same shape as the user's hand which the camera 5 image | photographs in real time. In another aspect, the generation unit 220 generates the hand image 30 with a size corresponding to the size of the display screen of the touch panel 4. Thereby, the hand image 30 is displayed in a size suitable for operating the display screen, and erroneous operation of the touch panel 4 can be reduced. In still another aspect, the generation unit 220 generates the hand image 30 with a size corresponding to the size of the object displayed on the display screen of the touch panel 4. Thereby, the hand image 30 is displayed in a size suitable for operating the object displayed on the display screen, and erroneous operation of the touch panel 4 can be reduced.

  The display control unit 230 displays the hand image 30 generated by the generation unit 220 so as to overlap the operation screen 32 of the touch panel 4. The display position of the hand image 30 on the operation screen 32 is determined by the position of the hand image 30 in the input image. That is, the display position of the hand image 30 is determined by the position of the user's hand with respect to the camera 5. In a certain situation, the display control unit 230 updates the display position of the hand image 30. Thereby, the display control unit 230 can link the movement of the user's hand with the display of the hand image 30. That is, when the user moves his / her hand up / down / left / right within the field of view of the camera 5, the display position of the hand image 30 also changes up / down / left / right according to the movement.

  The contact detection unit 240 detects that a first finger (for example, an index finger) included in the hand image 30 and a second finger (for example, a thumb) different from the first finger have approached or contacted each other. As a more specific process, first, the contact detection unit 240 detects each finger of the user included in the hand image 30. As a technique for detecting the user's finger from the hand image 30, for example, an image processing technique such as template matching is used. In this case, the contact detection unit 240 searches each finger image from the hand image 30 based on each finger image registered in advance in the storage device 20 or the like. As a result, the contact detection unit 240 can acquire the position of each finger in the hand image 30. As an example, the position of the user's finger in the hand image 30 is detected as a coordinate value in the input image. The contact detection unit 240 detects the proximity or contact between fingers based on the fact that the distance between fingers calculated from the coordinate value of each finger is within a predetermined distance.

  The execution unit 250 executes a command corresponding to the touch operation at the position of the hand image 30 in the touch panel 4. That is, the execution unit 250 executes the same process (that is, touch process) as the process executed when the CPU 2 detects a touch operation. The touch process is executed when the smartphone 100 detects a user operation (that is, an alternative operation) that takes the place of the touch operation.

[Alternative operation]
Various alternative operations can be considered as triggers for the execution unit 250 to perform touch processing. Examples of alternative operations are given below.

(Example of alternative operation 1)
An alternative operation includes, for example, the user's first finger (eg, forefinger) and second finger (eg, thumb) making a predetermined movement in front of the camera. That is, when the movements of the first finger and the second finger coincide with a predetermined movement, the execution unit 250 executes a touch process. The predetermined action includes, for example, bringing the first finger and the second finger into close proximity or contact with each other. More specifically, when the user does not want the smartphone 100 to perform the touch process, the user leaves a space between the first finger and the second finger in front of the camera. In addition, when the user wants the smartphone 100 to perform touch processing, the user touches or approaches the first finger and the second finger in front of the camera. As described above, whether or not the first finger and the second finger are in contact with each other is detected by the contact detection unit 240.

  The first finger and the second finger are arbitrarily selected from five fingers by the user. A method for selecting the first finger and the second finger is not particularly limited. For example, the smartphone 100 may be configured such that the user can select the first finger and the second finger in a user interface that selects an arbitrary finger from five fingers. In one aspect, the user selects a finger indicating a place to be touched as a first finger, and selects a finger instructing a touch operation as a second finger. In this case, when the user performs an alternative operation, the execution unit 250 executes a touch process at a position where the first finger included in the hand image 30 is pointing.

  In the above description, the alternative operation has been described by taking the operation of the two fingers of the first finger and the second finger as an example. However, the alternative operation is predetermined for one finger of the user. Operations may be included. Alternatively, the alternative operation may include a predetermined action of the user's three or more fingers.

(Example 2 of alternative operation)
As another example, the alternative operation includes the user rotating his / her hand in front of the camera 5. As an example, the execution unit 250 executes the touch process when the position of the fingertip of the first finger indicating the touch position does not change and rotation of the back of the hand is detected. At this time, the user turns the back of the hand toward the camera 5 when the smartphone 100 does not want to execute the touch process. From this state, when the user rotates the back of the hand, the execution unit 250 executes the touch process.

  In the above description, the operation in which the user rotates the back of the hand in front of the camera 5 has been described as the operation intended to execute the touch process. However, the operation intended to execute the touch process is not limited to this. For example, the operation intended to execute the touch process includes an operation of turning the back of the hand toward the camera 5 and other user's hand.

(Example 3 of alternative operation)
As another example, in the alternative operation, the user touches a specific place on the screen of the touch panel 4 using the hand on the side holding the smartphone 100, and the user touches a specific place of the smartphone 100. And the user may press an operation button (not shown) provided on the smartphone 100.

(Example 4 of alternative operation)
As another example, the alternative operation includes placing the user's own hand in a predetermined shape in front of the camera 5. That is, when the hand image having various hand shapes is registered in the smartphone 100 in advance, and the image obtained from the camera 5 matches the shape of the hand image registered in advance, the execution unit 250. Performs the touch process. The above-described alternative operation example 1 can also be regarded as an example of this example (alternative operation example 4). Specific examples of the alternative operation according to this example include the following (a) to (d), but the alternative operation is not limited to these.

  (A) The alternative operation according to this example includes an operation of raising a different finger (second finger) such as a thumb from a state where only one first finger indicating a place to be touched is raised. When the smartphone 100 detects this operation, the smartphone 100 performs a touch process.

  (B) In addition, the alternative operation according to the present example includes an operation of bending the second finger from a state where a finger (second finger) different from the first finger is raised in advance. When the smartphone 100 detects this operation, the smartphone 100 performs a touch process.

  (C) In addition, the alternative operation according to the present example includes an operation of using a plurality of second fingers different from the first in (a) and (b). For example, an operation in which all fingers other than the first finger are bent and only one first finger is raised from a state where the hand is opened can be cited. Moreover, the operation | movement which bends one or more fingers from the state which stood the thumb and the middle finger apart from the index finger (1st finger) is mentioned. The smartphone 100 executes the touch process when detecting these operations.

  (D) In addition, when an operation using a plurality of fingers different from the first finger matches a hand image registered in advance in the terminal, the smartphone 100 performs a touch process. For example, in front of the camera 5, the user uses two fingers of the thumb and middle finger separately from the index finger (first finger) to form a ring with two fingers (that is, two fingers) When the fingertip is brought into contact), the smartphone 100 executes a touch process.

[Control structure]
The control structure of the smartphone 100 will be described with reference to FIG. FIG. 5 is a flowchart showing a part of the process executed by smartphone 100. The processing in FIG. 5 is realized by the CPU 2 executing a program. In other aspects, some or all of the processing may be performed by circuit elements or other hardware.

  In step S <b> 510, the CPU 2 acquires an input image from the camera 5. In step S520, the CPU 2 determines, as the hand detection unit 210, whether or not the user's hand is included in the input image. When CPU 2 determines that the user's hand is included in the input image (YES in step S520), CPU 2 switches control to step S522. If not (NO in step S520), CPU 2 returns control to step S510.

  In step S522, the CPU 2 generates a hand image having the shape of the user's hand detected in step S520 as the generation unit 220. Preferably, the CPU 2 displays the hand image 30 by reducing it to an appropriate size within the screen of the touch panel 4. For example, the CPU 2 displays the hand image 30 in accordance with the screen size of the touch panel 4. Alternatively, the CPU 2 displays the hand image 30 according to the size of the object. In step S524, the CPU 2 displays the hand image 30 on the operation screen of the touch panel 4 as the display control unit 230, and displays the hand image 30 on the touch panel 4 in conjunction with the movement of the user's hand.

  In step S530, the CPU 2 determines whether or not the above-described alternative operation has been detected. If CPU 2 determines that an alternative operation has been detected (YES in step S530), it switches control to step S532. If not (NO in step S530), CPU 2 returns control to step S510.

  In step S532, the CPU 2 executes touch processing as the execution unit 250. As a more specific process, first, the CPU 2 detects from the hand image 30 a finger that the user intends to use for the touch operation. For example, if the hand shape in the hand image 30 is an index finger only, the CPU 2 determines that the user intends to use the index finger for a touch operation. The CPU 2 executes the touch process on the assumption that there is a touch operation at the detected tip position of the finger.

[Brief Summary]
As described above, the user can perform the same operation as the touch operation by operating the hand image 30 interlocked with the movement of his / her hand. Since the hand image 30 is linked to the movement of the user's hand, the user can operate the hand image 30 with a natural feeling. Further, since the camera 5 is provided on the opposite surface of the display screen, the hand image 30 on the back side of the hand is displayed as it is, as when the user performs a normal touch operation. For this reason, the user can operate the smartphone 100 with a natural feeling as if the touch panel 4 was directly operated using his / her finger.

  Further, the smartphone 100 displays the hand image 30 in a size suitable for the user to select an object such as an icon on the display screen of the touch panel 4. Therefore, even when a large number of objects are displayed densely, the user does not need to perform an operation for changing the screen layout such as an operation for enlarging a part of the screen. Since the user can perform the screen operation without changing the screen layout, it is possible to reduce the erroneous operation of the smartphone 100.

  Moreover, since the manufacturer of the smart phone 100 can implement | achieve the above-mentioned process using the function and components normally mounted in the general portable terminal, the manufacturing cost of the smart phone 100 can be held down. Moreover, since the manufacturer of the smart phone 100 does not need to provide other parts, such as a touch pen, in order to improve operativity, the portability of the smart phone 100 is not impaired.

<Second Embodiment>
[Overview]
Hereinafter, with reference to FIG. 6, the outline | summary of the smart phone 100A according to 2nd Embodiment is demonstrated. FIG. 6 is a diagram illustrating a state in which the user is operating the smartphone 100A. The smartphone 100A according to the second embodiment is different from the smartphone 100 according to the first embodiment in that the touch process is executed using a change in the distance to the user's hand as a trigger. Since other points such as the hardware configuration are the same as those of the smartphone 100 according to the first embodiment, description thereof will not be repeated.

  As illustrated in FIG. 6A, the smartphone 100 </ b> A includes the touch panel 4. On the touch panel 4, a hand image 30 obtained by photographing the user's hand with the camera 5 is displayed. In this state, as shown in FIG. 6B, the user moves his / her hand in a direction away from the smartphone 100A, so that the smartphone 100A performs a touch process. That is, the operation of moving his / her hand away from the smartphone 100 </ b> A becomes a user operation (that is, an alternative operation) instead of the touch operation.

  As a more specific process, the smartphone 100A calculates the distance to the user's hand (that is, the user distance) according to the size of the hand image generated by the generation unit 220 (see FIG. 4). The size of the hand image generated by the generation unit 220 is linked to the movement of the user's hand. That is, as shown in FIG. 6B, when the user moves his / her hand away from the camera 5, the size of the hand image generated by the generation unit 220 is reduced. Further, when the user brings his / her hand close to the camera 5, the size of the hand image generated by the generation unit 220 increases. Focusing on this point, the smartphone 100A calculates the user distance from the size of the hand image generated by the generation unit 220. The smartphone 100A executes the touch process when the change amount of the user distance becomes larger than a predetermined change amount.

  6A and 6B, the display size of the hand image 30 that is actually displayed is adjusted so as not to change depending on the user distance. That is, the display control unit 230 reduces or enlarges the hand image generated by the generation unit 220 according to the screen size, generates the hand image 30 having a certain size, and displays the hand image 30 on the touch panel 4. . Alternatively, the display control unit 230 may change the display size of the hand image 30 according to the user distance. In this case, the display control unit 230 increases the display of the hand image 30 as the user distance increases, and decreases the display of the hand image 30 as the user distance decreases.

[Function configuration]
With reference to FIG. 7, an example of the function of the smartphone 100A will be described. FIG. 7 is a block diagram illustrating a functional configuration of the smartphone 100A. The CPU 2 includes a hand detection unit 210, a generation unit 220, a display control unit 230, a contact detection unit 240, an execution unit 250, and a distance detection unit 260. Since hand detection unit 210, generation unit 220, display control unit 230, and contact detection unit 240 are as described above, description thereof will not be repeated.

  The distance detection unit 260 detects the distance between the user's hand and the smartphone 100A (that is, the user distance). More specifically, the distance detection unit 260 detects the user distance from the size of the hand image 30 generated by the generation unit 220. In this case, the user distance is closer as the size of the hand image 30 is larger. The smaller the size of the hand image 30, the longer the user distance. The distance detection unit 260 outputs the detected user distance to the execution unit 250. The distance detection unit 260 may detect the user distance using another method. In a certain aspect, the distance detection unit 260 may acquire the user distance from the above-described distance sensor 9 (see FIG. 3). Alternatively, the distance detection unit 260 refers to a table that defines the relationship between the size of the hand image 30 and the user distance, and acquires the user distance.

  The execution unit 250 executes a touch process at the position of the hand image on the touch panel 4 when the change amount of the user distance becomes larger than a predetermined change amount. Alternatively, the execution unit 250 may be configured to execute the touch process at the position of the hand image on the touch panel 4 when the user distance becomes larger than a predetermined distance.

  In one aspect, the execution unit 250 may acquire an adjustment value (contrast information) for adjusting the focus of the camera 5 from the camera 5 and execute the touch process according to a change in the adjustment value. When the user moves his / her hand back and forth with respect to the camera 5, the camera 5 changes the adjustment value to focus on the hand. The smartphone 100 </ b> A can determine whether or not the user has moved his / her hand back and forth in front of the camera 5 by detecting the change in the adjustment value.

[Brief Summary]
As described above, the smartphone 100 </ b> A according to the second embodiment executes the touch process by the user operation of moving the hand away from the camera 5. Since the direction in which force is applied to the touch panel when the touch operation is performed matches the direction in which the hand is moved, the user can operate the smartphone 100A with the same feeling as when the touch operation is performed. Thereby, the user can perform a more intuitive operation, and as a result, the operability of the smartphone 100A is improved.

<Third Embodiment>
[Overview]
Smartphone 100B according to the third embodiment is different from smartphone 100A according to the second embodiment in that a process different from the touch process is executed when a predetermined user action is detected. Since other points such as the hardware configuration are the same as those of smartphone 100A according to the second embodiment, description thereof will not be repeated.

  The smartphone 100B is configured such that the user can arbitrarily set the correspondence between the process executed by the smartphone 100B and the operation of the user's hand. That is, the smartphone 100B can associate various processes other than the touch process with alternative operations. As a more specific process, when the user sets the association between the process of the smartphone 100B and the alternative operation, the smartphone 100B stores information defining these correspondences in the storage device 20 or the like. When the smartphone 100B detects an alternative operation, the smartphone 100B executes a process corresponding to the detected alternative operation. There are various possible correspondences between an alternative operation and a process executed by the alternative operation. Examples thereof will be described below.

[Correspondence between alternative operation and processing executed by the alternative operation]
As an example, it is assumed that the user performs a setting for associating an operation of moving his / her hand back and forth in front of the camera 5 with a process of enlarging or reducing the display screen of the touch panel 4. In this case, when the user performs an operation of moving his / her hand away from the camera 5, the smartphone 100B enlarges the display screen. When the user performs an operation of bringing his / her hand close to the camera 5, the smartphone 100B reduces the display screen.

  As another example, it is assumed that the user performs a setting for associating an operation of moving his / her hand back and forth in front of the camera 5 with a process of scrolling the display screen of the touch panel 4. In this case, when the user performs an operation of moving his / her hand away from the camera 5, the smartphone 100B scrolls the display screen downward. Further, when the user performs an operation of bringing his / her hand close to the camera 5, the smartphone 100 </ b> B scrolls the display screen upward.

[Brief Summary]
As described above, the user can arbitrarily associate the operation of his / her hand with the process of the smartphone 100B. Thereby, since the user can customize the setting of the association between the process of the smartphone 100B and the operation for executing the process so that the user can easily use it, the operability of the smartphone 100B is further improved. .

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  1 ROM, 2 CPU, 3 RAM, 4 touch panel, 4A monitor, 4B touch sensor, 5 camera, 9 distance sensor, 10 network I / F, 10A antenna, 20 storage device, 30 hand image, 32 operation screen, 100, 100A , 100B smartphone, 210 hand detection unit, 220 generation unit, 230 display control unit, 240 contact detection unit, 250 execution unit, 260 distance detection unit.

Claims (5)

Electronic equipment,
A touch panel;
A camera provided on the opposite surface of the display screen of the touch panel;
A control device for controlling the electronic device,
The controller is
A hand detection unit for detecting a user's hand from an input image obtained from the camera;
A generating unit for generating a hand image having the shape of the user's hand detected by the hand detecting unit;
A display control unit for displaying the hand image on the operation screen of the touch panel, and displaying the hand image on the touch panel in conjunction with movement of the user's hand;
And an execution unit for executing a command corresponding to a touch operation at the position of the hand image in the touch panel. The control device further includes a distance detection unit for detecting a distance between the user's hand and the electronic device,
The electronic device according to claim 1, wherein the execution unit executes the command when a change amount of the distance detected by the distance detection unit is larger than a predetermined change amount.   The electronic device according to claim 2, wherein the distance detection unit detects a distance between the user's hand and the electronic device from a size of a hand image generated by the generation unit.   The electronic device according to claim 1, wherein the generation unit generates the hand image with a size corresponding to a size of the display screen. The control device further includes a contact detection unit for detecting that the first finger included in the hand image and a second finger different from the first finger are close to or in contact with each other,
5. The electron according to claim 1, wherein the execution unit executes the command when the contact detection unit detects proximity or contact between the first finger and the second finger. machine.

Images