JP2010102381A - Portable terminal and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a portable terminal which improves operability of input through a touch pad. <P>SOLUTION: The portable terminal includes: a touch pad which detects pressing force; and an imaging part imaging a face of the touch pad. When the touch pad detects the pressing force, a CPU of the portable terminal analyzes the latest image data which is obtained by imaging the face of the touch pad, and detects the number of fingers of a user on the face of the touch pad during pressing. Then the CPU determines a user's operation command based on the detected number of the fingers and the OS executes the determined operation command. Two kinds of operations to one object (for example, left and right click operations of a mouse) are facilitated, and operability is improved. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a portable terminal equipped with an input function using a touch pad and a program.

  A terminal in which a touch pad is mounted on a display unit of a portable terminal and input is performed by pressing a screen of the display unit has already been commercialized. However, only a simple screen press can perform processing equivalent to a left-click operation of the mouse in the computer, that is, processing such as selection / execution. In order to perform other operations such as a right-click operation of a mouse in a computer, an input command is determined according to the contact time with the screen (for example, Patent Document 1), and the contact area is large or small. In addition, an apparatus for determining an input command by detecting the number of contact points is proposed (for example, Patent Document 2).

JP2007-80291 JP 2000-284912 A

  However, in the above-mentioned Patent Document 1, there is a problem that it takes a long time to input a screen because a screen must be pressed for a predetermined time in order to input a desired operation command. In Patent Document 2, in order to input one operation command, a plurality of coordinate points on the screen must be pressed, the intuition that is the merit of touchpad input is lost, and the operation is complicated. There is a problem.

  The present invention has been made in view of the above problems, and an object thereof is to provide a portable terminal and a program that can improve the operability of input via a touch pad.

In order to achieve the above object, a mobile terminal according to the first aspect of the present invention provides:
A touchpad that detects pressing and outputs position information of the detected area;
Imaging means for imaging the surface of the touchpad;
Storage means for storing image data obtained by imaging by the imaging means;
When the touch pad detects a press, the image data stored in the storage means is analyzed, and the number of fingers detecting means for detecting the number of fingers existing on the surface of the touch pad at the time of the press When,
And an operation type determining unit that determines an operation type to be processed based on the number of fingers detected by the finger number detecting unit.

A portable terminal according to the second aspect of the present invention is
A touchpad that detects pressing and outputs position information of the detected area;
Imaging means for imaging the surface of the touchpad;
Storage means for storing image data obtained by imaging by the imaging means;
When the touchpad detects a press, the image data stored in the storage means is analyzed, and a press finger specifying means for specifying the press finger used for the press;
An operation type determining means for determining an operation type to be processed based on a positional relationship at the time of the pressing between the pressing finger specified by the pressing finger specifying means and another finger different from the pressing finger; It is characterized by providing.

  In the above case, the pressing finger specifying unit may analyze the image data for a predetermined time and specify a finger having the largest position change as the pressing finger.

A portable terminal according to the third aspect of the present invention is
A touchpad that detects pressing and outputs position information of the detected area;
Finger type specifying means for specifying the type of the pressed finger;
And an operation type determining unit that determines an operation type to be processed based on the finger type specified by the finger type specifying unit.

  In the above case, the finger type specifying means detects the fingerprint of the pressed finger, and collates the detected fingerprint with the fingerprint data of each finger of the user that is held in advance, whereby the type of the pressed finger May be specified.

A program according to the fourth aspect of the present invention is:
Computer
When the touchpad detects a press, it analyzes the image data obtained by the imaging means and stored in the storage means to detect the number of fingers present on the surface of the touchpad at the time of the press Finger number detection means to perform,
It is made to function as an operation type discrimination means for discriminating an operation type to be processed based on the number of fingers detected by the finger number detection means.

A program according to the fifth aspect of the present invention is:
Computer
When the touch pad detects a press, a pressing finger specifying unit that analyzes the image data obtained by imaging of the imaging unit and stored in the storage unit, and specifies the pressing finger used for the pressing,
Function as an operation type determining unit that determines an operation type to be processed based on a positional relationship between the pressing finger specified by the pressing finger specifying unit and another finger different from the pressing finger at the time of the pressing It is characterized by making it.

  According to the present invention, it is possible to easily input a plurality of operation commands with respect to an input operation via the touch pad without impairing intuition, and operability can be improved.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a mobile terminal 100 according to Embodiment 1 of the present invention. As illustrated in FIG. 1, the mobile terminal 100 includes a control unit 110, a wide area wireless communication module 120, a storage unit 130, a display unit 140, an input unit 150, and an imaging unit 160.

  The control unit 110 includes various programs including a CPU (Central Processing Unit) 111 that executes system control and various programs including OS (Operating System) processing, and a touchpad input control program that the CPU 111 executes. A ROM (Read Only Memory) 112 to be stored, a RAM (Random Access Memory) 113 used as a work area at the time of program execution, temporarily used to hold input information from the user, and the like are configured.

  The wide area wireless communication module 120 performs communication through a wide area wireless network in response to a request from the control unit 110. A cellular phone radio system is used as the wide area radio network communication system used by the wide area radio communication module 120. In this case, the wide area wireless communication module 120 performs communication with, for example, a wireless base station of a mobile phone system.

  The storage unit 130 includes, for example, a flash memory 131 and the like, and stores service subscription information, data acquired through communication, data for application programs, and the like. The display unit 140 is configured by a liquid crystal display (LCD) 141 or a display device using organic EL (Electro Luminescence) as in the present embodiment, and a screen (user I / O) for user operation based on a command from the control unit 110. F screen), the user input result, data desired by the user, and the like are displayed.

  The input unit 150 includes a keypad 151, a touchpad 152, and the like, and performs processing for receiving input from the user. The touch pad 152 is composed of a transparent flat plate capacitive sensor that detects a change in capacitance. The touch pad 152 is mounted on the LCD 141 so as to detect contact (press) of the touch pad surface (for the user, the display screen of the LCD 141) with a user's fingertip, a dedicated pen, or the like, and information on the position (coordinate data). Is output.

  The imaging unit 160 includes a CMOS image sensor 161, a lens unit 162, and the like, and captures moving images and still images. As shown in FIG. 2, the housing 170 that houses the imaging unit 160 is attached to the upper portion (side surface in the longitudinal direction) of the mobile terminal 100 so as to be rotatable in the arrow direction. In addition, the lens unit 162 is disposed so that the entire surface of the LCD 141 can be covered when the housing 170 is rotated to the maximum in the A1 direction, for example. Therefore, when the user wants to capture an image of his or her own pressing of the touch pad 152, the user can rotate the housing 170 to the maximum in the A1 direction, and to capture images of other subjects, the housing 170 can be moved in the A2 direction. The direction of the lens unit 162 can be switched as appropriate.

  Next, processing (touch pad input control processing) for determining the user's operation command from the press of the touch pad 152 by the user and causing the OS to execute the determined operation command will be described with reference to the flowchart of FIG. The touch pad input control process starts when the portable terminal 100 is powered on, and the touch pad input control program stored in the ROM 112 is read and executed by the CPU 111.

  When the touch pad input control process is started, the CPU 111 activates the imaging unit 160 and starts imaging the surface of the LCD 141 (that is, the surface of the touch pad 152) (step S101). The moving image data captured by the imaging unit 160 is stored in the RAM 113 (may be stored in the flash memory 131). During the execution of the touchpad input control process, the RAM 113 always stores moving image data for a predetermined time (moving image data from a predetermined time before to the current time).

  In order to reduce power consumption, the specification may be that the imaging unit 160 is not activated when the lens unit 162 is not facing the surface of the LCD 141 when the power is turned on (however, the user's keypad may be used). The imaging unit 160 can be activated by an operation such as 151. Alternatively, when the lens unit 162 is not facing the surface of the LCD 141 when the power is turned on, the touch pad input control process itself may not be started. In these cases, after the power is turned on, the imaging unit 160 automatically starts imaging or the execution of the touch pad input control process is started when the user rotates the casing 170 to the maximum in the A1 direction. It may be.

  Thereafter, when the touch pad 152 is touched (pressed) by the user (YES in step S102), the CPU 111 acquires coordinate data detected by the touch pad 152 (step S103) and the moving image data read from the RAM 113. And the number of fingers at the time of pressing is detected (step S104).

  In order to determine whether or not the finger is imaged, a known pattern recognition method using image data of some kinds of fingers stored in advance in the ROM 112 or the flash memory 131 is employed. Alternatively, a technique is adopted in which the contour of the image of the imaged object is taken out and the object is recognized as a finger if the contour is elongated (for example, the ratio of width to height is 1: 5). Also good.

  When the number of fingers detected as described above is one or less (YES in step S105), the CPU 111 determines that the first pressing operation has been performed by the user, and the first pressing together with the acquired coordinate data. The event is notified to the OS (step S106).

  On the other hand, when the number of detected fingers is two or more (NO in step S105), that is, when the user presses the touch pad 152 with another finger, the CPU 111 causes the user to Assuming that the second pressing operation has been performed, the OS notifies the OS of a second pressing event together with the acquired coordinate data (step S107).

  When the OS receives the notification of the first pressing event or the second pressing event, the OS performs processing corresponding to each event. For example, when the OS is notified of the first press event, when there is a symbol such as an icon indicating certain image data at the position indicated by the coordinate data associated with the first press event on the display screen of the LCD 141, The image data is displayed on the LCD 141.

  On the other hand, when the second press event is notified, the OS, when a symbol such as an icon indicating certain image data exists at the position indicated by the coordinate data associated with the second press event on the display screen of the LCD 141, Information on the image data, specifically, attribute information (so-called properties) such as data size, image compression format, image display size, image data creation date and time, is displayed on the LCD 141.

  The touch pad input control process continues until the mobile terminal 100 is turned off. When the user rotates the casing 170 in the A2 direction and switches the direction of the lens unit 162, the touch pad input control process may be terminated.

  By executing the touch pad input control process as described above, when the user presses the touch pad 152, the user does not attach another finger to the pressing finger (see FIG. 4A), and others. Different operations (first pressing operation and second pressing operation) can be performed on the portable terminal 100 when the finger is attached (FIG. 4B). Thus, the user can easily perform two types of operations (for example, operations corresponding to the left and right mouse clicks) on one object, and the operability can be improved.

  In the touch pad input control process, the CPU 111 may notify the OS of three or more types of pressing events corresponding to the detected number of fingers. In this way, more types of operations can be performed on one object.

(Embodiment 2)
Then, the portable terminal 100 which concerns on Embodiment 2 of this invention is demonstrated. The mobile terminal 100 according to the second embodiment has the same configuration (see FIGS. 1 and 2) as the mobile terminal 100 according to the first embodiment, but the content of the touch pad input control process to be executed is related to the first embodiment. Different from the mobile terminal 100. More specifically, in both touchpad input control processes, the determination methods of the pressing operations (first pressing operation and second pressing operation) performed by the user are different.

  FIG. 5 is a flowchart for explaining a technique for determining a pressing operation in the touchpad input control process according to the second embodiment. The flowchart of FIG. 5 shows only the parts different from the flowchart of FIG. Hereinafter, this part will be described in detail.

  When the touch pad 152 is pressed by the user and the number of detected fingers is two or more (NO in step S105 in FIG. 3), the CPU 111 specifies the actually pressed finger (step S501 in FIG. 5). Specifically, the CPU 111 analyzes the previously acquired moving image data and specifies the detected finger that has the largest movement as the finger that is actually used for pressing.

  Then, the CPU 111 analyzes the moving image data and determines whether or not another finger is present (captured) on the left side of the identified finger (finger that is regarded as pressed) ( Step S502). If there is no other finger on the left side (NO in step S502), the CPU 111 assumes that the first pressing operation has been performed by the user (see FIG. 6A), and the first pressing together with the acquired coordinate data. An event is notified to the OS (step S503).

  On the other hand, if there is another finger on the left side of the finger considered to be pressed (YES in step S502), the CPU 111 assumes that the user has performed a second pressing operation (see FIG. 6B). The second press event is notified to the OS together with the acquired coordinate data (step S504).

  As described above, according to the touch pad input control process of the present embodiment, the user places a plurality of fingers (for example, the right index finger and the right middle finger) in the vicinity of the touch pad 152 (LCD 141), that is, the image pickup by the image pickup unit 160. The first pressing operation and the second pressing operation can be performed while being placed within the range (within the visual field of the lens unit 162). Therefore, two types of operations on one object (for example, operations corresponding to the left and right mouse clicks) can be performed more easily, and the operability is further improved.

  In the touch pad input control process, the CPU 111 may notify the OS of a pressing event corresponding to the number of fingers existing next to the finger identified as pressed. In this way, more types of operations can be performed on one object.

(Embodiment 3)
Then, the portable terminal 100 which concerns on Embodiment 3 of this invention is demonstrated. The mobile terminal 100 according to the third embodiment has the same configuration (see FIGS. 1 and 2) as the mobile terminal 100 according to the first and second embodiments, but a pressing operation in the touch pad input control process to be executed ( The discrimination method of the first pressing operation and the second pressing operation) is different from that of the portable terminal 100 according to the first embodiment and the second embodiment.

  In the third embodiment, as shown in FIG. 7, the finger at the time of pressing is imaged from the lateral direction by the imaging unit 160. FIG. 8 is a flowchart for explaining a method for determining a pressing operation in the touchpad input control process according to the third embodiment. The flowchart of FIG. 8 shows only the parts different from the flowchart of FIG. Hereinafter, this part will be described in detail.

  When the touch pad 152 is pressed by the user, the CPU 111 analyzes the moving image data read from the RAM 113 and detects the number of fingers at the time of pressing (step S104 in FIG. 3). If the number of fingers is zero (YES in step S801 in FIG. 8), the CPU 111 determines that the first pressing operation has been performed by the user, and sends the first pressing event to the OS together with the previously acquired coordinate data. Notification is made (step S802).

  If the number of detected fingers is one (YES in step S803), the CPU 111 analyzes the previously acquired moving image data and determines whether there is an object that moves greatly behind (behind) the detected finger. Is determined (step S804). As a result, if there is an object that moves greatly behind (YES in step S804), the CPU 111 identifies the object as a finger that was actually used for pressing, and the user has performed a second pressing operation. The second press event is notified to the OS together with the previously acquired coordinate data (step S805). On the other hand, if there is no moving object behind (NO in step S804), the CPU 111 identifies the detected finger as the finger actually used for pressing, and the first pressing operation is performed by the user. A first pressing event is notified to the OS together with the previously acquired coordinate data (step S806).

  When the detected number of fingers is plural (NO in step S803), the CPU 111 identifies the finger actually pressed (step S807). Specifically, the CPU 111 analyzes the previously acquired moving image data and specifies the detected finger that has the largest movement as the finger that is actually used for pressing.

  Then, the CPU 111 analyzes the moving image data and determines whether or not the specified finger (finger that has been pressed) is blocked by another finger (step S808). For example, the CPU 111 performs such determination based on the degree of shape change during the movement of the identified finger. In other words, if the shape does not change significantly during operation, it is considered that the finger is not obstructed by another finger, and if the shape is greatly changed, it is assumed that the finger is blocked by another finger. .

  As a result, if the specified finger is not blocked by another finger (NO in step S808), the CPU 111 determines that the first pressing operation has been performed by the user (see FIG. 9A) first. A first pressing event is notified to the OS together with the acquired coordinate data (step S809). On the other hand, when the specified finger is blocked by another finger (YES in step S808), the CPU 111 obtains the first, assuming that the second pressing operation is performed by the user (see FIG. 9B). A second pressing event is notified to the OS together with the coordinate data (step S810).

  As described above, according to the touchpad input control process of the present embodiment, even when the user's finger is imaged from the lateral direction by the imaging unit 160, the user can While the finger (for example, the right index finger and the right middle finger) is placed in the vicinity of the touch pad 152 (LCD 141), that is, within the imaging range of the imaging unit 160 (in the field of view of the lens unit 162), the first pressing operation and the second The pressing operation can be performed.

  In the touch pad input control process, the press event that the CPU 111 notifies the OS is not limited to the first press event and the second press event. For example, the CPU 111 may notify the OS of a pressing event corresponding to the number of other fingers that block the pressed finger. In this way, more types of operations can be performed on one object.

  The present invention is not limited to any of the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  For example, the type of finger (for example, index finger or middle finger) that pressed the touch pad 152 is specified, the user's pressing operation is determined according to the type, and the OS is notified of the pressing event corresponding to the determined pressing operation. Also good. In this case, for example, the portable terminal 100 is provided with a function (such as a fingerprint sensor) for detecting the fingerprint of the finger that pressed the touch pad 152, and the fingerprint data (for example, flash memory) of each user's finger registered in advance. The type of the pressed finger can be specified by comparing the detected fingerprint with the detected fingerprint.

  Thereby, the user can easily perform a plurality of types of operations (for example, operations corresponding to the left and right mouse clicks) on one object by changing the finger to be pressed, and the operability is improved. I can plan.

  Further, the finger part (fingertip nail, finger belly, etc.) that pressed the touch pad 152 is specified, the user's pressing operation is determined according to the finger part, and the pressing event corresponding to the determined pressing operation May be notified to the OS. In this case, for example, the portable terminal 100 is provided with a function (color sensor or the like) for detecting the color of an object that presses the touch pad 152, and is registered in advance and stored in the flash memory 131 or the like. It is possible to determine whether the pressed object is a fingernail or a finger belly by comparing the color of the finger and the color of the finger belly with the color of the detected object.

  Thus, the user can easily perform two types of operations on one object without changing the finger to be pressed.

  Further, by applying the program, it is possible to cause an existing portable terminal (for example, a camera-equipped cellular phone) to function as the portable terminal according to the present invention. That is, by applying the program executed by the CPU 111 described above to an existing mobile terminal and executing the program by a computer (such as a CPU) that controls the mobile terminal, the above-described functional configurations and processes are realized. Can be done.

  Such a program distribution method is arbitrary. For example, the program can be distributed by being stored in a recording medium such as a memory card, or can be distributed through a communication network such as the Internet. Then, by installing and applying the distributed program in a mobile terminal such as a mobile phone, the same function as that of the mobile terminal 100 described above can be realized.

It is a block diagram which shows the structure of the portable terminal which concerns on each embodiment of this invention. It is the schematic which shows the external appearance of the portable terminal of FIG. 3 is a flowchart illustrating a procedure of touch pad input control processing according to the first embodiment. In Embodiment 1, it is a figure which shows the mode of the press operation by a user, (a) shows the mode at the time of 1st press operation, (b) shows the mode at the time of 2nd press operation. 10 is a flowchart for explaining a method for determining a pressing operation in a touchpad input control process according to the second embodiment. In Embodiment 2, it is a figure which shows the mode of the press operation by a user, (a) shows the mode at the time of 1st press operation, (b) shows the mode at the time of 2nd press operation. It is a figure which shows the usage example of the portable terminal in Embodiment 3. FIG. 14 is a flowchart for explaining a method for determining a pressing operation in a touchpad input control process according to the third embodiment. In Embodiment 3, it is a figure which shows the mode of the press operation by a user, (a) shows the mode at the time of 1st press operation, (b) shows the mode at the time of 2nd press operation.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Portable terminal, 110 ... Control part, 111 ... CPU, 112 ... ROM, 113 ... RAM, 120 ... Wide area wireless communication module, 130 ... Memory | storage part, 131 ... Flash memory, 140 ... Display part, 141 ... LCD, 150 ... Input unit 151 ... keypad 152 ... touch pad 160 ... imaging unit 161 ... CMOS image sensor 162 ... lens unit 170 ... housing

Claims (7)

A touchpad that detects pressing and outputs position information of the detected area;
Imaging means for imaging the surface of the touchpad;
Storage means for storing image data obtained by imaging by the imaging means;
When the touch pad detects a press, the image data stored in the storage means is analyzed, and the number of fingers detecting means for detecting the number of fingers existing on the surface of the touch pad at the time of the press When,
An operation type determination unit that determines an operation type to be processed based on the number of fingers detected by the finger number detection unit;
A portable terminal characterized by that. A touchpad that detects pressing and outputs position information of the detected area;
Imaging means for imaging the surface of the touchpad;
Storage means for storing image data obtained by imaging by the imaging means;
When the touchpad detects a press, the image data stored in the storage means is analyzed, and a press finger specifying means for specifying the press finger used for the press;
An operation type determining means for determining an operation type to be processed based on a positional relationship at the time of the pressing between the pressing finger specified by the pressing finger specifying means and another finger different from the pressing finger; Prepare
A portable terminal characterized by that. The pressing finger specifying means analyzes the image data for a predetermined time, and specifies a finger having a largest position change as the pressing finger.
The mobile terminal according to claim 2. A touchpad that detects pressing and outputs position information of the detected area;
Finger type specifying means for specifying the type of the pressed finger;
An operation type determining means for determining an operation type to be processed based on the finger type specified by the finger type specifying means;
A portable terminal characterized by that. The finger type specifying means detects the fingerprint of the pressed finger, and specifies the type of the pressed finger by comparing the detected fingerprint with the fingerprint data of each finger of the user held in advance.
The mobile terminal according to claim 4. Computer
When the touchpad detects a press, it analyzes the image data obtained by the imaging means and stored in the storage means to detect the number of fingers present on the surface of the touchpad at the time of the press Finger number detection means to perform,
Based on the number of fingers detected by the finger number detection means, function as an operation type determination means for determining an operation type to be processed;
A program characterized by that. Computer
When the touch pad detects a press, a pressing finger specifying unit that analyzes the image data obtained by imaging of the imaging unit and stored in the storage unit, and specifies the pressing finger used for the pressing,
Function as an operation type determining unit that determines an operation type to be processed based on a positional relationship between the pressing finger specified by the pressing finger specifying unit and another finger different from the pressing finger at the time of the pressing Let
A program characterized by that.

Images