JP4379214B2 - Mobile terminal device - Google Patents

Description

The present invention, by measuring the movement of tendons and arm of the user's finger or wrist, it is possible to perform the operation the user intends relates to a portable terminal device.

  Conventionally, input devices such as a keyboard, a keypad, and an input pen have been mainly used to input information to a mobile terminal device. However, it is difficult to mount such an input device on a small terminal such as a wristwatch type or a bracelet type while ensuring sufficient operability.

  Therefore, a method for inputting information by measuring a biological signal such as myoelectric potential, vibration or acceleration when a finger hits an object, and the like has been proposed. However, when measuring the myoelectric potential, the electrical resistance of the skin surface is affected by perspiration and ambient humidity. Therefore, in the medical field or the like, the electrode for measurement is applied after applying the electrolytic paste to the skin. Such work places a heavy burden on the user of the mobile terminal device. In addition, noise due to electromagnetic waves from surrounding electrical products or the like is often mixed, and the noise due to electromagnetic waves hinders measurement of myoelectric potential. In particular, when the mobile terminal device that performs input is a mobile phone or a PHS (Personal Handyphone System) terminal, the influence of electromagnetic waves cannot be avoided.

  An input system for recognizing input of characters and commands according to a force generated when an input sensor device having a motion sensor is attached to a user's fingertip, brought into contact with an arbitrary object, and moved along the surface of the object It has been proposed (see, for example, Patent Document 1).

  In addition, an accelerometer, gyro sensor, or inertial sensor is attached to the finger to measure the movement of the finger, and the measured finger movement is displayed superimposed on a virtual keyboard displayed on the screen. In response to this, an information input system for detecting a key input of a virtual keyboard has been proposed (see, for example, Patent Document 2).

JP 2001-142631 A (paragraphs (0029) to (0101), FIG. 1) JP 2002-278673 A (paragraphs (0012) to (0019), FIG. 3)

  However, the system described in Patent Document 1 requires an object to be brought into contact with the motion sensor, and cannot detect a motion such as a gesture.

  Further, the system described in Patent Document 2 requires display means that is large enough to display a keyboard, and cannot be applied to a portable terminal device that is required to be downsized.

Therefore, an object of the present invention is to provide a portable terminal device that can perform an operation intended by a user by measuring movements of the user's fingers and arms.

A portable terminal device according to the present invention is mounted on a user's arm, indirectly measures the movement of the user's finger and arm, and outputs a measurement amount, and a plurality of types of measurement amounts or measurement amounts. A control definition memory for storing input information corresponding to each of the combinations, a control means for extracting input information corresponding to the measurement amount output from the sensor from the control definition memory, and executing control according to the extracted input information; A part of the plurality of sensors is a sensor that measures a bulge of a tendon tied to a finger bone on a wrist of a user.

  Some of the plurality of sensors may be sensors that are mounted on the user's wrist and measure an acceleration vector of the movement of the user's arm. According to such a configuration, the movement of the user's arm can be indirectly measured.

  The control means may perform calculation using the measurement amount output from the sensor, and may extract input information corresponding to a value obtained by the calculation from the control definition memory. According to such a configuration, control can be executed based on the measurement amount output by the sensor.

  The sensor may be mounted on a slave unit that is wired or wirelessly connected to a housing including the control definition memory and the control means. According to such a configuration, since the device worn on the arm by the user is a slave unit, the device worn on the arm by the user can be further downsized.

  One housing may be equipped with a sensor, a control definition memory, and a control means. According to such a configuration, the mobile terminal device in one housing can measure the movement of the user's finger and arm, and can perform an operation intended by the user.

  The control definition memory may include a plurality of types of measurement amounts or a combination of measurement amounts and a distance between the cases. Input information corresponding to each may be stored, and the control means extracts the input information corresponding to the measurement amount output from the sensor and the measurement amounts output from the plurality of housings from the control definition memory, and extracts the input information. Control may be executed according to the input information. According to such a configuration, the mobile terminal device can execute control according to the distance between the casings in addition to the measured amount of the user's finger and arm movement measured by the sensor.

  According to the present invention, the control intended by the user can be executed in accordance with the movement of the user's finger and arm.

  Further, according to the present invention, since the control intended by the user can be executed without using a large-sized keyboard or keypad, the portable terminal device can be further downsized.

  Furthermore, according to the present invention, it is possible to detect a movement such as a gesture and execute a control intended by the user without touching any object.

  Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an explanatory view showing an example in which a portable terminal device 11 according to the present invention is mounted on a wrist. The wrist terminal may be attached to the mobile terminal device 11 or may be a handset that is connected to the mobile terminal device 11 in a wired or wireless manner and is equipped with a sensor. In the following description, it is assumed that the mobile terminal device 11 is mounted.

  By the way, the muscles of fingers and wrists are in the arms, and these muscles and the bones of the hands are connected by tendons. When the finger or wrist is bent, each tendon portion rises according to the angle. Therefore, a piezoelectric sensor (sensor) 25 that converts pressure into voltage is disposed at a position of the mobile terminal device 11 that is in contact with each tendon portion. The piezoelectric sensor 25 measures the contact pressure caused by the bulging of the tendon portion. The display unit 22 is realized by a monochrome or color liquid crystal panel, organic EL (electrofluorescence), or the like, and displays information. The keyboard 19 includes keys that are pressed to input information.

  FIG. 2 is a block diagram showing a configuration example of the mobile terminal device 11 according to the present invention. The mobile terminal device 11 includes a CPU (Central Processing Unit) 12 that controls each unit of the mobile terminal device 11, a keyboard 19 that inputs information, and a key input unit 18 that outputs information input to the keyboard 19 to the CPU 12. A display unit 22 for displaying information, a display control unit 21 for controlling the display unit 22, an acceleration sensor 23 for measuring a three-dimensional acceleration vector, a plurality of piezoelectric sensors 25, a piezoelectric sensor 25, and the piezoelectric sensor 25 for measuring A pressure input unit 24 for outputting the measured value to the CPU 12, a ROM (Read Only Memory) 14 for storing fixed data such as various control programs executed by the CPU 12, a dictionary used for kana character conversion, and the like. Stores data that is temporarily needed to execute a program The industrial memory 15, the control definition memory 16 that stores information on the relationship between the values measured by the piezoelectric sensor 25 and the acceleration sensor 23, and the processing to be executed or the characters to be input in response thereto, each sensor measures The timer part 17 which outputs the trigger which is a signal which shows a timing to CPU12, and the bus | bath 13 which connects CPU12 and each part are included.

  The working memory 15 is realized by, for example, a RAM (Random Access Memory). The acceleration sensor 23 measures the acceleration vector in the three-dimensional space of the user's arm and is realized by the three-dimensional acceleration sensor. However, the acceleration sensor 23 may measure an acceleration vector in a two-dimensional space or less. These one-dimensional acceleration sensors may be installed so as to measure the acceleration in the vertical direction.

  The control definition memory 16 is realized by, for example, a nonvolatile memory. For example, the movements of fingers, wrists, and arms are patterned based on values measured by the piezoelectric sensor 25 and the acceleration sensor 23 or combinations of values measured by the piezoelectric sensor 25 and the acceleration sensor 23. When the terminal device 11 is shipped from the factory, the control definition memory 16 stores in advance operation input information that is information indicating the relationship between the patterned movement and the process to be executed or the character to be input. is doing. Further, the user operates the keyboard 19 to store operation input information, which is information on the relationship between the patterned movement and the process to be executed or the character to be input, in the control definition memory 16. It may be memorized. The timer unit 17 outputs a trigger to the CPU 12 at a predetermined time interval (for example, every 10 ms), and when the trigger is input, the CPU 12 causes the acceleration sensor 23 to measure an acceleration vector and sends it through the pressure input unit 24. Then, the piezoelectric sensor 25 is caused to measure the contact pressure. The CPU 12 performs a predetermined calculation using the measurement amounts measured by the acceleration sensor 23 and the piezoelectric sensor 25, extracts operation input information corresponding to the value obtained by the calculation from the control definition memory, and extracts the extracted operation. You may perform control which performs a process according to input information, or inputs a character.

  The mobile terminal device 11 is realized by, for example, a mobile phone, a PHS terminal, a PDA (Personal Data Assistance, Personal Digital Assistant) or the like.

  Next, the operation of the embodiment will be described with reference to the drawings. FIG. 3 is a flowchart for explaining the operation of the embodiment of the present invention.

  When the mobile terminal device 11 is powered on, the timer unit 17 outputs a trigger to the CPU 12 at a predetermined time interval. When the trigger is input, the CPU 12 causes the acceleration sensor 23 to measure the acceleration vector, and causes the piezoelectric sensor 25 to measure the contact pressure via the pressure input unit 24. At this time, when the user moves the arm or finger wearing the portable terminal device 11, the acceleration sensor 23 measures the acceleration vector, and the piezoelectric sensor 25 measures the contact pressure due to the tendon bulge. The piezoelectric sensor 25 inputs the measured contact pressure to the pressure input unit 24. The CPU 12 stores the acceleration vector measured by the acceleration sensor 23 in the work memory 15 (step S101). Further, the CPU 12 stores the contact pressure input to the pressure input unit 24 by the piezoelectric sensor 25 in the work memory 15 (step S102).

  The CPU 12 recognizes the movement of the user's finger, wrist and arm based on the acceleration vector and the contact pressure stored in the work memory 15 (step S103). The motion input information stored in 16 is compared with each other to determine the motion pattern (step S104).

  When the movement recognized by the pattern matches the movement of the movement input information stored in the control definition memory 16, the CPU 12 performs a process to be executed according to the movement of the pattern according to the movement input information. Go or input characters (step S105).

  If the movement recognized by the pattern does not match the movement patterned by the motion input information stored in the control definition memory 16, the CPU 12 returns to step S101 after a predetermined time has elapsed (step S106).

  As described above, according to this embodiment, the user can input an intended operation or character to the mobile terminal device 11 by a combination of bending and stretching of a finger or wrist and movement or rotation of an arm. It can be performed. Therefore, a large-sized keyboard and keypad are not required, and the portable terminal device 11 can be further downsized.

  Further, since the mobile terminal device 11 is mounted near the wrist, it can be mounted in a natural form as a small mobile terminal device such as a wristwatch type or a bracelet type.

  Furthermore, since the measurement object by each sensor is the movement of the arm and the mechanical change amount of the finger and wrist, the measurement object is not affected by electromagnetic waves from the mobile terminal device 11 itself or from surrounding electrical products. .

  In addition, since the user can perform an operation or input characters without touching the object, it is possible to detect a movement such as a gesture.

  In this embodiment, the combination of the angle at which the finger or wrist is bent (contact pressure of the tendon measured by the piezoelectric sensor 25) and the movement or rotation of the arm (acceleration vector measured by the acceleration sensor 23) The processing performed by the mobile terminal device 11 and the character to be input are determined. Of these combinations, the processing performed by the mobile terminal device 11 and the character to be input may be determined using some combinations. The process performed by the mobile terminal device 11 and the character to be input may be determined in combination with the input to the. Further, based on the amount measured by one sensor, the process performed by the mobile terminal device 11 and the character to be input may be determined.

  Further, in this embodiment, the piezoelectric sensor 25 is used as a sensor for measuring the bulge of the tendon of the finger. However, the present invention is not limited to this, and the undulation of the skin such as an acceleration sensor or a strain gauge is used. Other sensors capable of measuring the displacement of may be used. Although the acceleration sensor 23 is used as a sensor for measuring the movement of the arm and the rotation, the mass sensor and the pressure sensor are used in combination, or the posture change amount, posture change speed, and posture change acceleration measured by the gyro sensor are used. It may be used to measure arm movement and rotation.

  The processing of the mobile terminal device 11 to be executed according to the user's finger, wrist, and arm movement patterns includes, for example, a menu selection function, a kana character conversion function, a cursor movement function, a screen scroll function, and various functions. And so on.

  Among the acceleration vectors measured by the acceleration sensor 23 for the movement of the user's arm, a velocity vector obtained by integrating the component due to movement of the mobile terminal device 11 once in time, or a displacement vector obtained by integrating twice. You may ask for.

  The reference for determining the movement pattern of the user's finger, wrist, and arm from the contact pressure measured by the piezoelectric sensor 25 and the acceleration vector measured by the acceleration sensor 23 may be changed by the user's operation. Then, for example, the user who uses the mobile terminal device 11 for the first time can suppress erroneous input by confirming the input with a large change amount, and the user who is used to the operation of the mobile terminal device 11 can make a small change. By confirming the input by the amount, the input operation can be performed quickly. It should be noted that the reference for discriminating the movement pattern of the user's finger, wrist and arm from the contact pressure measured by the piezoelectric sensor 25 and the acceleration vector measured by the acceleration sensor 23 must be changeable by the user's operation. Also good. In that case, the operation input information may be stored in the ROM 14.

  In order to reduce the power consumption of the portable terminal device 11, the CPU 12 turns off the power of the piezoelectric sensor 25 and the acceleration sensor 23 except when inputting to the piezoelectric sensor 25 and the acceleration sensor 23, and sets the power saving state. Also good.

  In this embodiment, as shown in the example of FIG. 1, the piezoelectric sensor 25 is arranged on a straight line along the circumference of the wrist, but the piezoelectric sensor 25 is arranged on a two-dimensional plane, A planar piezoelectric sensor that can measure the movement of tendon ridges may be used. Then, the change in the tendon bulge can be measured more accurately. That is, it is possible to measure more complicated finger movements.

  In addition, the control definition memory 16 may store information on the relationship between the values measured by the piezoelectric sensor 25 and the acceleration sensor 23 based on the movement of the hand and arm in sign language, and the corresponding characters and words. . Persons with hearing handicap generally communicate in sign language, but the control definition memory 16 stores information indicating the correspondence between the movement of the hand and arm in sign language and the character or word. If stored, characters and words can be input to the mobile terminal device 11 in accordance with the movement of the hand and arm in sign language. This eliminates the need for a person with hearing handicap to learn new finger and arm movements in order to input characters and words into the mobile terminal device 11.

  When there is an input from each sensor over a predetermined length of time, an input over a predetermined size, or an input whose measurement value changes over a predetermined size, the CPU 12 sends it to the mobile terminal device 11. It may be determined that the input is. Then, erroneous input due to chattering can be prevented.

  Moreover, the portable terminal device 11 has a function of measuring the distance between each other, and may be configured by a plurality of units connected by wire or wirelessly. At this time, a unit including the CPU 12, the keyboard 19, the key input unit 18, the display unit 22, the display control unit 21, the ROM 14, the work memory 15, the control definition memory 17, and the timer unit 17. (Hereinafter, referred to as a parent device) includes, for example, a distance measuring unit that measures a distance from another unit (hereinafter, referred to as a child device) using sound waves. The control definition memory 16 stores the relationship between the values measured by the piezoelectric sensor 25 and the acceleration sensor 23 based on the movement of the hand and the arm, the distance between the slave unit, and the corresponding character and processing. Information may be stored. Then, in addition to the movements of fingers and hands, processing and characters performed by the mobile terminal device 11 can be input in accordance with the distance between the parent device and the child device. At this time, each sensor may be mounted on the master unit or on the slave unit. A part of the plurality of sensors is mounted on the base unit, and the other part of the plurality of sensors is mounted. The slave unit may be installed. Moreover, the slave unit may be equipped with a distance measuring means.

  The present invention can be used for mobile terminal devices such as mobile phones, PHS terminals, and PDAs.

It is explanatory drawing which shows the example which mounted | wore the wrist with the portable terminal device by this invention. It is a block diagram which shows the example of 1 structure of the portable terminal device by this invention. It is a flowchart explaining operation | movement of embodiment of this invention.

Explanation of symbols

11 Mobile terminal device 12 CPU
13 bus 14 ROM
DESCRIPTION OF SYMBOLS 15 Memory for work 16 Memory for control definition 17 Timer part 18 Key input part 19 Keyboard 21 Display control part 22 Display part 23 Acceleration sensor 24 Pressure input part 25 Piezoelectric sensor

Claims (6)

A plurality of sensors that are attached to the user's arm and indirectly measure the movement of the user's finger and arm and output a measurement amount;
A control definition memory for storing input information corresponding to each of a plurality of types of measurement amounts or combinations of measurement amounts;
Control means for extracting input information corresponding to the measurement amount output by the sensor from the control definition memory and executing control according to the extracted input information ;
A part of the plurality of sensors is a sensor for measuring a bulge of a tendon tied to a finger bone on a wrist of a user . Among the plurality of sensors, some sensors, the portable terminal apparatus according to claim 1, wherein the sensor for measuring the acceleration vector of the movement of the user's arm is mounted on the user's wrist. Control means, sensor performs a calculation using the measured amount of the output, the portable terminal apparatus according to claim 1 or claim 2, wherein the input information corresponding to the value obtained is extracted from the control definition memory by calculation. The portable terminal device according to any one of claims 1 to 3 , wherein a sensor is mounted on a slave unit that is wired or wirelessly connected to a housing including a control definition memory and a control unit. The portable terminal device according to any one of claims 1 to 3, wherein a single housing includes a sensor, a control definition memory, and a control unit. Including a plurality of housings having a function of measuring each other's distance and outputting a measurement amount;
The control definition memory stores input information corresponding to each of a combination of a plurality of types of measurement amounts or combinations of measurement amounts and a distance between the housings,
The control means extracts input information corresponding to the measurement amount output from the sensor and the measurement amounts output from the plurality of housings from the control definition memory, and executes control according to the extracted input information. portable terminal device according to any one of claims 4.

Images