JP4015133B2 - Terminal device - Google Patents

Description

  The present invention relates to a portable terminal device, and more particularly, to a user interface technology for improving the efficiency of operation of the terminal device.

  Since the mobile phone inherits the operability with the fixed phone, an operation interface centered on a numeric keypad is often employed. Also, since mobile phones are required to be easy to carry, the case size is limited, and the number of keys that can be attached to the case is limited. On the other hand, mobile phones are becoming increasingly sophisticated, and it is necessary to select complicated functions with a combination of few key operations, so operability tends to deteriorate.

  As a method of solving such a problem, there is a method of providing an input device different from the numeric keypad. There are various extended keys as an example of such an input device. In addition, as a device using a device other than a key, there is a configuration that attempts to improve the operability of a mobile phone using a pressure-sensitive sensor (for example, Patent Document 1).

Japanese Patent Laid-Open No. 2001-69223 “Communication Device”

  Although the technique described in Patent Document 1 performs a scroll operation through a pressure-sensitive sensor, the pressure-sensitive sensor can be used for purposes other than the scroll operation, and to do so. Thus, it is possible to efficiently select a complicated function of the mobile phone. On the other hand, pressure-sensitive sensors often do not have a solid operational feeling like the click feeling of a numeric keypad, which makes the user uneasy. For this reason, an operation interface using a pressure sensor is often used for the purpose of pointing an item displayed on the screen, and is adopted as an operation interface for directly selecting a function like a key operation. It was hard.

  An object of the present invention is to provide a terminal device that allows a user to perform a desired operation from a wide range of functions with a certain sense of operation using a sensor that detects a contact position of a finger.

A terminal device according to the present invention has a housing that can be gripped with one hand, and is a terminal device that is used by operating a key with a finger of a hand that grips the housing. A part that covers a part of the front surface of the housing and a position where the hand holding the housing can be touched with a finger while maintaining the key operable position, and when the part is closed A sensor that is mounted at a position exposed to the outside both when the lid is opened and detects the contact position of the touched finger, a gesture identification unit that cuts out a gesture from the time series of the contact position detected by the sensor, and Control means for executing a function corresponding to the plurality of combined gestures by combining a plurality of gestures cut out by the gesture identification means, and the sensor further determines the size of the contact surface of the finger. Out, the gesture identification means, from the time series of the contact position detected in the sensor, Shi cut out gestures size of the contact surface dismissed time series of the contact position in the time zone below a predetermined value The distance value by matching is calculated by adding a penalty value to the absolute distance value calculated from the input time series and the vector element of the gesture pattern, and the distance value by the matching is used for calculating the gesture pattern .
In addition, a terminal device according to the present invention has a housing that can be held with one hand. In a terminal device that is used by operating a key with a finger of a hand that holds the housing, the terminal device is attached to the housing so as to be freely opened and closed. A part that covers a part of the front surface of the casing when the cover is closed, and a position where the hand holding the casing can be touched with a finger while maintaining the position where the key can be operated. A sensor that is mounted at a position exposed to the outside both when the lid is open and when the lid is opened, detects a contact position of the touched finger, and a gesture identification unit that cuts out a gesture from the time series of the contact position detected by the sensor, And a control means for executing a function corresponding to the plurality of combined gestures by combining a plurality of gestures cut out by the gesture identification means, and the sensor further includes a large contact surface of the finger. The gesture identifying means rejects the time series of the contact position in the time zone when the size of the contact surface is below a predetermined value from the time series of the contact position detected by the sensor, and performs the gesture. Cutting, and even if the operation of moving the finger away from the sensor surface occurs, it is not immediately determined that the gesture has ended, and the finger before the finger leaves the sensor surface is in contact with the sensor surface; Thereafter, the absolute value of the direction and the movement vector is given based on the difference from the state where the finger is in contact with the surface of the sensor, and the gesture is recognized.

Thus , according to the terminal device according to the present invention, an appropriate gesture pattern can be selected even if the operation pattern fluctuates with time. For example, the stroke operation from the top to the bottom and the bottom to the top can be selected. The finger is released based on the state where the finger is in contact with the front and back sensors, which can easily cope with simple deformations such as a stroke operation and a series of operations (up → down → up). By complementing the movement amount vector during the period, the movement of the gesture can be appropriately digitized.

Embodiments of the present invention will be described below with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a rear perspective view of a terminal device according to Embodiment 1 of the present invention configured as a mobile phone. In the terminal device 1 shown in the figure, the opening / closing part 2 is attached to the main body part 3 through a hinge mechanism 4 so as to be freely opened and closed. A sensor 5 is attached to the body part 3. The sensor 5 is a sensor that converts the contact position into an electrical signal and outputs it when an object comes into contact.

  In the example of FIG. 1, the sensor 5 is installed on the side surface of the main body part 3. However, the installation position of the sensor 5 is not limited to the side surface, and any position can be used as long as it can be touched by the finger of the hand while keeping the hand holding the housing at a key operable position. Also good. Here, “maintaining the hand holding the casing in a position where the key can be operated” means an act of holding the casing so that the majority of the keys can be operated with the hand holding the casing. For example, it means an act of gripping the terminal device by a method as shown in FIG.

  Generally, in a portable terminal device such as the terminal device 1, key buttons are concentrated and arranged at one place so that it can be easily operated with one hand. When the user grips the terminal, the user grips a position where these keys can be easily operated by hand. In such a case, even if the terminal device is held with a hand, it is often possible to move a part of fingers of the hand in a limited manner. And the sensor 5 is arrange | positioned in the position which can contact with the finger of such a state. Note that the position of the finger that contacts the sensor 5 need not be the tip of the finger, but may be any part of the finger.

  As shown in FIG. 3, the terminal device 1 has the opening / closing part 2 attached to the main body part 3 via the hinge mechanism 4 so as to be freely opened and closed, so that the casing can be folded. However, the sensor 5 is arranged so as to be exposed to the outside even when the opening / closing part 2 is closed (in a state in which the housing is folded). It is possible to contact. In addition, even when the lid is closed, the sensor 5 is installed at a position where it can be touched by a finger of a part of the hand while gripping the same position as the gripping position when the lid is opened. Have. As the installation position of such a sensor 5, the side surface and the back surface of the terminal device 1 are typical. By doing so, the terminal device 1 can be operated using the sensor 5 regardless of whether the lid is opened or closed.

  FIG. 4 is a block diagram illustrating a configuration of the terminal device 1. In the figure, the hinge mechanism 4 and the sensor 5 are the same as the components having the same reference numerals shown in FIG. The hinge mechanism 4 not only connects the opening / closing part 2 and the body part 3 so as to be freely opened / closed, but also has a built-in switch function that detects an operation to open / close the opening / closing part 2 and generates a signal in each operation. ing. Since such a hinge mechanism having a switch function is already widely used in a foldable mobile phone terminal or the like, its structure will not be described in detail here.

  The sensor 5 is a sensor that outputs a contact position of an object in contact with the surface as an electric signal. As a method for realizing such a sensor, when the sensor surface is pressed with pressure, the sensor body is deformed, and a change in the electric resistance characteristic of the sensor due to the deformation is output as an electric signal. Various methods are known, including a non-contact type capacitive sensor method that outputs a change that occurs in capacitance characteristics as an electrical signal when a conductor approaches the vicinity of the surface. Any sensor may be used as long as it can detect the contact position of an object. The contact position of the object can be output by any method as long as the circuit that processes the sensor signal can obtain the time series of the contact position (time change of the contact position) from the sensor. For example, the sensor may output an interrupt signal to the circuit, or the sensor state may be acquired by polling from the signal processing circuit of the sensor.

  FIG. 5 is a diagram showing a configuration example of such a sensor 5. The enlarged portion of the figure is shown with the sensor 5 enlarged. The sensor 5 in this example is composed of an array composed of small piezoelectric elements 5-1 to 5-N. Each of the piezoelectric elements 5-1 to 5-N is an independent piezoelectric element, and can output a signal independently when pressure is detected by contact of an object or the like with the surface. As a result, each of the plurality of parts is represented by a binary value in a pressed state and a non-pressed state. Here, a state where each of the piezoelectric elements 5-1 to 5-N is pressed is expressed as “1”, and a state where the piezoelectric elements are not pressed (normal state) is expressed as “0”. As described above, in the configuration example of the sensor 5, each piezoelectric element is expressed by a binary value indicating whether the piezoelectric element is pressed or not pressed, and thus the absolute value of the pressure is not acquired. However, as will be described later, in spite of such a sensor, stable operability can be realized according to the present invention. In the following, it is assumed that the sensor 5 can output an interrupt signal when the state changes, and the current state can be acquired if necessary (can be polled). The explanation will proceed as if it were.

  The gesture identifying means 6 is a circuit or element that processes a time series of contact positions output from the sensor 5 and cuts out a gesture. However, the gesture identifying means 6 may be configured to realize a similar function by a combination of a computer program and a central processing unit. The gesture dictionary 7 is a storage element or circuit or a storage medium that stores a gesture pattern cut out by the gesture identification unit 6 as data.

  The control means 8 is a function of the terminal device 1 and is a circuit or element that selects and executes a group of functions represented by the gesture according to the gesture cut out by the gesture identification means 6. In this case, the function to be executed is selected according to the current state of the opening / closing part 2 by storing a signal output when the hinge mechanism 4 is opened or closed.

  Here, “a group of functions” refers to a unit (group) of functions that an operator consciously selects by operation. For example, in the case of a mobile phone, various processes are performed in the operation of making a call, but the operator is not conscious of operating each of these processes in a disassembled form, but called “call”. A group of functions is selected by operation. In many cases, such a function can be selected by operating a numeric keypad or an extended key once or several times. Therefore, the “collective functions” can be considered as functions that the operator can select by operating the numeric keypad, the extended key, or the shortcut key. However, the group of functions mentioned here is not limited to functions that can be selected in the form of menu items in advance. For example, a macro function (multiple functions that an operator defines later by customization) A combination of functions) and functions expanded by introducing an application program are also included in the “collective functions”.

  Note that, similarly to the gesture identification unit 6, the control unit 8 may also be configured to realize the same function by a combination of a computer program and a central processing unit. In addition, the gesture identification means 6 and the control means 8 do not necessarily have to be configured separately, and may be configured by the same CPU so as to perform the respective functions at the subroutine level of the computer program or at the processing step level. is there.

  The timer 9 is a circuit or element that supplies a clock signal to the gesture identification unit 6 and the control unit 8 as necessary, and is configured using a crystal oscillator or the like. The function table 10 is a correspondence table of gestures and functions corresponding to the gestures, and is information stored in a storage element or circuit (not shown), a storage medium, or the like.

  Since the terminal device 1 is configured by taking a mobile phone as an example, in addition to the components shown in FIG. 4, a part for transmitting and receiving a radio signal, a part for power control, a key operation part, an LED and a display part However, the configuration and implementation method of these parts do not directly affect the configuration of the present invention, and thus the description thereof is omitted here.

  Next, the operation of the terminal device 1 will be described. The operation of the terminal device 1 is roughly divided into processing in the gesture identification unit 6 and processing in the control unit 7. First, processing in the gesture identification unit 6 will be described. FIG. 6 is a flowchart of the process of the gesture identification unit 6. First, when the finger touches the sensor 5, one of the piezoelectric elements 5-1 to 5 -N of the sensor 5 generates an interrupt signal and notifies the gesture identification unit 6 of the start of the gesture. In response to this, the gesture identification means 6 activates the processing of FIG.

  In the figure, the gesture identification means 6 polls the sensor 5 to acquire the current state (step S1). The current state acquired here is expressed by either “1” (pressed state) or “0” (not pressed state) of each of the piezoelectric elements 5-1 to 5 -N. Is a 1 and 0 array. Subsequently, it is checked whether or not the size of the contact surface with which the finger contacts the sensor 5 exceeds a predetermined value (step S2). Since the finger has a certain size, when the finger contacts the sensor 5, the plurality of piezoelectric elements can be simultaneously in a “1” state. In addition, in that case, the piezoelectric elements in the “1” state are adjacent to each other. Therefore, the size of the contact surface can be obtained by counting the number of adjacent piezoelectric elements whose state is “1”. Here, since the surface of the finger is soft, the degree of deformation is large and the contact surface is widened when pressed strongly (for example, FIG. 8) as compared to the state where the finger 5 is lightly touched (for example, FIG. 7). . From this, it can be said that the size of the contact surface has a correlation with the finger pressure.

  Thus, by acquiring the size of the contact surface, even if a sensor element that cannot acquire the absolute value of pressure is used, processing based on pressure can be indirectly performed. Then, it is possible to reject a signal generated when the operator accidentally touches the sensor 5 by examining whether or not the size of the contact surface exceeds a predetermined value.

  Further, in a user interface that performs an operation based on a position touched by a user, the shape of the gesture often becomes unstable because the contact position fluctuates immediately after the start of contact. For this reason, as in step S2, it is possible to obtain a more stable operational feeling by disposing an unstable contact position under some condition.

  If it is determined in step S2 that the size of the contact surface does not exceed the predetermined value, the process proceeds to step S3 (No in step S2). In step S3, the timer 9 is used to wait until a certain time, for example, 10 milliseconds elapses. After that time, the process returns to step S1 again to perform the next state. On the other hand, if the size of the contact surface exceeds a predetermined value in step S2, the process proceeds to step S4.

  The purpose of the above processing is to eliminate erroneous operation and instability of processing by rejecting the state when the size of the contact surface does not exceed a predetermined value, but in this case the state is not rejected. In addition, assuming that the state of all the piezoelectric elements is “0”, the control may proceed to step S4.

  Further, when the sensor is configured using a sensor that directly obtains the pressure value, step S2 may be verified based on the pressure value instead of the size of the contact surface.

  Subsequently, the information on the contact position acquired in this way is added to the time series of the contact position (step S4). The time series of contact positions is an array composed of a plurality of contact position data, and the data is arranged in order of occurrence time. Here, in order to save the storage capacity, only the median value of the distribution of piezoelectric elements in which “1” continues (the position of the piezoelectric element that is the center of the distribution, or information that identifies the piezoelectric element that is the center of the distribution) Adopt and store as time-series elements. At this time, the current time may be acquired from the timer 9 and stored as a time series element together with the contact position.

  Next, it is determined whether or not the gesture has been completed (step S5). As an example of a method for determining the end of the gesture, a method (first method) for determining whether or not the number of piezoelectric elements in the “1” state has become zero can be considered. During the gesture operation, since the finger is in contact with any part of the sensor 5, any one of the piezoelectric elements is in the “1” state. When the gesture ends, the finger leaves the sensor 5, and the number of piezoelectric elements in the “1” state becomes zero. Therefore, whether or not the finger is in contact with the sensor 5 can be determined based on whether or not the number of piezoelectric elements in the “1” state becomes zero. In this method, it is determined that the gesture has been completed when the finger is released from the sensor 5, and it is immediately determined that the gesture has been completed when the finger is released.

  In addition, a method (second method) for detecting that a certain period of time has elapsed in the steady state can be considered. This can be detected by obtaining a difference between time series elements. In other words, if the difference between a certain number of time-series elements is 0 (the preceding and following elements are the same), it means that no new operation has been applied to the sensor 5, and that is the end of the gesture. To do.

  However, if the state does not change with the finger touching the sensor 5, the operator is unfamiliar with the operation or operating while thinking, so it can be assumed that the gesture is in progress. If the state where the number of piezoelectric elements in the “1” state becomes zero continues for a certain period of time, it is determined that the gesture has ended.

  The second method, that is, the method of determining the end of the gesture by utilizing the fact that the state does not change during a certain period of time, is characterized in that it can be resumed even if the gesture is interrupted. In the case of the first method, since the gesture is immediately determined to end when the finger leaves the sensor 5, there is no concept of resuming the gesture. Further, in the case of the second method, since the gesture can be regarded as part of the operation even when the gesture is interrupted (the time when the finger is away from the sensor 5), the degree of freedom and combinations of the gestures are abundant and the expressive power increases. There are advantages. In the following description, it is assumed that the gesture end is determined based on the second method. However, since the gesture can be used even in the first method, the gist of the present invention is achieved.

  If it is determined in step S5 that the gesture has not ended, the process proceeds to step S3 (step S5: Yes), and the processing from step S1 is repeated.

  If it is determined in step S5 that the gesture has ended, the process proceeds to step S6. Here, the time series of the previous contact positions are converted into gestures. In this configuration, since a method (second method) for determining the end of the gesture is employed on condition that a certain time has elapsed, the state in which the finger is not in contact with the sensor 5 in the time series ( The state in which the piezoelectric elements are all “0”) is also included.

  Next, the time series is converted into a movement amount vector (step S6). The movement amount vector is the amount of change (difference) in the contact position between the time series elements, and for example, a symbol as shown in FIG. 9 is attached according to the changing direction. Then, a distance value between the input time series converted into the movement amount vector and each gesture pattern stored in the gesture dictionary 8 is calculated (step S7).

  The gesture patterns stored in the gesture dictionary 8 are basic time-series information of gestures that can be recognized by the terminal device 1. For example, movement amounts such as patterns (gesture 1 to gesture 11) as shown in FIG. Vector time-series information is stored in advance. The feature of the gesture pattern stored here is when the stroke operation (operation that traces the surface of the sensor 5 in only one direction with a finger) and the tap operation (operation that temporarily touches the surface of the sensor 5) are the minimum units. Further, it is configured as a combination including two or more of these minimum units.

  By adopting such a configuration, the following effects are produced. First, when the contact sensor is placed in a limited space such as the side of the case of a mobile phone, it can be traced in a fixed direction on the installation surface (stroke operation) or has only an operation variation equivalent to a tap operation. This makes it possible to create complex gesture patterns. Therefore, it becomes possible to make different gestures correspond to many functions, and the application range of the gesture operation can be expanded.

  Further, although it is conceivable to select a function with only one minimum unit of gesture, since the sensor 5 is exposed outside the housing, an erroneous operation is likely to occur. By combining a plurality of gestures as minimum units, the possibility of such an erroneous operation can be reduced.

  The calculation of the distance value with the gesture pattern is performed using a general pattern matching technique. Several such pattern matching techniques are known, and pattern matching may be performed by any method. In the following description, a method based on DP (Dynamic Programming) matching is introduced as an example of such a method.

The i-th vector element of the input time series and the gesture pattern are f (i) and g (i), respectively, and d (i, j) is the i-th vector element of the input time series and the j-th vector element of the gesture pattern. Absolute value distance d (i, j) = | f (i) −g (j) | Then, a DP evaluation value h (i, j) for calculating the difference between the i-th vector element of the input time series and the j-th vector element of the gesture pattern is calculated as follows, and this is calculated by DP matching. The distance value.

  In the above equation, DP_PENALTY represents a penalty value other than one-to-one correspondence. In other words, when the input time series vector element to be associated and the vector element of the gesture pattern do not have a one-to-one correspondence, a many-to-one or one-to-many association is performed. One-to-many correspondence needs to be expressed as an evaluation value. For example, when the input time series vector element is in the upward direction and the gesture pattern vector element is in the downward direction, the evaluation values when both are associated are -1 (upward) and +1 (downward). In this case, since the association is not correctly performed, the penalty value is added to the evaluation value.

  In addition, instead of adjusting the evaluation value by exaggerating the penalty value afterwards, it is possible to correctly associate by setting -N for the upward direction and + N for the downward direction at the initial numerical stage. It is also possible to increase the evaluation value when there is not.

  As described above, since the distance value to the gesture pattern is calculated by DP matching, an appropriate gesture pattern can be selected even if the operation pattern fluctuates with time. Further, for example, it is possible to easily cope with a simple deformation in which a stroke operation from the top to the bottom and a stroke operation from the bottom to the top are performed by a series of operations (up, down, up).

  In step S5, even if the movement of the finger away from the surface of the sensor 5 occurs, it is not immediately determined that the gesture has ended. Therefore, the number of piezoelectric elements of “1” is not included in the time series. A time series element of 0 states is included. The movement amount vector in this case is calculated as follows. That is, the absolute value of the direction and the movement vector is given based on the difference between the state in which the finger is in contact with the surface of the sensor 5 before that and the state in which the finger is in contact with the surface of the sensor 5 after that. For example, when the stroke operation that traces from the top to the bottom is continuously performed twice at high speed, it is considered that the moving speed of the finger increases while the finger is not in contact with the surface of the sensor 5. In addition, when the tap operation is repeated twice, the finger movement speed is considered to be almost zero because the same position is often tapped twice. As described above, it is possible to appropriately quantify the movement of the gesture by complementing the movement amount vector while the finger is separated based on the state where the finger is in contact with the sensor 5 before and after the finger is separated. It becomes.

  Finally, among the DP matching distance values calculated for each gesture pattern in this way, the gesture pattern having the smallest distance value with respect to the input time series is output as the processing result of the gesture identifying means (step S8).

Next, processing of the control means 8 will be described. The control means 8 acquires the gesture pattern specified by the gesture identification means 6, specifies the function corresponding to this gesture pattern from the function table 10, and executes the function. FIG. 11 is a diagram illustrating an example of the contents of the function table 10. As shown in the figure, the function table 10 is a correspondence table between information for identifying a gesture pattern, and a function at the time of opening and a function at the time of closing corresponding to the gesture pattern. In the table, “-” (hyphen) means that the corresponding function does not exist, that is, no function is executed even if such a gesture operation is performed. In the example shown in FIG. 11, the opening function and the closing function for the same gesture are always the same, or only the opening function is specified without the closing function. However, it is not limited to such an example,
Further, as described above, whether the opening / closing part 2 is in the open state or the closed state is determined based on a signal sent from the hinge mechanism 4 in accordance with the closing operation / opening operation. . In this way, the control means 8 executes a function corresponding to the gesture pattern specified by the gesture identification means 6 with reference to this table.

  As described above, according to the terminal device 1, a gesture operation with a finger is performed on the sensor 5 mounted at a position where the hand that holds the casing can be touched with the finger of the hand while the hand is held at a position where the key can be operated. Since it is possible to operate by performing, various operations can be performed with one hand.

  In addition, the gesture identifying means 6 cuts out a gesture, combines a plurality of the cut out gestures into a gesture pattern, and selects and executes a set of functions corresponding to the gesture pattern. The user interface can be operated with a solid operational feeling regardless of the user interface.

  Further, since the sensor 5 is disposed at a position where it can be operated both when the lid is opened and when the lid is closed, gesture operation is possible not only when the lid is opened but also when the lid is closed. Further, the function table 10 stores the function at the time of opening and the function at the time of closing corresponding to each gesture pattern, and any of the functions corresponding to the gesture pattern based on the information on the closing and opening of the cover obtained from the hinge mechanism 4 is stored. Since such a function is selected, an optimal function can be independently selected by a gesture according to the closed state and the open state.

  Further, by configuring the functions so as to be associated with each other when the lid is closed and when the lid is opened, the functions that can be selected by the gesture operation can be limited. For this reason, when carrying the terminal device 1 in a closed state, even if a gesture is mistakenly identified due to an object touching the sensor 5 or the like, it is limited so that some functions are not executed. Is possible.

  In the first embodiment of the present invention, a mobile phone terminal has been described as an example of the terminal device according to the present invention. However, this does not mean that the terminal device according to the present invention is limited to a mobile phone terminal. In other words, the present invention can be held with one hand such as a portable information device such as a PDA (Personal Digital Assistant) or an electronic dictionary, a receiver unit (including a slave unit) of a fixed telephone, or a remote control terminal of a home appliance. In addition, the present invention can be widely applied to all terminal devices that are further operated by selecting a function by pressing a key.

  Furthermore, in Embodiment 1 of the present invention, the sensor is used to output a one-dimensional contact position, but the contact position and the shape of the gesture are not limited to one dimension, but two-dimensional. There may be. It is well known that two-dimensional gesture matching can be performed by the above-described DP matching.

  In the first embodiment of the present invention, the gesture identification unit 6 cuts out a combination of gestures. In addition to this, each gesture (stroke operation or tap operation) is cut out by the gesture identification unit 6. In addition, the control means 8 may constitute a combination of gestures.

Embodiment 2. FIG.
In Embodiment 1, although the example which comprises the terminal device 1 of this invention using a contact sensor was demonstrated, it replaces with a contact sensor and can also comprise using a non-contact type sensor. The terminal device according to the second embodiment of the present invention has such a feature.

  FIG. 12 is a front view of a terminal device according to Embodiment 2 of the present invention. In the figure, a terminal device 11 is a mobile phone device having a rotatable part 12 (hereinafter referred to as a rotational part 12). The main body part 13 is rotatably coupled to the pivot part 12 by a hinge mechanism 14 (not shown). Similar to the sensor 5 in the first embodiment, the sensor 15 is mounted at a position where the hand that holds the casing of the terminal device 11 can be touched by the finger of the hand while maintaining the hand operable position. . FIG. 13 is a front view showing a state in which the angle of the rotating portion 12 is changed. FIG. 14 is a front view showing a state in which the rotating part 12 is superposed on the main body part 13. As shown in this figure, the rotating part 12 is a part that is mounted so as to be openable and closable so as to cover part of the front surface (key operation part) of the main body part 13, and as shown in FIG. A state in which the body part 13 and the body part 13 are almost completely overlapped is referred to as a closed state. The state that is not so is called when the lid is opened.

  FIG. 15 is a block diagram illustrating a configuration of the terminal device 11. In the figure, the hinge mechanism 14 sends a signal notifying that the operation has been performed to the control part of the terminal device 11 in accordance with the opening / closing operation in the same manner as the hinge mechanism 4 in the first embodiment. ing. The sensor 15 is the same as the sensor 15 of FIGS. 12 to 14 and is characterized by being a non-contact type sensor. As an example of such a non-contact type sensor, there is a capacitance type non-contact sensor such as TSM9910 manufactured by Circue USA or TouchPad manufactured by Syntactics USA. The main principle of these sensors is to detect a change in capacitance that occurs when a part of the human body that is a conductor approaches the sensor surface, and output this as an electrical signal. However, in the present invention, the approach to the sensor surface may be detected by any principle, for example, the approach to the sensor surface may be detected using infrared rays or temperature change. In the following description, it is assumed that the sensor 15 is configured using a capacitive non-contact sensor.

  The gesture operation history 16 is a gesture operation history file that accumulates and stores information indicating the speed of the gesture operation, for example, the time required for completion of each gesture among the information related to the gesture operation performed by the operator on the sensor 15. And is configured using a storage element, a storage circuit, or a storage medium. The required time calculation means 17 is a part that calculates the maximum value of the required time for completing the gesture operation based on the information indicating the speed of the gesture operation stored in the gesture operation history 16. The required time calculation means 17 may be configured to realize a similar function by a combination of a computer program and a central processing unit.

  The other components denoted by the same reference numerals as those in FIG. 4 are the same as those in the first embodiment, and thus the description thereof is omitted. The terminal device 11 is configured as a mobile phone as an example of the embodiment of the present invention. Therefore, in addition to the components shown in FIG. 15, components such as a wireless unit, a power supply control unit, a numeric keypad, an LED, and a display unit are provided. The implementation method of such components is the configuration of the present invention. Since these do not directly affect, explanation of these parts is also omitted.

  Next, the operation of the terminal device 11 will be described. FIG. 14 is a flowchart of the operation of the terminal device 11. In FIG. 14, steps denoted by the same reference numerals as those in the flowchart of FIG. 6 are the same as those in the first embodiment, and therefore only different operation step portions will be described here.

  First, when a finger approaches the sensor 15, the sensor 15 generates an interrupt signal and notifies the gesture identification unit 6 of the start of the gesture. In response to this, the gesture identification means 6 activates the process of FIG. 14 and starts the process. At the same time, the sensor 15 outputs a signal representing information on the distance between the finger and the sensor surface in several plots (representative points) for each plot. As shown in FIG. 15, this signal becomes maximum when the plot is in contact with the surface (distance 0), and becomes zero when the plot is separated by a certain distance or more. Therefore, if the number of plots having the maximum value is counted, the size of the contact surface can be obtained.

  Also, by replacing the maximum value plot with the number of plots having a signal value equal to or greater than a certain value, information equivalent to the size of the contact surface can be obtained. As a result, it is possible to obtain information on the state where the finger is in contact with the sensor surface even though it is non-contact, so that the number of identifiable gesture patterns can be increased, and therefore many functions can be indicated by specific gesture operations. It becomes possible to do.

  When the state of the sensor 15 is acquired in step S1 and the processing from step S2 to S4 is performed thereafter, the gesture identification unit 6 uses the required time calculation unit 17 to calculate the maximum value of the required time for gesture completion. To do. The required time calculation means 17 determines the maximum value of the required time for completing the gesture based on the information indicating the speed of the gesture operation accumulated in the gesture operation history 16. As a method for determining the maximum value of the required time for gesture completion, for example, if the information indicating the speed of the gesture operation stored in the gesture operation history 16 is the required time to completion of each gesture, the average value of the required time And a value obtained by adding a certain margin value to the average value is set as the maximum value of the required time.

  Also, the information indicating the speed of the gesture operation may be the finger moving speed calculated from the moving amount vector. In this case, for example, as shown in FIG. Alternatively, a timeout value may be provided according to each stage, and the determination may be made based on this timeout value.

  In step S5, it is checked whether or not the maximum required time for completion of the gesture calculated by the required time calculation means 17 has elapsed since the state of the sensor 15 becomes a steady state. Judge that it was done. Here, the case where the sensor 15 is in a steady state refers to a state in which the previous element in the time series and the current element do not change. If it is determined that the gesture has been completed (step S5: Yes), the process proceeds to step S52, and information indicating the speed of the gesture is stored in the gesture operation history 16. Thereafter, steps S6 to S8 are the same as those in the first embodiment.

  The control unit 8 is also similar to the first embodiment in that each function at the time of opening and closing is selected according to the gesture pattern based on a signal notified from the hinge mechanism 14.

  As described above, according to the terminal device according to the second embodiment of the present invention, in the terminal device having an openable / closable housing having the rotation part 12, it is exposed from the outside both when the lid is opened and when the lid is closed. Since the sensor 15 is mounted at the position to be operated, it is possible to perform an operation by performing a gesture on the sensor 15 in any state.

  The gesture operation history 16 and the required time calculation means 17 are provided, and the time until the completion of the gesture is changed based on the information indicating the speed of the gesture operation. As a result, when the operator's operation proficiency level increases and the operator can operate quickly, the operation can be performed more quickly and efficiently.

  In the above description, the processing has been described as resetting the time until completion of the gesture every time, but the time until completion of the gesture is changed only once, as in the learning mode / steady mode, and thereafter May be left unchanged.

  Moreover, although the terminal device of Embodiment 1 has been described as having a foldable housing, and the terminal device 11 has a housing having a rotatable part, the embodiment of the present invention has been described. Needless to say, the present invention can be applied to a case having a so-called flip type or slide type casing.

  The present invention is particularly applicable to a user interface for efficiently operating a portable information terminal such as a mobile phone, or a remote control terminal such as a television, audio, or home appliance.

It is a back perspective view of the terminal device of Embodiment 1 of this invention. It is a front view which shows the state which hold | gripped the terminal device of Embodiment 1 of this invention by hand. It is a front view at the time of lid closing of the terminal device of Embodiment 1 of this invention. It is a block diagram which shows the structure of the terminal device of Embodiment 1 of this invention. It is the figure which showed the detailed structural example of the sensor of Embodiment 1 of this invention. It is a flowchart of operation | movement of the terminal device by Embodiment 1 of this invention. It is a figure which shows the example of distribution of the signal which generate | occur | produces with the sensor of Embodiment 1 of this invention. It is a figure which shows another example of distribution of the signal which generate | occur | produces with the sensor of Embodiment 1 of this invention. It is a figure explaining the relationship between the time series of Embodiment 1 of this invention, and a movement amount vector. It is a table | surface which shows the example of the gesture pattern of Embodiment 1 of this invention. It is a figure which shows the correspondence of the gesture and function of Embodiment 1 of this invention. It is a perspective view at the time of lid opening of the terminal device of Embodiment 2 of this invention. It is another perspective view at the time of lid opening of the terminal device of Embodiment 2 of this invention. It is another perspective view at the time of lid closing of the terminal device of Embodiment 2 of this invention. It is a block diagram which shows the structure of the terminal device of Embodiment 2 of this invention. It is a flowchart which shows operation | movement of the terminal device of Embodiment 2 of this invention. It is explanatory drawing of the state of the signal which the sensor of Embodiment 2 of this invention generate | occur | produces. It is a figure which shows the example of a setting of the timeout value of the gesture completion by Embodiment 2 of this invention.

Explanation of symbols

5, 15 sensors, 6 gesture identification means, 8 control means, 16 gesture operation history, 17 required time calculation means.

Claims (2)

In a terminal device that has a casing that can be gripped with one hand and is used by operating a key with a finger of a hand that grips the casing,
A part that is attached to the casing so as to be freely opened and closed and covers a part of the front of the casing when the lid is closed;
A position where the hand holding the housing can be touched with the finger of the hand while maintaining the position where the key can be operated, and is exposed to the outside both when the part is closed and when the cover is opened. A sensor that detects the contact position of the finger that is worn and touched;
Gesture identification means for cutting out a gesture from a time series of contact positions detected by the sensor;
Control means for combining a plurality of gestures cut out by the gesture identification means and executing a function corresponding to the plurality of gestures combined;
The sensor further detects the size of the contact surface of the finger,
The gesture identification means, from the time series of the contact position detected in the sensor, to cut out the gesture size of the contact surface dismissed time series of the contact position in the time zone below a predetermined value, the input A terminal device characterized by calculating a distance value by matching in which a penalty value is added to an absolute distance value calculated from vector elements of a time series and a gesture pattern, and using the distance value by the matching for calculating a gesture pattern . In a terminal device that has a casing that can be gripped with one hand and is used by operating a key with a finger of a hand that grips the casing,
  A part that is attached to the casing so as to be freely opened and closed and covers a part of the front of the casing when the lid is closed;
  A position where the hand holding the casing can be touched with a finger of the hand while maintaining the position where the key can be operated, and is exposed to the outside both when the part is closed and when the lid is opened. A sensor that detects the contact position of a finger that is worn and touched;
  Gesture identification means for cutting out a gesture from a time series of contact positions detected by the sensor;
  Control means for combining a plurality of gestures cut out by the gesture identification means and executing a function corresponding to the plurality of gestures combined;
The sensor further detects the size of the contact surface of the finger,
The gesture identifying means cuts out the gesture by rejecting the time series of the contact position in the time zone in which the size of the contact surface is below a predetermined value from the time series of the contact position detected by the sensor, Does not immediately determine that the gesture has ended, and the finger is in contact with the sensor surface before the finger is released from the sensor surface. A terminal device characterized by giving an absolute value of a direction and a movement vector based on a difference from a state of being in contact with the surface of the object and recognizing a gesture.

Images