JP2013080999A - Portable terminal device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve operability and convenience in operating a portable terminal device.SOLUTION: A portable terminal device includes: a gripping hand pattern recognition part 16 for recognizing a gripping hand pattern indicating the gripping state and operation state of a user on the basis of information detected by a sensor; a slide locus detection part 13 for detecting the locus of a slide operation by a user with respect to a touch panel display; a slide direction discrimination part 18 for discriminating the direction of a slide operation by a user on the basis of a discrimination area for the discrimination of the detected slide operation locus and a slide operation direction; and a slide amount correction part 20 for correcting the slide amount of an image being displayed on the touch panel display in accordance with the recognized gripping hand pattern, the discriminated slide operation direction and the characteristics of an application for displaying the image.

Description

  The present invention relates to a technique for detecting a user's slide operation on a touch panel display and sliding an image displayed on the touch panel display in response to the detected slide operation.

  Conventionally, an information terminal device including a touch panel display is known. When operating this information terminal device, there is a case where the information terminal device is held in one hand and the baggage is held in the other hand. Even in such a state where one hand cannot be used, there are many situations in which operations are performed. In the information terminal device described above, an operation method called a gesture input operation that assigns functions such as a screen enlargement or a shortcut to a combination of a slide operation for sliding an input by sliding a finger up, down, left and right, or a combination of slide operations is proposed. ing.

  Patent Document 1 discloses a character input device that considers a slide operation distance in addition to a conventional method of assigning characters to a touch position and a slide direction in a flick input in a device capable of touch operation. Japanese Patent Application Laid-Open No. 2004-151867 discloses a device that can perform a touch operation and determines whether it is a gesture input operation or a character input operation from coordinate values acquired from a slide operation and returns a command assigned to the input operation. Is disclosed.

  Japanese Patent Application Laid-Open No. 2004-133620 discloses an apparatus that targets a one-handed operation of a mobile phone, discriminates a user's handle from an operation of tracing a diagonal line on a display, and shifts a menu or a scroll bar to the handle side. . Further, Patent Document 4 discloses an apparatus that discriminates the right and left sides of the hand on the hand determination screen and shifts the menu launcher on the HOME screen to the hand side.

  In addition to these patent documents, there are many proposals for determining a handle for a one-handed operation and changing a GUI (Graphical User Interface).

JP 2011-034289 A JP 2010-231480 A JP 2011-077049 A JP 2009-294725 A

  However, in the technologies disclosed in Patent Documents 1 and 2, a gesture input operation has the same operation and the same operation amount regardless of how to hold the gesture input. Operation occurs. Further, in Patent Documents 3 and 4, the left and right sides of the handle in one-hand operation are determined, and the display positions of GUI parts such as menus and scroll bars are physically shifted. For this reason, a finger can easily reach the GUI component during one-hand operation, but operability when the menu is shifted in the vertical direction or the content is scrolled is not considered.

  FIG. 10 is a diagram illustrating an example of a locus of a slide operation by a user. Even when the user performs the same slide operation, the finger trajectory tends to differ depending on the handle. For example, when the user slides from the bottom to the top with a single-handed operation with the right hand, the thumb does not move directly above, but tends to draw an arcuate locus that approaches the dominant hand as the screen moves upward. Also, it is difficult to slide from right to left (slide in the opposite direction of the handle) compared to slide from left to right (slide in the direction of the handle) by one-handed operation with the right hand. The trajectory tends to be shorter. One-handed operation with the left hand shows a tendency opposite to the above, and two-handed action tends to show a relatively linear locus.

  In this way, even with the same operation, the slide operation trajectory varies depending on the handle, so in the case of an operation with one hand, even if the user intends to perform the up / down operation, the finger actually moves in an arc shape. On the other hand, it may move sideways. For example, as shown in FIG. 11, in an application in which the screen is switched when the slide distance in the left-right direction exceeds the threshold, the user intends to perform the up-down operation as shown in FIG. The screen may be switched when it is erroneously recognized as an operation, and operability and convenience are greatly reduced. Also, in the left / right operation, it is somewhat burdensome to perform the slide operation in the opposite direction to the handle, as much as the amount of slide operation in the direction of the handle.

  Similarly, in an application that does not have a threshold for switching the screen and can be slid in the left and right directions, such as a web browsing application, the screen may move to the left and right although it is intended to slide up and down. Conventionally, the amount of vertical movement that the system reacts only considers the Y direction, so that the system moves the content up and down compared to the amount of finger movement for the slide operation performed by the user. The amount becomes smaller.

  Also, as shown in FIG. 12, in a web browsing application or the like, there is a gesture operation for assigning commands such as a previous screen, a next screen, and an enlargement to a combination of slide operations. Certain movements are difficult to perform with one hand.

  The present invention has been made in view of such circumstances, and determines the direction of the slide operation by the user, and determines the direction of the handle, the direction of the slide operation, and the touch panel display according to the characteristics of the application that displays the image. An object of the present invention is to provide a portable terminal device and a program that can improve the operability and convenience when the portable terminal device is operated with one hand by correcting the slide amount of the displayed image.

  (1) In order to achieve the above object, the present invention takes the following measures. That is, the mobile terminal device of the present invention is a mobile terminal device that detects a user's slide operation on the touch panel display and slides an image displayed on the touch panel display in response to the detected slide operation. Based on the detected information, a handle pattern recognizing unit for recognizing a handle pattern indicating a user's gripping state and operation state, a slide locus detecting unit for detecting a locus of a slide operation by the user on the touch panel display, Based on the detected slide operation locus and the determination region for determining the slide operation direction, a slide direction determination unit for determining the direction of the slide operation by the user, the recognized handle pattern, the determined direction of the slide operation and Special features of applications that display images Depending on, characterized in that it comprises a slide amount correction unit for correcting the amount of sliding of the image displayed on the touch panel display.

  Thus, the slide amount of the image displayed on the touch panel display is corrected according to the recognized hand pattern, the determined slide operation direction, and the characteristics of the application that displays the image. It is possible to improve operability and convenience when operating.

  (2) Moreover, the portable terminal device of the present invention further includes a discrimination area storage unit that stores the discrimination area for each handle pattern.

  As described above, since the determination area is stored for each handle pattern, it is possible to quickly determine the slide direction by reading out the determination area corresponding to the detected handle pattern.

  (3) Further, in the mobile terminal device of the present invention, the discrimination region is a slide direction with at least two curves or straight lines passing through the origin on a plane having the origin as coordinates of a point touched by the user on the touch panel display. It is defined for each.

  In this way, the discrimination area is defined for each slide direction by at least two curves or straight lines passing through the origin on a plane whose origin is the coordinates of the point touched by the user on the touch panel display. It is possible to determine the sliding direction to be performed.

  (4) In the mobile terminal device of the present invention, the application switches an image to be displayed by a slide operation in the horizontal direction on the touch panel display while sliding the image up and down by a slide operation in the vertical direction on the touch panel display. The slide amount correction unit changes a threshold value for switching an image to be displayed by a slide operation in the left-right direction.

  As described above, when the application has a characteristic of sliding an image up and down by a vertical slide operation on the touch panel display and switching an image to be displayed by a horizontal slide operation on the touch panel display, Since the threshold value for switching the image to be displayed is changed, it is possible to determine the slide direction intended by the user.

  (5) Further, in the mobile terminal device of the present invention, the application has a characteristic of sliding an image up and down and left and right by a slide operation in the up and down and left and right directions with respect to the touch panel display. For the slide operation, the slide amount of the image displayed on the touch panel display is corrected by changing the size of the minimum locus that can be detected by the slide locus detector.

  As described above, when the application has a characteristic of sliding the image up, down, left, and right by a slide operation in the up / down / left / right direction with respect to the touch panel display, the minimum track that the slide track detection unit can detect for the slide operation in the left / right direction. By changing the size, the slide amount of the image displayed on the touch panel display is corrected, so that it is possible to determine the slide direction intended by the user.

  (6) In the mobile terminal device of the present invention, the handle pattern may be any of left hand grip and left hand operation, right hand grip and right hand operation, right hand grip and left hand operation, left hand grip and right hand operation, or both hand grip and two hand operation. It is characterized by.

  With this configuration, it is possible to improve operability and convenience when operating the mobile terminal device.

  (7) The program of the present invention is a program for a mobile terminal device that detects a user's slide operation on the touch panel display and slides an image displayed on the touch panel display in response to the detected slide operation. Then, based on the information detected by the sensor, processing for recognizing a handle pattern indicating the user's gripping state and operation state, processing for detecting a locus of a slide operation by the user on the touch panel display, and the detected slide operation The process of determining the direction of the slide operation by the user on the basis of the determination region for determining the locus and the direction of the slide operation, and the characteristics of the application for displaying the recognized handle pattern, the determined direction of the slide operation and the image Depending on the touch panel device. A process of correcting the amount of sliding of the image displayed on the spray, the series of processing, and characterized by causing a computer to execute.

  Thus, the slide amount of the image displayed on the touch panel display is corrected according to the recognized hand pattern, the determined slide operation direction, and the characteristics of the application that displays the image. It is possible to improve operability and convenience when operating.

  According to the present invention, the slide amount of the image displayed on the touch panel display is corrected according to the recognized handle pattern, the determined direction of the slide operation, and the characteristics of the application displaying the image. It is possible to improve operability and convenience when operating with one hand.

It is a figure which shows the handle and batting of the portable terminal device by a user. It is a figure which shows the handle and batting of the portable terminal device by a user. It is a figure which shows the handle and batting of the portable terminal device by a user. It is a figure which shows the handle and batting of the portable terminal device by a user. It is a figure which shows the handle and batting of the portable terminal device by a user. It is a block diagram which shows schematic structure of the portable terminal device which concerns on this embodiment. It is a figure which shows the example which the screen of a portable terminal device changes. It is a figure which shows the example which the screen of a portable terminal device changes. It is a flowchart which shows operation | movement of a portable terminal device. It is a figure which shows the example of a PIN code input screen. It is a figure which shows an example of the discrimination | determination area | region according to a handle pattern. It is a figure which shows an example of the discrimination | determination area | region according to a handle pattern. It is a figure which shows an example of the discrimination | determination area | region according to a handle pattern. It is a figure which shows the example of the locus | trajectory of the slide operation by a user. It is a figure which shows an example of the locus | trajectory of slide operation. It is a figure which shows an example of allocation of gesture operation.

  FIG. 1A to FIG. 1E are diagrams showing a handle and a hand of a mobile terminal device by a user. The portable terminal device according to the present embodiment discriminates such a user's handle and hand (hereinafter simply referred to as “handle pattern”) and detects a slide operation for each handle pattern. And a function for changing a correction coefficient for the movement amount that the system reacts to the input movement amount.

  FIG. 2 is a block diagram illustrating a schematic configuration of the mobile terminal device according to the present embodiment. The mobile terminal device 10 displays an image on the display unit 11. The display unit 11 displays various operation screens for the user. The horizontal direction of the display unit 11 is referred to as the X direction, and the vertical direction is referred to as the Y direction. The touch panel input unit 12 receives an input from the user on the touch panel. And the touch coordinate information when a user touches is acquired. The slide locus detection unit 13 detects and stores a slide locus from when the user presses the touch panel until it is released. The acceleration sensor 14 detects the acceleration at the time of operation by the user.

  The handle pattern storage unit 15 stores the handle pattern and the acceleration in association with each other. The handle pattern recognition unit 16 recognizes the user's handle pattern. As shown in FIGS. 1A to 1E, there are five patterns of handle patterns: left-handed left-handed and right-handed right-handed, left-handed right-handed, right-handed left-handed, and double-handed both-handed. Let each be a handle pattern a, b, c, d, e. Of these, the operation patterns a and b correspond to the one-hand operation described above. For each of the above-mentioned handle patterns, physical operations such as shaking and tilting of the mobile terminal device due to the user's input operation have different characteristics. Therefore, by using these pieces of information, it is possible to determine the handle pattern by pattern recognition.

  The discrimination area storage unit 17 stores an area in which each direction is detected by a slide operation with respect to the handle pattern. The slide direction discriminating unit 18 detects in which direction the user's slide operation is a slide operation from the information of the discriminant area storage unit 17 and the slide trajectory detection unit 13. The slide amount correction value storage unit 19 stores a slide amount correction value for each handle pattern. The slide amount correction unit 20 corrects the slide amount. The above constituent elements are connected by a control bus 21 so that signals can be transmitted and received between them.

  The timing at which the handle pattern recognition unit 16 determines the handle pattern is preferably before the user picks up the portable terminal device 10 and performs a desired operation. In the present embodiment, a usage scene in which a handle pattern is determined at the timing of a password lock release operation by inputting a personal identification number of the mobile terminal device 10, and then a slide operation and a gesture input are illustrated.

  3 and 4 are diagrams illustrating an example in which the screen of the mobile terminal device changes. In the application shown in FIG. 3, the slide operation in the vertical direction is mainly used, and the slide operation in the horizontal direction is used for switching the screen. In the application shown in FIG. 4, a sliding operation not only in the vertical direction but also in the horizontal direction is possible as in the web browsing application.

  Next, the operation of the mobile terminal device according to the present embodiment configured as described above will be described. FIG. 5 is a flowchart showing the operation of the mobile terminal device. First, an input of a password is received from the user (step S1). Here, a password input screen as shown in FIG. 6 is displayed to allow the user to input the password. Next, acceleration information and touch information are acquired, and a handle pattern is determined (step S2).

  When the personal identification number is input in step S1, the acceleration and touch information during the personal identification number input are acquired. In determining the handle pattern, when the input is completed, the handle pattern recognition unit 16 determines the handle pattern based on the acceleration information. The handle pattern recognition unit 16 outputs the above-described operation patterns a to e using the acceleration information as an input value. In other words, based on the acceleration information during the input operation by the user, the user's handle pattern is a left-handed left-handed and right-handed right-handed, left-handed right-handed, right-handed left-handed, or double-handed double-handed pattern. Is determined (step S3). Instead of acceleration, touch information, both information of acceleration and touch information, or a value calculated from the information in addition to both information may be used as the input value. For discrimination, a widely used classifier such as SVM or k-nearest neighbor method is used. Alternatively, it may be an identification unit that presents a GUI for selecting an operation pattern to the user and allows the user to select one. In the present invention, the handle pattern recognizing unit 16 determines whether the user's handle pattern during the input operation is left-handed left-handed, right-handed right-handed, left-handed right-handed, right-handed left-handed, double-handed double-handed Any one of the five patterns can be identified as long as it identifies one of the five patterns. For example, a configuration in which the user inputs information indicating the handle pattern may be employed, or a configuration in which a pressure sensor is provided in the housing and the handle pattern is detected may be employed. Further, another physical sensor, for example, a temperature sensor may be provided to detect the handle pattern.

  Next, the movement amount that the system reacts to the slide area for detecting the slide direction and the input slide amount is changed. That is, in step S4, a slide operation detection area and an operation amount correction coefficient are determined. Next, a state of waiting for input is entered (step S5), and when an input is detected (step S6), the slide locus is stored (step S7). Next, a slide operation amount is detected (step S8), the slide operation amount is corrected (step S9), and operation control is performed (step S10). Then, the display is changed (step S11), and the process proceeds to step S5.

(A) In the case of an application in which the up / down operation is the main and the screen is switched by the left / right operation As an example, the application as shown in FIG. In such an application, the left and right screens are switched when the left and right slide distance exceeds a certain threshold. In the case of an operation with one hand, even if the user intends to perform an up / down operation, since the finger is actually moving in an arc shape, there is a possibility that the screen is erroneously recognized as left and right and the screen is switched.

  FIG. 7 to FIG. 9 are diagrams showing an example of the discrimination area according to the handle pattern. As shown in FIG. 7 to FIG. 9, the slide operation direction and distance conditions by the user are changed. The curve of the discrimination region may be a quadratic function, may be an ellipse or a part of a circle, and may be a function that is not convex downward.

For example, the curved portion F2 of the handle pattern a in FIG.

とするとΔｙ＞Ｆ１，Δｙ＞Ｆ２，Δｙ＞δｙの時に上方向へのスライド操作と判別する。 Straight line portion F1

Then, when Δy> F1, Δy> F2, Δy> δy, it is determined that the sliding operation is upward.

とすることで、ユーザのスライド操作量とシステムが反応する移動量とが直結する。 Compared with the amount of slide performed by the user, the amount of movement that the system reacts becomes smaller as shown in FIG. 11 and is different from the user's intuition. The amount of movement Δy 'that the system reacts to,

By doing so, the user's slide operation amount and the movement amount to which the system reacts are directly connected.

とする。 When switching the screen by sliding in the left-right direction, it is a little burden on the user to perform the slide operation in the direction opposite to the handle and as much as the amount of slide operation in the direction of the handle. Therefore, the threshold for switching the screen between the left slide and the right slide is changed according to the handle pattern. Assuming that the threshold value of the slide amount at which the screen is switched with both hands is δx, the threshold value δx ′ in the sliding operation in the opposite direction to the one-handed handle is

And

(B) In the case of an application that can be moved freely to the left and right The up / down operation is the same as (A) above, but the difference from (A) is that the screen can be freely moved to the left and right without switching. In (A), the threshold of the left / right slide operation for switching the screen is changed, but here, the minimum operation amount detected as the left / right slide operation and the movement amount to which the system reacts are changed.

とする。例えば、スライド操作を検知するには７ｐｘ（＝δｘ）スライドする必要があるアプリに対して、５ｐｘ（＝δｘ’）で検知するということである。 If the minimum operation amount to be detected and the slide operation defined in the application is δx, the minimum operation amount δx 'in the slide operation in the opposite direction to the one-handed handle is

And For example, an application that needs to slide by 7 px (= δx) to detect a slide operation is detected at 5 px (= δx ′).

とすることで、持ち手と反対方向へのスライド操作という動かしにくい操作に対して、小さな操作量で行なうことができる。例えば、片手持ちでの持ち手と反対方向へのスライド操作の場合、実際には２０ｐｘ（＝Δｘ）のスライド操作に対してシステムが反応する移動量は２５ｐｘ（＝Δｘ’）移動するということである。 Further, the movement amount Δx ′ that the system reacts to the operation amount Δx,

By doing so, it is possible to perform a small operation amount with respect to an operation that is difficult to move, such as a slide operation in the opposite direction to the handle. For example, in the case of a slide operation in the opposite direction to a one-handed handle, the movement amount that the system responds to a slide operation of 20 px (= Δx) actually moves by 25 px (= Δx ′). is there.

  Further, in (B), there may be a diagonal operation. As shown in FIG. 9, if the relationship between the change amounts Δx and Δy is an area surrounded by two curves, it is detected as oblique. The two curves may be a quadratic function as in the upper operation of (A), or may be a shape of a part of an ellipse and a circle.

  The above-described x1, x2, y1 and α, β, γ may be determined in advance, or may be estimated from the user's operation history. Further, it may be determined for each handle pattern and sliding direction.

  In this embodiment, the slide amount is changed only for the operation patterns a and b, but may be changed for all patterns. In particular, regarding the pattern e, it may be determined whether the finger is operated with the left or right finger, and left and right may be defined. Further, the correction may be canceled. The sliding directions are shown in the eight directions in FIG. 10, but may be more or less.

(Application examples)
As an application example, the command assignment may be changed for the gesture trajectory. In a web browsing application or the like, as shown in FIG. 12, there is a gesture operation in which commands such as a previous screen, a next screen, and an enlargement are assigned to a combination of slide operations. However, multi-touch and certain movements are difficult to perform with one hand.

  Therefore, a command for gesture input may be defined and the gesture input storage unit may be switched for each handle pattern. For example, with both hands, enlargement and reduction of the screen are assigned to multi-touch, while with one hand, switching to a single-touch gesture is given.

  As described above, according to the present embodiment, the system for the movement direction / movement amount condition for detecting the slide operation and the input movement amount according to the handle and the hand of the portable terminal device. It is possible to improve the input operability and convenience by correcting the movement amount to which the user responds and changing the assignment of the gesture operation to the handle pattern.

DESCRIPTION OF SYMBOLS 10 Mobile terminal device 11 Display part 12 Touch panel input part 13 Slide locus detection part 14 Acceleration sensor 15 Hand pattern storage part 16 Hand pattern recognition part 17 Discrimination area storage part 18 Slide direction determination part 19 Slide amount correction value storage part 20 Slide Quantity correction unit 21 Control bus

Claims (7)

A mobile terminal device that detects a user's slide operation on a touch panel display and slides an image displayed on the touch panel display in response to the detected slide operation,
A handle pattern recognizing unit that recognizes a handle pattern indicating a gripping state and an operation state of the user based on information detected by the sensor;
A slide locus detector for detecting a locus of a slide operation by a user on the touch panel display;
A slide direction discriminating unit for discriminating the direction of the slide operation by the user based on the discriminating area for discriminating the detected locus of the slide operation and the direction of the slide operation;
A slide amount correction unit that corrects the slide amount of the image displayed on the touch panel display according to the recognized handle pattern, the determined direction of the slide operation, and the characteristics of the application that displays the image. The portable terminal device characterized by this.   The mobile terminal device according to claim 1, further comprising a discrimination area storage unit that stores the discrimination area for each handle pattern.   2. The discrimination area is defined for each slide direction by at least two curves or straight lines passing through the origin on a plane having the origin of coordinates of a point touched by the user on the touch panel display. Or the portable terminal device of Claim 2. The application has a characteristic of switching an image to be displayed by a slide operation in the horizontal direction on the touch panel display while sliding an image up and down by a slide operation in the vertical direction on the touch panel display,
The mobile terminal device according to any one of claims 1 to 3, wherein the slide amount correction unit changes a threshold value for switching an image to be displayed by a slide operation in a horizontal direction. The application has a characteristic of sliding an image up and down and left and right by a slide operation in the up and down and left and right directions with respect to the touch panel display,
The slide amount correction unit corrects the slide amount of the image displayed on the touch panel display by changing the size of the minimum locus that can be detected by the slide locus detection unit in response to a slide operation in the horizontal direction. The mobile terminal device according to claim 1, wherein the mobile terminal device is a mobile terminal device.   The said handle pattern is any one of left hand grip and left hand operation, right hand grip and right hand operation, right hand grip and left hand operation, left hand grip and right hand operation, or both hand grip and two hand operation. Item 6. The mobile terminal device according to any one of Items 5 to 6. A program for a mobile terminal device that detects a user's slide operation on a touch panel display and slides an image displayed on the touch panel display in response to the detected slide operation,
A process of recognizing a handle pattern indicating a user's gripping state and operation state based on information detected by the sensor;
Processing for detecting a locus of a slide operation by a user on the touch panel display;
A process of determining the direction of the slide operation by the user based on the determination region for determining the locus of the detected slide operation and the direction of the slide operation;
A process of correcting the slide amount of the image displayed on the touch panel display according to the recognized handle pattern, the determined direction of the slide operation, and the characteristics of the application displaying the image, and a series of processes, A program characterized by being executed by a computer.

Images