JP2006099468A - Gesture input device, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent misidentification of a stroke and a gesture for allowing continuous gesture input by a simple operation for improvement in convenience. <P>SOLUTION: This gesture input device is provided with a display having a tablet function for inputting the positional coordinate instructed by a pen device, a mode switch means switching setting/releasing of a gesture input mode for receiving a gesture based on a time of contact by a pen device with a display area on the display having a tablet function, and a gesture processing part executing processing matching a gesture according to a locus drawn on the display with a tablet function. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a gesture input device, a gesture input method, and a gesture input program for performing gesture input on a coordinate input surface, and further relates to a technique for switching whether or not to perform gesture input.

  2. Description of the Related Art Conventionally, pen input interfaces have been incorporated into various devices such as tablet PCs (Personal Computers) and PDAs (Personal Digital Assistants) due to advances in handwriting recognition technology. The pen input interface improves convenience for users who are not familiar with the keyboard, and can be used as an input device for devices that cannot incorporate a keyboard or mouse. In particular, gesture recognition using a pen input interface can input various commands with a simple operation. However, in gesture recognition, it is difficult to determine the difference between gesture input and other pen input such as stroke input for writing characters.

  Therefore, a technique for determining whether or not the input stroke is a gesture has been proposed (see, for example, Patent Document 1). Also, a switching button for switching to gesture input is provided, and when the switching button is pressed, the pen touches the tablet, and the pen is dragged within a certain time without releasing the pen from the tablet. A technique for determining is proposed (see, for example, Patent Document 2).

JP 2003-308482 A JP 2001-117686 A

  However, in the technique described in Patent Document 1, since it is determined whether or not it is a gesture from the input stroke, if the input stroke is the same as or similar to the gesture, the stroke may be determined as a gesture. There is a problem of misrecognition.

  In the technique described in Patent Document 2, it is not possible to switch between a gesture input and another pen input only by a pen operation, and it is necessary to press a button every time a gesture is input. When inputting, there is a problem that the operation becomes complicated.

  The present invention has been made in view of the above, and provides a gesture input device, a gesture input method, and a gesture input program that are not erroneously recognized in strokes and gestures and that allow continuous gesture input and have improved convenience. It is intended to provide.

  In order to solve the above-described problems and achieve the object, the present invention provides an input means for inputting position coordinates from a coordinate input surface, and a specific trajectory by a plurality of continuous position coordinates input by the input means. Mode switching means for switching the setting and canceling of the gesture input mode for receiving a gesture indicating processing to be executed correspondingly based on the input timing when the position coordinates of an arbitrary region on the coordinate input surface are input from the input means. And gesture processing means for executing processing corresponding to the gesture by the trajectory input by the input means when the gesture input mode is set by switching by the mode switching means. And

  The present invention also provides an input step for inputting position coordinates from a coordinate input surface, and a gesture for receiving a gesture indicating processing to be executed corresponding to a specific trajectory by a plurality of continuous position coordinates input by the input step. An input mode is set and canceled based on an input timing at which the position coordinates of an arbitrary area on the coordinate input surface are input from the input step, and the gesture by the switching by the mode switching step. A gesture processing step for executing processing corresponding to the gesture by the trajectory input in the input step when the input mode is set.

  In addition, the present invention provides an input procedure for inputting position coordinates from a coordinate input surface, and a gesture for receiving a gesture indicating processing to be executed corresponding to a specific trajectory by a plurality of continuous position coordinates input by the input procedure. The input mode is set and canceled based on an input timing at which the position coordinates of an arbitrary region on the coordinate input surface are input from the input procedure, and the gesture by the switching by the mode switching procedure. When the input mode is set, the computer causes the computer to execute a gesture processing procedure for executing processing corresponding to the gesture by the trajectory input by the input procedure.

  According to the present invention, since the input unit switches between setting and canceling the gesture input mode based on the input timing when the position coordinates of an arbitrary region on the coordinate input surface are input, it is easy to switch the gesture input mode. There is an effect that can be done. In addition, since it is not necessary to determine whether or not it is a gesture in the gesture input mode, erroneous recognition of strokes and gestures does not occur, and continuous gesture input is possible in the gesture input mode, improving convenience. The effect of doing.

  Exemplary embodiments of a gesture input device, a gesture input method, and a gesture input program according to the present invention will be explained below in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a block diagram showing a configuration of the PDA 100 according to the first embodiment. As shown in the figure, the PDA 100 includes a display 101 with a tablet function, an input processing unit 102, an input mode switching unit 103, an input mode determination unit 104, a gesture recognition unit 105, a gesture processing unit 106, A stroke processing unit 107 and a display processing unit 108 are provided, and gesture input and stroke input are performed by a pen device 150 provided in the PDA 100.

  In addition, a gesture is different from tapping a button or selecting a menu by holding a predetermined trajectory in the PDA 100 and a process to be executed in association with each other and drawing a predetermined trajectory by the pen device 150. An operation for executing the associated function or process.

  The pen device 150 is a device for a user to input to the tablet function-equipped display 101 described later, and outputs a magnetic signal from the pen tip.

  The display with tablet function 101 is a flat display provided with a magnetic sensor for inputting the position coordinates of the pen device 150. In addition, since the display 101 with a tablet function of the present embodiment uses a magnetic sensor, even if the pen device 150 is not in contact, the pen in the three-dimensional space can be detected by detecting the magnetic signal output from the pen device 150. The position coordinates of the device 150 can be input. The display with tablet function 101 detects a magnetic signal emitted from the pen device 150 and outputs the detected signal to the input processing unit 102 described later. The user uses the pen device 150 to instruct position coordinates with the pen tip on the tablet function display 101 or draw a locus, thereby instructing the PDA 100 to process, characters, symbols, figures, etc. The image information can be input by handwriting.

  The display with tablet function 101 displays the result of processing performed by the display processing unit 108 described later.

  The input processing unit 102 analyzes the signal input from the tablet function-equipped display 101 and determines whether the pen device 150 is in contact with the position coordinates in the space, information on the trajectory of movement, the tablet function-equipped display 101, etc. Pen input information that holds information on whether or not, a time until the pen device 151 comes out of contact, and the like is generated. When the processed pen input information is information indicating that the pen device 150 is in contact with an arbitrary point within the display range of the display 101 with the tablet function, the input processing unit 102 is an input mode switching unit. To 103. In addition, when the locus is drawn with the pen device 150 on the display 101 with the tablet function, the input processing unit 102, for example, a process for connecting position coordinates of magnetic signals detected continuously or a process for calculating a locus using a function, etc. A trajectory is derived from successive points using a known derivation method.

  The input mode switching unit 103 indicates that the pen input information input by the input processing unit 102 indicates that the pen device 150 has not released an arbitrary point in the display area on the tablet function-equipped display 101 for two seconds or more. If it does, the input mode is switched. As a detailed example, the input mode switching unit switches to the gesture input mode if the input mode is a stroke input mode for inputting characters or graphics when the above switching condition is satisfied. If the input mode is the gesture input mode when the above switching condition is satisfied, the mode is switched to the stroke input mode. Further, since the input pen input information holds the position coordinates in the space and whether or not it is touching, the input mode switching unit 103 can input the pen input information continuously by inputting the pen input information. It becomes possible for the device 151 to obtain the contact time of the display area of the display 101 with the tablet function.

  In the present embodiment, there are two types of input modes, the stroke input mode and the gesture input mode, but the input mode is not limited to two types. In addition, the input mode switching condition by the input mode switching unit 103 is not limited to a case where an arbitrary point in the display area is in contact for 2 seconds or more. You may comprise so that an input mode may be switched if it contacts for the defined time. Moreover, you may decide to set the contact time for a user to switch an input mode.

  The input mode switching condition is not limited to the case where the pen device 150 is in contact for a predetermined time. For example, the pen device 150 contacts the display 101 with the tablet function twice within the predetermined time. In such a case, the input mode may be switched based on the number of contacts within a predetermined period.

  The input mode determination unit 104 determines the input mode switched by the input mode switching unit 103. When the input mode is determined to be the stroke input mode, the pen input information output from the input processing unit 102 is used as the stroke processing unit 107. When the input mode is determined to be the gesture input mode, the pen input information output from the input processing unit 102 is output to the gesture recognition unit 105.

  The gesture recognition unit 105 recognizes a gesture from a trajectory drawn on the tablet function-equipped display 101 held by the pen input information input from the input processing unit 102. Any existing method may be used as the gesture recognition method.

  The gesture processing unit 106 performs processing associated with the gesture recognized by the gesture recognition unit 105. For example, in a case where the process associated with the gesture “check” is “selection”, if the recognized gesture is “check” within a predetermined area, the gesture processing unit 106 is determined in advance. A process for selecting a candidate or the like pointed to by the area is performed.

  The stroke processing unit 107 extracts only the locus in the display area drawn on the display 101 with the tablet function from the input pen input information, and outputs it to the display processing unit 108. Thereby, the display processing unit 108 can display the locus drawn in the display area.

  The display processing unit 108 displays the result of processing associated with the gesture by the gesture processing unit 106, or displays the locus drawn on the display 101 with the tablet function by the pen device 150 extracted by the stroke processing unit 107. Process to do.

  Next, overall processing from the input by the pen device 150 to the display on the display 101 with the tablet function in the PDA 100 according to the present embodiment configured as described above will be described. FIG. 2 is a flowchart from the input by the pen device 150 to the display on the display 101 with the tablet function in the PDA 100 according to the present embodiment.

  The display 101 with the tablet function inputs the position coordinates instructed using the pen device 150 (step S201). Next, the input processing unit 102 generates pen input information based on the position coordinates and the like input from the display with tablet function 101 (step S202). Then, the input processing unit 102 determines whether or not the pen input information is information indicating that an arbitrary point within the display range of the tablet function display 101 by the pen device 150 is in contact (step S203). If it is information indicating that it is in contact, the pen input information is output to the input mode switching unit 103 (step S203: Yes).

  Then, the input mode switching unit 103 determines from the pen input information whether or not the pen device 150 has been in contact with an arbitrary point in the display area on the tablet function-equipped display 101 for two seconds or more (step S210). And when it determines with having been in contact for 2 seconds or more (step S210: Yes), the input mode switching part 103 switches an input mode (step S211). As a detailed example, the input mode switching unit 103 switches to the gesture input mode when the input mode is the stroke input mode, and switches to the stroke input mode when the input mode is the gesture input mode. In addition, when it is determined that the contact has not been made for 2 seconds or more (step S210: No), the input mode switching unit 103 does not perform any particular processing.

  If the pen input information is not information indicating that the touch has been made (step S203: No), the input mode determination unit 104 determines whether or not the input mode is the gesture input mode (step S204).

  When the input mode determination unit 104 determines that the gesture input mode is set (step S204: Yes), the pen input information generated by the input processing unit 102 is output to the gesture recognition unit 105. Next, the gesture recognition unit 105 recognizes the gesture from the locus drawn on the display 101 with the tablet function held by the pen input information (step S205). The gesture processing unit 106 performs processing associated with the gesture recognized by the gesture recognition unit 105 (step S206).

  When the input mode determination unit 104 determines that the mode is not the gesture input mode (step S204: No), the stroke processing unit 107 extracts the locus drawn on the display 101 with the tablet function held by the input pen input information. Is performed (step S207).

  Then, the display processing unit 108 performs a process for displaying the result of the process in step S206 or a process for displaying the locus drawn on the display with tablet function 101 extracted in step S207 (step S207). S208). The display with tablet function 101 displays the result processed by the display processing unit 108 (step S209).

  In the above processing procedure, after the input mode is switched by the input mode switching unit 103, no particular processing such as display is performed. However, a process for displaying that the input mode has been switched may be performed. .

  Further, according to the above-described processing procedure, the user can switch the input mode by touching the pen device 150 with any one point on the display 101 with the tablet function, and the user can continuously switch in the gesture input mode. Gesture input is possible.

  FIG. 3 shows an example of a screen in which gesture input and stroke input are performed on the PDA 100 according to the present embodiment. As a detailed example, first, in the gesture input mode, the user inputs a gesture by continuously drawing a trajectory associated with the check in the area indicated by 401 and 402. The PDA 100 recognizes the gesture in each area, and the display indicating that it is checked is made on the display 101 with the tablet function. Thereafter, the user touches one point in the tablet function-equipped display 101 with the pen device 150 for two seconds or more, so that the PDA 100 switches the input mode to the stroke input mode. Then, when the user inputs a stroke “sweat comes out” in the area indicated by 403 by the pen device 150, the PDA 100 displays the characters input in the area 403.

  FIG. 4 is a diagram showing a hardware configuration of the PDA 100 according to the present embodiment. The PDA 100 according to the present embodiment includes a CPU (Central Processing Unit) 601 serving as a control device, a ROM (Read Only Memory) 602 and a RAM 603 serving as an external storage device, a display unit 604 such as a display with a tablet function, and a tablet The hardware configuration includes an input I / F 605 for inputting a signal from the function-equipped display and a communication I / F 606 connected to an external device or a network. The ROM 602 stores a basic program such as an OS for controlling the entire PDA and application software that can be used by the PDA.

  The gesture input program executed by the PDA 100 according to the present embodiment is an installable or executable file, such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk), or the like. Recorded on a readable recording medium. Installation from an external device via the communication I / F 606 is possible.

  Further, the gesture input program executed by the PDA 100 of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the gesture input program executed by the PDA 100 of this embodiment may be provided or distributed via a network such as the Internet.

  Further, the gesture input program according to the present embodiment may be provided by being incorporated in advance in the ROM 602 or the like.

  The gesture input program executed by the PDA 100 according to the present embodiment includes the above-described units (input processing unit, input mode switching unit, input mode determination unit, stroke processing unit, gesture recognition unit, gesture processing unit, display processing unit). As the actual hardware, the CPU (processor) 601 reads the gesture input program from the storage medium and executes it to load the above-mentioned units onto the main storage device. A mode switching unit, an input mode determination unit, a stroke processing unit, a gesture recognition unit, a gesture processing unit, and a display processing unit are generated on the main storage device.

  In the present embodiment, the PDA 100 is used as a gesture input device. However, instead of the PDA 100, a PC (Personal Computer) or the like may be used. For example, in the case of a configuration in which gesture input is performed using a tablet placed on a table and a pen device, and the result of processing corresponding to the input gesture is displayed on the display of the PC, an arbitrary area of the tablet is displayed. It is conceivable to switch between the gesture input mode and the stroke input mode when the pen device is in contact for a predetermined time.

  As described above, in the first embodiment, there is no misjudgment between gesture input and stroke input, and continuous gesture input is possible even when there is an interval for gesture input at the time of gesture input. Can be recognized. Further, the input mode can be easily switched by the pen device 150 depending on the contact time of the display 101 with the tablet function. For this reason, in the PDA 100 according to the first embodiment, misrecognition is reduced, and continuous gesture input is possible, so convenience is improved.

(Second Embodiment)
In the first embodiment, the PDA 100 switches the input mode when an arbitrary point on the display with tablet function 101 is in contact with the pen device 150 for two seconds or more. However, when the input mode is gesture input, the user needs to know in advance which gesture can be input in which area on the tablet function-equipped display 101. For this reason, in the second embodiment, a gesture that can be input within an area on the display with tablet function 101 is displayed.

  FIG. 5 is a block diagram showing the configuration of the PDA 300 according to the second embodiment of the present invention. The PDA 100 according to the first embodiment described above includes a gesture candidate acquisition unit 301 and a storage unit 302, is changed to an input processing unit 305 that is different in processing from the input processing unit 102, and is different from the display processing unit 108. The difference is that the processing is changed to a different display processing unit 303. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted. Since the PDA 300 according to the present embodiment has the above-described configuration, the pen device 150 and the display with tablet function 101 are in a non-contact state due to the magnetic signal of the pen device 150, but within a certain range. When recognized, it is possible to display an inputable gesture in a pop-up format.

  The storage unit 302 stores a display gesture table and a gesture area table, and is used when a gesture to be displayed is acquired by a gesture candidate acquisition unit 301 described later.

  FIG. 6 is a diagram showing an example of a display gesture table stored in the storage unit 302 according to the present embodiment. As shown in the figure, a record in which a set ID is associated with a gesture ID, a gesture name, and a display flag is held. The set ID is an ID associated with the gesture area in the gesture area table. The gesture ID is an ID associated with a trajectory for recognizing the gesture. The gesture name is a name indicating a gesture, and is used when displaying a gesture that can be input on the display 101 with a tablet function. The display flag is a flag indicating whether or not to display, and is displayed when the display flag is “1”. The correspondence between the trajectory for recognizing the gesture and the gesture ID is omitted in this embodiment.

  FIG. 7 is a diagram showing an example of a gesture area table used in the present embodiment. As shown in the figure, a record in which a gesture area is associated with the set ID described above is held. Note that the gesture area 1 or the like indicating the gesture area refers to a predetermined area on the screen displayed on the display 101 with the tablet function.

  FIG. 8 is an explanatory diagram showing a predetermined area of the screen displayed on the display with tablet function 101 and a gesture area name in association with each other. As shown in the figure, a gesture area is assigned to each area of the screen displayed on the display 101 with the tablet function. In addition, a different gesture area is held every time the displayed screen is different. And the memory | storage part 302 memorize | stores the table mentioned above, It becomes possible to change the gesture recognized for every area | region.

  Returning to FIG. 5, the input processing unit 305 indicates that the pen device 150 is in a non-contact state with the tablet function display 101 from the signal input from the tablet function display 101. If the distance is determined to be within a certain range, the non-contact state and the position coordinate at which the magnetic signal is most strongly input on the tablet function display 101 are output to the gesture candidate acquisition unit 301 as pen input information. . The processing when the pen device 150 is in contact with the display with tablet function 101 is the same processing as the input processing unit 102 according to the first embodiment.

  The gesture candidate acquisition unit 301, based on the display gesture table and the gesture region table, from the input pen input information only when the input mode determination unit 104 determines the gesture input mode, the gesture ID and the gesture to be displayed. Get the name. A specific acquisition procedure will be described later.

  The display processing unit 303 is added with a gesture display processing unit 304 as compared with the display processing unit 108 according to the first embodiment. The display processing unit 303 performs the same processing as the display processing unit 108 according to the first embodiment for processes other than the gesture display.

  When the gesture display processing unit 304 receives a gesture ID and a gesture name from the gesture candidate acquisition unit 301, the gesture display processing unit 304 displays the gesture trajectory and the gesture name corresponding to the gesture ID in a pop-up format.

  Next, a process from detection of the pen device 150 to display of an inputable gesture in a pop-up format in the PDA 300 according to the present embodiment configured as described above will be described. FIG. 9 is a flowchart from when the pen device 150 is detected in the PDA 300 according to the present embodiment until an inputable gesture is displayed in a pop-up format.

  The tablet function-equipped display 101 receives a magnetic signal output from the pen device 150 (step S601). The input processing unit 305 performs an input process of position coordinates obtained by analyzing the input signal from the tablet function-equipped display 101 (step S602). For example, when the input processing unit 305 determines that the pen device 150 is not in contact with the tablet function display 101 as a result of analyzing the signal, the magnetic signal output from the pen device 150 is displayed on the tablet function display 101. The input process is performed using the position coordinate detected most strongly as the position coordinate detected by the pen device 150, and the pen input information generated from the non-contact and the position coordinate is output. When it is determined that the pen device 150 is in contact with the display 101 with the tablet function, the processing procedure shown in the first embodiment is performed (not shown).

  Next, the input mode determination unit 104 determines whether or not the input mode is the gesture input mode (step S603). If the input mode determination unit 104 determines that the mode is not the gesture input mode (step S603: No), the process ends without performing the process.

  When the input mode determination unit 104 determines that it is the gesture input mode (step S603: Yes), the gesture candidate acquisition unit 301 determines in which gesture region the position coordinates based on the pen input information input from the input processing unit 305 are. Then, the gesture ID and the gesture name of the gesture that can be input are acquired (step S604). A specific procedure for obtaining the gesture ID and the gesture name will be described later.

  The gesture display processing unit 304 performs a process for displaying, in a pop-up format, an icon and a gesture name indicating the gesture trajectory associated with the gesture ID acquired by the gesture candidate acquisition unit 301 (step S605). Then, the display with tablet function 101 displays an inputable gesture in a pop-up format based on the signal input by the gesture display processing unit 304 (step S606).

  Next, a process until the gesture candidate acquisition unit 301 of the PDA 300 according to the present embodiment configured as described above acquires a gesture candidate to be displayed based on the pen input information will be described. FIG. 10 is a flowchart showing the entire process until the gesture candidate acquisition unit 301 of the PDA 300 according to the present embodiment acquires a gesture candidate to be displayed based on the pen input information.

  The gesture candidate acquisition unit 301 specifies in which gesture region the position coordinate where the magnetic signal of the pen device 150 held by the pen input information input from the input processing unit 305 is detected is located (step S901).

  Next, the gesture candidate acquisition unit 301 searches the gesture area table from the specified gesture area and acquires a set ID (step S902). For example, when the specified gesture area is GestureArea1, the gesture candidate acquisition unit 301 acquires the set ID “01” when searching the gesture area table shown in FIG.

  Based on the acquired set ID, the gesture candidate acquisition unit 301 searches the display gesture table and acquires a gesture ID and a gesture name to be displayed (step S903). However, only the gesture ID and the gesture name when the display flag is “1” are acquired. For example, based on the condition of the display flag “1” with the set ID “01”, the gesture candidate acquisition unit 301 searches the display gesture table illustrated in FIG. 6, thereby gesturing the gesture name “Check” with the gesture ID “01”. The gesture name “Circle” is acquired with “and gesture ID“ 02 ”. Then, the gesture candidate acquisition unit 301 outputs the acquired gesture to the gesture display processing unit 304.

  FIG. 11 is a diagram illustrating a screen example of the tablet function-equipped display 101 on which gestures that can be input based on the above-described process are displayed in a pop-up format. Through the processing described above, the display 101 with the tablet function displays “Check” and “Circle”. The gestures 702 and 703 display a trajectory corresponding to the gesture ID. By drawing the trajectory displayed by the pen device 150, processing corresponding to the gesture is performed. The gesture name 'All' is a gesture for displaying all the gestures that can be input, and is always displayed when displayed in a pop-up format. When the trajectory 704 corresponding to the gesture “All” is drawn, all the gestures corresponding to the gesture area including the position coordinates detected by the pen gesture 150 held by the display gesture table regardless of the display flag are provided with the tablet function. It is displayed on the display 101.

  In addition, only when it is determined that it is the gesture input mode, it is not limited to displaying the input gesture in the pop-up format, but the input gesture in the stroke input mode is displayed in the pop-up format. Anyway.

  In addition, although the display 101 with the tablet function is in a non-contact state between the pen device 150 and the display 101 with the tablet function, the recognition that the display is within a certain range is determined by the strength of the magnetic signal. The distance between the display with tablet function 101 and the pen device 150 indicated by is determined in consideration of the strength of the actual magnetic signal and the convenience of use by the user.

  Note that the gesture that can be input when it is determined to be non-contact is not limited to being displayed in a pop-up format, and a gesture that can be input when the user is in contact may be displayed. However, it is necessary to prevent the input mode switching process from being confused with the pop-up display process when the input mode is switched twice within a predetermined time.

  In addition, the display of gestures that can be input is not limited to display in a pop-up format, and any gesture that can be input by a user may be displayed.

  As described above, the PDA 300 according to the second embodiment displays the gestures that can be input for each gesture region, so that the user can input the gesture after visually recognizing the input gesture. As a result, the convenience of the PDA 300 is improved.

(Third embodiment)
In the second embodiment, the PDA 300 displays all input gestures if the display flag is “1” according to the area. However, in the third embodiment, the gestures that can be input are displayed based on the frequency at which the gestures are input.

  FIG. 12 is a block diagram showing a configuration of a PDA 1000 according to the third embodiment. The PDA 300 according to the second embodiment described above includes a gesture history update unit 1003, is changed to a gesture candidate acquisition unit 1001 that is different in processing from the gesture candidate acquisition unit 301, and has a table stored in the storage unit 302. It is different in that it is changed to a different storage unit 1002. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  The storage unit 1002 stores a display gesture history table and a gesture region table, and is used when a gesture to be displayed is acquired by a gesture candidate acquisition unit 1001 described later. The structure of the gesture area table is the same as that of the gesture area table according to the second embodiment, and a description thereof will be omitted.

  FIG. 13 is a diagram showing an example of a display gesture history table used in the present embodiment. As shown in the figure, a record in which a set ID is associated with a gesture ID, a gesture name, and a frequency is held. Since the set ID, the gesture ID, and the gesture name have been described in the display gesture table according to the second embodiment, a description thereof will be omitted. The frequency indicates the number of times the associated gesture is input from the user.

  Returning to FIG. 12, when the input mode determination unit 104 determines the gesture input mode, the gesture candidate acquisition unit 1001 can input gestures that can be input from the input pen input information based on the display gesture table and the gesture region table. The gesture ID and the gesture name corresponding to one gesture are acquired in descending order of the input frequency. A specific acquisition procedure will be described later.

  The gesture history update unit 1003 updates the display gesture history table stored in the storage unit 1002 by adding 1 to the history associated with the gesture recognized by the gesture recognition unit 105. By including the gesture history update unit 1003, it is possible to display, in a pop-up format, gestures that can be input in the order of the highest frequency of input, based on the latest history information.

  In the PDA 1000 according to the present embodiment configured as described above, the processing from when the pen device 150 is detected until the inputable gesture is displayed in a pop-up format is the same as that in the second embodiment shown in FIG. Although it is the same as that of the process until it displays in such a pop-up format in principle, it is necessary to change only the process by the gesture candidate acquisition part 301 performed in step S604 to the process by the gesture candidate acquisition part 1001. This entire processing procedure is omitted.

  Next, a processing procedure until the gesture candidate acquisition unit 1001 of the PDA 1000 according to the present embodiment configured as described above acquires a gesture candidate to be displayed based on the pen input information will be described. FIG. 14 is a flowchart until the gesture candidate acquisition unit 301 of the PDA 1000 according to the present embodiment acquires a gesture candidate to be displayed based on the pen input information.

  First, the gesture candidate acquisition unit 1001 acquires a set ID based on the position coordinates at which the magnetic signal of the pen device 150 is detected in the same manner as steps S901 to S902 in FIG. 10 according to the second embodiment ( Steps S1201 to S1202).

  Then, the gesture candidate acquisition unit 1001 searches the display gesture history table, and acquires a gesture ID and a gesture name of the gesture having the same set ID and the highest frequency as the acquired set ID (step S1203). For example, the gesture candidate acquisition unit 1001 searches the display gesture history table illustrated in FIG. 13 under the condition that the frequency is the highest with the set ID “01”. Next, as a search result, the gesture candidate acquisition unit 1001 acquires a gesture ID “01” and a gesture name “Check” associated with the set ID “01” and the frequency “5”. The gesture candidate acquisition unit 1001 then outputs the acquired gesture ID and gesture name to the gesture display processing unit 304.

  Then, the gesture display processing unit 304 performs processing for displaying the gesture based on the input gesture ID and the gesture name, so that the most frequently input gesture that can be input to the tablet function-equipped display 101 is displayed. .

  FIG. 15 is a diagram illustrating a screen example of the display 101 with a tablet function in which a gesture that can be input based on the above-described process is displayed in a pop-up format. The display with tablet function 101 displays “Check”, which is the most frequently used gesture that can be input. Similarly to the second embodiment, the gesture name 'All' is a gesture for displaying all the gestures that can be input, and is always displayed when displaying in a pop-up format. When a trajectory corresponding to the gesture “All” is drawn, all the gestures that can be input corresponding to the gesture area of the position coordinates detected by the pen device 150 held in the display gesture history table regardless of the frequency are tablet functions. It is displayed on the attached display 101.

  FIG. 16 is a diagram showing a screen example of the display 101 with a tablet function displayed in a pop-up format when the PDA 1000 has not yet performed gesture recognition. In the initial state, since the frequencies of the display gesture history table stored in the storage unit 1002 are all set to “0”, the gesture candidate acquisition unit 1001 displays as shown in this figure because there is no gesture with the highest frequency. As another display method, displaying all gestures on the display with tablet function 101 in the initial state may be considered.

  It should be noted that the display is not limited to one in the descending order of frequency, but may be a predetermined number from the descending order of frequency. In addition, the present embodiment is not limited to display in the order of frequency, and may be anything that determines whether or not to display a gesture using frequency. As another example of the display method, it is conceivable to display the gestures in ascending order of frequency.

  In the present embodiment, the gesture to be displayed is determined based on the frequency. However, it is also possible to display the gesture based on information other than the frequency. For example, it is possible to display not the frequency but the last input gesture. In this case, the table held in the storage unit is a display gesture update time table that holds the column “update time” instead of the column “frequency” of the display gesture history table. A gesture time update unit that updates the update time with respect to the display gesture update time table is provided, and the update time corresponding to the gesture is updated each time a gesture is input. This improves the convenience because the last input gesture is displayed. Further, the gesture may be displayed in a pop-up format by combining the update time and the frequency.

  In the third embodiment, the gestures are displayed in the order of frequency of use for each gesture region. For this reason, gestures that are considered to be less important because they are used less frequently are not displayed, and only those that are considered important because they are used more frequently are displayed, so the user is important immediately after the gesture is displayed. A user can perform a desired operation by visually recognizing a gesture having a high degree, and the convenience of the PDA 1000 is improved.

  In addition, since it is determined whether or not to display a gesture based on the frequency of using the user's gesture, it is possible to display an appropriate gesture based on the user's usage status, and the convenience of the PDA 1000 is improved.

(Fourth embodiment)
Until the PDA according to the third embodiment, the size of the recognized gesture was not determined. In the PDA according to the fourth embodiment, the processing to be performed is changed depending on the size of the recognized gesture.

  FIG. 17 is a block diagram showing a configuration of a PDA 1600 according to the fourth embodiment. The PDA 100 according to the first embodiment described above is different from the PDA 100 according to the first embodiment in that the processing is changed to a gesture recognition unit 1601 that is different from the gesture recognition unit 105 and is changed to a gesture processing unit 1603 that is different from the gesture processing unit 106. . In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  The gesture recognition unit 1601 recognizes a gesture from a trajectory drawn on the display 101 with a tablet function held by the input pen input information. The gesture recognition unit 1601 includes an area calculation unit 1602 and calculates the size of the area based on the locus drawn as the gesture. Any existing method may be used as the gesture recognition method.

  The area calculation unit 1602 calculates the area of the area occupied by the trajectory drawn as a gesture. The area calculation unit 1602 is a rectangular area having a side parallel to the side of the drawing area of the tablet function display 101 as the area of the area occupied by the locus, and the area of the rectangle circumscribing the locus drawn as the gesture is calculated. calculate.

  The gesture processing unit 1603 performs processing based on the gesture recognized by the gesture recognition unit 105 and the area of the region calculated by the region calculation unit 1602. For example, when a map is drawn on the display with tablet function 101 and the recognized gesture is an enlargement / reduction, enlargement is based on the area occupied by the locus drawn as the gesture and the area of the display area of the map The rate is derived, and the display with tablet function 101 performs processing such as enlarging and displaying the map with the derived magnification rate. Specific processing will be described later.

  Next, processing from the input from the pen device 150 to the display on the display 101 with the tablet function in the PDA 1600 according to the present embodiment configured as described above will be described. FIG. 18 is a flowchart from the PDA 1600 according to the present embodiment until it is input by the pen device 150 and displayed on the display 101 with the tablet function. In this figure, the input mode switching process based on the contact time of pen input is omitted.

  The display 101 with the tablet function inputs the position coordinates instructed using the pen device 150 (step S1701). Next, the input processing unit 102 generates pen input information based on the position coordinates and the like input from the tablet function-equipped display 101 (step S1702). Then, the input mode determination unit 104 determines whether or not the input mode is the gesture input mode (step S1703). When the input mode determination unit 104 determines that the input mode is the stroke input mode (step S1703: No), the pen input information is sent to the stroke processing unit 107 in the same manner as the processing procedure described in the first embodiment. The stroke processing unit 107 performs processing (not shown).

  When the input mode determination unit 104 determines that the input mode is the gesture input mode (step S1703: Yes), the pen input information generated by the input processing unit 102 is output to the gesture recognition unit 105. The gesture recognition unit 1601 recognizes the gesture from the locus drawn on the display 101 with the tablet function held by the pen input information (step S1704). Then, the area calculation unit 1602 calculates the area of the area occupied by the locus drawn on the tablet function-equipped display 101 (step S1705).

  The gesture processing unit 1603 performs processing based on the gesture recognized by the gesture recognition unit 1601 and the area of the region calculated by the region calculation unit (step S1706). As a detailed example, the area of the calculated region is compared with the area of the display region of the map, and whether to enlarge or reduce is determined based on the comparison result. Further, it is conceivable to determine the enlargement ratio or reduction ratio according to the area ratio.

  Next, the display processing unit 108 performs processing for displaying the processing result in step S1706 (step S1707). The display with tablet function 101 displays the result processed by the display processing unit 108 (step S1708).

  By the processing procedure described above, processing based not only on the input gesture but also on the area occupied by the region drawn as the trajectory becomes possible.

  Next, in the processing procedure described above, processing when the gesture recognition unit 1601 recognizes a gesture indicating reduction / enlargement will be described. FIG. 19 is a flowchart showing a detailed processing procedure from step S1704 to step S1706 in FIG. 18 when the gesture recognition unit 1601 recognizes a gesture indicating reduction / enlargement. In this processing procedure, it is assumed that the display with tablet function 101 displays a map.

  The gesture recognition unit 1601 recognizes a gesture indicating enlargement / reduction from the locus drawn on the tablet function-equipped display 101 held in the pen input information (step S1801).

  Next, the area calculation unit 1602 calculates the area of the area occupied by the locus drawn on the tablet function-equipped display 101 (step S1802).

  The gesture processing unit 1603 performs enlargement / reduction from the information input from the gesture recognition unit 1601, and further determines whether or not the area area calculated by the area calculation unit 1602 is larger than the area of the map drawing area. (Step S1803).

  If the gesture processing unit 1603 determines that the region calculated by the region calculation unit 1602 is smaller (step S1803: No), the gesture processing unit 1603 calculates the area area calculated by the region calculation unit 1602 and the map area. The ratio of the area of the drawing area is calculated (step S1804).

  FIG. 20 is a diagram illustrating a screen example when the gesture processing unit 1603 determines that the area of the area calculated from the map drawing area is smaller. An ellipse indicated by a black line is a gesture drawn by the pen device 150, and an area calculated by the area calculation unit 1602 is an area 1901 indicated by a rectangle by a broken line. In step S1803, the gesture processing unit 1603 compares the area of the region 1901 with the area of the map drawing region 1902. Next, in step S1804, the gesture processing unit 1603 calculates the ratio of the area of the map drawing area 1902 and the area of the area 1901.

  Returning to FIG. 19, the gesture processing unit 1603 outputs to the display processing unit 108 that the map is drawn with the calculated ratio as an enlargement ratio (step S <b> 1805). That is, the display processing unit 108 to which the enlargement ratio is input performs a process for displaying the area indicated by the area 1901 in the entire drawing area 1902.

  If the gesture processing unit 1603 determines that the region calculated by the region calculation unit 1602 is larger (step S1803: Yes), the gesture processing unit 1603 displays that the map is drawn with the default reduction ratio. (Step S1806). The default reduction ratio indicates a preset reduction ratio, and is set based on user convenience. As the default reduction rate, for example, 1/2 may be considered.

  FIG. 21 is a diagram showing a screen example when the gesture processing unit 1603 determines that the area of the area calculated from the map drawing area is larger. An ellipse indicated by a black line is a gesture drawn by the pen device 150, and an area calculated by the area calculating unit 1602 is an area 2001 indicated by a rectangle indicated by a broken line. In step S1803, the gesture processing unit compares the area of the area 2001 with the area of the map drawing area 1902.

  The display processing unit 108 that has acquired the effect of reducing and enlarging through the above-described processing performs processing for reducing and enlarging the display, and the display with tablet function 101 can display the reduced and enlarged map.

  The PDA 1600 according to the present embodiment can select reduction or enlargement by changing only the gesture size when drawing with a pen device, and can also change the enlargement ratio.

  Note that the present embodiment does not limit the determination of the size of a gesture to the determination based on the area.

  Further, in the present embodiment, the case of enlarging / reducing based on the area of the trajectory drawn as a gesture has been described. However, when a gesture for changing the drawing area is recognized, based on the direction and direction of the gesture. It can be used for various processes such as changing the drawing range. For example, when an arrow is drawn as a gesture, when a map display process in the direction of the arrow is performed, the amount of movement of a map drawing area may be determined in proportion to the size of the arrow.

  In the present embodiment, the description has been given of the mode in which the gesture recognition unit 105 of the PDA 100 according to the first embodiment is changed to the gesture recognition unit 1601, but the PDA 300 according to the second embodiment or The above-described processing may be performed in a form in which the gesture recognition unit 105 of the PDA 1000 according to the third embodiment is changed to the gesture recognition unit 1601 and the gesture processing unit 106 is changed to the gesture processing unit 1603.

  In the fourth embodiment, the processing performed by the PDA 1600 can be easily changed based on the size of the region of the locus drawn by the user.

  The gesture input device, the gesture input method, and the gesture input program are useful for a device that performs gesture input, and are particularly useful as a technique for switching whether or not a trajectory input by a pen device is used as a gesture input.

It is a block diagram which shows the structure of PDA which concerns on 1st Embodiment. It is a flowchart until it displays on the display with a tablet function from the input by a pen device in PDA which concerns on 1st Embodiment. It is the figure which showed the example of a screen by which gesture input and stroke input were carried out about PDA which concerns on 1st Embodiment. It is the figure which showed the hardware constitutions of PDA which concerns on 1st Embodiment. It is a block diagram which shows the structure of PDA which concerns on 2nd Embodiment. It is the figure which showed an example of the display gesture table which the memory | storage part of PDA which concerns on 2nd Embodiment memorize | stores. It is the figure which showed an example of the gesture area | region table which the memory | storage part of PDA which concerns on 2nd Embodiment memorize | stores. It is explanatory drawing which matched and showed the predetermined area | region and gesture area | region name of the screen displayed on the display with a tablet function of PDA which concerns on 2nd Embodiment. It is a flowchart until it displays in a pop-up form the gesture which can be input after detecting a pen device in PDA concerning a 2nd embodiment. It is a flowchart until the gesture candidate acquisition part of PDA which concerns on 2nd Embodiment acquires the candidate of the gesture to display based on pen input information. It is the figure which showed the example of a screen of the display with a tablet function in which the gesture which can be input in PDA which concerns on 2nd Embodiment was displayed in the pop-up format. It is a block diagram which shows the structure of PDA concerning 3rd Embodiment. It is the figure which showed an example of the display gesture history table which the memory | storage part of PDA concerning 3rd Embodiment memorize | stores. It is a flowchart until the gesture candidate acquisition part of PDA which concerns on 3rd Embodiment acquires the candidate of the gesture to display based on pen input information. It is the figure which showed the example of a screen of the display 101 with a tablet function by which the gesture which can be input in PDA which concerns on 3rd Embodiment was displayed in the pop-up format. It is the figure which showed the example of a screen of the display with a tablet function displayed in the pop-up form in the state where PDA which concerns on 3rd Embodiment has not performed gesture recognition at all. It is a block diagram which shows the structure of PDA which concerns on 4th Embodiment. It is a flowchart until it inputs with a pen device and displays on a display with a tablet function in PDA concerning a 4th embodiment. It is a flowchart until it performs the process corresponding to the recognized gesture after the gesture recognition part recognizes it as the gesture which shows reduction expansion in PDA which concerns on 4th Embodiment. It is the figure which showed the example of a screen in case the gesture process part determines with the area of the area | region calculated in the area | region calculation part being smaller than the drawing area of a map in PDA which concerns on 4th Embodiment. It is the figure which showed the example of a screen when it is determined that the area of the area | region calculated by the area | region calculation part is larger than the drawing area of a map in the PDA which concerns on 4th Embodiment.

Explanation of symbols

100, 300, 1000, 1600 PDA
101 Display with Tablet Function 102, 305 Input Processing Unit 103 Input Mode Switching Unit 104 Input Mode Determination Unit 105, 1601 Gesture Recognition Unit 106 Gesture Processing Unit 107 Stroke Processing Unit 108, 303 Display Processing Unit 150 Pen Device 301, 1001 Get Gesture Candidate Unit 302, 1002 Storage unit 304 Gesture display processing unit 401, 402 A region for recognizing a predetermined gesture 403 A region where a stroke is input 501 GestureArea1
502 TextureArea2
503 TextureArea3
504 TextureArea4
505 TextureArea5
506 TextureArea6
601 CPU
602 ROM
603 RAM
604 Display unit 605 Input I / F
606 Communication I / F
701, 1301, 1401 Gesture pop-up display 702 to 704, 1302, 1303 Gesture trajectory 1003 Gesture history update unit 1602 Region calculation unit 1603 Gesture processing unit 1901 Area of region drawn as gesture 1902 Map drawing region

Claims (12)

An input means for inputting position coordinates from the coordinate input surface;
The setting and release of the gesture input mode for accepting a gesture indicating processing to be executed corresponding to a specific locus by a plurality of consecutive position coordinates input by the input means can be arbitrarily set on the coordinate input surface from the input means. Mode switching means for switching based on the input timing at which the position coordinates of the area are input;
Gesture processing means for executing processing corresponding to the gesture by the trajectory input by the input means when the gesture input mode is set by switching by the mode switching means;
A gesture input device characterized by comprising: The gesture input device according to claim 1, wherein the mode switching unit switches between setting and canceling the gesture input mode when a time input as the input timing is equal to or longer than a predetermined time. . The mode switching means switches the gesture input mode to cancel based on the input timing when the gesture input mode is set, and based on the input timing when the gesture input mode is cancelled. Switch gesture input mode to setting,
Stroke processing means for performing processing for displaying the shape of the trajectory input by the input means when the gesture input mode is canceled by switching by the mode switching means;
The gesture input device according to claim 1, further comprising: Area gesture correspondence storage means for storing area gesture correspondence information in which a predetermined detection area on the coordinate input surface is associated with gesture information indicating a gesture that can be input in the detection area;
Acquisition means for acquiring the gesture information associated with the detection area including the position coordinates on the coordinate input surface input from the input means, from the area gesture correspondence information stored in the area gesture correspondence storage means. When,
Gesture display means for displaying the gesture information acquired by the acquisition means;
The gesture input device according to any one of claims 1 to 3, further comprising: Gesture frequency correspondence storage means for storing gesture frequency correspondence information in which the gesture information is associated with frequency information with which the gesture indicated by the gesture information is input;
The acquisition means acquires the gesture information associated with the area gesture correspondence information from the detection area including the position coordinates on the coordinate input surface input from the input means from the area gesture correspondence information. The gesture input device according to claim 4, wherein the gesture information is acquired based on the frequency information associated with the gesture information acquired from the gesture frequency correspondence information.   The acquisition unit acquires, from the region gesture correspondence information, the gesture information associated with the region gesture correspondence information in the detection region including the position coordinates on the coordinate input surface input from the input unit. 6. The gesture information of a predetermined number is acquired in order from the highest frequency information associated with the acquired gesture information from the gesture frequency correspondence information. Gesture input device.   The gesture input device according to claim 4, wherein the gesture display unit displays the gesture information acquired by the acquisition unit in a pop-up format. When the gesture input mode is set, the gesture processing means determines whether the trajectory input by the input means and an area occupied by the trajectory are larger than a predetermined area on the coordinate input surface. The gesture input device according to any one of claims 1 to 7, wherein a process corresponding to the gesture determined by the operation is performed. The gesture processing means further includes area calculation means for calculating the area of the area occupied by the trajectory input by the input means, and when the gesture input mode is set, the gesture input means input by the input means A gesture determined by whether or not a locus and a region occupied by the locus are larger than a predetermined region on the coordinate input surface, and an area calculated by the region calculation unit and a predetermined on the coordinate input surface The gesture input device according to claim 8, wherein the processing is performed based on a ratio with the area of the region. Display means for displaying a result of processing performed by the gesture processing means;
The gesture input device according to claim 1, further comprising: An input step for inputting position coordinates from the coordinate input surface;
The setting and canceling of the gesture input mode for accepting a gesture indicating processing to be executed corresponding to a specific trajectory by a plurality of continuous position coordinates input in the input step can be arbitrarily set on the coordinate input surface from the input step. A mode switching step of switching based on the input timing at which the position coordinates of the area are input;
A gesture processing step for executing processing corresponding to the gesture by the trajectory input by the input step when the gesture input mode is set by switching by the mode switching step;
A gesture input method characterized by comprising: An input procedure for inputting position coordinates from the coordinate input surface,
The setting and release of the gesture input mode for accepting a gesture indicating processing to be executed corresponding to a specific trajectory by a plurality of consecutive position coordinates input by the input procedure can be arbitrarily set on the coordinate input surface from the input procedure. Mode switching procedure for switching based on the input timing at which the position coordinates of the area of
A gesture processing procedure for executing processing corresponding to the gesture by the trajectory input by the input procedure when the gesture input mode is set by switching by the mode switching procedure;
A gesture input program that causes a computer to execute.

Images