JP2005149178A - Gesture processing device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a gesture processing device and method capable of precisely recognizing gestures whose forms are difficult to recognize, such as those which could have characters similar in shape to them. <P>SOLUTION: The gesture processing device includes a coordinate extracting part 12 which outputs continuous coordinates inputted to a coordinate input part 11, and a gesture recognizing part 13 by which the newest stroke of a plurality of strokes outputted from the coordinate extracting part 12 is recognized as a gesture according to the positional relationship between the newest stroke and the strokes thus far and to the shape of the newest stroke. This makes it possible to discriminate between characters with similar shapes that are difficult to distinguish one from another according to only the shape of each stroke, and gestures. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a gesture process used in an information processing apparatus such as a computer, a portable information terminal, or a word processor that processes coordinate information input via a coordinate input apparatus such as a tablet.

  In an information processing device obtained from a coordinate input device such as a tablet, a user may instruct an operation on the information processing device by inputting a handwriting (gesture) having a special shape. Usually, the shape of a gesture is determined to be different from that of a character so that the character and the gesture can be distinguished. In order to operate with a gesture, for example, a gesture is entered in a handwritten circle, the direction vector of each coordinate point is obtained for this gesture, and the gesture is determined from the pattern of this direction vector. . (For example, see Patent Document 1)

  In addition, when recognizing a character string written by a tablet, a method of recognizing the entire character string to be input by writing a horizontal line as a gesture symbol after a character string that is partly omitted and written. there were. (For example, see Patent Document 2)

Japanese Patent Publication No. 5-84540 (Fig. 6) Japanese Patent Laid-Open No. 7-254050 ([0027] [0028], FIG. 3)

  However, in the input using the above-described tablet or the like, a gesture having a shape similar to a character may be used. For example, as shown in FIG. 1A, a gesture meaning a line break may be assigned (an arrow indicates an end point). In addition, as shown in FIG. 1B, a gesture that means recognition of a omitted character string may be assigned. In English, there is no problem because there is no character similar to the shape in Fig. 1 (a), but in Japanese, the shape is similar to the angle bracket "" ", so the key is a line feed gesture from the shape. In the case of Fig. 1 (b), the shape is similar to "_" (underline), "-" (hyphen), etc. Furthermore, the linefeed gesture is The shape is similar to the bracket "" ", but because it is derived from the proofreading symbol at the time of publication or printing, it is difficult to use another shape. Even if another shape is used, the gesture There is a problem that the burden on the user who gives an operation instruction is increased.

  The present invention has been made in view of the above problems, and is a gesture processing device capable of accurately determining a gesture having a shape that is difficult to determine, such as a character having a shape similar to a gesture. It is another object of the present invention to provide a gesture processing method.

  In order to solve the above-described problems, the present invention provides a coordinate extraction unit that outputs a stroke in which coordinates are continuous from input coordinates, a latest stroke and a plurality of strokes output from the coordinate extraction unit. It is characterized by comprising gesture determining means for performing gesture determination on the latest stroke based on the positional relationship with the previous stroke and the latest stroke shape.

  In addition, a coordinate extraction unit that outputs a stroke in which coordinates are continuous from input coordinates, a positional relationship between the latest stroke and the previous stroke, and a plurality of strokes output from the coordinate extraction unit, Based on the size relationship between the stroke and other strokes and the shape of the latest stroke, the gesture determination means for determining the gesture for the latest stroke, and the positional relationship between the latest stroke and the previous stroke And a gesture target selecting means for selecting an operation target corresponding to the gesture from the strokes up to that point.

  Furthermore, it comprises a stroke storage means for storing a stroke output from the coordinate extraction means, and deletes a stroke corresponding to a gesture from the stroke storage means when the gesture determination unit determines that it is a gesture. To do.

  According to the present invention, it is difficult to determine only from the shape of the stroke by adding not only the shape of the stroke to be determined by gesture but also the positional relationship or magnitude relationship with other strokes to the gesture determination condition. Characters with shapes can be distinguished from gestures and characters can be input accurately by writing.

  Details of embodiments of the present invention will be described below with reference to the drawings.

  FIG. 2 is a diagram showing the configuration of an embodiment of the present invention. The gesture processing device shown in FIG. 2 includes a coordinate input unit 11, a coordinate extraction unit 12, a gesture determination unit 13, a gesture processing unit 14, a stroke storage unit 15, and a gesture DB 16. The operation of this gesture processing device will be described with reference to the flowchart shown in FIG.

  The coordinates obtained from the coordinate input unit 11 are (X (1), Y (1)), ..., (X (a), Y (a)), (X (a + 1), Y (a + 1). )), .... (X (b), Y (b)), (X (b + 1), Y (b + 1)), ..., (X (i), Y (i)) , ... (X (N), Y (N)) are output to the coordinate extraction unit as a time series of two-dimensional coordinates.

The coordinate extraction unit 12 collects the coordinate information obtained from the coordinate input unit 11 in time as a single stroke and outputs it for each stroke (S11).
Stroke 1: (X (1), Y (1)), ..., (X (a), Y (a))
Stroke 2: (X (a + 1), Y (a + 1)), ..., (X (b), Y (b))
...
Stroke i; ..., (X (i), Y (i))
...
Stroke N: ..., (X (N), Y (N))
It becomes.
The gesture determination unit 13 includes a shape determination process 131, a position determination process 132, and a gesture determination process 133 as shown in FIG. , N-1 is used to determine and output the type of gesture (including the determination that it is not a gesture) based on the determination condition in the gesture DB 16 using information on N-1. The stroke determined as a gesture by the gesture determination unit 13 is deleted from the stroke recording unit.

 In the shape determination process 131, for example, a direction vector of each coordinate point of the stroke is obtained, and a gesture is determined from the pattern of the direction vector (S12). At this time, each gesture is represented by the direction pattern of FIG. The number of the direction pattern means the direction in FIG. 6, and “E” means the end. If it is determined that the gesture is not a gesture based on the determination condition in the gesture DB 16, shape determination processing is performed for the next stroke (S17, S18).

  When it is determined that the gesture number is 1 as shown in FIG. 7, the position determination process is performed because there is the position determination condition in FIG. If it is determined that the gesture number is 2, the position determination condition in FIG. 7 indicates “No need”, so that the gesture is determined only by the shape determination, and the gesture determination ends (S13). , S14). Usually, a gesture that does not have a similar-shaped character is “not required”.

  As an example of the process of the position determination process 132, the gesture 1 can be determined by the following method. The strokes stored in the stroke recording unit 15 so far are 1, 2,..., N−1, n (arranged in time order).

At this time, the stroke to be determined for gesture is n, and the other strokes are ... n-2, n-1. The coordinates of stroke i (i = 1,2, .., n) are
(X (i, 1), Y (i, 1)), (X (i, 2), Y (i, 2)), ..., (X (i, N (i)), Y (i , N (i)))
And N (i) is the number of coordinates of the stroke i.
Furthermore, the coordinate values where the x and y coordinates are the maximum in the coordinates from stroke 1 to n-1 are
Xmax (1 to n-1), Ymax (1 to n-1)
And At this time, the start point (X (n, 1), Y (n, 1)) and end point (X (n, N (n)), Y (n, Y (n))) of stroke n satisfy the following: Sometimes, it is determined that the condition “pass under the previous handwriting” is satisfied.

X (n, 1)> Xmax (1 to n-1) and Y (n, 1) <Ymax (1 to n-1) (conditions regarding the start point).
And X (n, N (n)) <Xmax (1 to n-1) and Y (n, Y (n)))> Ymax (1 to n-1) (conditions regarding the end point).
FIG. 8 shows this determination condition. That is, if there is a stroke n outside the coordinates where the coordinate values of the circumscribed rectangles of the strokes 1 to n-1 so far are maximum, it is determined as a gesture (S15, 16). As described above, the shape determination condition and the position determination condition in FIG. 7 are illustrated in a diagram for easy understanding, but in actuality, the determination is performed using equations.

  The gesture processing unit 14 executes a command corresponding to the determined gesture when the stroke from the gesture determination unit 13 is determined to be a gesture. When it is determined that the gesture number is 1 as shown in FIG. 7, the gesture processing unit 14 performs a character recognition process on the handwriting so far and deletes all information in the stroke storage unit. If it is determined that the gesture number is 2, a blank symbol (“”) is output.

  As described above, according to the present embodiment, it is possible to distinguish a character having a similar shape and a gesture, which are difficult to determine only from the shape of the stroke, and to execute an operation instruction using the gesture.

  Another embodiment of the present invention will be described with reference to FIGS. FIG. 9 is a diagram showing the configuration of another embodiment of the present invention. The same components as those shown in FIG. 2 described above are denoted by the same reference numerals and description thereof is omitted. The gesture processing apparatus shown in FIG. 9 includes a coordinate input unit 11, a coordinate extraction unit 12, a gesture determination unit 13, a stroke storage unit 15, and a gesture target selection unit 17.

  For example, there may be a gesture (predictive gesture) that gives an operation instruction to recognize a character string in which the stroke shown in FIG. 1B is omitted. Instead of writing the entire character string as shown in FIG. 10 (a), by inputting the character string as shown in FIG. 10 (b), the character following “con” is predicted and “computer” is entered. It is a function that converts it. However, it is difficult to distinguish from the character string “con_” only by looking at the shape and positional relationship of the handwriting. Therefore, a distinction is made using the magnitude relationship between the previous stroke and the last stroke. The size in the y direction of the circumscribed rectangle of strokes 1 to n-1 is Dy (1 to n-1), and the horizontal width of the last stroke is Dx (n).

Thereafter, the gesture is determined using the above two values (FIG. 10C).
Dy (1 to n-1) x K <Dx (n)
K is a margin for preventing erroneous determination of underline “_” and horizontal bar “−” as a prediction gesture, and is normally set to about 1.5, for example. As shown in FIG. 11, the configuration of the gesture determination unit when performing size determination is a configuration in which the position determination process 132 in FIG. 4 is replaced with a size determination process.

Further, the positional relationship may be added to the judgment condition for the prediction gesture.
For example,
Xmax (1 to n-1) <X (n, N (n)) and (Ymin (1 to n-1) + Ymax (1 to n-1)) / 2 <Y (n, 1)
The positional relationship judgment condition is added. This means that the prediction gesture (start point) is on the right side of the previous stroke and in the lower half (FIG. 10D).

  It is also possible to specify with a prediction gesture which part of the handwriting written by the gesture target selection unit 17 is to be predicted (target of prediction). In that case, it is determined that the prediction gesture is written so as to pass below the handwriting that is the target portion of the prediction as follows. In FIG. 10 (e), it means prediction of a character string following “CON” in the handwriting “TO CON”.

  The operation of the gesture target selection unit 17 at this time will be described with reference to the flowchart of FIG. According to the flowchart of FIG. 12, a prediction target is selected from the stroke numbers 1 to n-1 (S21).

First, the prediction target set Ω is emptied and selection of a prediction target is started (S22).
Set the maximum value and minimum value of stroke number i.
Xmin (i), Ymin (i), Xmax (i), Ymax (i)
And The prediction gesture is stroke number n.
Xmin (n) ≦ Xmin (i) and
Xmax (i) ≦ Xmax (n) and
Ymax (i) <Ymin (n)
The stroke number i that satisfies the condition is set as a prediction target (S23).
A stroke that matches the prediction target condition is added to the prediction target set Ω (S24), and a prediction target is selected for the next stroke (S25). When the prediction target is selected up to the previous stroke number of the prediction gesture, the process of the gesture target selection unit is ended (S26).

  As described above, in another embodiment of the present invention, a character having a similar shape that was difficult to determine only from the shape of a stroke is distinguished from a gesture, and the entire character string to be written is omitted without writing. Since the prediction gesture can be determined, it is possible to reduce the trouble of inputting characters by writing on a tablet or the like.

The figure which shows the written gesture. The block diagram which shows the structure of the gesture apparatus which concerns on one Example of this invention. The flowchart which shows operation | movement of the gesture apparatus which concerns on one Example of this invention. The block diagram which shows the detail of a gesture determination part. The figure which shows a shape determination process result. The figure which shows the reference | standard of a direction pattern. The figure which shows an example of the content of gesture DB. The figure explaining a position determination process. The block diagram which shows the structure of the gesture apparatus which concerns on the other Example of this invention. The figure explaining determination of a prediction gesture. The block diagram which shows the detail of a gesture determination part. The flowchart which shows operation | movement of a prediction object selection part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Coordinate input part 12 ... Coordinate extraction part 13 ... Gesture determination part 14 ... Gesture processing part 15 ... Stroke memory | storage part 16 ... Gesture DB
17 ... Gesture target selection unit 131 ... Shape determination process 132 ... Position determination process 133 ... Gesture determination process 134 ... Size determination process

Claims (9)

  Coordinate extraction means for outputting a stroke in which coordinates are continuous from the input coordinates, the positional relationship between the latest stroke and the previous stroke, and the latest stroke for a plurality of strokes output from the coordinate extraction means And a gesture determination means for performing gesture determination on the latest stroke based on the shape of the gesture processing device.   Coordinate extraction means for outputting strokes in which coordinates are continuous from the input coordinates, the magnitude relationship between the latest stroke and the other strokes for the plurality of strokes output from the coordinate extraction means, and the latest stroke And a gesture determination means for performing gesture determination on the latest stroke based on the shape of the gesture processing device.   Coordinate extraction means for outputting a stroke in which coordinates are continuous from the input coordinates, the positional relationship between the latest stroke and the previous stroke, and the latest stroke for a plurality of strokes output from the coordinate extraction means A gesture processing apparatus comprising gesture determining means for performing gesture determination on the latest stroke based on a magnitude relationship between the stroke and other strokes and a shape of the latest stroke.   Coordinate extraction means for outputting a stroke in which coordinates are continuous from the input coordinates, the positional relationship between the latest stroke and the previous stroke, and the latest stroke for a plurality of strokes output from the coordinate extraction means Based on the shape of the stroke, the gesture determination means for determining the gesture with respect to the latest stroke, and the target of the operation corresponding to the gesture from the previous stroke based on the positional relationship between the latest stroke and the previous stroke. A gesture processing apparatus comprising: a gesture target selecting means for selecting.   Coordinate extraction means for outputting strokes in which coordinates are continuous from the input coordinates, the magnitude relationship between the latest stroke and the other strokes for the plurality of strokes output from the coordinate extraction means, and the latest stroke Based on the shape of the gesture, the gesture determination means for determining the gesture for the latest stroke, and the position of the latest stroke and the previous stroke, the target of the operation corresponding to the gesture from the previous stroke A gesture processing apparatus comprising: a gesture target selecting means for selecting   Coordinate extraction means for outputting a stroke in which coordinates are continuous from the input coordinates, the positional relationship between the latest stroke and the previous stroke, and the latest stroke for a plurality of strokes output from the coordinate extraction means Based on the magnitude relationship between the stroke and other strokes and the shape of the latest stroke, the gesture determination means for performing gesture determination on the latest stroke, and the positional relationship between the latest stroke and the previous stroke. A gesture processing apparatus comprising gesture target selection means for selecting an operation target corresponding to a gesture from the strokes up to.   A stroke storage unit that stores a stroke output from the coordinate extraction unit is provided, and when the gesture determination unit determines that the gesture is a gesture, the stroke corresponding to the gesture is deleted from the stroke storage unit. The gesture processing device according to any one of claims 1 to 6.   A step of outputting a continuous stroke from the input coordinates, the positional relationship between the latest stroke and the previous stroke, and the shape of the latest stroke for the multiple strokes output from this step And a step of performing gesture determination on the latest stroke based on the method.   The step of outputting the stroke with continuous coordinates from the input coordinates, the positional relationship between the latest stroke and the previous stroke, and the latest stroke and other strokes for the multiple strokes output from this step Based on the magnitude relationship with the stroke and the shape of the latest stroke, the step of performing gesture judgment on the latest stroke, and the positional relationship between the latest stroke and the previous stroke, and the previous stroke And a step of selecting an operation target corresponding to the gesture.

Images