JP2018073202A - Information processing device, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a scheme of enabling selection of a phrase (words/sentence) with various gestures on the extension of a traffic line at the time of making an input with a pen.SOLUTION: The information processing device includes a touch panel, handwriting recognition means for recognizing handwritten characters, and a predictive conversion database for storing a handwriting (gesture) not recognized as a character by the handwriting recognition means and storing a character string determined based on an input character, the information processing apparatus having an application program running thereon, associates a predicted candidate character string following a character recognized by the handwriting recognition means and extracted from the predictive conversion database with the stored handwriting (gesture). The associated handwriting and the candidate character string are displayed, and when the associated handwriting is recognized by input from the user, the associated candidate character string is output to the application program.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus having a handwriting input support means on a touch panel, an information processing method, and a program thereof.

  There is a technique for predicting input of handwritten characters in order to smoothly input text on a smart device having a touch panel.

  In this technique, when characters are input, predicted conversion candidates (sentences and words) are displayed in a list in order of use frequency, and the user clicks and selects a desired character string from the candidates.

  Patent Document 1 is a method for improving the operability when selecting a word from a prediction candidate, and predicts the next handwriting based on the handwriting data of the input character and displays the prediction result. .

JP 2014-67147 A

  In patent document 1, it aims at the improvement of operativity by cutting out from a prediction candidate character string per character and making it selectable by one click.

  However, in the selection operation of Patent Document 1, it is necessary to move the pen once away from the handwriting input area and move it to the candidate selection area when predictive conversion candidates are selected, and wasteful movement increases against the flow line of the pen. Since the selection operation cannot be performed by a continuous operation from handwritten input, the input is troublesome. Further, it is necessary to release the pen once to make a selection, and a selection mistake is likely to occur.

  In other words, in handwritten input using a conventional smart device or the like, an operation for selecting a word from a predictive conversion candidate is complicated.

  An object of the present invention is to provide a mechanism capable of confirming selection of a word (word / sentence) by various gestures on an extension of a flow line when inputting with a pen.

The present invention is an information processing apparatus in which an application program having a touch panel, handwriting recognition means for recognizing handwritten characters, and a predictive conversion database for storing a character string determined by input characters operates, wherein the handwriting recognition Predictive conversion candidate extraction means for extracting a predicted candidate character string following the character recognized by the means from a prediction conversion database; gesture storage means for storing a handwriting (gesture) that is not recognized as a character by the handwriting recognition means; Gesture candidate character string associating means for associating the candidate character string extracted by the predictive conversion candidate extracting means with the handwriting (gesture) stored in the gesture storage means; and the gesture candidate character string associating means A gesture candidate character string display means for displaying the attached handwriting and the candidate character string; When the handwriting associated with the gesture candidate character string associating means is recognized by the handwriting recognition means, the character string for outputting the candidate character string associated with the gesture candidate character string associating means to the application program Output means.

  According to the present invention, it is possible to provide a mechanism capable of confirming selection of a word (word / sentence) by various gestures on an extension of a flow line when inputting with a pen.

It is an external view which shows an example of the external appearance of the information processing apparatus which concerns on embodiment of this invention. 2 is a block diagram illustrating an example of a hardware configuration of an information processing apparatus 100. FIG. 3 is a functional configuration diagram illustrating an example of a functional configuration of the information processing apparatus 100. FIG. It is a 1st flowchart which shows an example of the control processing means in the information processing apparatus 100 of embodiment of this invention. It is a 2nd flowchart which shows an example of the control processing means in the information processing apparatus 100 of embodiment of this invention. It is a 3rd flowchart which shows an example of the control processing means in the information processing apparatus 100 of embodiment of this invention. It is a 4th flowchart which shows an example of the control processing means in the information processing apparatus 100 of embodiment of this invention. It is an external view which shows an example of the screen image displayed with the information processing apparatus 100 which concerns on embodiment of this invention. It is an external view which shows an example of the screen image displayed with the information processing apparatus 100 which concerns on embodiment of this invention. It is an external view which shows an example of the screen image displayed with the information processing apparatus 100 which concerns on embodiment of this invention. It is an external view which shows an example of the screen image displayed with the information processing apparatus 100 which concerns on embodiment of this invention. It is a schematic diagram which shows an example of the image for demonstrating operation | movement with the information processing apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows an example of the image for demonstrating operation | movement with the information processing apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows an example of the data in the information processing apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows an example of the database in the information processing apparatus which concerns on the memory | storage device of this invention. It is a schematic diagram which shows an example of the database in the information processing apparatus which concerns on the memory | storage device of this invention.

  Hereinafter, embodiments of an information processing apparatus according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is an external view showing an example of the external appearance of an information processing apparatus according to the present invention. The information processing apparatus 100 of the present invention includes a display unit 206 having a touch panel function, such as a personal digital assistant (PDA) or a smartphone. The display unit 206 is an electronic component that combines a display control unit such as a liquid crystal panel and a position input control unit such as a touch pad, and is also referred to as a touch screen.

  The touch panel of the display unit 206 is a capacitive touch screen that detects a position by detecting a change in capacitance between a fingertip or a stylus pen and a conductive film.

  An application program (hereinafter simply referred to as an application) executed on the information processing apparatus is activated by double-clicking (touching twice) an application icon displayed on the screen using the touch panel function of the touch screen 206. Is executed.

  An area in the display unit (hereinafter referred to as a touch screen) 206 may be divided into a plurality of parts depending on the application, and is displayed divided into a text display unit 1 and a character input unit 11. FIG. 1 is an example in which a character such as the input character 12 is input by the stylus pen 10. Reference numeral 15 denotes an input instruction unit, which will be described in detail later.

  The information processing apparatus 100 may have other communication functions and the like, but will not be described because it is not directly related to the present invention.

  FIG. 2 is a diagram illustrating a hardware configuration of the information processing apparatus 100.

  As shown in FIG. 2, in the information processing apparatus 100, a central processing unit (CPU) 201, a random access memory (RAM) 203, a read only memory (ROM) 202, a storage device 204, and an input / output interface via a system bus 207. 205 and a touch screen 206 are connected.

  The CPU 201 comprehensively controls each device and controller connected to the system bus 207.

  The ROM 202 or the storage device 204 is a control program executed by the CPU 201 such as a basic input / output system (BIOS) or an operating system (OS), a computer-readable program for realizing this information processing method, and various necessary programs. Holds data (including data table).

  The RAM 203 functions as a main memory, work area, and the like for the CPU 201. The CPU 201 implements various operations by loading a program or the like necessary for executing the processing from the ROM 202 or the storage device 204 to the RAM 203 and executing the loaded program.

  The input / output interface 205 controls input from the touch screen 206. The touch screen 206 is a touch panel that can detect a position touched by a plurality of fingers, and a known technique can be used as described above. Various instructions can be performed when the user presses (touches with a finger or the like) in accordance with an icon, a cursor, or a button displayed on the touch screen.

  The storage device 204 stores dictionary information and input history information predicted from input characters, which will be described later, information for predicting and confirming characters input from handwriting entered with a stylus pen, and the like. ing.

  The storage device 204 is a medium for permanently storing information, and the form is not limited to a storage device such as a hard disk. For example, a medium such as SSD (Solid State Drive) may be used.

  FIG. 3 is a functional configuration diagram illustrating an example of a functional configuration included in the information processing apparatus 100 of the present invention.

  Reference numeral 300 denotes a touch screen control unit, which includes the touch screen 206 and the input / output interface 205 shown in FIG.

  Reference numeral 303 denotes a handwritten character recognition unit that recognizes an input handwriting input with a stylus pen or the like, and searches for a handwriting that matches from the handwriting dictionary 301 and picks up a corresponding character candidate.

  When the handwritten character recognizing unit 303 can identify the character that the user is trying to input, the process proceeds to the predictive conversion processing unit 305 to pick up the predicted conversion character or sentence of the character specified from the predictive conversion dictionary 302. .

  Further, the handwriting optimization gesture extraction unit 304 can easily remove the handwriting entered by the handwritten character recognition unit 303 from the character input unit 11 (for example, the handwriting easily moved to the input instruction unit 15 or the text display unit 1). ) Is extracted from the gesture candidate database.

  Character or sentence candidates picked up by the predictive conversion processing unit 305 are assigned to the gestures extracted by the handwriting optimization gesture extracting unit 304, listed in the predictive conversion candidate list creating unit 306, and a gesture = predictive conversion candidate list is displayed. To do.

  The gesture recognition unit 307 extracts a prediction conversion candidate corresponding to the gesture having the closest shape from the displayed gesture = prediction conversion candidate list, and transmits the extracted prediction conversion candidate to the application 310.

  With the above processing, a mechanism is provided that can select and confirm word / phrase (word / sentence) candidates by gestures on the extension of the flow line when inputting characters with a pen.

  Next, with reference to FIG. 4, the main process of the gesture input reception of the present invention will be described.

  FIG. 4 is a first flowchart illustrating an example of a control processing unit in the information processing apparatus 100 according to the embodiment of the present invention, and illustrates steps S1 to S30 in the drawing. The processing of each step is realized by the CPU 201 of each system loading and executing the application program stored in the external memory 211 of each system on the RAM 203.

  The flowchart of FIG. 4 is a flowchart started when the information processing apparatus 100 starts up an application program having a handwritten character input screen.

  First, in step S <b> 1 of FIG. 4, the CPU 201 of the information processing apparatus 100 accepts character input using the stylus pen 10 as shown in FIG. 1. An example of accepted characters is 12.

  Next, in step S <b> 2, the CPU 201 of the information processing apparatus 100 determines whether the handwriting input by the stylus pen 10 or the like is similar to the search target stroke in the handwriting dictionary 301, and extracts similar character candidates. To do. In addition, since the specification of the character by handwriting is a known technique, detailed description is abbreviate | omitted.

  Next, in step S3, the CPU 201 of the information processing apparatus 100 determines whether there is a character candidate corresponding to the handwriting detected in step S2. If there is a character candidate, the process proceeds to step S4. If there is no character candidate, the process returns to step S1 to continuously accept character input.

  When the process transitions to step S4, the CPU 201 of the information processing apparatus 100 refers to the character candidate extracted in step S2, and extracts a converted sentence candidate from the predictive conversion dictionary 302. An example in which data as shown in FIG. 16 is stored in the predictive conversion dictionary is described. That is, in this example, conversion candidates 1601 stored in a dictionary in advance, in which candidate classification 1602 is a word, and conversion candidates 1601, in which candidate classification 1602 stored from an input history or the like is a sentence candidate are stored. Needless to say, the example of the predictive conversion dictionary 302 may be in any format regardless of the format of FIG.

  In the example of FIG. 16, when the character candidate is “A”, the word candidates are “rain”, “Asahi”, “Asahikawa”, “Ashikaga”, “Ansei”, etc. If it rains tomorrow, would you like to go to a movie? "," What is the weather tomorrow? "," Blue sky "," Blue sea ".

  Next, in step S <b> 5, the CPU 201 of the information processing apparatus 100 performs processing for displaying the character candidates extracted in step S <b> 4 and the gesture candidates displayed for each character candidate at a position that is easy for the user to see. Details of the processing in step S5 will be described with reference to FIG.

  FIG. 5 is a second flowchart illustrating an example of the control processing unit in the information processing apparatus 100 according to the embodiment of the present invention, and illustrates steps S501 to S504 in the drawing. The processing of each step is realized by the CPU 201 of each system loading and executing the application program stored in the external memory 211 of each system on the RAM 203.

  The flowchart in FIG. 5 is a flowchart started when the information processing apparatus 100 transitions to the process in step S5 in FIG. 4 or step S14 in FIG.

  First, in step S501 of FIG. 5, the CPU 201 of the information processing apparatus 100 creates a list of gesture candidates to be input by the user. Details of the processing in step S501 will be described with reference to FIG.

  FIG. 6 is a third flowchart showing an example of the control processing means in the information processing apparatus 100 according to the embodiment of the present invention, and shows steps S601 to S603 in the figure. The processing of each step is realized by the CPU 201 of each system loading and executing the application program stored in the external memory 211 of each system on the RAM 203.

  The flowchart in FIG. 6 is a flowchart started when the information processing apparatus 100 transitions to the process in step S501 in FIG.

  In step S601 of FIG. 6, when the CPU 201 of the information processing apparatus 100 has recognized the character candidate from the transition between the current handwriting position of the character input area divided section obtained by dividing the character input area and the section where the previous handwriting was present. For the section at, it is specified from which “side” the handwriting entered the section. A specific example will be described later.

  Next, in step S602, the CPU 201 of the information processing apparatus 100 extracts gesture candidates to be input by the user from the position of the side where the character input area has entered. A specific example will be described later.

  Next, in step S603, the CPU 201 of the information processing apparatus 100 gives priority to the extracted gesture candidates, and ends the process of the flowchart of FIG.

  The flow of the flowchart of FIG. 6 will be specifically described with reference to the examples of FIGS. 12 and 13 and the gesture determination database of FIG.

  12 and 13 are schematic views showing an example of displaying the next gesture candidate when inputting characters according to the present invention.

  The handwriting input area 31 is divided into nine sections like 32 in the example of FIG. 12, and the handwriting to be input “a” is the current handwriting position 35. When the handwriting position is 35, it is in the E (middle) section 34 in the area divided into 9 sections, and the handwriting before coming to the E section is from the D section (left middle) 33, that is, the E section. Entering from the left side of. If the entry is from the left side, the gesture 1505 whose group key 1502 is “left side entry_E” is a gesture candidate based on the gesture determination database of FIG. Note that the priority of the gesture candidates is the order specified in ascending order 1506. In the example of FIG. 15, the top five gestures 1505 are gesture candidates.

  Similarly, in the example of FIG. 13, the handwriting to be input as “A” is the current handwriting position 40, and when the handwriting position is 40, E (middle) of the nine divided areas. At 39, the handwriting before coming to the E section enters from the B section (upper middle) 38, that is, from the upper side of the E section. If the entry is from the upper side, the gesture 1505 whose group key 1502 is “upper side entry_E” is a gesture candidate based on the gesture determination database of FIG. In the example of FIG. 15, the lower five gestures 1505 are gesture candidates.

  Through the above processing, gesture candidates to be input by the user from the handwriting of the character are picked up with priority. Returning to the flowchart of FIG.

  When a list of gesture candidates to be input by the user is created in step S501 in FIG. 5, in step S502, the CPU 201 of the information processing apparatus 100 uses the sentence conversion candidates extracted from the predictive conversion dictionary 302 in step S4. Create a list with priorities from the history. With reference to the example of FIG. 14 and FIG. 16, an example of extracting text conversion candidates extracted from the predictive conversion dictionary will be described.

  62 to 65 in 61 of FIG. 14 are examples of extracted sentences, and 63 to 65 are candidate classifications 1602 of the predictive conversion dictionary 302 as conversion candidates of sentence candidates. Reference numeral 62 is a list of conversion candidates whose candidate classification 1602 is a word candidate. The word candidates listed together are further subjected to processing for causing the user to select conversion candidates later. Note that the word candidate list may be each conversion candidate in the same manner as the sentence candidates.

  Next, in step S <b> 503, the CPU 201 of the information processing apparatus 100 associates the gesture candidate and the sentence conversion candidate in consideration of the priority, and stores them as a prediction conversion candidate list with selection gesture. An example of the prediction conversion candidate list with selection gesture will be described with reference to the lower diagram of FIG.

  The lower diagram of FIG. 12 is an example of the prediction conversion candidate list with selection gesture. The correspondence is made in consideration of the order of the gesture candidates and the priority of the conversion candidates. Returning to the flowchart of FIG.

  Next, in step S504, the CPU 201 of the information processing apparatus 100 performs a process of displaying the generated list like the screen image of FIG. The three lines displayed in the display column 14 of FIG. 8 are newly displayed screen images. The processing of the flowchart of FIG. 5 is completed by the above processing, and the processing is returned to the flowchart of FIG. 4 or the flowchart of FIG.

  Returning to the flowchart of FIG. 4, in the next step S <b> 6, the CPU 201 of the information processing apparatus 100 determines whether or not the handwriting such as the stylus pen is out of the text box frame (character input unit). This determination is made when the user operates the stylus pen in the text box frame (character input part) and recognizes that the character is being input, and when the stylus pen moves out of the text box frame (character input part) It becomes processing to judge. Details will be described with reference to FIG.

  The touch screen of FIG. 1 is divided into a text display unit 1, a character input unit 11, and an input instruction unit 15. While the stylus pen 10 is present in the character input unit 11, it is determined that the character is being input, and when the stylus pen 10 moves to the text display unit 1 or the input instruction unit 15, the user's instruction is given. By determining that the gesture has been changed, it is possible to determine whether the input is a gesture or a character is being input. Returning to the flowchart of FIG.

  In step S6, when a handwriting such as a stylus pen is not removed from the text box frame (character input unit), the process proceeds to step S1 in order to continue the handwritten character. On the other hand, when a handwriting such as a stylus pen is out of the text box frame (character input unit), it is determined as a gesture operation (that is, a character candidate selection operation), and the process proceeds to step S7.

  In step S <b> 7, the CPU 201 of the information processing apparatus 100 recognizes the gesture operation. Step S7 may be an operation of waiting for an operation to end the gesture operation.

  Next, in step S <b> 8, the CPU 201 of the information processing apparatus 100 detects the feature amount of the handwriting of the operation that is outside the input field. This detection may be performed to the extent that it is possible to specify whether it is similar to the operation that has been raised as a gesture candidate in FIG.

  In step S9, the CPU 201 of the information processing apparatus 100 determines whether or not the feature amount of the gesture operation detected in step S8 is similar to the gesture candidate picked up in step S5. If the feature amount of the gesture operation detected in step S8 has a shape similar to the gesture candidate picked up in step S5, the process proceeds to step S10 and is similar to the picked-up gesture candidate. If there is no shape, the handwriting of the gesture part is deleted (step S20), the process proceeds to step S1, and the character input process is continued. That is, the handwriting after the character input part is removed is deleted, and the process returns to the character input process. If the feature amount of the gesture operation detected in step S8 is not similar to the gesture candidate picked up in step S5, the process proceeds to step S7, and the gesture recognition may continue. . A specific determination example will be described with reference to FIG.

  FIG. 9 is a screen image when an operation of the stylus pen 10 such as 16 is received by the user. The gesture 16 in FIG. 9 is an operation of drawing a polygonal line after extending to the lower right, and is similar to the operation of 22 of the gesture candidates. Therefore, when a gesture operation as shown in FIG. 9 is accepted, the processing flow in FIG. 4 transitions to step S10. Returning to the flowchart of FIG.

  Next, in step S10, the CPU 201 of the information processing apparatus 100 determines whether the accepted gesture operation is an operation for designating a word candidate BOX (that is, 23 in FIG. 8 or 41 in FIG. 12). In the case of a gesture operation for designating a word candidate BOX, the process proceeds to step 11, and in the case of a gesture operation for designating a word candidate BOX, the process proceeds to step S30.

  When the process transitions to step S <b> 30, the CPU 201 of the information processing apparatus 100 determines a sentence with a sentence candidate corresponding to a gesture candidate whose gesture operation is similar (matches), and outputs the sentence to the application program 310. An example output to the application program 310 will be described with reference to FIGS.

  In FIG. 9, when the gesture operation 16 is performed and there is a gesture candidate 22 that matches the gesture, the sentence 13 corresponding to the gesture candidate 22 is output to the application program. The output result is shown in FIG. The 17th column in FIG. 10 is a sentence input by a gesture.

  On the other hand, when the gesture operation received in step S10 receives an operation (23 in FIG. 8) for designating a word candidate BOX, the process proceeds to step S11.

  When the process transitions to step S11, the CPU 201 of the information processing apparatus 100 performs a process of selecting from word candidates by the next gesture. Details of the processing in step S11 will be described with reference to FIG.

  FIG. 7 is a fourth flowchart illustrating an example of the control processing unit in the information processing apparatus 100 according to the embodiment of this invention, and illustrates steps S13 to S19 in the figure. The processing of each step is realized by the CPU 201 of each system loading and executing the application program stored in the external memory 211 of each system on the RAM 203.

  The flowchart in FIG. 7 is a flowchart started when the information processing apparatus 100 transitions to the process in step S11 in FIG.

  In step S13 of FIG. 7, the CPU 201 of the information processing apparatus 100 picks up characters 1601 that are word candidates from the prediction conversion dictionary 302 and associates these characters with gesture candidates. FIG. 14 shows an image of the associated data. The word candidate group BOX expansion display 71 shown in the lower part of FIG. 14 is an image diagram of data in which a word candidate is associated with a corresponding gesture candidate.

  Next, in step S14, the CPU 201 of the information processing apparatus 100 displays the word candidate and the gesture candidate associated in step S13 at an easily viewable position. FIG. 11 shows the displayed image. FIG. 11 is a screen image displayed when a gesture such as 18 is accepted in the screen image as shown in FIG. Data associated with step S13 is displayed in the middle portion 19 of FIG.

  Next, in step S15, the CPU 201 of the information processing apparatus 100 receives a gesture input from the user. In this case, it is assumed that the gesture input is input from outside the input field (text box 11). If the stylus pen is operated from the input field (in the text box 11), it is determined that the character input is a normal handwriting.

  Next, in step S16, the CPU 201 of the information processing apparatus 100 detects the feature amount of the input gesture, and in step S17, determines whether there is a shape similar to the gesture candidate picked up in step S13. . If the feature amount of the gesture operation detected in step S16 is similar to the gesture candidate picked up in step S13, the process proceeds to step S19, and word candidates corresponding to the similar gesture candidate are selected. Confirm and output. On the other hand, if the feature amount of the gesture operation detected in step S16 is not similar to the gesture candidate picked up in step S13, the handwriting input as the gesture is cleared, and the process proceeds to step S15. return.

  With the above processing, there is an effect that the selection of a phrase (word / sentence) can be confirmed and output by various gestures on the extended line of the flow line when inputting with a stylus pen or the like. Further, it is possible to determine whether or not the input with the stylus pen and the instruction by the gesture are entered after being removed from the input field (text box).

  It should be noted that the configuration and contents of the various data described above are not limited to this, and it goes without saying that the various data and configurations are configured according to the application and purpose.

  The program according to the present invention is a program that allows a computer to execute the processing method of the flowcharts shown in FIGS. 4 to 7. Is remembered.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by executing the reading.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk, solid state drive, or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program is actually It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the processing and the processing is included.

  Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention.

  In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

DESCRIPTION OF SYMBOLS 1 Text display part 10 Stylus pen 11 Character input part 15 Input instruction | indication part 100 Information processing apparatus 206 Touch screen

Claims (5)

An information processing apparatus on which an application program operates, having a touch panel, handwriting recognition means for recognizing handwritten characters, and predictive conversion storage means for storing a character string determined by input characters,
A predictive conversion candidate extraction means for extracting a predicted candidate character string following the character recognized by the handwriting recognition means from a predictive conversion database;
Gesture storage means for storing a handwriting (gesture) that is not recognized as a character by the handwriting recognition means;
A gesture candidate character string linking means for linking a candidate character string extracted by the prediction conversion candidate extraction means and a handwriting (gesture) stored in the gesture storage means;
Gesture candidate character string display means for displaying the handwriting and the candidate character string linked by the gesture candidate character string linking means;
When the handwriting recognized by the gesture candidate character string linking means is recognized by the handwriting recognition means, the character string that outputs the candidate character string linked by the gesture candidate character string linking means to the application program And an output unit.   The information according to claim 1, wherein the handwriting stored in the handwriting (gesture) storage means associated with the gesture candidate character string associating means is a handwriting outside an input area for recognizing handwritten characters. Processing equipment. The predictive conversion candidate extraction means has a rank of a character string as a candidate from the predictive conversion database determined by the input characters,
A gesture extraction unit that associates the ranks extracted by the predictive conversion candidate extraction unit in order of easily deviating from the input area for recognizing the handwritten character from the position of the handwritten character recognized by the handwriting recognition unit;
The information processing apparatus according to claim 2, further comprising: A touch panel, a handwriting recognition unit that recognizes handwritten characters, a gesture storage unit that stores a handwriting (gesture) that is not recognized by the handwriting recognition unit, and a predictive conversion that stores a character string determined by an input character An information processing method in an information processing apparatus having a storage unit on which an application program operates,
A predictive conversion candidate extraction step of extracting a predicted candidate character string following the character recognized by the handwriting recognition unit from a predictive conversion database;
A gesture candidate character string associating step for associating the candidate character string extracted in the predictive conversion candidate extraction step with a handwriting (gesture) stored in the gesture storage unit;
A gesture candidate character string display step for displaying the handwriting and the candidate character string linked in the gesture candidate character string linking step;
When the handwriting associated with the gesture candidate character string associating step is recognized by the handwriting recognition unit, the character string that outputs the candidate character string associated with the gesture candidate character string associating step to the application program And an output step.   A program for causing an information processing apparatus to operate according to the information processing method according to claim 4.

Images