JP2012014517A - Handwritten character recognition method and system thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method of efficiently recognizing handwritten characters in a tablet type computer and a system thereof.SOLUTION: In a character recognition method for a tablet type computer, there are used a plurality of program modules, among which a program module group that executes processing of relatively high priority is configured aiming at quick user feedback, and a program module group that executes processing of low priority is configured aiming at text data generation processing executed in background and processing of accuracy improvement for character recognition. Through forming a plurality of program module groups, user experience superior to that obtained from a mere combination of conventional techniques can be offered by the handwritten character recognition method and system.

Description

  The present invention relates to a handwritten character recognition method and system. More specifically, the present invention relates to a method and system for recognizing handwritten characters input via a touch panel mounted on a tablet computer.

  In recent years, a type of mobile phone called a smartphone has become widespread. Among these smartphones, there are few switches and buttons on the case, and it is known that most operations are performed by touching a touch panel display with a finger (also referred to as “gesture”).

  In recent years, tablet computers that inherit the functions of this type of smartphone are also becoming popular. Since this tablet computer has a size several times as large as the smartphone (for example, a size of about A4 size or B5 size notebook), the above-mentioned smartphone can view video content that is restricted by the size of the screen. It is possible to provide an extended user experience (user experience) such as browsing electronic books.

  This tablet computer can download various applications (software) from the Internet via a wireless network such as a wireless LAN or a cellular network. Users can use the application according to their individual needs. Some of these applications can create a memo by a handwriting method. The user can provide the created memo data to other users, and can utilize the data in cooperation with other applications.

In applications installed in personal computers (hereinafter referred to as “PCs”), mobile phones, and PHS, conventionally, one or more of the following functions must be combined to create and edit notes: Was known.
1) Characters input in one area (area A) are reduced to another area (area B) and displayed 2) An area to be input from a plurality of specified areas is selected, and an input character string is displayed in the selected area 3 ) Handwritten character recognition 4) Improvement of handwritten character recognition accuracy using probability related to dictionary and character continuity 5) Automatic recognition of specific words in sentences and attachment of annotations 6) Gestures with touch screen / touchpad Operation of electronic equipment used 7) Text search 8) Service via network such as e-mail, SNS, online information storage / sharing 9) Predictive conversion at the time of text input using input history 10) By voice recognition 2. Input of Words and Sentences In addition to the above-described conventional techniques, many improvements have been made for handwritten character recognition (see Patent Documents 1 and 2).

JP 2003-308484 A JP-A-6-332610

  However, when the conventional techniques 1) to 6) are simply combined and implemented on a tablet computer, there is a problem that a response to a user input becomes slow and a user experience deteriorates.

  In addition, since kanji conversion is not performed in handwritten character recognition on a tablet computer, the above-mentioned 9) predictive conversion and 10) speech recognition, which have greatly improved the efficiency at the time of inputting words and sentences especially in mobile phones and PHS, etc. There was a problem of not being able to take advantage of the function.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a more efficient handwritten character recognition method and system in a tablet computer.

  In order to solve the above problems, the present invention provides a handwritten character recognition method in a tablet computer. The method executed by the tablet computer includes: (a) generating a program module that performs processing with a relatively low priority among a plurality of program modules as a thread with a low processing priority; ) Determining whether to execute the thread; (c) receiving user input if it is determined not to execute the thread; and (d) the user input is a display on the tablet computer. And (e) when it is determined that the user input is performed on the first region, the first step is performed to determine whether the first input is performed on the first region. A drawing corresponding to the user input is performed in the area of the display, and the drawing is displayed as bitmap data in the second area of the display. (F) storing the handwriting data corresponding to the drawing in a shared list accessible to the thread's program module; and (g) until the thread is determined to be executed. Repeating steps (b) to (f); (h) determining whether or not handwriting data input by the user is stored in the shared list when it is determined to execute the thread; and (I) when the handwriting data is stored, reading the handwriting data from the shared list; and (j) estimating one or more text data corresponding to the read handwriting data. The one or more text data includes at least accuracy information; and (k) the estimated one or more To comprising the step of storing in conjunction with the handwriting data corresponding to a plurality of text data.

  The present invention also provides a handwritten character recognition system in a tablet computer. The system includes means for storing a first program module group including one or a plurality of program modules, and one or a plurality of programs for performing a process having a lower priority than the first program module group. A means for storing a second program module group including modules, and a recognition result database, wherein the first program module group receives received user input in a first area of the display of the tablet computer; When it is determined that the image is applied to the first area, drawing corresponding to the user input is performed in the first area, and the drawing is written as handwriting data which is bitmap data in the second area of the display. Display and store the handwriting data in a shared list accessible by the second program module group The second program module group reads out the handwriting data from the shared list and reads out the handwriting data when the handwriting data is stored in the shared list. A specific phrase recognition program module for estimating one or more text data corresponding to the handwriting data, wherein the text data includes at least a certain phrase recognition program module, The estimated text data is combined with the corresponding handwriting data and stored in the recognition result database.

  According to the present invention, it is possible to provide a more efficient handwritten character recognition method and system in a tablet computer.

  Also, according to the present invention, user feedback for handwritten input is performed more quickly, and a predictive conversion function and a voice recognition function that have not been conventionally used when handwritten character recognition is performed can be used.

  In addition, according to the present invention, since there is one or more text-based character string information stored together with handwriting information, a full-text search is performed on the created memo, or various types provided on the network It is also possible to cooperate by transmitting text-based character string information to the web service.

1 is a system configuration diagram of a tablet computer according to the present invention. It is a figure which shows the program module for performing the handwritten character recognition method which concerns on this invention. It is a figure which shows the external appearance of the tablet type computer which concerns on this invention. It is a figure which shows the flow of the handwritten character recognition which concerns on this invention. It is a figure which shows the flow of the handwritten character recognition which concerns on this invention.

  Hereinafter, exemplary embodiments of the present invention will be described in detail.

(System configuration)
FIG. 1 is a system configuration diagram of a tablet computer 100 in which a handwritten character recognition method according to the present invention is implemented.

  The tablet computer 100 includes a control unit 101, a main storage unit 102, an auxiliary storage unit 103, an interface (I / F) unit 104, a display unit 105, a recognition result DB 106, a dictionary DB 107, a character pattern DB 108, and an individual history DB 109. . These components 101 to 109 are connected to each other via a bus 120, and the components 102 to 109 are controlled by the control unit 101. The tablet computer 100 can receive user input via the stylus pen 160. In FIG. 1, the stylus pen 160 is indicated by a dashed arrow to indicate that it is not connected to the tablet computer 100 by wire.

  The control unit 101 is also called a central processing unit (CPU), controls the components 102 to 109 and calculates data as described above, and loads various program modules stored in the auxiliary storage unit 103. Read to the main storage unit 102 and execute. The main storage unit 102 is also called a main memory, and stores various data received by the tablet computer 100, computer-executable instructions, data after arithmetic processing by the instructions, and the like.

  The auxiliary storage unit 103 is a storage device typified by a hard disk (HDD) and is used when data and programs are stored for a long period of time. Since the main storage unit 102 has a relatively smaller storage capacity than the auxiliary storage unit 103, it is used for temporary data storage, arithmetic processing, and the like, whereas the auxiliary storage unit 103 requires necessary data and information. Used for long-term storage and storage. In other words, when the control unit 101 executes a program to perform data calculation, the necessary data and program are read from the auxiliary storage unit 103 to the main storage unit 102 and the calculation result data is stored / saved for a long time. In this case, the control unit 101 writes calculation result data in the auxiliary storage unit 103.

  The interface (I / F) unit 104 serves as an interface for transmitting and receiving data to and from the stylus pen 160 for inputting handwritten characters. Data received from the outside of the tablet computer 100 is temporarily stored in the main storage unit 102. The interface unit 104 also provides the display unit 105 with an interface (an application program such as an input screen) that accepts various commands and input data (such as various masters and tables) from the user. The display unit 105 displays data processed by the tablet computer 100 and the interface.

  The recognition result DB 106 stores handwriting information input by the user recognized by the tablet computer 100 and one or more character strings and / or voice information combined with the handwriting information.

  The dictionary DB 107 stores information on dictionaries such as national language, synonyms, new words, English-Japanese / Japanese-English, and encyclopedias used in the tablet computer 100. These dictionaries and encyclopedias may be built in the tablet computer 100 in advance, or may be downloaded and stored via a network. The dictionary DB 107 also includes information such as intonation used when recognizing voice information. In addition, various dictionaries and encyclopedias stored in the dictionary DB 107 hold, as metadata, management information that can be used by applications running on the tablet computer 100 (for example, priority usage order according to usage frequency). Information, etc.).

  The character pattern DB 108 stores character patterns (templates) used when estimating corresponding characters and phrases based on handwriting information input by the user. This character pattern may be built in the tablet computer 100 in advance, or may be downloaded and stored via a network.

  The individual history DB 109 is a database that stores handwriting information input previously, the frequency of use of the dictionary and department dictionary stored in the dictionary DB 107, and voice characteristics for each user. By using this database, for example, it is possible to memorize the characteristics of handwriting by individuals for the characters “meeting”, and also to memorize the characteristics of individual voices. It can be used to improve the accuracy of recognition.

  As the stylus pen 160, any known stylus pen can be used as long as it can draw characters in an input field displayed on the tablet computer 100.

(Function of program module)
FIG. 2 is a diagram showing a program module for executing the handwritten character recognition method according to the present invention. These program modules are executed on the main storage unit 102 by the control unit 101 in response to a command (input) from the user or a specific occurrence event.

  The tablet computer 100 includes a first module group 201 and a second module group 202. The first module group 201 includes a reduced display module 211, a selection area display module 212, and a gesture recognition module 216. The second module group 202 includes a handwritten character recognition module 213, a character recognition accuracy improvement module 214, and a specific phrase recognition module 215.

  The tablet computer 100 also includes a text search module 217, a network service module 218, a prediction conversion module 219, and a speech recognition module 220. The text search module 217 and the network service module 218 can be activated in cooperation with the second module group 202 in accordance with a user instruction.

  The text search module 217 provides a function for searching text-based character string data stored in the recognition result DB 106. The handwritten character input by the stylus pen 160 is stored in the recognition result DB 106 as handwriting information (bitmap data) that is the handwritten character itself and as one or more text-based character string data. The text search module 217 can perform a search based on the one or more character string data.

  The network service module 218 provides an electronic mail transmission / reception function, an SNS access function, and a function for downloading and storing information existing on a network such as the Internet and uploading and sharing the network. . The tablet computer 100 generates an image from handwriting information input by the stylus pen 160 and a template stored in the character pattern DB 108 based on a user operation, or generates a sentence from a series of recognized character strings. Then, the generated image / text can be transmitted to the network via the network service module 218, and services by various applications provided from the network can be used.

  The predictive conversion module 219 provides a function of predicting and presenting a conversion candidate for the character and a subsequent character string in response to the input of the character by the user. This function presents, for example, a post-conversion word or a succeeding character that is expected based on the input character when a part of the character is input when inputting the character.

  The voice recognition module 220 recognizes a voice input input via a microphone built in the tablet computer 100 as voice information that is an electrical signal. At the time of such recognition, if there is handwriting information input almost simultaneously by the stylus pen 160, the speech recognition module 220 can combine the speech information with the handwriting information. The information used for this combination process is, for example, date / time information such as sysdate. For example, when handwriting input of “meeting place” using the pen 160 (or immediately after the input), when inputting “first-floor meeting room” by voice, handwriting information “meeting place” and voice information “ The first floor conference room "is combined and stored in the main storage unit 102 (finally the recognition result DB 106).

  The voice recognition module 220 can determine the accuracy of the voice-recognized word or sentence based on the dictionary DB 107. For example, when the voice input “Kyoiku” is recognized as voice information “going today” or “education”, the recognition result is relatively high based on the context and voice intonation. Information such as accuracy can be determined. This accuracy information (degree of certainty) is used to improve the accuracy of predictive conversion.

  As described herein, when there is speech information combined with handwriting information (bitmap data), the speech recognition module 220 also performs an estimation process on the speech information, and includes one or more speech information. Character string data (text data) is acquired with accuracy information.

  The first module group 201 is a group of a plurality of program modules for processing user feedback with priority. That is, the first module group 201 is a module group in which additional functions are mounted based on the functions 1), 2), and 6) described in the “Background Art” column.

  As shown in FIG. 3, the reduced display module 211 uses a character input by the stylus pen 160 in a specific area (area A) of the display of the tablet computer 100 as another bitmap area. B) provides a function for reducing and displaying. For example, when the user uses the stylus pen 160 to input characters to the memo creation application, the character input field (area A) is an input field that is easy to write by hand and displays handwriting information. The column (area B) is a display column large enough to read a series of input text. The reduced display module 211 transmits handwriting information input by the stylus pen 160 to the second module group 202. When the speech information recognized by the speech recognition module 220 is combined with the handwriting information, the handwriting information and the speech information are transmitted while being combined. The transmitted handwriting information and / or voice information is stored in, for example, a variable length list on the main storage unit 102, but the stored information is stored as it is until a process described later is started.

  The selection area display module 212 provides a function of selecting an area to be input from a plurality of specified areas and displaying an input character in the selected area. In the example shown in FIG. 3, the user can select an input area in either area A or area B. The selection area display module 212 displays characters input in the area selected by the user. The selection area display module 212 automatically performs line feed processing when the right end of the area is reached in the selected area. The interval between line breaks depends on a predetermined template or setting information set in advance by the user. Furthermore, the selection area display module 212 automatically increases the character reduction rate until the display unit 105 becomes displayable when characters written in the area cannot be entered depending on the template or setting information. To make the string displayable.

  The gesture recognition module 216 enables an operation using a gesture (for example, touching the display unit 105 with a human finger) on the touch screen / touch pad of the tablet computer 100. For example, when a gesture for touching an icon of an arbitrary application displayed on the display unit 105 once is detected, the icon is displayed when a gesture for starting the application or dragging the icon is detected. Move it within.

  Next, the modules 213 to 215 of the second module group 202 will be described. The second module group 202 is a module group in which additional functions are mounted based on the functions 3), 4), and 5) described in the “Background Art” column. The second module group 202 performs processing when there is room in the processing of the first module group 201. The determination as to whether or not there is room in the processing of the first module group 201 is made based on a predetermined event, for example, when the user pauses input by the stylus pen 160 and rests his hand.

  The handwritten character recognition module 213 reads and estimates handwritten characters input by the stylus pen 160, that is, handwriting information transmitted from the reduced display module 211 and stored therein, for example, from a variable length list. The handwritten character recognition module 213 estimates one character at a time from the read handwritten information based on input timing information accompanying stroke information, stroke start position information, and the like. If a single character cannot be determined here and a plurality of candidates are estimated, the plurality of candidates are recognized and stored with certainty. Such estimation can be performed by collating with a character pattern stored in the character pattern DB 108.

  The character recognition accuracy improvement module 214 provides a function of improving the accuracy of handwritten character recognition by using probability information regarding continuity of a dictionary and characters. The dictionary is a dictionary included in the dictionary DB 107, and the continuity of characters indicates information as to whether or not it is established as a sentence. In addition, the character recognition accuracy improvement module 214 improves the accuracy of handwritten character recognition using information for each user stored in the individual history DB 109.

  The specific word / phrase recognition module 215 provides a function of recognizing a specific word / phrase in a sentence and attaching an annotation (annotation) thereto in cooperation with the handwritten character recognition module 213 and the character recognition accuracy improving module 214. That is, the specific word recognition module 215 comprehensively determines one or more character delimiter candidates, one or more character recognition candidates, probability information regarding continuity of the dictionary and characters, and the result of speech recognition. Character candidates corresponding to one or more character breaks are calculated with certainty. Character candidates corresponding to the calculated one or more character breaks are stored in the recognition result DB 106 without being always visible on the display of the tablet computer 100, and these stored data are stored by the user. Used in search operations. This search operation is performed in descending order of accuracy when there are a plurality of character delimiters and corresponding character candidates.

  The specific word recognition module 215 can also display character candidates corresponding to the calculated one or more character breaks on the display unit 105 in descending order of accuracy, and obtain character recognition confirmation information from the user. Further, the specific word / phrase recognition module 215 can receive an indication / correction from the user regarding an error in character recognition. Confirmation information and indication / correction information from the user can be obtained by presenting the application and selecting the correct candidate by the user, or by carefully writing the misrecognized character again. Can do.

  The sentence search module 217 performs a search on the calculated character recognition result based on the text. In one embodiment, this search is not performed only on character string data that has been finalized by the user, but can be performed in parallel on a combination of characters having a predetermined accuracy or higher.

(Processing flow)
FIG. 4 is a diagram showing a flow of handwritten character recognition according to the present invention using the above modules. In the following description, as shown in FIG. 3, an area close to the user's hand in the display of the tablet computer 100 is defined as an area A, and the remaining area is defined as an area B. Characters input to the area A are reduced and displayed in the area B as handwriting information.

  In step S401, the tablet computer 100 generates the process of the second module group 202 as a low priority thread. Here, “low priority” is because the processing of the second module group 202 is performed when there is room in the processing of the first module group 201 as described above. A “thread” is an execution unit of software and can include one or a plurality of program modules.

  In step S402, the tablet computer 100 determines whether the process of the second module group 202 can be started. Specifically, it is determined whether or not there is a user input for a predetermined period such as when there is no user input to the tablet computer 100 for a predetermined time. During that period, if there is a user input, the process proceeds to step S403, and if there is no user input, the process proceeds to the process flow of the second module group 202 shown in FIG.

  In step S403, the tablet computer 100 receives a user input. Here, the user input includes gestures, input with the stylus pen 160, voice input, input to the tablet computer 100 main unit, and the like.

  In step S404, the tablet computer 100 determines whether the user input corresponds to a gesture. As described above, the gesture refers to a contact act with a user's finger or the like performed on the display of the tablet computer 100. In step S405, the tablet computer 100 performs processing corresponding to the gesture. Specifically, the tablet computer 100 operates a corresponding application. When this process ends, the tablet computer 100 returns the process to step S402.

  In step S <b> 406, the tablet computer 100 determines whether the user input corresponds to an input to the area A via the stylus pen 160.

  In step S407, the tablet computer 100 performs drawing corresponding to the input by the stylus pen 160 in the area A, and uses the reduced display module 211, and the bit corresponding to the drawing for the area B at a predetermined timing. Map data is reduced and displayed in area B. This reduced display is newly added to the bitmap data already displayed in the area B in consideration of the size of the bitmap data already displayed in the area B and the newly added bitmap data. This is done by reducing the bitmap data. In addition, when the character string input to the area A reaches the right end of the screen, the tablet computer 100 automatically performs line feed processing using the selection area display module 212 according to the template and setting information being used.

  In step S408, the tablet computer 100 transmits the characters input in the area A as handwriting information to the second module group 202 via the reduced display module 211, and shares a list with the area B (for example, a variable length list). ). When this process ends, the tablet computer 100 returns the process to step S402.

  In step S409, the tablet computer 100 determines whether the user input is a voice input. The voice input is converted into an electric signal by the microphone of the tablet computer 100. In step S410, the tablet computer 100 uses the speech recognition module 220 to combine the handwriting information input when speech input is performed and the speech information, and then saves them in a shared list with the region B. To do. When this process ends, the tablet computer 100 returns the process to step S402.

  In step S <b> 411, the tablet computer 100 determines whether the user input is an input from the device body or an input operation other than the area A via the stylus pen 160. In step S412, the tablet computer 100 performs processing corresponding to the input (for example, displaying an initial display screen). When this process ends, the tablet computer 100 returns the process to step S402.

  FIG. 5 is a diagram showing a process related to the second module group 202 in the handwritten character recognition flow according to the present invention.

  In step S501, the tablet computer 100 determines whether data is stored in the shared list of the area A and the area B. As described above, handwriting information is transmitted to the shared list by the reduced display module 211 and stored. If unprocessed handwriting information is stored in this shared list, the process proceeds to step S502. On the other hand, if unprocessed handwriting information is not stored, the process flow of FIG. 5 ends. Whether the data is unprocessed or processed can be determined based on the value of a flag attached to the handwriting information data or the value of predetermined metadata.

  In step S <b> 502, the tablet computer 100 reads handwriting information and voice information from the shared list using the handwritten character recognition module 213 and stores them in the main storage unit 102.

  In step S503, the handwritten character recognition module 213 estimates one character at a time from the read handwriting information based on input timing information and stroke start position information stored in the header portion of the handwriting information. If a single character cannot be determined and multiple candidates are estimated, the multiple candidates are estimated with certainty. Such estimation can be performed by collating with a character pattern stored in the character pattern DB 108. The tablet computer 100 can also estimate the read character string and the subsequent character string using the predictive conversion module 219. The handwritten character recognition module 213 can improve the accuracy of the estimation by performing processing in cooperation with the character recognition accuracy improving module 214 and the specific phrase recognition module 215 as described above.

  When there is speech information associated with the handwriting information, the speech recognition module 220 performs an estimation process on the speech information and acquires one or more estimation results (text data) with accuracy. Predictive conversion can be performed by using accuracy information during speech recognition.

  In step S504, as an arbitrary process, the tablet computer 100 uses the specific word / phrase recognition module 215 to display on the display unit 105 if the estimated accuracy of one or more characters or phrases is equal to or higher than a predetermined accuracy. indicate. One or more characters and phrases displayed on the display unit 105 are subjected to a confirmation process and an error correction process by the user. For example, the tablet computer 100 causes the display unit 105 to display a screen for allowing the user to perform a confirmation process for one or more characters or phrases, or input for correction to cause the user to correct an error. Fields and edit screens can be displayed. In step S505, the tablet computer 100 performs character recognition according to a determination process or an error correction process performed by the user as an arbitrary process.

  In step S506, the tablet computer 100 combines one or a plurality of character string data (text data), which is a recognition result, with handwriting information (bitmap data) and stores it in the recognition result DB 106. When the storage is completed, the process returns to step S501.

(Other embodiments)
The following functions that can be added to the above embodiment can also be added to the program module or database of the tablet computer 100.

  As shown in FIG. 1, the tablet computer 100 has a personal history DB 109. When estimating one character at a time from the handwriting information in the above-described step S503, higher accuracy can be estimated by collating with the handwriting information of the user stored in the individual history DB 109. The stored handwriting information is associated with character string data that is a previous character recognition result. When performing collation while referring to the individual history DB 109, the tablet computer 100 uses the handwriting information in the individual history DB 109 as a reference while expanding or reducing the handwriting information input by the user as necessary. Alternatively, collation is performed while shifting little by little.

  The voice recognition module 220 can also recognize the voice of the user of the tablet computer 100 by recognizing the voice of the user of the tablet computer 100 based on the user-specific voice characteristics stored in the individual history DB 109. For example, assuming that a plurality of people speak in a meeting or the like, in a normal mode, the microphone built in the tablet computer 100 performs voice recognition on a plurality of people's voice input. If the function is used, it is possible to perform voice recognition by selecting only the voice input of the user of the tablet computer 100.

  The tablet computer 100 can also determine priorities of various dictionaries stored in the dictionary DB 107 based on input characters and phrases, or words that have been voice-recognized. For example, when a dictionary that stores characters and phrases used in the field of computer science is often used, the tablet computer 100 may change the order of referring to the dictionary to a higher priority. it can. In this way, when a specific dictionary is frequently used, it is possible to improve the accuracy of estimation of characters and phrases by changing the priority order of the dictionary stored in the dictionary DB 107. As described above, a dictionary in a new field can be downloaded from the network and added to the dictionary DB 107.

(Summary)
The methods, functions, and means described in the above-described embodiments can be implemented as a computer-executable program, or can be implemented as a computer-readable recording medium that stores the program.

  The embodiment described above is merely an example, and the present invention is not limited to the above embodiment, and various modifications are possible. That is, as long as it is based on the technical idea described in the scope of claims, the modifications are included in the technical scope of the present invention.

100 tablet computer 106 recognition result DB
107 Dictionary DB
108 Character pattern DB
109 Personal history DB
160 Stylus Pen 201 First Module Group 202 Second Module Group 211 Reduced Display Module 212 Selected Area Display Module 213 Handwritten Character Recognition Module 214 Character Recognition Accuracy Improvement Module 215 Specific Word Recognition Module 216 Gesture Recognition Module 217 Sentence Search Module 218 Network Service Module 219 Predictive conversion module 220 Speech recognition module

Claims (10)

A method for recognizing handwritten characters in a tablet computer,
The method performed by the tablet computer includes:
(A) generating a program module that performs processing with a relatively low priority among a plurality of program modules as a thread with a low processing priority;
(B) determining whether to execute the thread;
(C) receiving a user input if it is determined not to execute the thread;
(D) determining whether the user input is made to a first area of the display of the tablet computer;
(E) When it is determined that the user input is performed on the first area, drawing corresponding to the user input is performed on the first area, and the drawing is performed on the second area of the display. Displaying as bitmap data in the area;
(F) storing handwriting data corresponding to the drawing in a shared list accessible by the program module of the thread;
(G) repeating the steps (b) to (f) until it is determined that the thread is to be executed;
(H) a step of determining whether or not handwriting data input by a user is stored in the shared list when it is determined to execute the thread;
(I) if the handwriting data is stored, reading the handwriting data from the shared list;
(J) estimating one or more text data corresponding to the read handwriting data, wherein the one or more text data includes at least accuracy information;
(K) combining and storing the estimated one or more text data with corresponding handwriting data. The estimating step (j) includes:
(J1) further including the step of estimating the corresponding one or more text data based on input timing information and stroke start position information included in the read handwriting data,
The method according to claim 1, wherein the estimation is performed by collating with a character pattern stored in the tablet computer, and the accuracy information is determined according to the collation result. . The estimating step (j1) includes:
3. The method of claim 2, further comprising predicting text data candidates that follow the estimated one or more corresponding text data.   The method according to claim 2, wherein the estimation is performed by comparing the read handwriting data with previously inputted handwriting data for each user. Presenting the estimated one or more text data having information of the accuracy exceeding a predetermined threshold to a user;
The method further comprises: determining one text data of the estimated one or more text data according to a user input in response to presenting to the user. The method described in 1. Confirming according to the user input comprises:
Displaying a first user input screen for allowing a user to select one text data of the one or more text data as correct data;
6. The method of claim 5, further comprising: displaying a second user input screen for correcting one of the one or more text data to correct data.   2. The bitmap data corresponding to the drawing displayed in the second area is reduced and displayed according to the size of the second area of the display of the tablet computer. The method described in 1. The storing step (f) includes audio data that is a recognition result of the audio input when there is an audio input that is coupled to a drawing corresponding to the user input and that is input via a microphone of the tablet computer. And combining the handwriting data and storing in the shared list,
The determining step (h) further includes a step of determining whether or not the handwriting data and the voice data combined with the handwriting data are stored in the shared list,
The step of reading (i) further includes a step of reading the handwriting data and the voice data from the shared list when the handwriting data and the voice data are stored,
The estimating step (j) further includes estimating one or more text data corresponding to the read handwriting data and the speech data, the text data including at least accuracy information,
The method according to claim 1 or 5, characterized in that: The estimating step (j) includes:
(J2) Estimating one or more text data corresponding to the voice data based on at least the intonation information of the read voice data, the intonation information being stored in the tablet computer Step, and
(J3) Based on the corresponding one or more text data estimated based on the input timing information and the stroke start position information included in the read handwriting data, The method of claim 8, comprising at least one of estimating a plurality of text data. A handwritten character recognition system for a tablet computer,
Means for storing a first group of program modules including one or more program modules;
Means for storing a second program module group including one or more program modules that perform processing with a lower priority than the first program module group;
A recognition result database and
The first program module group includes:
When it is determined that the received user input is performed on the first area of the display of the tablet computer, drawing corresponding to the user input is performed on the first area, and the drawing is performed. At least a reduced display program module for storing the handwriting data in a shared list accessible by the second program module group in the second area of the display. Including
The second program module group includes:
A specific phrase recognition program for reading the handwriting data from the shared list and estimating one or more text data corresponding to the read handwriting data when the handwriting data is stored in the shared list A module, the text data including at least accuracy information, including at least a specific word recognition program module;
The estimated text data is combined with the corresponding handwriting data and stored in the recognition result database.
A system characterized by that.

Images