JP3893013B2 - Character recognition method, computer-readable recording medium on which character recognition program is recorded, and character recognition device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a character recognition method, a computer-readable recording medium on which a character recognition program is recorded, and a character recognition apparatus, and in particular, recognizes characters (character strings) input by handwriting without using a keyboard without using a keyboard. In addition, the present invention relates to a character recognition method, a computer-readable recording medium on which a character recognition program is recorded, and a character recognition apparatus that can easily correct the recognized character (character string).
[0002]
[Prior art]
In recent years, when inputting characters into a computer equipped with a display device, handwriting input is performed within a set frame using an appropriate desktop digitizer (coordinate reading device) such as a mouse or tablet without using a keyboard. And a method for recognizing the handwritten character is known. Here, the handwriting input is also referred to as stroke input, and refers to input of characters and lines that can be expressed, such as character input, drawing stroke, and drawing lines between two points.
According to this method, even people who are unfamiliar with the keyboard operation can input desired characters on a display device such as a computer and create various sentences.
[0003]
[Problems to be solved by the invention]
However, in the conventional method, depending on the computer application (application software), a frame to be entered for each character may be set in advance. In such a case, when the user (user) performs handwriting input, the entry position is limited within the frame, and it becomes difficult to write, or only one character can be input by handwriting within one frame. It was assumed that the user would need time and effort when creating continuous sentences.
[0004]
In order to prevent the above-mentioned matters, what is necessary is to propose so-called “no frame” character recognition without setting a frame in advance. However, for example, when displaying a character recognition result for a handwritten input character on a display device such as a display provided in the computer, if the character recognition result is displayed on the handwritten character in exchange for the handwritten character, erroneous recognition may occur. If it happens, you won't know what you wrote. In addition, when the character delimiter position is wrong as a result of character recognition (for example, one character is recognized as two characters or conversely two characters are recognized as one character), the correct delimiter position is indicated. An operation for redisplaying the original pattern becomes necessary and troublesome. Furthermore, if the recognition result is displayed at a position away from the handwritten character, the user must move his / her line of sight each time the recognition result is confirmed or corrected, resulting in poor efficiency.
[0005]
SUMMARY OF THE INVENTION In view of the above, an object of the present invention is to easily perform character recognition and correction of recognition for a character input without handwriting.
For example, an object of the present invention is to correct a plurality of recognized characters as one input character with respect to a character that is input by handwriting without a frame and recognized. Another object of the present invention is to correct one recognized character as a plurality of input characters for a character that has been input by handwriting without a frame and has been character-recognized.
[0006]
It is another object of the present invention to display a candidate character in the vicinity of a recognition result and correct the recognized character by selecting a correct candidate character. Another object of the present invention is to correct an original handwritten input character by deleting the original handwritten input character and performing character recognition again on the newly input handwritten input character.
[0007]
Another object of the present invention is to search for candidate characters by applying the Viterbi algorithm. Further, the present invention corrects the character recognition and the recognition by re-searching only the handwritten input character behind the corrected handwritten input character after confirming the recognition result ahead of the corrected handwritten input character. The purpose is to perform efficiently.
[0008]
[Means for Solving the Problems]
According to the first solution of the present invention,
A display step for displaying input characters input by handwriting without a frame and recognition results of the input characters;
A position determining step for determining whether the input position of the stroke drawn for instructing the correction in the correction mode with respect to the recognition result displayed in the display step is inside or outside the first circumscribed rectangle of the input character. When,
A determination step of determining whether the correction instruction is an enclosure for a plurality of recognized characters or a delimiter for one recognized character when it is determined by the position determining step that the input position of the drawn line is outside the first circumscribed rectangle When,
A correction recognition step for correcting a plurality of recognized characters as one input character based on the determination step, or correcting one recognized character as a plurality of input characters, and performing character recognition again according to the correction instruction. A computer-readable recording medium on which a character recognition method and a character recognition program are recorded is provided.
[0009]
According to the second solution of the present invention,
An application that displays input characters handwritten without a frame and recognition results of the input characters;
A display / correction mode processing unit for displaying recognition results and correcting erroneous recognition;
A recognition engine that performs character recognition,
The display / correction mode processing unit
Position determining means for determining whether the input position of the stroke drawn for instructing correction in the correction mode with respect to the recognition result displayed by the application is inside or outside the first circumscribed rectangle of the input character; ,
When the position determination means determines that the input position of the drawn line is outside the first circumscribed rectangle, the determination means determines whether the correction instruction is an enclosure for a plurality of recognized characters or a break for one recognized character When,
Correction recognition for correcting a plurality of recognized characters as one input character or correcting one recognized character as a plurality of input characters based on the determination means and performing character recognition again by the recognition engine in accordance with the correction instruction Means and
A character recognition device comprising:
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a schematic configuration diagram of a character recognition system to which a frameless handwritten character recognition apparatus according to the present invention is applied.
The character recognition system 1 includes, for example, a user 2, a frameless handwritten character (character string) recognition control unit 10, an application 3, and a frameless handwritten character string recognition engine unit 4. The recognition control unit 10 includes, for example, a handwritten character input mode processing unit 11 that performs a “handwritten character input mode” process for inputting a handwritten character string, and displays a recognition result and corrects a recognition error. A display / correction mode processing unit 12 for performing the processing of “correction mode” is provided.
[0011]
The user 2 inputs a handwritten character string without a frame, and outputs an instruction for executing recognition of the handwritten character string to the input mode processing unit 11. An instruction for executing recognition of the handwritten character string can be output from the application 3 to the input mode processing unit 11 at an arbitrary timing. Further, the application 3 outputs the recognized character type setting (Hiragana / Katakana / Kanji ...) to the input mode processing unit 11. The input mode processing unit 11 that has received the instruction for executing the recognition of the handwritten character string outputs the input handwritten character string and the setting of the recognized character type to the recognition engine unit 4. The recognition engine unit 4 recognizes the handwritten character string and outputs the recognition result to the recognition control unit 10. The recognition control unit 10 displays the recognition result on the application 3.
[0012]
On the other hand, when the user determines that the character string displayed on the application 3 is erroneously recognized, the user 2 displays an instruction for correcting the erroneously recognized character string (misrecognition correction instruction). Output to the correction mode processing unit 12. Upon receiving this instruction, the display / correction mode processing unit 12 outputs a command (result correction command) for correcting the recognition result displayed on the application 3 to the recognition engine unit 4. The recognition engine unit 4 re-recognizes the misrecognized character string and outputs the corrected recognition result to the display / correction mode processing unit 12 again. As described above, the display / correction mode processing unit 12 displays the recognition result on the application 3 again. If the recognition result is correct, the recognition process is terminated. If the recognition result is incorrect, the user 2 outputs an instruction for correcting the erroneously recognized character string to the display / correction mode processing unit 12 again. .
[0013]
FIG. 2 is a flowchart of a character recognition method according to the present invention. For convenience of explanation, the explanation of the flowchart here is schematic, and the specific explanation will be described later.
First, a handwritten character string is input (for example, pen-down input) by a user using a pointing device such as a mouse or an electronic pen on a display screen without a frame (S101). Next, it is determined whether the processing mode is the handwritten character input mode or the recognition result display / correction mode (S103). As a criterion for determining these two modes, for example, if nothing is input on the display screen, the handwritten character input mode is selected. On the other hand, if the handwritten character string is already input, the recognition result display / The correction mode may be determined, or the two modes may be displayed with a mode switching button (not shown) or the like, or the user may be switched between the two modes by an instruction from a pointing device. In the handwritten character input mode, the character recognition is performed on the handwritten character that has been stroked (here, from the start to the end of pen-down input for a certain character or character string) (S105).
[0014]
Next, the recognition result recognized in step S105 is displayed on the display screen (S121). At this time, the recognition result is displayed together with the handwritten input character and its circumscribed rectangle. The user determines whether or not the displayed recognition result is correct (S123). If the displayed recognition result is correct, the process ends. If not, the process returns to step S101 again.
[0015]
On the other hand, if it is determined in step S103 that the display / correction mode is selected, a circumscribed rectangle indicating the recognized character separation position is obtained, and whether the pen-down input position is, for example, outside or inside the circumscribed rectangle is determined. Determination is made (S107, details will be described later). If the pen-down input position in step S107 is outside the circumscribed rectangle, it is determined whether the stroke drawn by the stroke input is “separation” or “enclosure” (S109). Here, “separation” means that when a handwritten character string that should be divided into a plurality of characters is erroneously recognized as one character, the erroneously recognized character is divided into a plurality of characters from the outside of the circumscribed rectangle of the character. This means that it is separated as handwritten characters for the characters. On the other hand, “enclosed” means that when a handwritten character that should be recognized as one character is erroneously recognized as two or more characters, the two or more characters that are erroneously recognized are represented as one character from the outside of the circumscribed rectangle of the character. Enclose as minute handwritten characters (details will be described later). If the drawn line is a break, correction processing and character recognition by a break described later are performed (S111). If the drawn line is a box, correction processing and character recognition by a box described later are performed (S113).
[0016]
On the other hand, if the position of the pen-down input in step S107 is inside the circumscribed rectangle, for example, it is determined whether the pen-down input is shorter than a predetermined length (or an operation for a shorter time than the predetermined time) (S115). ). If it is short (or if it is a short time operation), the candidate character is displayed, and if there is a correct character in the candidate character, the correct candidate character is selected by the user (S117). In step S115, if the left mouse click (or middle click or the like) or the pen-down input is longer than a predetermined length (or an operation longer than a predetermined time), it is determined that the input is a single character. Character recognition is performed on a single character re-input inside the circumscribed rectangle (S119). In addition, the process with respect to the result recognized or selected by step S111, step S113, step S117, and step S119 performs the same process as the process after step S105 mentioned above (step S121, S123). In step S115, in addition to branching the next process (step S117 or S119) depending on the length or time of the pen-down input, for example, other predetermined operations such as another click operation of the mouse, a button on a keyboard (not shown), or the like. The subsequent processing may be branched by this operation. In addition to the operation mainly using the electronic pen as described above, the operation may be performed using an appropriate input device such as a trackball, a mouse, or a keyboard.
[0017]
Hereinafter, the process of each step of the above-described flowchart will be described in detail.
(About Step S101)
[0018]
FIG. 3 is an explanatory diagram of a display screen when the character recognition program according to the present invention is started.
The recognition control unit 10 executes a “character recognition program” for recognizing a handwritten character string input on a display screen without a frame by a user, and enters a display device such as a display provided in the computer. The output screen is displayed, and the user can visually and easily perform operations such as input of a handwritten character string, execution of frameless character string recognition, and correction of erroneous recognition. On this display screen, an input / output display screen 101 without a frame, a recognition result display screen 102, and a function input unit 103 are displayed. The function input unit 103 corresponds to a plurality of functions, and performs an appropriate operation on the handwritten input character, for example, by clicking and executing an appropriate instruction with respect to the handwritten input character to be recognized. Has been placed. The function input unit 103 includes, for example, a recognition button, an input end button, a transmission button, a one-screen deletion button, an SPC button, a line feed button, a BS button, a delete all button, a recognition cancel button, a minimize button, and a scroll represented by an arrow. A button and a check box for arbitrarily selecting a character type to be recognized, such as “English letters, numbers, kanji, hiragana, katakana, and others”, are provided.
[0019]
FIG. 4 is an explanatory diagram regarding input of a handwritten character string.
In the “handwritten character input mode”, the user inputs handwritten characters on a display screen without a frame using, for example, a mouse or an electronic pen (for example, FIG. 4A is Japanese and FIG. 4B is English). Each example.) If a computer equipped with a display means is used when inputting handwritten characters, not only a mouse and an electronic pen but also a tablet used for inputting graphic data or drawing CG (computer graphics) A pointing device such as a suitable desktop digitizer (coordinate reading device) is also applicable. In addition, the mouse includes, for example, a left click button, a right click button, and, in some cases, a middle click button. Whether to have the function of starting cue can be changed as appropriate. In addition, in the handwritten character input mode, for example, a function that deletes one input at a time from the last input image and a function that deletes all input at once in order to correct incorrect input. With.
[0020]
FIG. 5 is an explanatory diagram regarding screen scrolling.
If the character string to be input cannot be written on the screen, for example, the user can input all the desired character strings by scrolling the screen (for example, FIG. 5A shows Japanese, FIG. 5). (B) shows examples of English, respectively, which are examples of scrolling further downward on the display screens of FIGS. 4 (A) and 4 (B).
[0021]
(About Step S105)
FIG. 6 is an explanatory diagram of handwritten character string recognition.
When the user finishes inputting the handwritten character string (stroke input ends), does the user select “recognition” from the four menus displayed by performing an appropriate operation such as right click, for example? Alternatively, an instruction to execute recognition of a handwritten character string is made by selecting a “recognition” button from a toolbar displayed on the screen of the application. Based on this instruction, the character recognition device starts frameless character string recognition (for example, FIG. 6A shows an example of Japanese and FIG. 6B shows an example of English).
[0022]
7 and 8 are display examples (1) and (2) of recognition results.
The recognition result displayed on the screen is indicated by, for example, a circumscribed rectangle indicating the separation position of each character and a recognition result of each character displayed in the circumscribed rectangle (for example, FIG. 7A shows Japan). (FIG. 7B shows an example in English). In this example, a circumscribed rectangle is displayed. However, a circumscribed rectangle area, which is an area where there is a circumscribed rectangle, may be assumed and the circumscribed rectangle may not be displayed. In addition, when the handwritten character is an English sentence, not only the recognition in English character units but also the recognition in English word units as shown in FIG. 8 is possible. In this case, for example, it can be recognized by separating adjacent characters as one word and searching for candidates in the word dictionary stored in the recognition engine unit 4 or the like. It should be noted that the display of the frame is an option, and the frame may not be displayed according to the user's preference. In addition, when the recognition result is displayed, the original handwritten character pattern is displayed in a different color, darkness, dot pattern, etc. to emphasize that the recognition result is a target for various operations described later, It is possible to rewrite the handwritten characters at any time. In the standard state, the recognition target is all character types, but one or more arbitrary character types from “English letters, numbers, kanji, hiragana, katakana, and others” are recognized. It can also be set.
[0023]
FIG. 9 is a display example (3) of the recognition result.
The color of the recognized normal size character is displayed in blue, for example, while the color of the small character such as a prompt sound is displayed in green, for example, between the normal size character and the small character. By displaying in different colors, it is possible to easily distinguish characters of normal size, for example, “Ya, Yu, Yo” and small characters “Ya, Yu, Yo”, such as sound prompts, from the recognition results. (For example, FIGS. 9 (A), (B), and (C) show normal size characters, and FIGS. 9 (D), (E), and (F) show examples of small characters such as prompt sounds). .
[0024]
(About Step S111)
FIG. 10 is an explanatory diagram of correction of a break position by a break.
First, when a handwritten character string (eg, “o” “ke”) that should be divided into a plurality of characters is erroneously recognized as one character (eg, “column”) as shown in FIG. Is assumed. In this case, as shown in FIG. 10B, the “column” is corrected to “o” and “ke” as shown in FIG. In addition, when a handwritten character string (for example, “o” and “f”) that should be divided into a plurality of characters is erroneously recognized as one character (for example, “H”) as shown in FIG. As shown in FIG. 10E, “H” is corrected to “o” and “f” as shown in FIG. 10F by drawing a dividing line from the outside of the circumscribed rectangular area.
[0025]
(About Step S113)
FIG. 11 is an explanatory diagram of correction of the break position by the enclosure.
First, when a handwritten character that should be recognized as one character originally, for example, “frame” is erroneously recognized as two or more characters (for example, “tree” and “medium”) as shown in the upper part of FIG. Is assumed. In this case, the “tree” and the “middle” are surrounded as one handwritten character from the outside of the circumscribed rectangle or circumscribed rectangle area as shown in FIG. “Medium” is corrected to “Frame” as shown in FIG.
[0026]
Further, when a handwritten character that should be recognized as one character, for example, “Y”, is erroneously recognized as two or more characters (for example, “e” and “S”) as shown in FIG. Encloses “e” and “S” as one handwritten character from the outside of the circumscribed rectangle or circumscribed rectangular area as shown in FIG. As shown in FIG. 11F, it is corrected to “Y”.
[0027]
FIG. 12 is an explanatory diagram for determining the enclosure.
When the points constituting the handwritten character string are included in the encircling line at a ratio or more set in advance by the application, it is determined that the handwritten character string is surrounded. For example, the handwritten character “shi” as shown in FIG. 12A is recorded internally as a collection of points as shown in FIG. 12B (the enclosed line is also internally a collection of points. Recorded). For example, when the application side is set so that 80% or more of the points are included in the encircling line, the encircling line is regarded as being encircled as shown in FIG. Since 7 points out of 8 points (87.5%) are in the box, it is determined that “shi” is surrounded. On the other hand, in the case of being surrounded as shown in FIG. 12D, since only 6 points (75%) of the 8 points are included in the box, it is determined that “shi” is not surrounded. To do.
[0028]
(About Step S109)
FIG. 13 is an explanatory diagram relating to a determination process for a box and a delimiter.
Here, a circumscribed rectangle of the enclosing line or dividing line is defined. That is, there are two circumscribed rectangles (circumscribed rectangles of a surrounding line or a separator line and a handwritten input character). FIG. 13A is a geometric explanatory diagram when the user draws an appropriate drawn line (enclosed or separated) with respect to a circumscribed rectangle having a larger horizontal width (X coordinate). First, the width of a circumscribed rectangle (broken line in the figure) of the enclosing or dividing line (solid line in the figure) is CX, and the vertical width is CY. Here, CX> CY, and the difference X (two-dot chain line in the figure) of the X coordinate of the start point and the end point (white circle in the figure) of the enclosing or dividing line by the user is “X ≦ CX × n”. If there is, it is determined as an enclosure (right side in the figure), while if “X> CX × n”, it is determined as a break (left side in the figure). “N” is a ratio set in advance by the application and is 0 ≦ n ≦ 1.0 (however, usually about 0.7).
[0029]
FIG. 13B is a geometric explanatory diagram when the user draws an appropriate drawn line (enclosed or separated) with respect to a circumscribed rectangle having a larger vertical width (Y coordinate). First, the width of a circumscribed rectangle (broken line in the figure) of the enclosing or dividing line (solid line in the figure) is CX, and the vertical width is CY. Here, CX ≦ CY, and the difference Y (two-dot chain line in the figure) of the Y coordinate of the start point and the end point (white circle in the figure) of the enclosing or dividing line by the user is “Y ≦ CY × n”. If there is, it is determined as an enclosure (right side in the figure), while if “Y> CY × n”, it is determined as a break (left side in the figure).
[0030]
FIG. 14 is a flowchart of the enclosing / separating determination process.
First, the circumscribed rectangle of the enclosing line or the dividing line is obtained, and it is determined which is larger in the horizontal width (expressed by the X coordinate) and the vertical width (expressed by the Y coordinate) of the circumscribed rectangle (S201). Next, when the horizontal width is larger, it is determined whether or not “the difference between the X coordinates of the start point and the end point ≦ the horizontal width of the circumscribed rectangle × n” (S203). The calculation formula “the difference between the X coordinate of the start point and the end point ≦ the width of the circumscribed rectangle × n” is: “If the size of the mouth of the line enclosed by the user is much smaller than the width of the circumscribed rectangle of the handwritten input character, , Otherwise it means “separator”. In step S203, when the size of the mouth of the line enclosed by the user is smaller than the preset ratio (n × 100% of the circumscribed rectangle) compared to the width of the circumscribed rectangle of the handwritten input character (for example, less than 30%) Is determined to be an enclosure. In addition, when the size of the mouth of the line enclosed by the user is not smaller than the width of the circumscribed rectangle of the handwritten input character, that is, the coordinate difference between the start point and the end point is n × 100% or more of the circumscribed rectangle Is determined as a break.
[0031]
On the other hand, if the horizontal width and the vertical width of the circumscribed rectangle are larger in step S201, it is determined whether “the difference between the Y coordinates of the start point and the end point ≦ the vertical width of the circumscribed rectangle × n”. (S205). This calculation formula is different from the above calculation formula only in the coordinate axis (Y coordinate: vertical width), and the description thereof is omitted. In step S205, if the size of the mouth of the line enclosed by the user is smaller than a preset ratio (n × 100% of the circumscribed rectangle) compared to the vertical width of the circumscribed rectangle of the handwritten input character, it is determined as an encircling. The Further, if the size of the mouth of the line enclosed by the user is not smaller than a preset ratio (n × 100% of the circumscribed rectangle) compared to the vertical width of the circumscribed rectangle of the handwritten input character, it is determined as a break. The
[0032]
Next, processing for specifically determining the position of the dividing line drawn by the user will be described.
FIG. 15 is an explanatory diagram of the determination by the circumscribed rectangle.
The position of the dividing line (solid line in the figure) is determined by adding an additional line to the dividing line for each character of the recognition result (for example, “a”) and converting it to a range of one character. Character recognition may be performed in the character range area. First, specifically, it is determined whether or not the circumscribed rectangle area (dotted line in the figure) of the recognition result (“a” in the figure) overlaps the circumscribed rectangle of the dividing line (two-dot chain line in the figure). . If they do not overlap as shown in FIG. 15A, it is assumed that they are not separated, and the process described later is omitted. That is, the processing time can be shortened by omitting the processing for the handwritten character (“A” in FIG. 15A) that is clearly unrelated to the dividing line. On the other hand, as shown in FIG. 15B, when both of the above are overlapped, processing described later is performed.
[0033]
FIG. 16 is an explanatory diagram for determining the separation direction.
Next, the horizontal width and vertical width of the circumscribed rectangle of the dividing line are compared. When the vertical width is longer as shown in FIG. 16A (vertical width> horizontal width), the drawn line drawn by the user is On the other hand, when the horizontal width is longer as shown in FIG. 16B (vertical width <horizontal width), it is determined as a horizontal partition.
[0034]
FIG. 17 is an explanatory diagram of a case where character delimiter positions and range areas are determined based on delimiters.
FIGS. 17A, 17B, and 17C are explanatory diagrams for determining the character delimiter positions based on vertical delimiters (for example, three patterns). In FIG. 17A, the end point of the dividing line (solid line in the figure) drawn from the top to the bottom by the user and the end point are extended in the vertical direction of the circumscribed rectangle of the delimiter line so as to contact the circumscribed rectangle of the recognition result. When the point “17a”, the lower left vertex “17b” of the circumscribed rectangle of the recognition result, the starting point “17d” of the dividing line, and the starting point are extended in the horizontal direction of the circumscribing rectangle of the dividing line, the circumscribing rectangle of the recognition result The point “17c” that intersects with the extended vertical side is connected by an additional line (thick line in the figure) to determine the position of character separation based on the vertical direction separation line. A state in which an additional line is added to the line is set as a one-character range area, and character recognition is performed in the one-character range area.
[0035]
In FIG. 17B, when the end point of the dividing line and the end point are extended in the horizontal direction of the circumscribed rectangle of the delimiter line, the point “17a” intersected with one side of the extended vertical direction of the circumscribed rectangle of the recognition result. , And a starting point “17c” of the dividing line and a point “17b” that intersects one side of the circumscribed rectangle of the circumscribed rectangle when the starting point is extended in the horizontal direction of the circumscribing rectangle of the dividing line Each character is connected by an additional line (indicated by a thick line in the figure) to determine the character delimiter position based on this vertical delimiter line, and the state where the additional line is added to this delimiter line is defined as a single character range area. Character recognition is performed in the range of one character.
[0036]
In FIG. 17C, the end point of the dividing line, and when the end point is extended in the vertical direction of the circumscribed rectangle of the dividing line, the point “17a” that intersects one side of the circumscribed rectangle in the recognition result and the recognition result The lower left vertex “17b” of the circumscribed rectangle, the upper left vertex “17c” of the circumscribed rectangle of the recognition result, the starting point “17e” of the dividing line, and the starting point extending in the vertical direction of the circumscribing rectangle of the dividing line By connecting the point “17d” that intersects the extended horizontal side of the circumscribed rectangle of the recognition result with an additional line (thick line in the figure), the character delimiter position based on this vertical delimiter line And a state in which an additional line is added to the dividing line is defined as a one-character range area, and character recognition is performed in the one-character range area.
[0037]
FIGS. 17D, 17E, and 17F are explanatory diagrams for the case where the character delimiter positions are determined based on the horizontal delimiters (for example, three patterns). In FIG. 17D, the end point of the separation line (solid line in the figure) drawn from the left to the right by the user and the end point are extended in the horizontal direction of the circumscribed rectangle of the delimiter line to contact the circumscribed rectangle of the recognition result. When the point “17a”, the vertex “17b” at the upper right of the circumscribed rectangle of the recognition result, the starting point “17d” of the dividing line, and the starting point are extended in the vertical direction of the circumscribing rectangle of the dividing line, the circumscribing rectangle of the recognition result By connecting the point “17c” that intersects one side of the extended horizontal direction with an additional line (thick line in the figure), the position of character separation is determined based on the horizontal direction separation line. A state in which an additional line is added to the line is set as a one-character range area, and character recognition is performed in the one-character range area.
[0038]
Further, in FIG. 17E, when the end point of the dividing line and the end point are extended in the vertical direction of the circumscribed rectangle of the delimiter line, the point “17a” intersected with the extended horizontal side of the circumscribed rectangle of the recognition result. , And a starting point “17c” of the dividing line and a point “17b” that intersects one side of the circumscribed rectangle that is the recognition result when the starting point is extended in the vertical direction of the circumscribing rectangle of the dividing line. Each character is connected with an additional line (indicated by a bold line in the figure) to determine the character delimiter position based on this horizontal delimiter line, and the state where the additional line is added to this delimiter line is defined as a one character range area. Character recognition is performed in the range of one character.
[0039]
In FIG. 17F, when the end point of the dividing line, and when this end point is extended in the horizontal direction of the circumscribed rectangle of the delimiter line, the point “17a” that intersects one side of the circumscribed rectangle in the recognition result and the recognition result The upper right vertex “17b” of the circumscribed rectangle, the upper left vertex “17c” of the circumscribed rectangle of the recognition result, the starting point “17e” of the dividing line, and the starting point extending in the horizontal direction of the circumscribing rectangle of the dividing line By connecting the point “17d” intersecting the extended vertical side of the circumscribed rectangle of the recognition result with an additional line (bold line in the figure), the character delimiter position based on this horizontal delimiter line And a state in which an additional line is added to the dividing line is defined as a one-character range area, and character recognition is performed in the one-character range area. FIG. 17 shows an example in which the dividing line is drawn from top to bottom or from left to right. However, when the dividing line is drawn from bottom to top or from right to left, The same processing may be performed with the columns reversed.
[0040]
Next, a process for searching for a character string by applying the Viterbi algorithm is described.
FIG. 18 is a flowchart of misrecognition correction processing.
The display / correction mode processing unit 12 inputs a surrounding line and / or a dividing line by the user (S301). The candidate character lattice in the front part is determined for the correction position by the input surrounding line or dividing line (S303). Here, the lattice is a path that connects between preceding and succeeding candidate characters. By determining the candidate character lattice in the front part, the candidate character lattice that contradicts the correction is deleted (S305). Next, the Viterbi algorithm is applied to the candidate character lattice limited to some extent, and the candidate character lattice is searched again (S307). The recognition result is output to the display / correction mode processing unit 12 (S309).
[0041]
Specific examples will be described below.
FIG. 19 is an explanatory diagram of the process of correcting erroneous recognition.
When the user corrects the character division using the enclosing line or the dividing line, the display / correction mode processing unit 12 and the recognition engine unit 4 perform the correction process of the recognition result. Specifically, the character separation is corrected for the character recognition and the candidate character that are the current recognition results, and candidate characters are listed for the corrected portion. Then, possible paths of candidate characters are derived. In addition, the path to be considered in the candidate character lattice is limited by determining the front part from the correction part and deleting the part inconsistent with the correction part. By applying the Viterbi algorithm to this candidate character lattice, character recognition, character delimitation, and connection as a character string are comprehensively evaluated, and an optimum character delimiter and character recognition result are obtained.
[0042]
FIG. 19A is an example of a handwritten character string that has not been correctly recognized and a candidate character lattice at that time. For example, the handwritten character string “Tomorrow is” is misrecognized as “day”, “month”, and “target” as a result of dividing it by “day”, “month”, and “day”, which are incorrect character delimiters. (Shaded characters in the figure). In this case, as a plurality of candidate characters, for example, “white” “day” for “day”, “month” “white” for “month”, and “day” and “month” For “”, “light”, for “ha”, “ke” “ha”, for “day” and “ha” “elbow” “target”, and “month” For “day”, “bold” and “yester” are cited as recognized candidates, and a candidate character lattice is derived based on the plurality of candidate characters.
[0043]
Next, with respect to the “day” and “month”, as shown in FIG. 19B, the “day” and “month” are surrounded by a surrounding line as a single character, thereby correcting the delimiter. Accordingly, candidate characters contradicting the correction, that is, “white” “day” for “day”, “month” “white” for “month”, and “bold” “yester” for “month” and “day”. Are deleted from the candidate characters, and the candidate character lattice that contradicts the correction is also deleted. As shown by the thick line in FIG. 19B, when the Viterbi algorithm is applied to this restricted candidate character lattice and a “probable” item is selected, recognition after correction is performed as shown in FIG. 19C. As a result, “Ming”, “Day”, and “Ha” that follow the shaded characters are obtained.
[0044]
Here, the Viterbi algorithm will be described. When a certain handwritten character string is recognized, the recognition results are as follows: “Probability of division” for each division by character delimiter, “Probability for character recognition” for each character recognition candidate, There is a continuous "probability of connection", and along the path connecting the candidates from the beginning to the end of the sentence, "probability as character recognition", "probability of division", "probability of connection" is added Then, the route with the largest sum of these is selected. An example of how to do this efficiently is the Viterbi algorithm. In this example, “morning”, “day”, and “ha” are obtained as the most probable routes.
[0045]
FIG. 20 is an explanatory diagram of an example in which erroneous recognition correction is spread.
When correction is performed by enclosing or delimiting, or by selecting or rewriting candidate characters, the Viterbi algorithm is applied to the candidate character lattice after correction after confirming character delimiters and character recognition before that. As a result, when misdividing and misrecognition are corrected from the front, the corrected part is confirmed accordingly, and the possibility of automatically correcting misdividing and misrecognition behind the correction increases, Necessary correction work can be reduced. Specifically, if the handwritten character “Is it sunny in tomorrow?” Was misdivided or misrecognized as “Tomorrow”, “Sun”, “Ha”, “Sunny”, “Target”, “Do”, or “Rejected” Divide the space between the two lines with a dividing line. As a result, the recognition result until “Tomorrow is fine” is fixed, and “Target” is corrected to “Re” and “Ru”. In addition, the erroneously recognized “rejection” in the back is automatically corrected to “na” and “?” By the Viterbi algorithm.
[0046]
(About steps S115 and S117)
FIG. 21 is an explanatory diagram of erroneous recognition correction for each single character.
As shown in FIGS. 21A, 21B, and 21C, when the delimiter position is correct but the recognition result of a single character (Japanese) is wrong (for example, “sentence” is erroneously recognized as “six”) ) The user performs, for example, a left click (or middle click) or tap of the mouse within the circumscribed rectangular area of the wrong character. As a result, a recognition candidate list for each single character is displayed as shown in FIG. 21B (for example, “six”, “missing”, “sentence”, “hole”, “how”). When the user selects “sentence”, which is a correct recognition result, from the list, the recognition result is corrected to “sentence” as shown in FIG. Also, as shown in FIGS. 21D, 21E, and 21F, when the delimiter position is correct but the recognition result of the single character (English) is wrong (for example, “T” is misrecognized as “J”) The user performs, for example, a left click (or middle click) or tap of the mouse within the circumscribed rectangular area of the wrong character. As a result, a recognition candidate list for each single character is displayed as shown in FIG. 21E (for example, “T” “f” “J” “I” “P”). When the user selects “T” which is the correct recognition result from the list, the recognition result is corrected to “T” as shown in FIG.
[0047]
(About Step S119)
FIG. 22 is an explanatory diagram of re-input for each single character.
First, since the recognition candidates for each single character are displayed in order of decreasing similarity, they are registered in the dictionary when the handwritten character string is written too loosely or when it is a habitual character. If the similarity to the shape of the character to be recognized is low, the similarity of the correct answer candidate is low, and the above-described recognition candidate list for each single character may not include a correct result. At this time, re-input is performed for each single character and the recognition result is corrected. The reason why the correct result is not included in the recognition candidate is that the similarity is too low to be displayed in the candidate, and characters indicating the correct result are not necessarily stored in the dictionary. Specifically, when “correction” is misrecognized as “trust” as shown in FIG. 22 (A), the user re-appears again from the inside of the circumscribed rectangular area as shown in FIG. 22 (B). By inputting characters (for example, “correction”) by handwriting, “trust” is corrected to “correction” as shown in FIG. Further, for example, as shown in FIG. 22D, when “S” is erroneously recognized as “B”, the user again reads the character (for example, from the inside of the circumscribed rectangular area as shown in FIG. 22E). , “S”) by handwriting input, “B” is corrected to “S” as shown in FIG.
[0048]
Here, the discrimination between the click (tap) and the re-input in step S115 is performed, for example, when the moving distance from the pen-down position to the up position with a pointing device such as a mouse is a preset threshold value (for example, 3 dots). Click (tap) if it is less than the threshold), re-input if it is greater than the threshold, or click if the time from pen down to up is less than the preset threshold. For example, there is a method for determining re-input. Further, by clicking (tapping) or re-inputting in the circumscribed rectangular area, the processing in steps S117 and 119 described above is executed, so that the circumscribed rectangular area plays a role as a user interface button or the like. It will also be.
[0049]
The character recognition method of the present invention includes a computer-readable recording medium on which a character recognition program is recorded, a program product that includes the character recognition program and can be loaded into the internal memory of the computer, and a recording that can be used by the computer including the character recognition program. It can be provided by a program product or the like stored in a medium.
[0050]
In the present invention, it is also possible to improve recognition accuracy by accumulating user's habits of inputting characters as data. In this case, the priority of the candidates can be weighted with data selected in the past, or when the input character is tilted, processing for correcting the tilt can be executed in advance.
[0051]
【The invention's effect】
According to the present invention, as described above, it is possible to easily perform character recognition and recognition correction for characters input by hand without a frame.
For example, according to the present invention, a plurality of recognized characters can be corrected as one input character with respect to a character that is input by handwriting without a frame and recognized. Further, according to the present invention, it is possible to correct one recognized character as a plurality of input characters with respect to a character that is input by handwriting without a frame and recognized.
[0052]
In addition, according to the present invention, it is possible to correct the recognized character by displaying the candidate character in the vicinity of the recognition result and selecting the correct candidate character. According to the present invention, the original handwritten input character can be erased and corrected by performing character recognition again on the newly input handwritten input character.
[0053]
In addition, according to the present invention, candidate characters can be searched by applying the Viterbi algorithm. Furthermore, according to the present invention, after confirming the recognition result ahead of the corrected handwritten input character, by re-searching only the handwritten input character behind the corrected handwritten input character, character recognition and recognition can be performed. Correction can be performed efficiently.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of a character recognition system to which a frameless handwritten character recognition apparatus according to the present invention is applied.
FIG. 2 is a flowchart of a character recognition method according to the present invention.
FIG. 3 is an explanatory diagram of a display screen when a character recognition program according to the present invention is started.
FIG. 4 is an explanatory diagram regarding input of a handwritten character string.
FIG. 5 is an explanatory diagram relating to screen scrolling.
FIG. 6 is an explanatory diagram of recognition of a handwritten character string.
FIG. 7 is a display example (1) of a recognition result.
FIG. 8 is a display example (2) of a recognition result.
FIG. 9 is a display example (3) of a recognition result.
FIG. 10 is an explanatory diagram illustrating correction of a break position by a break.
FIG. 11 is an explanatory diagram regarding correction of a break position by enclosure.
FIG. 12 is an explanatory diagram for determining the enclosure.
FIG. 13 is an explanatory diagram related to an enclosure / separation determination process.
FIG. 14 is a flowchart of a surrounding / separation determination process.
FIG. 15 is an explanatory diagram of determination based on a circumscribed rectangle.
FIG. 16 is an explanatory diagram for determining the separation direction.
FIG. 17 is an explanatory diagram of a case where a character delimiter position and a range area are determined based on delimiters.
FIG. 18 is a flowchart of misrecognition correction processing;
FIG. 19 is an explanatory diagram of a process of erroneous recognition correction.
FIG. 20 is an explanatory diagram of an example in which erroneous recognition correction is spread.
FIG. 21 is an explanatory diagram of erroneous recognition correction for each single character.
FIG. 22 is an explanatory diagram of re-input for each single character.
[Explanation of symbols]
1 Character string recognition system
2 users
3 Application
4 recognition engine part
10 Recognition control section
11 Input mode processing section
12 Display / correction mode processing section

Claims (15)

An input mode processing unit inputting a handwritten character string without a frame from the input device;
The display / correction mode processing unit displays on the display screen of the display device the input character input by handwriting without a frame by the input mode processing unit and the recognition result of the input character by the recognition engine that performs character recognition . 1 display step,
A step in which the display / correction mode processing unit inputs a drawn line indicating an enclosure or a break from the input device;
Display and correction mode processing unit, wherein the first display recognition result displayed by step, the input position of the drawn line input to direct the correction in the correction mode, the first circumscribed rectangle of the input character A position determination step for determining whether it is inside or outside,
When the display / correction mode processing unit determines that the input position of the drawn line is outside the first circumscribed rectangle in the position determining step, the correction instruction is an enclosure for a plurality of recognized characters or a single recognized character A determination step for determining whether or not a break for
Based on the determination step , the display / correction mode processing unit corrects a plurality of recognized characters as one input character or corrects one recognized character as a plurality of input characters, and again performs a character by the recognition engine according to the correction instruction. A correction recognition step for recognition ;
A display / correction mode processing unit including a second display step of displaying a recognition result recognized by the correction recognition step on a display screen of a display device;
The correction recognition step includes
A recognition engine extracting candidate characters that have been corrected and re-recognized by surrounding or delimiting, and forming a path that connects between the candidate character and other recognition character candidates;
The recognition engine deleting a path that is related to non-candidate characters and that conflicts with the correction;
A character recognition method including: a recognition engine searching for an optimum route based on the candidate character and the formed route . The correction recognition step includes
Recognition engine further comprises a step of determining the forward recognition results than corrected written text than corrected written text was to re-search only the rear of the written text according to claim 1 Character recognition method. An input mode processing unit inputting a handwritten character string without a frame from the input device;
The display / correction mode processing unit displays on the display screen of the display device the input character input by handwriting without a frame by the input mode processing unit and the recognition result of the input character by the recognition engine that performs character recognition. 1 display step,
A step in which the display / correction mode processing unit inputs a drawn line indicating an enclosure or a break from the input device;
The display / correction mode processing unit inputs the drawing line input to instruct correction in the correction mode with respect to the recognition result displayed in the first display step, and the first circumscribed rectangle for the input character A position determination step for determining whether it is inside or outside,
When the display / correction mode processing unit determines that the input position of the drawn line is outside the first circumscribed rectangle in the position determining step, the correction instruction is an enclosure for a plurality of recognized characters or a single recognized character A determination step for determining whether or not a break for
Based on the determination step, the display / correction mode processing unit corrects a plurality of recognized characters as one input character or corrects one recognized character as a plurality of input characters, and again performs a character by the recognition engine according to the correction instruction. A correction recognition step for recognition;
A display / correction mode processing unit including a second display step of displaying a recognition result recognized by the correction recognition step on a display screen of a display device ;
In the determination step, the display / correction mode processing unit
Forming a second circumscribed rectangle for the drawn line input to instruct correction, a difference between a starting point and an ending point of the drawn line in the longitudinal direction of the second circumscribed rectangle, and one side in the longitudinal direction of the first circumscribed rectangle By comparing with the length of
If the difference is smaller than a predetermined percentage of the length of one side, it is determined that the correction instruction is a box,
A character recognition method in which a correction instruction is determined to be a break if the difference is not smaller than a predetermined ratio of the length of one side. In the correction mode, the display / correction mode processing unit has an input position of a drawn line input for instructing correction inside the first circumscribed rectangle, and the drawn line is not longer than a predetermined length or input time is not longer than a predetermined time. If
Written text, in the vicinity of the first circumscribed rectangle or the recognition result display candidate characters, according to any one of claims 1 to 3 correct candidate character further comprises display selection step of correcting the recognized character by being selected Character recognition method. In the correction mode, the display / correction mode processing unit has the input position of the drawn line input to instruct correction within the first circumscribed rectangle, and the drawn line is longer than a predetermined length or input time is longer than a predetermined time. If
The character recognition method according to any one of claims 1 to 4 , further comprising a re-input recognition step for performing character recognition by a recognition engine on a newly input handwritten input character. The recognition engine, when the handwritten input character is an English word,
Separate close to that letters as one word, the stored word by searching the candidate in the dictionary, the character recognition method according to any one of claims 1 to 5 so as to character recognition English words . In the first and second display steps, the display / correction mode processing unit displays the input character, the recognized character, and / or the circumscribed rectangle by appropriately changing the color, density, or thickness of the drawn line. character recognition method according to any one of claims 1 to 6 were. When the handwritten input character is a small character indicating a prompt sound , the display / correction mode processing unit
In the first and second display steps,
The character recognition method according to any one of claims 1 to 7 , wherein a character having a normal size and a small character indicating a prompt sound are distinguished and displayed. Said inner region of the first circumscribed rectangle of each input character determined by the recognition step, the character recognition method according to any one of claims 1 to 8 so as to function as a button of the user interface. An input mode processing unit inputting a handwritten character string without a frame from the input device;
The display / correction mode processing unit displays on the display screen of the display device the input character input by handwriting without a frame by the input mode processing unit and the recognition result of the input character by the recognition engine that performs character recognition . 1 display step,
A step in which the display / correction mode processing unit inputs a drawn line indicating an enclosure or a break from the input device;
Display and correction mode processing unit, wherein the first display recognition result displayed by step, the input position of the drawn line input to direct the correction in the correction mode, the first circumscribed rectangle of the input character A position determination step for determining whether it is inside or outside,
When the display / correction mode processing unit determines that the input position of the drawn line is outside the first circumscribed rectangle in the position determining step, the correction instruction is an enclosure for a plurality of recognized characters or a single recognized character A determination step for determining whether or not a break for
Based on the determination step , the display / correction mode processing unit corrects a plurality of recognized characters as one input character or corrects one recognized character as a plurality of input characters, and again performs a character by the recognition engine according to the correction instruction. A correction recognition step for recognition ,
A recognition engine extracting candidate characters that have been corrected and re-recognized by surrounding or delimiting, and forming a path that connects between the candidate character and other recognition character candidates;
The recognition engine deleting a path that is related to non-candidate characters and that conflicts with the correction;
A correction recognition step including a step of a recognition engine searching for an optimum route based on the candidate character and the formed route;
A computer reading in which a display / correction mode processing unit records a character recognition program for causing a computer to execute a second display step of displaying a recognition result recognized in the correction recognition step on a display screen of a display device. Possible recording media.   An input mode processing unit inputting a handwritten character string without a frame from the input device;
  The display / correction mode processing unit displays on the display screen of the display device the input character input by handwriting without a frame by the input mode processing unit and the recognition result of the input character by the recognition engine that performs character recognition. 1 display step,
  A step in which the display / correction mode processing unit inputs a drawn line indicating an enclosure or a break from the input device;
  The display / correction mode processing unit inputs the drawing line input to instruct correction in the correction mode with respect to the recognition result displayed in the first display step, and the first circumscribed rectangle for the input character A position determination step for determining whether it is inside or outside,
  When the display / correction mode processing unit determines that the input position of the drawn line is outside the first circumscribed rectangle in the position determining step, the correction instruction is an enclosure for a plurality of recognized characters or a single recognized character A second circumscribing rectangle for the drawn line that is input to instruct correction, and the start and end points of the drawn line in the longitudinal direction of the second circumscribed rectangle By comparing the difference with the length of one side in the longitudinal direction of the first circumscribed rectangle, if the difference is smaller than a predetermined ratio of the length of one side, it is determined that the correction instruction is a box, and the difference is The determination step of determining that the correction instruction is a break if it is not smaller than a predetermined ratio of the length;
  Based on the determination step, the display / correction mode processing unit corrects a plurality of recognized characters as one input character or corrects one recognized character as a plurality of input characters, and again performs a character by the recognition engine according to the correction instruction. A correction recognition step for recognition;
  A computer reading in which a display / correction mode processing unit records a character recognition program for causing a computer to execute a second display step of displaying a recognition result recognized in the correction recognition step on a display screen of a display device. Possible recording media. An application that displays input characters handwritten without a frame and recognition results of the input characters;
A display / correction mode processing unit for displaying recognition results and correcting erroneous recognition;
A recognition engine that performs character recognition,
The display / correction mode processing unit
Position determining means for determining whether the input position of the stroke drawn for instructing correction in the correction mode with respect to the recognition result displayed by the application is inside or outside the first circumscribed rectangle of the input character; ,
When the position determination means determines that the input position of the drawn line is outside the first circumscribed rectangle, the determination means determines whether the correction instruction is an enclosure for a plurality of recognized characters or a break for one recognized character When,
Correction recognition for correcting a plurality of recognized characters as one input character or correcting one recognized character as a plurality of input characters and performing character recognition again by the recognition engine according to the correction instruction based on the determination means and means,
The correction recognition means includes
Extracting candidate characters that have been corrected and re-recognized by surrounding or separating, and forming a path connecting the candidate character and other recognized character candidates,
Delete paths that are related to non-candidate characters and conflict with corrections,
A character recognition device that searches for an optimum route based on a candidate character and a formed route .   An application that displays input characters handwritten without a frame and recognition results of the input characters;
  A display / correction mode processing unit for displaying recognition results and correcting misrecognition,
  A recognition engine that performs character recognition,
  The display / correction mode processing unit
  Position determining means for determining whether the input position of the stroke drawn for instructing correction in the correction mode with respect to the recognition result displayed by the application is inside or outside the first circumscribed rectangle of the input character; ,
  When the position determination means determines that the input position of the drawn line is outside the first circumscribed rectangle, the determination means determines whether the correction instruction is an enclosure for a plurality of recognized characters or a break for one recognized character When,
  Correction recognition for correcting a plurality of recognized characters as one input character or correcting one recognized character as a plurality of input characters based on the determination means and performing character recognition again by the recognition engine in accordance with the correction instruction Means and
  In the determination means,
  Forming a second circumscribed rectangle for the drawn line input to instruct correction, a difference between a start point and an end point of the drawn line in the longitudinal direction of the second circumscribed rectangle, and one side in the longitudinal direction of the first circumscribed rectangle By comparing with the length of
  If the difference is smaller than a predetermined percentage of the length of one side, it is determined that the correction instruction is a box,
  If the difference is not smaller than a predetermined ratio of the length of one side, a character recognition device in which a correction instruction is determined to be a break. The character recognition according to claim 12 or 13 , further comprising an input mode processing unit for performing character recognition on the input character in the input mode, and obtaining a first circumscribed rectangle of the input character and a recognition result by the recognition engine. apparatus. The character recognition device according to claim 12, wherein an area inside the first circumscribed rectangle is an input button of a user interface.

Images