JP5352757B2 - Handwritten character recognition method, handwritten character recognition system, handwritten character recognition program, and storage medium - Google Patents

Description

  The present invention relates to a handwritten character recognition method and a handwritten character recognition system that perform online handwritten character recognition, a handwritten character recognition program that realizes the recognition method, and a storage medium that stores the program.

  Many character recognition systems have been proposed and put into practical use, but there are two basic principles, one is a structural analysis, the other is a pattern matching, and the former is In general, the recognition system is light, so if input restrictions are strong, that is, if the stroke number and stroke order are fixed, or if either is fixed, or if the latter is free of both stroke number and stroke order It has been applied to cases close to that.

  From the standpoint of structural analysis, IEICE Transactions, 56-D, 5, pp. 312-319, “Online real-time recognition of handwritten numerals and katakana characters” and Japanese Patent Publication No. 59-131972 issued by the Japan Patent Office, there are so-called basic stroke methods. It is classified into simple stroke (4 types) and compound stroke (7 types) and recognized by the identification automaton. Although it is easy, there are problems in creating a dictionary, dealing with continuous characters and abbreviations. It has been said that there is a problem in development.

  There are two types of pattern matching methods. One is IEICE Transactions, J63-D, 2, pp. 153-160, “Online recognition of handwritten characters by point approximation of strokes”, strokes are approximated by a small number of points, and these are used as feature points, and the direction of brush movement at the end points is estimated. A feature vector is constructed with the focus. The dictionaries are broken down into strokes, which also have feature vectors, take the correspondence between the input vectors and the feature vectors prepared for each category, calculate the distance for the matched dictionaries, and give the smallest distance A dictionary name is a recognized character name, which is basically free to stroke order and stroke count.

  There is another pattern matching method, and there is “Handwritten character recognition by Rubber String Matching method” described in the IEICE Technical Committee paper PRL74-20) as the original paper of the feature point correspondence method. As disclosed in Japanese Laid-Open Patent Publication Nos. 57-45679 and 8-24942 issued by the Agency, the input character and the feature point vector of the dictionary are supported by the DP (Dynamic Programming) method, which is a handwritten character. The mainstream of online recognition.

Recently, offline character recognition technology has been applied online.
There is OCR technology that has been accumulated so far, which can also be used for online character recognition.
From this standpoint, looking at OCR technology, the mainstream is direction feature matching. There is an enormous amount of literature on this, but the basic idea method is, for example, as an original paper [The Institute of Electronics, Information and Communication Engineers Journal, J62-D, 3, pp. 217-224, “Improved Correlation Method for Character Recognition”]. The fundamental difference between this method and the method of structural analysis is that the features are generally assigned to an n × m grid plane, and the feature distribution on this plane is the final input character representation. Scan to the right to make an n × m dimensional vector. For identification, the inner product (similarity) between the standard direction feature vector and the input character direction feature vector is calculated, and the category name of the standard direction feature vector having the highest value is used as the answer. At this time, a highly non-linear normalization preprocessing is performed particularly for handwritten characters that are significantly deformed. This is necessary because the method is based on an n × m lattice plane. The advantage of this approach is that, in general, vector space, especially Hilbert space theory in which inner products are defined, can be applied, so that advanced discriminating theory is used. In practical terms, it is resistant to noise. However, the biggest advantage of online is killing the ease of segmentation. Online, for example, it is possible to make a machine recognize even if a number of characters are written in the same place. Furthermore, even highly nonlinear normalization is not sufficient for truly significant deformations, and for example, rotational deformation requires considerably high degree of normalization with a considerable degree of calculation. [S. Mori, H. Nishida, H. There are detailed descriptions in Chapter 3 of Yamada, Optical Character Recognition, Wiley.

  As the above feature, it is also possible to take, for example, a curvature. From this point of view, [The Journal of the Institute of Electronics, Information and Communication Engineers, J62-D, 3, pp. 217-224, “Improved Correlation Method for Character Recognition”], especially because cursive letters “g”, “y” and Arabic numeral “9” are easily mistaken for handwritten characters. The tangent angle difference of the curved part is obtained, appropriate quantization is performed, the character is represented by the conventional direction feature vector and the local rotation feature vector, and all feature vectors obtained by combining them are obtained, and the blurring process is performed. (This is actually done on the grid plane). A method has been proposed in which a blur total feature standard vector is obtained for each category, similarity calculation is performed, and an answer is obtained. By the way, from the viewpoint of structural analysis, cursive “g” and “y” are quite different from the Arabic numeral “9”. It is because the structure of the upper part is seen explicitly. However, in feature matching, it is mixed in the inner product process to make one scalar quantity, so the upper structure will be seen in the shadow, and since all three letters have a strong linear structure, they are buried in this straight line. End up. Therefore, we prepared a local feature plane. However, as we will see later, in our method, what is called a rotation feature above is sought globally, not locally, and is easily expressed naturally in a consistent manner. Therefore, the above three characters can be recognized very easily.

  Recognition systems that are invariant to rotation, such as graphics, objects placed in logistics systems, and airplanes in the military, are required for a wide range of objects.

  Therefore, research has been conducted since a long time ago, and many papers are still published. The research up to 1990 is described in detail in Shunji Mori and Atsuko Sakakura, the basics of image recognition (II)). Mori, H. Nishida, H. There is a detailed description in Optical Character Recognition by Yamada, the mainstream of this research is the method of moments, combining higher-order moments so that the phase angle cancels, and the Fourier called the Fourier descriptor name There is an application of the conversion method, which reflects the recent increase in the speed of PCs, but the research itself is thriving, but it has not yet appeared in the market as a practical application, but this trend and Separately, the Journal of Information Processing Society of Japan, Vol. 27, No. 5: May, 1986, “Linear line segment, arc sequence, subject to“ Online handwritten figure recognition method without stroke number, stroke order, rotation, separation ”” Approximate with, express with their relative angle change, match the object (input figure) with similar expression in the dictionary, measure each other's distance with the sum of the absolute value of the difference of each angle change, rotation invariant One Although the disadvantage of being susceptible to acute detection have been described in the paper itself.

  The so-called matching method is a feature (for example, stroke direction) matching on a two-dimensional plane where characters are placed. On-line handwritten character recognition is the so-called DP (Dynamic Programming) matching, also known as elastic string matching. In the former case, the distribution of features in two dimensions is represented by a vector, the distance between characters is defined as the inner product of these vectors, and the statistical method is used as an identification problem in the vector space where the inner product is defined. Perform character recognition. In the latter, simple superposition matching is extended, and character recognition is performed by matching input characters with standard characters adaptively and flexibly.

  For such a method, a character recognition method generally called a structural analysis method has been studied. This is applicable to general graphics and is a good method, but it requires symbolization of the object, specifically character strokes, and has been matched by symbols. However, there is a problem in this symbolization, the symbolization loses flexibility, the design is not mechanical, and research and development are stuck. For example, as described in [Electronic Communication Society Journal, 56-D, 5, pp. 312-319, “Online Real-Time Recognition of Handwritten Numerals / Katakana Characters”] was published in 1973. At this stage, the clockwise and counterclockwise on-line characters are used as a feature, but these sequences are all symbolized. These are detected by increasing or decreasing the X-coordinate value of the input pattern, and are expressed by 11 symbols. Such symbolic representation is flexible and is actually only partially used as a special case. After that, the structural analysis method was introduced in 1981 by the IEICE Transactions J64-D, 8, p705-712, “Algebraic Structural Representation of Shapes”. IEEE Trans. On Pattern Analysis and Machine Intelligence Vol. 14, No. 5, pp. 1029-1058, “Algebraic Description of Curve Structure”, a practical algebraic system was constructed, but it was still a symbol expression. In this way, structural analysis techniques have remained at the symbol expression level. In order to break through this wall, the fact that it must be analog, not symbol, has been often said by academic societies, but until now, no concrete method of analogization has been found.

However, the above conventional techniques have the following basic problems.
Although structural analysis is simple, it is not flexible, the boundary of basic patterns is a problem, it is discrete, awkward, and it takes time to create a dictionary.
The pattern matching method, especially the DP method, is heavy.
The main purpose of the pattern matching method, including offline recognition, is simply to read, that is, to force the input characters into the dictionary, and the cause / result is misunderstood. It's not uncommon for a person to know something.

  The recognition of single characters has been described above. A method for efficiently recognizing, for example, a cursive character string written continuously will be described below. Again, various technologies have been proposed and put into practical use, but they can be broadly divided into two methods, a “segment method” and a “holistic method”.

  The “holistic method” is a method of recognizing a character string as a whole as if it is a single character. However, this problem is that the number of templates becomes very large. In the case of English words, at least 50,000 are required.

  On the other hand, the “segment method” does not cause such a problem because each character constituting the character string is cut out and recognized. But the question is how to segment / cut out, and this has long been known as a challenge. (For a systematic and detailed introduction to the work up to 1995, there is a paper by Richard G. Cassy and Elic Lecolinet: IEEE Transactions on Pattern and Machine Intelligence, Vol. 18, No. 7, July 1996, pp690- 706).

  There are two main solutions to this difficult problem. One is a method of using various features, cutting out characters first, and then recognizing the cut out characters. This is called a “cutting method” and is a classic method, and the object is mainly used for printing character strings. In such an object, there are some gaps between characters, and such a gap is detected from the projection of the density on the X axis of the object. However, when the characters are inclined as a whole, the density projection becomes invalid, so it is necessary to normalize the inclination, and the “cutting method” is basically limited.

  The second is a “recognition-based method” in which recognition and clipping are performed simultaneously. That is, character recognition is used as means for cutting out characters.

  The first idea came out in 1962, and is a method of continuously scanning one window frame from the beginning of the character string and continuously recognizing characters in the window frame. However, in this method, for example, “W” is recognized as two “V”. Therefore, a method has been considered in which the entire character string is considered, a window frame string in a conceivable range is assumed, and the character string recognized in the window frame is evaluated as a whole. At this time, since the combination becomes large, the DP (Dynamic Programming) method has been adopted as an efficient calculation method. This method was published by V.A.Kovalevsky and is in “Character Readers and Pattern Recognition,” Washington, D.C., Spartan Books, 1968.

  There, as an example, three types of characters, “A”, “N”, and “H”, which are very deteriorated, are taken up. The string is assumed to be a Russian character. When this character string is continuously scanned, the character “R” is recognized and detected in the middle of NH because of the “N” and “H” lines (decoration of European print). However, since the recognition evaluation is low for the entire character string, the three characters A, N, and H are recognized and evaluated as the entire character string. That is, correct segmentation and recognition are performed simultaneously.

  This idea was innovative, but has not received much attention for a long time. The rediscovery of this method is by Richard G. Cassy, who belonged to IBM (one of the authors of the above paper). Based on this idea, he and N.Nagy proposed a new method called recursive clipping.

  As an example, r and m are completely connected by the degraded type letters “r” and “m”. Initially, the window frame is taken to include all character examples. In this case, it is “rm”. If the example of the character in the character frame is recognized and can be recognized, it is sufficient. This is the end. However, if it cannot be recognized, the window frame is narrowed from the right end, recognition is performed there, and it is sufficient if the inside of the window frame and the remaining part can be recognized. In the example of “rm”, “r” and “m” are recognized at this stage, and this is the end. This method is described in “Recursive Segmentation and Classification of Composite Patterns,” Proc., Sixth International Conference on Pattern Recognition, p.1923, 1982.

  The method described above is off-line character recognition, and the flow of matching is in the positive direction on the X axis. This forms the natural axis of matching. Matching takes the inner product of density levels. In this respect, the local features are composed of 25 types of 3 × 3 masks based on the nth-order autocorrelation method that is invariant to translation, and 1 × M perpendicular to the X axis (M is the height of the character frame) ) Is detected, these features are detected, and a method of recognizing an object from their histogram distribution with an appropriately designed discriminant function has been proposed. This is mainly an invention by Otsu and has been granted a US patent.

Patent Document 1 is a US patent by Otsu et al.
U.S. Pat. No. 4,288,777

However, it is difficult to say that these methods are all complicated and sufficiently satisfy the practical level in terms of performance.
As flexible structural matching, extensive research has been done in the field of scene recognition rather than character recognition. However, they are two-dimensional feature arrangements and matching on a two-dimensional graph that generally expresses their relationship. There is a tremendous amount of research on this.

  On the other hand, in online character recognition, the image seen by humans is two-dimensional, but it is strictly on the time axis and is one-dimensional. That is, it can be expressed by a simple one-dimensional linear graph. This point of view dramatically simplifies the problem. Moreover, natural cutout candidate points are implicitly aligned on the time axis according to the winding angle. This winding angle and a linear (one-dimensional) graph are the core of the present invention.

  The present invention has been made in view of the above points, and basically belongs to the structural analysis method described above. However, the present invention overcomes the conventional problems and provides a basis for a flexible structural analysis method. Therefore, the object is to avoid the problem of symbolization, express the structure in an analog manner, and perform flexible and simple matching with the standard.

In the present invention, when recognizing online handwritten characters, the input handwritten character string is captured by the parameter expression for each image, the polygonal line approximation is performed for each image, and each polygonal line approximated by the line is approximated from the start point. As a vector to the end point, the angle between the reference axis and each polygonal line is obtained as a polygonal line angle series, the external angle series of each vertex of the obtained polygonal line is obtained, and the same sign of positive or negative of the external angle series continues. The sum of the outer angles of the code is the winding angle series, and the starting point, the end point, the point that gives a winding angle that is an integer multiple of 90 degrees, and the point at which the winding angle changes are taken as nodes , based on the feature extraction by each obtained sequence. , by the attribute of the edges between attributes and nodes as a point of the node, it gives a graphical representation, the matching with the template in the mask structure, especially an open does not define the start and end points , And it performs character recognition.

  According to the present invention, when recognizing on-line handwritten characters, it is flexible and resistant to noise and deformation at the edges, even for characters that maintain normal top / bottom and left / right relationships and when rotation invariance is required. Robust recognition is performed, and even in a continuous character string, recognition can be performed in the same manner as a method for recognizing characters written in isolation without cutting them out as preprocessing.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

  The example of the present embodiment is applied to a system that performs online handwritten character recognition, and FIG. 1 shows a configuration example in which each processing unit has a hardware configuration. In addition, as shown in FIG. 1, it is good also as a structure which performs each process part by a common arithmetic processing part, or what made the handwritten character recognition of this example into the program in general purpose arithmetic processing apparatuses, such as a personal computer apparatus, It may be mounted so that similar handwritten character recognition is performed.

  Moreover, in the following description, the concept required for handwritten character recognition in this example is defined by the terms shown in Table 1 below.

  The configuration shown in FIG. 1 will be described. By writing a character on the paper 1 with the pen 2, a pen stroke (handwriting) 1a on the paper 1 is detected on the pen 2 side. The pen 1a is detected by a camera built in the pen 2, for example. Alternatively, the movement of the pen 2 itself may be detected from an acceleration sensor or the like. Furthermore, instead of detecting on the pen side, the paper 1 side may be configured by some panel so that the handwriting can be detected electrically. In any case, in the case of this example, since it is online handwritten character recognition, it is configured to be able to determine a change in handwriting over time.

  The handwriting data detected by these processes is sent to the input processing unit 3, and an input process for detecting character information is performed. The input data is sent to the polygonal line approximation unit 4, the feature extraction unit 5, the identification unit 6, and the identification result output unit 7, and the corresponding processing is performed in each processing unit. The result output unit 7 performs output processing such as displaying the identified character and outputting the identified character code. The identification character may be displayed or printed based on the identified character code.

  The flowchart of FIG. 2 shows an example of the entire character recognition process in this example. Hereinafter, referring to FIG. 2, the character / graphic pattern input from the input processing unit 3 (step S11) is approximated by a broken line by the broken line approximation unit 4 (step S12). From this approximation, the input pattern is expressed as a vector whose elements are differences in length, direction angle, and direction angle between adjacent broken lines when each broken line is regarded as a vector (step S13). Further, the sum of the angle differences of the same sign is obtained from the vector expression of the direction angle difference, and a vector expression named as the winding angle is obtained as one element including the sign. Next, the feature extraction unit 5 extracts features from the polygonal line approximate representation according to the situation (step S14), gives a one-dimensional linear graph representation based on the feature extraction result (step S15), and the graph. Character recognition is performed by matching the expression with a template having an open mask configuration that does not particularly define the start point and end point (step S16), and the character recognition result is output (step S17).

  Here, an example of the details of the character recognition processing in step S16 will be described with reference to the flowchart of FIG. First, it is checked that the arrangement of the node shapes in the input graph representation is the same (step S21). Then, matching of attributes as node points is checked (step S22), and then matching of attributes of edges (edges) between nodes is checked (step S23). Finally, matching of other attributes such as the presence / absence of intersections and inter-node distance relationships is checked (step S24) and identified. If all of these check results match, the recognition result is OK (step S25), and if any one does not match, it is eliminated (step S26).

  The recognition method in the present embodiment basically belongs to the structural analysis method described above, but overcomes the problems so far and provides a basis for a flexible structural analysis method. Therefore, it avoids the problem of symbolization, expresses the structure in an analog manner, and performs flexible and simple matching with the standard. In addition, since structural analysis is performed, it is inevitably possible to describe the subject appropriately, and the correspondence of the cause and effect is clear from the viewpoint of human vision. Accordingly, it is possible to evaluate the shape of an object such as a character, to set a correct refusal range, and to provide a recognition system having a human-like ability.

  Up to this point, it is basically the same as the structural analysis method previously proposed by the inventors of the present application. In addition, this method is very convenient for efficient character recognition in continuous character strings (for example, cursive letters). In the present invention, attention is paid to the fact that natural cutout candidate points are implicitly aligned on the time axis according to the winding angle, and for the input character string, between the candidate points (nodes) and the nodes. Gives a one-dimensional linear graph representation using edge attributes, cuts out at the same time as recognition ("cutout recognition"), and cuts out characters written in isolation without cutting out as preprocessing Recognition is possible in the same way as the recognition method.

  In describing the effect of the present invention, it is easy to understand from the recognition of continuous character strings, so the description starts with the specific method for recognizing numeric strings in FIG. This is “888”. Each of these characters is connected. Therefore, in the conventional character recognition, it is necessary to cut out these characters and send them to the recognition system. This process is called “pre-cut process”. However, this method does not require this “pre-cutout process”, and recognition and cutout are performed simultaneously. This will be described below.

  The image of FIG. 4 is written from the left, but these basic expressions are the length of each connecting polyline, the polyline angle, the external angle composed of the corners of adjacent polyline, and the external angle of the same sign Are a length series, an angle series, an outside angle series, and a winding angle series with the winding angle as an element.

  Therefore, these are converted into one-dimensional graph representations suitable for “cutout recognition” using these as materials. The graph here is the simplest linear graph. FIG. 5 where the first “8” portion of FIG. 4 is intentionally cut out is described below as an example. Here, the start point, the end point, the point that gives a winding angle that is an integral multiple of 90 degrees (± n × 90 degrees long point), the point where the sign of the winding angle changes is a “node”, and the outer angle of that point as a node attribute (△), the winding angle value (Θ) of the winding angle region to which the node belongs, and the maximum value of the absolute value of the outer angle in the polygonal line group between the nodes (maximum | Δ |) To give a graph representation.

  Where <0> [s], <1> [-], <2> [-+], <3> [+], <4> [+], <5> [+], ... <28> [e] is a node number of this graph and a symbol expressing their characteristics. Starts with <0> [s] and ends with <28> [e]. [s] and [e] are start and end symbols, respectively. <1> [−] is the first winding angle of −90 degrees in the outer angle (Δ) series. Although the winding angle is discrete, since it is linearly interpolated, it can generally be obtained in analog form as an α-degree long point. <1> The “-” in [-] represents the sign of the winding angle, counterclockwise.

<Graphic representation of FIG. 5>
<0> [s] 0 (326, 121), Θ = -172.83
↓
↓ Number of broken lines (0 to 4) = 4, Length = 0.07, Maximum | Δ | = | -24.52 |
↓
<1> [−] 4 (156, 121), Θ = -172.83, Δ = -33.26
↓
↓ Number of broken lines (4-6) = 2, Length = 0.09, Maximum | Δ | = | -72.64 |
↓
<2> [− +] 6 (347, 279), Θ (-) = -172.83, Θ (+) = 304.42, Δ (-) = -72.64, Δ (+) = 21.27
↓
↓ Number of broken lines (6 to 8) = 2, Length = 0.02, Maximum | Δ | = | 45.06 |
↓
<3> [+] 8 (365, 346), Θ = 304.42, Δ = 49.31
↓
↓ Number of broken lines (8 to 12) = 4, Length = 0.07, Max | Δ | = | 30.07 |
↓
<4> [+] 12 (195, 329), Θ = 304.42, Δ = 20.39
↓
↓ Number of broken lines (12 to 15) = 3, Length = 0.03, Max | Δ | = | 35.79 |
↓
<5> [+] 15 (194, 258), Θ = 304.42, Δ = 38.45
↓
↓ Number of broken lines (15 to 16) = 1, Length = 0.05
↓
<6> [+-] 16 (330, 205), Θ (-) = 304.42, Θ (+) = -56.33, Δ (-) = 38.45, Δ (+) = -56.33
・
・
・
<28> [e]

  <1> [-], that is, "-90 degree point" is just beyond the fifth vertex. Therefore, as the attribute of the point of this node, the winding angle sequence (Θ = −172.83) to which this node belongs and the outer angle (Δ) value of the closest vertex in time (in this case, the fifth vertex), Δ = − Give 33.26. In addition, <2> [− +], which indicates the boundary of the winding angle, is important. In other words, this is a point where the winding angle (−) is changed to the winding angle (+). The boundary of the wrapping angle is not a point, but shares one line. The tip of the shared broken line in the plus direction of the time axis is a node. The attributes of this point are Θ (−) = − 172.83, Θ (+) = 304.42, Δ (−) = − 72.64, Δ (+) = 21.17. That is, the winding angle Θ (-) = -172.83 before the time change, the winding angle Θ (+) = 304.42 after the time change, and the outer angle (Δ) values at both ends of the common broken line, Δ ( -) = -72.64, Δ (+) = 21.27.

  <2> After [− +], the polygonal line sequence enters the winding angle (+) region. The first is <3> [+], which is a 90 degree long point. The attributes are Θ = 304.42, Δ = 49.31. <4> [+] is a 180 degree long point. <5> [+] is a 270 degree long point. In other words, at this node <5>, the winding angle exceeds 270. Next is <6> [+-], which is the point where the winding angle changes from + to-.

  Next, as the attributes of the edges connecting these nodes, the number of broken lines (0 to 4) = 4, length = 0.07, maximum | Δ | = | -24.52 | , Take the maximum value of | Δ |. These properties are rotation invariant.

  The above is the graph representation of the input, and matching with the template based on the mask configuration is performed. An example of the mask configuration in the case of “8” is shown below.

<Example of “8” Mask>
Condition 1: * =-+ (first key node)
Condition 2: −200 <Θ (* −) <− 100 & 200 <Θ (* +) <360
(Note: “Θ (* −)” is the first “−” winding angle, “Θ (* +)” is the succeeding “+” winding angle)
Condition 3: -100 <Δ (*-) <-20 & 10 <Δ (* +) <100
(Note: “Δ (*-)” is the outer angle value of the “−” winding angle boundary, and “Δ (* +)” is the outer angle value of the succeeding “+” winding angle boundary. -Side and + side outside angle values in
Condition 4: (*-, * +), Cross: nxm: n∈ (*-) to (* +)
(Note: There is an intersection Cross between both ends of the boundary line)
Condition 5: * + 1 = + − (second key node)
Condition 6: 200 <Θ (* + 1+) <360 & -200 <Θ (* + 1-) <-10
(Note: “/ Θ (* + 1+)” is the + winding angle, “Θ (* + 1-)” is the next − winding angle condition 7: 10 <Δ (* + 1+) <100 & -100 < Δ (* + 1−) <-20
(Note: “Δ (* + 1+)” is the outside angle value at the + winding angle boundary, and “Δ (* + 1−)” is the outside angle value at the next − winding angle boundary.)
Condition 8: (* + 1-, * + 1 +), Cross: nxm: m∈ (* + 1+) to (* + 1−)
(Note: Cross exists between both ends of the boundary line)

Character [8] = condition 1 & condition 2 & condition 3 & condition 4 & condition 5 & condition 6 & condition 7 & condition 8.

  This mask is very simple, with two nodes (“* = − +” and “* + 1 = + −”), with only a boundary from − to + and a boundary from + to −, respectively. “± n × 90 degrees long point” is not used. Here, the meaning of “*” in “* = − +” indicates that this node is the key to this mask, and that there is always a node whose winding angle changes from “minus” to “plus”. ing. “* + 1 = + −” is the next necessary node, and on the other hand, the node whose winding angle changes from “plus” to “minus” is raised.

  In addition, the attributes of the first key node in conditions 2 and 3 and the second key node in conditions 6 and 7 are the winding angle value and the outer angle Δ at both ends of the boundary line segment. Gives detailed conditions and tightens the structure. In addition, intersection information is used as an important edge characteristic. That is condition 4 and condition 8. m∈ (* + 1+) to (* + 1−) requires that an intersection exists between the boundary lines at the next winding angle change point. The intersection is expressed as Cross: nxm. These n and m are numbers of intersecting broken lines, and n∈ (* −) to (* +) indicate that the intersecting broken lines coincide with the winding angle boundary line segment. Of course, the intersection is a rotation invariant feature.

  This is a very simple mask. This is combined with the input graph expression to search for a node range where this type fits, and if it exists, it is assumed that there is “8” there. Note that the [s] and [e] nodes are not used in the above mask. Therefore, both ends are open. Therefore, anyway, if there is a place (range) that applies to this mask in the flow of time, it means that there is “8”. It is unnecessary.

  In fact, with this mask, three “8” s are recognized in FIG. In FIG. 6, connected “8” s having completely different inclinations are recognized. In FIG. 7, two “8” s having significantly different sizes are connected, and these are correctly recognized as “8”. As a matter of course, it is possible to recognize a single character by combining a similar mask and an input graph expression regardless of the connected character.

  Here, some points to note are described. That is, the mask described above does not include the property of distance or length. Therefore, regardless of the length of the connection, the shape was fitted everywhere and the cut-out was recognized. However, on the other hand, for example, as in the case of FIG. 7, two pieces of information of “8” are lost, and there is a question that the difference in size as seen by humans is not known. However, this is actually easy to find.

  This input representation has the position coordinates of each node. Therefore, for example, in “8”, a node that is returned 180 degrees from the “*” node that is the mask key, in FIG. 7, this is almost an <s> node, and a node that is advanced 180 degrees from the * + 1 node. Find the Euclidean distance between positions.

  Specifically, it is the square root of (286-38) 2+ (562-449) 2 for the larger “8” = 273, and the square root of (174-55) 2 + (: 625-611) 2 for the smaller “8”. = 120, the ratio of the two is about 0.43, which gives results that match intuition. Although slightly complicated, in the case of FIG. 2 as well, the vector of the major axis and the minor axis of “8” can be obtained, and the tilt information can also be obtained. In this way, the coordinate values of the node can be effectively used to obtain a geometric metric such as length, corner and direction.

<Use of ± n × 90 degree node>
In the example of the mask configuration of “8” described above, in order to explain the essence of the present invention, the case of a simple configuration in which ± n × 90 degree nodes are not added as conditions is described, but here, ± n × 90 The effectiveness of degree nodes will be described with an example. This is particularly effective for round shapes, typically “circles”.

This is a symbol often used as “◯” in practice. The challenge is to create a mask that retains the shape of this circle and that withstands significant deformation.
This mask is made as follows.
<Example of “○” mask>
Condition 1: * = +90
Condition 2: 350 <Θ (*) <600 & 0 <Δ (*) <95
Condition 3: (*. * + 1); 0.1 <length <0.35 & 0 ≤ | Δ | <95
Condition 4: * + 1 = +180
Condition 5: Θ (* + 1) = Θ (*) & 0 <Δ (*) <95
Condition 6: (* + 1. * + 2); 0.1 <length <0.35 & 0 ≦ | Δ | <95
Condition 7: * + 2 = +270
Condition 8: Θ (* + 2) = Θ (*) & 0 <Δ (*) <95
Condition 9: (* + 2. * + 3); 0.1 <length <0.35 & 0 ≦ | Δ | <95
Condition 10: * + 2 = +360
Condition 11: Θ (* + 3) = Θ (*) & 0 <Δ (*) <95
Condition 12: (* + 3. * + 4); 0.0 <length <0.35 & 0 ≦ | Δ | <95

Symbol “◯” = condition 1 & condition 2 & condition 3 & condition 4 & condition 5 & condition 6 & condition 7 & condition 8 & condition 9 & condition 10 & condition 11 & condition 12

  There are four nodes in this mask (“* = + 90”, “* + 1 = + 180”, “* + 2 = + 270”, “* + 3 = + 360”), and the existence of ± n × 90 degree nodes (this case) In the case of + n × 90 degrees node), this shape is roughly defined. That is, in this case, n is 1 to 4, and the winding angle is requested to be wound up to 360 degrees in units of 90 degrees. On the other hand, as an attribute of the node, in the vicinity of the long point of + n × 90 degrees, a sudden increase in angle is suppressed by 0 <Δ (*) <95. In addition, as a side attribute, the length between nodes is requested to be within a predetermined range. This is “0.1 <length <0.35”. It also stipulates that there is no sudden increase in corners during that time. This is “0 <| Δ | <95”.

  This mask is sufficiently robust against noise and is open at both ends. Therefore, as can be seen in FIG. 8, “◯” is recognized even when there is considerable noise outside. As described above, if each length is normalized by the relative length, for example, the Euclidean distance between the 180-degree long point and the 360-degree long point, this can be applied to any number of connected character sequences. Applicable.

<Uneven node expression>
Here, a graph representation in the case of character recognition that does not require normal rotation invariance will be described. In such a case, it is more intuitive and easy to understand that it is more difficult to use irregularities as nodes based on horizontal and vertical orthogonal coordinate systems than to use ± n × 90 degree nodes. Please see the explanatory diagram of Fig. 9 as an image. Here, ○ is a node and → with an arrow is a side.

  Here, the upper and lower irregularities are denoted as ∪ and ∩, and the irregularities viewed from the left and right are denoted as ⊃ and ⊂. In addition, there are actually upper and lower unevenness and left and right unevenness simultaneously. For example, when it is pointed to the upper right, it is indicated as ∩ + ⊃ where ∩ and ⊃ overlap.

  The node attributes are as described above, but the average direction angle of the line group between the nodes is added as the edge attribute. This is a very important feature in the uneven node representation. In addition, there is a directional dispersion of a broken line group indicating the degree of bending. Next, what is important is the distance ratio between nodes. For example, referring to FIG. 9, the start point must be somewhat higher than the end point. When these are calculated mechanically, they become a considerable number, but in practice, only the main points need to be suppressed, and not so many. Here, for example, a value obtained by subtracting the start point <s> node Y-axis value from the Y-axis value of the node <5> and dividing by the ratio of the length of the whole character is obtained.

Hereinafter, an example of the mask of “8” by this concavo-convex graph expression is shown.
<Example of “8” Mask>
Condition 1: * -1 = s / any
Condition 2: (*, * -1), 110 <Weighted average angle <170
Condition 3: * = ∩
Condition 4: -420 <Θ (*) <-160
Condition 5: -110 <Δ (*) <-10
Condition 6: * + 1 = ⊂
Condition 7: Θ (* + 1) = Θ (*)
Condition 8: -110 <Δ (* + 1) <-10
Condition 9: (* + 1, * + 2), -80 <Weighted average angle <-30
Condition 10: (* + 1, * + 2), length> 0.15
Condition 11: * + 2 = ⊃
Condition 12: 250 <Θ (* + 2) <500
Condition 13: 20 <Δ (* + 2) <100
Condition 14: * + 3 = ∪
Condition 15: Θ (* + 3) = Θ (* + 2)
Condition 16: 10 <Δ (* + 3) <120
Condition 17: * + 4 = ⊂
Condition 18: Θ (* + 4) = Θ (* + 2)
Condition 19: 10 <Δ (* + 4) <100
Condition 20: (* + 4, * + 5), 20 <Weighted average angle <75
Condition 21: (* + 4, * + 5), Cross: nxm: n∈ (* + 1) to (* + 2), m∈ (* + 4) to (* + 5)
Condition 22: * + 5 = e / any
Condition 23: ((* + 1, * + 2) CrossY value-(*) Y value) / height 0.20 <Y <0.70
// The intersection is located almost in the center when viewed in height. Pay attention to the sign

Character “8” = condition 1 & condition 2 & condition 3 & condition 4 & condition 4 & condition 6 & condition 7 & condition 8 & condition 9 & condition 10 & condition 11 & condition 13 & condition 14 & condition 15 & condition 16 & condition 17 & condition 18 & condition 19 & condition 20 & condition 21 & condition 22 & condition 23

  In this case, the inter-node distance ratio is not particularly required. Instead, condition 23, which is an edge attribute for the intersection, plays that role. Thus, the inter-node distance ratio is actually much less than the number produced by the combination. This is because it is implicitly limited by the limitation of the winding angle, outer angle, and polygonal line angle. Also, the length limitation in the upper mask only limits the length of the polyline that goes down from left to right. In fact, for this reason, it is a very strong mask against edge noise.

  This is not a cut-out recognition mask for continuous character sequences. This is because of this length limitation. In this respect, if this length is normalized with the Y-axis value between node * = ∩ and node * + 3 = ∪, a cut-out recognition mask for a continuous character sequence is obtained.

  With this method, even when extreme noise occurs at the edge or when extreme deformation occurs, if it has the shape of “8” as the core, correct recognition is possible regardless of the surrounding circumstances. Done.

<Inclusive relation of character shape>
The cut-out recognition methods described above inevitably have the potential to give multiple answers.
A specific example of this is shown in FIG. FIG. 10 is written with the intention of “8”, but “6” is hidden in this figure. In other words, the character shape “6” is “cut out and recognized” from the original figure. This is therefore an inevitable result.

  Therefore, an intended mechanism for correctly outputting “8” is necessary. Intuitively speaking, this means giving priority to complex shapes, but quantitatively speaking accurately, it matches the length of the polygonal line that matches the mask “6” and the mask “8”. If the lengths of the broken lines are compared, the latter is naturally longer.

  From the following example, “8” is longer by 0.27 than “6”. Therefore, “6” is quantitatively buried in the image of “8”, and “8” has a matching portion 0.27 longer than “6”, and is determined to be “8”. In addition, it is possible to consider matching measures such as the number of matching nodes and the number of broken lines.

  The handwritten character recognition according to the present invention is not limited to the processing configuration shown in FIG. 1 as described in the beginning of the description of the embodiment, and a configuration in which substantially the same handwritten character recognition is performed. If so, it is possible to perform recognition processing with various apparatus and system configurations. For example, the handwritten character recognition of the present invention may be converted into a program (software) and mounted on a general-purpose personal computer device. The handwritten character recognition program can be stored in various storage media and distributed.

Although online characters are targeted here, character recognition may also be performed for offline characters by appropriate thinning or contour tracking.
Furthermore, in the above-described embodiment, the case where numbers are mainly recognized has been described as an example. However, handwritten character recognition according to the present invention is basically applicable to characters and symbols in any language.

It is a block diagram which shows the example of a system by one embodiment of this invention. It is a flowchart which shows the example of a process of the whole character recognition by one embodiment of this invention. It is a flowchart which shows the detailed example of the character recognition process by one embodiment of this invention. It is explanatory drawing which shows the example of the broken line approximation by one embodiment of this invention. It is explanatory drawing which shows the example of the node by one embodiment of this invention. It is explanatory drawing which shows the example of the broken line approximation by one embodiment of this invention. It is explanatory drawing which shows the example of the broken line approximation by one embodiment of this invention. It is explanatory drawing which shows the example of the broken line approximation by one embodiment of this invention. It is explanatory drawing which shows the example of the relationship between the node and edge by one embodiment of this invention. It is explanatory drawing which shows the example of determination by one embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Paper, 1a ... Brush stroke, 2 ... Pen, 3 ... Input processing part, 4 ... Broken line approximation part, 5 ... Pre-processing part, 6 ... Feature extraction part, 7 ... Identification part, 8 ... Identification result output part

Claims (8)

In the handwritten character recognition method for recognizing online handwritten characters,
The input handwritten character string is captured by parameter expression for each image, and a polygonal line approximation is performed for each image,
Each polygonal line approximated by the polygonal line is determined as a vector from the start point to the end point, and the angle between the reference axis and each polygonal line is obtained as a polygonal line angle series.
Obtain the outer angle series of each vertex of the obtained polyline,
A sum of outer angles of the same sign in which the same plus or minus same sign of the outer angle series continues, and a winding angle series,
Based on the feature extraction by each of the obtained series, the start point, the end point, the point that gives a winding angle that is an integer multiple of 90 degrees, and the point at which the winding angle changes are nodes, and the attribute between the node and the node Given the attribute as an edge of
Matching with a template with an open mask configuration that does not specify the start point and end point, it is flexible, even for characters that maintain the normal vertical and horizontal relationship, and when rotation invariance is required, edge noise and A character recognition method characterized by robust recognition against deformation. The character recognition method according to claim 1,
For recognition of normal character forms that do not require rotation invariance, the top and bottom unevenness and the left and right unevenness are used as nodes, and a graph representation is given by the attributes as points of the nodes and the attributes as edges between the nodes. A character recognition method, wherein character recognition is performed by matching a template with an open mask configuration that does not particularly define an end point. The character recognition method according to claim 1 or 2 ,
A character recognition method characterized in that, when a plurality of characters or symbols are in an inclusive relationship, priority is given to a character or symbol having a long matching portion quantitatively. The character recognition method according to any one of claims 1 to 3 ,
A character recognition method characterized by performing recognition without performing a special process of cutting out a continuous character string. In a handwritten character recognition system that recognizes online handwritten characters,
An input means for entering handwritten characters online;
The input handwritten character string is captured by the parameter expression for each image, and the broken line approximation means for performing the broken line approximation for each image,
Each polygonal line approximated by the polygonal line approximation means is used as a vector from the start point to the end point, and the angle formed between the reference axis and each polygonal line is obtained as a polygonal line angle series, and the outer angle series of each vertex of the obtained polygonal line is obtained. A processing means for determining a sum of outer angles of the same sign in which the same plus or minus same sign of the outer angle series continues as a winding angle series;
Based on the feature extraction by each series obtained by the processing means, the start point, the end point, the point that gives a winding angle that is an integer multiple of 90 degrees, and the point at which the winding angle changes are nodes, and the attributes as points of that node Also, a graph expression is given by the attribute as an edge between the nodes, and a character having an ordinary vertical and horizontal relationship is also obtained by matching with a template with an open mask configuration that does not particularly define the start point and the end point. A character recognition system comprising a step of performing robust recognition that is flexible and resistant to edge noise and deformation even when rotation invariance is required. In a handwritten character recognition program for causing a computer to recognize online handwritten characters,
Capturing the input handwritten character string with a parameter expression for each image and performing a polygonal line approximation for each image;
Obtaining each polygonal line approximated by the polygonal line as a vector from the start point to the end point, and determining an angle formed between the reference axis and each polygonal line as a polygonal line series;
Obtaining an outer angle series of each vertex of the obtained polygonal line;
A step of setting the sum of the outer angles of the same sign that the same plus or minus same sign of the outer angle series continues as a winding angle series;
Based on the feature extraction by each of the obtained series, the start point, the end point, the point that gives a winding angle that is an integer multiple of 90 degrees, and the point at which the winding angle changes are nodes, and the attribute between the node and the node A step of giving a graph representation by attributes as edges of
Matching with a template with an open mask configuration that does not define the start and end points, it is flexible, even for characters that maintain the normal vertical and horizontal relations, and when rotation invariance is required, edge noise and The step of performing robust recognition that is resistant to deformation ,
A handwritten character recognition program to be executed by a computer . In the handwritten character recognition program according to claim 6,
When multiple characters or symbols are in an inclusive relationship, priority is given to the characters or symbols with long matching parts quantitatively.
Handwritten character recognition program. In a storage medium capable of online handwritten character recognition by mounting a stored program on a predetermined arithmetic processing device,
As a program stored in a storage medium,
Capturing the input handwritten character string with a parameter expression for each image and performing a polygonal line approximation for each image;
Obtaining each polygonal line approximated by the polygonal line as a vector from the start point to the end point, and determining an angle formed between the reference axis and each polygonal line as a polygonal line series;
Obtaining an outer angle series of each vertex of the obtained polygonal line;
A step of setting the sum of the outer angles of the same sign that the same plus or minus same sign of the outer angle series continues as a winding angle series;
Based on the feature extraction by each of the obtained series, the start point, the end point, the point that gives a winding angle that is an integer multiple of 90 degrees, and the point at which the winding angle changes are nodes, and the attribute between the node and the node A step of giving a graph representation by attributes as edges of
Matching with a template with an open mask configuration that does not define the start and end points, it is flexible, even for characters that maintain the normal vertical and horizontal relations, and when rotation invariance is required, edge noise and And a step of performing robust recognition resistant to deformation.

Images