JPS6129980A - On-line recognizer of hand-written figure - Google Patents

Abstract

PURPOSE:To obtain recognition independent of the number of strokes at the time of inputting a figure by dividing hand-written strokes into plural line segments to recognize them with the aid of features obtained from a straight line for connecting two arbitrary points among plural sampled points of hand-written strokes. CONSTITUTION:When a user draws a hand-written figure on a tablet 100, it samples coordinates of hand-written strokes by a certain period, and transmits coordinate signals to a figure recognizer 200. It fetches the sampled coordinate signals by stroke, and afterward it repeats the recognition processing by stroke. Graphic codes of the recognized result are transmitted to a figure display device 300 together with the size of the inputted figure and a parameter for showing an angle. The figure display device 300 develops graphic codes of the recognized result, etc., in a bit map memory and displays them on a CRT400 by a video signal. Hard copy of the display recognition result can be obtained by a printer 500 with the aid of a bit map signal, and moreover a graphic code signal can be stored in an external memory device 600.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Application of the Invention] The present invention relates to a handwritten figure recognition method, and particularly to a figure data input method suitable for an online handwritten document creation system.

Publicly known example closest to the invention 0 "Research is actively progressing on online recognition of handwritten characters" Nikkei Electronics 1973, 5, 7 P45P59 0 Murase et al. "Online recognition of handwritten flowcharts using candidate lattice method" Transactions of the Institute of Electronics and Communication Engineers '84/3 vol. J67-D3
[Background of the Invention] Due to the spread of large computers and the increasing functionality of information processing terminals, the user base of computers is rapidly expanding. Along with this, handwriting input devices have attracted attention as one of input means to computers.

Information to be input is broadly classified into characters and figures, and conventional examples of these will be explained below.

[Conventional Example 1] Handwritten character online recognition in general is introduced in Nikkei Electronics, No. 1973.5-7, pages 46 to 59. Among these methods, we will describe a method for recognizing handwriting by representing it as a series of short line segments and comparing it with the characteristics of a dictionary.

FIG. 11 shows handwritten handwriting smoothing processing.

Assuming that the input handwriting is Po=Ps, first, the point Po where the pen is lowered is set as the first sample point S1, a circle with a radius of 5 is drawn around this S1, and the next point P+ outside this circle is obtained. And P. The point that divides the area between 21 and 21 at a ratio of 3:1 is defined as a new sample point S+.

The above process is continued until the pen is raised to obtain sample points S+ to S3. At this time, line segment So S+ and line segment 8
1 If the angle of 82 is less than a predetermined value, one line segment S extends from the sample point So to 82. Let it be S2.

This process is repeated to approximate handwritten strokes using polygonal lines (segments) (FIG. 12).

Segment data that approximates handwriting is recognized as a specific character using a character recognition tree as shown in FIG.

This example uses a method called the direction code method, in which the points So to S3 sampled in the smoothing process are successively decomposed into a plurality of line segments.

As a result, it is effective for line segmentation of short handwriting (for example, strokes of kanji, etc.) because there is less camera shake during input, but it absorbs camera shake that occurs with long handwriting when inputting shapes. It's not enough to do that. In other words, in order to absorb camera shake, it was necessary to repeat line segmentation multiple times.

In addition, in this example, since the number of handwriting is used as an element of character classification, it was necessary to devise a new identification method for writing.

[Conventional Example 2] Regarding other conventional technologies, Journal of the Institute of Electronics and Communication Engineers '84/3
VOn, J67 D A The figure recognition method described in "Online Handwritten Line Figure Recognition Using Candidate Lattice Method Introducing the A3° Connection Rule" Murase et al. (Electric Corporation Musashino Tekisho) will be explained below.

The above figure recognition method is called the "candidate lattice method."

FIG. 14 shows the flowchart of the algorithm of the above method.

[Processing 1 Candidate figure extraction] Correlate the start and end point coordinates P of the input stroke with the feature points of the symbol G (targeting all dictionary figures), and extract the symbol G.
A plurality of combinations of single-stroke stroke sequences that pass through all of the feature points Pi (i=1...n, n number of feature points of Nishinpol G) are generated. At this time, if the input stroke string and the generated single-stroke stroke string do not conflict,
Let this symbol G be a candidate figure Gc. This is called "extraction of candidate shapes Gc by brush strokes" and is a method for broadly classifying dictionary shapes while absorbing the deformation of shapes by handwriting.Hereinafter, the above-mentioned brush strokes will be referred to as candidate strokes. Say.

However, this method does not take into consideration the detection of coincidence between the feature points and strokes when the input figure is significantly tilted.

[Process 2 Calculation of dissimilarity] This candidate lattice method uses dynamic programming (DP) matching, which evaluates not only the Euclidean distance between feature points and strokes, but also the tangential direction of the stroke portion corresponding to each feature point. ing. At this time,
The degree of dissimilarity d2 is d2=min[Σt(xm−x'u(m
))2+(ym Y'u(m))"um=1 +α・h(m, u)))] ・・・・・・
(1) Here, (xm + Ym) is the coordinate series of the input stroke (x / (2) + Y'u (e) is the coordinate series of the candidate stroke u (m) = u (1) = 1: starting point 11 ( M)=M
: End point The first and second terms in parentheses on the right side of equation (1) above indicate the Euclidean distance between the input stroke end point and the feature point of the dictionary figure, and the third term is the feature point (X'm + Y'm ) indicates the tangential difference between the input stroke and the candidate stroke.

[Process 3 Measures for candidate lattice] The dissimilarity calculation in Process 2 is performed for all the figures extracted in Process 1, and the results are created in a table as shown in FIG. Therefore, the optimal graphic sequence is selected from among all possible graphic sequences from the above table, the one having the smallest sum of dissimilarities.

By performing processing 1-3 above, it is possible to recognize handwritten figures without depending on the number of strokes, stroke order, or segmentation of the figure. No consideration is given to the means of recognizing time.

[Purpose of the invention]

An object of the present invention is to provide a method for automatically segmenting and recognizing figures when a continuation of all figures drawn in freehand and other figures with two or more connecting lines is drawn. There is a particular thing.

[Summary of the invention]

When drawing figures (flowchart figures, etc.) that are often used in general documents, it is often the case that both the figure (or a part thereof) and the connecting line are drawn with a brush.

On the other hand, in conventional figure recognition methods, the range that can be drawn with one stroke is limited to one figure. Therefore, to cut out the figure part from a single stroke that exceeds the figure, as described in the present invention, the operation of extracting a part of the single stroke is performed for each handwriting as shown in Figure 2, and the combination of these parts is This becomes possible by performing recognition processing.

[Embodiments of the invention]

FIG. 1 shows an online handwritten figure recognition device using the present invention.

The user draws a handwritten figure on the tablet 100. At this time, the tablet 100 samples the coordinates of the handwritten handwriting at regular intervals and sends a coordinate signal to the figure recognition device 200.

The figure recognition device 200 incorporates the sampled coordinate signals for each stroke, and thereafter repeats the recognition process for each elutriation stroke. As a recognition result, a graphic code is sent to the graphic display device 300 along with parameters indicating the size and angle of the input graphic.

The graphic display device 300 expands the graphic code etc. resulting from the above recognition into a pit map memory, and outputs the graphic code etc. resulting from the above recognition into a pit map memory, and displays it on a CRT using a video signal.
400. The recognition result displayed on the CRT 400 is transmitted to the graphic display device 30 using a bitmap signal.
By using the printer 500 connected to the printer 500, it is possible to take the /...-tocobee, and it is also possible to store the graphic code signal in the external storage device 606 connected to the printer 500.

Since the processing of these parts can be executed in real time, this device can function as an online handwritten figure recognition device.

When inputting a figure from the tablet 100, there are no restrictions on the "number of strokes, order of strokes,""rotation," etc., so in the present invention, the constituent unit of a figure is a line segment, and the shape of the figure is It is constructed on the premise that the connection order of line segments is determined again by the difference.

Here, °number of strokes ° is 1 from pen down to bent up.
When considered as a stroke, the number of strokes required to draw one shape, °stroke order ° is the order of strokes when drawing a shape, “rotation ° is the angle of the input shape with respect to the standard shape, and ° separation” is the input number. This is an indication of the range corresponding to the figure being drilled.

The present invention has two main features (Fig. 2).

That is, a line segment segment means performs a matching process with dictionary figures, and a one-dimensional line segment dynamic program means cuts out figures from input strokes.

The line segment method further includes line segment segmentation, which divides the stroke into basic units for processing described later, code rat pink, which generates a code list by tracing the shape as if it were wrapped around it, and a universal angle for rotation of the input shape. It consists of three steps: calculating the degree of difference due to change.

By this means, the above-mentioned restrictions on "number of strokes or rotation" are relaxed, and on the other hand, line segment elements are reconstructed by the -dimensional line segment DP means, and restrictions on "separation" are relaxed.

(1) Outline of processing procedure Manually drawn handwritten figures are processed according to the illustrated flow (FIG. 2). The main processing procedure of the present invention will be explained next.

■ Converting strokes into line segment elements The handwritten figure input from the tablet must be defined as a set of sample points for each stroke.

For this purpose, the direction from sample point S+ to S2 is quantized using the direction code shown in FIG. 3, and line segments are divided into elements.

(2) Code wrapping Based on the connection information between the end points of the line segment data extracted in (2) above, a code list L4 (FIG. 7) in which direction codes corresponding to each direction code are arranged is generated.

At this time, the line segment data will be traced in order so as to wrap around the input figure.

■ Calculating the degree of dissimilarity based on angle changes From the dictionary figure (roughly classified by the number of line segments) that has been made into elements as described above, a code list (referred to as L5) is generated in the above ■, and each adjacent direction of the L4 is generated. The code difference is compared with that of L5 (the following equation).

f, gives the unit dissimilarity.

[1=ShiL4(i)-1,4(i-1) 1-(L5(
i)-L5 (i-1111However,'”'2+'3+
"'+ n n: Number of line segments in L5... (1) Define the average value of f as the degree of difference F of the handwritten figure. If F = 0, the input figure and the dictionary Everyone is sad when the shapes match.

The F value often takes a value of O to 10, and the minimum value of the F value obtained from the input figure and various dictionary figures is used for the recognition figure.

■ Reconstruction of line segment elements If there is a continuation between the shape and the connection line in the input shape,
The above process cannot be performed. Therefore, after converting handwritten handwriting into line segment elements, it is necessary to cut out combinations of line segment elements corresponding to figures from among the line segment elements. This extraction operation directly affects the number of loops of the entire recognition process, that is, the processing time. Therefore, it is necessary to perform the cutting operation efficiently.

The cutting operation is generally performed using the "total drawing method" that combines all line segment elements, but since the number of operations to cut out one figure tends to be large, here we will focus on the characteristics of handwritten figure input. We use the ``-dimensional line segment dynamic programming (DP) method 2 that we focused on.As a result, it is possible to cut out figures with a fraction of the number of operations compared to the ``brute force method.'' The above recognition process is repeated for the line segment elements that have been identified.

(2) Description of processing procedure The basic figure recognition algorithm described in (1) above will be specifically explained below using an example.

Converting Strokes to Line Elements (FIG. 4) The sample point coordinate data S1 of the handwritten figure input from the tablet 100 is input from the pen down.

The entire stroke up to the start-up is included in this process.

As shown in figure (a), sample point coordinate data S! obtained from handwritten strokes! The area from the starting point SP+o to the ending point SPn is defined as the unidentified area (UA)', and the starting point SP+.

is the "unidentified area starting point (UAS)", and the end point SPn is the "unidentified area ending point (UAE)".

Next, as shown in EQ (b), the area A of the area sandwiched between the handwritten handwriting S and the straight line connecting its starting point and ending point is determined. This area A can be easily obtained using the trapezoidal method in geometry.

Next, if the calculated area A is smaller than a preset threshold value Ath, this handwritten handwriting S is regarded as a straight line, and the process moves to line segment registration processing.

As shown in the same figure (C), in the above (8), if the area A is smaller than the threshold value Aih, the end point is moved to the midpoint between the UAS point and the UAE point, the area is calculated again, and the area A' get.

Here, the magnitude relationship between the area A' and the threshold value Ath is checked again, and if A'< Ath, the current end point is determined as a new UAS point as shown in FIG.
If t h, the current end point is set as a new UAE point and the process moves to calculation of the area A'', as shown in FIG. 4(e).

The above process is performed when there is no more UA, that is, when UAE point = UAS
This is done until a point is reached. Thereafter, the process moves to line segment registration processing, regarding the starting point to the ending point as one straight line.

At the same time, the end point is reset to the start point and the UAS point, and the final point of the handwritten handwriting S is reset to the end point and the UAE point.

Coordinates SPj, SPk of the start point and end point determined to be a line segment (L) through the above processing, up/dOWn of the pen at each point
(F) State 8Tj, STk, angle θ from the start point to the end point
Lk etc. as shown in the same figure (f)) Seg-1
Register on ist. These processes are continued until all handwritten strokes S are registered.

To perform the code wrapping process, it is necessary to know the connections and relationships between the end points of each line segment element. Therefore, normalization of input graphics and creation of the connection relationship list will be explained.

FIG. 5 shows how the input figure (triangle) is normalized. As shown in (a) of the same figure, the normalized size (t
n). Therefore, as shown in Figure (b), the values of the starting and ending points of the line segment L
/lI times to create a normalized line segment list L2.

Next, creation of connection information between line segment element end points will be explained using FIG. 6. Figure (a) shows the input figure (triangle) after normalization.
It is. As shown in Figure (b), the normalized size (z
10% of the line segment element endpoints is set as the "permissible connection distance" between the endpoints of line segment elements.If the distance between the endpoints of each line segment is smaller than the permissible connection distance, the two endpoints are considered to be connected. This information is written to the connection list (L3) shown in FIG.

In this example, line segment element 1 has two pieces of connection information: ``The starting point is connected to the starting point of line segment element 3°,'' and ``The end point is connected to the starting point of line segment element 2.'' .

The manner in which code wrapping is performed using the line segment connection list L3 created in this manner and the normalized line segment list L2 will be described with reference to FIG.

The starting point for wrapping each line segment element may be any line segment element, but here the line segment elements are traced clockwise from the leftmost end point.

■In the same figure (a), among the line segment elements that make up the input figure,
Find the one whose endpoint is farthest to the left.

■ Considering the end point found in the above ■ as the new starting point, the line segment element that is most oriented in the X direction from among the line segment element that includes this end point and other line segment elements connected to that end point ( In the figure, line segment element 1) is selected, and the direction code (-
4, FIG. 7) is registered in the code list L4.

■Line segment connection list (Fig. 6) Since we can see that the starting point of line segment element 1 is connected to line segment element 3, as shown in (b), the direction code ( =28)
is registered in code list L4.

■Create code list L4 in the same manner as in ■below.

If some of the input line segment elements have not been used for wrapping even once, the process from (2) is repeated.

If not, this process ends.

As a result of this processing, a clockwise code list (L4) is generated with respect to the input figure as shown in FIG. 2(d).

Using the code list L4 obtained in the code wrapping process for calculating the degree of difference based on angle changes, matching process with dictionary figures is performed according to the following procedure.

First, three angular difference data shown in the following equations are obtained from codelith) L4.

θa = L 4 (2) −L 4 (1) θb =
L 4 (8) −L 4 (2) θc = L 4
(4) L 4 (8) Next, a method for calculating the degree of difference between the input figure and the dictionary figure will be explained (FIG. 8).

The angular difference data θa to θC of the input figure shown in FIG.
′, the sum f1 + f2 + . . . of the absolute values of the differences between the corresponding data shown in FIG. In this figure, CG') IF - As shown, f for the input figure.

A dictionary figure with the minimum value is found and output as a recognition figure ((d) in the same figure).

In the example in this figure, θa=24. θb--12. θc=-12°θ
,=23. θ2 = -12, θ3 = -11°0!
'=20102'-1810. If '=-12, then f+=2. f2=8, and by averaging this fl+f2 by the number of line segment elements, fl73=0.67, f2/3=2[7, and the recognized @ figure in this case becomes figure A.

Reconstruction of line segment elements From converting the stroke into line segment elements, the calculation size of the above-mentioned dissimilarity is -(
The matters explained in FIG. 2 (corresponding to the line segment means) are based on the premise that graphic parts are independently extracted from a plurality of handwritings.

However, in general, when inputting a flowchart or the like by hand, it often happens that the connecting line and the figure are continuations of handwriting, as shown in FIG.

Therefore, cutting out a figure by the ``-dimensional line segment DP method'', which is an application of the dynamic progeraminking (DP) method, will be explained with reference to the drawings.

When inputting handwritten figures, the input order for this figure cutting is connecting line → figure, figure → connecting line, or connecting line → ■
This is done by focusing on the fact that it can be divided into one of three cases: shape → connecting line. By removing the line segment element input first and the line segment element input last among the plurality of input line segment elements, a graphic portion can be efficiently cut out.

The above processing will be specifically explained using FIG. 9.

FIG. 4(a) shows the input figure after it has been converted into line segment elements.

In this example, line segment element 1 needs to be removed.

FIG. 3B shows the process of reconstructing line segment elements using the -dimensional line segment DP method.

First, line segment elements 1 to 4 are input to this process.

In the first processing, only the line segment element input last is increased or removed.

In the second time, we remove the line segment element that was input first,
Extraction was successful.

In the third and subsequent processes, (1) the number of line segment elements to be removed is 2, indicating that no matter how the removal is performed, the figure will not be cut out.

In this way, one or more line segment elements are removed from the beginning and end of the input line segment elements to efficiently cut out the figure.

〔Effect of the invention〕

According to the present invention, handwritten handwriting is divided into a plurality of line segments and recognized using features obtained from a straight line connecting any two of a plurality of sample points of handwritten handwriting. 2) Even if one figure is drawn with an arbitrary number of handwritings, by dividing the handwritings into line segments, which are the constituent units of the figure, it is possible to recognize them without depending on the number of handwritings when inputting the figure. This has the effect of improving man-machine performance compared to before.

More specifically, as shown in FIG. 10, this has the effect of significantly relaxing restrictions on the number of strokes, stroke order, rotation, and division when inputting handwritten figures.

For example, as shown in FIG. 1 (1), when one figure is drawn with an arbitrary number of strokes, the figure is decomposed into line segment elements, which are units of the figure, so there is no limit to the number of strokes.

Furthermore, as shown in FIG. 2 (2), when one figure is drawn from any part, line segment elements are ordered clockwise by ``code wrapping'', so there are no restrictions on the order of strokes.

When a figure is input at an angle, the degree of difference is evaluated based on the angular difference between adjacent line segment elements, so the rotation of the input figure is not restricted ((8) in the same figure).

A figure part can be cut out from multiple line segment elements (-
dimensional line segment DP method), it is possible to input figures with connecting lines, and there are no restrictions on so-called separation.

[Brief explanation of the drawing]

Figure 1 is a diagram showing an example of the configuration of an online handwritten figure recognition system, Figure 2 is a flowchart of Kitoaki's desired procedure, and Figure 3 is a diagram showing an example of the configuration of an online handwritten figure recognition system.
The figure shows direction codes, Figure 4 is an explanatory diagram of converting handwritten handwriting into line segment elements according to the present invention (area method), Figure 5 is an explanatory diagram of figure normalization processing, and Figure 6 is an illustration of connection list. 7 is an explanatory diagram of code wrapping, FIG. 8 is an explanatory diagram of difference calculation, FIG. 9 is an explanatory diagram of reconfiguration of line segment elements, and 10th
The figure shows an example of figure recognition, Fig. 11 shows a conventional example of smoothing, Fig. 12 shows a conventional example of approximating handwriting with polygonal lines, and Fig. 13 shows an example of a character recognition tree. Figure, 14th
The figure is a flowchart of a conventional handwritten figure recognition procedure, and FIG. 15 is an explanatory diagram of a conventional example using the candidate lattice method. 100...replets, 200...computer, 600
...Graphic dictionary.

Claims (1)

[Claims] 1. Evaluate the degree of similarity between a handwritten figure and a dictionary figure stored in advance, and if there is a dictionary figure among the dictionary figures that can be recognized as the same as the handwritten figure, that dictionary figure is recognized. In a handwritten figure recognition device that outputs the result and repeats the above process by inputting the next handwritten figure if there is no result, the combination of some or all of the handwriting of the handwritten figure is compared with the dictionary figure, and the combination is An online handwritten figure recognition device, characterized in that the handwriting is selected from a group of line segments constituting the handwriting, and the comparison is performed from combinations with a large number of line segments. 2. In the online handwritten figure recognition device according to claim 1, when the line segment groups are numbered in a chronological sequence, the line segment with the minimum or maximum number is deleted, and the remaining line segment group is An online handwritten figure recognition device as one of the combinations. 3. In the online handwritten figure recognition device according to claim 2, the number of line segments to be deleted is limited to n (0≦n<number of input line segments, n is an integer), and the deletion is performed from the line segment number. An online handwritten figure recognition device that performs handwriting alternately from both ends of a sequence of numbers. 4. In the online handwritten figure recognition device according to claim 2 or 3, when the line segment group includes a line segment group (referred to as a curved part) connected at an angle of a predetermined value or more, Online handwritten figure recognition device that deletes multiple line segments. 5. In the online handwritten figure recognition device according to claim 2 or 3, when the line segment group includes a line segment group (referred to as a curved part) connected at an angle of a predetermined value or more, the curved part An online handwritten figure recognition device that deletes in one go.