SK3096A3 - Method of stroke segmentation for handwritten input - Google Patents

Abstract

The method of the present invention includes a step of calculating the derivative (140), or instantaneous rate of change, of the curvature at points in the handwritten input (110). The method then selects as stroke boundary points certain points (or pixels) in the input which lie at a midpoint between a point of high curvature derivative and a succeeding point of low curvature derivative (150). Such boundary points are not influenced by absolute curvature values, but rather only by relative changes in the curvature. The stroke segmentation boundary points are provided to a stroke-based recognizer for interpretation of the handwritten input (170).

Description

METHOD OF SEGMENTATION OF STRAPS FOR GRAPHIC ENTRY

Technical field

The present invention relates generally to handwriting recognition and, in particular, to a method of stroke segmentation for handwriting input.

BACKGROUND OF THE INVENTION

Machine recognition of the human handwriting is a very complex matter and, with the simultaneous rapid expansion of pen-based computing, it has become an important task to be addressed. There are many different approaches to its solution. One useful approach is to divide the manuscript into a sequence of basic movements or strokes, and to use those strokes, which are somewhat parameterized, as character recognition inputs.

A key condition for a stroke-based recognizer is that many instances of the same character class (eg, A letters written at different times by different writers) should each be divided into a similar set of strokes. This helps to ensure that recognition is not too complicated because the character case description will appear similar to the character recognizer. Ideally, all instances of a given feature would always contain the same number of strokes, all strokes would be in the same relative position, and the descriptions of the stroke properties would be very similar in all cases. This ideal is not achievable in practice, but the accuracy of recognition can be improved to the extent that it can be approached.

In one of the methods used hitherto, the stroke boundaries are located at points where the speed of the pen in the vertical (or y) direction is zero, i. at points where the font begins to move up or down. The resulting sets of strokes can thus be called upstrokes and downstrokes. This method is discussed in Mermelstein & Eden, Experiments on Computer Recognition of

Connected Handwritten Words, in Information and Control, Vol. 7, p. 255-270, 1964. One of the disadvantages of this method is that it is too sensitive to changes in the vertical direction and insensitive to changes in the horizontal direction. But many characters are composed of horizontal parts - eg. the horizontal line t and the three legs E are usually much more horizontal than vertical, even in a sloppy manuscript. A y-speed-based stroke segmentator sometimes splits the horizontal portion into one stroke, but often splits it into two, three, or more strokes, simply because of a slight flick of the pen in the vertical direction. This leads to low recognition accuracy because many instances of the same feature will often be segmented into different looking stroke sets. Attempts to correct the inaccuracies of this method, including requiring a minimum change in the vertical direction before a new move was made, had only limited success and many of the same underlying problems still remain.

Another current method addresses this problem by placing stroke boundaries at points where the curvature is at its local maximum and exceeds a certain threshold that corresponds to sharp bends in the manuscript. Since a sharp bend may occur regardless of the direction in which the pen moves, this method is not sensitive to the orientation of various parts of the handwriting input, such as words or characters. The curvature-based technique also has its drawbacks. For example, suppose someone writes L with a very gradual curvature rather than a sharp curvature, so that it starts to look more than C. In this case, the method may fail to segment L until the threshold curvature required for the stroke boundaries has been reached. Simply lowering the threshold does not solve the problem, as this would only lead to a disproportionate number of moves. Too many single moves are as undesirable as too few, because again, many cases of the same feature are often segmented into different types of moves.

There is therefore a need for a segmentation method that is more accurate and does not have the drawbacks of said methods, such as the y-speed rate method and the existing curvature method.

SUMMARY OF THE INVENTION

In accordance with the present invention, a method of segmenting handwriting input into strokes, the number of which is consistent for many cases of each individual character class, is implemented.

The present invention provides a method of segmenting handwritten entry into strokes that are similar in shape and location for many cases of each individual character class at the entry.

The method of the present invention generally comprises the step of calculating the derivative or the instantaneous value of the curvature change at the handwriting entry points. The method then selects certain points (or pixels) in the input that lie between the high curvature derivative point and the subsequent low curvature point as the stroke boundary points. Such boundary points are not affected by absolute curvature values, but rather by relative changes in curvature.

Overview of the drawings

Fig. 1 illustrates a flow chart of a stroke boundary identification process in accordance with a preferred embodiment of the present invention.

Fig. 2 illustrates a handwriting stroke segmentation generated by a prior speed y-direction.

Fig. 3 illustrates the handwriting stroke segmentation created by the preceding curvature method.

Fig. 4 illustrates a segment of handwritten stroke entry created by a preferred embodiment of the present invention.

Fig. 5 shows the points forming the letter L as received from the digitizer.

Fig. 6 shows the points forming the letter L after resampling at constant distances in accordance with a preferred embodiment of the present invention.

In FIG. 7 is an exploded view showing the curvature calculation in a preferred embodiment of the present invention.

Fig. 8 graphically illustrates the curvature values calculated for each point in FIG. 7th

Fig. 9 graphically depicts the curvature value derivatives calculated for each point of FIG. 7th

DETAILED DESCRIPTION OF THE INVENTION

Typically, handwritten character input is obtained from the user in the form of separate continuous segments. A separate continuous segment consists of one or more pen strokes, where the pen stroke is the trace that the pen leaves during its contact with the input device, such as a digitizing table or paper.

In the present invention, handwriting input units are formed by one or more separate continuous segments. Handwriting input is an input that is electronically scanned, including but not limited to: handwriting input, electronic input, pressure sensed input such as a print input, input received electronically, such as by fax, a radio search system receiver, or other device.

The thrust is expressed as a sequence of points sampled at approximately regular intervals by the input device. Each point is taught at least by x and y coordinates. The strokes may be scanned electronically using a digitizing table, or otherwise may be derived from a scanned or faxed image by a line detection process in the image; such electronic scanning methods are known in the art. In a preferred method, handwriting input is received by a device such as a personal digital assistant (PDA) or other device. Other means of receiving handwriting include, but are not limited to, computers, modems, radio search system receivers, telephones, digital televisions, interactive televisions, digitizing tablet devices, fax machines, scanning devices, and other devices capable of capture handwriting input. Generally, when strokes are sensed electronically, each point is represented by a pixel such that the stroke is expressed as a series of pixels on the device.

In accordance with the present invention, the handwriting input may be in the form of alphanumeric characters, ideographic characters or other forms of characters or symbols of written communication.

Fig. 2 and FIG. 3 illustrates stroke segmentation of alphanumeric handwriting input with a high probability of inaccuracy in input interpretation when stroke segmentation arrives at the stroke recognizer. Fig. 4 shows stroke segmentation of the same alphanumeric input as in FIG. 2 and FIG. 3, where stroke segmentation is performed in accordance with the present invention, and such stroke segmentation has a high likelihood of accurate interpretation upon arrival at the stroke recognizer.

Fig. 1 shows a flow chart of a preferred method in accordance with the present invention. The handwriting input from the digitizer or other device is received in the form of x and y coordinates 110 (with associated pen raised or lowered states). Typically, these dots are represented by pixels. The present method generally resample handwriting input to obtain points that are equally spaced over the length of handwriting input 120. FIG. 5 shows an example of an L 500 as a series of dots or pixels prior to resampling. Fig. 6 shows the same letter L 600 as a series of dots or pixels after resampling. Sampling is done using a point-to-point distance d 610 that is constant throughout the handwriting input. Preferably, d is selected such that the mean value of the handwriting input height is approximately 15 to 30 times d. For example, in FIG. 6, the value of d is selected such that the mean value of the letter height in the word is approximately between 15 and 30 times d.

The preferred embodiment of FIG. 1 calculates the curvature at each resampling point 130. FIG. 7 illustrates graphically a description of the data for calculating the curvature at point R 710. The curvature at the resampled point R 710. is defined as the distance to the point following R (point R + 1 730) from point P 720 obtained by linear projection from the point preceding R (R). This distance is shown in FIG. 7 as the element 740. The curvature at the endpoints of the handwriting entry is defined as the same at the corresponding nearest adjacent points. The curvature at the internal points of the handwriting entry can also be calculated by projecting two points distant from R rather than one point (and using a point that is two points ahead of R rather than a point R-1) to obtain a more error-tolerant estimate. Fig. 8 shows graphically the curvature values obtained for the points shown in FIG. 7th

For example, the aforementioned example of a gradual curve L may be segmented into a vertical and a horizontal stroke if the two straight portions of L are substantially equal to the bend between them. Thus, the curvature would increase towards the bend (i.e., the derivative of the curvature would be high) and decrease from the bend (the derivative of the curvature would be low), allowing the bending tensile limit to be near or close to it if desired.

If, in a preferred embodiment of the present invention, the curvature has already been obtained for each resampling point, the curvature group for the resampling points may be smoothed to minimize any known artifacts introduced by the digitization process. The type of anti-aliasing to be performed should be a standard method that is selected based on specific digitizing characters. This may include averaging the point value with its adjacent points (weighting the point itself and the nearest higher points) and replacing the curvature value at said point with a calculated average. Ideally, the size of the smoothing window used herein should be wider in areas of low font curvature and narrower in areas of large curvature to minimize the loss of important information in the smoothing process. Since the object of smoothing is the curvature itself, one preferred smoothing method is to calculate the initial curves, where smoothing is based on these curves, and again smoothing based on the newly calculated curves.

In a preferred embodiment of the present invention, the absolute curvature value is calculated for each resampling point by multiplying each negative curvature value by -1. Absolute curvature values are used to calculate curvature derivatives rather than actual curvature values, since the generally preferred embodiments of the present invention deal only with the sharpness of the font bends and not in which direction the curve is curved.

As shown in FIG. 1, the process continues by calculating the curvature derivative at each resampling point 140. Referring to FIG. 7, the curvature derivative at R-point is defined as the absolute value of curvature at R-point +1 minus the absolute value of curvature at R-1 divided by two (i.e. the slope of the curve of plotted curvature values). Fig. 9 graphically illustrates the curvature derivative obtained at each point shown in FIG. 7. As with using more than two points to obtain a more accurate curvature value as described above, the curvature derivative could be calculated using a wider window (five points vs. three) where there are relevant points and a narrower window (two points vs. three) , where necessary. Since the curvature derivative cannot be calculated at the endpoints of the written segment, the curvature derivative at the endpoints is simply defined as the same at the corresponding adjacent points.

According to FIG. 1 and 9, a preferred embodiment of the method further examines a recalculated array of curvature derivative values to determine points where the derivative is at its local maximum of 910 (defined to include points at the end of the curve turnover and starting to decrease) or where the derivative is at its local minimum 920 (defined to include points at the end of curve turnover and beginning to rise). For each input area (in time) after the local maximum and before the local section (expressed by the minimum arc length, the center point of the section is specified) 930. This center point is defined as M 930.When the difference between the local maximum and local minimum values exceeds the threshold value T '940 and the absolute value of the curvature at the point M exceed a certain threshold value T (820), the point M being selected as the tension limit 150.

The parameters T 'and T must be estimated and depend on the units in which the curvature and the curvature derivative are measured. The exact values here are not critical if an error tolerant character recognizer is used. When performing any experimental tuning of these or any other parameters to create a specific embodiment of the invention, the desired goal is to achieve as uniform segmentation as possible for many cases of individual character classes. This should be done empirically, by examining how the procedure segments the different actual font samples to be recognized.

In addition to the already described selected stroke boundary points, the stroke points are where the pen is raised or lowered. In a preferred embodiment of the present invention, the curvature-based boundary points may be shifted by up to two points, causing them to fall to points where the absolute curvature values are a maximum of 160. that both the curvature rate and the curvature derivative will provide the same tension limits. However, this preferred application of the present invention is performed for a given point only if no thrust selected from less than three points is created.

The set of stroke boundary points defines a set of corresponding strokes in accordance with the present invention. These strokes can then be passed to the character recognizer based on the strokes to recognize handwriting input.

The present invention and its preferred embodiments relate to new more precise methods of stroke segmentation. In accordance with the present invention, in many handwritten input cases, the input is repeatedly divided into similar sets of strokes each time. For example, if the handwritten entry is an L written at different times by different writers, the present invention and its preferred embodiment would each time more precisely divide the L entry into similar stroke segmentation points, regardless of the differences of the different writers. Such stroke segmentation would help to provide a more accurate interpretation of stroke-based recognizer.

Those skilled in the art will recognize that a number of embodiments of the present invention can be used. One obvious extension is from the case of the printed manuscript described herein for the case of the combined script. The present stroke segmentation method is independent of the character separation method, and hence, techniques allowing the processing of the combined font can easily utilize the stroke segmentation method described herein. Another embodiment could be segmentation of the scanned or unlinked font into strokes. A straightforward method of applying the present invention to such a problem would be to reduce the font to achieve a constant width of the written curve. The stroke boundaries could then be determined at both the curvature derivative points and the intersection points, because the lack of time information makes the intersections and touching bends look similar.

Claims (8)

A method for recognizing a handwriting feature composed of a plurality of colored pixels, comprising the steps of: - calculating the value of the curvature derivative for each of the plurality of colored pixels, wherein each value of the curvature derivative represents the rate of change of the absolute curvature at the corresponding pixel, - selecting a set of stroke boundaries such that each stroke boundary lies between a colored pixel with a high value of curvature derivative and a subsequent colored pixel with a low value of curvature derivative, - determining the set of strokes so that each stroke boundary is located at the end of the stroke, - calculating at least one stroke property value for each stroke to create a feature set - using a feature set to determine the identity of said handwriting. The method of claim 1, wherein the colored high curvature derivative pixel has a locally maximum curvature derivative value and the subsequent low curvature derivative colored pixel has a locally minimal curvature derivative value. The method of claim 2, wherein each stroke boundary lies at a point half the distance between the colored pixel with a locally maximum curvature derivative value and the colored pixel with a locally minimal curvature derivative value. The method of claim 1, wherein each tension limit lies at a point of a locally maximum absolute curvature value. 5. A method of recognizing a handwriting feature composed of a sequence of points, wherein each point consists of three spatial coordinate values, comprising the steps of: - calculating the value of the curvature derivative for each of the set of points, where each value of the curvature derivative represents the rate of change of the absolute curvature at the corresponding point, - selecting a set of stroke boundaries such that each stroke boundary lies between a point with a high curvature derivative value and a subsequent point with a low curvature derivative value, - specifying a set of strokes such that each stroke boundary is located at the end of the stroke, - calculating at least one stroke property value for each stroke to create a set of character properties, - using a feature set to determine the identity of said handwriting. The method of claim 5, wherein the high curvature derivative point has a locally maximum curvature derivative value and the subsequent low curvature derivative point has a locally minimal curvature derivative value. The method of claim 6, wherein each stroke boundary lies at a point half the distance between the locally maximum curvature derivative point and the locally minimal curvature derivative point. The method of claim 5, wherein each stroke boundary lies at a point of the locally maximum absolute curvature value. Description of picture no. 1 110 120 130 140 150 160 170 input input as a sequence (x, y) of resampling points such that the points are spaced at equal distances calculating the curvature size at each point calculating the curvature derivative at each point identifying the stroke boundaries as points between the local maximum and the curvature derivative handover of detected stroke boundaries for further use in the recognition process to correct small measurement errors