CA2162489A1 - Method of stroke segmentation for handwritten input - Google Patents
Method of stroke segmentation for handwritten inputInfo
- Publication number
- CA2162489A1 CA2162489A1 CA002162489A CA2162489A CA2162489A1 CA 2162489 A1 CA2162489 A1 CA 2162489A1 CA 002162489 A CA002162489 A CA 002162489A CA 2162489 A CA2162489 A CA 2162489A CA 2162489 A1 CA2162489 A1 CA 2162489A1
- Authority
- CA
- Canada
- Prior art keywords
- curvature
- stroke
- point
- derivative value
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/32—Digital ink
- G06V30/333—Preprocessing; Feature extraction
- G06V30/347—Sampling; Contour coding; Stroke extraction
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Character Discrimination (AREA)
- Image Analysis (AREA)
Abstract
The method of the present invention includes a step of calculating the derivative (140), or instantaneous rate of change, of the curvature at points in the handwritten input (110). The method then selects as stroke boundary points certain points (or pixels) in the input which lie at a midpoint between a point of high curvature derivative and a succeeding point of low curvature derivative (150). Such boundary points are not influenced by absolute curvature values, but rather only by relative changes in the curvature. The stroke segmentation boundary points are provided to a stroke-based recognizer for interpretation of the handwritten input (170).
Description
21 62~ 89 METHOD OF STROKE SEGMENTATION
FOR HANDWRITTEN INPUT
Background of the Invention Field of the Invention This invention relates generally to handwriting recognition, 1 0 and more particularly to a method of stroke segmentation of handwritten input.
Description of the Prior Art 1 5 MAçhine recognition of human handwriting is a very difficult problem, and with the recent explosion of pen-based computing devices, has become an important problem to be addressed. There exist many very different approaches to this problem, but one useful approach has been to divide the writing into a sequence of 2 0 filn~l~mental movements, or "strokes", and use the strokes (which are parametrized in some way) as inputs to a character recognizer.
A key requirement in a stroke-based recognizer is that multiple instances of the same character class (e.g. the letter "A" written at different times and by different writers) should be divided into a 2 5 .simil~r set of strokes each time. This helps insure that recognition is not too difficult, because the description of the character instances will "look" simil~r to the character recognizer itself. In the ideal case, all instances of a given character would always contain the same~umber of strokes, the strokes would all be in the same relative 3 0 locations, and the featural descriptions of the strokes would all be very .simil~r across the instances. This ideal is not achievable in practice, but to the extent it can be approached, recognition accuracy can be improved.
2 2162~89 In one technique in the prior art, stroke boundaries are set at points where pen velocity in the vertical (or "y") direction is zero --that is, at points where the writing begins moving upward, or begins 5 moving downward. The resulting set of strokes may then be called "upstrokes" and "downstrokes". This method is discussed in Mermelstein & Eden, "Experiments on Computer Recognition of Connected Handwritten Words", in Information And Control vol. 7, pp. 255-270, 1964. One problem with this method is that it is overly l 0 sensitive to changes in the vertical direction, and not sensitive at all to changes in the horizontal direction. Yet many characters are composed of horizontal pieces -- for example, the cross of a "t" and the three prongs of an "E" are normally much more horizontal than vertical, even in sloppy writing. A y-velocity based stroke segmenter l 5 will sometimes break a horizontal piece into one stroke, but often it will be broken into two, three, or even many more, simply because of small jitters of the pen in the vertical direction. This leads to poor recognition accuracy, because multiple instances of the same character will often be segmented into different-looking sets of 2 0 strokes. Attempts to correct the inaccuracies of this method, including requiring a minimum vertical direction change before creating a new stroke have had only limited success, and many of the same basic problems still remain.
In another existing technique, this problem is solved by setting 2 5 the stroke boundaries at points where curvature is at a local m~3~imum and exceeds some threshold value that corresponds to sharp bends in the writing. Since a sharp bend can occur regardless of the direction the pen is moving, this method is not sensitive to the orientation of the various parts of handwritten input, such as words 3 0 or characters. However, the curvature-based technique has its own problems as well. Suppose, for example, a person writes an "L" with a very gradual bend, rather than a sharp bend, so that it starts to look more like a "C". The method might fail to segment the "L" in this case if the threshold curvature required for stroke boundaries was 3 5 not met. Simply lowering the threshold does not solve the problem, - ` 3 2~ 62~89 because this can simply lead to excessive numbers of strokes. Having too many extra strokes is just as bad as having too few, because again it means that multiple instances of the same character are often segmented into different types of strokes.
Accordingly a need exists for a stroke segmentation technique - that is more accurate and not subject to the problems in the methods discussed, such as the y-velocity method and the existing curvature method.
1 0 Sllmm~ry of the Invention Accordingly, the present invention provides a method of segmenting handwritten input into strokes which are consistent in number across multiple instances of each particular character class.
The present invention provides a method of segmenting handwritten input into strokes which are ~imil~r in shape and - location across multiple instances of each particular character class of input.
Generally, the method of the present invention includes a step 2 0 of calculating the derivative, or instantaneous rate of change, of the curvature at points in the handwritten input. The method then selects as stroke boundary points certain points (or pixels) in the input which lie between a point of high curvature derivative and a succeeding point of low curvature derivative. Such boundary points 2 5 are not influenced by absolute curvature values, but rather only by relative changes in the curvature.
- _ 4 2162~89 Brief Description of the Drawings FIG.l Illustrates a flow diagram of operation for identifying stroke boundaries in accordance with a preferred embodiment of the S present invention.
FIG.2 Illustrates segmentation into strokes of handwritten input produced by the y-velocity prior art method.
FIG.3 Illustrates segmentation into strokes of handwritten input produced by the curvature prior art method.
FIG 4 Illustrates segmentation into strokes of handwritten input produced by a preferred embodiment of the present invention.
FIG.5 Illustrates points m~king up the letter "L" as received from a digitizing device.
FIG.6 Illustrates points m~qking up the letter "L" after 15 resampling to a constant distance in accordance with a preferred embodiment of the present invention.
FIG.7 Is an exploded view Illustrating a curvature calculation of a preferred embodiment of the present invention.
FIG.8 Illustrates graphically the curvature values calculated 2 0 for each point in FIG.7.
FIG.9 Illustrates graphically the derivative of curvature values calculated for each point in FIG.7.
2 5 Detailed Description of the P.efeI~ed Embodiments Typically, handwritten character input is collected from the user in the form of discrete continuous segments. A discrete continuous segment consists of one or more pen strokes, where a pen 3 0 stroke is the mark left by a pen during its period of contact with an input device such as a digitizing tablet or paper.
In the present invention one or more discrete continuous segments are the units of handwritten input being recognized.
Handwritten input is input which is captured electronically that 3 5 includes but is not limited to the following: handwritten input;
5 21 62~ 89 electronic input; input captured through pressure, such as stamped input; input that is received electronically, such as via f~csimile, pager, or other device.
A stroke is represented as a sequence of points sampled at approximately regular intervals by the input device. Each point is described at minimum by an X and a Y coordinate. Strokes may be captured electronically using a digitizing tablet, or alternatively may be derived from a scanned or faxed image through a process of line detection in the image; such methods of capturing electronically are understood in the art. In a ~lererled method handwritten input is accepted by a device, such as a personal digital assistant (PDA) or other device. Other devices that function to receive handwritten input include, but are not limited to, the following: computers, modems, pagers, telephones, digital televisions, interactive televisions, devices having a digitizing tablet, f~csimile devices, sc~nning devices, and other devices with the ability to capture handwritten input.
Generally, when strokes are captured electronically, each point is represented by a pixel, such that a stroke is represented by a series of pixel on the device.
2 0 In accordance with the present invention, handwritten input maybe in the form of alphanumeric characters, ideographic characters or other forms of characters or symbols of written communication.
Referring now to the FIGs. and FIGs. 2 and 3 illustrate stroke 2 5 segmentation of alphanumeric handwritten input with a high probability of inaccuracies in interpretation of the input when stroke segmentations are passed to a stroke-based recognizer. FIG. 4 illustrates stroke segmentation of the same alphanumeric input of FIGs. ~ and 3 where the stroke segmentation is made in accordance 3 0 with the te~çhing.c of the present invention and such stroke segmentation has a high probability of accurate interpretation when passed to a stroke-based recognizer.
Referring now to Fig. 1, illustrating a flowchart of a preferred the method in accordance with the te~çhings of the present 3 5 invention. Handwritten input from the digitizer or other device is - ~ 6 21621~9 received in the form of xy coordinates (110) (with associated penup or pendown states). Typically these points are represented by pixels.
Generally, the present method resamples the handwritten input to obtain points which are equally spaced along the length of the S handwritten input (120). Figure 5 illustrates an example of the letter "L" (500) as a series of points or pixels before resampling. Figure 6 illustrates the same letter "L" (600) as a series of points or pixels after resampling. Resampling is done using an interpoint distance d (610) that is constant throughout the handwritten input. Preferably, the l 0 value of d chosen is such that the median input height in the handwritten input is approximately in the range of 15 to 30 times d.
For ~mple, in FIG. 6 the value of d chosen is such that the median letter height in the word is approximately in the range of 15 to 30 times d.
l S The preferred embodiment of FIG. 1 calculates the curvature at each resampled point (130). Figure 7 illustrates graphically a depiction of the data for calculation of curvature at a point R (710).
Curvature at a resampled point R (710) is defined as the distance to R's successor (point R+1 (730)) from a point P (720), obtained by 2 0 projecting linearly from R's predecessor (R-1, 750) through R itself.
This distance is shown in Figure 7 as element 740. Curvature at endpoints of the handwritten input is defined as equal to that at the corresponding nearest neighbor points. Curvatures at the interior points of the handwritten input may also be computed by projecting 2 S two points away from R, rather than one point (and using a point two points prior to R, rather than R-1), to obtain a more robust estimate.
Figure 8 illustrates graphically the curvature values obtained for the points shown in Figure 7.
-For example, the above mentioned example of an "L" with a 3 0 gradual bend could be segmented into a vertical and a horizontal stroke as long as the two "straight" parts of the "L" were significantly straighter than the bend between them. The curvature would then be increasing going into the bend (i.e., the curvature derivative would be high) and decreasing going away from the bend (the curvature 7 2162 1~9 derivative would be low), thus allowing a stroke boundary at or near the bend, as desired.
In a preferred embodiment of the present invention, once the curvatures have been obtained for each resampled point, the array of S curvatures for the resampled points may be smoothed to minimi7e any known artifacts introduced by the digitizing process. The type of smoothing to be performed should be a standard method which is selected based on the particular digitizing characteristics at hand.
This might include averaging the value of a point with its neighhor l 0 points (weighting the point itself and nearest points higher), and repl~ing the curvature value of the point in question with the computed average. The size of the smoothing window used here should ideally be wider at low-curvature sections of the writing and narrower at high-curvature sections, to minimi7e loss of important l S information in the smoothing process. Since it is the curvature itself that is being smoothed, though, one ~efelable way to smooth is to compute initial curvatures, smooth based on those curvatures, and then re-smooth based on the new curvatures computed.
In a preferred embodiment of the present invention, for each 2 0 resampled point the absolute value of the curvature is computed by multiplying any negative curvature values by the value -1. Preferably the absolute values are used in computing curvature derivatives, rather than the actual curvatures values, because the overall preferred embodiments 2 5 of the present invention process are only concerned with the sharpness of bends in the writing, and not which direction a given bend curves.
As illustrated in FIG. 1, the process next calculates the derivative of curvature at each resampled point (140). Referring to 3 0 Figure 7, the derivative of curvature at point R is defined as the absolute value of curvature at point R+1 minus the absolute value of curvature at point R-1, divided by two (i.e., the slope of the curve of plotted curvature values). Figure 9 illustrates graphically the derivative of curvature obtained at each point shown in Figure 7.
3 5 .~imil~r to the use of more than two points to get a more accurate 2l62~89 measure of curvature as described above, the derivative of curvature should be computed using a wider window (five points versus three) where the relevant points exist, and a narrower one (two points versus three) where necessary. Since the derivative of curvature cannot be calculated at the endpoints of the ink segment, the derivative of curvature at the endpoints is simply defined to be equal to that at the corresponding neighhor points.
Referring now to FIGs. 1 and 9, a preferred embodiment of the process next çx~mines the newly calculated array of curvature derivative values to locate points where the derivative is at a local m~imum (910) (defined to include points at the end of an inflection and about to decrease) or where the derivative is at a local minimum (920) (defined to include points at the end of an inflection and about to increase). For each section of input (in time) after a local m~ximum 1 5 and before a local minimum, the midpoint of the section (in terms of arc length of the section) is located (930). This midpoint is defined as M (930). If the difference between the local m~imum and the local minimum values for a section exceeds a threshold value T' (940), and the absolute curvature value at M exceeds some threshold value T"
2 0 (820), the point M is selected as a stroke boundary (150).
The parameters T' and T" must be estimated and depend on the units in which curvature and curvature derivative are measured.
There exact values are not critical as long as an error-tolerant character recognizer is used. In performing any experimental 2 5 tuning of these or any other parameters for creating a specific embodiment of the invention, the desired goal is to attain segmentation which is as consistent as possible across multiple instances of particular character classes. This should be done empirically by e~mining how the procedure segments various 3 0 actual samples of the writing to be recognized.
In addition to the selected stroke boundary points described, points where the pen is lifted or put down are also selected as stroke boundary points. In a preferred embodiment of the present invention, curvature derivative-based boundary points maybe shifted by as much 3 5 as two points to cause them to fall on points where absolute curvature 9 2162~9 value is m~q~im~l (160). By shifting the curvature derivative based boundary points, location of the stroke boundary points can be improved by m~king both the measure of curvature and the derivative of curvature yield the same stroke boundary. This preferred 5 application of the present invention is only done for a given point if no stroke of less than three points will be produced, however.
The set of stroke boundary points in accordance with the present invention defines a set of corresponding strokes. These strokes can then be forwarded to a stroke-based character recognizer 1 0 for recognition of the handwritten input.
The present invention and its preferred embodiments relate to novel more accurate methods of stroke segmentation. In accordance with the present invention, in multiple instances of handwritten input the input is repeatedly divided into a simil~r set of strokes each l 5 time. For example, if the handwritten input is the Letter "L" written at different times by different writers, the present invention and its preferred embodiments would more accurately divide each time the input of the letter "L" into simil~r stroke segmentation boundary points regardless of the variances of the different writers. Such 2 0 stroke segmentation would aid in providing more accurate interpretation by a stroke-based recognizer.
Those skilled in the art will find many embodiments of the present invention to be useful. One obvious extension is from the case of printed handwriting described here to that of cursive writing. The 2 5 actual method of segmentation of strokes is independent of the method of segmenting characters, and thus techniques which allow processing of cu~sive writing may readily make use of the stroke segmentation process described here. Another embodiment would be the segmentation of scanned, or "off-line" writing, into strokes. A
3 0 straightforward way of applying the present invention to such a task would be to perform thinning of the writing to obtain a constant-width ink curve. Stroke boundaries could then be set at both curvature derivative-based points and at intersection points, since the lack of temporal information makes intersections and touching bends look 3 5 similar.
FOR HANDWRITTEN INPUT
Background of the Invention Field of the Invention This invention relates generally to handwriting recognition, 1 0 and more particularly to a method of stroke segmentation of handwritten input.
Description of the Prior Art 1 5 MAçhine recognition of human handwriting is a very difficult problem, and with the recent explosion of pen-based computing devices, has become an important problem to be addressed. There exist many very different approaches to this problem, but one useful approach has been to divide the writing into a sequence of 2 0 filn~l~mental movements, or "strokes", and use the strokes (which are parametrized in some way) as inputs to a character recognizer.
A key requirement in a stroke-based recognizer is that multiple instances of the same character class (e.g. the letter "A" written at different times and by different writers) should be divided into a 2 5 .simil~r set of strokes each time. This helps insure that recognition is not too difficult, because the description of the character instances will "look" simil~r to the character recognizer itself. In the ideal case, all instances of a given character would always contain the same~umber of strokes, the strokes would all be in the same relative 3 0 locations, and the featural descriptions of the strokes would all be very .simil~r across the instances. This ideal is not achievable in practice, but to the extent it can be approached, recognition accuracy can be improved.
2 2162~89 In one technique in the prior art, stroke boundaries are set at points where pen velocity in the vertical (or "y") direction is zero --that is, at points where the writing begins moving upward, or begins 5 moving downward. The resulting set of strokes may then be called "upstrokes" and "downstrokes". This method is discussed in Mermelstein & Eden, "Experiments on Computer Recognition of Connected Handwritten Words", in Information And Control vol. 7, pp. 255-270, 1964. One problem with this method is that it is overly l 0 sensitive to changes in the vertical direction, and not sensitive at all to changes in the horizontal direction. Yet many characters are composed of horizontal pieces -- for example, the cross of a "t" and the three prongs of an "E" are normally much more horizontal than vertical, even in sloppy writing. A y-velocity based stroke segmenter l 5 will sometimes break a horizontal piece into one stroke, but often it will be broken into two, three, or even many more, simply because of small jitters of the pen in the vertical direction. This leads to poor recognition accuracy, because multiple instances of the same character will often be segmented into different-looking sets of 2 0 strokes. Attempts to correct the inaccuracies of this method, including requiring a minimum vertical direction change before creating a new stroke have had only limited success, and many of the same basic problems still remain.
In another existing technique, this problem is solved by setting 2 5 the stroke boundaries at points where curvature is at a local m~3~imum and exceeds some threshold value that corresponds to sharp bends in the writing. Since a sharp bend can occur regardless of the direction the pen is moving, this method is not sensitive to the orientation of the various parts of handwritten input, such as words 3 0 or characters. However, the curvature-based technique has its own problems as well. Suppose, for example, a person writes an "L" with a very gradual bend, rather than a sharp bend, so that it starts to look more like a "C". The method might fail to segment the "L" in this case if the threshold curvature required for stroke boundaries was 3 5 not met. Simply lowering the threshold does not solve the problem, - ` 3 2~ 62~89 because this can simply lead to excessive numbers of strokes. Having too many extra strokes is just as bad as having too few, because again it means that multiple instances of the same character are often segmented into different types of strokes.
Accordingly a need exists for a stroke segmentation technique - that is more accurate and not subject to the problems in the methods discussed, such as the y-velocity method and the existing curvature method.
1 0 Sllmm~ry of the Invention Accordingly, the present invention provides a method of segmenting handwritten input into strokes which are consistent in number across multiple instances of each particular character class.
The present invention provides a method of segmenting handwritten input into strokes which are ~imil~r in shape and - location across multiple instances of each particular character class of input.
Generally, the method of the present invention includes a step 2 0 of calculating the derivative, or instantaneous rate of change, of the curvature at points in the handwritten input. The method then selects as stroke boundary points certain points (or pixels) in the input which lie between a point of high curvature derivative and a succeeding point of low curvature derivative. Such boundary points 2 5 are not influenced by absolute curvature values, but rather only by relative changes in the curvature.
- _ 4 2162~89 Brief Description of the Drawings FIG.l Illustrates a flow diagram of operation for identifying stroke boundaries in accordance with a preferred embodiment of the S present invention.
FIG.2 Illustrates segmentation into strokes of handwritten input produced by the y-velocity prior art method.
FIG.3 Illustrates segmentation into strokes of handwritten input produced by the curvature prior art method.
FIG 4 Illustrates segmentation into strokes of handwritten input produced by a preferred embodiment of the present invention.
FIG.5 Illustrates points m~king up the letter "L" as received from a digitizing device.
FIG.6 Illustrates points m~qking up the letter "L" after 15 resampling to a constant distance in accordance with a preferred embodiment of the present invention.
FIG.7 Is an exploded view Illustrating a curvature calculation of a preferred embodiment of the present invention.
FIG.8 Illustrates graphically the curvature values calculated 2 0 for each point in FIG.7.
FIG.9 Illustrates graphically the derivative of curvature values calculated for each point in FIG.7.
2 5 Detailed Description of the P.efeI~ed Embodiments Typically, handwritten character input is collected from the user in the form of discrete continuous segments. A discrete continuous segment consists of one or more pen strokes, where a pen 3 0 stroke is the mark left by a pen during its period of contact with an input device such as a digitizing tablet or paper.
In the present invention one or more discrete continuous segments are the units of handwritten input being recognized.
Handwritten input is input which is captured electronically that 3 5 includes but is not limited to the following: handwritten input;
5 21 62~ 89 electronic input; input captured through pressure, such as stamped input; input that is received electronically, such as via f~csimile, pager, or other device.
A stroke is represented as a sequence of points sampled at approximately regular intervals by the input device. Each point is described at minimum by an X and a Y coordinate. Strokes may be captured electronically using a digitizing tablet, or alternatively may be derived from a scanned or faxed image through a process of line detection in the image; such methods of capturing electronically are understood in the art. In a ~lererled method handwritten input is accepted by a device, such as a personal digital assistant (PDA) or other device. Other devices that function to receive handwritten input include, but are not limited to, the following: computers, modems, pagers, telephones, digital televisions, interactive televisions, devices having a digitizing tablet, f~csimile devices, sc~nning devices, and other devices with the ability to capture handwritten input.
Generally, when strokes are captured electronically, each point is represented by a pixel, such that a stroke is represented by a series of pixel on the device.
2 0 In accordance with the present invention, handwritten input maybe in the form of alphanumeric characters, ideographic characters or other forms of characters or symbols of written communication.
Referring now to the FIGs. and FIGs. 2 and 3 illustrate stroke 2 5 segmentation of alphanumeric handwritten input with a high probability of inaccuracies in interpretation of the input when stroke segmentations are passed to a stroke-based recognizer. FIG. 4 illustrates stroke segmentation of the same alphanumeric input of FIGs. ~ and 3 where the stroke segmentation is made in accordance 3 0 with the te~çhing.c of the present invention and such stroke segmentation has a high probability of accurate interpretation when passed to a stroke-based recognizer.
Referring now to Fig. 1, illustrating a flowchart of a preferred the method in accordance with the te~çhings of the present 3 5 invention. Handwritten input from the digitizer or other device is - ~ 6 21621~9 received in the form of xy coordinates (110) (with associated penup or pendown states). Typically these points are represented by pixels.
Generally, the present method resamples the handwritten input to obtain points which are equally spaced along the length of the S handwritten input (120). Figure 5 illustrates an example of the letter "L" (500) as a series of points or pixels before resampling. Figure 6 illustrates the same letter "L" (600) as a series of points or pixels after resampling. Resampling is done using an interpoint distance d (610) that is constant throughout the handwritten input. Preferably, the l 0 value of d chosen is such that the median input height in the handwritten input is approximately in the range of 15 to 30 times d.
For ~mple, in FIG. 6 the value of d chosen is such that the median letter height in the word is approximately in the range of 15 to 30 times d.
l S The preferred embodiment of FIG. 1 calculates the curvature at each resampled point (130). Figure 7 illustrates graphically a depiction of the data for calculation of curvature at a point R (710).
Curvature at a resampled point R (710) is defined as the distance to R's successor (point R+1 (730)) from a point P (720), obtained by 2 0 projecting linearly from R's predecessor (R-1, 750) through R itself.
This distance is shown in Figure 7 as element 740. Curvature at endpoints of the handwritten input is defined as equal to that at the corresponding nearest neighbor points. Curvatures at the interior points of the handwritten input may also be computed by projecting 2 S two points away from R, rather than one point (and using a point two points prior to R, rather than R-1), to obtain a more robust estimate.
Figure 8 illustrates graphically the curvature values obtained for the points shown in Figure 7.
-For example, the above mentioned example of an "L" with a 3 0 gradual bend could be segmented into a vertical and a horizontal stroke as long as the two "straight" parts of the "L" were significantly straighter than the bend between them. The curvature would then be increasing going into the bend (i.e., the curvature derivative would be high) and decreasing going away from the bend (the curvature 7 2162 1~9 derivative would be low), thus allowing a stroke boundary at or near the bend, as desired.
In a preferred embodiment of the present invention, once the curvatures have been obtained for each resampled point, the array of S curvatures for the resampled points may be smoothed to minimi7e any known artifacts introduced by the digitizing process. The type of smoothing to be performed should be a standard method which is selected based on the particular digitizing characteristics at hand.
This might include averaging the value of a point with its neighhor l 0 points (weighting the point itself and nearest points higher), and repl~ing the curvature value of the point in question with the computed average. The size of the smoothing window used here should ideally be wider at low-curvature sections of the writing and narrower at high-curvature sections, to minimi7e loss of important l S information in the smoothing process. Since it is the curvature itself that is being smoothed, though, one ~efelable way to smooth is to compute initial curvatures, smooth based on those curvatures, and then re-smooth based on the new curvatures computed.
In a preferred embodiment of the present invention, for each 2 0 resampled point the absolute value of the curvature is computed by multiplying any negative curvature values by the value -1. Preferably the absolute values are used in computing curvature derivatives, rather than the actual curvatures values, because the overall preferred embodiments 2 5 of the present invention process are only concerned with the sharpness of bends in the writing, and not which direction a given bend curves.
As illustrated in FIG. 1, the process next calculates the derivative of curvature at each resampled point (140). Referring to 3 0 Figure 7, the derivative of curvature at point R is defined as the absolute value of curvature at point R+1 minus the absolute value of curvature at point R-1, divided by two (i.e., the slope of the curve of plotted curvature values). Figure 9 illustrates graphically the derivative of curvature obtained at each point shown in Figure 7.
3 5 .~imil~r to the use of more than two points to get a more accurate 2l62~89 measure of curvature as described above, the derivative of curvature should be computed using a wider window (five points versus three) where the relevant points exist, and a narrower one (two points versus three) where necessary. Since the derivative of curvature cannot be calculated at the endpoints of the ink segment, the derivative of curvature at the endpoints is simply defined to be equal to that at the corresponding neighhor points.
Referring now to FIGs. 1 and 9, a preferred embodiment of the process next çx~mines the newly calculated array of curvature derivative values to locate points where the derivative is at a local m~imum (910) (defined to include points at the end of an inflection and about to decrease) or where the derivative is at a local minimum (920) (defined to include points at the end of an inflection and about to increase). For each section of input (in time) after a local m~ximum 1 5 and before a local minimum, the midpoint of the section (in terms of arc length of the section) is located (930). This midpoint is defined as M (930). If the difference between the local m~imum and the local minimum values for a section exceeds a threshold value T' (940), and the absolute curvature value at M exceeds some threshold value T"
2 0 (820), the point M is selected as a stroke boundary (150).
The parameters T' and T" must be estimated and depend on the units in which curvature and curvature derivative are measured.
There exact values are not critical as long as an error-tolerant character recognizer is used. In performing any experimental 2 5 tuning of these or any other parameters for creating a specific embodiment of the invention, the desired goal is to attain segmentation which is as consistent as possible across multiple instances of particular character classes. This should be done empirically by e~mining how the procedure segments various 3 0 actual samples of the writing to be recognized.
In addition to the selected stroke boundary points described, points where the pen is lifted or put down are also selected as stroke boundary points. In a preferred embodiment of the present invention, curvature derivative-based boundary points maybe shifted by as much 3 5 as two points to cause them to fall on points where absolute curvature 9 2162~9 value is m~q~im~l (160). By shifting the curvature derivative based boundary points, location of the stroke boundary points can be improved by m~king both the measure of curvature and the derivative of curvature yield the same stroke boundary. This preferred 5 application of the present invention is only done for a given point if no stroke of less than three points will be produced, however.
The set of stroke boundary points in accordance with the present invention defines a set of corresponding strokes. These strokes can then be forwarded to a stroke-based character recognizer 1 0 for recognition of the handwritten input.
The present invention and its preferred embodiments relate to novel more accurate methods of stroke segmentation. In accordance with the present invention, in multiple instances of handwritten input the input is repeatedly divided into a simil~r set of strokes each l 5 time. For example, if the handwritten input is the Letter "L" written at different times by different writers, the present invention and its preferred embodiments would more accurately divide each time the input of the letter "L" into simil~r stroke segmentation boundary points regardless of the variances of the different writers. Such 2 0 stroke segmentation would aid in providing more accurate interpretation by a stroke-based recognizer.
Those skilled in the art will find many embodiments of the present invention to be useful. One obvious extension is from the case of printed handwriting described here to that of cursive writing. The 2 5 actual method of segmentation of strokes is independent of the method of segmenting characters, and thus techniques which allow processing of cu~sive writing may readily make use of the stroke segmentation process described here. Another embodiment would be the segmentation of scanned, or "off-line" writing, into strokes. A
3 0 straightforward way of applying the present invention to such a task would be to perform thinning of the writing to obtain a constant-width ink curve. Stroke boundaries could then be set at both curvature derivative-based points and at intersection points, since the lack of temporal information makes intersections and touching bends look 3 5 similar.
Claims (8)
1. A method of recognizing a handwritten character composed of a plurality of inked pixels, comprising the steps of:
-- calculating a curvature derivative value for each of a plurality of the inked pixels, whereby each curvature derivative value represents the rate of change of absolute curvature at the corresponding pixel, -- selecting a set of stroke boundaries, such that each stroke boundary lies between an inked pixel with a high curvature derivative value and a succeeding inked pixel with a low curvature derivative value, -- locating a set of strokes, such that each stroke boundary is located at the end of a stroke, -- calculating at least one stroke feature value for each stroke to produce a character feature set, -- using the character feature set to determine the identity of said handwritten character.
-- calculating a curvature derivative value for each of a plurality of the inked pixels, whereby each curvature derivative value represents the rate of change of absolute curvature at the corresponding pixel, -- selecting a set of stroke boundaries, such that each stroke boundary lies between an inked pixel with a high curvature derivative value and a succeeding inked pixel with a low curvature derivative value, -- locating a set of strokes, such that each stroke boundary is located at the end of a stroke, -- calculating at least one stroke feature value for each stroke to produce a character feature set, -- using the character feature set to determine the identity of said handwritten character.
2. The method of Claim 1 wherein the inked pixel with a high curvature derivative value has a locally maximal curvature derivative value, and the succeeding inked pixel with a low curvature derivative value has a locally minimal curvature derivative value.
3. The method of Claim 2 wherein each stroke boundary lies on a point midway between the inked pixel with locally maximal curvature derivative value and the inked pixel with locally minimal curvature derivative value.
4. The method of Claim 1 wherein each stroke boundary lies on a point of locally maximal absolute curvature value.
5. A method of recognizing a handwritten character composed of a sequence of points, each point comprising three spatial coordinate values, comprising the steps of:
-- calculating a curvature derivative value for each of a plurality of the points, whereby each curvature derivative value represents the rate of change of absolute curvature at the corresponding point, -- selecting a set of stroke boundaries, such that each stroke boundary lies between a point with a high curvature derivative value and a succeeding point with a low curvature derivative value, -- locating a set of strokes, such that each stroke boundary is located at the end of a stroke, -- calculating at least one stroke feature value for each stroke to produce a character feature set, -- using the character feature set to determine the identity of said handwritten character.
-- calculating a curvature derivative value for each of a plurality of the points, whereby each curvature derivative value represents the rate of change of absolute curvature at the corresponding point, -- selecting a set of stroke boundaries, such that each stroke boundary lies between a point with a high curvature derivative value and a succeeding point with a low curvature derivative value, -- locating a set of strokes, such that each stroke boundary is located at the end of a stroke, -- calculating at least one stroke feature value for each stroke to produce a character feature set, -- using the character feature set to determine the identity of said handwritten character.
6. The method of Claim 5 wherein the point with a high curvature derivative value has a locally maximal curvature derivative value, and the succeeding point with a low curvature derivative value has a locally minimal curvature derivative value.
7. The method of Claim 6 wherein each stroke boundary lies on a point midway between the point with locally maximal curvature derivative value and the point with locally minimal curvature derivative value.
8. The method of Claim 5 wherein each stroke boundary lies on a point of locally maximal absolute curvature value.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US24040794A | 1994-05-10 | 1994-05-10 | |
| US08/240,407 | 1994-05-20 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2162489A1 true CA2162489A1 (en) | 1998-06-01 |
Family
ID=22906392
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002162489A Abandoned CA2162489A1 (en) | 1994-05-10 | 1995-05-03 | Method of stroke segmentation for handwritten input |
Country Status (14)
| Country | Link |
|---|---|
| EP (1) | EP0710384A4 (en) |
| JP (1) | JP2002515144A (en) |
| CN (1) | CN1128074A (en) |
| AU (1) | AU2431695A (en) |
| BR (1) | BR9506197A (en) |
| CA (1) | CA2162489A1 (en) |
| CZ (1) | CZ6196A3 (en) |
| FI (1) | FI960110A (en) |
| HU (1) | HUT75820A (en) |
| IL (1) | IL113659A0 (en) |
| NO (1) | NO955064L (en) |
| PL (1) | PL312469A1 (en) |
| SK (1) | SK3096A3 (en) |
| WO (1) | WO1995032485A1 (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB9701793D0 (en) | 1997-01-29 | 1997-03-19 | Gay Geoffrey N W | Means for inputting characters or commands into a computer |
| US7603684B1 (en) | 1998-05-19 | 2009-10-13 | United Video Properties, Inc. | Program guide system with video-on-demand browsing |
| US6563515B1 (en) | 1998-05-19 | 2003-05-13 | United Video Properties, Inc. | Program guide system with video window browsing |
| CN1317664C (en) * | 2004-01-17 | 2007-05-23 | 中国科学院计算技术研究所 | Confused stroke order library establishing method and on-line hand-writing Chinese character identifying and evaluating system |
| EP1562138B1 (en) | 2004-02-06 | 2009-08-19 | Dassault Systèmes | A process for drafting a curve in a computer-aided design system |
| EP1562130A1 (en) * | 2004-02-06 | 2005-08-10 | Dassault Systèmes | A process for modifying a curve in a computer-aided design system |
| CN100405389C (en) * | 2004-08-06 | 2008-07-23 | 摩托罗拉公司 | Identifying character from stroke mark |
| JP2006162692A (en) * | 2004-12-02 | 2006-06-22 | Hosei Univ | Automatic lecture content creating system |
| CN100338621C (en) * | 2005-04-07 | 2007-09-19 | 上海交通大学 | Eigenvalue error compensation on limited sample collection and parameter distribution correcting method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2658137B2 (en) * | 1988-03-11 | 1997-09-30 | 沖電気工業株式会社 | Character recognition method |
| JP3017740B2 (en) * | 1988-08-23 | 2000-03-13 | ソニー株式会社 | Online character recognition device and online character recognition method |
| US5590220A (en) * | 1992-08-12 | 1996-12-31 | International Business Machines Corporation | Bending point extraction method for optical character recognition system |
-
1995
- 1995-05-03 JP JP53030695A patent/JP2002515144A/en active Pending
- 1995-05-03 CA CA002162489A patent/CA2162489A1/en not_active Abandoned
- 1995-05-03 CN CN95190347A patent/CN1128074A/en active Pending
- 1995-05-03 CZ CZ9661A patent/CZ6196A3/en unknown
- 1995-05-03 PL PL95312469A patent/PL312469A1/en unknown
- 1995-05-03 WO PCT/US1995/005409 patent/WO1995032485A1/en not_active Application Discontinuation
- 1995-05-03 BR BR9506197A patent/BR9506197A/en unknown
- 1995-05-03 EP EP95918350A patent/EP0710384A4/en not_active Withdrawn
- 1995-05-03 HU HU9503882A patent/HUT75820A/en unknown
- 1995-05-03 AU AU24316/95A patent/AU2431695A/en not_active Abandoned
- 1995-05-03 SK SK30-96A patent/SK3096A3/en unknown
- 1995-05-08 IL IL11365995A patent/IL113659A0/en unknown
- 1995-12-14 NO NO955064A patent/NO955064L/en unknown
-
1996
- 1996-01-10 FI FI960110A patent/FI960110A/en not_active Application Discontinuation
Also Published As
| Publication number | Publication date |
|---|---|
| NO955064D0 (en) | 1995-12-14 |
| CZ6196A3 (en) | 1996-06-12 |
| AU2431695A (en) | 1995-12-18 |
| IL113659A0 (en) | 1995-08-31 |
| EP0710384A1 (en) | 1996-05-08 |
| CN1128074A (en) | 1996-07-31 |
| WO1995032485A1 (en) | 1995-11-30 |
| EP0710384A4 (en) | 1997-05-02 |
| HUT75820A (en) | 1997-05-28 |
| BR9506197A (en) | 1996-04-24 |
| FI960110A0 (en) | 1996-01-10 |
| NO955064L (en) | 1995-12-14 |
| SK3096A3 (en) | 1996-10-02 |
| PL312469A1 (en) | 1996-04-29 |
| FI960110A (en) | 1996-01-10 |
| HU9503882D0 (en) | 1996-02-28 |
| MX9600189A (en) | 1998-11-29 |
| JP2002515144A (en) | 2002-05-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5740273A (en) | Method and microprocessor for preprocessing handwriting having characters composed of a preponderance of straight line segments | |
| US5287417A (en) | Method and system for recognizing a graphic object's shape, line style, and fill pattern in a pen environment | |
| EP0708944B1 (en) | Method for segmenting handwritten input | |
| EP1507192B1 (en) | Detection of a dwell gesture by examining parameters associated with pen motion | |
| EP0739521B1 (en) | Method of splitting handwritten input | |
| US7515752B2 (en) | Sketch recognition and enhancement | |
| US5737443A (en) | Method of joining handwritten input | |
| EP0727754A2 (en) | Handwritten character recognition apparatus with an improved feature of correction to stroke segmentation and method for correction to stroke segmentation for recognition of handwritten character | |
| CA2162489A1 (en) | Method of stroke segmentation for handwritten input | |
| US6208756B1 (en) | Hand-written character recognition device with noise removal | |
| MXPA96000189A (en) | Method of segmentation of traces for the entry of characters manuscri | |
| JP2633523B2 (en) | Handwriting input device | |
| CA2479564C (en) | Sketch recognition and enhancement | |
| JP2000251013A (en) | Method and device for character recognition and storage medium | |
| JPH0821064B2 (en) | Character writing input device | |
| JP2004133951A (en) | Character recognition method, character recognition device, and storage medium | |
| JP2000259768A (en) | Method and device for recognizing character and storing medium | |
| JPH05189616A (en) | On-line character recognizing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| FZDE | Discontinued |