WO1995019013A1

WO1995019013A1 - Method and apparatus for interpreting handwritten symbols in a computer system

Info

Publication number: WO1995019013A1
Application number: PCT/US1994/014980
Authority: WO
Inventors: Michael Libes; William J. Adams; Neal B. Lesh; Nicholas Kushmerick
Original assignee: Nimble Corporation
Priority date: 1994-01-06
Filing date: 1994-12-29
Publication date: 1995-07-13
Also published as: AU1596695A

Abstract

Disclosed is a method and apparatus for translating character symbols drawn on a computer system of the type including a touch-sensitive screen that produces a position indication signal indicating where the character was drawn on the touch screen (90, 94 and 100). A bounding box is calculated having a width equal to the maximum horizontal space occupied by the symbol and a height equal to the maximum vertical space occupied by the symbol (104). The system detects horizontal line, vertical line, angular stroke and circle stroke (110, 112, 118). The system also analyzes each point in the stroke and produces search string indicative of where each point in the stroke lies within the rectangular grid (102).

Description

METHOD AND APPARATUS FOR INTERPRETING HANDWRITTEN SYMBOLS IN A COMPUTER SYSTEM

Related Application The present application is a continuation-in-part of our copending application serial No. 08/178,218, filed January 6, 1994, titled Method and Apparatus for Interpreting Handwritten Symbols in a Computer System, which is expressly incorporated by reference herein.

Field of the Invention The present invention relates to handwriting recognition systems in general, and in particular to a method and apparatus for translating handwritten, shorthand symbols into alphanumeric characters.

Background of the Invention

As computers have become smaller and more portable, personal, hand-held microcomputers are achieving widespread popularity with consumers. Such computers currently have the ability to act as electronic date books or notepads as well as to perform more sophisticated functions such as sending and receiving facsimile data. Due to the popularity of these hand-held computers, a great deal of research and development has been directed to improving the user interfaces for these machines. In particular, considerable effort has been directed to improving the way in which data is entered into a hand-held computer.

Currently, the most common type of data entry mechanism for any type of computer system is a keyboard. Unfortunately, keyboards have not proved to be very usable with small, hand-held computers. First, the individual keys of a keyboard must have some minimum size in order for a user to be able to type comfortably. The minimum size of a keyboard is a predominant factor in determining how small a computer housing can be made. Second, keyboards are not very compatible with a graphical user interface displayed by most computer systems.

To solve the problems associated with keyboards as the data entry mechanism, newer, hand-held computers have begun to incorporate handwriting recognition technology whereby a user is provided with a pen and enters data by writing directly on an area of a touch-sensitive screen. These pen-based computers read the pen's movements and translate what was drawn on the screen into alphanumeric characters using a character recognition program. Typically, once the handwritten character has been translated, the alphanumeric equivalent is displayed on the screen for the user to see and manipulate.

The problem with recognizing handwritten characters is the large variation in the way different people write the same letter. Therefore, most currently available handwriting recognition programs include sophisticated algorithms for analyzing what was drawn on the screen and translating the handwriting character to an alphanumeric character. Such algorithms typically operate by looking for unique features in a handwritten character and associating the unique features with a particular alphanumeric character. Due to the complexity of the feature recognition algorithms, most character recognition programs require a high level of computing power and large amounts of computer memory—neither of which is available in most pen-based computers.

In an effort to increase the speed at which characters are recognized by a pen- based computer, shorthand alphabets have been developed that are more easily recognizable by computers than standard alphanumeric characters. One such example of a shorthand alphabet is the Unistroke™ character set developed by Xerox. This alphabet contains five basic character shapes that can be oriented in four different directions and written forwards or backwards to represent a total of 40 different alphanumeric characters.

Despite the simplicity of the Unistroke™ character set, previously developed translators all use character recognition algorithms of the type referred to above to recognize these characters. Therefore, despite their advantages, shorthand character sets, such as Unistroke™, have not yet achieved widespread acceptance. This is primarily due to the size and computational requirements of prior art shorthand character set translators. Therefore, there exists a need for a method of operating a pen-based computer so that it will recognize a simplified character set using a translator program that is accurate, does not require a great deal of computer power, and does not occupy large amounts of memory.

Summary of the Invention In accordance with this invention, the problems associated with prior art character recognition systems are solved by providing a method of translating handwritten character symbols suitable for implementation in a hand-held computer system of the type that includes a touch-sensitive screen or other digitizing technology that is sensitive to an electronic pen or other writing instrument. The method controls the operation of the computer system in a way that translates alphanumeric characters quickly and without requiring sophisticated computations or large amounts of memory. The character symbols that the present invention is designed to detect include dots, horizontal and vertical "single" lines, as well as horizontal and vertical "double" lines, angular symbols such as "L" or "N" strokes, a "P" stroke, as well as circular strokes. Upon detecting that a user has written a character symbol on the touch- sensitive screen, the invention determines the maximum horizontal and vertical distance occupied by the character symbol. A bounding box is created having dimensions equal to the maximum horizontal and vertical distance determined. To detect the presence of a dot, the invention determines whether the width and height of the bounding box are less than a predetermined minimum. If so, the invention determines that the user has drawn a dot.

To detect the presence of a horizontal line, the invention determines whether the height of the bounding box is less than a predetermined fraction of the width of the bounding box. If so, the bounding box can be divided into a plurality of sections each of which is assigned a section code. A search code then is produced that is composed of the section codes traversed by the character stroke. A dictionary data structure is searched for the search code in order to determine the corresponding alphanumeric character. In a similar manner, to detect the presence of a vertical line, the invention determines if the width of the bounding box is less than or equal to a predetermined fraction of the height of the bounding box and, if so, the box can be divided, a search code assigned based on the path of the stroke, and the code can be searched in the dictionary data structure.

To detect the presence of an angular shape, the bounding box is coarsely divided into a rectangular grid that is comprised of a plurality of rectangular sections. Each rectangular section is assigned a section code. The invention analyzes each point in the character symbol to determine the section of the rectangular grid in which the point is located. As described above, a search code is produced that is composed of the section codes traversed by the character stroke. A dictionary data structure is searched for the search code in order to determine the corresponding alphanumeric character. To detect the presence of a circular shape, the bounding box is divided into a radial grid that is composed of a plurality of radial sectors. Each radial sector is assigned a radial sector number. The invention first determines the radial sector of the first and last point of the character symbol in order to determine if the first and last points are located more than 180° apart. If the endpoints are more than 180° apart, the invention determines that the user has not drawn a proper circle shape. However, if the end points are located at a position less than or equal to 180° apart, the invention then determines if the user has traced a path around the center point of the radial grid. If so, the invention indicates that the user has drawn a valid circle shape.

In the event that the end points of the character symbol are less than 180° apart but the invention does not detect a circle drawn around the center of the radial grid, the invention then searches for the crossing point of the circular shape. Once the crossing point is found, the invention calculates a new bounding box for the points that lie between the crossing point of the character symbol. A second radial grid is centered in the middle of the new bounding box and the invention analyzes whether a path has been traced around the center of the second radial grid. If so, the invention indicates that the user has drawn a valid circle shape.

Brief Description of the Drawings The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:

FIGURE 1 is a chart of the prior art Unistroke™ character set that the present invention is designed to recognize, and the mapping of the Unistroke™ character set to standard alphanumeric characters; FIGURE 2 is a block diagram of a pen-based computer system suitable for implementing the method of the present invention;

FIGURE 3 is a flow chart of the steps performed by the present invention to recognize a handwritten character;

FIGURES 4A-4C show an example of a stroke drawn by a user and how the stroke is normalized and placed within a bounding box by the present invention; FIGURE 5 is a flow chart of the steps taken by the present invention to calculate the bounding box and to determine whether a handwritten stroke is a dot, a vertical line or a horizontal line;

FIGURE 5A is a flow chart showing the steps taken by the present invention to detect a line, the direction that the line was drawn, and whether the line is a single line or a double line;

FIGURES 6, 6A and 6B show how the present invention uses a grid to recognize shapes of the Unistroke™ alphabet as well as additional strokes that can be recognized and mapped by the present invention; FIGURES 7A-7B are a series of flow charts showing the steps taken by the present invention to translate a handwritten stroke using the grids shown in FIGURE 6A;

FIGURES 8A-8C show how the present invention uses a radial grid to translate a handwritten stroke; FIGURES 9A-9B are a series of flow charts showing the steps taken by the present invention to translate a stroke using the radial grid shown in FIGURES 8A-8C;

FIGURE 10 is a flow chart showing the steps taken by the present invention to determine the sector of the radial grid in which a portion of a stroke is located; FIGURES 11A-1 ID are a series of flow charts showing the steps taken by the present invention to calculate where to relocate the center of the radial grid as shown in FIGURE 8C;

FIGURE 12 is a flow chart of the steps taken by the present invention to determine if the stroke is a proper circle shape; and FIGURE 13 is a flow chart of the steps taken by the present invention to recalculate the bounding box to translate a handwritten stroke.

Detailed Description of the Preferred Embodiment

FIGURE 1 is a chart showing the Unistroke™ character set developed by the

Xerox Corporation to facilitate data entry in a pen-based computer system. The Unistroke™ character set consists of five basic character shapes: a straight line, an

"L" shape (also referred to as a corner), a "V" shape (also referred to as a caret), an "N" or Z" shape (depending on the orientation), and a circle (also referred to as an alpha shape). These five shapes, oriented in any of four different directions, produce twenty different characters or strokes that can be mapped to an alphanumeric character. A dot is an additional character usable in the Unistroke™ system. In addition to the orientation of a Unistroke™ character, the direction in which the character is drawn determines how the character is mapped to a particular alphanumeric character. For example, the letters "c" and "d" are assigned to the same Unistroke™ character, but the "c" is drawn in the clockwise direction while the "d" is drawn in the counterclockwise direction. Similarly, the letter "t" is represented as a horizontal line drawn from left to right, while the letter "e" is represented as a horizontal line drawn from right to left. Thus, each Unistroke™ character can be drawn in two directions each of which is a different character, thereby allowing the Unistroke™ character set to represent a total of forty different letters or symbols, plus a dot. In addition, the characters can be used with different modes by using two or more modality characters such as the upper case, lower case, or symbol characters in order to represent additional alphanumeric characters.

The benefit to using a shorthand scheme such as the Unistroke™ character set is that the characters are more easily recognizable by a computer system despite variations in the way they may be written by different users. Therefore, a shorthand character set allows handwritten data to be entered more rapidly than a standard (alphanumeric) character set. As can be appreciated by reviewing the Unistroke character set, not all alphanumeric symbols are provided. For example, there is no provision for an "=" sign. To provide such symbols, the inventors have added several new characters or symbols. As will be explained in greater detail below, the new symbols include a double line (a continuous stroke drawn linearly in a first direction and then in the reverse direction) and what are referred to as "P" and "M" strokes. With the addition of these symbols, many more alphanumeric characters can be provided.

As noted above, the problem with present handwriting character recognition methods implemented in computer systems is that they are too slow, require large amounts of computational power, and occupy too much memory. Furthermore, existing character recognition programs are not extremely accurate and often require a user to redraw a letter until the computer can translate it correctly. The present invention is directed to a method of recognizing handwritten shorthand characters and translating those characters into alphanumeric letters that, when implemented in a computer system, does not have the foregoing disadvantages. As will be seen, the present invention is accurate, does not require difficult or time-consuming computations to be performed, and can be implemented by a computer program that does not occupy significant memory space. FIGURE 2 is a block diagram of a pen-based computer system suitable for implementing the presently preferred embodiment of the method of the invention. The pen-based computer system 50 shown in FIGURE 2 comprises a processing unit 60, a display unit 76, and a pen 80. The processing unit 60 includes a central processing unit (CPU) 62, a random access memory (RAM) 64, a read only memory (ROM) 66 and an input/output port (1/0) 68. Preferably the CPU 62 is a microprocessor. The CPU, RAM, ROM and I/O are all shown coupled together by a common address/data bus 70.

Coupled to the processing unit 60 through the I/O port 68 and a set of leads 74 is the display unit 76 on which the computer system displays alphanumeric and graphic characters. The display unit 76 is touch sensitive and preferably includes a digitizing tablet 78 upon which characters can be written with the pen 80. The digitizing tablet 78 may occupy the entire area of the display unit 76 or may be confined to a small portion of the display. Characters (referred to as strokes) written on the digitizing tablet 78 with the pen 80 or other instruments, including fingers, cause a position indication signal to be generated. The position indication signal is comprised of a stream of binary numbers that are read by the CPU 62 and interpreted as the X, Y coordinates of adjacent points that make up the stroke. The X, Y coordinates are typically stored in a memory array that contains, in order, the coordinates of all the points of the stroke from the time the pen was first placed on the touch-sensitive screen to the time the pen was removed from the touch-sensitive screen.

As will be described below, the present invention analyzes the array of coordinate data to translate the stroke drawn by the user into an alphanumeric character for display on the display unit 76 and or storage in the processor's memory. Although the following discussion describes how the present invention translates a Unistroke™ character into an English language alphanumeric character, those skilled in the art will appreciate that a stroke, once recognized, could be translated to virtually any symbol in any language, including mathematical symbols, chemical symbols, etc.

FIGURE 3 is a flow chart showing the steps taken by the present invention to decode a handwritten character or stroke and to translate the stroke into an alphanumeric character according to the mapping scheme shown in FIGURE 1 or any alternate mapping scheme desired. In the preferred embodiment of the invention, the method is carried out using a suitable computer program which is stored in the ROM 66 of the processing unit 60. The program begins with a step 90 that detects when a user has placed the pen on the touch-sensitive screen. At a step 94, the CPU reads the X, Y coordinate of the points that comprise the stroke and stores the coordinate data in the RAM, substantially in real time. The process of deciphering or translating the stroke begins at a step 100 when the computer system detects that the pen has been lifted from the surface of the digitizing tablet 78. The program then begins to analyze the stroke point coordinate data for all the points that comprise the stroke at a step 102.

At a step 104, the program calculates the height and width of the stroke in order to construct a bounding box encompassing the stroke. As will be discussed in further detail below, the bounding box is used by the present invention to decode the stroke by calculating the dimensions of the stroke and the position of each point in the stroke within the bounding box.

After the bounding box has been created, the program determines, at a step 106, whether the stroke is the equivalent of a dot drawn on the digitizing tablet. As can be seen from the charts on FIGURE 1, a space character is represented in the Unistroke™ character set as a dot. Therefore, if the user has drawn a dot, the deciphering program returns a character (i.e., a space) that matches the character drawn at a step 108. Additionally, the program may display the matching character on the touch-sensitive screen at a step 109 before the program ends. If the user has not drawn a dot, the program determines at a step 110 whether the user has drawn a horizontal line or a vertical line. If so, the program returns the character that matches the vertical or horizontal line drawn depending whether the stroke was drawn from left to right, right to left, top to bottom, or bottom to top. If the program does not detect that the user has drawn a horizontal or vertical line, the program proceeds to a step 112 where the bounding box is coarsely divided into a rectangular grid in an attempt to decipher the character drawn.

FIGURES 4A-4C show how the coordinates of a stroke are normalized and how a bounding box is computed to contain the stroke by the present invention. FIGURE 4A shows a depiction of a stroke drawn on the digitizing tablet. The beginning of the stroke starts at a point (15, 45), has a midpoint (67, 543), and ends at a point (642, 512). The program first normalizes the points of the stroke so the minimum X coordinate is zero and the minimum Y coordinate is zero. In the example shown in FIGURE 4B, the normalization procedure has the effect of placing the beginning of the stroke at the origin of a Cartesian coordinate system. As can be seen in FIGURE 4B, the normalization procedure is accomplished by subtracting the minimum X and Y coordinate values from each point that comprises the stroke. As a result, the end point becomes (627, 467) and the midpoint becomes (52, 498).

FIGURE 4C shows how a bounding box is constructed by the present invention around the stroke. The bounding box has dimensions just large enough to contain the stroke. Thus, in the example shown, the corners of the bounding box have the normalized coordinates (0, 0), (0, 498), (627, 498) and (627, 0). As will be discussed in further detail below, the bounding box is used by the present invention to decipher the stroke drawn by the user based upon the dimensions of the bounding box and the location of the points within the bounding box. FIGURE 5 is a more detailed flow chart of the steps 104, 106, and 110 shown in FIGURE 3. The method represented by the flow chart of FIGURE 5 operates to normalize the stroke, calculate the dimensions of the bounding box and determine whether the stroke is a dot, a horizontal line or a vertical line. Beginning at a step 150, the program scans the array of coordinate values to determine a minimum X and Y coordinate value (X^, Ymin) and a maximum X and Y coordinate value (^m_ax» Ym_a ) °f ^tne points that comprise the stroke. At a step 152, the height of the bounding box is determined by subtracting the minimum Y value from the maximum

Y value (Ym_ax - Ymin). The width of the bounding box is determined by subtracting the minimum X coordinate value from the maximum X coordinate value (Xm_a ~ -^min)- At a step 154, the points of the stroke are normalized by subtracting Xπ_ύn and Ymin from each coordinate pair that comprise the stroke.

Once the stroke has been normalized and the dimensions of the bounding box have been computed, the program determines whether the stroke should be interpreted as a dot. At a step 156, the program calculates whether the difference between the maximum X coordinate value and the minimum X coordinate value multiplied by the resolution of the digitizing tablet is less than or equal to 2 mm. In the present embodiment of the invention, strokes drawn more than 2 mm in the X direction are not defined to be dots and the program skips to step 162. However, as will be appreciated, the 2 mm value is arbitrary and could be easily varied depending upon the dimensions of the digitizing tablet, the resolution of the digitizing tablet, or other factors. If, in the described embodiment, the X dimension of the bounding box is equal to or less than 2 mm, the program proceeds to a step 158 to determine whether the difference between the maximum Y coordinate and minimum

Y coordinate multiplied by the resolution of the digitizer is less than or equal to 2 mm. If so, the program returns to the "Yes" side of step 106 in FIGURE 3, indicating the stroke is a dot. If the stroke exceeds 2 mm in either the X direction or the Y direction, the program assumes the stroke was not a dot and cycles to steps designed to determine if the stroke is a vertical or horizontal line. The presence of a vertical line is detected by determining if the width of the bounding box is less than or equal to one-third the height of the bounding box at a step 162. If so, the program returns at a step 164 indicating the stroke is a vertical line. If the stroke is not detected to be a vertical line, the program determines if the stroke is a horizontal line by calculating if the height of the bounding box is less than or equal to one-third the height of the bounding box at step 166. If so, the program returns at a step 168 indicating the stroke is a horizontal line. If the height of the bounding box is greater than one-third the width of the bounding box, the program assumes the stroke is not a horizontal line and returns at a step 170 to the step 112 shown in FIGURE 3.

Alternatively, if a line is detected as described above, the line can be analyzed to determine the direction that the stroke was made and/or whether the line was a "double line," i.e., a line drawn first in one direction and then in the reverse direction approximately back over itself. Recognizing these features increases the number of different strokes that can be used, at least some of which may be easier to draw than those of the Unistroke™ system. FIGURE 5 A is a flow chart showing in detail the steps performed to determine whether a vertical line is a single or double vertical line as well as whether the stroke was drawn top to bottom or vice versa. In general, the program interprets the stroke by dividing the bounding box into a series of sections, each of which is assigned a numeric code. The program analyzes the sequential array of coordinate data to trace the stroke through the sections. A code is built that comprises the section numbers in the order traversed by the stroke. As can be seen in FIGURE 6, to detect the presence of a single line shape, first the bounding box is divided into equal thirds, wherein each third is assigned a number 0, 1 or 2 (from the bottom to the top). The width of the bounding box is shown wider than the stroke for purposes of illustration. The program determines the sequence of section numbers through which the stroke passes by analyzing the coordinate data for each point of the stroke. The section numbers are concatenated into a search string and a dictionary data structure is used to correlate the search string to a corresponding alphanumeric character.

For example, for the single line upward stroke shown in FIGURE 6, the stroke begins in section 0 and ends in section 2. Therefore, a search code of 012 would be produced. The dictionary data structure is searched for this code. Upon finding the code, the program returns the corresponding alphanumeric character that is represented by the single upward stroke. Horizontal strokes are determined in a similar fashion, except that the bounding box is divided into three horizontal sections. As will be appreciated, the number of sections may be more or less than three, depending upon the complexity of the dictionary data structure and the speed at which the translation program is desired to operate. Also, the search can be directed to a portion of the dictionary data structure dedicated to vertical or horizontal line codes.

More specifically, as shown in FIGURE 5 A, the program begins in a loop 800 that analyzes the coordinate data for each point in the stroke. At a step 802, the program determines whether the Y coordinate of a point is less than or equal to one- third of the height of the bounding box. If the answer to step 802 is yes, then the section number in which the point is located is assigned 0. If the answer to step 802 is no, then the program determines whether the Y coordinate is less than or equal to two-thirds of the height of the bounding box at a step 806. If so, than the section number is assigned the value 1 at a step 808. If the answer to step 806 is no, then the section number is assigned the number 2 at a step 810.

At a step 812, the program determines whether there has been a change in section number since the last point was analyzed. If not, the program proceeds to the next point in the stroke. If the answer to step 812 is yes, then the program concatenates the section number into a search string at a step 814. At a step 816, the program determines whether all the points in the stroke have been analyzed. If the answer to step 816 is no, the program loops back and analyzes the next point in the stroke.

Once all the points in the stroke have been analyzed, the direction of the stroke will be known based on the code sequence. An upward single line stroke will have a search code of 012; and a single line downward stroke will have a search code of 210. A double line stroke, shown in FIGURE 6, will have a search code of 01210 or 0121 or 1210 ( to be valid, the return portion of a double line stroke must be greater than one-third of the overall length of the stroke, assuming the bounding box is divided into thirds). The program searches the dictionary data structure for the search string produced at step 814. The program returns the character that matches the search string at a step 822. The double line stroke shown in FIGURE 6 has a downward return portion that is shown offset from the upward stroke for purposes of illustration. In the preferred embodiment of the invention, the return portion is drawn directly over the first portion, but in the reverse direction. The same steps are performed to detect a single or double horizontal line.

Each point in the stroke is analyzed to determine the sections in which the point is located, each section constituting one-third of the bounding box. A search code is developed consisting of the section numbers. The search code is compared to the dictionary data structure.

Returning to FIGURE 3, if the stroke drawn by the user is not a dot, a horizontal line or a vertical line, the program divides the bounding box into a rectangular grid and uses the grid to determine whether the stroke is a diagonal line, "L" stroke (corner), "V" stroke (caret), or an "N" or "Z" stroke at a step 112. FIGURE 6A shows how a rectangular grid can be used to divide the bounding box into nine equal sections. Each section has associated with it a numeric identifier. As will be described in further detail below, the program calculates the grid section numbers through which the stroke passes. The section numbers are concatenated into a search string and a dictionary data structure is used to correlate the search string to the corresponding alphanumeric character. For example, the diagonal line shown in FIGURE 6A begins in section 0 and passes through sections 4 and 8. Therefore, upon analyzing the stroke, the program would compute a search string of 048. The dictionary data structure is then interrogated to see if it contains the search string. If the search string is in the dictionary, the program looks up the corresponding alphanumeric character. For the diagonal line drawn from the lower left to the upper right, the dictionary data structure returns the letter "k." For the corner or "L" shape shown in FIGURE 6 A, the stroke begins in section 0 and ends in section 8. The program would compute a search string of 03678 and look for this search string in the dictionary data structure. If the search string is found, the program returns the appropriate alphanumeric character. Using the Unistroke™ mapping scheme of FIGURE 1, the dictionary data structure would return a comma. In a similar fashion, the program uses the rectangular grid to analyze whether the stroke is a "V" shape or an "N" shape. Again, this is accomplished by determining the section numbers of the rectangular grid through which the stroke passes, building a search string of the section numbers, and searching the dictionary data structure for the search string.

FIGURES 7A-7B are a more detailed set of flow charts showing the steps performed by the program to complete step 112 shown in FIGURE 3. Beginning at a step 180, the program begins a loop that analyzes the coordinate data for each point in the stroke starting from the first point to determine where in the bounding box the point lies. At a step 182, it is determined whether the X coordinate of the stroke has a value less than or equal to one-third of the width of the bounding box. If so, the program knows the point is in the left third of the bounding box and assigns a value of zero to a variable that holds the section number of the rectangular grid at a step 184. If the X coordinate is greater than one-third the width of the bounding box, the program determines whether the X coordinate of the point is less than or equal to two-thirds the width of the bounding box at a step 186. If so, the program knows the point is in the middle vertical third of the bounding box and a value of 1 is assigned to the variable that holds the section number at a step 188. If the X coordinate of the point is greater than two-thirds the bounding box width, the program knows the point is in the right third of the bounding box and a value of 2 is assigned to the variable that holds the sector number at a step 190.

After analyzing the X coordinate of the stroke point, the program determines if the Y coordinate of the point is less than or equal to one-third the bounding box height at a step 192. If so, then the program knows the point is in the bottommost section of the bounding box and the section number is unchanged. If the Y coordinate is greater than one-third the bounding box height, the program then determines if the Y coordinate is less than or equal to two-thirds the bounding box height at a step 194. If so, the program knows the point is in the middle horizontal third of the bounding box and adds 3 to the variable that holds the section number at a step 196. If the Y coordinate is greater than two-thirds the height of the bounding box, the program knows the point is in the upper third of the bounding box and adds 6 to the variable that holds the section number at a step 176. After thusly determining in which section of the grid the stroke point is located, the program proceeds to the flow chart shown in FIGURE 7B. Beginning at a step 200, the program determines whether there has been any change in the section number since the last pass through the steps illustrated in FIGURE 7A and described above. If not, the program loops back to step 180 and analyzes the next point in the stroke. If the section number has changed, the program concatenates the section number in a search string at a step 202. At a step 204, it is determined whether all the points that comprise the stroke have been analyzed. If not, the program loops back to step 180 and analyzes the next point in the data array.

If all points in the data array have been analyzed, the program searches the dictionary data structure that maps the search strings to particular alphanumeric characters. As described above, the diagonal line shown in FIGURE 6A that begins in the lower left-hand corner and ends in the upper right-hand corner produces a search string of 048. Searching the dictionary for this string will return the letter "k." However, other search strings that would return the letter "k" would include 0348, 03458, 01478, etc., depending upon how straight the user drew the diagonal line. Therefore, the dictionary data structure must contain all possible search strings that can validly map to a particular alphanumeric character. The details of how to construct such a dictionary data structure are considered within the skill of one of ordinary skill in the art of computer programming and therefore will not be discussed in further detail. At a step 210, a test is made to determine whether the search string is found in the dictionary data structure. If so, the program returns the matching alphanumeric character corresponding to the search string. If not, the program returns to the next step 114 shown in FIGURE 3 and described next because the stroke is not a diagonal line, a corner or "L" stroke, a caret or a "V" stroke, or an "N" or "Z" stroke. FIGURE 6B shows how a five by five rectangular grid can be used to divide a bounding box into 25 sections. As described above with reference to a three by three grid, each section is assigned a unique code or number. Thereafter, the angular stroke is analyzed as described above, for the more specific grid. An initial estimate of the section in which a point is located is determined based on the X coordinate of the point. If the X coordinate has a value between 0 and 1/5 of the bounding box width, then the section number is assigned a zero. X coordinates between 1/5 and 2/5 of the bounding box width are assigned a section number 1. Similarly, X coordinates between 2/5 and 3/5 of the bounding box width are assigned 2, between 3/5 and 4/5 are assigned 3 and greater than 4/5 of the bounding box width are assigned 4. After analyzing the X coordinate of the stroke point, the program calculates the section number based on the Y coordinate of a point. If the Y coordinate is between 0 and 1/5 of the bounding box height, nothing is added to the section number. If the Y coordinate is between 1/5 and 2/5 of the bounding box height, 5 is added to the section number. Y coordinates between 2/5 and 3/5 of the bounding box height add 10 to the section number. Coordinates between 3/5 and 4/5 add 15 to the section number, while those Y coordinates between 4/5 and the height of the bounding box add 20 to the section number. As will be appreciated, the logic of determining the section number is obviously dependent on the number of sections in a grid and the way in which each section is numbered. The sequence of section numbers through which a stroke passes is formed to constitute the search code.

As previously described, if all points in the data array have been analyzed, the program searches the dictionary data structure that maps the search strings to particular alphanumeric characters. In a five by five grid, a diagonal line of the type shown in FIGURE 6A that begins in the lower left corner and ends in the upper right corner produces a search string of 0, 6, 12, 18, 24 if the line was drawn at exactly 45° to the dimensions of the bounding box. Searching the dictionary for this string will return the letter "k." However, other search strings that would return the letter "k" would include 0, 6, 12, 17, 18, 23, 24 , etc., depending upon how straight the user drew the diagonal line. Therefore, the dictionary data structure must contain all possible search strings that can validly map to a particular alphanumeric character. FIGURE 6B illustrates additional stroke shapes that can be used in addition to the Unistroke™ shapes or be substituted for the Unistroke™ shapes which are most difficult to master. The "M" stroke can be drawn in different orientations, namely, with the acute angles at the top, to the right, at the bottom, or to the left, and also can be drawn in different directions so as to correspond to eight different alphanumeric characters. The "P" stroke includes a single line, and a double line drawn approximately perpendicular from an end of the single line, and also can be drawn in four different orientations and in different directions. For example, the upper "P" stroke shown in FIGURE 6B starts at the bottom left of the bounding box and progresses with a single upward directed vertical line, then to a double horizontal line first proceeding toward the right at the top; whereas the lower "P¹" stroke shown in FIGURE 6B begins at the upper left and proceeds with a single downward vertical line followed by a horizontal double line that first proceeds toward the right and returns toward the left. A diagonal double line stroke extends linearly in a first direction and returns linearly in a second direction that is substantially opposite to the first direction. These additional "M" and "P" and diagonal double line strokes can be interpreted as described above by sequentially analyzing the section numbers of each point to develop a search code consisting of the section numbers through which the stroke passes, which then is compared to the dictionary data structure to determine the corresponding alphanumeric character.

Referring again to FIGURE 3, if the program is not able to decipher the stroke using the rectangular grid and the dictionary data structure, the program calculates a radial grid from the center of the bounding box at a step 114 and determines if the stroke is a circle or alpha shape. The first test made by the program to determine if a stroke is a valid circle or alpha shape is to determine whether the end points of the stroke are on the same side of the radial grid at a step 116. If the end points are on opposite sides of the radial grid (i.e., more than 180° apart), the stroke cannot be a valid circle shape and the program produces an error message at a step 132 indicating the stroke was not recognized. If the end points are on the same side of the radial grid, the program determines at a step 118 if the stroke points of the stroke trace a circle around the center of the radial grid. If so, the program returns the appropriate character that matches the stroke, depending upon whether the stroke was drawn in the clockwise or counterclockwise direction at a step 108.

FIGURES 8A-8C show in detail how the radial grid is used to decipher a circle or alpha shape. As can be seen in FIGURE 8A, the bounding box is divided into eight sectors that are consecutively numbered in the counterclockwise direction. Upon detecting that the end points of the stroke are on the same side of the radial grid, the program determines whether the stroke traces a circle around the center of the radial grid. If so, the program determines the stroke is a valid circle shape and returns the appropriate alphanumeric character. For example, in FIGURE 8 A, the program returns a letter "x."

It is possible that even if the user has drawn the stroke correctly, it cannot be decoded using a radial grid that is centered in the middle of the bounding box. FIGURE 8B shows a stroke that also corresponds to the letter "x"; however, the circular portion of the stroke does not contain the center of the radial grid. Therefore, the test performed in step 118 of FIGURE 3 would fail. In order to avoid this result, the present invention operates to move the center of the radial grid to the center of the looped or enclosed portion of the stroke as shown in FIGURE 8C and then again evaluates the loop to determine if the user has completed a circle around the center of the radial grid. As can be seen, the stroke drawn in FIGURE 8C does complete a circle around the center of the radial grid and therefore the program correctly determines that the user has drawn a letter "x."

FIGURE 9A shows the steps taken by the present invention to decode a circle shape drawn by a user. To begin with, the center of the radial grid is placed in the center of the bounding box. At a step 220, the coordinates for the center of the radial grid are determined by dividing the width of the bounding box by 2 and the height of the bounding box by 2. At a step 222, the slope of the diagonal lines that divide the 4 quadrants of the radial grid is determined by dividing the height of the bounding box by its width. Given the center of the bounding box and the slope of its diagonal lines, the program calculates in which sector of the radial grid the first point in the stroke is located and in which sector of the radial grid the last point of the stroke is located at a step 224.

Figure 10 shows the steps taken by the program to calculate in which sector of the radial grid a particular point in the stroke is located. Beginning at a step 280, the program calculates the coordinates of the center of the radial grid (X center, Y center) and the slope of the diagonal line that creates the radial grid (as determined by steps 220 and 222 in FIGURE 9A). At a step 282, the program transposes the coordinates (X, Y) of the point being analyzed to a new coordinate system having the origin at the center of the radial grid. Two local variables X' and Y are created by subtracting the center points of the radial grid from the X and Y coordinates of the point under consideration.

At a step 284, it is determined whether both X and Y¹ are greater than or equal to 0. If so, the point under consideration must be in the first quadrant of the radial grid as conventionally measured in the counterclockwise direction, i.e., the quadrant composed of sections 0 and 1 (FIGURE 8 A). At a step 286, it is determined whether the Y¹ coordinate is above or below the diagonal line that divides the first quadrant. This is accomplished by calculating whether Y is greater than the quantity

(slope*X). If so, the program knows the coordinate must be in sector 1. If the answer to block 286 is no, the program assigns 0 to the sector number at a step 290.

At step 292, it is determined whether the point being analyzed is in the second quadrant (composed of sections 2 and 3) by calculating whether X is less than 0 and is greater than or equal to 0. If so, it is then determined at a step 294 whether the

Y coordinate is above or below the diagonal line that divides the second quadrant. Again, this is accomplished by calculating whether Y is greater than the quantity - (slope*X) at step 294. If so, the program assigns 2 to the sector number at step 296. If the answer to step 294 is no, the program assigns 3 to the sector number at a step 298.

If the answers to steps 284 and 292 are no, the program determines whether the point under consideration is in the third quadrant (composed of sections 4 and 5) at a step 300. This is accomplished by determining whether both X and Y¹ are less than 0. If so, it is then determined whether the Y coordinate is above or below the diagonal line that divides the third quadrant by calculating whether Y¹ is greater than the quantity (slope*X) at step 302. If so, the program assigns 4 to the sector number at a step 304. If the answer to step 302 is no, the program assigns 5 to the sector number at a step 306. Finally, if the point under consideration is not in the first, second or third quadrants, the program knows that the point must lie in the fourth quadrant (composed of sections 6 and 7) and therefore can determine the sector number by calculating whether Y¹ is greater than the quantity - (slope*X) at a step 308. If the answer to step 308 is yes, then 7 is assigned to sector number at a step 310. If the answer to step 308 is no, then 6 is assigned to the sector number at a step 312. Once the sector value has been determined, the program returns the sector number to the flow chart of FIGURE 9A at a step 316.

Once the program has determined in which sector the first and last points of the stroke are located, the program then determines whether the sectors of the first and last points are the same at a step 226. If the answer to step 226 is yes, then the program determines at a step 228 whether the first point is located in sector 1 or sector 2. If the answer to step 228 is yes, then the program assumes that the stroke drawn by the user must be either the character that corresponds to the letter "x" or the character that corresponds to the letter "o" depending whether the stroke was drawn in the clockwise or counterclockwise direction. After determining an initial estimate of what the stroke drawn by the user may be, the program proceeds to the flow chart shown in FIGURE 12, wherein it is determined if the user has completed a valid circle around the center of the radial grid. As will be described below, the flow chart shown in FIGURE 12 not only determines if the user has completed a valid circle, but also determines the direction of the stroke (clockwise or counterclockwise) thereby allowing the program to determine the proper alphanumeric character corresponding to the stroke drawn. Before detailing the steps of FIGURE 12, the remainder of FIGURE 9 is explained to describe how the program makes an initial estimate of the character based on the location of the first and last endpoints of the stroke. If the answer to step 228 is no, the program then determines whether the sector of the first point is equal to 3 or 4. If so, the program assumes that the stroke corresponds to the letter "p" as shown in FIGURE 1 and proceeds to FIGURE 12. If the first point is not located in sector 3 or 4, the program determines at a step 236 whether the first point is located in sector 5 or 6. If so, the program assumes that the stroke drawn by the user corresponds to an alpha shape that is located 90° to the right (at the present time, there is no mapping of this shape to an alphanumeric character in the Unistroke™ character set). However, in order to recognize this character the program continues to the flow chart shown in FIGURE 12 to make sure that the user has drawn a valid circle around the center of the radial grid. If the answer to step 236 is no, the program assumes that the stroke drawn by the user corresponds to the letter "q" as shown in FIGURE 1 and proceeds to the flow chart shown in FIGURE 12.

If the sector of the first and last point in the stroke are not equal, the program proceeds from step 226 to step 244 wherein it is deteπnined if the first and last points in the stroke are in sectors 0, 1, 2, or 3, i.e., in the top half of the bounding box. If so, the program proceeds to a step 246 wherein the stroke is decoded as the character that corresponds to the letter "x" or "o" depending upon the direction in which the stroke was drawn. If the answer to step 244 was no, the program proceeds to step 248 wherein it is determined if the first and last points of the stroke lie between sectors 2 and 5, i.e. , in the left half of the bounding box. If so, the stroke is decoded as the Unistroke character corresponding to the letter "p" at a step 254 before the program proceeds to the flow chart of FIGURE 12.

If the answer to step 248 is no, the program determines if the first and last end points are located in sectors 0, 1, 6, or 7 (in the right half of the bounding box). If so, the stroke is decoded as the character corresponding to the letter "q" before the program proceeds to the flow chart of FIGURE 12.

If the answer to step 252 is no, the program determines at a step 256 whether the first and last end points of the stroke lie in sectors 4, 5, 6, or 7 (in the bottom half of the bounding box). If so, then the program assumes the shape drawn by the user is an alpha shape rotated 90° to the right for which there is not yet implemented a corresponding character in the Unistroke™ character set before proceeding to FIGURE 12.

If the end points of the stroke are not located on the same side of the radial grid, the program assumes the user has not drawn a valid circle shape and declares the stroke as unrecognized at a step 260 and the program will end at step 132 of FIGURE 3 by declaring the stroke cannot be recognized.

As indicated above, even if the ends of the stroke are located on the same side of the radial grid, the program determines whether the user has drawn a full circle around the center of the radial grid in order to confirm that a valid circle shape has been drawn and, if so, to determine the direction of the stroke. This is done with the flow chart shown in FIGURE 12. Therefore, at a step 264 (FIGURE 9B), the program determines if a user has drawn a valid circle. If so, the program returns the character determined by the endpoints of the stroke at a step 268. If the user has not drawn a circular path around the center of the radial grid, the program jumps to step 120 of FIGURE 3 and begins scanning for the crossing point of the stroke in order to move the center of the radial grid.

Turning now to FIGURE 12, the program begins as a loop at a step 580 that analyzes each point in the stroke beginning with the first point in the stroke. At a step 582, the sector number of each point is determined according to the steps shown in FIGURE 10. At a step 584, it is determined whether the sector number of the point under consideration has changed from the previous point under consideration. If not, the program gets the next point in the stroke at a step 586 and determines its sector number. This process continues until the sector number changes.

In order to be a valid circle, the sector number cannot change by more than 3 at any given time. In the presently preferred embodiment of the invention, sector 7 is contiguous with sectors 6 and 0. Therefore, a change from sector 7 to sector 0 is considered a change of 1 sector in the counterclockwise direction. The program checks this at a step 588. If the sector number has changed by more than 3, an error is declared at step 590 indicating that the stroke cannot be recognized and the program then returns at step 591. As will be appreciated, the constant 3 by which the sector number cannot change is selected based on the number of sectors, resolution of the digitizer, etc., and may be changed if necessary.

If the sector has not changed by more than 3, the program determines whether the sector number has decreased at a step 600. If the answer to step 600 is yes, the program knows the stroke is moving in a clockwise direction and a variable is set to indicate the direction is moving clockwise. If the answer to step 600 is no, then the program sets the direction variable equal to counterclockwise at a step 604. At a step 606, it is determined whether the direction has changed from its previous direction. If the answer is yes, then an error is declared at step 610 indicating the stroke is not recognized and the program returns at step 611. If the direction has not changed, the program determines whether all points in the stroke have been analyzed at a step 608. If not, the program loops back to step 586 and the next point in the stroke is analyzed.

Once all the points in the stroke have been analyzed, the program determines whether the direction in which the stroke was drawn was clockwise. If so, the system returns the alphanumeric character corresponding to the clockwise version of the stroke determined in FIGURE 9. If the direction was counterclockwise, the system returns the counterclockwise version of that stroke at step 616. As will be appreciated, if an alphanumeric character is not assigned to a stroke detected, the program returns an error message indicating an invalid shape was drawn. As described above and shown in FIGURE 8B, it is possible that a user can draw a valid circle shape that is not detected checking to see if a circle has been drawn around the radial grid that is centered in the middle of the bounding box. Therefore, the program of the present invention tries to center the radial grid within the points enclosed by the circle shape drawn by the user. The flow charts of FIGURES 11A-11D describe the steps performed by the program to calculate the point at which the stroke crosses itself and to calculate a new bounding box that contains the points located between where the stroke crosses in one direction and recrosses in the opposite direction as is shown in FIGURE 8C.

To determine where the stroke crosses, the program first assigns two variables F and L that are pointers to the array of coordinates that comprise the stroke. The variable F holds the memory location of the first point in the stroke while a variable L holds the memory location of the last point in the stroke in the array at a step 350. At a step 352, it is determined whether the Y coordinates of both the first and last points are less than two-thirds the height of the bounding box. If so, the program assumes the alpha character was drawn in the orientation shown in FIGURE 11 A (with the tail ends at the bottom) begins scanning for the crossing point from the bottom of the bounding box toward the top. In general, pointer F contains the address in memory of the first set of coordinates representing the beginning of the stroke. The F pointer is advanced along the sequence of coordinates from the first set of coordinates in the array to the second, then the third, and so on. Pointer L is "advanced" from the sequence of coordinates representing the end of the stroke, from the last coordinates to the next-to-last, and so on. Advancement of the pointers is incremental, the pointer associated with the coordinates having the smallest Y designation (i.e., the lower of the two) always having priority. Eventually, it will be discovered that the coordinates associated with both pointers have the same X value, in which case the crossing point has been found. In the absence of a crossing point, the F and L pointers contain the same memory address in the array, in which case an error is declared.

Before the crossing point can be located, it must be determined whether the stroke is drawn in the clockwise or counterclockwise direction. This is accomplished at a step 354 by determining whether the X coordinate of the first point is less than the X coordinate of the last point. If the answer to step 354 is yes, the program assumes the stroke is drawn in the counterclockwise direction. The program then begins looking for the crossing point.

Beginning at a step 356, it is determined if the pointers F and L contain the same memory address in the array. If the answer to step 356 is yes, then the program declares an error at a step 358 indicating the stroke was not recognized and proceeds to step 126 of FIGURE 3 in order to scan the stroke horizontally for the crossing point. At the outset, the answers to step 356 will be "no" and the program proceeds to step 360 that deteπriines whether the X coordinate of the first point is greater than or equal to the X coordinate of the last point. Again, the answer to step 360 will initially be "no". When the answer to step 360 is yes, the crossing point of the stroke has been found and processing proceeds to the flow chart shown in FIGURE 13 that recalculates the center of the radial grid at the points contained between the F and L pointers.

If the answer to step 360 was no, the program determines whether the

Y coordinate of the set pointed to by F is less than or equal to the Y coordinate of the set pointed to by L. If so, then the F pointer is advanced to the next memory location in the array. If the answer to step 366 is no, then the L pointer is advanced to the previous memory location in the array. This process continues until the X coordinate of the set pointed to by F is greater than or equal to the X coordinate of the set pointed to by L as detected by a step 360 or the test in step 356 succeeds. If the answer to step 354 was no, the program assumes the step was drawn in the clockwise direction. The program begins looking for the crossing point by first determining at a step 380 whether the F and L pointers are pointing at the same memory location in the array. If so, an error is declared at step 382 indicating the stroke is unrecognized and the program proceeds to step 126 of FIGURE 3 in order to scan the stroke horizontally for the crossing point. If the answer to step 380 is no, the program then determines whether the X coordinate of the set pointed to by F is less than or equal to the X coordinate of the set pointed to by L at step 384. If so, the program proceeds to the flow chart of FIGURE 13 to confirm that the user has drawn a circular path around a radial grid that is centered in a new bounding box that encircles the closed portion of the stroke between the F and L pointers.

If the answer to step 384 is no, the program then determines whether the

Y coordinate of the set pointed to by F is less than the Y coordinate of the set pointed to by L at step 390. If so, the F pointer is advanced to the next memory location in the array at step 392. If the answer to the step is no, then the L pointer is advanced to the previous memory location in the array at step 394. This process continues until the answer to step 384 is yes or until the F and L pointers point to the same memory location in the array.

If the Y coordinates of the first and last points in the array are not less than two-thirds the height of the bounding box, the program proceeds to the flow chart shown in FIGURE 1 IB and the stroke is scanned from the top to the bottom looking for the crossing point. In that case, the pointer F or L associated with the higher Y value has priority in the incremental movement through the array of coordinates.

Before scanning the stroke from the top of the bounding box to the bottom of the bounding box, the program determines whether the stroke is drawn in the clockwise or coimterclockwise direction. This is determined by step 420 which checks if the X coordinate of the set pointed to by F is less than the X coordinate of the set pointed to by L. If so, the program knows the stroke is drawn in the clockwise direction. The program then begins searching for the crossing point of the stroke.

Beginning at a step 422 the system determines if the F and L pointers are pointing to the same memory location in the array. If so, an error is declared at step 424 indicating the stroke was unrecognized and processing proceeds to step 126 of FIGURE 3 to scan the stroke horizontally for the crossing point. If the answer to step 422 is no, the program determines whether the X coordinate of the set pointed to by F is greater than or equal to the X coordinate of the set pointed to by L at a step 426. If the answer to step 426 is yes, the crossing point has been found and the program can proceed to FIGURE 13 to determine if the user has drawn a circular path around the center of the radial grid.

If the answer to step 426 is no, the program determines whether the Y coordinate of the set pointed to by F is greater than the Y coordinate of the set pointed to by L at step 432. If the answer to step 432 is yes, the program advances the F pointer to the next memory location in the array at a step 434. If the answer to step 432 is no, then the L pointer is advanced to the previous memory location in the array at a step 436. This process continues until the X coordinate of F exceeds the X coordinate of L or until the F and L pointers point to the same memory location in the array.

If the answer to step 420 is no, the program assumes that the stroke was drawn in the counterclockwise direction. The program then determines the crossing point by deteπnining if the F and L pointers point to the same memory location in the array at a step 440. If yes, an error is declared at step 442 indicating that the stroke was unrecognized and processing proceeds to step 126 of FIGURE 3 in order to scan the stroke horizontally for the crossing point. If the answer to step 440 is no, the program determines whether the X coordinate of the set pointed to by F is less than or equal to the X coordinate of the set pointed to by L at a step 444. If the answer to step 444 is yes, the program proceeds to verify the user has drawn a proper circle shape at a step 448 according to the flow chart of FIGURE 13.

If the answer to step 444 is no, the program then determines at a step 450 whether the Y coordinate of the set pointed to by F is greater than the Y coordinate of the set pointed to by L. If so, the F pointer is advanced to point to the next memory location in the array at a step 452. If the answer to step 450 is no, the L pointer is advanced to the previous memory location in the list at a step 454. Again, this process continues until the X coordinate of the set pointed to by F is less than or equal to the X coordinate of the set pointed to by L or the F and L pointers point to the same memory location in the array.

If the stroke is not deciphered by scanning vertically for the crossing point, the stroke is scanned horizontally for the crossing point at step 126. If the crossing point is found, a new radial grid is calculated and a test is made to determine whether the user has drawn a circle around the center of the new radial grid. If this test fails, the stroke is declared unrecognized by the program.

FIGURE 11C-11D show the steps taken by the program to horizontally scan the stroke for the crossing point. The initial determination of whether to scan from right to left or left to right is made in step 460, wherein it is determined if the X coordinates of the first and last points are less than or equal to two-thirds the width of the bounding box. If so, the program scans the stroke from left to right, priority in the incremental movement of F and L being given to the one associated with the smaller X coordinate. The crossing point of the stroke being determined when the Y coordinate of the set pointed to by F is less than or equal to the Y coordinate of the set pointed to by L.

Beginning at a step 462, it is determined whether the stroke is drawn in the clockwise or counterclockwise direction. The program determines this by checking to see if the Y coordinate of the set pointed to by F is greater than the Y coordinate of the set pointed to by L. If so, the program assumes the shape is drawn in the counterclockwise direction. The program then searches for the crossing point by first determining whether F and L point to the same memory location in the array at a step 464. Initially, the answer to step 464 will be no, and the program proceeds to a step 468. If the answer to step 464 is yes, an error is declared at a step 466 indicating the stroke is unrecognized and the program proceeds to step 132 of FIGURE 3 and processing stops.

If the answer to step 464 is no, the program determines at a step 468 whether the Y coordinate of the set pointed to by F is less than or equal to the Y coordinate of the set pointed to by L. If yes, the crossing point has been found and the program proceeds to FIGURE 13 to verify the user has drawn a circular path around the center of the radial grid.

If the answer to step 468 is no, the crossing point has not been found and it is determined at a step 474 whether the X coordinate of the set pointed to by F is less than the X coordinate of the set pointed to by L. If so, the F pointer is assigned to the next memory location in the array at a step 476. If the answer to step 474 is no, the L pointer is set to point to the previous memory location in the array at a step 478 and the process continues until the crossing point is found at a step 468 or the answer to step 464 is yes.

If the answer to step 462 is no, then the program assumes the shape was drawn in the clockwise direction. The program then searches for the crossing point of the stroke by first determining whether the F and L pointers point to the same memory location in the array at a step 480. Again, the answer to step 480 will initially be no. If the answer to step 480 is yes, an error is declared at step 482 indicating the stroke is unrecognized and the program then proceeds to step 132 of FIGURE 3 and processing stops. If the answer to step 480 is no, then it is determined whether the Y coordinate of the set pointed to by F is greater than or equal to the Y coordinate of the set pointed to by L at a step 484. If the answer to step 484 is yes, then the crossing point has been found and the program proceeds to the flow chart shown in FIGURE 13 to verify the user has drawn a valid circle shape at step 488. If the answer to step 484 is no, the program determines whether the

X coordinate of the set pointed to by F is less than the X coordinate of the set pointed to by L at step 490. If the answer is yes, then the F pointer is advanced to the next memory location in the array at step 492. If the answer to step 490 is no, then the L pointer is advanced to the previous memory location in the array at a step 494. Again, this process continues until either the F and L pointers point to the same memory location in the array or the crossing point is found as determined by step 484.

If the answer to step 460 is no, then the program scans the stroke from right to left according to the steps shown in FIGURE 1 ID. Before the program can begin to search for the crossing point, it is first necessary to determine whether the stroke is drawn in the clockwise or counterclockwise direction. The program determines this at step 510 by checking to see if the Y coordinate of the set pointed to by F is greater than the Y coordinate of the set pointed to by L. If the answer to step 510 is yes, the program assumes that the shape was drawn in the clockwise direction and begins to look for the crossing point. In the same manner as was described above, the program looks for the crossing point in the stroke by first determining whether the F and L pointers point to the same memory location in the array at step 512. If the answer to step 512 is yes, then an error is declared at step 514 indicating that the stroke is unrecognized. The program then proceeds to step 132 of FIGURE 3 and processing stops. If the answer to step 512 is no, then it is determined whether the Y coordinate of the set pointed to by F is less than or equal to the Y coordinate of the set pointed to by L. If the answer to step 516 is yes, then the crossing point has been found and the program proceeds to the flow chart shown in FIGURE 13 to verify the user has properly drawn a circle. If the answer to step 516 is no, the program determines whether the X coordinate of the set pointed to by F is greater than the X coordinate of the set pointed to by L at a step 522. If the answer to step 522 is yes, then the F pointer is advanced to point to the next memory location in the array at a step 524. If the answer to step 522 is no, then the L pointer is advanced to the previous memory location in the array at a step 526. Again, this process continues until either the F and L pointers point to the same memory location in the array or the crossing point is found as determined by step 516.

If the answer to step 510 is no, then the program assumes the stroke is drawn in the counterclockwise direction. The program then searches for the crossing point by first determining if F and L point to the same memory location in the array at a step 530. Initially, the answer to step 530 should be no, but if the answer to step 530 is yes, then an error is declared at step 532 indicating the stroke is unrecognized and the program proceeds to step 132 of FIGURE 3.

If the answer to step 530 is no, then it is determined at a step 534 whether the Y coordinate of the set pointed to by F is greater than or equal to the Y coordinate of the set pointed to by F. If the answer to step 534 is yes, then the crossing point has been found and the program performs the steps of the flow chart of FIGURE 13. If the answer to step 534 is no, then it is determined at a step 540 whether the X coordinate of the set pointed to by F is greater than the X coordinate of the set pointed to by L. If the answer is yes, then the F pointer is advanced at a step 542 to the next memory location in the array. If the answer to step 540 is no, then the L pointer is set to the previous memory location in the array at a step 544. Again, this process continues until either the F and L pointers point to the same memory location in the array or until the crossing point is detected by step 534.

As shown in FIGURE 8C, once the crossing point of the stroke has been determined, a new bounding box is created to enclose those points that lie in the closed portion of the stroke (i.e., those points that are between the F and L pointers to the array). FIGURE 13 operates to calculate the new bounding box and the center of the new radial grid. Beginning at a step 640, the new bounding box for the points between the F and L pointers is completed. This is accomplished using the steps 150-154 as shown in FIGURE 5. Once the new bounding box has been calculated, the center of the radial grid is determined at a step 642 by placing the center point in the middle of the new bounding box. The X coordinate of the new radial grid is determined by dividing the width of the new bounding box by two and adding the minimum X coordinate, Xm^-,, of the points contained in the enclosed portion of the stroke. Similarly, the Y coordinate is determined by dividing the height of the new bounding box by two and adding the minimum Y coordinate, Ymin, of the points contained in the enclosed portion of the stroke.

The slope of the diagonal lines that divide the quadrants of the new radial grid is computed by dividing the height by the width of the new bounding box at a step 644. Once the center of the radial grid and the slope of the diagonal lines that define the new radial grid have been determined, the program proceeds to the steps shown in FIGURE 12 to determine whether the user has created a valid circle shape according to the steps described above except that the new bounding box and radial grid are used.

When the program returns from performing the steps shown in FIGURE 12, the program tests whether the user has drawn a proper circle shape at a step 648. If the answer is yes, the program returns the character that matches the stroke at step 658. If the stroke is not a proper circle shape, the program determines whether the stroke has been scanned both vertically and horizontally for the cross point at step 650. If the answer to step 650 is yes, the program returns to step 132 and processing stops. If the answer to step 650 is no, the program proceeds to step 126 and scans the stroke horizontally for the crossing point.

As can be seen in FIGURE 8C, moving the center of the radial grid allows the program to detect that a user has drawn a proper circle shape that could not be detected when the center of the radial grid was placed in the center of the original bounding box. Thus, by finding the crossing point, creating a new bounding box, and placing the center of the radial grid in the middle of the new bounding box, the program can detect shapes that would otherwise not be translated by simply checking the end points of the stroke and determining if the user has drawn a circle shape around the middle of the bounding box.

As can be seen, the present invention operates to translate hand-drawn characters in a way that is simple, does not require large or complex computations, and can be easily coded in a computer program. The present invention is thus able to quickly and accurately detect handwritten characters in a pen-based computer system with a speed and accuracy that has not been obtainable with prior art handwriting recognition programs. Although the present embodiment of the invention has been disclosed with respect to translating the Unistroke™ character set, those skilled in the art will realize that other shapes could easily be detected. For example, by encoding other shapes into the dictionary data structure, other shapes can be detected with the rectangular grid. Similarly, the radial search system as described above could be used to detect other shapes besides the circle or alpha shapes contained in the Unistroke™ alphabet such as a number "8." Thus, the present invention is not limited to detecting just the Unistroke™ character set but could be used to detect many possible characters that could be written with a pen-based computer system.

While the preferred embodiment of the invention has been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the invention.

Claims

The embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:

1. A method of translating a handwritten symbol drawn on a computer system of the type that includes a touch-sensitive screen that produces a position indication signal indicative of where on the touch-sensitive screen a user has written the symbol, the method comprising the steps of: reading the position indication signal to determine a first point and a last point of the symbol; reading the position indication signal to determine a maximum horizontal distance and a maximum vertical distance occupied by the symbol on the touch-sensitive screen; determining from the maximum horizontal distance and the maximum vertical distance an area occupied by the symbol on the touch-sensitive screen; dividing the area occupied by the symbol into a plurality of sections, each section being identified by a section code; reading the position indication signal to determine the sections of the area in which the symbol is drawn; producing a search code that indicates the sections of the area in which the symbol is drawn; and matching the search code to an alphanumeric character that corresponds to the symbol drawn on the touch-sensitive screen.

2. The method of Claim 1, wherein the step of matching the search code to an alphanumeric character further comprises the step of: searching a dictionary data structure that relates one or more search codes to a particular alphanumeric character.

3. The method of Claim 1, wherein the step of deterrnining an area occupied by the symbol further comprises the step of: computing a bounding box that contains the symbol, the bounding box having a width that is substantially equal to the maximum horizontal distance occupied by the symbol and a height that is substantially equal to the maximum vertical distance occupied by the symbol.

4. The method of Claim 3, wherein the symbol is comprised of a plurality of consecutive points and the position indication signal is indicative of a vertical and a horizontal coordinate of each point on the touch-sensitive screen, and wherein the step of determining the sections of the area in which the symbol is drawn further comprises the steps of: dividing the width of the bounding box into a plurality of horizontal segments; dividing the height of the bounding box into a plurality of vertical segments; reading the position indication signal to determine in which horizontal segment the horizontal coordinate of each point in the symbol is located and to determine in which vertical segment a vertical coordinate of each point in the symbol is located; determining the section code from the horizontal and vertical segments in which a point of the symbol is located; and concatenating the section codes to produce the search code.

5. The method of Claim 3, wherein one of the handwritten symbols drawn on the touch-sensitive screen is a dot, the method further comprising the step of determining if the symbol drawn is a dot by: deteπnining if the height and width of the bounding box are less than a predetermined minimum value and, if so, indicating the symbol drawn is a dot.

6. The method of Claim 4, wherein one of the symbols drawn on the touch-sensitive screen is a circular shape, the method further comprising the step of determining if the symbol drawn is a circular shape by: dividing the bounding box into a radial grid that comprises a plurality of radial sections, each radial section being identified by a radial section code; determining in which radial section the first and the last points of the symbol are located; deteπnining whether the radial section of the first point is located at a position less than 180° from the radial section in which the last point is located; and if the radial section of the first and last points are less than 180° apart, analyzing the position indication signal to determine if the points of the symbol trace a path around a center point of the radial grid.

7. The method of Claim 6, further comprising the steps of: analyzing the position indication signal to determine a crossing point of the symbol such that two points in the symbol have substantially the same horizontal and vertical coordinates; analyzing the position indication signal to determine a maximum horizontal distance and a maximum vertical distance occupied by the points of the symbol that lie between the crossing point; computing a second bounding box having a width substantially equal to the maximum horizontal distance occupied by the points that lie between the crossing point and a height substantially equal to the maximum vertical distance occupied by the points that lie between the crossing pomt; locating a center of a second radial grid at a center of the second bounding box; and analyzing the position indication signal to determine if the points in the symbol that lie between the crossing point trace a path around the center of the second radial grid and, if so, producing an indication that the symbol is a circle shape.

8. The method of Claim 4, wherein one of the symbols drawn on the touch-sensitive screen includes a vertical line, wherein the method further includes the step of deteπnining if the symbol drawn is a vertical line by: determining if the width of the bounding box is less than or equal to a predetermined fraction of the height of the bounding box.

9. The method of Claim 4, wherein one of the symbols drawn on the touch-sensitive screen is a horizontal line, and wherein the method further includes the step of determining if the symbol is a horizontal line by: determining if the height of the bounding box is less than or equal to a predeterrnined fraction of the width of the bounding box.

10. A pen-based computer system for translating a handwritten symbol into an alphanumeric character, comprising: a touch-sensitive screen including means for producing a position indication signal that is indicative of where on the touch-sensitive screen a user has drawn the symbol; a central processing unit coupled to the touch-sensitive screen for receiving the position indication signal produced by the touch-sensitive screen; a memory storage unit coupled to the central processing unit for storing a computer program that causes the central processing unit to: determine from the position indication signal a first point and a last point of the symbol; determine from the position indication signal a maximum horizontal distance and a maximum vertical distance occupied by the symbol on the touch-sensitive screen; determine from the maximum horizontal distance and the maximum vertical distance an area occupied by the symbol on the touch-sensitive screen; divide the area occupied by the symbol into a plurality of sections, each section being identified by a section code; read the position indication signal to determine the sections of the area in which the symbol is drawn; produce a search code that indicates the sections of the area in which the symbol is drawn; and match the search code to an alphanumeric character that coπesponds to the symbol drawn on the touch-sensitive screen.

11. The pen-based computer system of Claim 10, wherein the computer program that causes the central processing unit to match the search code to the alphanumeric character further causes the central processing unit to search a dictionary data structure stored in the memory storage unit, the dictionary data structure relating one or more search codes to a particular alphanumeric character.

12. The pen-based computer system of Claim 10, wherein the computer program that causes the central processing unit to determine an area occupied by the symbol further causes the central processing unit to: compute a bounding box that contains the symbol, the bounding box having a width that is substantially equal to the maximum horizontal distance occupied by the symbol and a height that is substantially equal to the maximum vertical distance occupied by the symbol.

13. The pen-based computer system of Claim 12, wherein the symbol is comprised of a plurality of consecutive points and the position indication signal is indicative of a vertical and a horizontal coordinate of each point on the touch- sensitive screen, and wherein the computer program that causes the central processing unit to determine the sections of the area in which the symbol is drawn further causes the central processing unit to: divide the width of the bounding box into a plurality of horizontal segments; divide the height of the bounding box into a plurality of vertical segments; read the position indication signal to determine which horizontal segment the horizontal coordinate of each point in the symbol is located and to determine in which vertical segment a vertical coordinate of each point in the symbol is located; determine the section code from the combination of the horizontal and vertical segments in which a point of the symbol is located; and concatenate the section codes to produce the search code.

14. The pen-based computer system of Claim 12, wherein the one of the handwritten symbols drawn on the touch-sensitive screen is a dot, the computer program further causes the central processing unit to determine if the symbol drawn is a dot by: deteπnining if the height and width of the bounding box are less than a predetermined minimum value and, if so, to indicate that the symbol drawn is a dot.

15. The pen-based computer system of Claim 12, wherein one of the symbols drawn on the touch-sensitive screen is a circular shape, the computer program further causing the central processing unit to determine if the symbol drawn is a circular shape by: dividing the bounding box into a radial grid that comprises a plurality of radial sections, each radial section being identified by a radial section code; determining in which radial section the first and the last points of the symbol are located; determining whether the radial section of the first point is located at a position less than or equal to 180° apart from the radial section in which the last point is located; and if the radial section of the first and last points are less than or equal to 180° apart, analyzing the position indication signal to determine if the points of the symbol trace a path around a center point of the radial grid.

16. The pen-based computer of Claim 15, wherein the computer program further causes the central processing unit to: analyze the position indication signal to determine a crossing point of the symbol such that two points in the symbol have substantially the same horizontal and vertical coordinates; analyze the position indication signal to determine a maximum horizontal distance and a maximum vertical distance occupied by the points of the symbol that lie between the crossing point; compute a second bounding box having a width substantially equal to the maximum horizontal distance occupied by the points that lie between the crossing point and a height substantially equal to the maximum vertical distance occupied by the points that lie between the crossing point; locate a center of a second radial grid at the center of the second bounding box; and analyze the position indication signal to determine if the points in the symbol that lie between the crossing point trace a path around the center of the second radial grid and, if so, produce an indication that the symbol is a circle shape.

17. The pen-based computer system of Claim 12, wherein one of the symbols drawn on the touch-sensitive screen is a horizontal line, and wherein the computer program further causes the central processing unit to determine if the symbol is a horizontal line by: determining if the height of the bounding box is less than or equal to a predetermined fraction of the width of the bounding box.

18. The pen-based computer system of Claim 12, wherein one of the symbols drawn on the touch-sensitive screen includes a vertical line, wherein the computer program further causes the central processing unit to determine if the symbol drawn is a vertical line by: determining if the width of the bounding box is less than or equal to a predetermined fraction of the height of the bounding box.

19. A method of inputting data into a computer system of the type that includes a touch sensitive screen that produces a position signal indicative of where on the touch sensitive screen a user has drawn a character, comprising the steps of: drawing on the touch sensitive screen a character that comprises a first portion that extends substantially linearly in a first direction and a second portion that extends linearly in a second direction that is substantially opposite the first direction; detecting when the user has drawn the character; and translating the character to an alphanumeric character and storing the alphanumeric character in a memory of the computer system.

20. A method of inputting data into a computer system of the type that includes a touch sensitive screen that produces a position signal indicative of where on the touch sensitive screen a user has drawn a character, comprising the steps of: drawing on the touch sensitive screen a character that comprises a first portion that extends linearly in a first direction, a second portion at an end of the first portion that extends linearly in a second direction that is perpendicular to the first direction and a third portion that extends in a third direction substantially opposite to the second direction; detecting when the user has drawn the character; and translating the character to an alphanumeric character and storing the alphanumeric character in a memory of the computer system.

21. A computer system including: a touch sensitive screen that produces a position signal indicative of where on the touch sensitive screen a user has drawn a character; a memory; means for detecting whether a user has drawn a character that comprises a first portion that extends linearly in a first direction and a second portion that extends substantially over the first portion in a reverse direction; and means for translating the character into an alphanumeric character and storing the character in the memory.

22. A computer system including: a touch sensitive screen that produces a position signal indicative of where on the touch sensitive screen a user has drawn a character; a memory; means for detecting whether a user has drawn a character that comprises a first portion that extends linearly in a first direction, a second portion at an end of the first portion that extends linearly in a second direction perpendicular to the first direction and a third portion that extends over the second portion in a direction substantially opposite to the second direction; means for translating the character into an alphanumeric character and storing the character in the memory.

23. A method for encoding a symbol written on a touch screen for accessing an associated character of a character set stored at positions in memory each associated with a code sequence, said method including the steps of storing the sequence of position on said touch pad which are activated during the writing of said symbol, forming a bounding box about said symbol, dividing said bounding box into a plurality of sections, each of said sections having a code associated with it, determining the succession of said sections corresponding to said sequence of position for forming a code sequence representative of said successions of sections and selecting a character of said character set stored at a position in memory corresponding to the code sequence formed.

24. A method as in Claim 23 wherein the sequence of positions encountered during the writing of a symbol on said touch pad corresponds to a first linear succession of sections of said bounding box followed by a second succession which is the reverse of said first succession.

25. A method as in Claim 23 wherein the sequence of positions encountered during the writing of a symbol on said touch pad corresponds to a first succession of sections of said boundary box which is linear followed by a second succession of sections extending perpendicularly from the end of said first succession, said second succession being followed by a third succession but in a reverse direction.

26. A method as in Claim 23 wherein said bounding box is rectangular and said sections comprise a grid within said rectangle.

27. A method as in Claim 23 wherein said bounding box is a circle and said sections are wedge-shaped formed by radii of said circle.

28. A pen-based computer including a touch screen and a memory, said memory including a character set, each character of which is stored at a position corresponding to a code sequence, said computer also including means responsive to the formation of a symbol on said screen for recording the consecutive positions on said screen encountered during the formation of said symbol, said computer also including a means for forming a boundary box about said symbol, means for dividing said boundary box into a plurality of sections each having an associated position code, means for forming a code sequence representative of the sequence of said sections corresponding to said consecutive positions, and means for selecting the character of said character set corresponding to the code sequence so formed.