EP0210237A1

EP0210237A1 - Input device for handwritten characters

Info

Publication number: EP0210237A1
Application number: EP86901014A
Authority: EP
Inventors: Jeffrey Malcolm Macauley
Original assignee: Tybar Engineering Pty Ltd
Current assignee: Tybar Engineering Pty Ltd
Priority date: 1985-01-31
Filing date: 1986-01-30
Publication date: 1987-02-04
Also published as: WO1986004704A1

Abstract

Un stylo à écrire manuel (10) peut détecter la direction du déplacement de la pointe d'écriture (18). Les déplacements dans des directions diverses sont détectés par des commutateurs (22, 20), et le fonctionnement du ou des commutateurs appropriés peut être contrôlé par un ordinateur, de sorte que la reconnaissance des caractères devient possible.A manual writing pen (10) can detect the direction of movement of the writing nib (18). Movements in various directions are detected by switches (22, 20), and the operation of the appropriate switch or switches can be controlled by a computer, so that character recognition becomes possible.

Description

INPUT DEVICE FOR HANDWRITTEN CHARACTERS

The present invention relates to a handheld computer input device for allowing handwritten characters to be converted into machine readable form and relates particularly, although not exclusively, to a handwriting instrument that can be coupled to the input of a computer.

In the past, handwriting recognition devices have consisted of a handheld wand or stylus which is connected to a so-called "digitizing tablet". The wand or stylus is pressed onto the tablet and the x,y position of the wand or stylus on the tablet is detected by its proximity or contact with horizontal and vertical multiplexed grids within the tablet. By accurately positioning a preprinted form on the tablet, the wand or stylus can be moved over the form and the desired characters drawn on the form. The sensed x,y co-ordinates generated by successive positions of the pen are stored and' the actual handwritten characters are computed from the stored co-ordinates by suitable character recognition software. Devices of this type are shown in Australian Patent Application No. 25935/78.

The problem with these devices is that they are not really portable and the wand or stylus must be coupled to the tablet. As the tablet has limited dimensions there is a physical limit on the amount of characters that can be written on a preprinted form. Preprinted forms must often be accurately aligned with the tablet. In addition, many of the prior art digitizing tablets require characters to be written in a particular manner and/or within ^"predefined borders. It is an object of the present invention to provide a handwriting instrument which requires no digitizing tablet for computer input.

A further object of the invention is to provide a handwriting instrument capable of simultaneously producing handwritten copy on any suitable writing surface and storage within a computer memory device.

A still yet further object is to alleviate, to a substantial degree, the writing of characters in a particular manner. With these objects in view the present invention may provide--a computer input device for converting handwritten characters into machine readable form, said device including means for detecting or sensing direction of movement of said device across a writ-ing surface.

Preferably said device includes a writing tip e.g. pencil or felt tip; or alternatively a nib or ball supplied with ink from a reservoir. "

In ^'one embodiment said means includes a plurality of light responsive sensors focussed to detect ink or lead left by said device on said writing sur ace.

In a further embodiment said means includes a plurality of switch contacts inside said device actuable by a pivotted switch blade.

In yet a further preferred embodiment said means for detecting or sensing direction includes a plurality of feeler wires surrounding an ink reservoir whereby angular movement of said reservoir is detected by said feeler wires. Preferably each feeler wire is formed from a spring metal and each feeler wire is deflected from its undeflected position when contacted by said ink reservoir.

The feeler wires may, in use, be coupled, directly or indirectly, to 1/0 lines or an expansion bus of a computer means in order that the directional changes may be interpreted thereby.

In still yet a further embodiment said means includes a plurality of optically sensitive means each indicating a direction of movement and an optical source coupled to movement of a writing tip of said device whereby changes in direction of said writing tip will cause particular optically sensitive means to be activated. If a self-contained device is required the device may include computational means for determining the handwritten characters from analysis of said directions. The device may also include memory means for storing text translated by said computational means.

These and other objects and aspects of the present invention will be more fully described with reference to the preferred^' non-limitative embodiments shown in the accompanying drawings, in which:- Fig. 1 is a longitudinal cross-sectional view of a first embodiment of a handwriting instrument made in accordance with the invention;

Fig. 2 is a cross-sectional view along and in the direction of arrows 2-2 of Fig. 1; Fig. 3 is a diagrammatic electronic circuit representation showing connection of the handwriting instrument shown in Fig. 1;

Fig. 4 is a longitudinal cross-sectional view of a second embodiment of a handwriting instrument made in accordance with the invention;

Fig. 5 is a cross-sectional view along and in the direction of arrows 5-5 of Fig. 4;

Fig. 6 is a diagrammatic electronic circuit representation showing connection of the handwriting instrument shown in Fig. 4; Fig. 7 is a longitudinal cross-sectional view of a third embodiment of a handwriting instrument made in accordance with the invention;

Fig. 8 is a cross-sectional view along and in the direction of arrows 8-8 of Fig. 7;

Fig. 9 is a longitudinal cross-sectional view of a fourth embodiment of a handwriting instrument made in accordance with the invention;

Fig. 10 is a cross-sectional view along and in the direction of arrows 8-8 of Fig. 7;

Fig. 11 is an enlarged view of the circled area shown in Fig. 9;

Figs. 12A and 12B are flow charts showing one example of a character recognition algorithm; and Fig. 13 shows a pictorial representation of directions of movement of the writing instrument.

Turning to Figs. 1 to 3 there is shown a writing intsrumeήt or pen which has a body 12. A ball 14 is fitted in the end of body 12 and is held captively therein, e.g. by flange 15, to allow for swivel movement. Secured within ball 14 is an ink reservoir 16 with a ball writing tip 18 at one end thereof protruding through aperture 19. At the other end of reservoir 16 is a switch blade 20 in the form of a disk. Circumferentially located inside body 12 are a plurality of switch contacts 22. These contacts 22 are equispaced and in this embodiment there are eight such contacts. Accordingly when writing with pen 10 the swivel action of ball 14 will cause one of the switch contacts 22 to be closed by switch blade 20. Thus each change in direction of writing tip 18, while writing, will cause switch blade 20 to contact a particular switch contact 22.

The actuation of the different contacts 22 is monitored by a preprocessor mounted inside body 12, in a separate case e.g. a pocket-type device, or directly by a computer. In this embodiment there are eight switch contacts 22 to be monitored and additional switches (not shown), if required. Such additional switches may inclue a "pen write" contact switch

(confirming writing tip 18 is in contact with a writing surface) and special function switches e.g. for punctuation. The signals from pen 10 can be hardwired to the preprocessor or computer or an aerial transmission device (e.g. infra red). Fig 3 shows the' switch contacts 22 being connected to I/O lines 23 of a latching buffer or peripheral interface adapter 24 e.g. a Synertek 6522. Such an adapter can monitor the signals from pen 10 and generate interrupts to a computer when changes of direction occur. These changes can be stored by the computer for immediate or later analysis to determine the character written.

Suitable software to recognize the handwritten characters^' and to store the subsequent text is implemented on the computer or preprocessor. One method is to consider the various switch contacts as defining north, south, east and west directions and first, second, third and fourth quadrants. Thus the first quadrant will fall between the north and east directions. By giving each of the directions and quadrants a particular value the changes in direction can be sequentially stored. The data from the changes in direction can be filtered, if required, and compared with a lookup table. It has been found that the lookup table can be personalized to suit a particular writer's style and various combinations for the same character can be included in the lookup table. It has also been found that there is very little conflict in recognizing a particular character. A simple example is the letter "L" which consists of the sequential data of the south and east positions. Letters like an "S" will pass through many quadrants and directions but these changes are easily handled by the lookup table and the speed of the computer. An example of suitable software to achieve character recognition will be described later in this specification.

In the embodiment shown in Figs. 4 to 6 the switch contacts 22 and switch blade 20 are replaced by optical means. In body 30 there is a lead 32 or similar writing tip. Equispaced around the lead 32 are eight optical fibres 34 which have their inside ends coupled to a light source (not shown). To sense changes in reflected light from a writing surface eight additional optical fibres 36 are circumferentially mounted within the circle of optical fibres 34.

Photodetectors 38 are coupled to the inside ends of optical fibres 36 to provide a voltage dependent on the light reflected from the corresponding optical fibre 34. Photodetectors 38 are coupled to respective voltage comparators 40. As discussed with reference to the embodiment shown in Figs. 1 to 3 a latching buffer or peripheral interface adapter 42 is used to monitor the signals from the pen. The adapter 42 may be located externally to the pen if required. Again an aerial transmission device may be used to send the data to the computer.

In this embodiment the optical fibres 36 will be detecting the ink or lead left on the writing surface as the pen is moved thereacross. Thus a loss of reflectivity will be detected which indicates the direction travelled. The software required will be similar to that discussed with reference to the previous embodiment. Thus if the pen is travelling in an easterly direction the west direction photodetector will detect the ink or lead trail from the pen for analysis by the computer. This embodiment can detect cross-overs e.g. as occurs in the numeral "8" and the letter "Q". Suitable adjustments to the software will be necessary to allow for simultaneous detection from more than one photodetector.

Turning now to Figs. 7 and 8 there is shown a writing instrument or pen 50 which has a main outer barrel 52. Outer barrel 52 is slipped or moulded over an inner barrel 54. At the rear of pen 50 is a plug 56 to close off barrels 52 and 54. A metal ink reservoir 58 having ball point 60 is secured into plug 56 with the ink reservoir free to be deflected from its vertical position. Mounted along inner barrel 54 are eight feeler wires 62 formed from a spring metal. As shown the feeler wires 62 extend a short distance along barrel 54 and are soldered to a copper wire (not shown) which is coupled to a contact pin.64 at the rear of pen

50. A pin is also provided for the ink reservoir 58.

'. The ends of feeler wires 62 are bent ^"inwardly to adjust their position to be adjacent but not contacting ink reservoir 58.

Feeler wires 62 are equispaced about reservoir 58. Accordingly when writing with pen 50 the pivotting action of reservoir 58 will cause one of the feeler wires 62 to contact ink reservoir 58 and be deflected from its position shown in Fig 7. Thus each change in direction of writing tip 60, while writing, will cause ink reservoir 58 to contact a particular feeler wire 62. The resilient nature of the feeler wires will return the deflected wire to its normal position when a further change in direction is made and another wire is deflected.

The actuation of the different feeler wires 62 is monitored by a preprocessor mounted inside pen 50, in a separate case e.g. a pocket-type device, or directly by a computer. In this embodiment there are eight feeler wires 62 to be monitored and additional switches, if required. Such additional switches may include a "pen write" contact switch and special func- tion switches e.g. for punctuation as discussed with reference to Figs. 1 to 3. The signals from the pen can as previously discussed, be hardwired to the pre¬ processor or computer through an expansion bus, or an aerial transmission device e.g. ultrasonics, radio or infrared. Similarly the feeler wires 62 can be connected to I/O lines of a latching buffer or peripheral interface adapter as discussed in Fig. 3.

Turning to Figs. 9 to 11 there is shown a pen 70 which has a main barrel 72. At the rear of pen 70 is a plug 74 to close off the end. An ink reservoir 76 having ball point 78 is secured inside pen 70 by a plug 80. Plug 80 is formed of resilient material e.g. silicone rubber so that ink reservoir 76 may be deflected angularly and vertically. At the end of the ink reservoir opposite to ball point 78 is a light source 82 e.g. a light emitting diode (LED). Adjacent light source 82 and mounted within barrel 72 is a disk 84 having segmental opto-sensitive receivers 86 and a central opto-sensitive receiver 88. Suitable electronics and power supply to drive light source 82 and receivers 86,88 are generally designated at 99. This type of circuitry is familiar to the man skilled in the art and requires no further explanation.

Pen 70 includes switches 90 for detecting when the pen is in the writing position. On ink reservoir 76 are a pair of switch contacts 92 and 94 which co-operate with further switch contacts 96 and 98. Accordingly when pen 70 is not being used switch contacts 94 and 98 will be closed. On initial pen pressure by ball point 78 on a writing surface ink reservoir 76 will move upwardly in view of the resilient nature of plug 80 and no switch contacts will be closed. This intermediate position will indicate a "pen ready" condition. Once writing commences switch contacts 92 and 96 will be closed to enable character recognition to commence. If desired the switch contacts could be replaced by other suitable means e.g. magnetic or opto-sensitive devices.

In use the ink reservoir 76 is free to pivot as the pen is used and this pivotting action will cause light source 82 to move and accordingly activate a respective receiver 86,88 to indicate the change in direction of the pen movement. As has been previously discussed the condition of receivers 86,88 and switch contacts 92-98 can be monitored e.g. through contact pins- 100.

It is clear that light source 82 and opto- receivers 86,88 can be replaced by similar acting devices e.g. a feeler wire may replace light source 82 and opto-receivers 86,88 could be switch Contacts. Other alternative arrangements are envisaged as magnetic devices e.g. reed and Hall-effect switches, could also be used.

A basic description of the software required for character recognition from the changes in direction of movement will now be described with reference to Figs. 12A, 12B and 13. In the description of the various embodiments direction movements have been limited to 8 directions. This limit is convenient from a practical point of view but the inventive concept has no such limitations. As can be seen from Fig. 13 each direction has been assigned a particular value e.g. the north direction is $01 whilst the south direction is $ 10. (Throughout this specification reference to "$" will signify_.a number in hexadecimal notation). Accordingly for each change in the clockwise direction there is a doubling of the direction value i.e. a "left bit" shift. As previously discussed each character is recognized by the various sequential changes in direction resulting from movement of the pen. Each change in direction is checked against a list of characters kept in a "lookup" table.

Typical entries for the characters of "7" and "2" are listed below:- Entry for "7"

1 byte $37 (ASCII code for "7") 2 bytes Accumulative weight counter

2 bytes Current Table Pointer (Direction) Sequential Direction

Entries 1 2 3

Combined Directions $0E $30 $00 First weight + 5 5 0

First weight - 5 5 3

Successive matches + ^{1 *} . ° 0 0

Successive matches - ^' 2 2 3^'

Entry for "2" 1 byte $32 (ASCII code for "2")

2 bytes Accumulative weight counter

2 bytes Current Table Pointer (Direction)

Sequential Direction

Entries 1 2 3 4 Combined Directions $β'3 $0E $20 $0E $00

First weight + 0 3 3 4 0

First weight - 0 3 3 4 3

Successive matches + 0 3 0 4 0

Successive matches - 0 3 0 4 3 For each character to be recognized a similar table would be prepared. The software starts by initializing the weight results by resetting the

"Accumulative Weight Counter" for each entry to "5".

Thus for the entry for the "7" character the "Accumulative Weight Counter" would contain "5". The software filters the directional changes from the pen and then compares the direction with each character entry in the lookup table. Assuming a "7" was being drawn the pen would indicate that firstly the pen was travelling in an easterly, direction i.e. value $04 in

Fig. 13. As there is no "pen lift" (indicating the end of a particular character) the "Current Table Pointer" is reset and incremented to the first "Sequential Direction Entry", i.e. $0E. A comparison is made between the indicated direction, i.e. $0^*4, and the

"Sequential Direction Entry". This comparison is done by performing a logical AND operation on the two values, i.e. $0E .AND. $04. As the result $04 is non-zero, the directions match, and a "positive first weight" value of "5" is added to the "Accumulative Weight Counter" giving a total of $ 0A. If the comparison had failed, the "negative first weight" value of "5" would have been subtracted from the "Accumulative Weight Counter" giving a total of $00. The importance of the value of $0E for the first table entry for direction is chosen to allow for persons' whose horizontal bar for character "7" is slanted, i.e. directions $02 and $08 in Fig. 13. Accordingly the software allows for peculiarities in writing styles of individuals. Thus, in this case, movements in the $02, $04 and $08 directions will result in a match, as these directions have been added together ($02 + $04 + $08 = $0E). The software then loops back to check for further directional changes. If the direction remains the same, or stays within directions $02, $04 and $08, then no further additions will be made to the "Accumulative Weight Counter" as successive matches have a $00 positive value. Thus the horizontal bar can wander in these three directions, or any combination thereof, and still be matched. When the pen starts the angular downward movement to complete the character "7", the direction change will normally be in the $20 direction. The software for the table entry has allowed for both the $20 direction and the $10 direction giving a table entry of $30 for the second matched direction. Accordingly, assuming the pen changes to the $20 direction, the first table direction will fail to match, and the "Table Direction Pointer" is changed to point to the next direction i.e. $30.

As there is a match the "positive first weight" value of "5" will be added to the "Accumulative Weight Counter" giving a total of $0F. Subsequent matches for this direction will not change the "Accumulative Weight Counter" as the "positive successive match" value is $00. If, as outlined in the previous paragraph, no match occurred the "negative first weight" value of "5" _will be subtracted from the "Accumulative" Weight Counter".^' Thus, if there was no match for the first and second directions, the

"Accumulative Weight Counter" would be negative. As can be seen from the flow chart, if the "Accumulative Weight Counter" becomes negative that character is skipped completely for further direction matching. In this manner the software has a reduced selection from which to determine a particular character match.

At the completion of formation of the downward slash of the "7" the pen will be lifted. A pen lift results in a $00 direction and this is shown as the third entry in the table. Final weights are added to (or subtracted from) the "Accumulative Weight Counter" ($00 if the "7" has been completed). The software then checks for the highest value in the "Accumulative Weight Counter" for each character entry to select the table character which best fits the input data. The character recognition software then proceeds to recognize the next written character. As can be seen from a comparison of the entries for the "2" and "7" characters, the number of directional changes determines the number of

"Sequential Direction Entries". For a simple character like a "7" very few directions are involved for a match, whereas the multiple direction changes for a "2" are more involved, in view of the "loopy" nature of the character.

From the above it is clear that the table entries may be personalized to an individual allowing peculiarities in writing to be accommodated. It is also possible to have several different entries for the same character. Experimentation has shown that the software can detect characters fairly accurately, once fine-tuned to an individual's writing- style. • The invention has particular application for oriental type scripts. It is believed that the invention and many of its attendant advantages will be understood from the foregoing description and it will be apparent that various changes may be made in the form, construction and arrangement of the parts and that changes may be made in the form, construction and arrangement of the input device described without departing from the spirit and scope of the invention^*or sacrificing all of its material advantages, the form hereinbefore described being merely preferred embodiments thereof.

Claims

The claims defining the invention are as follows:-

1. A computer input device for converting handwritten characters into machine readable form, said device including means for detecting or sensing direction of movement of said device across a writing surface.

2. The computer input device as claimed in claim 1, further including a writing tip, especially a pencil or felt tip.

3. The computer input device as claimed in claim 1, further including a nib or ball supplied with ink from a reservoir.

4. The computer input device as claimed in claim 2 or 3, wherein said means includes a plurality of light responsive sensors focussed to detect ink or lead left by said device on said writing surface.

5. The computer input device as claimed in any one 'of claims 1 to 3, further including a longitudinal hollow body member having a plurality of direction sensors -therein, an activation means attached to the free end of an extension of said writing tip for activating at least one of said plurality of direction sensors and said writing tip being pivotally, swivellably or resiliently mounted in said longitudinal hollow body member.

6. The computer input device as claimed in claim 5, wherein said activation means is a switch blade and said direction sensors are switch blades.

7. The computer input device as claimed in claim 5, wherein said activation means is an optical source and said direction sensors are opto-sensitive devices.

8. The computer input device as claimed in claim 5, wherein said activation means is a magnetic member and said direction sensors are magnetic-responsive members.

9. The computer input device as claimed in any one of claims 5 to 8, wherein said writing tip is secured within said longitudinal hollow body member by a swivelling ball arrangement.

10. The computer input device as claimed in any one of claims 5 to 8, wherein said writing tip is secured in a resilient flexible material for allowing pivotal and longitudinal deflections of said writing tip.

11. The computer input device as claimed in any one of claims 5 to 10, wherein said extension is the ink reservoir.

12. The computer input device of any one of claims 5 to 11, wherein said direction sensors may, in use, be coupled directly, or indirectly, to I/O lines or an expansion bus of a computing means in order that the directional changes may be interpreted thereby.

13. The computer input device as claimed- in any one of claims 1 to 3, wherein said means includes a plurality of switch contacts inside said device actuable by a pivotted switch blade.

14. The computer input device as claimed in claim 3, wherein said means for detecting or sensing direction includes a plurality of feeler wires surrounding said ink reservoir whereby angular movement of said reservoir is detected by said feeler wires.

15. The computer input device as claimed in claim

14, wherein each feeler wire is formed from a spring metal and each feeler wire is deflected from its undeflected position when contacted by said ink reservoir.

16. The computer input device as claimed in claim

15, wherein feeler wires may, in use, be coupled directly, or indirectly, to I/O lines or an expansion bus of a computer means in order that the directional changes may be interpreted thereby.

17. The computer input device as claimed in any preceding claim, wherein said device is self-contained and includes computational means for determining the handwritten characters from analysis of said directions.

18. The computer input device as claimed in claim 17, further including memory means for storing text translated by said computational means...

19. The computer input device as claimed in claim 12, wherein said indirect connection to said computing means is by ultrasonics, radio or infrared.

20. A method of character recognition using a computer input device as claimed in any preceding claim, said method including the steps of monitoring the changes in direction of said writing tip and comparing the sequential changes in direction with a set of stored entries for each written character to^" be recognized to establish the particular written character.

21. A computer input device substantially as hereinbefore described with reference to the accompanying drawings.

22. A method of character recognition substantially as described herein.