GB2087616A - Apparatus and Method for Generating an Alpha Numeric Output from Shorthand - Google Patents

Abstract

The deaf can be helped if speech can be immediately reproduced in written form on a display. The invention allows a shorthand writer to write speech on a tablet 10 which after processing in a microcomputer (12, 16, 22 and 26) is reproduced on a VDU 38. Each shorthand outline representing a word is broken down into features representing consonants (or vowels) by detecting angular discontinuities and the feature is recognised mainly by its angular orientation. The string of features found for each outline is converted into a string of phonemes according to predetermined rules and the phonemes are transliterated to give the final output text. <IMAGE>

Description

SPECIFICATION Apparatus and Method for Generating an Alpha Numeric Output from Shorthand The present invention relates to apparatus and methods for providing an output, for example in the form of a readable display or a printout from a shorthand input.

An existing aid to the deaf is described by A. C. Downton in his paper "Speech Transcription as a Communication Aid for the Deaf", Child: care, health and development, 1979, Volume 5, pages 41 to 48. In this aid the output from a Palantype shorthand machine is electronically processed to provide a transliterated display equivalent to the shorthand keyed into the machine by the Palantype operator.

With this aid a transcript of verbatim speech is available to deaf persons and this is clearly very useful in almost any situation where the deaf person is at a disadvantage by not being able to hear speech.

One disadvantage of this apparatus is however that a Palantype operator is required and such operators are relatively scarce.

The Palantype machine and its associated electronic equipment car also be used in any situation where there is a need for a record of verbatim speech, such as in court proceedings, but it should be mentioned that the display or printout generated is often graphically incorrect and far from grammatically correct, as yet. However the output is readable by most adults although a little training is helpful.

A brief descrition of Pitman's New Era Shorthand is now given as a basis for explaining the invention. This shorthand is phonetically based; words are described by their sound rather than their spelling. The set of approximately forty English speech sounds is divided into two convenient subsets; the phonetic vowels and consonants.

Consonants are essentially the subset of sounds produced by an obstruction of the air passage; for example, the 'P' sound in 'peat' or the 'S' sound in 'seat'. The group is further subdivided in to 'voiced' or 'unvoiced' consonants, depending on whether the vocal chords contribute to the final sound.

In the shorthand notation, each of these consonant sounds is assigned a simple, single stroke, geometric symbol, generally either a straight line or shallow curve written in one of several orientations. The thickness of the written stroke is used to denote voicing. The essential principle of the shorthand is that the consonant sounds are regarded as the most important fraction of a word to be recorded. Hence, the New Era representation of a spoken word consists mainly of the appropriate consonant symbols sequentially joined to form a pseudo-cursive outline.

Vowels are the subset of sounds produced without an obstruction of the air passage. This group includes the sound of 'A' as in 'ate', and the sound of 'I' as in 'kite'. In this notation, vowels are mainly represented by dots and dashes; again written as one or two distinct thicknesses. The vowel symbols are written in one of three pre-defined positions alongside the consonant symbols that they complement within the main outline. Quite often, vowels will not be added to every outline, particularly when recording fast speech.

The complete Pitman phonetic alphabet is given in the book "Pitman Shorthand New Course", New Era Edition, Pitman publishing International Standard Book Number 0 273 42231 6 and is illustrated in Tables la and Ib. In Table I, the sound is that generally attributed to the grapheme(s) shown in capitals; the arrowhead attached to each stroke denotes the direction of writing only and is not actually written. Also for vowels the column headed "Position" refers to the vowel position with respect to the consonant that it complements, i.e.

'Ist, ,2nd, I 3rd.

In addition to this range of basic symbols, there are a variety of convenience symbols available to represent quickly common phonetic and graphemic forms. In particular, these include specialised notation for common prefixes, suffixes, and compound consonants such as TR; PL, BL, etc. These forms include hooks, loops or circles, attached to the consonant symbol that they modify within the main outline. A further range of very abbreviated outlines, called short forms are used to record very common words and enable a substantial increase in recording speed. Normally, each spoken word is given a discrete Pitman outline, but at high speeds, it is generally accepted that short forms may be attached to the other short forms or word outlines. This process is called 'phrasing'.

Table la-Consonants

Sound Notation Pitman form Peter /P/ # Beat /B/ # Type /T/ Doll /D/ # eaCH /CH/ # Judge /J/ # Fill /F/ # Vow /V/ # THought /TH/ # THe IDHI Say /S/ # Zoo /Z/ # SHip /SH/ # pleaSure /ZH/ # Cork /K/ # Go Moat, beaM /M/ d a d u r c h g e k e n n z e i c h n e t Not, iN /N/ goNG /NG/ Late, fiLL /L/ # Water /W/ # Yule Red /R1/ aRm /R2/ # Heat /H/ # # Table Ib-Vowels (Including Dipthongs)

Sound Notation Pitman form Position kite /I/ V 1st fOII /OY/ > 1st psALm /AH/ # 1st hAt /AI/ . 1st mAUI /AW/ # 1st pOp /OI/ - 1st cIAY /A/ # 2nd sEt /EI/ 2nd sIOpe /O/ # 2nd rOUgh /UI/ - 2nd abOUT /OW/ # 3rd fUme /U/ # 3rd sIEEp /E/ # 3rd untII, sillY /II/ . 3rd sUE /OO/ # 3rd shOUld /OU/ - 3rd A stenoghrapher may also use strokes to represent words by placing a particular stroke, such as one representing the first phonetic consonant of a word, so that it intersects the preceding outline.

Such strokes are known as "intersections".

According to a first aspect of the present invention there is provided apparatus for generating a readable display from a written shorthand input, comprising graphical input means for producing signals representative of the position of a stylus used in writing shorthand on an input surface of the graphical input means, analysis means for analysing the output signals from the graphical input device according to rules based on a predetermined type of shorthand to derive output signals representative of symbols written in the predetermined shorthand on the said surface of the graphical input means, and display means for displaying characters derived from output signals of the analysis means.

The predetermined shorthand may for example be a Pitman shorthand or a Gregg shorthand, or a special shorthand designed with machine recognition in mind.

The display means may include a visual display unit or a printer for example; and it will be understood that the term 'stylus' means any suitable writing means.

Advantageously, the analysis means is adapted to derive its output signals from intermediate signals representing shorthand symbols by recognising and transliterating text represented by the intermediate signals.

One of the main advantages of the present invention is that any shorthand writer can, perhaps with a little additional training, use the apparatus and the availability of shorthand writers is much greater than that of Palantype operators. Further persons likely to use the apparatus are more likely to be motivated to learn shorthand than they would be to become a Palantype operator, in view of the much greater field of use of shorthand.

A further application of the present invention is seen in word processing where the display provided is sufficiently understandable if used for example for internal memoranda and internal reports.

The advantage again being that such reports and memoranda can be written directly by a shorthand writer without the usual process of typing from shorthand notes.

When the predetermined shorthand is Pitman's New Era or Pitman's 2000, the analysis means may be adapted to determine boundaries between strokes representing consonants by detecting angular discontinuities in shorthand outlines representing words. Detection may be carried out from vectors representing increments of the outline. Since increments of outlines can be made available in X and Y co-ordinate directions from many types of graphical input means, a function representing angular discontinuity at a point in an outline can be formed, for example by summing (for each coordinate direction) a predetermined number of signed unit increments over the outline, or a portion thereof including the point. The magnitude of the function at each point is then a good indication of angular discontinuity and therefore probability of a stroke boundary at that point.

The analysis means may be adapted to recognise hooks and loops in the written shorthand and to simplify signals representing outlines by removing those signals representing hooks and loops before determining boundaries between strokes.

For the two types of Pitman shorthand specifically mentioned above, the analysis means may for example be adapted to calculate values for one or more of the following features of each stroke in order to recognise the character corresponding to the stroke.

(a) the angle of the stroke relative to a predetermined arrangement of sectors, (b) the area enclosed by the stroke and a line joining the beginning of the stroke to the end thereof (a sign may be allotted to the area in dependence upon the direction of curvature of the stroke), (c) the length of the stroke, and (d) average pressure applied to the pen or stylus making the stroke.

The analysis means may then compare each value calculated with reference values for the strokes in order to effect stroke recognition.

When analysis means is also constructed to carry out transliteration, the transliteration may employ logical processes based upon spelling rules which depend not only upon other characters recognised in an outline but also upon the position of the character in the outline.

Furthermore where the predetermined shorthand includes a number of short form symbols each representing a certain word then the analysis means may be adapted to recognise such symbols and provide an output representative of the appropriate word.

If intersections are used, as is usual, the analysis means is adapted to detect each intersection and provide output signals representing the word corresponding to that intersection.

The analysis means may also be adapted to recognise certain predetermined combinations of consonants and the position of the combination in a word and, if necessary, to insert a symbol representing the likely presence of a vowel (or vowels) among the further characters of such a predetermined combination.

According to a second aspect of the invention there is provided a method of generating a readable display including writing with a stylus in a predetermined type of shorthand on an input surface of a graphical input means which generates output signals representative of the position of the stylus on the said surface, analysing the output signals of the graphical input means according to rules based on the predetermined type of shorthand to recognise which shorthand symbols were written on the said surface, and displaying characters derived from the symbols so recognised.

The characters displayed may be obtained by transliterating text represented by the shorthand symbols recognised.

Certain embodiments of the invention will now be described by way of example, with reference to the accompanying drawings, in which: Figure 1 is a schematic block diagram of apparatus according to the invention.

Figures 2a and 2b together form a flowchart of the ASTRID algorithm used by the microprocessor of Figure 1.

Figures 3a, 3b and 3c are flowcharts of interrupt routines used by the apparatus of Figure 1.

Figures 4a, 4b and 4c together form a flowchart of the "READ" algorithm used by the microprocessor of Figure 1.

Figures 5a, 5b together form a flowchart of a subroutine for detecting and classifying hooks and loops.

Figures 6a and 6b together form a flowchart of a subroutine used in the "READ" algorithm.

Figures 7a and 7b are diagrams illustrating measures used in segmenting and recognising strokes.

Figures 8a, 8b and 8c together form a flowchart of a subroutine used in the "READ" algorithm.

Figures 9a and 9b together form a flowchart of a subroutine for sectoring shorthand strokes.

Figures 1 Oa and 1 Ob together from a flowchart of the "WRITE" algorithm used by the microprocessor of Figure 1.

Figures 1 a and 1 b together form a flowchart of an algorithm for inserting vowels.

Figure 12 shows an input menu for operator use in Figures 1 3a, 1 3b, 1 3c and 1 3d show shorthand strokes used in explaining the process of shorthand feature to phoneme conversion.

In Figure 1 a graphical input tablet 10, for example of the "Quest Datapad" made by Quest Automation Limited, Ferndown, Dorset, is connected to a universal asynchronous receive and transmit (UART) circuit 11 forming part of a single card computer 12. In this embodiment an intrumented pen 13 which provides an output signal represeritative of pressure applied to the pen is used to write shorthand on the tablet 10 and its output signal is applied to the UART 1 A printer 14 which prints a transliterated printout of alphanumeric characters corresponding to the shorthand written on the tablet 10 is also connected to the UART 11.

The single card computer 1 2 includes a central processing unit (CPU) 1 5 formed by a Z80 microprocessor which is coupled to the UART by way of a data bus 14 and interrupt connections 21.

The CPU 1 5 is connected to an S-100 busboard 1 6 comprising a bus structure 1 7 in which connections are, in this embodiment, divided into three groups: an address bus, a data bus and a control bus. The CPU 1 5 is connected to the address bus, data bus and control bus by groups of connections 1 8, 1 9 and 20, respectively.

The single card computer 12 may be of the type known as the Cromemco Single Card Computer which includes 1 K bytes of random access memory (RAM) and up to 8K bytes of programmable readonly memory (PROM).

Programs, subroutines and look-up tables are stored in a read only memory (ROM) 22 coupled to the data bus, address bus and control bus by groups of connections 23, 24 and 25, respectively. Some programs, subroutines and look-up tables may also be stored in the 8K PROM. The Cromemco 32K Bytesaver card is suitable for use as the ROM 22 and may include up to 32K of PROM.

A random access memory (RAM) 26 is mapped into an input first-in-firstout (FIFO) 27, an output FIFO 28 and an area for storing variables 29. The RAM 26 is coupled to the data bus, address bus and control bus by groups of connections 31, 32 and 33, respectively. A Wamemco Mem 1 8K static RAM card is suitable for use as the RAM 26.

An additional visual display corresponding to shorthand written on the tablet is provided by a visual display unit (VDU) 38 which is coupled to a memory mapped VDU interface card 34 (which may be of the type Solid State Music VB1-B). The interface card generates a standard video signal from data written into the onboard display RAM by the processor 1 5 via the data, address and control buses; 35, 36 and 37, respectively.

The tablet 10 and the pen 1 3 each include an interface for connection to the UART 11, allowing signals corresponding to shorthand outlines to be passed by way of the CPU 15 and the bus 17 to the input FIFO 27. Similarly information from the output FIFO passes by way of the bus 17, the CPU 1 5 and the UART 11 to the printer 14. Programs carried out by the CPU 1 5 which are stored in the ROM 22, can be interrupted when information is to pass from the tablet or pen to the FIFO 27 or from the FIFO 28 to the printer 14 by means of the interrupt connections 21 between the UART 11 and the CPU 1 5.

It will be understood that the general circuit arrangement shown in Figure 1 is well known in microprocessor engineering, and that all the integrated circuits and circuit boards shown are available as commercial items.

Depending on the graphical input tablet used as the tablet 10 and the interface connected thereto, different sequences of bytes containing control information and the location of the point of contact of a pen on the writing surface of the tablet, may be generated following each contact of the pen with the writing surface and while the pen moves in contact with the surface. In the following example the sequence contains a predetermined control byte to signify that contact has been made between the pen and the writing surface, information bytes representing the X and Y co-ordinates of the point of contact of the pen when a new contact is made (such points are known as the origins of outlines), and information bytes representing X and Y increments of pen movement.

Each of these latter bytes forms a differential vector relating to an increment of pen movement along the writing surface.

In order to supervise the segmentation of signals from the tablet representing outlines into strokes representing characters, the recognition of strokes and transliteration, the microprocessor is programmed according to an algorithm entitled "ASTRID" and a flowchart for this algorithm is shown in Figures 2a and 2b. "ASTRID" calls a number of subroutines which are explained below, and continues to operate unless interrupts are received for example from the tablet in order to read information from the tablet into the FIFO.

Referring now to the flowchart of Figures 2a and 2b, certain characteristics of the present stenographer's shorthand including usual size and pressure ranges are measured in operation 51 and entries are made in an operation 52 to the operator's personal lexicon held by microprocessor storage, these entries concerning short forms and intersections used by that stenographer. The operation 52 and the compiling of the operator's lexicon are discussed in more detail later.

A search is now carried out for a new outline origin in operation 53 by testing for a control byte (signifying a new contact), from the input FIFO 27 (and if required for other fixed bits near the beginning of each outline sequence). In operation 54 those parts of the memory which are to contain the variables representing measurements and calculations carried out on a new outline are reset to zero.

The next output from the input FIFO 27, which is a differential vector, is read in operation 55 and examined in test 56 to see whether it represents the end of an outline. If not then, in an operation 57, the data from the vector and other similar vectors is accumulated in a linear processing buffer, forming part of the RAM 29, where it is arranged in a suitable order for processing. The spatial limits of the outline are next updated in an operation 58, that is limits, held in storage, of a "box" round the outline are modified as necessary. This "box" is used to distinguish new outlines and to recognise vowels associated with outlines.

When the end of an outline is reached as indicated by a control byte supposedly signifying a new outline, a positive output from the test 56 occurs, the bytes representing the co-ordinates of the next new origin are isolated in an operation 60 and then tested to decide whether a position error has occurred (test 61-for example: is the new origin near the last pen position), or whether the second part of a current outline has occurred (test 62-for example is the new origin within the current "box"), the second part of the outline perhaps representing a vowel or an intersection or a prefix or a suffix. The "vowel" flag is then set in operation 64 signifying that a second part of the outline has to be processed when the end of the first part is reached.

The next differential vector is now read in operation 65 and tested in operation 66 to determine whether a new outline has occurred. If not the data from the new vector is decoded and transferred to the linear buffer mentioned above in operation 67. Similarly, the spatial limits of the outline are updated in operation 68. These operations are continued until a new outline is detected in test 66 when the next origin is located in an operation 69 and a test 70 is performed to check the total number of vowels, intersections, prefixes and suffixes already assigned to this outline. Unless the number exceeds four a loop 72 causes a jump back to operation 61.

In summary operations and tests 53 to 70 delimit the next complete shorthand outline and, rejecting spurious data, transfer the data in a form suitable for processing to a linear buffer.

The complete outline is now analysed by means of a subroutine "READ" called in operation 73 and signals representing characters recognised in the outline are output by means of a subroutine "WRITE" called in operation 74. Should a new outline have been recognised by means of tests 61 and 62 operations 64 to 70 are omitted by means of a forward jump 75 to the point where the "READ" subroutine is called in operation 73.

After the subroutines "READ" and "WRITE" have been called the algorithm "ASTRID" jumps back (jump 76) to operation 53 where all outline variables are reset, thus allowing the next outline to be read in and analysed.

While "ASTRID" is supervising the overall operation of the system as described in relation to the flowchart of Figure 2 the microprocessor is also controlling the operations of reading data into the FIFO and outputting data to the printer. In order that these operations are carried out the CPU 1 5 and the UART 11 are wired for the three interrupts shown in Figures 3a, 3b and 3c. In Figure 3a an output signifying that data is available from the graphical input tablet interrupts the CPU in operation 77. The data is read into the input FIFO 27 in operation 78. A pen interrupt is then enabled in operation 79 before control is handed back to "ASTRID" in operation 80.As soon as a signal representing the pen pressure is available from the pen the pen interrupt 82 enables information to be read into the input FIFO 27 in operation 83 whereupon the pen interrupt is disabled (operation 84) and control handed back to "ASTRID" in operation 85.

The CPU 15 runs a timer routine 86 which periodically tests the contents of the output FIFO 28 in test 87 to determine whether any output is to be printed. If so data is sent to the printer in operation 88 and the control is handed back in operation 89.

When "ASTRID" has determined that all data to be attributed to a particular outline is available "READ" is called, in operation 73 as is mentioned above. "READ" and its subroutines are based on Pitman's New Era Shorthand. "READ" (see Figures 4a, 4b and 4c) first carries out operation 90 to filter the data which is stored in the buffer to remove "high frequency" noise from the outline data. A suitable algorithm for filtering is a common "gear backlash" algorithm such as is described in "Character Recognition in an Experimental Reading Machine for the Blind" (Mason, S. J., Clemens, J.

K.) which is a paper in "Recognising Patterns" Edited by Kolers and Eden, MIT Press, 1968, pages 156 to 167. A test 94 is next carried out to determine whether the outline is initially highly curved and therefore probably features an initial. hook or loop. If the data appears to fall into this category the initial structure is classified in operation 95 as a hook, a loop or a circle. Test 94 and operation 95 are described in more detail later. The outline is then simplified by the removal of that part of the data representing the hook or loop (operation 96). The same type of sequence of operations is then carried out again in test 100 and operations 101 and 102 to search for final hooks or loops, recognise such features and remove them from the outline structure.A forward jump 104 enables operations 95 and 96 to be omitted if the test 94 indicates that no initial hook or loop is present and a jump 105 carries out the same function with regard to final hooks or loops.

The consonant outline is then re-examined in operation 106 and a test 107 carried out to determine whether the outline contains a closure, that is if the same co-ordinates (or near same coordinates) occur twice in the same outline. If so the closure is classified, in operation 108, as large or small, on the basis of the area enclosed (represented by area variable described later). The outline representation, as held in the linear buffer, is simplified by removal of the data representing the closure in operation 109. If no closure is present jump 111 bypasses operation 108 and 109.

Separation of the outlines into strokes representing characters is now carried out by first calculating "boundary values" for each point in the outline, this being carried out in operation 11 2 according to a subroutine, which is described below. The values obtained are weighted in operation 11 3 according to a rule which gives higher weights to points likely to correspond to boundaries between outlines. On the basis of the weighted boundary values available the first, and subsequently the next, boundary is located in operation 114 whenever a boundary value peaks above a reference value.

Having isolated a stroke it is now classified in operation 11 5, as described later.

Should the area variable, which will be described later, reach a maximum mid-way through the stroke and then be cancelled by later data, a check is carried out to determine whether the stroke consists of two strokes representing separate consonants. To carry this out an area inversion test is first made in operation 11 6 and then the stroke is split at the point representing maximum unsigned "area" in operation 11 7, the two consonants thus segmented are classified in operation 11 8 in the same way as in the operation 11 5. If no area inversion occurs a jump 11 9 is used to bypass the operations 117 and 118.

Test 121 determines whether a stroke has been successfully classified. If not "READ" carries out a jump 122 back to the operation 114 provided not more than three classification failures have occurred as revealed by operation 1 23 and test 1 24. In this way if a boundary between strokes is incorrectly located it is later ignored and the next possible boundary considered. Should three attempts at classification fail then the output of test 124 leads to operation 125 where an error string (that is a string of characters denoting an error) is assembled for later output.

Successful classification as indicated by the test 1 21 causes data representing the classified stroke to be deleted from the outline representation in operation 126 and a test 127 is carried out to determine whether strokes representing consonants have come to an end. If not a jump 128 takes the algorithm back to the operation 11 3 where boundary potentials are first formed and then the next boundary is located in operation 114. In this way the outline representation is truncated each time a character is recognized and the remaining portion is processed to find the next boundary and corresponding character until the whole of the consonant part of the outline has been processed.

If the test 1 27 signals the end of consonants then a "Recognised Consonant String" is assembled in operation 130 and a test 131 is carried out to determine whether the end of the outline has been reached, this test depending on whether the "vowel" flag was set in the operation 64 of "ASTRID". If vowels (or intersections etc.,) are present they are then examined and classified one by one in operations 1 32 and 133 which are repeatedly carried out by means of a jump 134 until the end of the outline has been reached. At this point the final text string is assembled in operation 135 and the end of the "READ" algorithm has been reached.

The test 94 and the algorithm used in the operation 95 are now described. i3Oth hooks and loops exhibit (at least partial) self-closure and it is this phenomenon which is used to help detect and isolate them.

The overall classification procedure is exactly the same for either type of feature at either outline extrema, and so the detection of an initial hook is considered.

The hook is represented in the filtered data by a sequence of vectors in which the first maximum in the distance from the origin of the stroke occurs at the end of the ith vector and the first minimum at the end of the jth vector.

The exact location of these points (that is, the numbers of the ith and jth vectors) is found by searching for an initial maximum and following minimum of values in a function related to the "distance from origin" of each vector end point.

Essentially, hooks and loops are distinguished by the value of this function at the first minimum (after the maximum) of the feature; a small value being expected for loops, for example. Thus the test 95 is based on the discovery of a maximum followed by a minimum but the 'distance from origin' at the minimum must be below a threshold value for the test 94 to register a hook or loop.

The 'shape' of the hook may be checked more accurately by examining the relative directions of the pre- and post-maximum vectors, which should, in theory, be opposite.

Similarly, the sub-categories of loops (i.e. circles and ellipses) are distinguished by the ratio of the major and minor axes.

The algorithm used is shown in Figure 5. The positions of the 'distance-frorh-origin' (DFO) of the initial maximum and following minima are located in operations 215 and 216 and the test 217 for a further peak in DFO is carried out to ensure that the stroke continues and is not simply a circle. The maximum and minimum DFO values are stored in operation 21 8 and then the enclosed area is calculated in operation 219 and used to classify the size of the feature in operation 221. Next the DFO of the first minimum is compared with a threshold value in test 222, and if the threshold is not exceeded the feature is classified as loop. The eccentricity of the loop regarded as an ellipse is calculated in operation 223 and a test 224 is carried out to determine whether a threshold value is exceeded. If not operation 225 classifies the feature as a circle, but if the threshold is exceeded the feature is classified as an ellipse in operation 226. The subroutine is then finished and operation 96 of Figure 4a is carried out.

If the test 222 indicates that the loop is not closed, that is the DFO of the first minimum is greater than the threshold value, the average direction of the vectors occurring before the initial maximum is calculated and the sector in which this average occurs is determined in operation 227. The description of Figure 7b which follows late in this specification explains the concept of sectors. A similar averaging process is carried out in operation 228 on vectors between the initial maximum and the following minimum to determine the sector in which the post maximum average vector occurs. For a loop the two average vectors should be in opposite directions, that is their sectors should be opposite.

Test 229 is therefore carried out and if passed the feature is classified as a hook in operation 301. If test 229 is negative or no further peak is detected (see test 217), operation 302 classfies the feature as not being a hook or loop.

When the above algorithm is used to process features at the end of strokes, the 'origin' used is the end of the stroke, not the beginning, and the first maximum and following minimum are as encountered by starting at the end of the stroke and proceeding towards the beginning.

The algorithm used in the operation 112, is based on boundary values in which the ith boundary value is defined as:

Bpotith = | Xpotith | + | y potith where each of Xpot1, and Ypot, are defined by the following equation:

where: K is X or Y as appropriate C is a constant to reduce the effect of writing tablet 'Distortion'. C is defined as 11 for X, 8 for Y.

A is the averaging distance (in number of vector steps) of the boundary function.

S(AK1) is the mathematical sign function of the ith vector K (that is, the magnitude of the vector is disregarded).

This has the value: S(AKI)4 if (AKI)= S(6K,)=--1 if (AKj)=negative S(AK1)=+1 if (AK1)=positive The boundary value is essentially an averaged first finite differential of the sequential vector description of an outline.

A possible junction between strokes representing an "S" and an "M" is shown in Figure 7a. The "S" is represented by differential vectors 1' to 14' and the "M" by vectors 15' to 32'. As has been mentioned the differential vectors are represented by signed four bit X and Y incremental values in the form of a string of bytes output from the graphical input tablet after the origin of a new outline has been output. A table of values of Xpot and Ypot, is given in Table II below and it is clear that high values of these variables occur at and near the junction between the two strokes.

Table II

lth element #X1 #Y1 Xpot1 Ypot1 9 0, -1 -22 40 10 0, -1 0 56 11 -1, -1 33 64 12 -1, -1 55 80 13 0, -1 99 88 14 -1, -1 121 104 15 1, 1 132 96 16 1, 0 99 72 17 1, 1 88 56 18 1, 1 88 24 19 1, 0 77 8 20 1, 1 55 -8 21 1, 0 33 -32 22 1, 1 22 -48 23 1, 0 0 -72 24 1, -1 0 -64 A flowchart of a subroutine for calculating Bpot,th is shown in Figures 6a and 6b. First three initialising operations 140, 141 and 142 are carried out and then the X element of the Pth vector is examined the Pth value having been initialised to equal (i+1 ), this value can be any integer between -8 and 7. When the examination has been carried out in operation 143 the test 144 is carried out to determine whether the vector element is zero and if not a test 145 is carried out to determine the sign of the vector element.Either operation 146 or 147 is carried out to update the value of Xpot by adding or subtracting eleven and then similar operations 148 to 1 52 are carried out to obtain a current value of Ypot by adding or subtracting eight. The constant C for tablet distortion is represented by the standard increments eleven and eight.

Up to this point the value of Bpotith has been found corresponding to the first value required to calculate the first term in the above equation for Knot, and operations are now carried out to calculate the first value required to calculate the second term in Kpot,. These operations 143' to 1 52' are similar to operations 143 to 1 52 except that it is then p-(A+ 1 )th vector which is examined and in operations 146',147',151 and 152' signs are reversed in relation to operations 146, 147, 151 and 152, respectively, in order to allow for the minus sign which precedes the second term of Kpot,.

The data pointer is now incremented in operation 153 and a loop counter is decremented in operation 154. If the loop counter has not reached zero as determined by test 155 jump 156 is carried out so that the next term in Bpoti,, can be evaluated and when the ith boundary value has been calculated and test 1 55 is positive the modulii of the accumulated values of Xpot1 and Ypot are calculated (operations 156 and 157, respectively) and are added in operation 158 to give Bpot,th.

Control is then returned to "READ" so that when all the appropriate boundary values have been calculated, operation 11 3 can be carried out.

Before the subroutine which is used to recognise strokes is described the four features on which it is based are first described.

(i) Consonant Sector This features is used to distinguish the type of consonant stroke written. The sector boundaries are illustrated in Figure 7b and are formed by lines with the following equations: (a) 4Y=-X (b) 4Y=X (c) Y=4X (d) Y=-4X The sectors are numbered 0 to 7 as shown in Figure 7b. However improved results can be obtained if the number of sectors is increased to 40, for example. The sector value is derived from the total displacement value calculated for each consonant. The feature is useful as a 'preclassifier' because it enables an easy test of the possible validity of that stroke. (For example, it is known that single stroke Pitman consonants should never enter sections 2, 3 or 4).

The sector value is calculated in the following manner. Firstly, the total X and Y displacement values describing the stroke are calculated. These values are then subject to tests based on the above four equations (a) to (d). Each one of these tests is arranged to set a flag (labelled A, B, C and D respectively) if TRUE. When all four tests have been completed, the 3 bit binary value representing the sector number is assembled.

The following logical condition codes may be used to set each of the sector bits.

BIT 2, the most significant bit, is set if A is true; that is: BIT 2=A Bit 1 , the second most significant bit, is set if the following expression is true: BIT l=(AO) Similarly, BlT, the least significant bit, is set only if the following expression is true: BIT=((At3C)(3(B(3D)) In all these expressions, the symbol (3 has been used to denote an 'exclusive-OR' operation. The exact form of these expressions determines into which sector boundary values (i.e. those on any of the lines (a), (b), (c) or (d)) are allocated. The logical rules may, therefore, be modified slightly if the current assignments needed to be changed.

The sectoring algorithm is described below in greater detail with reference to Figure 9.

(ii) Enclosed Area and Area Sign These features are used to distinguish straight and curved consonant strokes; if curved, the area sign also distinguishes the direction of curvature. The area value is calculated from a summation over the vector series describing the total stroke:

where: N=the number of vectors in the entire stroke.

At the Xth element of the ith vector.

AY≅the yth element of the ith vector.

Y1=the intermediate Y sum defined by:

Xtot the total stroke displacement values, tot and * has been used to denote multiplication.

Note that the digits 2 and 3 are present to allow for tablet distortion, more accurately 8 and 11 would be used but multiplication in microprocessors is easier using 2 and 3. The 'Area' parameter is related to the area enclosed within the consonant stroke and the minimum closing vector of that stroke. The sign follows the usual mathematical convention for rotations; positive denotes and anticlockwise rotation.

A stroke is defined as curved if the 'Area' parameter exceeds a predetermined threshold value.

(iii) Consonant Length This feature is used to disintuish between halved, normal and double length consonant strokes.

The 'length' value is calculated from the X and Y total stroke displacements:

The (11/8) scale factor is intended to counteract tablet distortion.

A stroke is defined as halved, normal or doubled, if the 'Length' parameter lies between two predetermined threshold values.

(iv) Average 'Pen Pressure' This features is used to distinguish thick and thin consonant strokes. The average value (calculated from the sum of the pressure values corresponding to vectors in the series describing the outline) is compared against a threshold value to determine the thickness of a stroke.

A written stroke is classified as a specific consonant only when each and every one of the above mentioned features lies within some predefined range of the 'ideal' value for that consonant. In other words, the 'tolerance' of the recognition algorithm is an inherent property of each of the features used, and is NOT achieved by means of any distinct 'nearest' matching process performed with the ENTIRE feature set.

The subroutine which is used to recognise or classify simplified segmented consonants (that is with hooks and loops removed) in operations 115 and 118 of "READ" is shown in Figures 8a, 8b and 8c. Three initialising operations 1 61, 1 62 and 163 are first carried out and then each vector is read in turn (operation 164). The area increment is calculated in operation 1 65 by determining the increments in X and Y which the vector represents: AX and AY respectively, determining the current Y (as defined above), forming the product (3*AX1)*(2*Y1)+AY) and subtracting the result from the area accumulated so far. This operation builds up the second term in the expression for 'Area' mentioned above.Next increments for Xtot and Ytot are calculated in operations 166 and 167, respectively. After incrementing the data pointer to correspond to the next vector in operation 1 68 and decrementing the loop counter in operation 169, the next vector is considered by means of jump 170 back to the operation 164.

When a test 1 72 indicates that the complete segmented stroke has been dealt with in this way and therefore the second term of the 'area', and Xtot and Ytot have been calculated, the enclosed area corresponding to the consonant is calculated in operation 1 74 by forming the product (3*Xtot)*(2*Ytot) and adding it to the completed second term of the 'Area' expression. The sign of the area found is now tested (173) and either a positive or negative flag is set in operation 350 or 351. Next the modulus of the area is found in operation 352, and in tests 353 and 354 is tested against two threshold values. If the lower threshold is not exceeded a "straight flag" is set (operation 355) and if the higher threshold is not exceeded a "curved flag" is set.Exceeding the second threshold causes a recognition failure to be registered in operation 1 83. The stroke length is now calculated in the way described above (operation 357) and tested for length against four threshold values in tests 358 and 361. If a high threshold value is exceeded, the stroke is impossibly long and a recognition failure (operation 183) is registered. Very short strokes are also rejected against a low threshold value in test 359. Test 360 against a higher intermediate threshold sets a "double flag" (operation 362) if the threshold is exceeded or leads to a test against a lower intermediate threshold which either sets a 'halved flag' or a 'normal flag' in operations 363 and 364 depending on whether the threshold is not reached or is exceeded.

The 'sector flag' denoting the sector (see Figure 7b) in which the stroke falls, as determined in operation 1 76 is now set and the average pen pressure is calculated from transducer output values in operation 1 77. Test 1 78 determines whether a pressure threshold is exceeded and if so sets a "thick flag" (operation 365) or if not a "thin flag" (operation 366).

The various flags set make up the bits of a word according to the significance of the feature concerned. Thus the most important features (such as sector) are assigned the most significant bits within the words and the least significant bits are assigned to the least important feature (such as pressure). The word is then tested against a table of words held in ROM in an operation 184. If a match is found (test 179), the character corresponding to the word matched is read from the table and is output in operation 181, if not the nearest character in the table is output in operation 1 80 (since the table is arranged in numerical order of the word formed by the flags this is the most likely alternative).

The thresholds used in tests 353,354 and 358 to 361 are set according to the operator variables as measured in the operation 51. Operation 182 returns control to the READ algorithm.

To carry out the operation 176, the sectoring algorithm of Figure 9 is carried out. First the bits 0, 1 and 2 which will represent the sector number are set to zero and flags A, B, C and D are reset in operation 310. A test 311 based on equation (a), mentioned above, is carried out substituting Xtot and Ytot for X and Y. If the test 311 is satisfied flag A and bit 2 are set to 1 in operations 312 and 313. Then tests 314,315 and 31 6 are carried out in sequence with flags B, C and D being set in dependence on their outcome in operations 317,318 and 319.

The logical product AOC is then formed from the flags set in operation 320 and bit 1 set accordingly (test 321 and operation 322), and the logical product (A(3C)(3(BG)D) is formed in operation 323 and bit 0 set by means of test 324 and operation 325.

Bits 2, 1 and 0 are now set to represent the binary equivalent of the sector numbers shown in Figure 7b for the stroke represented by Xtot and Ytot.

Figures 1 0a and 1 Ob show a flowchart for the "WRITE" algorithm which transliterates the blocks of recognised characters produced by "READ". In operation 1 85 and test 186 intersections are detected but if no intersections are present a test 187 is carried out for the presence of a vowel. In the absence of vowels operation 1 88 is carried out to search through a short form lexicon, held as a table stored in part of the Rom 22, with test 1 89 giving a positive output if a match is found. The search may employ a simple list searching strategy (commonly known amongst microprocessor engineers) using the 'position' of the short form as the basis of an index to the appropriate area of the dictionary. Exit from the search brings a representation of a short form when an entry is found.

A similar procedure is carried out by operation 191 and test 192 in the stenographer's personal short form lexicon stored in the portion 29 of the RAM 26. In this case the search may be a dictionary search of the type described by Longuet-Higgins and Ortony. A., in "The Adaptive Memorization of Sequences" in Machine Intelligence Vol. 3, edited by Michie.

If the string does not contain an intersection, or if it is not either form of short form it is then transliterated and enhanced, as described below in operations 1 93 and 194. The resultant of operations 1 93 and 1 94 is, in these circumstances used to format the output string in operation 195, this string being passed to the printer 14 by way of the output FIFO 28, and to the VDU 38 by way of the interface card 34 in operation 1 96.

If an intersection is found in test 186 a jump 197 takes place to a test 198 for a standard intersection and if such an intersection is present, the underlying outline is process in operation 199, and the standard intersection lexicon is searched in operation 200. Similarly if the intersection is nonstandard a search through the non-standard, personal intersection lexicon belonging to the stenographer is carried out in operation 201. Test 202 determines whether a match is found in either intersection lexicon and if so the output string is formatted and printed in operations 1 95 and 1 96. If no intersection match is found the operation is informed in operation 203 and the next best alternative string is substituted in operation 204 before formatting the output string and printing the final output.

If the test 187 detects a vowel then a jump 205 takes place direct to operation 1 93 avoiding searching either short form dictionary.

After operation 1 96 "WRITE" hands control back to "ASTRID".

Before operation 1 93 is carried out it is advantageous to carry out a process in which features of outlines are converted to phonemes.

However, a description of the strikes available at this stage is first given.

Forty-eight strokes can be indentified, for example strokes 1 and 2 may designated horizontal strokes drawn from left to right with 2 distinguished from 1 by being thicker. As a further example, strokes 45 and 46 may designate arcs of circles drawn anti-clockwise from 9 on the clock face to 6, and again distinguished from one another by the latter being thicker than the former. Strokes may be horizontal, vertical, or at an angle between the horizontal and vertical, or they may form arcs subtending angles of 900.

The recognition algorithm ASTRID operates by segmentation of the physical shorthand outline into a number of smaller and simpler elements. These elements have been chosen to be comparatively easy to classify accurately in isolation. Such is the nature of Pitman's shorthand that these elements, on the whole, are important shorthand features. However, this is not always a simple relationship between these elements and the intended shorthand characters.

For example in the first of the three examples of strokes shown in Figures 13a, 13b and 1 3c, the outline consists of an initial positive small hook attached to a horizontal straight stroke (number 1). This can be written: &commat;,+small hook, stroke 1,&commat; where (Bb denotes the outline boundaries. This outline consists of two shorthand features, representing two shorthand characters, forming the phonetic 'word': IKIJV In Figure 1 3b, the outline consists of an initial positive small hook attached to an upward straight stroke (number 7 which is a light stroke at angle usually of about 300 to the horizontal drawn from the lower end towards the upper end).Again, this outline consists of two shorthand features, but represents only one shorthand character: In Figure 13c, the outline consists of a final positive small hook attached to a horizontal straight stroke (number 1). Again this represents two shorthand characters, but the meaning of the hook has now changed: or It is because of this complicated relationship between the recognised features and implied shorthand characters that a contextual feature to phoneme conversion process is required.

The algorithms used to perform this process may be classified into two groups.

Firstly, a controlling algorithm which monitors the overall process of phoneme 'recognition'.

Since a suitable algorithm is simply derived no further description is given.

Secondly, there are quite a large number of specialised rules each concerned with the conversion of shorthand features adjacent to a particular stroke. The entire conversion process is performed relative to the constituent consonant elements within the outline.

The rules, examples of which are specified below, are grouped according to the consonant stroke to which they are applied. Each of the rules within the group relate to the linguistic value of a particular series of features when written adjacent to the stroke.

When this conversion process is required, the individual elements comprising the shorthand outline will have been segmented and classified, and an equivalent feature string listing details of the strokes, hooks, loops and circles comprising the base part of that outline will already have been assembled.

The controlling algorithm sequentially examines each of the features within the source string, searching for the next simple stroke element. Once this has been located, the algorithm attempts to match each of the relevant production rules against the feature sequence occurring in the source string. If the arrangement of features in the source string successfully satisfies one of the specific production rules, then that feature sequence represents the phoneme sequence indicated by the rules.

In the event that no particular production rule is satisfied, the additional features are ignored, and the stroke element produces the following sequence of phonemes: /Vb/,/*/,/Va/,/Re/ When all the source consonant elements have been converted, the contents of a target buffer are prepared for transliteration.

The following set of symbols are used throughout the rules: denotes a feature sequence boundary (effectively a word boundary).

* indicates the current consonant stroke 'cursor' position within the feature sequence.

loop denotes any eliptical (but not circular) closure.

circle denotes any circular closure hook denotes any partial closure +/- denotes the mathematical sign of feature curvature. (+ means the feature was drawn anticlockwise and - means it was drawn clockwise) 1/ denotes that the symbol between the slashes represents that of a phonetic code.

denotes the main phonetic representation of the current consonant stroke.

/Re/ denotes the phonetic remainder of the current consonant stroke. This term is due to the additional linguistic significance of stroke length. For example, the remainder of a halved iT/ consonant might well be either/T/ or depending on the syllable count.

/Va/ represents the phonetic code of any vowel (assigned to the current stroke) occurring AFTER the current stroke.

/Vb/ similarly, BEFORE the current stroke.

Unusuai hook a hook formed with a reversal of direction, for example as in writing the top of a Y S represents any other stroke in the feature context sequences. If followed by a terms enclosed in brackets, this term must also be satisfied in order that the production rule be applied.

Examples:- S(s )-represents any stroke featuring sector 1 S(n4)-represents stroke number 4 Commas are used to delimit individual features and phonemes within the sequence specified in the feature-phoneme conversion rules.

Many of the conversion rules (called 'production rules') are common to a large number of strokes.

In particular, production rules tend to be sensitive to stroke curvature, so that all straight strokes, for example, have a group of production rules in common. These are known as general production rules, GROUP A. Curved strokes share general production rules, GROUP B.

In general, production rules are applied only to the range of simple consonant strokes. These rules then test for other features (such as hooks, loops and circles) and produce the equivalent phonetic representation. Vowel codes, on the whole, are converted simply by look-up table as these feature codes are not context sensitive. However, vowels are assigned to the phonetic representation of an outline only via the appropriate consonant stroke production rules.

For example: Rule: +small hook,+ This rule is read as follows: If the current stroke were preceded by a small hook with positive arithmetic curvature, the feature sequence would best represent the phonetic sequence, preceding vowel, phonetic consonant, phonetic /v.

The trailing dots indicate that this is the first half of the representation of the consonant stroke feature. The remainder (invariably due to the linguistic consequences of stroke length in Pitman's shorthand) is dealt with by another production rule.

For example Rule: *,+small circle;S(n7) ...,/Va/,/Re/,/S/ This rule may be read: If the current stroke were followed by a small circle of positive curvature, and if that circle where to be followed by any stroke apart from stroke number 7, then this feature sequence would best represent the phonetic sequence, following vowel, phonetic remainder, phonetic/S/.

The conversion of the following outline feature string is now considered: &commat;,-small hook,stroke 7,-small hook,&commat; (this is as shown in Figure 13d). The production rules for relating to stroke number 7 are examined. In this case, this stroke would normally represent phonetic /R/. However, the specific additional producution rules illustrated below must be applied before those more general ones in GROUP A. In fact, the following rule is satisfied: Rule: small hook,* NbA/YA...

The feature therefore represents phonetic /Y/ rather than phonetic /R/ as normal. Consider the next feature. In this case, there are no additional specific production rules and so only the general rules are applied. The following rule is satisfied: Rule: *,-small hook . . . ,iVa/,/N/,/Re/ Hence, the final phonetic representation of the source feature string is: ,'Vb",/YANaA/N/,/R e/ and therefore if there are no vowels, and if the stroke is of normal length (so that there is no phonetic remainder), the feature sting represents.

/Y/,/N/ In more complicated outlines it would, at least in theory, be possible to include some features in more than one production rule. This is not permitted. Once a feature (apart from a stroke) has been used to satisfy a production rule, it cannot be used again to satisfy further production rules.

Also in the embodiment here described it is necessary to order the application of production rules. This is as follows: 1. Specific rules dealing with features before, the current stroke 'cursor'.

2. General rules dealing with features before the current stroke 'cursor'.

3. Specific rules dealing with features after the current stroke 'cursor'.

4. General rules dealing with features after the current stroke 'cursor'.

The general production rules for Group A straight strokes are as follows: Context Production &commat;,+large circle,* /SW/,iVb/,/*/, &commat;,+small loop,* /ST/,/Vb/,/*/,...

+small hook,* /Vb/,/*/,/L/,...

-small hook,* /Vb/,/*/,/R/,...

+small circle,* /S/,/VbA/*/,...

-small circle,* /S/,/Vb/,/*/,/R/,...

*,+small hook ...,/Va/,/F/ or /V/,/Re/ *,-small hook ...,/Va/,/N/,/Re/ *,+small circle,S(n7) ...,/Va/,/Re/,/S/ *,+large circle ...,/Va/,/Re/,/SES/ *,+small loop ...,/Va/,/Re/,/ST/ *,+large loop ...,/Va/,/STER/ *,-small circle,&commat; ..., /Va/,/N/,/Re/,/S/ *,+large hook,&commat; ...,/Va/,/SHUN/ *,-large hook,&commat; ...,/Va/,/SHUN/ The general production rules for Group B are given below. All of the following rules have the additional constraint that the sign of the enclosed AREA of the attached feature MUST agree with that of the base stroke.

Context Production &commat;,small loop,* /ST/,/Vb/,/*/,...

&commat;,large circle,* /SW/,/Vb/,/*/,...

small sircle,* /S/,/Vb/,/*/,...

small hook,* /Vb/,/*A/R/,...

large hook,* /Vb/,/*/,/L/,...

*,small circle,S(n7) ...,/Va/,/Re/,/S/ *,large circle ...,/Va/,/Re/,/SES/ *,large loop ...,/Va/,/Re/,/STR/ *,small hook ...,/Va/,/N/,/Re/ *,large hook,&commat; ...,/Va/,/SHUN/ Some examples of specific production rules are given below and further such rules may be derived by considering the Pitman shorthand convention and the segmentation of outlines described above.

The production rules for stroke 1 are: -normally denotes /K/ -assignments: 1.

2. /Re/=iT/ for halved strokes, st syllable.

3. /Re/=/D/for halved strokes, after 1 st syllable.

4. /Re/=iTDTH/for all doubled strokes.

-additional production rules: &commat;,-small unusual hook,* /W/,/Vb/,/*/,...

&commat;,+large hook,* /Vb/,/KW/,...

*,-small circle,(S(n7). S(so, 1, 2, 3,)) ...,/Va/,/Re/,/S/ -general production rules: Group A The production rules for stroke 7 are: -normally denotes /R/ -assignments: 1./*/=/R/ 2. /Re/=/T/ for all halved strokes.

3./Re/=/TDTH/ for all doubled strokes.

-additional production rules: &commat;,-small unusual hook,* /W/,/Vb/,/*/,...

&commat;,+large hook,* /Vb/,/WH/,...

+small hook,* /Vb/,/W/,...

small hook,* /VbA/YA...

-small circle,* /Vb/,/H/,...

general production rules: Group A The production rules for stroke 45 are: -normally denotes/Fl i-assignments: 1.1*1=/Fl 2. /Re/=/T/ for halved strokes, Ist syllable.

3. /Rel=/Dlfor halved strokes, after 1 st syllable.

4. lRe/=/TDTHlfor all doubled strokes.

-additional production rules: +small hook,* /Vb/,/*/,/R/,...

-general production rules: Group B The process of transliteration, for example as carried out in the operation 193 is now described.

Transliteration is based on recognising patterns of spelling from the characters identified. Since these characters are largely phonetic the transliteration takes the form of substituting a letter or a group of letters which usually represents a sound in a particular context, the context determining the character used mainly according to spelling patterns.

A set of spelling rules is first drawn up by analysis of English linguistic patterns, for example those desribed in "Phoneme-Grapheme Correspondences as Cues to Spelling Improvenemt" by Hanna, P.

R., et al., Standford University, School of Education, CRP-1991, 1965, (microfische ED 003 321) and by Horne, E., "Phonetics and Spelling", Elementary School Journal (USA), May 1957, pages 424 to 432.

Some examples of such rules are given below but first a key to the symbols used in the logic used by the rules is given.

Key: The following set of symbols has been used throughout the attached list of spelling patterns: cons any phonetic consonant vowel any phonetic vowel (including diphthong) dipth any phonetic diphthong (i.e. l+OW+Y+U). Note + means or.

s. vowel any short vowel (i.e. A!+E!+I !+O !+U !+OU) I. vowel any long vowel (i.e. A+O+AH+AW+U+E+OO+I+OY+OW).

any word boundary (e.g. space) stop any word boundary or inflection * denotes that the transliteration shown represents that of several phonemes processed as a group) denotes that the default form should be doubled subject to a positive result from the fol lowing additional test: (s. vowel), cons, suffix means that the test is applied to the adjacent consonant, regardless of any intermediate vowels.

+'e' denotes the addition of a following silent 'e'. This is a common graphemic structure, which usually occurs finally. (e.g. 'make', 'cake', 'have').

denotes that any following silent 'e' due should be cancelled.

'X' an abbreviation used to denote the following group of phonemes: (II+I+A!+A+E!+E+O !+O+N+R+L+U J2, etc. the subscript conditions refer to the consonantal position of the labelled consonant with respect to the beginning of the word. (e.g. C10 IN2S3O lN4E lN5T6). When a subscript is used in a logical condition statement, the subscript condition must be satisfied in order that the 'rule' be applied. (e.g.

J > 1-/J/ must not be the first consonant in a word.

J < 3-/J/ must be either the first or second consonant.

J < 4-/J/ can be any except the fourth consonant).

For the sake of clarity, commas have used to delimit individual phonemes (or logical groups of phenomes) within the logical condition statements. When reading a logical condition statement each comma may be thought of as meaning 'followed by'. A bar over an expression denotes absence of that expression, thus cons means no consonant present. A phoneme transliteration summary for some consonants and vowels is given in Table Ill. In this context a phoneme means the character recognised by "READ".

Table Ill Transliterated Phoneme Logical Conditions Representation /F/ (s. vowel),F,(cons) ff F,O,(N+B+T) ph F,l l,L,(vowel) ph F,l !,S,(vowel) ph default f /G/ (s. vowel)G,l l,&commat; gg2 (s, vowel)G,...,(R+D),&commat; gg (s. vowel)G,...,L,&commat; gg &commat;,G,(l l+E l),(V.T.cons) gu2 (I. vowel),G,&commat; > gue2 OIG > 2,&commat; gue2 default g /L/ L,D 12 (s. vowel),L,(stop) 112 (cons),L,&commat; 112 default /E/ (D+R),E e (D.R.SH F.K.G.W.),E,(CH+T+G+K+S+P) ea (P.B),E,R ea &commat;,E,J i default ee Some examples of the use of Table Ill to perform transliteration are now given for the words "fill" and "colleague".

Example 1 Phonetic code: (Z F II L (1) (2) (3) (4) (5) Transliteration: Comments: (1) /&commat;/-space (2) /F/-f /F/ is: (i) preceded by a space, (ii) followed by a short vowel, then a consonant, then another space. If the final space had been a vowel, the condition: F, l,L, (vowel) would have been satisfied.

However, as no specific condition was satisfied only the default form would be output.

(3)/li/-i /11/is: (i) preceded by a consonant; (ii) followed by a consonant.

Once again, this satisfied no specific condition and the default form would be output.

(4) IL/-lI /L/ is: (i) preceded by a short vowel, (ii) followed by a space Hence the condition: (s. vowel) ,L,(stop) is satisfied and the double 'I' form would be output.

(5) /&commat;/-space 'F', II would therefore be transliterated as "fill" which is, of course, completely correct.

Example 2 Phonetic code: (B K 0 | L E G (1) (2) (3) (4) (5) (6) (7) Transliteration: Comments: (1) /&commat;/-space (2) /K/-c /K/ is: (i) preceded by a space, (ii) followed by a short vowel.

Hence no condition is satisfied and the default form would be output.

(3) /OI/-o /OI/ is: (i) preceded by a consonant, (ii) followed by a consonant.

Once again, no specific condition is satisfied and only the default form would be output.

Transliteration: Comments: (4) /L/-l /2 is: (i) preceded buy a short vowel, (ii) followed by a long vowel.

Once again, no specific condition is satisfied and only the default form would be output.

(5)/E/-ea /E/ is: (i) preceded by a consonant/2, (ii) followed by a consonant /G/.

This situation satisfies the following condition: (D,R.SH.F.K.G.W), E,(CH+T+G+K+S+P) This test is satisified because both logical conditions surrounding the /E/ phoneme are true.

That is: (a) the individual terms within the left AND product are all one; therefore, the logical product is one. (Only the inverse of L, that is L, is zero).

(b) the term /G/ in the right logical expression is one; therefore the complete OR function is one.

The alternative form 'ea' would be output.

(6)/G/-gue /G/ is: (i) preceded by a long vowel, (ii) followed by a space.

This situation satisfies the following condition: (I. vowel),G,&commat; and therefore the alternative form 'gue' would be output.

'K,O !,L,E,G', would therefore be transliterated as 'coleague', which, despite being incorrect, represents a substantial improvement over the original form.

In those instances where the transliterated form is to include an additional silent 'e', the 'e' is added after the transliterated representation of any following consonant. All intermediate vowels, and some consonants cancel this additional 'e' term. In the event of there being no following consonant, the 'e' is added immediately after the transliterated representation of the vowel which caused it.

It will be realised of course that Table Ill only represents a small proportion of the transliterations which are in practice carried out. Other transliterations can be derived from the references given and will vary according to circumstances of use and the amount of transliteration it is desired to incorporate. Another useful reference for this purpose is the book "Regularized English", by A. Wijk published by Almquist and Wiksell.

By the time that transliteration is required, the recognition algorithm will have assembled a phonetic text string representing the shorthand outline.

The transliteration algorithm will sequentially examine each of the phonemes within this string.

For each of the source phenomes, the algorithm will attempt to match one of the phonetic sequences listed in the rules against that occurring in the source string. Only the rules relating to the particular source phoneme are tried.

If the arrangement of phonemes in the source script successfully satisfies one of the specific transliteration rules, then the phoneme is replaced in the target script by the transliteration indicated. If no particular rule is satisfied, then only the appropriate 'default' form is added to the target script.

The algorithms used to perform transliteration may be grouped into two categories.

Firstly, a controlling algorithm which monitors the overall script conversion may be used. Since a suitable algorithm is simply derived no further description is given.

Secondly, there are a large number of specialised algorithms each concerned with the transliteration of a single phoneme. Algorithms designed to process prefixes, suffixes and other similar non-phonetic features may also be included in this category.

Flowcharts of algorithms in the second category are not described in this specification since they follow directly from the rules given by way of example and other rules which are derived in the way outlined. If no specific transliteration rule is satisfied then only the most common 'graphemic' representation of the phoneme is passed to the output string (see operation 195).

The process 1 94 of enhancing the string received from the "READ" algorithm is now discussed.

One of the main problems experienced when trying to regenerate text from a Pitman shorthand transcript is that, quite often, the stenographer has written insufficient (phonetic) vowels to enable a good quality orthographic transcript to be made. Indeed, shorthand secretaries are often encouraged not to bother with vowels, particular if this means that they can increase their 'recording' speed. Usually worst affected are short common words, since without vowels these become totally unrecognisable.

Longer words on the other hand, can quite often still be read even if a large percentage of the vowels are missing. This is particularly true of words appearing in context.

In the embodiment described the problem of missing vowels is dealt with in two ways.

Firstly, the most common short words are available to the stenographer as short forms.

Effectively, this means that the stenographer need not write vowels at all into over 40% of the shorthand script. Of course, since these forms will be dictionary processed they will appear correctly spelt. In addition a special dictionary is used which may be easily modified (during operations) to suit the stenographer's current transcription task. This is particularly useful in any situation where the stenographer is required to transcribe a small group of long, less common words unusually often, as might occur in a lecture, for example. Again, these words benefit from dictionary processing and appear correctly spelt at the output.

Secondly, outlines which fail to be matched in either of these dictionaries are processed by an 'enhance' algorithm described later which helps to restore the correct orthographic 'shape' to the word by inserting an ambiguous marker (+) where appropriate.

The 'enhance' algorithm performs the insertion based on purely statistical considerations. If the probability of a particular consonant digram occurring with an intermediate vowel is greater than it occurring alone, the algorithm inserts a vowel marker between the digram whenever it occurs alone in the mutilated text. Since such information clearly cannot be 'calculated', it has to be stored within the algorithm in the form of a look-up table.

A reasonable improvement can be made in the quality and accuracy of vowel insertion by taking into account an additional variable; the position of the digram within the word. This ensures that the algorithm faithfully reflects the different initial and final common graphemic structures. Furthermore, by considering only the relative consonant position, the look-up process is made insensitive to the type of mutilated source material expected from transliterated shorthand.

The dimensions of the look-up table can be reduced to a minimum by careful encoding of the data. For instance, it is known that vowel digrams (and trigrams) are comparatively rare, so there is no need to include any vowel terms within the table. Furthermore, the consonant 'Q' is almost always surrounded by vowels (or initial space), so that need not be included in the table either. Less obvious perhaps, is the fact that 'e' is by far the most common final vowel. As a result of these simple considerations, only pairs of the following letters need be included as digrams within the look-up table: BCDFGHJKLMNPRSTVWXYZ The 'enhance' algorithm uses only five of the eight bits of each data byte; bits 3, 4, 5, 6, 7.The five groups reflect the range of permitted locations for each digram; the assignments are: (1) Group 1, bit 3 - is used to reflect initial vowel structure. If 'I', a vowel marker is inserted BEFORE the initial consonant digram. (e.g. +NT) (2) Group 2, bit 4 - is used to reflect initial consonant digram structure. lf 'I', a vowel marker is inserted between the initial digram. (e.g. N+T) (3) Group 3, bit 5 - is used to reflect all intermediate consonant digram structure. If 'I', a vowel marker is inserted between the appropriate intermediate digram. (e.g. PR+MPT) (4) Group 4, bit 6 - is used to reflect final consonant digram structures. If 'I', a vowel marker is inserted between the final consonant digram.

(5) Group 5, bit 7 - is used to reflect the final vowel structure. If 'I', a final 'e' is added after the final consonant. (e.g. ABLe) Table V is an example of part of a look-up table, and gives a five bit word for each possible digram commencing with V or W. The complete table used comprises digrams for all pairs of consonants from BB to ZZ (except pairs which include Q). A suitable look-up table can be compiled by statistical analysis of digram structures contained in English words, the probability of occurrence of each word being taken into account. In compiling a complete look-up table the book "Computational Analysis of Present Day English" by Kucera and Francis, published by Brown University Press is useful.

The complete look-up table is located in the ROM 22 and each digram is translated into a code representing the address of a location. The location is addressed and the corresponding five bit word obtained.

Table IV

Data Byte Data Byte Location Digram Bits 3 to 7 Location Digram Bits 3 to 7 301 VB 0 1 1 1 0 321 WB O 1 1 1 0 302 VC 1 1 1 1 0 322 WC 1 1 1 0 0 303 VD O 1 1 1 1 323 WD 0 1 0 1 0 304 VF 0 1 1 1 0 324 WF 1 1 1 1 0 305 VG 1 1 1 1 0 325 WG 1 1 1 1 0 306 VH 0 1 1 1 0 326 WH 1 1 0 0 0 307 VJ 0 1 1 1 0 327 WJ 0 1 1 1 0 308 VK 0 1 1 1 0 328 WK 0 1 1 1 0 309 VL 0 1 1 1 0 329 WL 0 0 0 1 0 310 VM 0 1 1 1 0 330 VM 0 1 1 1 0 311 VN 0 1 1 1 1 331 WN 0 0 1 1 0 312 VP O 1 1 1 0 332 WP 0 1 0 1 0 313 VR O 1 1 1 1 333 WR O 1 1 0 0 314 VS O 1 1 1 0 334 WS O 0 0 1 0 315 VT 1 1 1 1 0 335 WT 0 1 0 1 0 316 W 1 1 1 1 0 336 WV 1 1 1 1 0 317 VW 0 1 1 1 0 337 WW 0 1 1 1 0 318 VX 0 1 1 1 0 338 WX 0 1 1 1 0 319 vY 0 0 1 1 0 339 WY 0 1 1 1 0 320 VZ 0 1 1 1 0 340 WZ 0 1 1 1 0 An example of the use of the look-up table is given in Table Vl with reference to Table V. In the comments column the numbers adjacent to the consonants refer to the position of that consonant in the word, ignoring all intermediate vowels.

TableVl Original Vowel4ess Enhanced Comments VOWEL VWL V+WL V1,W2 bit 3=41 V1,W2 bis 4=1 W2,L3 bit 64 W2,L3 bit 7=ii The enhance algorithm is structured in two parts; a controlling routine isolates consonant digrams setting appropriate variables reflecting the relative location of the digram within the word, but calls a subroutine to actually perform the vowel insertion process. Flow charts explaining the first of these routines are shown in Figures 1 la and11 1b.

In order to provide the best possible insertion accuracy, the insertion process has been arranged to reflect the different initial, intermediate, and final structures which are common in English words.

Special attention has been given to consonant digrams occurring near word boundaries. (Psycholinguists have determined that the structures adjacent to word boundaries play a particularly important part in the reading process). Contextual sensitivity is achieved by having not one, but five discrete lookup tables; each table summarising exactly which digrams are permitted and which should be split in a given context.

Thus when a new string is produced in operation 193 (see Figure 1 Oa) the initial consonant diagram is read in operation 330 of Figure 11 a. If no digram can be located (that is there are less than two consonants) test 331 fails and the unenhanced word is output in operation 332 of Figure 11 b.

Otherwise a test 333 checks for a preceding vowel and when present this vowel is output in operation 334. Alternatively bit 3 of the word from the complete vowel-enhancement look-up table corresponding to the digram determined in operation 330 is selected in operation 335 and a "+" inserted if bit 3 is a "I". Operation 335 is carried out by a subroutine INSERT which is mainly concerned with addressing the correct location of the look-up table, and selecting the specified bit. It is not therefore described further except for a reference below to the action taken when bit 7 is selected.

The initial consonant of the digram is then passed to an output buffer (operation 336), and a bit counter is incremented in operation 337 unless bit 5 has been reached as detected by test 338. Test 339 tests for an intermediate vowel which is output if present (operation 341) but otherwise the INSERT subroutine is called (operation 342).

If further digrams are present as indicated by test 343 the next digram is formed in operation 344 and if the current digram is not the last (test 345), tests 338, 339 and 343, and operations 336, 337 and 341 or 342 are repeated. If the test 345 indicates that the last digram is present, the initial consonant of the digram is output in operation 346 and the bit value is set to 6 in operation 347. Tests 339 and 343, and operation 341 or 342 are repeated.

If the test 343 indicates that the last digram is present the final consonant is output in operation 348 and a test 349 for final vowels is made. If present such vowels are inserted (operation 351) but if not the subroutine INSERT is called (operation 352). This subroutine contains a test for bit 7 and when it is present the character "e" is inserted.

The enhance algorithm then returns to the "WRITE" algorithm and operation 195 (Figure 1 Ob)'is carried out.

The processes 1 88 and 1 91 in "WRITE" search short form lexicons for symbols recognised using a dictionary search of the type already mentioned. Tables of short forms can be obtained from shorthand textbooks but examples of two short forms are now given: BeenX We et When using apparatus of this embodiment the stenographer has to ensure that every short form symbol is written as a distinct outline and independent of any other outline or shortform, and further the permitted number of standard shortforms is limited to those pre-programmed into the transcription system.

The non-standard shortform lexicon provides a means for supplementing the standard shortforms with non-standard shortforms which the stenographer decides it would be convenient to use to save repeatedly writing an outline in full. Where a fairly uncommon word is used frequently for a short period, for example while discussing a particular subject, the stenographer enters and uses a nonstandard shortform.

Operations 200 and 201 of the "WRITE" algorithm employ dictionary searches through two tables; a first table giving the comple range of permitted standard intersections (such tables can be found in shorthand textbooks) and a second table which consists of pairs of recognised consonants written as an intersection and their equivalent meanings. Examples of two permitted standard intersections are now given: The intersection I (T) means 'attention', and the intersection / (J) means 'journal'.

The second table is entered by the stenographer during operation 52 of the "ASTRID" algorithm.

Returning now to operations 51 and 52 of "ASTRID", the way in which operator variables and the operator's lexicon are entered is now described.

When operation 51 is reached the VDU 38, or a special operator's display (not shown) displays a number of requests for certain outlines. In response, the operator writes symbols so exemplifying the size of a personal shorthand unit and half a unit and enabling these values to be extracted and stored.

One of the facilities obtainable with a writing tablet is the use of a form (called a menu), such as that shown at 210 in Figure 12, in an interactive mode. The form is placed on the writing surface of the tablet 10 and a cross 211 is touched using the pen signifying that a form of the type illustrated is in use. When used in conjunction with a simple partitioning algorithm, it is possible to arrange that pen contacts within a box 212 are interpreted as representing shorthand characters, but if the pen is used to contact a character in a box 21 3 a keyboard response is decoded. Further, if a character in a box 214 is contacted, special control codes are decoded. The keyboard facility may be used by the operator in response to a question displayed during operation 52. For example, the question "Do you wish to enter a short form?" is displayed in operation 52 and the operator may touch the "Y" in the box 213 to answer "yes" or an "N" to answer "no". If a short form is to be written the operator writes the corresponding outline in box 212 and touches characters spelling out the word in the box 213. In this way recognized short form outlines are stored against words in a table in the variables portion 29 of the RAM 26 for use as the operator's personal short form lexicon. The algorithm for interactive operation with a Datapad tablet is not described in more detail as it simply requires examination and sub-division of the absolute position of pen contact in each case. However, it will be realised that a short form lexicon can be entered in a more conventional way by using the tablet to write an outline in response to a display originating from a programme operated by the CPU, the operator then using a keyboard to enter the word corresponding to the outline.

Similar methods can be used to enter the operator's personal intersections.

While a specific embodiment of the invention has been described it will be realised that the invention can be put into practice in many other ways. For example, different algorithms can be used to provide segmentation of strokes representing consonants and for recognising consonants and vowels.

Other microprocessors can be used and it will then be necessary to interface and connect them as described by the manufacturer.

Claims (17)

Claims

1. Apparatus for generating a readable display from a written shorthand input, comprising graphical input means for producing signals representative of the position of a stylus used in writing shorthand on an input surface of the graphical input means, analysis means for analysing the output signals from the graphical input device according to rules based on a predetermined type of shorthand to derive output signals representative of symbols written in the predetermined shorthand on the said surface of the graphical input means, and display means for displaying characters derived from output signals of the analysis means.

2. Apparatus according to Claim 1 wherein the analysis means is adapted to derive its output signals from intermediate signals representing shorthand symbols by recognising and transliterating text represented by the intermediate signals.

3. Apparatus according to Claim 1 or 2 wherein the predetermined shorthand is Pitman's New Era or Pitman's 2000.

4. Apparatus according to Claim 3 wherein the analysis means may be adapted to determine boundaries between strokes representing consonants by detecting angular discontinuities in shorthand outlines representing words.

5. Apparatus according to Claim 4 wherein the graphical input means, in operation, provides signals representing increments of stylus movement, referred to rectangular co-ordinate axes, in writing on the input surface, and the analysis means forms a function representing angular discontinuity at a point in a shorthand outline written on the input surface by summing, for each coordinate direction, a predetermined number of signed unit increments over the outline or a portion thereof including the point

6. Apparatus according to any of Claims 3 to 5 wherein the analysis means is adapted to recognise hooks and loops in the written shorthand and to simplify signals representing outlines by removing those signals representing hooks and loops before determining boundaries between strokes.

7. Apparatus according to any of Claims 3 to 6 wherein the analysis means is adapted to calculate values for one or more of the following features of each stroke in order to recognise the character corresponding to the stroke (a) the arigle of the stroke relative to a predetermined arrangement of sectors; (b) the area enclosed by the stroke and a line joining the beginning of the stroke to the end thereof (a signal may be allotted to the area in dependence upon the direction of curvature of the stroke), (c) the length of the stroke, and (d) average pressure applied to the pen or stylus making the stroke, and to compare each value calculated with reference values in a stroke recognition process.

8. Apparatus according to Claim 2 or any Claims 3 to 7 insofar as dependent on Claim 2, wherein the analysis means is adapted to recognise the intermediate symbols by a process including converting a sequence of shorthand symbols making up all or part of an outline to a sequence of phonemes according to predetermined rules.

9. Apparatus according to Claim 2 or any Claims 3 to 8 insofar as dependent on Claim 2, wherein the transliteration employs logical processes based on spelling rules which depend on characters recognised and the position of the characterin a shorthand outline.

10. Apparatus according to any preceding claim wherein the predetermined shorthand employs a number of short form symbols each representing a certain word, and the analysis means is adapted to recognise the short form symbols.

11. Apparatus according to any preceding claim wherein the predetermined shorthand employs intersections, and the analysis means is adapted to detect intersections and provide output signals representing words corresponding to respective detected intersections.

12. Apparatus according to any preceding claim wherein the analysis means is adapted to recognise certain predetermined combinations of consonants and the position of the combination in a word and, if necessary, to insert a symbol representing the likely presence of a vowel (or vowels) among the characters of such a predetermined combination.

13. Apparatus according to any preceding claim wherein the analysis means comprises a computer.

14. A method of generating a readable display including writing with a stylus in a predetermined type of shorthand on an input surface of a graphical input means which generates output signals representative of the position of the stylus on the said surface, analysing the output signals of the graphical input means according to rules based on the predetermined type of shorthand to recognise which shorthand symbols were written on the said surface, and displaying characters derived from the symbols so recognised.

1 5. A method according to Claim 14 wherein the characters displayed are obtained by transliterating text represented by the shorthand symbols recognised.

16. Apparatus for generating a readable display from shorthand, written on graphical input means, substantially as hereinbefore described and as shown in Figure 1 of the accompanying drawings.

17. A method of generating a readable display from shorthand, written on graphical input means, substantially as hereinbefore described.