GB2151055A - Word processing - Google Patents

Abstract

A word processor, especially an editing transcriber producing text from machine shorthand chords, is provided with additional editing commands which facilitate rapid correction of text. A cue command automatically moves the editing cursor to the next word likely to need editing because it includes an impossible letter combination or is not in a dictionary, specifically was therefore produced by use of a transliteration algorithm. A try command provides predetermined changes to a word which are likely to effect the required correction, e.g. predetermined transposition of letters, correction of common spelling errors and correction of common mis-keyings. The repertoire of changes may be built up by the typist or built up automatically by logging commonly effected corrections. In the case of an editing transcriber the try command may repeat transcriptions, then implement a spelling corrector, then implement an extended transliteration algorithm.

Description

SPECIFICATION Improvements relating to word processing The present invention relates to word processing in general and in particular an editing transcriber for machine shorthand chords.

There are currently two well-known machine shorthand systems known as Palantype and Stenotype respectvely. In such systems, the machine shorthand writer (typist) presses different combinations of keys to write different syllables. Each combination is known as a chord and the machines print out the chords on a strip of paper in a form which can be deciphered by the uninitiated and which can be read easily by a skilled operator. Reference may be made to GB 1 601 215 for general background information on the nature of the machines, their printed output and so on.

Although shorthand machines make it possible to write at extremely high speed, transcription to normal orthography can only be performed at a later time by the typist. A need exists for more rapid transcription, indeed virtually simultaneous transcription, and systems have been devised and used for automatic transcription from the chords to normal orthography text, referred to for simplicity as text, without the implied qualification. These systems are computer-based systems which incorporate a chord-to-text dictionary and search algorithms which find the text words from the coded inputs representing the chords. The text (e.g. in ASCII code) may be displayed and/or printed out and/or stored, e.g. on magnetic disk.

It is necessary to supplement the dictionary by transliteration algorithms which convert unmatched chords into likely letter groups.

Some dictionary matches will be erroneous and words produced by transliteration are very likely to be in error. Apart from obscure English or foreign words, proper names frequently cause incorrect output. Operator errors are also inevitable at high speed. The overall result is text which is quite easy to read but which is nevertheless significantly garbled. A known transcriber is described in the aforementioned GB 1 601 215.

If a perfect transcript is required, the output from the automatic transcription system will need to be edited. The cost or time necessary to produce a perfect report of a meeting is a combination of the cost of producing the automatic transcription and the cost of editing this initial draft transcription to perfection.

These two factors are interrelated in that, the more accurate the draft transcript is, the less editing needs to be done on it.

The accuracy of a draft transcript depends on the skill of the typist, the size of the dictionary employed and the algorithm used to spell non dictionary matched chords. With competent operators and a 20,000 word dictionary somewhere between two and five percent of the words are incorrectly spelled in the draft transcript. The figure however will rise if the listening conditions are poor or if the speaker is not fluent. The amount of editing to be done is thus likely to be significant if a perfect transcription is required. Therefore an important feature of any transcriber is the editor program, its ease of use and its efficiency.

Draft transcripts could be edited using a standard word processing program. This would be less than ideal, however, because the task of editing draft transcripts from a machine shorthand system is different from normal editing in a number of ways.

The invention also concerns improvements to normal (non-transcribing) word processors.

The editing of text requires many cursor operations to move from word to word and, even with "find and replace" available, editing involves repetitive operations to correct errors to which the particular typist is prone or happens to have made consistently in a particular document.

The broad object of the invention is to provide a word processor with additional facilities designed to make editing even more efficient.

A more specific object of the invention is to provide an editing transcriber which will greatly facilitate the editing of transcribed text.

The word processor according to the invention has, in addition to standard editing commands either or both of: (1) A cue command which automatically moves the cursor to the next text word which bears an indication that it may need editing, (2) A try command which provides predetermined changes to words, which changes are likely to effect the connection required thereto.

So far as the cue command is concerned, the indication that a word may need editing may be absence of the word from a dictionary used by the word processing software or the inclusion of an impossible or highly unlikely letter combination within the word, for example the typist can use the cue command to effect rapid clearing up of most typing errors by cuing from word to word, correcting those that actually do need correction. This can be done without reading the text and without laborious use of cursor controls.

In the case of an editing trascriber, the indication that a word may need editing may be that the word was produced by transliteration, which indication may be used alone or in combination with other indications such as are discussed above.

The try command can have a repertoire of changes which automatically invoked when the command is used. If the change works it is accepted. If it fails the typist will have to override it with normal editing commands.

Provided the repertoire is well chosen, correction of many errors will be greatly speeded up.

Examples of changes are: Transposition of letters in groups where the typist is prone to make an erroneous transposition.

Correction of common spelling errors.

Correction of common mis-keyings.

The repertoire of changes may be entered by the typist or be built up automatically by logging corrections and putting them into the try repertoire when they reach a certain frequency of occurrence.

In the case of an editing transcriber, the changes preferably comprise alternative transliterations of chords.

Thus, the invention further provides an editing transcriber comprising a computer with a keyboard and display device, software for transcribing chords to text displayable on the display device and editing software providing a plurality of editing commands. These may comprise a set of standard editing commands such as a Full cursor controls Insert command Typeover command Delete command Move command Find command Substitute command (Find and replace).

In addition, and in accordance with the invention, the editing commands comprise one or more of the following: (1) A cue command which automatically moves the cursor to the next text word produced by transliteration (2) An unprocessed command which causes a predetermined section of text marked by the cursor to be replaced by the chords from which that text was transcribed (3) A try command which applies at least selected ones of a sequence of algorithms to the chords of marked text words to try a succession of alternative transliterations thereof.

The cue command speeds up editing because it will enable the operators to go immediately to the majority of words needing correction, without the relatively slow use of conventional cursor control commands. These remain available to move to other words (dictionary-transcribed) which nevertheless need editing.

The unprocessed command preferably replaces the whole line of text wherein the cursor lies by the corresponding chords, which will normally spill over into the next line of text. The operator can thus get assistance, when he is puzzled or baffled by the text word, by going back to the chord which he typed.

The try command may initially repeat transcription. This is of value when the dictionary, or a supplementary word list, is amplified or updated during editing, as described below.

Another attempt at transcription can then replace an erroneous transliteration by a correct dictionary transcription.

The try command may next implement a spelling corrector which replaces the incorrect word with a common word which looks very similar.

The try command may finally implement an extended transliteration algorithm. The initial transcription uses a relatively simple algorithm for speed whereas the extended algorithm can provide a series of possible transliterations.

preferably in order of decreasing likelihood, allowing the operator to adopt that which he finds acceptable.

The utility of these commands can be seen when the editing task is analysed. The editing task can be divided into four main parts 1. Locating a mis-spelled word 2. Deciding what the correct words should be Clues to the correct word come from a number of sources: A. Graphemes bphonemes bword Letter groups in the mis-spelled word may give a clue to the sound of the correct word.

B. Sentence structure The sentence structure will provide clues as to the type of the word, such as noun, verb, adjective, etc.

C. Context The subject matter of the manuscript may also aid the selection of the correct word.

3. Translating the correct word into its orthographic Word#Graphemes 4. Implementing editing operations to bring about a transformation of the mis-spelled word to the correctly spelled word.

The cue command assists in step 1. The unprocessed and try commands assist both in steps 2 and 3 and the try command assists also in step 4 since it is easier to accept a word offered by the try command than to edit by normal word-processing techniques.

The invention will be described in more detail, by way of example, with reference to the accompanying drawings, in which: Figure 1 shows an example of text to be edited, Figure 2 illustrates operation of the cue command with this text, Figure 3 illustrates operation of a link command, Figure 4 illustrates operation of the unprocessed command, Figure 5 illustrates operation of the try command, Figure 6 is a flow chart for the cue command, Figure 7 is a flow chart for the unprocessed command, Figure 8 is a general flow chart for the try command, Figures 9 and 10 show a flow chart for updating the dictionary, and Figure 11 is a flow chart for discarding words to make room in the dictionary.

Description is now given in the specific context of Palantype.

An example of a draft transcript is shown in Fig. 1. This contains rather more errors than would normally be expected in order to illustrate more clearly the types of errors which can occur.

It can be seen that the command menu is displayed along the bottom of the screen. This menu only contains those commands which are valid at the time. If there are more valid commands than can be fitted on a single line an "Other" command is shown which allows access to a second menu. A command is selected, from a menu displayed, by typing the initial letter of the command name on the keyboard. It is assumed that "Try" is among the other commands.

As the final document is intended to be a verbatim report based on a shorthand report there will be little large-scale editing, such as moving paragraphs which sometimes occur in some editing tasks. In addition the editing of draft transcripts differs from that of normal typed text in a number of other important ways.

A. It is easier to predict where editing is required in the transcript because information about the transcription process is present.

Where a transcription algorithm is weak, spelling errors are likely to occur.

B. Due to the syllabic nature of palantype coding there are a large number of false word boundaries.

C. A significant proportion of the misspelled words are unrecognisable without reference to context. These are generally uncommon words which have been mis-keyed by the palantypist.

D. There is substantially more repetition of particular errors than would be expected in normal text.

E. Correctly spelled but wrong words occur fairly frequently.

On the basis of the above characteristics the specialised editing commands have been added to the standard word processing commands. Some of these commands make use of information which is not normally available in conventional editing systems. The file from the transcription software contains the translated draft transcript interleaved with semiphonetic data (Palanforms) and processing information. Although normally only the draft transcript is displayed, this interleaved data is always available for use by the editor software.

The characteristics A, B and C above have been noted and the specialised commands explained above have been incorporated into the editor to utilise them and to improve editor efficiency.

Cue command The cue command allows the user quickly to locate likely editing points. When this command is used (by typing C) the cursor rapidly scans from its current position to the next most likely editing point. Transliterated words are seldom correctly spelled and would be some of the words which the cursor would locate. Fig. 2 shows the places where the cursor would stop on an example of text.

Link command The Link command is used to remove false word boundaries. This feature takes the form of a word joining command. The effect of using this command on a fragmented word is shown in Fig. 3. The link command must be issued four times to link the five fragments together.

Unprocessed command It may be the case that weaknesses in the transcription algorithm obscure the correct wording of a sentence and the palantypist may wish to refer back to the original palanforms. The Unprocessed command gives the user the ability to display the original palanforms from which a piece of text originated.

Fig. 4 shows the effect of this command. A single line of text has been shown in its unprocessed form but, as can be seen, the unprocessed text is generally slightly longer than the processed version and spills over onto the next line. When in this mode cursor commands can be used to change the line which is being displayed in its unprocessed form.

Try Command When activated the Try command will attempt automatically to perform the correct edits for the user, in order to reduce the number of mental and physical processes required to perform a given editing task. Automatic editing can be very successful when dealing with the following commonly occurring error types in the draft transcript: 1. Uncommon words which have been correctly palantyped but are not in the dictionary.

2. Common words which have been slightly mis-keyed.

The automatic editing techniques which may be employed are: 1. Deriving editing operations from edits made in the past Because of the number of spelling errors which occur frequently in the draft transcript the user may find herself/himself repeating sequences of edits many time. If we could recall editing operations which have occured in the past we could avoid the need to repeat these same editing operations in the future.

This requires the editor to have the ability to learn from edits which have been performed, and is discussed in detail in a later section of this specification.

2. Spelling corrector Using a dictionary of approximately 2000 common words we try to find a word which closely matches a mis-spelled word. It is to be expected that the spelling correction technique will be most successful when employed to correct 'small' spelling mistakes on common words. A 'small' spelling error is one which falls into one of the following categories: (a) One letter incorrect (b) One letter missing (c) An extra letter inserted (d) Two adjacent characters transposed.

This method however is likely to fail, or indicate an incorrect word, if the word is uncommon or badly spelled.

A program has been developed to demonstrate the possibilities of a spelling corrector, and this has confirmed earlier expectations.

The spelling corrector performs well under the condition mentioned above but, when a spelling correcting technique is used on a 'large' spelling mistake, or on an uncommon word, the technique does not always fail but will sometimes offer a wrong word in place of the mis-spelled word. The following corrections are examples: tuning to during whic to which thits to this numbler to number rom to from.

3. Transliteration technique As editing progresses, more accurate word boundary information becomes available.

Transliteration rules using this extra information have a higher probability of producing the correct spelling of the word, than the transliteration rules used during the original transcription.

The transliteration technique has been implemented and proved quite successful in coping with words which had been correctly palantyped. Fig. 5 shows some examples of transliteration applied within an actual draft transcript.

Although transliteration does not always yield the correct spelling of a word, it will often reduce significantly the amount of editing that the user is required to perform.

There are a number of possible ways in which the Try command may work.

1. The Try command chooses the technique which it thinks will yield the best result.

This choice may be based upon certain characteristics of the mis-spelled word. For example, the chosen characteristic may be: (a) The number of syllables in the word, e.g. a multi-syllabic word may trigger the transliteration technique.

(b) Digrams which occur in the word, e.g.

an uncommon digrams may trigger the spelling corrector.

2. The Try command uses all the techniques available and then chooses the word which it thinks is most likely to be correct.

Possible bases for deciding between words may be: (a) Most agreement: it may be that two of the techniques result in the same word and the other technique produces a different one. The Try command may then choose the word for which there is most agreement.

(b) Word length: Another possibility is that the Try command may choose the word which is closest in length to the mis-spelled word.

The editor software automatically writes the information that it has acquired during the editing process to a file on the disk. The information is written to the file in a form which is compatible with the source file used to create the transcription dictionary. This file can thus be used to update the transcription dictionary and hopefully provide better draft transcripts in the future.

The editor's function is to correct errors in the draft transcript caused by a combination of palantypist's errors and short-comings in the transcription algorithms. The user provides the information necessary to correct these errors, in the form of editor commands. In a conventional editor this information is discarded and cannot be used again. The information provided by the user however reveals details about the individual palantypist and the subject matter of the transcript.

The details revealed about the palantypist are: 1. Common strokes mis-keyed by the palantypist.

2. Individual preference when selecting a palanform to represent a word, when several different palanforms can be used.

Subject matter details are revealed in the form of unusual words which have occurred in the conversation but were not found in the dictionary.

If this information is retained within the editing system, it could be made use of in a number of useful ways.

In the Try command the information can be used to repeat editing operations. The user can ask the editor to sift through its memory and try to find out if the information required to correct a particular word, is available. If this information is found, the editor can perform the edits automatically.

In the overall speech transcription system the information can be used constructively to increase system efficiency. If the information is made available to, and made use of by, the transcription process, we have a system which can not only adapt to the characteristics of the individual user but also to the type of job the user is performing. Such a system will be able to improve gradually the quality of its original draft transcript and therefore reduce the amount of time required to edit the draft to produce a final copy.

The user however may not wish all of the information gathered to be included in the system dictionary. An error palanform of one word may correspond to the correct palanform of another word and could only be included in the dictionary by displacing the original word.

The information not sent to the dictionary could still be retained within the editing system so that the user would have the ability of selectively overriding the original processing by using the Try command.

The volume of information gathered during the editing of a large draft transcript may become too large to manage efficiently. Some of the information gathered may prove to ge of only limited value and if it were to be retained within the system the available memory would soon become exhausted. A memory system which could estimate the value of an item of information and be able to discard information if necessary has been developed.

The implementation of the Cue, Unprocessed and Try commands will now be described.

The Cue command Description: The Cue command is used to automatically move the cursor onto the nearest word, following the cursor, which was transliterated and is thus likely to require editing.

Format: C where C is the character used to start the cueing operation Method of use: Initiate Cueing operation When the cueing operation is initiated, by pressing C, the cursor will move forward through the text of a likely editing point in the text. If the Esc key is pressed, during the cueing operation, the cueing operation will be terminated and the cursor will stop before it reaches a likely editing point. The flow chart is shown in Fig. 6.

The Unprocessed command Description: The Unprocessed command allows the operator to view the original source Palanforms from which a line of text was generated.

Format: U < Escape > where U is the character used to put the editor into Unprocessed mode Escape is the character used to return the editor to the command mode Unprocessed mode menu line indication: Unprocessed: < Esc > The flow chart is shown in Fig. 7.

The Try Command Description: When the Try command is initiated, the editor will attempt to automatically perform the edits required to correct a mis-spelled word. The Try command will not always succeed but has a good chance of success particularly if the word has occurred previously in the text and was corrected by the user on that occasion.

Form: T where T is the character which activates the Try command.

The overall flow chart is shown in Fig. 8.

The first part of the algorithm makes use of a supplementary word list. Words which have been modified during the editing process are stored, along with their palanforms, in this word list within the memory. The information which has gathered in this word list can be used by the Try command to repeat past edits.

A section of memory, of fixed size, is allocated for the word list. The of amount memory allocated may be sufficient for approximately 100 words.

Each entry in the word list is made up of a word or phrase and its palanform. Entries are added to the word list until the memory allocation is used up. Before an additional entry can then be added to the word list, space must be created by removing an old entry. This function is performed by a "discard" algorithm which chooses an entry in the word list to be removed. This choice is based on criteria of age and popularity.

The word list is implemented using a linked list data structure. The memory space allocated for the word list is divided up into a number of equal sized records. An entry in the word list can occupy one or more of these records. Each record in the word list has the form: Pointer or link Flag and age counter Popularity counter Palanform Word or phrase The link points either to the next entry in the list or to a continuation record if the entry is too big to fit into a single record. The flag is used to indicate that the record contains the end of an entry.

As the editor performs changes to a word, under the instructions of the user, it stores information about these changes in temporary buffers. The word list is only updated with this information when the user moves the cursor on to and then proceeds to edit a different word. The relevant flow charts are shown in Figs. 9 and 10. The effect of the "update" operation shown in Fig. 9 and in more detail in Fig. 10 is that the word buffer should contain the English equivalent of the old palanform.

Only limited space is available in memory for building the word list and there is therefore a high probability that the word list wil become full during the editing of a large manuscript. Information about potentially frequently occuring incorrect words would therefore then be excluded from the word list unless some method of discarding entries in the word list exists.

The object of the "discard" algorithm is to try and ensure, that at the end of editing session, the word list contains information about those incorrect words which have occurred most frequently during the editing of the manuscript.

If an entry has to be discarded from the word list, it would be desirable if it was an entry which was unlikely to be of any further use. Some method of estimating the future value of an entry is required.

A "discard" algorithm has been developed to perform this function. To estimate the future value of an entry the "discard" algorithm looks at how valuable the entry was in the past. This is realised by storing a popularity count along with each entry in the word list.

Each time an entry in the word list is used by the editor its popularity count is increased by one.

Each entry in the word list has also an age associated with it. An entry in the word list cannot be discarded until it reaches a certain age. This gives an entry time to become established in the word list. Entries above this certain age are eligible to be discarded. When an entry in the word list is discarded all the remaining entries in the word list are aged by one.

Entries are added to the word list until the space allocated to the word list becomes exhausted. Before a new entry can be added to the word list, a space must be created by discarding one of the entries already in the word list. The "discard" algorithm will choose the entry which is to be discarded. This entry will be the least popular of all those which are eligible to be discarded. If there is a number of entries which are equally unpopular then the oldest of these entries will be discarded.

The flow chart appears in Fig. 11.

The procedure for removing an entry in the word list is as follows: 1. Break the chain of entries before the entry to be removed.

2. Detach the entry to be discarded from the start of the final half of the chain.

3. Join the two halves of the chain back together again.

4. Free the space occupied by the detached entry, for further use.

If use of the supplementary word list fails, the Try algorithm (Fig. 8) moves on to the spelling corrector.

A number of algorithm can be used to measure the similarity of words. The particular implementation used at present is essentially to calculate a correlation function between the test word and a dictionary word. To do this a count of the number of matching letters is made for all relative positions of the two words see diagram below. Large peaks towards the centre of the correlation function indicates that the words are very similar. The two highest number of matches near to the centre of the function are added together and, if the sum is of the same magnitude as the length of the test word, a close match is indicated.

Matches whic 0 which 0 which 0 which 0 which 4 which 0 which 0 which O which thits 0 this 0 this 0 this 3 this 1 this 0 this 1 this 0 this Matches turing 0 during O during 0 during O during 0 during 5 during 0 during 0 during 0 during 0 during 0 during The final feature of Fig. 8 is the extended transliteration. An example of a known transliteration technique based on Boolean logic (a series of yes/no decisions) is given in GB 1 601 215, especially Figs. 18 to 32 thereof. This may be the technique which is employed in the original production of the draft transcript. The extended transliterations of the Try command, on the other hand, Transliteration from the phonetic Palantype shorthand code into English is a difficult task, prone to error.For each Palantype chord (or syllable) there are a number of possible English outcomes, depending on numerous factors, such as the position of the chord within the word, etc.

In the Palantype system, normally only the most frequently correct transliteration is shown. The extended transliteration algorithm incorporates a range of different transliteration rules for each Palantype code, and applies them successively in order of decreasing frequency of occurrence. Thus, the most likely spelling of a Palantype code will be shown first, and if this is not correct, the user will select the "Try" function, and this will automatically show the next most likely transliteration, and so on, for each successive operation of the T key. There may be as many as four or five different transliterations available for the user to select from. Finally, if none of the suggested transliterations is correct, the user must correct the error in the normal way, using either the "Delete" and "Insert'' or "Replace" functions.

The extended transliteration may moreover make use of information about phoneme position within a word to increase the predicted accuracy of the phoneme-grapheme transliteration.

The transliteration rules may be implemented using a data structure known as a trie structure to hold the spelling rule, instead of the boolean logic of the original transliteration software. A structure of tries is a method of dictionary searching originally proposed for matching variable length names in compilers (De la Briandais, R., "File searching using variable length keys", Proc. of the Western Joint Computer Conference, v15, pp 295-298, 1959).

The editor software automatically writes the information that it has acquired during the editing process to a file on the disk. The information is written to the file in a form which is compatible with the source file used to create the transcription dictionary. This file can thus be used to update the transcription dictionary and hopefully provide better draft transcripts in the future.

Claims (10)

1. A word processor comprising a computer with a keyboard and display device and editing software providing a repertoire of editing commands including a cue command which automatically moves an editing cursor to the next text word bearing an indication that it may need editing.

2. A word processor according to claim 1, which includes a dictionary and wherein the said indication is absence of the word from the dictionary.

3. A word processor according to claim 1, wherein the said indication is occurrence of an impossible or highly unlikely letter combination within the word.

4. A word processor according to claim 1, wherein the word processor is an editing transcriber which produces the text being edited by automatic transcription of machine shorthand chords, the transcription being effected by use of a chord-to-text dictionary and supplementary transliteration algorithms, and wherein the said indication is production of the word by transliteration.

5. A word processor comprising a computer with a keyboard and display device and editing software providing a repertoire of editing commands including a try command which provides predetermined changes to words, which changes are likely to effect the correction required thereto.

6. A word processor according to claim 5, wherein the processor is arranged to log corrections and to put them into the repertoire of try changes when they reach a certain frequency of occurrence.

7. A word processor according to claim 5, wherein the word processor is an editing transcriber which produces the text being edited by automatic transcription of machine shorthand chords, the transcription being effected by use of a chord-to-text dictionary and supplementary transliteration algorithms, and wherein the try changes comprise alternative transliterations of chords.

8. An editing transcriber comprising a computer with a keyboard and display device, software for transcribing machine shorthand chords to text displayable on the display device, and editing software providing a repertoire of editing commands including one or more of the following: (1) A cue command which automatically moves the cursor to the next text word produced by transliteration (2) An unprocessed command which causes a predetermined section of text marked by the cursor to be replaced by the chords from which that text was transcribed (3) A try command which applies at least selected ones of a sequence of algorithms to the chords of marked text words to try a succession of alternative transliterations thereof.

9. An editing transcriber according to claim 8, wherein the unprocessed command replaces the whole line of text wherein the cursor lies by the coresponding chords.

10. An editing transcriber according to claim 8, wherein the try command implements in sequence firstly repeat transcription, secondly a spelling corrector and then an extended transliteration algorithm.