GB2165974A - A system for communication with a machine - Google Patents

Abstract

Musical intervals are used as codes to communicate preselected commands or inputs to a computer. The system can use the human voice, sounding notes (DO-RE, etc.) successively to establish the interval and can use electrical circuitry and programs for converting the received notes or tones into pulse trains of the same fundamental period and for storing and calculating the period and interval. When a particular interval is received the apparatus including a digital micro-computer may execute a pre-set recorded subroutine and restore the circuitry for the reception of a second interval. <IMAGE>

Description

SPECIFICATION Computer communication system BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to communication to machines and is especially concerned with vocal communication of commands to a computer.

Description of the Prior Art A great deal of interest in and research has been done on vocal communication by humans to computers. Advantages of, applications for, and common systems for vocal communication with computers were surveyed in an article entitled "Toss Your Keyboards and Just Tell Your Computer What to Do" by A. E. Conrad in the January 1984 issue of Research & Development, Vol. 26, No. 1, pages 86-89.

As pointed out in this publication, most work in this field has been on speech recognition systems which are speaker-specific or speaker dependent and speaker independent systems.

The former is able to recognize only words spoken by an individual while the latter may recognize the same word spoken by a number of individuals. The speaker-specific system can currently recognize (or be taught to recognize) a larger number of words than the speaker independent system and consequently can control more functions. Both systems rely on the creating of "templates" of spoken words and matching the received word to the templates.

A great deal of effort has been expanded in this area and numerous patents and articles exist as evidenced by the survey of technical publications found in the Description of the Prior Art section of the Marley U.S. Patent No. 4,284,846. This prior patent teaches one system for analyzing and comparing words by comparing certain waveform characteristics with pre-stored ratios.

Other patents disclosing speech recognition systems are: Sahoe, U.S. Patents Nos. 4,286,114 and 4,319,221; B. H. An U.S. Patent No. 4,292,470; Welch, et al U.S. Patents Nos. 4,319,085 and 4,336,421; Kellett U.S. Patent No. 4,343,969; Pirz U.S. Patent No. 4,349,700; Taniguchi et al U.S. Patent No. 4,389,109; Hitchock U.S. Patent No. 4,388,495; Duifhuis et al U.S. Patent No. 4,384,335; and Rothschild et al U.S. Patent No. 4,399,732.

Such systems are quite complex and expensive or else extremely limited in their abilities. For example, a typical speaker independent system might recognize about 10 words (e.g. the ten digits) while a speaker-specific system can recognize perhaps an order of magnitude higher, of 100 to 200 words.

SUMMARY OF THE INVENTION The present invention differs from the systems disclosed in the reference patents cited above by providing a relatively less complex system which takes advantage of the physical fact that the human voice can extend over a much wider range than is normally necessary to be used, or is normally used in speech, to economically achieve a speaker independent system that can recognize a larger number of different vocal commands. A system constructed in accordance with the teachings of the present invention senses and recognizes tonal differences which are treated as coded signals to perform preset defined functions.

The tonal differences are, in accordance with one feature of the present invention, tones in a musical scale such as the common diatonic scale and means are provided for setting the key for each different user by comparing different tones such as the user's subjective middle C tone and his/her D tone voiced subsequently. The system of the present invention employs and recognizes musical intervals as commands or inputs.

The system lends itself to many applications, including input into and control of a computer in an environment where it is not practical to use other types of input systems. As in dark rooms (for example, electronic microscope rooms) or where the user must employ his hands in other tasks, as in a production process. The system is of great utility to the disabled, especially those who may easily not operate conventional computer terminals.

The same recognition system of the present invention, together with the advantages thereof, may best be understood by reference to the following description taken in connection with the accompanying drawings, in the several figures of which, like reference numerals identify like elements.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a block circuit diagram of a sound recognition system constructed in accordance with the teachings of the present invention with waveforms at various points indicated; Figure 2 is a circuit diagram of the wave shaping circuit portion of the system shown in Fig.

1; Figure 3 is a circuit diagram of a period measurement circuit and other portions of the system shown in Fig. 1; Figure 4 is a set of wave forms useful in understanding the operation of the circuit of Figs. 2 and 3; Figure 5 is a flow chart useful in illustrating the overall operation of the system shown in Figs.

1-3; Figures 6, 7 and 8 are flow charts illustrating the operation of alternative embodiments of the system; and Figure 9 and 10 are alternative flow chart sub-routines that may be substituted for a portion of each of the flow charts shown in Figs. 6, 7 or 8.

DESCRIPTION OF THE PREFERRED EMBODIMENTS The principles of the present invention are capable of being applied with hardware and/or with software in a variety of manners, several of which, with variations, will be described herein.

A A system constructed in accordance with the present invention is shown in Fig. 1 and is generally identified therein by reference numeral 10. The system 10 includes a transducer 12 such as microphone, which serves to pick up sound waves such as depicted by waveform 14.

Such waves, as in the case of a person signing the note "C", has a basic period "T" for a rate of that pitch. To simplify the electronics, the electric analog of the received sound waveform 14 is converted to a pulse wave train 16 having a period "T" by a wave shaping circuit 18.

As an alternative to the transducer 12, an auxiliary electric signal input 20 may be provided.

This can be, for example, a telephone input for remote activation, or a tone generator.

The wave train 16 is received by a period measurement circuit 22 which measures and digitizes its period and feeds this information to an input/out interface 24 which interfaces with a a digital computer 26.

In the overall operation of the system 10, the system 10 receives tonal signals at-microphone 12 or input 20 and responds to selected ones of these to operate pre-programmed routines in the computer.

To understand the principles behind the operation of the system 10 of the invention, its application will be described with reference to the familiar diatonic scale, it being understood that it can be applied to other scales. The modern diatonic scale has intervals that are dependent of the frequency of any particular tone (although once one frequency is set it determines the frequency of the remainder of the tones) and can be expressed in part by the following table:: ANY ARBITRARY INTERVALS DIATONIC SCALE MI' 2.5000 RE' 2.2500 DO' 2.0000 TI 1.8750 LA 1.6667 SO 1.5000 FA 1.3333 MI 1.2500 RE 1.1250 THE REFERENCE DO 1.000 TI, 0.93750 LA, 0.83333 SO, 0.75000 FA, 0.66667 MI, 0.62500 RE, 0.56250 DO, 0.50000 TI, 0.46875 Since most humans are familiar with this scale and even children can readily do a DO, RE, ME, FA, SO, LA, TI, DO (subconsciousiy choosing a reference frequency and relating it to the other tones by the above intervals) the present invention determines and reacts to intervals as its means of receiving information.

Thus, with reference to the flow chart of Fig. 5, a start command may be the reception of a detected tonal voice ("RE") at Stage A. This signal is measured to determine if it has a repeating period of sufficient duration (to avoid false activations) at Stage "B". If a reference signal exists (e.g. "D") and is stored, the two are compared and if an interval of 1.125 is calculated at Stage "C" a fetch and execute sub-routine is executed at Stage "D" recalling and executing a pre-recorded sub-routine for the interval 1.125. At the conclusion the system is rest at "E" and ready to receive a second signal (e.g. "FA") and respond to it in the same general manner.

To prevent the system from being speaker-specific, and allow its use by anyone with minimal musical ability, the reference signal is established by the same process. As a start up, the user need only sound "DO" into the microphone 12 for a short period of time and then sound "RE".

The system treats the first tonal sound received as establishing the reference signal and the second one and subsequent ones as possible command signals.

A A preferred embodiment for the wave shaping circuit 18 and its interconnection to the microphone 12 and auxiliary input 20 is shown in Fig. 2. Specific electrical values are given in Fig. 2 for the components employed, but of course many other values can be employed as is well known to those skilled in this art. However, the values and the connections of the circuit element shown in Fig. 2 worked very well in a prototype.

More specifically, the microphone 12 was connected between chassis ground and a reactive impedance 27 to a circuit point 28. This point 28 was also connected through a resistor 32 to chassis ground and through a capacitor 34 to the auxiliary input 20.

Whichever signal input 12 or 20 receives a signal, that input 12 or 20 feeds it through to a low frequency amplifier 36 formed of an operational amplifier 38 whose negative input (pin 6) is connected to the point 28 and whose positive input (pin 5) is connected to bias voltage (plus 5 volts) through resistor 40 and to chassis ground through the parallel connection of capacitor 42 and resistor 44. The operational amplifier 38 has its pin 4 connected to a source of positive bias (12 volts) through a current limiting resistor 45 and to chassis ground through a capacitor 46. Some of the output of the operational amplifier 38 is fed back to its negative input via the parallel connection of a resistor 47 and capacitor 48.

The primary output of the amplifier 36 is fed to a low pass filter consisting of a resistor 51, one side of which is connected to the output of the amplifier 36 and the other side of which is connected through a capacitor 52 to ground and through a resistor 53 to a circuit junction 54.

The output of the low pass filter 50 is fed to a junction 54 and from there to a comparator 55 and to a peak-voltage-follower-with-decay circuit 56. The signal is fed (a) through a current isolating diode 57 to the primary positive signal input (pin 12) of an operational amplifier 58 of comparator 55 which input is also connected to ground through a resistor 59 and (b) to the peak-voltage-follower-with-decay circuit 56. The output of the circuit 56 provides the primary negative signal input to an input pin (pin 13) of the comparator 55.

The circuit 56 is preferably formed by an operational amplifier 60 whose primary positive input (pin 3) is connected directly to junction 54 and whose output (pin 1) is fed back directly to its negative input (pin 2) and through an isolating diode 61 to the negative signal input of amplifier 58 with the output also being connected through a resistor 63 and capacitor 64 to ground. The resistor 63 and capacitor 64 in parallel have such a discharge time constant that at the comparator 55 the inverting input always has a greater effect than the non-inverting input except for the most significant peak of each cycle of the signal appearing at junction 54.

The output, waveform 16, of the comparator 55 is taken from the junction of a pair of resistors 65 and 66 connected in series from the output (pin 14) of the operationa! amplifier 58 to ground. This output is fed to input, ST, of the period measurement circuit 22.

Referring now to Fig. 3, there is illustrated a preferred embodiment of the period measurement circuit 22 and the input/output interface 24 along with interconnections of same to the computer 26. The specific computer 26 is preferably Apple II plus (R.T.M), and the specific interconnections for that unit are shown.

The pulse train 16 is coupled through an operational amplifier 70 which serves as a Schmitt Trigger, to a shift register 72 whose outputs (pins 3 and 4) are coupled through an inverter 74 and period measurement gate 75 to a counter 78 which is coupled through buffers 80 to the computer 26.

Outputs from the computer 26, are taken from its R/W, AO, Al, A2 and DS (device select) outputs as well as from a source of bias (5 v.) and timing pulses and are supplied as shown in Fig. 3. Gates 81 and 82 serve to deliver the reset command over a line 83.

The functioning of the circuitry of Figs. 2-3 and computer 26 is better understood by reference to Fig. 4 which interrelates the waveforms 14, 16 and the inputs at CP to the shift register 72, the timing pulses from the computer 26 also delivered to a period measurement gate 75 as well as the output of the shift register 72 corresponding to the start and stop of the measured period "T".

In operation, the circuits of Figs. 2 and 3 first reduce the input signal 14 to a pulse train 16 shaped so as to have one pulse per cycle at point ST (Fig. 2 and Fig. 3), the input to the Schmitt Trigger 70. The computer 26 resets the shift register 72 and Counter (over line 83 and through an inverter 86) and prepares the circuit for the next period measurement.

Upon arrival of a first rising edge of the shaped wave train 16 at CP of the shift register 72 its output QO goes from low to high (voltage levels). The counter 78 starts measuring the current period of the input wave train.

Upon the arrival of a second rising edge of the shaped wave train at input CP of the register 72, its output Q1 goes from low to high. The counter 78 stops counting. At the same instant, the input at the most significant digit of the higher byte buffer 90 of the buffers 80 is also set from low to high representing that the period measurement has been completed.

The computer 26 reads the higher byte buffer 90. If the most significant digit is high it is neglected and the computer 26 values the least significant 7 bits as Q8 to Q14 of a 15 bit binary number. (If the most significant digit is low that means period measurement is unfinished.) The computer 26 also reads the lower byte buffer 92, values the reading as the least significant 8 bits of the 15 bit binary number, that is QO to Q7. The computer 26 interprets the magnitude of this 15 bit binary number as the magnitude of the period just measured.

The operation of the system 10 can be further appreciated from the flow chart of Fig. 6.

Here, the start sequence A feeds through a reset measurement circuit (gates 81-82 and their associated leads) for which it decides at B1 as to whether or not a periodic signal has been received. If not the system cycles back to await such a signal. If yes, it takes a reading at B3, and if the "Jth" (any number, e.g. 30th) reoccurrence of the same pulse period T in succession at stage B4 allows an average to be computed at Stage C.

The flow chart of Fig. 6 is for single-interval messages. It is for isolated inputs of reference and signal wave trains wherein there is always a break between successive wave trains. The block I serves to detect these breaks.

If the output of block I is the first tonal signal detected it is treated as the reference signal and it is stored. If a second or successive signals are identified they are compared with the reference and an interval computed. The computed interval is attempted then to be matched, and if a match is found, the associated subroutine is executed and the program reset.

In practical use a computer terminal could display a menu such as the following: I AM AT YOUR SERVICE. 'SING' YOUR CHOICE.

(DO DO) FOR (LIST PROGRAMME IN MEMORY) (DO RE) FOR (DISPLAY PATTERN 'HO') (DO ME) FOR (TEXT MODE DISPLAY) (DO FA) FOR (FLASH MODE DISPLAY) (DO so) FOR (PLAY RUNNING TONES) (DO LA) FOR (ACTIVATE EXTERNAL DRIVE TO CATALOG PROGRAMMES ON DISK) (DO TE) FOR (DISPLAY 'TE') (DO DO') FOR (ACTIVATE EXTERNAL DRIVE TO SAVE THIS PROGRAMME ON DISK, AND EXECUTE ANOTHER PROGRAMME ON DISK, AND RETURN) And the user need only "sing" the requested commands to activate the computer.

A suitable listing for use in the program of Fig. 6 is as follows.

In particular, system 10 will operate with a new reference for each command if line 93 of the above listings is modified as follows: 93 W=O Such operation permits the same or a different speaker to freely change his or her reference from command to command.

Referring now to Fig: 7 there is illustrated therein an alternative flow chart for the system of the present invention and is designated by reference numeral 100. This chart depicts the program for N-interval messages. That is, it is for a multi-tonal coding. E.g. where DO-RE-ME and DO-RE-FA are different signals.

A suitable listing for carrying out this program is as follows: A further alternative flow chart for an oral program is shown in Fig. 8. By inputting a sequence of sounds the speaker effectively defines an oral program (e.g. a process) consisting of a desired sequence of vocal commands for subsequent execution.

A suitable listing for carrying out this program in accordance with the flow chart of Fig. 8 is set forth below: Fig. 9 illustrates an alternative subroute, labeled II, which can be substituted for the flow diagram block labeled I in Figs. 6, 7 and 8 and which allows the system to handle slurred inputs, i.e. slurred wave trains, when substituted for any of the blocks I in Figs. 6, 7 and 8.

Such operation is advantageous to a human speaker as less effort is required in producing slurred vocal sounds than isolated ones.

A suitable listing for implementing the program of block II is: This may be used, e.g. in place of lines 2420-2485 of the program listing for the flow chart of Fig. 6 listed above. It should be noted that the pause of Fig. 9 should be long enough to prevent the current wave train (of the flow charts of Figs. 6, 7 and 8) being mistaken for a follow on reading, i.e. mistaken for a "next wave train" and the user should take care not to continue to produce such a wave train for longer than the pause.

Fig. 10 illustrates another alternative subroute labeled III. The flow diagram block labeled Ill, modifies the system 10 to allow the user to in effect extend his or her frequency range by producing and holding a wave train output for longer than normally required. That is, for longer than the pause period. When the machine detects a wave train lasting longer than a predetermined duration it modifies the data by a scaling factor to get a virtual interval before message identification (i.e. before interpretation).

The term m is block ill of Fig. 10, is a scaling factor which may take any particular value in a range. Two particular values, 0.5 and 2 are especially useful for the human speaker. When m=0.5, the machine listener performs upper octave transposition and when m=2, lower octave transposition. (I.e. DO-RE, interval 1.125, if m=0.5 and if the RE is held, measures interval 2.5 or DO-RE'). Repeated transpositions are realized when the speaker or singer further maintains the wave train. This means that the frequency range of the speaker is virtually extended. Also, verbal inputs permit the user to operate within a comfortable frequency range and yet achieve a large number of intervals as if he or she had a much wider voice frequency range. Thus a greater number of different "word" signals may be given using only a few notes.

A suitable listing for the program in block Ill of Fig. 10 is as follows: From the foregoing description it will be apparent that the present invention teaches a novel system for communication with a computer. The system uses the concept of communicaton by a musical interval code and this substantially extends the number and ease of recognition of oral commands.

The system is economical and efficient to implement and easily learned and effectively used. It is not user specific and yet provides for a large number of "words" (intervals) to be recognized while being economical to construct and use.

While several embodiments of the system of the present invention have been shown and described, it will be understood by those skilled in the art that changes and modifications may be made to the system 20, 200 or modifications thereto described herein without departing from the teachings of the present invention. Accordingly, the scope of the present invention is only to be limited as necessitated by the accompanying claims.

Claims (18)

1. A system for communication by means of tonal signals with a computer having a series of subroutines prerecorded therein in association with different tonal intervals, comprising: (a) means for converting several tonal signals into a signal representative of the period of the fundamental tone thereof; (b) means coupled to said tonal signal converting means for calculating the interval of a received tonal signal and a reference tonal signal; (c) means responsive to said converting and calculating means for selecting and running a prerecorded subroutine associated with the particular interval calculated by said interval calculating means.

2. The system of claim 1 being operable to respond to a reference signal established by a tonal signal inputed into said converting means in sequential order with a command tonal signal.

3. The system of claim 2 being operable to respond to musical tones in a musical scale.

4. The system of claim 3 wherein the scale is the modern scale with intervals including 1.000, 1.250, 1.500 and 2.000 operable with tolerance.

5. A voice actuated computer control apparatus comprising: a microphone for converting sounds to electrical signals; means coupled to said microphone for converting the electrical signals into a pulse train at the fundamental period of the received voice sounds; means coupled to said converting means for measuring the period of said pulse train and for comparing it with a reference period; means for calculating the interval between the period of said pulse train and said reference period; means responsive to the above named means for causing the computer to execute different routines in response to selected calculated different specific interval values.

6. In a human voice recognition system, the improvement residing in means for responding to commands in the form of tonal signals that are related by specific musical intervals and in which system a number of specific different commands are recognized and associated with a like number of different intervals.

7. The system of claim 6 wherein said means are operable to create a different interval by scaling the interval detected if one of the tones is repeated over a continuous period beyond a preselected time.

8. In a voice recognition system the improvement residing in: a wave shaping circuit operable to respond to a tonal input to produce a pulse train of the same period as the fundamental component of the tonal signal; a period measurement and comparison circuit for receiving and measuring the pulse train output of said wave shaping circuit including means for comparing the pulse train period with a reference period and means for calculating the musical interval thereof; and means for responding to specific intervals and not to others.

9. A method for human voice communication with a computer through interfacing circuitry comprising the steps of: presenting commands to the interfacing circuitry in the form of tonal signals; relating said tonal signals to specific musical intervals; calculating the interval to a received tonal signal to a reference to tonal signal; and selecting and running in the computer a prerecorded subroutine protocol associated with the particular interval calculated.

10. The method of claim 9 wherein said tonal signals are musical tones in a musical scale.

11. The method of claim 10 wherein said musical scale is the modern musical scale with intervals including 1.000; 1.250; 1.500; and 2.000 operable with tolerance.

12. The method of claim 11 wherein said step of relating said tonal signals comprises converting, first, a reference tonal signal to a signal representative of the period of the tone thereof, and subsequently, in sequential order, the command tonal signals into signals representative of the period of the tones thereof; and then calculating said intervals.

13. The method of claim 9 including the steps of: detecting when a tonal signal is repeated over a continuous time period; creating a different interval by scaling the detected repeated interval.

14. The method of claim 9 wherein said step of relating tonal signals includes the steps of: converting each tonal signal to a pulse train having a specific time period related to the tone of that tonal signal.

15. The method of claim 123 wherein said step of relating tonal signals includes the steps of: converting said reference signal to a pulse train having a specific time period related to the tone of that reference tonal signal; subsequently converting each tonal signal to a pulse train related to the tone of that tonal signal; and then calculating said interval.

16. A system for communication as hereinbefore described with reference to the accompany ig drawings.

17. A voice actuated computer control apparatus as hereinbefore described with reference to the accompanying drawings.

18. A method for human voice communication with a computer as hereinbefore described with reference to the accompanying drawings.