CA1301338C - Frame comparison method for word recognition in high noise environments - Google Patents

Abstract

Abstract A method and arrangement for a speech recognition system employs channel bank information to represent speech. The method considers background noise included with the speech. The method includes determining three energy levels for each channel, the first representative of background noise, the second representative of the input frame and the third representative of the word template frame. Values representing energy level differentials are assigned at each channel. If the second energy level is less than the first energy level, then a predetermined constant value is assigned at that particular channel. These values are combined to generate a distance measure depicting the similarity between the two frames.

Description

3~338 ' `~ 5 FRAME COMPARISON METHOD FOR WORD RECOGNITION
IN HIGH NOISE ENVIRONMENTS
'``' ~ Backqround of_the Invention `:;
; The present invention relates generally to the ~ practice of word recognition in a speech recognition ;~ 15 system and, more particularly, to the recognition of words in the presence of high noise.
; A long standing problem in speech reoognition systems ~ has been the difficulty in effecting acceptable ;~ performance in high noise environments. Often speech recognition systems are in an environment where background noise severely degrades the recognition proces~.
Earlier developments of recognition systems for high noise environments have typically utilized head mounted or hand-held microphones. Such systems have attempted to overcome the problem of background noise by requiring that the microphone be positioned close to the mouth.
Although this may be a somewhat effective solution for this type of system, a head mounted or hand-held microphone arrangement is not practical, nor acceptable, for many systems.
The most desirable types of recognition systems are those which operate hands-free. A hands-free system is extremely practical in cases where the operator is , '' ~, , :

.. .

.

~3G~331~3 required to manually handle controls, such as while driving. Due to high background noise in such an environment, the speech recognition system must be able to accurately distinguish words from the background noise, as well as free the operator from manual control of a microphone. A system of this kind offers substantial improvement to the operator's productivity and concentration.
There have of course been previous attempts to accurately effect a word recognition system in high noise environments. Some approaches substract an estimate of the background noise from the speech using spectral subtraction and then match the speech to word template memory. Typically, the template memory is segmented into frames of equal time intervals. Likewise, the incoming speech is split into frames before the matching process begins. Each frame from the incoming speech is then compared to frames from the template memory. A match is depicted by a sequence of frames from the incoming speech corresponding to ~rames of a template in memory.
Notwithstanding a particular word template matching technique, spectral subtraction usually requires that an estimate of the background noise be subtracted from the incoming speech before matching to the template.
Some of the more successful recognition systems actually implement the consideration of the background noise into a specific spectral matching technique.
However, these systems often re~uire a complex method of comparing frames within word templates to the input frames representing the speech. Adding such complexity has resulted in either a substantially slower recognition process or a restriction to a very specialized, high speed system architecture.

~ ~3~33~3 ~ 3 -What is needed is a simple method ~or comparing a word template frame to an input frame which compensates for the presence of background noise. Such a method should be computationally fast and should not require speaialized hardware architecture.

Obiects and SummarY of the Invention Accordingly, it is an object of the present invention to provide an improved method for comparing an input frame to a word template frame for a speech recognition sys~em in high noise which results in high recognition accuxacy and is computationally efficient.
It is an additional object of the present invention to provide such an improved method which can be used in an unlimited num~er of word template searching techniques, including truncated search techniques.
The present invention teaches an improved method of matching an input frame to a word template frame for speech recognitlon in a high noise environment. the method employs the use of speotral channels for representing both the word template frames and the input frames. Speci~ia steps of this method include determining three energy levels for each channel used.
The energy levels include a first level representative of background noise energy, a second level representative of the input frame energy and a third level representative of the word template frame energy. Values are assigned at each channel including a constant value when the second level is less than the first level at one or more channels. These values are then used to generate a distance score between the input frame and the word template frame.

, . . .

:
, ~3l~1338 _ief Description of the Drawings ~; Figure l is a block diagram of a speech recognition system illustrating an arrangement of blocks pertinent to the present invention.
Figure 2 is an illustration of a prior art state model for a word template used by a speech recognition system.
Figure 3 is a graph illustrating an example of a input frame being compared to a word template frame, ;~ 10 according to the present invention.
Figure 4 is a general flowchart, illustrating steps for generating a distance score between an input frame and word template frame, according to the present invention.
Figure 5a is an expanded flowchart showing, more specifically, the steps illustrated in Figure 4.
Figure 5b is a continuation of the flowchart of Figure 5a.

Detailed Description of the Preferred Embodiment Figure 1 is a block diagram of a speech recognition system illustrating blocks pertinent to the present invention. Initially, input speech is converted into channel bank information by the acoustic processor 2 for traini~g the system, i.e. establishing a system vocabulary. The acoustic processor converts the speech into filter bank information. One method for generating filter bank information is described by R.W. Shafer, L.R.
Rabiner and O. Herrmann in "FIR Digital Filter Banks for Speech Analysis" Bell System Technical Journal, Vol. 54 No. 3 pp. 531-544, March 1975.
The training processor 4 detects endpoints from the converted speech and generates word templates for storage in template memory 6. A variety of training processors . i 3~133~

can be used for this purpose, such as is described by L.R. Rablner and M.R. Sambur in "An Algorithm forDetermining the Endpoints ofIsolated Utterances", Bell System Technical Journal, Vol. 54 No. 2, pp 297-315, Feb.
1975.
Once training is completed, the training processor 4 is disabled and the system configures itself for speech recognition. Speech which is processed by the acoustic processor 2, is used by the distance calculator 10 and the background noise estimator 8. The background noise estimator 8 calculates an approximation of noise which is input with the speech. Such an estimator is described in by Robert J. McAulay and Marilyn L. Malpass in a paper entitled "Speech Enhancement Using a Soft-Decision Suppression Filter", IEEE Transactions on Acoustics, Speech, and Signal Pr~cessing, Vol. ASSP-28, No. 2, pp.
137-145, April 1980.
The distance calculator 10 employs the converted speech from the acoustic processor 2, the wor~ templates ~rom the template memory 6, and background noise levels from the estimator 8 to generate a measure of similarity, or a distance, between each input frame and frames from the wor~ template. The recognizer 12 provides the ; distance calculator 10 with the appropriate frames from template memory 6 for comparison. There are a number of ` well known recognition frameworks that can be used to provide this control, one of which is described by Bridle, Brown and Chamberlain in "An Algorithm for Connected Word Recognition", Proceedings of the 9182 IEEE
International Conference on Acoustics, Speech and Signal Processing, PP. 899-902. However, the distances calculated by the distance calculator 10 are critical for whichever recognizer is used by the system. Regardless of the manner in which the word templates are controlled or located, if the calculated distances are not accurate, the recognizer cannot identify an accurate match to a .-- .

.. . ~ , .

~3(~33~

word template. Subsequent discussion is directed to ~rame comparison performed within the distance calculator 10.
ImprovQments in isolated word searching techniques have recently used silence states in recognition models for matchiny an input speech utterance~ represented as a series o~ frames, to a word template in a frame-by ~rame manner. Typically, a speech recognition system will represent speech utterances using feature data such as LPC parameters, channel bank information or line spectral pairs. A vocabulary is formed by representing the spoken words as word templates. When a frame of input speech is !
compared to a word template frame, a distance score is generated. For a number of word templa~es compared to a series o~ input frames representing an input word, the template with the lowest distance score usually indicates that the input word is matched to that word template.
Silence states are employed in a typical word searchiny technique by matching input frames, representative o~ silence, to silence frames within a word model using a suitahle distance measure. ~igure 2 illustrates a word template model with an initial silence state 13 and an ending silence state 15 appended. The inner states 14 represent prestored frames o~ the actual word template. The use of silence states can greatly increase the per~ormance of a word recognition system operated in a high noise environment. In high noise it is very difficult to accurately determine the endpoints of words since the beginning and/or ending of the words may be buried in the noise. The silence states 13 and 15 can help spot the beginning and ending of the spoken word to which the model is being compared. Employing a state sequence estimation method, such as that described by G.D. Forney, Jx. in a paper entitled "The Viterbi Algorithm:, IEEE Proceedings on Acoustics, Speech_and Siqnal Processinq, Vol. 61, No. 3, March 1973, with . . .

~3~L338 . ~
silence states in a word model can facilitate the decoding process without requiring that the beginning and ending of an input word be known.
~ 5 Figure 2 illustrates one model which can be used for ; a variety of method for comparing input frames to a word template. However, despite the type of method used, using a frame-by-frame comparison technique requires that a distance score be generated for each pair of frames compared.
Notwithstanding the type of method used, the method must generate a distance score accurate enough for the word template searching technique to distinguish between many potential word templates. When significant levels of background noise exist, levels that often exceed the levels of the speech, this method must still be able to generate a distinguishable distance score.
Figure 3 is a graph illustrating an example of such a method in accordance with the present invention. In this example each frame contains channel bank information pertaining to energy levels in each of 'k' frequency channels. The~e channels are seen on the horizontal axis. On the vertical axis, depicted is a relative log magnitude scale. Plotted in the graph are three energy ¦ 25 levels 16, 18 and 20 representing an input frame 16, a word template frame 18 and an estimated noise level 20, respectively. A dotted line represents a buffer level 22. The buffer level 22 is determined from the noise `~ floor at a particular channel i plus a constant. Though this constant may vary for different applications, in the present embodiment, it is preferred to be a value ~ representing above 3dB. At each channel i, the input ^~ frame level 16 is compared to the buffer level 22. If the input frame level 16 is greater than the buffer 22, then an assignment of 1 is made for that channel as shown '~ .

, , ~3~.33~

24, i.e. Ci = 1. Ci 24 is used to determine which input frame channels, if any, are substantially above the noise level 20. "Substantially" refers to the 3dB buffer.
An energy normalization of the channels, required to change the levels to relatively similar magnitudes, is subsequently discussed. It should be noted, however, that the levels shown in Figure 3 are meant only to help illustrate the present invention. Actual levels, after energy normalization, would probably differ from those shown.
After normalization, at those channels where the input frame level 16 is above the buffer level 22, i.e.
where Ci = 1, the input frame level is subtracted from the word template frame level 18. In this example, since Ci =1 only at channels 1, 2, 3 and 9, there are only four subtractions. The absolute value representing each subtraction is accumulated for a partial distance score.
At those channels where Ci = 0, channels 4-8, a value is assiyned. This value depends on the difference ¦ between t~e word template frame level 18 and the input frame level 16. If the difference between the word template level 18 and the input ~rame level 16 is not j greater than a preselected nominal differential, then the ¦ 25 nominal differential becomes the assigned value for that particular channel. Otherwise, the assiyned value is equal to the subtracted difference. This nominal difference is a predetermined constant, used to prevent the partial distance score from accumulating nonrepresentative values. this occurs when input frame levels very close to the noise level 20 are predominantly noise rather than the actual speech itself. A total accumulation of assigned values from every channel is used to form a total distance score. This total distance score represents a measure of acoustic similarity between 3(~:~L338 ~ - 8a -: the input frame and the word template frame. A
relatively low total distance score indicates two ':

.
.

~ .

~3~L33~3 g ; similarly acoustic frames. A total distance score of zero indicatzs a perfect match.
By implementing a method in conformance with the example given, efficient computation can be sued to determine the similarity between a word template and a series of input frames representing a spoken word.
Now referring to Figure 4, shown in a general flowchart of the ~rame comparison method illustrated above. The flowchart begins by determining for which channels the input frame has energy significantly above ~; the background noise level, block 30. This is done by subtracting the buffer level from the input frame level and comparing the results to zero. If the difference is positive, then Ci = 1 and the channel is considered to be signi~ican~ly above background noise.
At block 34, a test is then performed to determine if Ci = 1 for any channels. If the test results in the affirmative, a normalization is required for both the input ~rame and the word template frame energy levels, block 32. Without normalization, if the energy of the word template is much different than the energy of the input ~rame, the distance between them would indicate very little acoustic similarity. Prior art has typically normalized by subtracting the mean log energy over all the channels in the frame from the log energy at each particular channel. However, this does not work very well when a significant amount of background noise is present. A more representative normalization can be calculated by normalizing each level based on the mean energy levels at only those channels where significant energy in the input frame exists. In the example from Figure 3, this would be where Ci = 1.
For example, an input frame energy level, X, and a word template frame energy level, Y, can be normalized over k channels as follows:

' :

, ~3~a3313 ' ~ 1 0 --Let Ci = l where significant energy exists fro the input frame at channel i. Let Ci = O where it does not. Then for each channel i, Xi is normalized by subtracting (Sumi=l to k Xi*Ci)/Sumi=l to k Ci) and similarly Yi is normalized by subtracting (Sumi=l to k Yi*ci)/sumi=l to k Ci).
Therefore, the mean energy is determined only from channels having Ci = l.
It should be noted that for most recognizer,s the determination of active channels (Ci = l) and the normalization o~ the input frame must be performed only once for each input frame, even though several word template frames are typically compared each time a single input frame is processed.
If no channel has significant input frame energy, i.e. Ci = O ~or all channels, then the value assignment at every channel i9 the same. This assignment is predetermined, representing the expected differential between the input frame level and the template frame level. Letting this expected differential equal "e", then for k channels, the total distance score can easily be calculated as 'k * e'. This is shown in block 38.
If any channels are active, an appropriate value is assigned which indicates an absolute differential between the normalized input frame level and the normaliæed word template frame level for that channel. An accumulation of these value as~ignments determines the total distance score (DIST), block 36. For channels without significant energy, additional tests are made ~or determining a proper value assignment. These additional tests and assigned values are discussed in the flowchart of Figure 5a and 5b.
Now re~erring to Figure 5a, shown in the flowchart of Figure 4 expanded. A definition of each variable used in this expanded flowchart is listed below.

: . . ~ . - ::

~.3~L33~3 X = input frame levels for each channel Y = word template frame levels for each channel N = estimated background noise levels for each channel k = total number of channels d = buffer differential over N
e = predetermined expected value for channel distance C = indicators ~or significant energy for each channel of the input frame.
i - channel number T = word template frame channel level minus input frame channel level.
DIST = total distance score Block 30 of Figure 4 is expanded into blocks 40 through 50 of Figure 5a. At each channel i, a decision is made, block 42, to determine if the energy level of the input frame X is significantly greater than the estimated background noise level, N. This is done by comparing the buffer level to the input frame level. The buffer level is defined as the noise level plus the buffer differential. If the input fame level is greater than or equal to the buffer level, it is indicated by setting Ci ~ 1, block 44. If the input frame level is less than the buffer level, it is indicated by setting Ci = O, block 46. The procedures from blocks 42 through 46 are repeated for every channel, as illustrated by blocks 40, 48 and 50.
~ Block 34 of Figure 4 requires no further expansion.
; To determine if the input frame level has significant energy at any channels, a test is made to indicate whether Ci = 1 at any channel. If Ci = 1 at least one channel, flow proceeds to block 32. If Ci = O at all channels, then the predetermined total distance score 'k * e' is assigned as the distance measure, block 38.
Continuing on to Figure 5b, block 32 of Figure 4 is expanded into blocXs 52 through 58. At every channel ~: i :

3~L33~

both the input frame level and the word template frame level are normalized to compare spectrally similar waveforms of different energy levels. blocks 52, 58 and 59 indicate the normalization at each channel. The input frame is normalized at each channel by subtracting its means level from the input frame level. As previously discussed, the mean level of the input frame is determined as 'the sum of the channel levels fsr which Ci = 1' divided by 'the number of channels for which Ci =13. This is illustrated in block 54. Block 56 illustrates the parallal normalization for the word template frame level. This special mean level ~ determination is important because in the instance where ; the input frame level exceeds the buffer level at only a few channels, then the remaining channels should not significantly weigh the resulting normalization level.
` If they did significantly weigh the resulting level, then the normalization process would be affected by the background noise rather than depending only on the speech energy.
In Figure 5h, block 36 is shown expanded into blocks 60 through 76. To assign an appropriate value at each channel, two measurements are made. The first measurement is the difference between the normalzied word template frame level and the normalized input frame level. This difference is denoted 'Tl, as seen in bloc k62. The absolute value of T becomes the value assigned at the particular channel if Ci = 1 for that channel, i.e. if the input frame level is greater than the buffer level. The Ci test is illustrated in block 64 and the absolute value of T is shown accumulated into the distance score, DIST, in block 66.
The second measurement tests if T is greater than ; zero, block 68, only for those channels where the buffer level is greater than the input frame level, i.e. where Ci = 0. By comparing T to zero in this instance, block : . ' `
,. ~

, ` ~3~33~

68, the word template frame level can be detected beneath the buffer level. This is because at this point the input frame level is known to be below the buffer level, since Ci = 0. Hence, if the word template level is below the input frame level, then it follows that the word template frame is below the buffer level. This is important because if both levels are below the buffer level, no true level comparison can be made. Therefore, the expected value 'e' is assigned. the total distance score is shown accumulating 'e' in block 70.
If the word template frame level is not detected below the buffer level (T > 0), then it is conceivable that a somewhat accurate measurement can be made between this level and the input frame level which is somewhere below the buffer level. If this measurement is not greater than a preselected nominal differential, then its differential becomes the assigned value at that channel.
Althou~h t~is value may differ depending on the application, it is preferred that this value be the value, 'e'. If the measurement is greater than the differential, then the value of "t" becomes the value assigned to the channel. This is illustrated in block 72, with the greater of either 'e' or the value of 'T' accumulated into the total distance score, DIST. This value assignment process and accumulation is done at each channel as indicated by ~locks 70, 74 and 76. These accumulated values become the total distance score, DIST, ; representing an accurate distance measure between a frame from a spoken word and a fra~e from a potentially matching word template.
It should be noted that the buffer differential, d, and the predetermined expected channel value, i, may be different for different channels. In addition, the above described technique can be modified to work with `~ ~30~3:~æ

- 13a -Euclidean or weighted Euclidean distance measures by making appropriate changes to blocks 66 and 72.
., ' '' ' ."~
.

` ~3~33~

Although this method may be applied to any speech recognition system which uses channel bank type information in representing the speech utterances, a problem arises when this method is used with truncated searching technique. A truncated searching technique, such as Beam Decoding, only extends decoding paths whose accumulated distance is within a threshold of the accumulated distance for the best curren~ path. This searching strategy reduces searching time and is well known in the art. Rsference B. Lowerre, "The Harpy Speech Recognition System'i, Ph.D. Dissertation, Computer Science Department, Carnegie-Mellon University, 1977.
The problem that arises, when using this method with a truncated search, pertains to energy levels in the input frame which are close to the background noise energy levels. This is indicated by a majority of the channels having Ci = O. When such an input frame is compared to a number o~ word template frames, each resulting total distance score will be very similar.
This does not effect a distinguishing distance measure between the frames. In this in~tance, the searching is not longer truncated since all the distance scores will be less than the threshold. Hence, no decoding path can be eliminated from exhaustive comparisons.
one solution to this problem, in accordance with the present invention, is referred to as "energy threshalding". Energy thresholding in this sense prohibits matching a high energy portion of a word template to relatively low input frame energies. If the energy of the input frame, preferably the average energy for all the channels, is only slightly greater than the energy of the background noise, then the input frame will be considered dissimilar to a word template frame if the word template frame's energy is about equal to the peak energy of the entire word template. This offers an ;

~a 3~L33~

additional similarity measure for the truncated search strategy. If the distance scores for each frame of the different word templates are not distinguishing, then energy thresholding can be used. This is because an input frame from a spoken word, with relatively low energy, cannot correspond to a matching word template ` frame, if the latter frame has relatively high energy.
The preferred energy threshold test is as follows:
If the average energy of the word template frame is within 12dB of the peak energy in the word template;
and the average energy of the input frame is less than the "valley" plus 6dB, then the word template frame ; does not correspond to the input frame.
The term "valley is sued to represent the last previously detected, minimum energy level o~ speech relative to the present frame. For further information regarding valley detectors, reference may be made to U.S.
Patent No. 4,378,603.
By using the frame comparison method described in Figures 5a and 5b, in combination with "energy thresholding", this frame comparison method can be used in an unlimited number of word template searching techniques.
What ls claimed is:
-'

Claims (6)

1. A method for word recognition of the spoken word, comparing stored speech recognition templates which are framed into time segments and channelized into at least two channels which are frequency band-limited to an input signal which has been contaminated by high levels of noise and which is framed into a time segment and channelized into at least two channels which are frequency band-limited, comprising the steps of:
determining, from the input signal, a first noise level associated with a first of the at least two channels and a second noise level associated with a second one of the at least two channels;
adding a buffering level to each of said first and second noise levels to create respectively first and second buffered noise levels;
determining, from the input signal, a first signal level associated with a first of the at least two channels and a second signal level associated with a second of the at least two channels;
normalizing the level of each said first and second signal levels to create, respectively, normalized first and second signal levels and normalizing the level of first and second channelized speech recognition template signal levels to create, respectively, first and second normalized template signal levels;
subtracting said normalized first signal level from said normalized first template signal level to determine a first difference and subtracting said normalized second signal level from said normalized second template signal level to determine a second difference; and generating a distance measure by at least:
(a) adding the absolute value of said first difference if said first signal level is greater than said first buffered noise level, adding said first difference if said first signal level is less than said first buffered noise level and said first difference is a positive value, or adding a predetermined nominal differential value if said first signal level is less than said first buffered noise level and said first difference is a negative value; and (b) adding the absolute value of said second difference if said second signal level is grater than said second buffered noise level, adding said second difference if said second signal level is less than said second buffered noise level and said second difference is a positive value, or adding a predetermined nominal differential value if said second signal level is less than said second buffered noise level and said second difference is a negative value.

2. A method in accordance with the method of claim 1 wherein said step of generating a distance measure further comprises the steps of:
assigning a first binary value to a storage associated with said first channel if said first signal level is grater than said first buffered noise level;
assigning a second binary value to said storage associated with said first channel if said first signal level is less than said first buffered noise level;
if said first channel has an associated first binary assigned storage value, summing the absolute value of said first difference into said distance measure;
if said first channel has an associated second binary assigned storage value, determining if said first difference is a positive value and adding said first difference to said distance measure; and if said first channel has an associated second binary assigned storage value, determining if said first difference is a negative value and adding a predetermined nominal differential value to said distance measure.

3. A method in accordance with the method of claim 1 wherein said step of generating a distance measure further comprises the steps of:
determining whether both said first and second signal levels exceed a predetermined threshold level; and inhibiting said generating step if both said first and second signal levels do not exceed said predetermined threshold level.

4. A word recognition detector for the spoken word, utilizing stored speech recognition templates which are framed into time segments and channelized into at least two channels which are frequency band limited to compare to an input signal which has been contaminated by high levels of noise and which is framed into a time segment and channelized into at least two channels which are frequency band-limited, the word recognition detector comprising:
means for determining, from the input signal, a first noise level associated with a first of the at least two channels and a second noise level associated with a second one of the at least two channels;
means for adding a buffering level to each of said first and second noise levels to create respectively first and second buffered noise levels;
means for determining, from the input signal, a first signal level associated with a first of the at least two channels and a second signal level associated with a second of the at least two channels;
means for normalizing the level of each said first and second signal levels to create, respectively, normalized first and second signal levels and means for normalizing the level of first and second channelized speech recognition template signal levels to create, respectively, first and second normalized template signal levels;
means for subtracting said normalized first signal level from said normalized first template signal level to determine a first difference and means for subtracting said normalized second signal level from said normalized second template signal level to determine a second difference; and means for generating a distance measure further comprising:
(a) means for adding the absolute value of said first difference if said first signal level is greater than said first buffered noise level, means for adding said first difference if said first signal level is less than said first buffered noise level and if said first difference is a positive value, and means for adding a predetermined nominal differential value if said first signal level is less than said first buffered noise level and if said first difference is a negative value;
and (b) means for adding the absolute value of said second difference if said second signal level is greater than said second buffered noise level, means for adding said second difference if said second signal level is less than said second buffered noise level and if said second difference is a positive value, and means for adding a predetermined nominal differential value if said second signal level is less than said second buffered noise level and if said second difference is a negative value.

5. A word recognition detector in accordance with claim 4 wherein said means for generating a distance measure further comprises:
means for assigning a first binary value to a storage associated with said first channel if said first signal level is greater than said first buffered noise level;
means for assigning a second binary value to said storage associated with said first channel if said first signal level is less than said first buffered noise level;
means for summing the absolute value of said first difference into said distance measure if said first channel has an associated first binary assigned storage value;
means for determining whether said first difference is a positive value and for adding said first difference to said distance measure if said first channel has an associated second binary assigned storage value; and means for determining whether said first difference is a negative value and for adding a predetermined nominal differential value to said distance measure if said first channel has an associated second binary assigned storage value.

6. A word recognition detector in accordance with claim 4 wherein said means for generating a distance measure further comprises:
means for determining whether both said first and second signal levels exceed a predetermined threshold level; and means for inhibiting said means for generating if both said first and second signal levels do not exceed said predetermined threshold level.