EP0255529A4

EP0255529A4 - Frame comparison method for word recognition in high noise environments.

Info

Publication number: EP0255529A4
Application number: EP19870900768
Authority: EP
Inventors: Ira Alan Gerson; Brett Louis Lindsley
Original assignee: Motorola Inc
Current assignee: Motorola Solutions Inc
Priority date: 1986-01-06
Filing date: 1986-12-29
Publication date: 1988-06-08
Also published as: FI873567A0; WO1987004294A1; FI873567A; EP0255529A1; CA1301338C; JPS63502304A

Abstract

Method and arrangement for a speech recognition system using channel bank information to represent speech. The method considers background noise included with the speech. The method includes determining three energy levels for each channel, the first representative of background noise energy (20), the second representative of the input frame energy (16) and the third representative of the word template frame energy (18). Values representing energy level differentials are assigned at each channel. If the second energy level is less than the first energy level, then a predetermined constant value is assigned at that particular channel. These values are combined to generate a distance measure depicting the similarity between the two frames.

Description

FRAME COMPARISON METHOD FOR WORD RECOGNITION IN HIGH NOISE ENVIRONMENTS

Background of the Invention

The present invention relates generally to the practice of word recognition in a speech recognition system and, more particularly, to the recognition of words in the presence of high noise.

A long standing problem in speech recognition systems has been the difficulty in effecting acceptable performance in high noise environments. Often speech recognition systems are in an environment where background noise severely degrades the recognition process.

Earlier developments of recognition systems for high noise environments have typically utilized head mounted or hand-held microphones. Such systems have attempted to overcome the problem of background noise by requiring that the microphone be positioned close to the mouth. Although this may be a somewhat effective solution for this type of system, a head mounted or hand-held microphone arrangement is not practical, nor acceptable, for many systems.

The most desirable types of recognition systems are those which operate hands-free. A hands-free system is extremely practical in cases where the operator is - 2 -

required to manually handle controls, such as while driving. Due to high background noise in such an environment, the speech recognition system must be able to accurately distinguish words from the background 5 noise, as well as free the operator from manual control of a microphone. A system of this kind offers substantial improvement to the operator's productivity and concentration.

There have of course been previous attempts to

ID. accurately effect a word recognition system in high noise environments. Some approaches substract an estimate of the background noise from the speech using spectral subtraction and then match the speech to word template memory. Typically, the template memory is segmented into

15 frames of equal time intervals. Likewise, the incoming speech is split into frames before the matching process begins. Each frame from the incoming speech is then compared to frames from the template memory.. A match is depicted by a sequence of frames from the incoming speech

20- corresponding to frames of a template in memory.

Notwithstanding a particular word template matching technique, spectral subtraction usually requires that an estimate of the background noise be subtracted from the incoming speech before matching to the template.

25_ Some of the more successful recognition systems actually implement the consideration of the background noise into a specific spectral matching technique. However, these systems often require a complex method of comparing frames within word templates to the input

³⁰ frames representing the speech. Adding such complexity has resulted in either a substantially slower recognition process or a restriction to a very specialized, high speed system architecture.

35 - 3 -

What is needed is a simple method for comparing a word template frame to an input frame which compensates for the presence of background noise. Such a method should be computationally fast and^" should not require specialized hardware architecture.

Objects and Summary of the Invention

Accordingly, it is an object of the present invention to provide an improved method for comparing an input frame to a word template frame for a speech recognition system in high noise which results in high recognition accuracy and is computationally efficient.

It is an additional object of the present invention to provide such an improved method which can be used in an unlimited number of word template searching techniques, including truncated search techniques.

The present invention teaches an improved method of matching an input frame to a word template frame for speech recognition in a high noise environment, the method employs the use of spectral channels for representing both the word template frames and the input frames. Specific steps of this method include determining three energy levels for each channel used. The energy levels include a first level representative of background noise energy, a second level representative of the input frame energy and a third level representative of the word template frame energy. Values are assigned at each channel including a constant value when the second level is less than the first level at one or more channels. These values are then used to generate a distance score between the input frame and the word template frame. Brief Description of the Drawings

Figure 1 is a block diagram of a speech recognition system illustrating an arrangement of blocks pertinent to the present invention.

Figure 2 is an illustration of a prior art state model for a word template used by a speech recognition system.

Figure 3 is a graph illustrating an example of a input frame being compared to a word template frame, according to the present invention.

Figure 4 is a general flowchart, illustrating steps for generating a distance score between an input frame and word template frame, according to the present invention.

Figure 5a is an expanded flowchart showing, more specifically, the steps illustrated in Figure 4.

Figure 5b is a continuation of the flowchart of Figure 5a.

Detailed Description of the Preferred Embodiment

Figure 1 is a block diagram of a speech recognition system illustrating blocks pertinent to the present invention. Initially, input speech is converted into channel bank information by the acoustic processor 2 for training the system, i.e. establishing a system vocabulary. The acoustic processor converts the speech into filter bank information. One method for generating filter bank information is described by R.W. Shafer, L.R. Rabiner and 0. Herrmann in "FIR Digital Filter Banks for Speech Analysis" Bell System Technical Journal, Vol. 54 No. 3 pp. 531-544, March 1975.

The training processor 4 detects endpoints from the converted speech and generates word templates for storage in template memory 6. A variety of training processors - 5 -

can be used for this purpose, such as is described by L.R. Rabiner and M.R. Sambur in "An Algorithm forDetermining the Endpoints ofIsolated Utterances", Bell System Technical Journal, Vol. 54 No. 2, pp 297-315, Feb. 1975.

Once training is completed, the training processor 4 is disabled and the system configures itself for speech recognition. Speech which is processed by the acoustic processor 2, is used by the distance calculator 10 and the background noise estimator 8. The background noise estimator 8 calculates an approximation of noise which is input with the speech. Such an estimator is described in by Robert J. McAulay and Marilyn L. Malpass in a paper entitled "Speech Enhancement Using a Soft-Decision Suppression Filter", IEEE Transactions on Acoustics,

Speech, and Signal Processing, Vol. ASSP-28, No. 2, pp. 137-145, April 1980.

The distance calculator 10 employs the converted speech from the acoustic processor 2, the word templates from the template memory 6, and background noise levels from the estimator 8 to generate a measure of similarity, or a distance, between each input frame and frames from the word template. The recognizer 12 provides the distance calculator 10 with the appropriate frames from template memory 6 for comparison. There are a number of well known recognition frameworks that can be used to provide this control, one of which is described by Bridle, Brown and Chamberlain in "An Algorithm for Connected Word Recognition", Proceedings of the 9182 IEEE international Conference on Acoustics, Speech and Signal Processing, PP. 899-902. However, the distances calculated by the distance calculator 10 are critical for whichever recognizer is used by the system. Regardless of the manner in which the word templates are controlled or located, if the calculated distances are not accurate, the recognizer cannot identify an accurate match to a word template. Subsequent discussion is directed to frame comparison performed within the distance calculator 10.

Improvements in isolated word searching techniques have recently used silence states in recognition models for matching an input speech utterance, represented as a series of frames, to a word template in a frame-by frame manner. Typically, a speech recognition system will represent speech utterances using feature data such as LPC parameters, channel bank information or line spectral pairs. A vocabulary is formed by representing the spoken words as word templates. When a frame of input speech is compared to a word template frame-, a distance score is generated. For a number of word templates compared to a series of input frames representing an input^'word, the template with the lowest distance score usually indicates that the input word is matched to that word template.

Silence states are employed in a typical word searching technique by matching input frames, representative of silence, to silence frames within a word model using a suitable distance measure. Figure 2 illustrates a word template model with an initial silence state 13 and an ending silence state 15 appended. The inner states 14 represent prestored frames of the actual word template. The use of silence states can greatly increase the performance of a word recognition system operated in a high noise environment. In high noise it is very difficult to accurately determine the endpoints of words since the beginning and/or ending of the words may be buried in the noise. The silence states 13 and 15 can help spot the beginning and ending of the spoken word to which the model is being compared. Employing a state sequence estimation method, such as that described by G.D. Forney, Jr. in a paper entitled "The Viterbi Algorithm: , IEEE Proceedings on Acoustics, Speech and Signal Processing, Vol. 61, No. 3, March 1973, with silence states in a word model can facilitate the decoding process without requiring that the beginning and ending of an input word be known.

Figure 2 illustrates one model which can be used for a variety of method for comparing input frames to a word template. However, despite the type of method used, using a frame-by-frame comparison technique requires that a distance score be generated for each pair of frames compared. Notwithstanding the type of method used, the method must generate a distance score accurate enough for the word template searching technique to distinguish between many potential word templates. When significant levels of background noise exist, levels that often exceed the levels of the speech, this method must still be able to generate a distinguishable distance score.

Figure 3 is a graph illustrating an example of such a method in accordance with the present invention. In this example each frame contains channel bank information pertaining to energy levels in each of 'k' frequency channels. These channels are seen on the horizontal axis. On the vertical axis, depicted is a relative log magnitude scale. Plotted in the graph are three energy levels 16, 18 and 20 representing an input frame 16, a word template frame 18 and an estimated noise floor 20, respectively. A dotted line represents a buffer level 22. The buffer level 22 is determined from the noise floor at a particular channel i plus a constant. Though this constant may vary for different applications, in the present embodiment, it is preferred to be a value representing above 3dB. At each channel i, the input frame level 16 is compared to the buffer level 22. IF the input frame level 16 is greater than the buffer 22, then an assignment of 1 is made for that channel as shown 24, i.e. Ci = 1. C-_L 24 is used to determine which input frame channels, if any, are substantially above the noise floor 20. "Substantially" refers to the 3dB buffer.

An energy normalization of the channels, required to change the levels to relatively similar magnitudes, is subsequently discussed. It should be noted, however, that the levels shown in Figure 3 are meant only to help illustrate the present invention. Actual levels, after energy normalization, would probably differ from those shown. After normalization, at those channels where the input frame level 16 is above the buffer level 22, i.e. where Cf = 1, the input frame level is subtracted from the word template frame level 18. In this example, since C = 1 only at channels 1, 2, 3 and 9, there are only four subtractions. The absolute value representing each subtraction is accumulated for a partial distance score.

At those channels where Cj_ = 0, channels 4-8, a value is assigned. This value depends on the difference between the word template frame level 18 and the input frame level 16. If the difference between the word template level 18 and the input frame level 16 is not greater than a preselected nominal differential, then the nominal differential becomes the assigned value for that particular channel. Otherwise, the assigned value is equal to the subtracted difference. This nominal difference is a predetermined constant, used to prevent the partial distance score from accumulating nonrepresentative values. This occurs when input frame levels very close to the noise floor 20 are predominantly noise rather than the actual speech itself. A total accumulation of assigned values from every channel is used to form a total distance score. This total distance score represents a measure of acoustic similarity between the input frame and the word template frame. A relatively low total distance score indicates two similarly acoustic frames. A total distance score of zero indicates a perfect match.

By implementing a method in conformance with the example given, efficient computation can be sued to determine the similarity between a word template and a series of input frames representing a spoken word. Now referring to Figure 4, shown in a general flowchart of the frame comparison method illustrated above. The flowchart begins by determining for which channels the input frame has energy significantly above the background noise level, block 30. This is done by subtracting the buffer level from the input frame level and comparing the results to zero. If the difference is positive, then Cj = 1 and the channel is considered to be significantly above background noise.

At block 34, a test is then performed to determine if Cj_ = 1 for any channels. If the test results in the affirmative, a normalization is required for both the input frame and the word template frame energy levels, block 32. Without normalization, if the energy of the word template is much different than the energy of the input frame, the distance between them would indicate very little acoustic similarity. Prior art has typically normalized by subtracting the mean log energy over all the channels in the frame from the log energy at each particular channel. However, this does not work very well when a significant amount of background noise is present. A more representative normalization can be calculated by normalizing each level based on the mean energy levels at only those channels where significant energy in the input frame exists. In the example from Figure 3, this would be where Cj_ = 1.

For example, an input frame energy level, X, and a word template frame energy level, Y, can be normalized over k channels as follows: Let Cj_ = 1 where significant energy exists fro the input frame at channel i. Let Ci =^** 0 where it does not. Then for each channel i, Xj_ is normalized by subtracting (Sumi___ to k to k 5 _Ci) and similarly Yj_ is normalized by subtracting Therefore, the mean energy is determined only from channels having Ci = 1. ^* ID It should be noted that for most recognizer,s the determination of active channels (Ci = 1) and the normalization of the input frame must be performed only once for each input frame, even though several word template frames are typically compared each time a single 15 input frame is processed.

If no channel has significant input frame energy, i.e. Ci = 0 for all channels, then the value assignment at every channel is the same. This assignment is predetermined, representing the expected differential 20- between the input frame level and the template frame level. Letting this expected differential equal "e", then for k channels, the total distance score can easily be calculated as 'k * e'. This is shown in block 38. If any channels are active, an appropriate value is 25 ^" assigned which indicates an absolute differential between the normalized input frame level and the normalized word template frame level for that channel. An accumulation of these value assignments determines the total distance score (DIST) , block 36. For channels without significant 30.^" energy, additional tests are made for determining a proper value assignment. These additional tests and assigned values are discussed in the flowchart of Figure 5a and 5b.

Now referring to Figure 5a, shown in the flowchart of 35 Figure 4 expanded. A definition of each variable used in this expanded flowchart is listed below. X = input frame levels for each channel Y = word template frame levels for each channel N = estimated background noise levels for each channel k = total number of channels d = buffer differential over N e = predetermined expected value for channel distance C = indicators for significant energy for each channel of the input frame. i = channel number

T = word template frame channel level minus input frame channel level. DIST = total distance score

Block 30 of Figure 4 is expanded into blocks 40 through 50 of Figure 5a. At each channel i, a decision is made, block 42, to determine if the energy level of the input frame X is significantly greater than the estimated background noise level, N. This is done by comparing the buffer level to the input frame level. The buffer level is defined as the noise level plus the buffer differential. If the input fame level is greater than or equal to the buffer level, it is indicated by setting Ci = 1, block 44. If the input frame level is less than the buffer level, it is indicated by setting Ci = 0, block 46. The procedures from blocks 42 through 46 are repeated for every channel, as illustrated by blocks 40, 48 and 50.

Block 34 of Figure 4 requires no further expansion. To determine if the input frame level has significant energy at any channels, a test is made to indicate whether Ci = 1 at any channel. If Ci = 1 at least one channel, flow proceeds to block 32. If Ci = 0 at all channels, then the predetermined total distance score 'k * e' is assigned as the distance measure, block 38. Continuing on to Figure 5b, block 32 of Figure 4 is expanded into blocks 52 through 58. At every channel both the input frame level and the word template frame level are normalized to compare spectrally similar waveforms of different energy levels, blocks 52, 58 and

59 indicate the normalization at each channel. The input frame is normalized at each channel by subtracting its means level from the input frame level. As previously discussed, the mean level of the input frame is determined as 'the sum of the channel levels for which Ci = 1' divided by 'the number of channels for which Ci =1'. This is illustrated in block 54. Block 56 illustrates the parallel normalization for the word template frame level. This special mean level determination is important because in the instance where the input frame level exceeds the buffer level at only a few channels, then the remaining channels should not significantly weigh the resulting normalization level. If they did significantly weigh the resulting level, then the normalization process would be affected by the background noise rather than depending only on the speech energy.

In Figure 5b, block 36 is shown expanded into blocks-

60 through 76. To assign an appropriate value at each channel, two measurements are made. The first measurement is the difference between the normalzied word template frame level and the normalized input frame level. This difference is denoted 'T', as seen in bloc k62. .The absolute value of T becomes the value assigned at the particular channel if Ci = 1 for that channel, i.e. if the input frame level is greater than the buffer level. The Ci test is illustrated in block 64 and the absolute value of T is shown accumulated into the distance score, DIST, in block 66.

The second measurement tests if T is greater than zero, block 68, only for those channels where the buffer level is greater than the input frame level, i.e. where Ci = 0. By comparing T to zero in this instance, block 68, the word template frame level can be detected beneath the buffer level. This is because at this point the input frame level is known to be below the buffer level, since Ci = 0. Hence, if the word template level is below the input frame level, then it follows that the word template frame is below the buffer level. This is important because if both levels are below the buffer level, no true level comparison can be made. Therefore, the expected value 'e' is assigned. The total distance score is shown accumulating 'e' in block 70.

If the word template frame level is not detected below the buffer level (T > 0) , then it is conceivable that a somewhat accurate measurement can be made between this level and the input frame level which is somewhere below the buffer level. If this measurement is not greater than a preselected nominal differential, then this differential becomes the assigned value at that channel. Although this value may differ depending on the application, it is preferred that this value be the value, 'e'. If the measurement is greater than the differential, then the value of "t" becomes the value assigned to the channel. This is illustrated in block 72, with the greater of either 'e' or the value of 'T^! accumulated into the total distance score, DIST. This value assignment process and accumulation is done at each channel as indicated by blocks 70, 74 and 76. These accumulated values become the total distance score, DIST, representing an accurate distance measure between a frame from a spoken word and a frame from a potentially matching word template.

It should be noted that the buffer differential, d, and the predetermined expected channel value, e, may be different for different channels. In addition, the above described technique can be modified to word with Euclidean or weighted Euclidean distance measures by making appropriate changes to blocks 66 and 72. Although this.method may be applied to any speech recognition system which uses channel bank type information in representing the speech utterances, a problem arises when this method is used with truncated 5 searching technique. A truncated searching technique, such as Beam Decoding, only extends decoding paths whose accumulated distance is within a threshold of the accumulated distance for the best current path. This searching strategy reduces searching time and is well

10. known in the art. Reference B. Lowerre, "The Harpy

Speech Recognition System", Ph.D. Dissertation, Computer Science Department, Carnegie-Mellon University, 1977.

The problem that arises, when using this method with a truncated search, pertains to energy levels in the

15 input frame which are close to the background noise energy levels. This is indicated by a majority of the channels having Ci = 0. When such an input frame is compared to a number of word template frames, each resulting total distance score will be very similar.

20 This does not effect a distinguishing distance measure between the frames. In this instance, the searching is not longer truncated since all the distance scores will be less than the threshold. Hence, no decoding path can be eliminated from exhaustive comparisons. 25 One solution to this problem, in accordance with the present invention, is referred to as "energy threshσlding". Energy thresholding in this sense prohibits matching a high energy portion of a word template to relatively low input frame energies. If the

30 energy of the input frame, preferably the average energy for all the channels, is only slightly greater than the energy of the background noise, then the input frame will be considered dissimilar to a word template frame if the word template frame's energy is about equal to the peak 35 energy of the entire word template. This offers an additional similarity measure for the truncated search strategy. If the distance scores for each frame of the different word templates are not distinguishing, then energy thresholding can be used. This is because an input frame from a spoken word, with relatively low energy, cannot correspond to a matching word template frame, if the latter frame has relatively high energy. The preferred energy threshold test is as follows: If the average energy of the word template frame is within 12dB of the peak energy in the word template; and the average energy of the input frame is less than the "valley" plus 6dB, then the word template frame does not correspond to the input frame. τh_e term "valley is sued to represent the last previously detected, minimum energy level of speech relative to the present frame. For further information regarding valley detectors, reference may be made to U.S. Patent No. 4,378,603.

By using the frame comparison method described in Figures 5a and 5b, in combination with "energy thresholding", this frame comparison method can be used in an unlimited number of word template searching techniques.

What is claimed is:

Claims

1. In a speech recognition system, wherein channel bank information is used to represent speech as frames and templates, represented by frames, are stored for retrieval during the process of speech recognition, a method thereof, comprising the steps of: determining a first energy level representative of input background noise energy for each channel; determining a second energy level representative of input frame energy for each channel; retrieving a third energy level representative of template frame energy for each channel; assigning values corresponding to the differential between said second and third energy levels at each channel, but including a predetermined constant value when said second energy level is less than said first energy level at any particular channels; and generating a distance measure between said input frame and said word template frame from said values.

:

2. The method, in accordance with claim 1, wherein said values include at least one predetermined value assigned at a channel where said second and third energy levels are below said first energy level.

3. The method, in accordance with claim 1, further including the step of normalizing said second energy level based on the mean for those channels in which said second energy level is greater than first said energy level.

4. In a speech recognition system wherein channel bank information is used to represent speech as frames and templates,- represented by frames, are stored for retrieval during a truncated template searching process 5 of speech recognition, a method thereof, comprising the steps of: determining a first energy level representative of input background noise energy; determining a second energy level representative ° of input frame energy; retrieving a third energy level representative of template frame energy; comparing said second energy level to said first energy level when said third energy level is less than a 5 predetermined constant to indicate a similarity between said input frame and said word template frame for the truncated search process.

0

5

0

5 5. The method, in accordance with claim 4, wherein said second energy level is representative of the average channel values of the input frame.

6. In a speech recognition system, wherein channel bank information is used to represent speech as frames and templates, represented by frames, are stored for retrieval during the process of speech recognition, an arrangement therein, comprising: means for determining a first energy level representative of input background noise energy for each channel; means for determining a second energy level representative of input frame energy for each channel; means for retrieving a third energy level representative of template frame energy for each channel; means for assigning values corresponding to the differential between said second and third energy levels at each channel, but including a predetermined constant value when said second energy level is less than said first energy level at any particular channels; and means for generating a distance measure between said input frame and said word template frame from said values.

7. The arrangement, in accordance with claim 6, wherein said values include at least one predetermined value assigned at a channel where said second and third energy levels are below said first energy level.

8. The arrangement, in accordance with claim 6, further including means for normalizing said second energy level based on the mean for those channels in which said second energy level is greater than first said energy level.

9. In a speech recognition system, wherein channel bank information is used to represent speech as frames and templates, represented by frames, are stored for retrieval during^'a truncated template searching process of speech recognition, an arrangement therein, comprising: means for determining a first energy level representative of input background noise energy; means for determining a second energy level representative of input frame energy; means for retrieving a third energy level representative of template frame energy; means for comparing said second energy level to said first energy level when said third energy level is less than a predetermined constant to indicate a similarity between said input frame and said word template frame for the truncated search process.

10. The arrangement, in accordance with claim 9, wherein said second energy level is representative of the average channel values of the input frame.