EP0782127A2

EP0782127A2 - A time-varying feature space preprocessing procedure for telephone based speech recognition

Info

Publication number: EP0782127A2
Application number: EP96309114A
Authority: EP
Inventors: Alexandros Potamianos; Richard C. Rose
Original assignee: AT&T Corp
Current assignee: AT&T Corp
Priority date: 1995-12-29
Filing date: 1996-12-13
Publication date: 1997-07-02
Also published as: CA2191377A1; US5765124A; JPH09198085A

Abstract

An improved speech recognition system, in which transformation process parameters are generated in response to selected characteristics derived from speech inputs obtained from both carbon and linear microphones. The transformation process parameters are utilized in conjunction with selected digitized speech models to improve the speech recognition process based on the carbon microphone property of suppressing speech spectral energy for low energy invoiced sounds, and also for low energy regions of the spectrum between formant peaks for voices sounds.

Description

Field of the Invention

The instant invention relates generally to speech recognition and, more particularly, to method and apparatus for improving recognition based on the generation of transformation process parameters which suppress speech spectral energy for low energy unvoiced sounds, and also for low energy regions of the spectrum between formant peaks for voiced signals.

Background of the Invention

Speech recognition is a process by which one or more unknown speech utterances are identified. Speech recognition is generally performed by comparing the features of an unknown utterance, with the features of known words, phrases, lexical units and/or strings, which are often referred to as "known words." The unknown speech utterance is typically represented by one or more digital pulse code modulated ("PCM") signals. The features, or characteristics of the known words, are typically defined through a process known as training, and in conjunction with apparatus known as speech recognizers.
Speech recognizers typically extract features from the unknown utterance in order to characterize the utterance. There are many types of speech recognizers, such as, for example, conventional template-based and Hidden Markov Model ("HMM") recognizers, as well as recognizers utilizing recognition models based on neural networks.
It is, of course, understood that audio input to any speech recognition system must first pass through a transducer such as a carbon or linear (electret) microphone. Such transducers introduce distortion into the audio signal, which distortion may have an adverse (or in same cases a beneficial) effect on the recognition process. It is known, for example, that carbon transducers suppress certain speech information which heretofore has been deemed a reason to minimize use of carbon transducers in speech recognition systems. However, when utilizing speech recognition in the telephone network, the use of carbon transducers cannot be avoided, as it is estimated that fifty (50%) percent of existing telephones utilize carbon transducers.
Advantageously, the instant invention recognizes that certain characteristics inherent in carbon transducers are in fact beneficial to speech recognition, particularly in the telephone system, when properly identified and utilized. The instant invention makes use of these characteristics to improve the speech recognition process.

Summary of the Invention

In accordance with the present invention, method and apparatus is described which provides improved speech recognition by essentially suppressing information in those regions of the speech signal where the signal variability is high, or the modeling accuracy is poor.
More particularly, the invention takes advantage of the fact that one type of microphone, such as the carbon microphone, suppresses speech spectral energy for low energy unvoiced sounds, and also for low energy regions of the spectrum between formant peaks for voiced sounds. This observation is utilized in the invention described below to improve speech recognition for various types of microphones, including the carbon and linear (electret) microphone.
HMM digit models trained from carbon utterances are used by a Viterbi decoder, with the output of the Viterbi decoder utilized in a process parameter generator to generate a set of transformation process parameters. Also applied to the process parameter generator are speech utterances obtained from the outputs of both carbon and linear microphones. The output of the process parameter generator is a carbon-linear transformation process parameter, which parameter is indicative of certain significant differences in the properties of carbon and linear microphones.
The transformation process parameter is then combined with HMM digit models trained from combined linear-carbon speech via a Viterbi decoder and a speech utterance from a carbon microphone to generate a transformed speech observation vector.
This vector is in turn applied to a speech recognizer in combination with the HMM digit models trained from combined linear-carbon speech to produce a recognized word string.

Brief Description of the Drawings

In the drawings:

FIG. 1 illustrates examples of smoothed spectral envelopes taken from individual frames of speech for a single speaker utilizing a carbon and electret transducer,
FIG. 2 illustrates one portion of the inventive system that computes a carbon-linear transformation process parameter, and
FIGS. 3A and 3B illustrate the remaining portions of the inventive system, in which a transformed speech observation vector is utilized to improve the speech recognition process.

Detailed Description

It has been found that the use of a carbon microphone suppresses speech spectral energy for low energy unvoiced sounds, and also for low energy regions of the spectrum between formant peaks for voiced sounds. This is illustrated by the plots in FIG. 1. More particularly, FIG. 1 shows examples of smoothed spectral envelopes taken from individual frames of speech for a single speaker, through an electret transducer (solid line), and a carbon transducer (dotted line) simultaneously. The three plots depict filter bank envelopes for the sounds, corresponding to phonetic symbols of "/iy/," "/ah/," and "/s/" respectively, shown in FIGS. 1A, 1B and 1C. From the foregoing, it has been determined that the use of certain characteristics of a carbon transducer can be useful in speech recognition. Support for the proposition that the transformation introduced by a carbon transducer is beneficial to obtain improved speech recognition is given in Table I below. Table I

Error Rate (Per Digit)

Testing Condition

Training Condition: Carbon Electret

Carbon 1.3% 4.1%

Electret 1.6% 2.4%

Combined 1.3% 2.8%
The data shown in Table I, was obtained from an AT&T Bell Laboratories speech database, where voice samples were recorded from subjects recruited in a mall and stored in the database. The speech database contained connected digit utterances spoken over the telephone network with the speech being stored in directories per speaker labeled as either speech originating from a carbon or electret transducer. Utterances utilized to create the data in Table I consisted of 1-7 digits spoken in a continuous manner. Training data consisted of 5,368 utterances (16,321 digits) from 52 speakers. Testing data consisted of 2,239 utterances (6,793 digits) from 22 speakers. Five dialect regions were available: Long Island, Chicago, Boston, Columbus and Atlanta. The data in Table 1 utilized the Columbus dialect region.
Known hardware and software was used for the speech recognition process that generated the data in Table I, which hardware and software consisted of a front end processing system that computed cepstrum coefficients from a smooth spectral envelope. Also utilized was the Bell Laboratories Automatic Recognition System (BLASR™), which is a Hidden Markov Model (HMM) based system, and a Viterbi decoder/recognizer. Such hardware and associated software are well known and are described for example in "Fundamentals of Speech Recognition" by L.R. Rabiner and B.H. Juang, Prentice Hall, 1993.
As is shown in Table I, the error rate (per digit) recognition of connected digits over the telephone network is substantially lower for speech received through a carbon transducer, than for speech received through an electret transducer regardless of the training conditions. Thus, it is apparent from the foregoing experiment that due to the fact that carbon transducers suppress speech information where signal variability is high, speech recognition is improved. The invention advantageously utilized this type of transformation in the feature space created by a carbon transducer to generally improve speech recognition whether a carbon or electret transducer is utilized.
Referring now to FIG. 2, there is shown a system that computes a carbon transducer-linear transducer transformation process. More particularly, the system shown in FIG. 2 is designed to generate carbon-linear transformation parameters in response to speech from both carbon and linear transducers, and using the HMM Models trained from carbon utterances.
Stored at 10, are HMM models trained from carbon utterances. Such HMM models are applied to Viterbi Decoder 20, which type of Decoder is well known in this technical area and is described, in the Rabiner and Juang reference mentioned above.
Speech to be recognized is applied to carbon transducer 40, and linear transducer 50, and the respective transducer outputs are applied to the ASR front-end. Such ASR front-ends are well-known and described, for example, in "Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Sentences" by S.B. Davis and P. Mermelstein, IEEE Transactions of Acoustic Speech and Signal Processing, 1980. The outputs of the ASR front-ends are $\vec{x}$
$\binom{c}{t}$
which is the cepstrum observation vector spoken through the carbon transducer and an output of $\vec{x}$
$\binom{l}{t}$
which is the ceptrum observation vector spoken through the linear (electret) transducer.
The observation vector $\vec{x}$
$\binom{c}{t}$
, is applied to the Viterbi Decoder 20. The output of Viterbi Decoder 20, θ $\binom{c}{t}$
is applied to block 30, which estimates the parameters of the transformation process. The function performed by block 30 is to estimate α_k(y _t) ∼ N (µ_αk, σ_2αk) associated with "carbon-linear" observations decoded in state k for k=l,...K = total number of states.
The observation vectors $\vec{X}$
$\binom{c}{t}$
and $\vec{X}$
$\binom{l}{t}$
are subtracted at block 60 to form $\vec{Y}$
t which is the carbon-linear distortion process also applied to block 30.
Block 30 generates µ_{a k} and σ $\binom{2}{a_{k}}$
, which are the carbon-linear transformation process parameters with:
and where N_k is the number of vectors $\vec{Y}$
_t assigned to class K.
The problem of generating the carbon-linear transformation process parameters is treated as a signal recovery problem. The cepstrum vectors derived from the carbon transducer, $\vec{x}$
$\binom{c}{t}$
, are taken as the "desired" signal, and the cepstrum vectors derived from the electret transducer $\vec{x}$
$\binom{l}{t}$
, are taken as the "corrupted" signal. It is assumed that these vectors are realizations of random processes that are related according to $\vec{x}$
$\binom{c}{t}$
= $\vec{x}$
$\binom{l}{t}$
+ $\vec{y}$
_t, where $\vec{y}$
_t represents a simple linear filtering operation. This can be modeled as an additive bias in the mel-frequency cepstrum domain. It is also assumed that $\vec{x}$
$\binom{c}{t}$
and $\vec{y}$
_t are both represented by Gaussian densities that are tied to the states of the hidden Markov digit models. The parameters of the HMM state dependent Gaussian densities associated with $\vec{y}$
_t are obtained from the simultaneous carbon/electret recordings of the database according to the process illustrated in FIG. 2. Viterbi alignment of each training utterance spoken through a carbon transducer is performed against the known word transcription for the utterance. All frames where $\vec{x}$
$\binom{c}{t}$
are assigned to state θ_t=k are used to estimate the mean, $\vec{µ}$
_k, and variance, $\vec{σ}$
$\binom{2}{k}$
of $\vec{y}$
_t= $\vec{x}$
$\binom{c}{t}$
- $\vec{x}$
$\binom{l}{t}$
for state k, which is done in Block 30.
In the particular embodiment shown in FIG. 2, parameters were estimated from a "Stereo Carbon-Electret" database where four speakers spoke triplets of digits simultaneously into two handsets.
In FIG. 2, a transformation vector is estimated for each state of the HMM. The underlying goal was to approximate the highly non-linear characteristics of the carbon transducer using a segmental linear model. It is assumed that over a single HMM state, the transformation is a simple linear filter which can be modeled as an additive bias in the log spectral domain, or in the mel-frequency cepstrum domain.
It is important to note that the parameters of the transformation are estimated from a stereo database where speakers uttered connected digit strings simultaneously through carbon and electret telephone handsets. Hence, the parameters of the transformation were trained completely independent from the utterances that were used to test speech recognition performance. Furthermore, the speakers and the telephone handsets used for training the transformations were also separate from those used during testing.
The test utterances were transformed during recognition using the two pass procedure described in FIG. 3A (First Pass) and FIG. 3B (Second Pass). The two pass procedure is utilized for transforming the feature space prior to speech recognition. In the first pass, a state dependent transformation is applied to the input speech. Then, in the second pass, compensation and rescoring are performed on the transformed features.
In FIG. 3A, a list of N most likely string candidates (N- best list) is generated from the original test utterance. Then, a state dependent transformation is performed for each string candidate by replacing each observation $\vec{x}$
$\binom{l}{t}$
with $\vec{y}$
_t-µ_{θ t}. Finally, the best string is chosen as the one associated with the transformed utterance with the highest likelihood, as shown in FIG. 3B.
More particularly, HMM models trained from combined carbon-linear speech are stored at block 70. Similarly, the carbon-linear transformation process parameters obtained from block 30 in FIG. 2 are stored in block 80.
The HMM models from block 70 are applied to Viterbi Decoder 90, along with the input test speech observation vector $\vec{X}$
_t. The input test speech observation vector is also applied to summation block 110. The output of the Viterbi decoder Θ_t is applied to Select Transformation Vector Block 100, along with the carbon-linear transformation process parameters. Block 100 is a standard look-up table, where the input θ_t is used as a parameter to access the data stored in block 80.
The output of block 100 is also applied to summation block 110, whose output is Z_t, which is the transformed speech observation vectors, providing a plurality of recognition hypothesis.
On the second pass, the transformed speech observation vectors Z₊ are applied to Compenate/Rescore Block 130, along with the HMM Digit models (Block 120) trained from the combined carbon-linear speech.
It is to be understood that FIG. 3A produces N recognition hypotheses. However, the best scoring hypothesis may not be the best in terms of speech recognition. Accordingly, in FIG. 3B, a decision is made on the "best" (i.e., most recognizable) recognition hypothesis by rescoring the compensated utterances. For example, suppose that P ( $\vec{Z}$
$\binom{2}{t}$
/λ )> P ( $\vec{Z}$
$\binom{1}{t}$
/λ ) or the score of the compensated second candidate was higher than the first. Then, the second candidate would be chosen as additional information was used to reorder the recognition hypotheses.
The output of the speech recognizer 130 is the recognized word string. It must be emphasized that the feature vectors in the test utterances are not used to estimate any aspect of the transformation process. The parameters are obtained strictly from knowledge of the distortion process that existed prior to recognition. The result of the inventive speech recognition procedure is set forth in the digit recognizer error rate shown below in Table II. In Table II, it is to be understood that baseline conditions for Carbon were 1.3 and 2.8 for Electret, while with compensation Carbon would be at 1.0 and Electret at 1.9. Table II

Error Rate (Per Digit)

Testing Condition

Training Condition: Carbon Electret

Carbon

0. 8% 2.1%

Electret 1.7% 1.9%

Combined 1.0% 1.9%
As indicated in Table II, the approach outlined above resulted in substantial improvement in the speech recognition process, when compared to the data shown in Table I. Recognition performance improved not only for the electret, but also for the carbon utterances. Recognition performance improved across the board when Table II is compared with Table I, and is even approximately the same (1.6% vs. 1.7%) for the mismatched electret training and carbon testing case. It should be particularly noted that overall performance was significantly improved for both the matched and mismatched case, while the error rate for carbon data is still (after compensation) almost half of the error rate for the electret data (matched case).

Claims

An improved speech recognition system comprising,
means for generating transformation process parameters in response to selected characteristics derived from speech inputs obtained from a plurality of different types of microphones,

means for utilizing said transformation process parameters, in conjunction with selected digitized speech models to generate a transformed speech observation vector, said digitized speech models being generated from combined speech inputs received from said different types of microphones, and

means for applying said digitized speech models to a speech recognizer, along with said transformed speech observation vector to recognize individual words from said speech inputs.
An improved speech recognition system in accordance with Claim 1, wherein said generating means includes means for accessing and utilizing stored digitized speech models trained from speech utterances stemming from one of said different types of microphones, and accessing and utilizing combined speech utterances stemming from two or more of said different types of microphones.
An improved speech recognition system in accordance with Claim 2, wherein there is further included a first Viterbi decoder to which said digit models trained from speech utterances stemming from one of said different types of microphones is applied prior to access by said generating means.
An improved speech recognition system in accordance with Claim 3, wherein said utilizing means includes means for accessing and utilizing stored digitized speech models trained from speech utterances stemming from two or more of said different types of microphones, and for applying said digitized speech models to a second Viterbi decoder.
An improved speech recognition system in accordance with Claim 1, wherein said different types of microphones includes a carbon microphone and a linear microphone.