FR2864319A1 - Speech detection method for voice recognition system, involves validating speech detection by analyzing statistic parameter representative of part of frame in group of frames corresponding to voice frames with respect to noise frames - Google Patents

Abstract

The method involves analyzing power level of each frame of audio signal for detecting speech in an audio signal. The detection of speech is validated by analyzing a statistic parameter representative of a part of the frame in a group of predetermined frames corresponding to voice frames with respect to another group of frames corresponding to noise frames. An independent claim is also included for a device for detecting speech in an audio signal.

Description

Method and device for detecting speech in an audio signal

  The present invention relates to speech recognition systems in an audio signal, particularly in a noisy environment.

  More particularly, the invention relates to a method and system for detecting speech in an audio signal.

  The detection of voice activity is a major component of speech processing in a telecommunication system. It constitutes a preliminary or simultaneous phase to the recognition of the speech which can then be carried out on isolated words.

  In a noisy environment, the speech detection is often faulty, the speech detection errors can lead to a significant degradation of the performance of the speech recognition system.

  In order to overcome this drawback, it has been proposed to perform speech detection in a noisy environment by analyzing the energy of an audio signal frame, assuming that the distribution of energy in speech and in noise follows a Gaussian distribution. This technique is essentially based on a likelihood ratio calculation for the signal studied, in each of the noise and speech hypotheses. In this regard, reference can be made to the Solutions for Robust Speech / Non-Speech Detection in Wireless Environment, Karray L., Mokbel C. and Monné J.; Interactive Wireless Technology for Telecommunication Application, IVTTA 98 proceedings, 1998 IEEE 4h workshop. However, the hypothesis on which this technique is based, which assumes that the distribution of energy in speech and in noise follows a Gaussian distribution, is unrealistic and leads to a very weak improvement of speech detection by compared to a detection based solely on an estimate of the energy in the noise.

  According to another technique, it has been proposed to model speech no longer from an energy parameter alone, but by using a set of parameters. For example, reference may be made to US Pat. No. 5,732,392 which describes a speech detection technique by elaborating, from a signal frame, a spectral parameter vector which is used to classify the frame either in a speech category, ie in a noise category, by comparison with a quantization dictionary. This technique, which uses a modeling of the speech category using a quantization dictionary, is much richer than the previously mentioned unimodal modeling. However, it has a major disadvantage, since the noise is no longer modeled, hence the difficulty of defining a threshold on the quantization distortion for speech, this distortion can be very different, depending on whether found in noisy environment or not.

  The object of the invention is therefore to overcome the disadvantages of the state of the art and to provide a method and a speech detection system with increased performance.

  The subject of the invention is therefore a method for detecting speech in an audio signal in which the energy level of each frame of the audio signal is analyzed in order to detect the speech in the signal, and the speech detection is validated by analyzing for each frame a statistical parameter representative of the membership of the frame to a first class of predetermined frames corresponding to speech frames, with respect to a second class of frames corresponding to noise frames, so as to deduce the nature of the the frame of the audio signal.

  According to another characteristic of the invention, the Gaussian mixtures of the first and second classes of frames are extracted from a voice recognition system using a hidden Markov model (HMM).

  For example, the audio signal frame is represented by a cepstral coefficient vector.

  According to one embodiment, said parameter representative of the membership of the frame to the first frame class is developed from the likelihood of the frame in each of the classes. For example, the parameter is computed by calculating the ratio of the likelihood of the frame in the first class to the likelihood of the frame in the second class.

  According to another embodiment, the belonging of the frame to one of the first and second classes is determined from a summation of the likelihood calculated for each Gaussian of each class.

  It is also possible to determine the membership of the frame in one of the first and second classes from a method of: determination of the nearest neighbor in each of the classes.

  In one embodiment of the invention, the detection of the speech is performed by means of a finite state machine whose states comprise at least one state corresponding to a noise or a silence, a state corresponding to a presumption of speech and a state corresponding to speech, the transitions between the states being governed by the result of energy criterion evaluations of the frame and the result of the step of analyzing said parameter.

  According to the invention, there is also provided a device for detecting speech in an audio signal comprising means for analyzing the energy level of the audio signal frame for the detection of speech in the signal and calculation means for a parameter representative of the membership of each audio signal frame to a first class of predetermined frames corresponding to speech frames, compared to a second class of frames corresponding to noise frames, the calculation means being further adapted to analyze said parameter to deduce the nature of the audio signal frame and to validate the speech detection performed by the analysis means.

  It further comprises a finite state machine whose states comprise at least one state corresponding to a noise or a silence, a state corresponding to a presumption of speech and a state corresponding to speech, the transitions between the states being governed by the result of evaluations of energy criteria of the frame validated by the result of the analysis of the parameter of the frame.

  Other objects, features and advantages of the invention will become apparent on reading the following description, given solely by way of nonlimiting example, and with reference to the appended drawings, in which: FIG. 1 represents the general structure of FIG. a voice recognition system incorporating a speech detection device according to the invention; FIG. 2 represents a state machine forming part of the speech detection device according to the invention; and FIG. 3 is a curve illustrating the performance of the detection device according to the invention.

  FIG. 1 shows the general structure of a speech recognition system designated by the general numerical reference 10. The recognition system represented comprises a speech detection module 12 and a voice recognition module 14. The function of the speech detection module 12 is to detect in an incident audio signal S the speech periods in order to prevent the voice recognition module 14 from seeking to recognize speech over periods of the input signal S corresponding to phases of silence or noise phases. Its presence then makes it possible both to improve the performances and to reduce the cost of the voice recognition system.

  This determination is preceded by the analysis of the audio signal S by an analysis module 16, so as to extract coefficients relevant for the detection module 12 and for the recognition module 14.

  According to one embodiment, the extracted coefficients are cepstral coefficients, also called MFCC coefficients (ME1 Frequency Cepstrum Coefficients). For example, the analysis module 16 comprises a bank of eight filters delivering eight cepstral coefficients, which are supplemented by an energy parameter, and calculates the time derivatives of order 1 and 2 of the nine coefficients thus produced to deliver, in output of the analysis module 16, vectors of 27 coefficients.

  Upstream, the speech recognition system 10 is further provided with a detection stage 18 for dividing the incident signal S into a set of audio frames. For example, this detection module 18 comprises a Hanning window providing the cutting of the signal S in a 32 ms frame with an overlap of 16 ms. Thus, according to this example, the analysis module 16 generates, for each frame, a vector of 27 coefficients, which is used by the speech detection module 12 and by the recognition module 14 for, on the one hand, the recognition of isolated words and continuous speech and, on the other hand, provide the result. R of recognition via a decision module 20.

  The speech recognition module 14 and decision 20 of the speech recognition system consist of modules of conventional type, within the reach of a skilled person. They will not be described in detail later.

  With regard to the speech detection module 12, it essentially comprises, according to an exemplary implementation of the invention, a finite state machine. Such an automaton may, for example, be constituted by a two-state automaton, in the simplest case, when it is simply a question of detecting a vocal activity, or with three or four states, or even five states.

  The decision as to whether the frame belongs to speech is taken at each of the frames of the input signal, whose rate may be 16 ms, as indicated in the example mentioned above. In general, the use of an automaton having a large number of finite states allows a finer modeling of the decision to be taken, by taking into account the structural considerations of speech.

  FIG. 2 shows an example of a finite state machine that can be used for the realization of the speech detection module 12. Such an automaton can be produced in accordance with the teaching of the document entitled Improving the Performance of Voice Servers. of L. Mauuary, doctoral thesis, University of Rennes 1, 1994. Of course, other detection automata can also be used in the context of the present invention.

  The five states of the PLC are defined as follows: - state 1: noise or silence; - state 2: presumption of speech; - state 3: speech; state 4: unvoiced plosive or silence; and - state 5: possible recovery of speech.

  According to a characteristic of the invention, the transitions between the states are governed by the result of evaluations made on energy criteria, of duration of speech and of silence and by the result of the analysis of a parameter associated with each incident frame representative of the membership of the frame to a first class of determined frames corresponding to frames of 30 E1-speech, with respect to a second class of frames corresponding to noise frames.

  The development of such a parameter is essentially based on the calculation of statistical likelihood of the current frame in each of the classes of frames say Noise and Word. More particularly, in an exemplary implementation, the membership of the current frame to the noise class or to the speech class is determined on the basis of calculating the ratio of the calculated likelihoods, on the one hand, in the class of frames Speech, and, on the other hand, in the class of noise frames.

  It will be noted that the classes of Noise and Word frames are represented by a Gaussian mixture or mixture, that is to say a set of densities of weighted Gaussian prolbabilities each corresponding to a density of probability of belonging to the noise or noise class. Word, based on a characteristic parameter.

  Such modeling by mixing Gaussian is a widespread modeling, within the reach of a skilled person. It will not be described in detail later.

  It should be noted, however, that different techniques can be used for the creation of the mixture of Gaussians characterizing the classes Noise and Speech. These techniques are based on the use of a hidden Markov model, HMM (Hidden Markov model). This model consists of a set of states, transitions between these states, and Gaussian probability densities of emission in acoustic space related to transitions. Each probability density function has three parameters, namely a vector of means, a covariance matrix and a priori probability. Such a model is widely used in any speech recognition system and therefore will not be further detailed.

  For the development of these Gaussian reference, according to a first technique, a set of training frames are developed from sounds uttered by speakers. In order to be able to reliably learn speech gaussiennes that represent all the possible speech components independently of the speaker, a large number of example frames, typically of the order of several hundreds of thousands of frames, are used. It is the same for noise frames. The frames belonging to the Noise and Word classes are then manually labeled, and the parameters of Gaussian mixtures Noise or Word are then learned from these frames, starting from the HMM model. As is conceivable, such a technique, which requires listening to a large number of frames, is relatively tedious to implement.

  Preferably, the speech recognition system is used to automatically label the speech and Noise class frames. In this case, it is necessary to use frames whose known word sequence is known. Thus, by aligning the sequence of pronounced words with its reference in the HMM modeling, we obtain the boundaries of the phonemes and thus the classification of the 1.5 frames in class Noise or Word. The HMM model is then responsible, as before, to recover the parameters of Gaussian mixtures, noise or speech.

  Finally, according to a third example, the Gaussian mixture of gaussian models of phonemes and models of silence and noise already available in the HMM model are used for the generation of the Gaussian mixture. All gaussian models of HMM modeling phonemes and all the Gaussian models of silence and noise models of HMM modeling are used.

  From Gaussian mixtures previously elaborated or recovered, firstly, for each current frame represented by its vector of cepstral coefficients as elaborated by the analysis module 16, the likelihood lpa, oie (X) of the frame is calculated first. X in the class Word.

  This computation is carried out by means of the relation: lparoie (X) = Oi = i, .., N, ro, e7igi (X) (1) in which: Nparole is the number of Gaussian of the class Word, 7ti denotes the probability a priori associated with Gaussian speech g, parameters for which the likelihood of the frame X is calculated; and Oi = 1 Npaa, e can be considered, for example, as the operator maxi_1 Nparol or as the operator E i = 1, .. 9Nolaise With regard to the Gaussian of speech, this one is given by the relation : gl (X) = 1 exp- 1 (X -, uj) 'E-1 (X-1) (2) (27t Ej 2 1 In each case, a particular likelihood is obtained; to say: the "word (X) = mwCi = 1, ... Nparole7igi (X)

IF

speech (X) = l.

  arole (X) = Ei 7r = 1, .., NparaiPigi (X) Similarly, we calculate the lbraif likelihood (X) of the frame X in the noise class, according to the relation: 20, lbrui, (X) = 0 i =], .., Nn, a ki gi (x) (3) in which: Nbrai, is the number of Gaussians of the noise class; Sorting the a priori probability associated with the noise Gaussian gi of parameters for which the likelihood of the X-frame is calculated; ei = l Npa can be considered as the operator maxi = l, .., Npa.a, P or as the operator Ei = 1, .., Nparol As far as noise Gaussian is concerned, it is given -30 by the relation: l.5 If (x) = 1 exp-2 (X -) 'E.' (X-Pl) (4) In each case, we obtain a particular likelihood, that is, say: brmuit (X) = maxi = l, .., N ig! (X) 6 ui brui! (X) = Ei = 1,..., Na, i Rigi (X) The likelihood ratio for the current frame 10 is then calculated from the relation: 11r (X) = Iparol (X) noise (X) with the two following special cases Imax X llrux (X) _ speech max (X) noise l llr (X) = speech (X) noise (X) In the first special case of llrmax (X), this ratio corresponds to the likelihood ratio between maximum likelihood among Gaussian speakers of the speech class and maximum likelihood among Gaussians of the noise class. According to a geometric interpretation, this can be interpreted as a search method of the nearest neighbor among the representatives of the Word and search class of the nearest neighbor among the representatives of the class Noise. The criterion of belonging of the current frame X to the word class then becomes a function (for example, difference or ratio) of the distance to the nearest neighbor of each class. El

(27r)%. (5) 15.

  One can also, from Gaussian modeling, obtain a simplified modeling, of vector quantization type, which uses a quantization dictionary derived from Gaussian parameters. It is then a question of looking for the closest neighbor in each class, as being the one giving rise to a minimal quantization distortion. We can then deduce a value reflecting the belonging of the frame to the Word class relative to the noise class by taking, for example, the ratio of the minimum distortions for each of these two classes.

  As indicated above, the likelihood ratio thus developed is used to govern the transitions of the automaton, together with an analysis of the energy level of the frame.

  We will now describe, with reference to Figure 2, the operation of the automaton.

  The conditions governing the transitions between the states are, for this automaton, the following ones: Condition Cl: this first condition is based on an energetic criterion, itself based on the use of noise statistics. The classical assumption is that the logarithm of the noise energy E (n) follows a normal distribution of parameters (, E2).

  In this example, the logarithm of the noise energy E (n) of a frame n is the logarithm of the short-term noise energy, that is, the logarithm of the sum of squares of the samples. of the frame n. The statistics of the logarithm of noise energy are 25. estimated when the automaton is in a state 1 noise or silence.

  Mean and standard deviation are respectively estimated by the following equations (6) and (7): (n) = (n -1) + (1 2.) (E (n) (n -1)), ( 6) (n) = 6 (n -1) + (1) t) (IE (n) (n -1) I cs (n -1)), (7) in which n represents the index of frame, E is a forgetting factor (typically equal to 0.995), and E (n) is the logarithm of energy for the current frame.

  We consider the logarithm of the energy of each frame and we try to verify the hypothesis according to. which the automaton is in the noise or silence state, which corresponds to an absence of speech. The decision will then be taken as a function of the difference in the logarithm of the energy E (n) of the frame considered n, relative to the estimated average of the noise, that is to say according to the value of the critical ratio r (E (n)) defined as follows: r (E (n)) = E (n) (n) (8) The critical ratio is then compared with a predefined detection threshold, according to the following relation (condition C1): r (.E (n))> Threshold (9) Typically, threshold values between 1.5 and 3.5 can be used.

  This first condition, based on the use of an energy information E (n) of the input signal is called SB criterion in the following description. However, it will be appreciated that other criteria utilizing input signal energy information may also be used within the scope of the present invention.

  - Condition C2: This condition is based on the likelihood ratio calculated for the current frame.

  As previously stated, this condition is based on a likelihood ratio calculation, according to the previously mentioned relation. For example, consider the likelihood ratio llr "ax (X) and compare this ratio with a threshold, according to the following relation (10) llrmaX (X)> S euilr 13 (10) In other words, this controller essentially uses two conditions to govern the transitions between the states and detects the speech if there have been enough consecutive energy frames and if, among these frames, a sufficient number of frames have been classified as belonging to the class of Word frames using Gaussian mixtures.

  - Condition C3: This condition is based on a speech time control.

  This condition is based on the use of a DP counter incremented at each decision of the PLC tending to remain in the speech state. Condition C2 is verified when the DP counter reaches a predetermined minimum speech threshold value. In other words, according to this condition, the automaton verifies that it has remained in the speech state beyond a minimum number.

  - Condition C4: This condition is based on a classification of the frame in the Word frame class.

  To validate this condition, the automaton checks that incident consecutive frames 20 have been classified in the first class of speech frames a number of times greater than a threshold value.

  This condition uses a DPGMM counter and a GMM minimum speech threshold value.

  - Condition C 5: This condition is based on a silence time control.

  This condition is satisfied if the DS silence time represented by a counter is greater than a fine silence threshold value.

  The initial state of the automaton is state 1 noise or silence.

  The automaton remains in this state as long as no energy frame is received (condition not_C1), that is to say a frame whose energy is greater than a predetermined detection threshold. At each reception 14G d an insufficiently energetic frame, the action Al is carried out, that is to say that the counter DS is incremented.

  When receiving the first frame whose energy is greater than the detection threshold (condition C1), the automaton goes into state 2 presumption of speech. In doing so, the counter DP is set to 1 (action A2) and the counter DPGM: M is set to 0 (action A3). In state 2, the reception of a non-energy frame (condition not_C1) causes the return to state 1 noise or silence and the assignment to the value DS of the sum of counters DS and DP (action A4).

  Note that in the presumptive state of speech, the automaton calculates for each frame its likelihood ratio by gaussian mixture. The automaton remains in the presumptive speech state as long as condition C1 is fulfilled and conditions C3 and C4 are not fulfilled.

  If the conditions C3 and C4 are fulfilled, the automaton goes into the speech state, that is to say when the automaton has remained in state 2 for a predetermined minimum minimum number of frames, and that a sufficient number of frames have been placed as words using Gaussian mixtures. I1 then remains in the speech state, as long as the frames are energetic (condition C1). At each reception of a frame, the counter DP is incremented (action A5).

  On the other hand, it goes to the unvoiced plosive 4 state or silence as soon as the current frame is non-energetic (non_C1 condition). In this case, the DS counter is set to 1 (action A7). In this state 4, the reception of a number of successive non-energetic frames (condition no._C1) whose cumulative duration is greater than fine silence (condition C5) confirms a state of silence and causes the return of the automaton in the state 1 noise or silence. The DS and DP counters are then reset to O. The silence end variable is therefore used to confirm a state of silence due to the end of speech. For example, in the case of continuous speech, end silence can reach a second.

  On the other hand, if in the non-voiced plosive 4 state or silence, the current frame is energetic (condition C1), the automaton goes into the state of possible speech recovery, it remains in this state as long as the condition Cl is fulfilled and condition C3 is not fulfilled. In this state 5, the reception of a non-energy frame (condition not_Cl) makes it return to the state 1 noise or silence or in the state 4 non-voiced plosive or silence depending on whether the silence duration DS is greater (C5 ) or not (non_C5) to a predetermined number of frames (fine silence). The duration of silence represents the time spent in the non-voiced plosive state 4 or silence and in the state of possible speech recovery. Finally, if the condition Cl 15. & C3 (& denotes the logical and logical) is verified, that is to say if the automaton remained during a minimum number (minimum word) of energy frames in the state 5 ( possible recovery of speech), it then returns to state 3 (speech).

  FIG. 3 shows curves showing the variation of definitive error rates as a function of rejectable error rates, on the one hand using an algorithm based on energy criteria alone (curves Ti and T2) and, d on the other hand, using energetic criteria combined with classification using Gaussian mixture modeling (curves T '1 and T'2).

  Definitive errors are errors that systematically cause errors in the speech recognition module 14. They are essentially fragmentations, for which a word is divided into several detections, omissions, for which a word is not detected, or groupings, for which several words are grouped into a single detection.

  By rejectable errors is meant errors that may possibly be rejected by the voice recognition module 14. These are essentially inserts, for which noises are detected as being speech.

  For each of the algorithms considered, the variation of the definitive error rate is represented as a function of the rejectable error rate, on the one hand under strongly noisy conditions, ie on a lower Signal / Noise ratio. at 20 dB (T2 and T'2 curves) and, on the other hand, under low noise conditions, ie for a signal-to-noise ratio greater than 20 dB (in low noise conditions), c that is, for a Signal / Noise ratio greater than 20 dB (curves T1 and T '1). There is a marked improvement in the detection performance of

  using the classification of the frames from the calculation of the likelihood ratio of Gaussian mixtures with an energetic criterion, compared to a detection using only an energetic criterion, whether under strongly noisy or slightly noisy conditions. In particular, there is a fall in the rate of both definitive and rejectable errors.

- 17 -

Claims (12)

  A method of detecting speech in an audio signal, characterized in that the energy level of each frame of the audio signal is analyzed to detect the speech in the signal, and the speech detection is validated by analyzing for each frame a statistical parameter representative of the membership of the frame to a first class of predetermined frames corresponding to speech frames, with respect to a second class of frames corresponding to noise frames, so as to deduce the nature of the frame of the audio signal.   2. Method according to claim 1, characterized in that the frame classes are each represented by a mixture of Gaussians representing probability densities of each class.   3. Method according to one of claims 1 and 2, characterized in that the Gaussian mixtures of the first and second classes of frames are extracted from a voice recognition system (14) using a hidden Markov model (HMM).   4. Method according to any one of claims 1 to 3, characterized in that the audio signal frame is represented by a vector of cepstral coefficients.   5. Method according to any one of claims 1 to 4, characterized in that said parameter representative of the membership of the frame to the first frame class is developed from the likelihood of the frame in each class.   6. Method according to claim 5, characterized in that the parameter is constituted by the ratio of the likelihood of the frame in the first class on the likelihood of the frame in the second class.   7. Method according to any one of claims 1 to 5, characterized in that one determines the membership of the frame to one of the first and second classes from a summation of the likelihood calculated for each Gaussian of each class.   8. Method according to any one of claims 1 to 5, characterized in that the membership of the frame is determined to one of the first and second classes from a method of determining the nearest neighbor in each class.   9. Method according to any one of claims 1 to 8, characterized in that the detection of speech is performed by means of a finite state machine (12) whose states include at least 1.0 a state corresponding to a noise or a silence, a state corresponding to a presumption of speech and a state corresponding to speech, the transitions between the states being governed by the result of evaluations of energy criteria of the frame and by the result of the step of analyzing said parameter.   10. Device for detecting speech in an audio signal, characterized in that it comprises means for analyzing the energy level of the audio signal frame for the detection of speech in the signal and means (12) for calculation a parameter representative of the membership of each audio signal frame to a first class of predetermined frames corresponding to speech frames, with respect to a second class of frames corresponding to noise frames, the computing means being in further adapted to analyze said parameter to deduce the nature of the audio signal frame and to validate the speech detection performed by the analysis means.   11. Device according to claim 10, characterized in that the calculation means (12) comprise means for calculating the likelihood of the frame in each of the frame classes.   12. Device according to one. claims 10 and 11, 30. characterized in that it further comprises a finite state machine (12) whose states comprise at least one state corresponding to a noise or a silence, a state corresponding to a presumption of speech and a state corresponding to speech, the transitions between the states being governed by the result of energy criterion evaluations of the frame validated by the result of the analysis of the parameter of the frame.