EP1406244B1 - Voice activity detection based on unsupervised trained clustering - Google Patents

Description

A Voice Activity Detector (VAD) is a device that allows to distinguish between speech including background noise ("speech") and background noise alone ("non-speech"). The input of a VAD can be, for example, a voice signal of a communication terminal recorded by a microphone. As the user speaks, the signal is composed of his voice and the background noise (for example, street noise). In contrast, during the pauses between speaking, the signal alone consists of the background noise. The output of a Voice Activity Detector now adds information to the input signal, whether it contains speech or not.

The applications of a VAD are many. Thus, a data reduction VAD can be used to store or transmit only the speech signal. In speech recognition, a VAD allows faster and better recognition because the recognition can focus on the pure speech passages of the audio signal.

VADs are either set based on heuristics or trained during a training phase. The input signal used in each case is the appropriately pre-processed audio signal. In a property extraction, depending on the number of properties used, different-sized property vectors are obtained.

The simplest, but still widely used, heuristic is to judge a signal against a specific, fixed energy threshold. If the signal energy exceeds the threshold, then "language" is assumed, otherwise "non-language".

Another example is the determination of the zero crossing rate of the autocorrelation function of the speech signal and a corresponding threshold for discriminating whether a speech signal is present or not.

In addition, there are more complex methods to make the desired distinction based on a variety of properties using a more or less large number of thresholds.

For example, VADs that are trained during a training phase include statistical VADs or neural networks. These are trained with data that is known when speech and when a noise occurs. So these are data that are previously labeled by hand, for example. Examples of methods with which it can be decided in this way whether a voice signal is present or not, for example, in Stadermann J .: "Speech / Pause Detection in Automatic Speech Recognition", University of Duisburg, Diploma Thesis, 1999, pages 28-36 , stated.

Other VADs, in particular for wireless communication, are described in El-Maleh, K. and Kabal, P .: "Comparison of voice activity detection algorithms for wireless personal communication systems", Proc. IEEE Canadian Conference on Electrical and Computer Engineering, St. John's, Newfoundland, May 1997, pages 470-473.

The document according to Venkatesha Prasad et al. "Comparison of Voice Activity Detection Algorithms for VoIP", Proc. 7 ^th, ISCC, July 2002, pp. 530-535, discloses a device for detecting whether or not there is a voice signal. For this purpose, a speech signal is divided into one of the three classes "speech", "noise" or "low-energy speech phonemes". Subsequently, depending on the class into which the speech signal has been divided, it is decided whether the signal is a speech signal or not. The disadvantage here is that the Division of the speech signal is bound to the three fixed predetermined classes and thus the differentiated characteristics of speech signals is not taken into account. This results in a greater inaccuracy in the decision as to whether a speech signal is present or not.

Based on this, the object of the invention is to make possible a more precise distinction between speech and non-speech. It should also be placed on an automatic adaptability to different noise situations, speakers or languages value.

This object is achieved by the inventions specified in the independent claims. Advantageous embodiments emerge from the subclaims.

The invention is based on the idea that a VAD can, in principle, be regarded as a classifier with N = 2 classes (speech / non-speech). However, it has been found that a much better classification can be made if a signal is not immediately assigned to the speech or non-speech class, but if the signal is first in a class of a plurality of more than is divided into three classes. In this way, the numerous different properties of speech and noise can be better taken into account.

According to these numerous different characteristics, the plurality is preferably greater than or equal to 10, in particular greater than or equal to 64. Depending on the class into which the signal is divided, it is then decided whether the signal is a speech signal or not.

While it is known in further speech processing, speech signals that are recognized as such, ie after the Voice Activity Detection, to divide into more than two classes, as a special feature of the invention, two or more classes may be provided to be decided that the signal is not speech when it is divided into it.

For this purpose, the classes may be clustered in clusters so that similar classes are grouped adjacent or in groups. For this purpose, the classes are automatically formed in a self-organizing cluster process to be trained in a training phase, in particular by means of test signals.

In this case, a neural network is preferably used, in particular a Kohonen network with the network architecture of a self-organizing card.

This trained and structured network is then preferably also used immediately in the detection phase, in which it is decided whether a signal is a voice signal or not.

Particularly advantageously, the device described can be used in the biometric speech recognition during enrollment to capture the voice of the enrollenden person as a reference and no more or less large parts of the background noise. Otherwise, a person who has a similar noise environment during verification may be authenticated by the system.

A method for detecting whether a speech signal is present or not can be constructed analogously to the described device. This also applies to its preferred embodiments.

A program product for a data processing system, which contains code sections with which one of the described methods can be executed on the data processing system, can be implemented by suitable implementation of the method in a programming language and translation in code executable by the data processing system. The code sections are stored for this purpose. In this case, the program is understood as a tradable product under a program product. It can be in any form, such as on paper, a computer-readable medium or distributed over a network.

Figur 1

die Trainingsphase einer Vorrichtung mit Mitteln zur Detektion, ob ein Sprachsignal vorliegt oder nicht;

Figur 2

die Assoziationsphase der Vorrichtung nach Figur 1;

Figur 3

ein Beispiel für eine Detektion, ob ein Sprachsignal vorliegt oder nicht.

Further essential advantages and features of the invention will become apparent from the description of an embodiment with reference to FIGS. Showing:

FIG. 1

the training phase of a device with means for detecting whether a speech signal is present or not;

FIG. 2

the association phase of the device of Figure 1;

FIG. 3

an example of detection of whether a voice signal is present or not.

The prior art known VADs have the problem that properties extracted from the signal are divided into only two classes, although their characteristics differ widely within one and the same class. For example, in a speech signal, characteristics that represent unvoiced sounds tend to be very different from those that reflect voiced sounds. Nevertheless, both are assigned to the same class ("language").

It is therefore proposed to use a two-phase learning method to discriminate whether or not there is a speech signal.

In the first phase of the process, a self-organizing, self-organizing cluster process with N> 2 classes is used. N is given arbitrarily, but meaningfully. For training, therefore, only extracted from an audio signal property vectors are used, without at the same time a class affiliation is specified. More generally, therefore, there are a larger number m of classes of classifier representing "language" and a larger number n of classes representing "non-language" (m + n = N> 2). Thus, for example, it becomes possible to assign voiced and unvoiced sounds to different classes.

This first phase will be illustrated with reference to FIG. There one recognizes an audio database 1 with audio signals. These are fed to preprocessing 2. This preprocessing is preferably the same as that used for later speech recognition. This can save a second preprocessing.

The preprocessing 2 extracts from the audio signals of the audio database 1 property vectors 3, in which properties of the audio signals are specified. These property vectors 3 are supplied to the input neurons of a neural network 4.

The neural network 4 is a Kohonen network with the network architecture of a self-organizing map (SOM: Self-Organizing Map). It has the property that there is a local neighborhood relationship between the individual neurons, so that the reference vectors representing the individual classes are spatially ordered after completion of the training.

The neural network is trained on the basis of a database which, for example, has speech and noise in equal frequency.

The training of such a network represents a self-organizing cluster process with unsupervised learning.

The result of the classifier training is a class representation 5.

After successful classifier training, in a second phase, the association phase, the assignment of each individual class of the classifier 4 in the form of the neural network to one of the two classes speech or non-speech takes place. This is now the classifier 4 itself in Classification mode operated, that is, it outputs to each property vector 3, the associated class 6. This is shown in FIG. On the other hand, the association unit 7 is operated in the training mode, that is, it learns to associate each of the classifier classes with "speech" or "non-speech" based on the labeled audio signals 8. It is ascertained to which classes in each case how many test signals have been assigned which are "language" or the "non-language". Depending on this result, each class in an association step is declared as a language or non-language class. The result is the class assignment 9 of the VAD.

After the association step, the results obtained are further enhanced by using a mean value filter to eliminate individual outliers.

In FIG. 3, the amplitude A of the German word "twenty" (20) is plotted over time t. For this signal, the result of the detection is shown below the graph, whether a voice signal is present or not.

The actual classification is indicated in the first line labeled "Real". Where "noise" stands for "non-language" and "speech" for "language".

The second line ("Label") indicates the classification made by a conventional VAD for labeling.

In the third line, labeled "N-VAD", the detection is finally indicated, as achieved by the device according to the invention and the method according to the invention with a predetermined class number N = 625. As you can see, this detection and classification is much more in line with reality than with the traditional VAD. This makes especially noticeable by the fact that even pauses between individual syllables are detected as "non-speech".

The invention provides the following advantages in particular:

Dissimilar property vectors are no longer forced into the same class, but are assigned to a class on the basis of a similarity criterion alone. This increases the accuracy of the classification.

Inaccuracies in the labeling of the audio signals do not affect the actual training process, as there is unmonitored learning. Thus, typical short speech pauses between individual syllables are not detected when labeling, but assigned to the class "speech", although in this pause the background noise predominates. In the proposed method based on unsupervised learning, this short pause is assigned to its corresponding property vectors.

The method is independent of the language and / or content of the spoken text.

Overall, the accuracy of the VAD is improved, which translates into better results in applications based on it.

According to the increased accuracy, the invention can also be used preferably in the context of enrollment in a biometric speech recognition to recognize the word boundaries, after previous methods based on the signal energy again and again lead to errors and thus to a security risk in the biometric authentication.

Claims (9)

1. Device for detecting whether a speech signal is present or not, with

   - Means for classifying a signal into one of more than two classes,

   - Means for deciding whether the signal is a speech signal or not, depending on the class to which the signal is classified,

   characterized in that, that the following means are provided:

   - Means for extracting feature vectors from the speech signal,

   - Means for classifying in a learning process the extracted feature vectors with the aid of a self-organizing cluster method into one of more than two automatically formed classes.

   - Means in an association process, for classifying the classes from the learning process as "speech" or "non-speech" respectively.
2. Device in accordance with claim 1,
   characterized in that
   the number of the more than two classes is equal to or greater than 10, especially equal to or greater than 64.
3. Device in accordance with claim 1,
   characterized in that
   the automatically formed classes are classes which are formed by a neuronal network.
4. Device in accordance with one of the previous claims,
   characterized in that
   the device features a neuronal network for classification of the signal into one of more than two classes.
5. Device in accordance with one of the claims 3 or 4,
   characterized in that
   the neuronal network is a Kohonen network.
6. Device in accordance with one of the previous claims,
   characterized in that
   the device is a mobile terminal, especially a mobile telephone.
7. Biometric method, in which a device in accordance with one of the claims 1 to 6 is used.
8. Method for detecting whether a speech signal is present or not, in which

   - a signal is classified into one of more than two classes which are clustered into self-organizing clusters,

   - depending on the class into which the signal is classified, a decision is made as to whether the signal is a speech signal or not,

   characterized in that,

   - feature vectors are extracted from the speech signal,

   - in a learning process the extracted feature vectors are classified with the aid of a self-organizing cluster process into one of more than two automatically formed classes,

   - in an association process the classes from the learning process are classified as "speech" or "non-speech" respectively.
9. Program product for data processing system which contains code sections, with which all steps of a method in accordance with one of the claims 7 to 8 are executed, if the program product is running on a data processing system.

Images