WO2018166316A1 - Speaker's flu symptoms recognition method fused with multiple end-to-end neural network structures - Google Patents

Abstract

A speaker's flu symptoms recognition method fused with multiple end-to-end neural network structures consisting of four end-to-end neutral networks, the method comprising: when the input is an original speech or speech spectrum, extracting optimal features by means of a convolutional neural network, and finally performing classification by means of a long/short-term memory network or a fully connected network; and when the input is mel frequency cepstral coefficient (MFCC) or constant Q cepstral coefficient (CQCC), directly performing classification by means of the long/short-term memory network, and finally fusing these systems together. The whole process integrates feature extraction and model classification, such that the whole speaker process of recognizing flu symptoms of a speaker can be simpler and quicker.

Description

 Speaker cold symptom recognition method combining various end-to-end neural network structures

Technical field

The invention relates to the field of speech processing technology, and proposes a speaker cold symptom recognition method combining various end-to-end deep learning structures

Background technique

1. Speaker recognition, also known as voiceprint recognition, refers to the technique of automatically identifying a speaker by using pattern recognition technology by including unique speaker information in the voice. The current speaker technology achieves good performance in experimental conditions, but in practice, speech is affected by environmental noise and speaker health conditions, making the robustness of existing speaker recognition techniques less, and the cold speech recognition method. By classifying the existing speech to determine whether it is a cold speech, it is possible to improve the robustness of the speaker recognition by using the cold speech recognition method to determine in advance whether the speech is a cold speech or not, and then performing speaker recognition.

2. In the research of speech technology, researchers always hope to find the characteristics of the target type, and find out the characteristics of the normal speech from the recognition target speech. The speech feature extraction is to extract the speaker's speech features and vocal features. At present, the mainstream characteristic parameters including MFCC, LPCC, CQCC, etc. are all based on a single feature, which lacks the information to characterize the symptoms of the speaker's cold, and affects the recognition accuracy. At the same time, a large amount of knowledge of classification target speech is needed. In the speech recognition algorithm, the method based on vocal tract model and speech model knowledge is earlier, but because of the complexity of the model, it has not achieved good practical effects, and the model Matching methods such as dynamic time warping, hidden Markov model, vector quantization, etc. begin to exert good recognition results. Separating feature extraction and pattern classification is a common method for identification research, but there are problems that the features and models do not match, the training is difficult, and the features are not easy to find. The classical recognition framework has the above problems.

3. In recent years, with the development of deep learning, the recognition of images and speech based on deep neural networks has shown great energy, and a series of neural network structures have also been proposed, such as automatic coding networks, convolutional neural networks and circulating nerves. Network, etc. Many scholars have found that learning speech through neural networks can obtain better hidden features of speech. The end-to-end identification method is to deal with feature learning and feature recognition with as little prior knowledge as possible. Good recognition effect.

Summary of the invention:

According to the existing identification techniques, the features and patterns are separately classified, and there are problems such as mismatching of features and models, difficulty in training, and difficulty in finding features. The present invention proposes a method for identifying a cold symptom of a speaker that incorporates various end-to-end deep learning structures. We constructed four different end-to-end deep learning networks, and finally merged four different end-to-end neural network structures for speaker cold symptom recognition.

The four end-to-end deep learning structures are: 1. Input is voice, the network is multi-layer convolutional neural network and long-term and short-term memory network; 2. Input is voice spectrum, network is multi-layer convolutional neural network and long Short-term memory network; 3, the input is the speech spectrum, the network is a multi-layer convolutional neural network and a fully connected network; 4. The input is the Mel cepstrum coefficient and the constant Q cepstral coefficient, and the network is a long-term and short-term memory network;

The beneficial effects of the present invention are: based on the uncertainty of the traditional features, the output obtained by the neural network training can better express the characteristics of the speaker's cold symptoms, and the input is relatively simple, without excessive feature processing. . Because voice has timing information, we have better results through long- and short-term memory networks. By unifying feature learning and pattern classification, the whole speaker's cold symptom recognition process is simpler and faster, and has broad application prospects.

DRAWINGS

Figure 1 shows the flow of voice extraction of the Mel Cepstral Coefficient (MFCC).

Figure 2 shows the flow of the speech extraction constant Q cepstrum coefficient (CQCC).

Figure 3 shows the first end-to-end neural network. The input is voice and the network is CNN+LSTM.

Figure 4 shows the second end-to-end neural network. The input is the voice spectrum and the network is CNN+LSTM.

Figure 5 shows the third end-to-end neural network, the input is the voice spectrum, and the network is the CNN+ fully connected network.

Figure 6 shows the fourth end-to-end neural network. The input is the Mel cepstrum coefficient or the constant Q cepstral coefficient. The network is LSTM.

detailed description:

 In order to make the technical solutions and advantages of the present invention more clear, the technical solutions of the invention will be clearly and completely described below with reference to the accompanying drawings:

Step 1: Construct an end-to-end neural network with input as voice and network as CNN+LSTM. Specifically, the input speech is divided into segments of the same size, such as 40ms, and then averaged, and the corresponding convolutional neural network. It consists of 8 modules, each of which consists of a one-dimensional convolution layer, a ReLU active layer, and a one-dimensional maximum pooling layer. Each convolution kernel has a size of 32 and a pooled core has a size of 2. The pooling step size is 2. Then use long and short-term memory networks for classification.

Step 2: Construct the input as the speech spectrum, and the network is the end-to-end neural network of CNN+LSTM. Specifically, the input speech is divided into segments of the same size, and the fast Fourier transform is performed to obtain the spectrum of the speech segment. The convolutional neural network consists of six modules, each consisting of a two-dimensional convolutional layer, a ReLU active layer, and a two-dimensional maximum pooling layer. Among them, the first convolution layer uses 7*7 convolutional layer, the second layer uses 5*5 convolution kernel, the remaining 4 layers use 3*3 convolution kernel, and all the largest pooling layer uses 3 *3 pooled core, the pooling step size is 2. Finally, it is classified by the LSTM network.

Step 3: Construct the input as the speech spectrum, and the network is the end-to-end neural network of CNN+LSTM. Specifically, the input speech is divided into segments of the same size, and the fast Fourier transform is performed to obtain the spectrum of the speech segment. The convolutional neural network consists of six modules, each consisting of a two-dimensional convolutional layer, a ReLU active layer, and a two-dimensional maximum pooling layer. Among them, the first convolution layer uses 7*7 convolutional layer, the second layer uses 5*5 convolution kernel, the remaining 4 layers use 3*3 convolution kernel, and all the largest pooling layer uses 3 *3 pooled core, the pooling step size is 2. After a full connection layer, it is finally classified by Softmax.

Step 4: Construct an end-to-end neural network whose input is MFCC feature or CQCC feature, and the network is LSTM. The MFCC feature pre-emphasizes the speech, windowing frame, fast Fourier transform, and Meyer scale triangular filter bank. The filtering, logarithmic operation, and discrete cosine transform are finally obtained, and the CQCC feature is obtained by performing constant Q transform on the speech, obtaining the energy spectral density, taking the logarithm operation, and cosine transform. Perform long- and short-term memory networks for classification. The speech extraction MFCC or CQCC features are used as input to the neural network, and finally classified by long-term and short-term memory networks.

Step 5: Combine the above four networks to perform speaker cold speech recognition.

Claims (5)

 A method for identifying a cold symptom of a speaker that incorporates a variety of end-to-end deep learning structures, including: S1, constructing the input as voice, and the network is an end-to-end neural network of a convolutional neural network plus a long-term and short-term memory network; S2, constructing the input as a speech spectrum, and the network is an end-to-end neural network of a convolutional neural network plus a long-term and short-term memory network; S3, constructing the input as a speech spectrum, and the network is a convolutional neural network plus a fully connected end-to-end neural network; S4. The input is constructed as a voice MFCC/CQCC feature, and the network is an end-to-end neural network of a long-term and short-term memory network; S5, combining the above four end-to-end neural networks for speaker cold symptom recognition; The speaker cold symptom recognition method according to claim 1, wherein the input in S1 is voice, and the network is an end-to-end neural network of CNN+LSTM. Specifically, the input speech is divided into segments of the same size, for example, 40 ms, and then averaged, and the corresponding convolutional neural network is composed of 8 modules, each of which is composed of a one-dimensional convolution layer and a ReLU activation layer. The one-dimensional largest pooling layer, wherein each convolution kernel has a size of 32, the pooled core has a size of 2, and the pooling step has a size of 2. Then use long and short-term memory networks for classification. The speaker cold symptom recognition method according to claim 1, wherein the input in S2 is a speech spectrum, and the network is an end-to-end nerve of CNN+LSTM. The network is specifically as follows: the input speech is divided into segments of the same size, and a fast Fourier transform is performed to obtain a spectrogram of the speech segment. The convolutional neural network is composed of six modules, and each module is composed of a two-dimensional convolution layer. The ReLU active layer and the two-dimensional maximum pooling layer are composed. Among them, the first convolution layer uses 7*7 convolutional layer, the second layer uses 5*5 convolution kernel, the remaining 4 layers use 3*3 convolution kernel, and all the largest pooling layer uses 3 *3 pooled core, the pooling step size is 2. Finally, it is classified by the LSTM network. The speaker cold symptom recognition method according to claim 1, wherein the input in S3 is a speech spectrum, and the network is an end-to-end nerve of CNN+LSTM. The network is specifically as follows: the input speech is divided into segments of the same size, and a fast Fourier transform is performed to obtain a spectrogram of the speech segment. The convolutional neural network is composed of six modules, and each module is composed of a two-dimensional convolution layer. The ReLU active layer and the two-dimensional maximum pooling layer are composed. Among them, the first convolution layer uses 7*7 convolutional layer, the second layer uses 5*5 convolution kernel, the remaining 4 layers use 3*3 convolution kernel, and all the largest pooling layer uses 3 *3 pooled core, the pooling step size is 2. After a full connection layer, it is finally classified by Softmax. The speaker cold symptom recognition method according to claim 1, wherein the MFCC feature of S4 is pre-emphasized, windowed, and fast Fourier transformed by voice. The Meyer scale triangular filter bank filter, the logarithm operation, and the discrete cosine transform are finally obtained, and the CQCC feature is obtained by performing constant Q transform on the speech, finding the energy spectral density, taking the logarithm operation, and cosine transform. Perform long- and short-term memory networks for classification. The speech extraction MFCC or CQCC features are used as input to the neural network, and finally classified by long-term and short-term memory networks.