CN111653289A - A kind of playback voice detection method - Google Patents

Abstract

The invention discloses a playback voice detection method, which is characterized by comprising the following steps: 1) training phase: 1.1) inputting training voice samples, the training voice samples including original voice and playback voice; 1.2) extracting training voice samples Cepstral features; 1.3) Train the residual network model according to the extracted features to obtain network model parameters; 2) Test stage: 2.1) Input test voice samples; 2.2) Extract cepstrum features of test voice samples; 2.3) Utilize step 1) The trained residual network identifies and scores the features of the extracted test speech samples; 2.4) judges whether the test speech samples are playback speeches. Compared with the prior art, the advantages of the present invention are: based on the deep learning method, the cepstral feature of the speech signal is combined with the deep residual network, which effectively improves the detection performance of the system and makes the algorithm have better robustness. Awesomeness.

Description

A kind of playback voice detection method

technical field

The invention relates to voice detection technology, in particular to a playback voice detection method.

Background technique

With the continuous development of voiceprint authentication technology, the various deceptive voice attacks it is subjected to are becoming more and more severe. Among them, the playback voice attack mainly relies on the use of recording equipment to secretly record the voice of a legitimate user when entering the system and playback with a high-fidelity playback device to attack the voiceprint authentication system. The playback voice attack is easier to implement because the voice samples are derived from real voice, and the operation is simple, easy to obtain, and does not require the attacker to have relevant professional knowledge.

The early playback speech detection technologies were all based on traditional handcrafted features. However, the traditional handcrafted features could not express the high-level semantics of speech well. Therefore, more current playback speech detection technologies are based on deep learning. The network model extracts the shallow features in a deeper level to improve the performance of the playback speech detection algorithm. For example, prior art 1: "Lavrentyeva G, Novoselov S, Malykh E, et al.Audio ReplayAttack Detection with Deep Learning Frameworks[C]//Interspeech.2017:82-86.":, the author proposes a lightweight The convolutional neural network LCNN uses the maximum output method in each convolutional layer in the LCNN to form a Max-Feature-Map (MFM) layer, and eliminates the smaller output part in a competitive learning manner. Another example is prior art 2: "Jung J, Shim H, Heo H S, et al. Replay attack detection with complementary high-resolution information using end-to-end DNN for the ASVspoof 2019 Challenge [J]. arXiv preprint arXiv:1904.10134, 2019.":, the authors propose a method of dataset partitioning by dropping data points to build their models to ensure that there is no overlap between spoofing attacks in the training and testing phases, while they combine the methods of deep traditional machine learning with theirs. The dataset partitions are combined to improve the generalization ability of the model. Another example is prior art 3: "Chettri B, Stoller D, Morfi V, et al.Ensemblemodels for spoofing detection in automatic speaker verification[J].arXivpreprint arXiv:1904.04589,2019.", the author proposes a method based on multi-feature integration And the framework of multi-task learning, which contains the complementary information of multiple spectral features in the network, but since a single traditional feature is not enough to generalize the information of the speech signal in the full frequency band, it is better expressed through multi-feature integration. In addition, the authors propose a butterfly unit for multi-task learning to facilitate parameter sharing in the propagation process of binary classification tasks and other auxiliary class tasks.

Existing technology 1 has the problem of low robustness, and its detection algorithm performance is not high in the evaluation set, mainly due to the weak generalization ability of its model, when faced with unknown attacks in the evaluation set, its detection performance is significantly reduced ; Prior art 2 and 3 have the problem that detection algorithm is more complicated, in prior art 2, the author performs data set partitioning so that the playback voice attack in the test set and training set does not overlap, but its data set partitioning process is more complicated, so It takes more time; in the prior art 3, the author proposes a multi-feature and multi-task learning method, and also performs other auxiliary tasks during network training to help the recognition and judgment of the two-class classification, but the detection algorithm is relatively relatively complex.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a playback voice detection method for the shortcomings of the above-mentioned prior art, which can improve the detection performance and robustness of the playback voice algorithm.

The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a playback voice detection method, which is characterized in that it comprises the following steps:

1) Training phase:

1.1) input training voice samples, the training voice samples include original voice and playback voice;

1.2) Extract the cepstral features of the training speech samples;

1.3) Train the residual network model according to the extracted features to obtain network model parameters;

2) Test phase:

2.1) Input test voice samples;

2.2) Extract the cepstrum feature of the test speech sample;

2.3) use the residual network obtained in step 1) to identify and score the features of the extracted test speech samples;

2.4) Determine whether the test voice sample is a playback voice.

Preferably, in order to retain more detailed information on the spectrum, in step 1.2) and step 2.2), the extracted features are full-frequency cepstral coefficients.

Further, the extraction method of full-frequency cepstral coefficient feature is, 1) by the speech signal of training speech sample or test speech sample is divided into frame and windowed, then the speech signal after framed is carried out Fourier transform to obtain. Its spectral coefficient X _i (k):

Among them, i represents the ith frame after framing, k represents the frequency point in the ith frame, k=0, 1, 2,..., N-1, j represents a complex number, m represents the speech signal after framing The number of frames, N represents the number of Fourier transform points;

2) Then find the absolute value to get the corresponding amplitude spectral coefficient E _i (k):

3) Then perform logarithmic operation and DCT transformation to obtain the frequency cepstral coefficient BFCC(i) of the i-th frame:

In order to make the training models of different features cooperate with each other to obtain better fusion results, in step 1.2) and step 2.2), the Mel-frequency cepstral coefficient feature and the constant-Q cepstral coefficient feature are also extracted respectively. The full-frequency cepstral coefficient feature, the Mel-frequency cepstral coefficient feature, and the constant-Q cepstral coefficient feature obtain the corresponding three residual networks. Combined with the residual network identification results obtained by the constant Q cepstral coefficient feature, the three identification results are combined to make a comprehensive judgment.

According to an aspect of the present invention, in step 1.2) and step 2.2), the extracted features are Mel-frequency cepstral coefficients.

According to another aspect of the present invention, in step 1.2) and step 2.2), the feature of constant-Q cepstral coefficients is extracted.

Preferably, the residual network includes a two-dimensional convolutional layer, a residual block sequence, a dropout layer, a first fully connected layer, an activation function layer, a GRU layer, a second fully connected layer and a network output layer that are connected in sequence.

Preferably, the activation function layer adopts a leakage correction linear unit.

Preferably, in order to speed up the convergence speed of learning, a batch normalization processing layer is also provided between the two-dimensional convolution layer and the activation function layer.

In order to improve the detection accuracy, in step 2.4), the score output by the residual network is combined with the ASV system score to determine whether the test speech sample is the original speech or the playback speech.

Compared with the prior art, the present invention has the advantages that: the neural network can be used to extract deeper features, and the detailed information in the speech signal can be more completely represented; The residual network can effectively improve the detection performance of the system and make the algorithm more robust; the residual network can model the distortion in the time domain and frequency domain well, thereby improving the neural network classification. By extracting the features of full-frequency cepstral coefficients, no filter is needed, and more detailed information on the spectrum can be preserved.

Description of drawings

1 is a flowchart of a playback voice detection method according to an embodiment of the present invention;

Fig. 2 is the MFCC extraction flow chart of the playback speech detection method of the embodiment of the present invention;

Fig. 3 is the CQCC extraction flow chart of the playback speech detection method of the embodiment of the present invention;

Fig. 4 is the BFCC extraction flow chart of the playback speech detection method of the embodiment of the present invention;

FIG. 5 is a schematic diagram of a residual network of a playback speech detection method according to an embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", " Rear, Left, Right, Vertical, Horizontal, Top, Bottom, Inner, Outer, Clockwise, Counterclockwise, Axial, The orientations or positional relationships indicated by "radial direction", "circumferential direction", etc. are based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying the indicated devices or elements. must have a specific orientation, be constructed and operated in a specific orientation, and since the disclosed embodiments of the present invention may be arranged in different orientations, these directional terms are for illustration only and should not be regarded as limiting, such as "up", "Down" is not necessarily defined as a direction opposite or in line with the direction of gravity. Furthermore, features delimited with "first", "second" may expressly or implicitly include one or more of that feature.

Referring to Fig. 1, a playback voice detection method includes the following steps:

1) Training phase:

1.1) Input training speech samples;

1.2) Extract cepstral features;

1.3) Train the residual network model to obtain the network model parameters;

2) Test phase:

2.1) Input test voice samples;

2.2) Extract cepstral features;

2.3) Use the residual network trained in step 1) to identify and score;

2.4) Combining the ASV system score and step 2.3) to get a decision.

In the above steps, for the feature extraction method of the training speech samples and the test speech samples, the extractor may be a filter, and then the features in the speech samples may be determined according to the filter, wherein the filter may be a preconfigured filter according to actual requirements , which is used to extract features in speech samples, such as traditional Mel-frequency cepstral coefficient features and constant-Q cepstral coefficient features.

1. Features of Mel-frequency cepstral coefficients: Mel-Frequency Cepstral Coefficients (MFCC) is a speech feature parameter commonly used in the field of speaker recognition. It conforms to the auditory characteristics of the human ear. Sound waves have different hearing sensitivities. The Mel frequency cepstral coefficient is the cepstral coefficient extracted in the frequency domain of the Mel scale. The Mel scale can reflect the nonlinear characteristics of the human ear frequency. The relationship between it and the frequency is expressed as follows:

where _Fmel is the perceived frequency in Mel (Mel) and f is the actual frequency in Hertz (Hz). Converting the speech signal to the perceptual frequency domain, rather than simply expressing it in a Fourier transform, usually better simulates the processing of the human auditory process.

Referring to Fig. 2, it is the specific extraction process flow chart of MFCC, including the following steps:

1) The signal x (n) of the speech sample (training speech sample or test speech sample) is preprocessed to be x _i (m), and then the Fourier time-frequency transform (STFT) of each frame of the framed signal is performed. ) to obtain the spectral coefficient of the frame signal;

Among them, i represents the ith frame after frame division, k represents the frequency point in the ith frame, k=1,2,...,N-1;

2) Calculate the amplitude spectral coefficient of each frame of the obtained spectral coefficient:

3) The obtained energy spectral coefficients are sent to a set of mel filters, and the energy in the mel filters is calculated. The resulting mel spectral coefficients are multiplied and summed by the energy spectral coefficients and the frequency response H _m (k) in the mel filter, namely:

Among them, 0≤m≤M refers to the mth Mel filter, and there are M filters in total;

4) then carry out logarithmic operation and DCT transformation to Mel spectral coefficients to obtain its Mel frequency cepstral coefficients:

The standard MFCC only reflects the static characteristics of the parameters in the speech signal. Generally, the dynamic characteristics of the speech signal can be obtained through the differential spectral coefficients of these static characteristics. Features are combined to improve the recognition performance of the system.

2. Features of constant Q cepstral coefficients: Constant Q Cepstral Coefficients (CQCC) features are obtained by converting speech signals through CQT time-frequency transformation. What is used in the CQT transform is a set of filters whose center frequency to bandwidth ratio is constant Q. The frequency of the horizontal axis of the spectrum obtained by CQT transformation is nonlinear, because its center frequency obeys an exponential distribution, and the window length used by the filter bank changes according to the frequency. When it converts a time domain speech signal into a frequency domain speech signal, it can provide higher frequency resolution in the low frequency region and higher time resolution in the high frequency region.

Referring to Figure 3, it is a flowchart of the specific extraction process of the constant-Q cepstral coefficient, including the following steps:

1) The signal x(n) of the speech sample (training speech sample or test speech sample) is converted by CQT to the perceptual frequency domain signal X ^CQ (k), and the calculation formula of X ^CQ (k) is as follows:

Where k represents the frequency point, _k =1,2,..., _K , fs is the sampling rate of the speech sample, fk is the center frequency of the filter, which obeys the exponential distribution and is defined as follows:

B is the number of frequency points in each multiplication, and f1 is the center frequency of the lowest frequency point, which is calculated by the following formula:

The Q factor is the ratio of the center frequency f _k to the bandwidth B _k , which is a constant independent of k. Its basic definition formula is as follows:

The window function adopts the Hanning window. As the frequency resolution increases, the time resolution gradually decreases. Therefore, the window length N _k changes with k and is inversely proportional to k. The definition of the window function is as follows:

2) Find the amplitude of the frequency value X _i (k) of the k-th frequency point of the i-th frame:

3) Calculate the logarithmic spectral coefficient for the above amplitude:

Wherein T _k represents the total number of frames of speech signals on the kth frequency band, k=1, 2, . . . , K, and K is 420 in this paper.

4) Uniform resampling of the obtained logarithmic spectral coefficients, where the relationship between the new frequency representation and the original frequency representation is as follows:

5) DCT transform is performed on the resampled spectral coefficients, namely:

Among them, p=0,1,...,L-1, l represents the frequency point after resampling.

The above two are existing feature extraction, and in the present invention, the following feature extraction methods can also be used:

In addition, it is also possible to extract features of speech samples without using filters, as follows:

3. Full-frequency cepstral coefficients: Full-Band Frequency Cepstral Coefficients (BFCC), compared with some other traditional features, such as CQCC and MFCC, this feature abandons the use of filters in traditional features, namely The advantage of directly performing logarithmic operation and DCT transformation on the spectral coefficients obtained by the Fourier transform is that more detailed information on the frequency spectrum can be preserved.

Referring to Fig. 4, it is the specific extraction process flow chart of BFCC, including the following steps:

1) By performing a frame-by-frame windowing process on a piece of speech, and then performing Fourier transform on the frame-by-frame speech signal to obtain its spectral coefficient:

Among them, i represents the ith frame after framing, k represents the frequency point in the ith frame, k=0, 1, 2,..., N-1, j represents a complex number, m represents the speech signal after framing The number of frames, N represents the number of Fourier transform points, in this embodiment, N=512;

2) Then find the absolute value of it to get the corresponding amplitude spectral coefficient:

3) Then perform logarithmic operation and DCT transformation to obtain its cepstral coefficient:

In the BFCC feature, it is preferable to add the logarithmic energy coefficient and the first-order and second-order difference coefficients of the feature as the final feature vector.

In the present invention, the network model adopts a residual network, and its overall framework is shown in Figure 5. When different features are used as inputs, the overall structure of the network model remains unchanged, and only changes in the input feature dimension of the input end.

Residual network, including sequentially connected two-dimensional convolutional layer, four identical residual block sequence, Dropout layer, first fully connected layer, activation function layer, GRU layer, second fully connected layer and network output layer (classification) device), set the value of the Dropout layer function to 0.5, and the network output layer preferably adopts the Softmax layer.

The activation function layer adopts the leaky rectified linear unit (LeakyRelu), which is evolved from the Rectified linear unit (Relu) activation function. The Relu activation function only activates the neuron when the input exceeds the threshold, turning all negative values to 0 and leaving positive values unchanged. But its complement is that it is very fragile during training, especially when the input is negative, the learning speed of Relu will be very slow or even make the neuron directly useless, resulting in its weights cannot be updated, making this neuron no longer There will be activation on any data, and the gradient will always be 0. Therefore, in order to solve the complementary point of the Relu function, the present invention adopts the LeakyRelu activation function in the network. Unlike the Relu function, which sets all negative values to 0, the LeakyRelu function adds a very small non-zero slope to all negative values, which solves the problem that the neurons of the Relu function do not learn when the input is negative. Its mathematical expression is as follows:

In addition, the GRU layer can not only re-aggregate the frame-level features extracted from the upper network into a single utterance feature, but also its simple model is suitable for building deeper networks. In the residual network, its two gates can also make the entire network more efficient, save time in computation, and make the model converge significantly faster during training.

The extracted three cepstral features are respectively sent to the input of the residual network. First, a two-dimensional convolutional layer is used to extract its frame-level time-frequency domain features, and then the output of this layer is sent to four In the same sequence of residual blocks, to promote deeper training of the network, the output of the last residual block is sequentially sent to the Dropout layer, the first fully connected layer, the activation function layer and the GRU layer. After the GRU layer, its utterance-based features are mapped to the new space through the second fully connected layer, converted into new features, and sent to an output layer with only two node units, where the classification logarithm is generated, Finally, the output of the second fully connected layer is sent to the Softmax layer to logarithmically transform it into a score probability distribution.

Since the number of layers of the neural network is too deep during training, the gradient may disappear. Therefore, a batch normalization (BN) layer is added to the residual network. BN is a method of normalization, which will deviate from the normal range. The distribution is pulled into a standardized distribution range, which is located between the two-dimensional convolution layer and the activation function layer, so that the data can be distributed in the area where the activation function is more sensitive, so that the gradient becomes larger and the convergence speed of learning is accelerated. .

During the training phase, the residual network can also be optimized using the Adam algorithm. Meanwhile, with 10e ^-4 as the learning rate of the network and 32 as the batch value, the training process stops after 50 epochs. The loss function takes the binary cross-entropy function between the predicted value and the target value, and takes the node output of the last fully connected layer as the predicted score.

In step 1) training phase, use the training speech samples (including the training set of the original speech and the replayed speech) to extract the traditional features and send them to the neural network, and train the residual network models of the original speech and the replayed speech respectively; In the first stage, the features in the test voice samples are extracted and sent to the residual network model trained in the training stage. The test voice is classified and judged according to the score results of the network output layer, and the results are compared with the ASV (Automatic Speaker Recognition) system. The scores are combined as the final decision to test whether the speech sample is the result of playback speech.

Through the above training and testing process, the subsystems about the three cepstral features can be obtained respectively, and the scores of the three subsystems can be fused. The fusion method is to fuse the scores of the three subsystems. The formula is:

S=i·S _BFCC +j·S _MFCC +k·S _CQCC

Among them, i, j, and k are the weight coefficients of the scores of the three subsystems respectively, and the constraint condition is i+j+k=1, and S _BFCC , S _MFCC , and S _CQCC are the scores of the normalized subsystems, respectively. By letting the trained models of different features cooperate with each other to obtain better fusion results.

Claims (10)

1. a playback voice detection method, is characterized in that: comprise the steps: 1) Training phase: 1.1) input training voice samples, the training voice samples include original voice and playback voice; 1.2) Extract the cepstral features of the training speech samples; 1.3) Train the residual network model according to the extracted features to obtain network model parameters; 2) Test phase: 2.1) Input test voice samples; 2.2) Extract the cepstrum feature of the test speech sample; 2.3) use the residual network obtained in step 1) to identify and score the features of the extracted test speech samples; 2.4) Determine whether the test voice sample is a playback voice. 2. The playback speech detection method according to claim 1, characterized in that: in step 1.2) and step 2.2), the extracted features are full-frequency cepstral coefficients. 3. playback voice detection method according to claim 2 is characterized in that: full frequency 1) by the voice signal of training voice sample or test voice sample is carried out framing and window processing, then the voice signal after framing is carried out. Fourier transform to find its spectral coefficient X _i (k):

Among them, i represents the ith frame after framing, k represents the frequency point in the ith frame, k=0, 1, 2,..., N-1, j represents a complex number, m represents the speech signal after framing The number of frames, N represents the number of Fourier transform points; 2) Then find the absolute value to get the corresponding amplitude spectral coefficient E _i (k):

3) Then perform logarithmic operation and DCT transformation to obtain the frequency cepstral coefficient BFCC(i) of the i-th frame:

4. playback speech detection method according to claim 2 is characterized in that: in step 1.2) and step 2.2), also extract Mel frequency cepstral coefficient feature and constant Q cepstral coefficient feature respectively, in training stage respectively According to the full-frequency cepstral coefficient feature, the Mel-frequency cepstral coefficient feature and the constant-Q cepstral coefficient feature, the corresponding three residual networks are obtained. The residual network identification results obtained from the feature and the constant Q cepstral coefficient feature are used to comprehensively judge the fusion of the three identification results. 5. The playback speech detection method according to claim 1, characterized in that: in step 1.2) and step 2.2), the extracted features are Mel frequency cepstral coefficients. 6. The playback speech detection method according to claim 1, characterized in that: in step 1.2) and step 2.2), the extracted features are constant-Q cepstral coefficients. 7. The playback speech detection method according to any one of claims 1 to 6, wherein the residual network comprises a two-dimensional convolutional layer, a residual block sequence, a Dropout layer, and a first fully connected layer that are sequentially connected , activation function layer, GRU layer, second fully connected layer and network output layer. 8 . The playback speech detection method according to claim 7 , wherein the activation function layer adopts a leakage correction linear unit. 9 . 9 . The playback speech detection method according to claim 7 , wherein a batch normalization processing layer is further provided between the two-dimensional convolution layer and the activation function layer. 10 . 10. The playback voice detection method according to any one of claims 1 to 6, characterized in that: in step 2.4), the score of the residual network output is combined with the ASV system score to judge that the test voice sample is the original Voice or playback voice.

Images