TWI749709B - A method of speaker identification - Google Patents

Abstract

The present invention includes three-stage identification. The first stage of identification is to detect whether a test sentence related to this text is a spoofing attack for recording playback. The second stage of identification is the detection of a synthetic speech spoofing attack on an unrelated test sentence in the text. The third stage of identification uses the one-speaker identification system to determine the registered speaker of the irrelevant test sentence in this article. If it is not a registered speaker, it is judged as an imposter. The first two stages use different features and have their own binary classifiers. The third stage uses a complex classifier, through Ensemble Learning and Unanimity Rule, with conditional retry. Mechanism to determine whether the irrelevant test sentence in this article is the target speaker or the imposter; based on this, the target speaker’s blocking rate can be effectively reduced without losing the impostor’s blocking rate.

Description

A method of speaker identification

The present invention relates to speech technology, especially a method of speaker identification.

The number of voice application products is increasing. Therefore, for voice application products, speaker identification technology is very important. It needs to be able to authenticate the identity of the speaker and prevent impersonation, so as to meet the requirements of anti-theft and confidentiality.

In particular, with the maturity of technologies such as Speech Synthesis and Voice Conversion, voice attacks have also become an important issue to be resolved in speaker identification technology. In speaker identification, test sentences that meet the registered speakers are called Targets, those that do not meet the registered speakers are called Imposters, and those who use artificial voices to attack (trying to pass authentication) are called Targets. Spoofing Attacks include recording playback (Replay), speech synthesis (Speech Synthesis), or voice conversion (Voice Conversion). Recording playback refers to the playback of pre-recorded sentences of the registered speaker, while speech synthesis and speech conversion refers to fake speech generated by artificial technology.

The speaker identification technology must be able to correctly identify the target speaker and prevent deception attacks and the intrusion of imposters. In order to judge the quality of speech recognition technology, the LA (Logical Access) corpus provided by ASVspoof-2019 can be used as the training/testing corpus. ASVspoof-2019 is developed by the University of Edinburgh in the United Kingdom and the National Institute of Information and Automation in France. , Japan’s NEC and other organizations initiated the automatic speaker identification deception and countermeasures competition. The competition is divided into two major challenges: LA and PA. There are 107 authentic speakers in the LA corpus, including 46 males and 61 females. There is no obvious channel or background noise in the speech.

In practical applications, first of all, it is unlikely to find a feature extraction technology that can simultaneously distinguish between "recording playback", "synthetic speech" and "real human speech". In addition, in the language In the application of user identification, any non-registered speakers are imposters, so the number of imposters is far greater than the number of target speakers, and malicious imposters will even try to break into the system multiple times. Combining these two situations, selecting appropriate feature engineering, and having the ability to accept entry of all target speakers and block any intrusion by imposters are the keys to the successful realization of a voice recognition technology.

Known speaker recognition technology, such as the paper "Speaker Verication using I-vector Features" published by PhD Thesis. Or, for example, the paper "Front-End Factor Analysis for Speaker Verification" published by Najim Dehak et al. in IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL.19, NO.4, MAY 2011. Or, for example, the paper "Joint Factor Analysis versus Eigenchannels in Speaker Recognition" published by Patrick Kenny et al. in IEEE Transaction on Audio Speech and Language Processing, 15(4): 1435-1447. Neither can effectively solve the problem of spoofing attacks and pretenders trying to break.

In addition, the conventional method used to detect recording playback attacks is the development of key acoustic features, but there is currently no sufficiently robust feature that can completely block recording playback. The conventional feature of detecting synthesized speech is extracted as a constant Q cepstral coefficient (CQCC), but this feature still cannot provide complete information to develop the ability to fully recognize the synthesized sentence. For distinguishing target speakers and impersonators, the conventional feature extraction technology is i-vector.

The spoofing attacks against recording playback and artificially synthesized sentences are both the detection issues of the conventional binary classification. Regarding the identification technology of the accustomed target speaker and the impostor, most of them adopt the threshold value decision method, which uses such as deep neural network (DNN), linear support vector machine (Linear-SVM), kernel support vector machine ( Classifiers such as Kernel-SVM and Random Forest. The classifier generates a score for the test sentence. When the score is higher than the threshold, it is judged as passed (the test sentence is the target language person), and if the score is lower than the threshold, it is judged as not passing the authentication (the test sentence is a pretender). This threshold decision-making method is used to "increase the blocking rate of imposters" and "reduce the target". There is a tradeoff between "slogan blocking rate", that is, no threshold can satisfy both of these effects at the same time.

Therefore, Xizhi will use the Score Normalization method to separate the score distribution between the target speaker and the impostor, hoping to find an appropriate threshold more easily. However, in practical applications, the regularization method still cannot improve the acceptance rate of the target speaker and the rejection rate of the imposter at the same time. This is because the score distribution of the target speaker and the impostor will partially overlap, which makes the scores regular The method of transformation cannot separate the two fractional distributions.

The main purpose of the present invention is to disclose a speaker identification method, which can detect deception attacks of recording playback and speech synthesis, and at the same time, for speaker identification, it can effectively reduce the target without losing the blocking rate of imposters. The speaker's blocking rate.

To achieve the above objective, the present invention provides a speaker identification method to identify a text-dependent test sentence and a text-independent test sentence spoken by a user. Related test sentences in this article are specified sentences, and irrelevant test sentences in this article are sentences that are spoken at will.

The present invention sequentially performs the first, second, and third stages of identification to determine whether the relevant test sentence of this text is a spoofing attack of recording playback, whether the unrelated test sentence of this text is a spoofing attack of synthesized speech, and is judged by the one-speaker identification system The irrelevant test sentences in this article are target speakers or pretenders, and the steps are as follows.

In the first stage of identification, when judging whether the relevant test sentence of this text is a spoofing attack of recording playback, the present invention uses the characteristics of Mel-frequency Spectral Coefficients (MFSC) to construct a speech recognition model related to this text. The recognition model can choose to use Speech Recognition or Template Matching to form a binary classifier. The binary classifier can be used to determine whether the relevant test sentence in this article is recorded. Deception attacks on audio playback. If the test sentence related to this article is not a spoofing attack for recording playback, pass the first stage identification and enter the second stage identification, otherwise the test sentence related to this article is rejected.

The second stage of identification is to determine whether the irrelevant test sentence in this text is a spoofing attack of synthesized speech. For this reason, the present invention first constructs a dimensional reduced hybrid feature for the irrelevant test sentence in this text. The mixed feature is derived from the irrelevant test sentence in this text. The constant Q cepstral coefficient (CQCC) and the spectrogram (Spectrogram) construct a mixed feature vector. First, let the irrelevant test sentence of this text go through step P1: pre-processing and framing, and then proceed to step P2- separately. 1: Extract CQCC features and step P2-2: Short-time Fourier Transform (STFT), where step P2-1 is to obtain constant Q cepstral coefficients, and step P2-2 is to obtain spectrograms, and finally step: P3: composite mixing The feature is to generate the mixed feature vector, and pass the mixed feature vector to another binary classifier to determine whether it is a synthesized speech. If it is judged that it is not a synthetic speech, it will pass the second stage of recognition. The irrelevant test sentence in this article will be regarded as a real person on the scene, and then enter the third stage of recognition, otherwise the irrelevant test sentence of this article will be rejected.

The third stage of identification is speaker identification, which is used to determine whether the unrelated test sentence of this text is a target language person or a pretender. The third stage of identification is performed by a text-independent speaker identification system with plural registered speakers and plural classifiers. Step a~Step c.

First, proceed to step a: extract the i-vector feature of the irrelevant test sentence in this article, and reduce its dimensionality, and based on the feature vector after the i-vector dimensionality reduction, let the complex classifier each determine whether the irrelevant test sentence in this article is Any one of the plural registered speakers is a pretender, and each produces a judgment result.

Then proceed to step b: make a decision based on the result of the plural judgment. When the result of the plural judgment is all judged to be the same registered speaker, judge the irrelevant test sentence in this text as the target speaker, and end the speaker identification of the third stage of identification ; When the plural judgment result is judged to be the same registration If the number of speakers is less than half, the irrelevant test sentence in this article is judged to be an impostor, and the speaker identification of the third stage of identification is ended. If it is not the two types of situations mentioned above, but when the plural judgment result judges that the number of persons with the same registered language is not less than half, then proceed to step c and give a maximum number of retries with a positive integer number.

In step c, when the number of retries by the user does not exceed the upper limit, the user is asked to re-speak another unrelated test sentence, and then step a is repeated, but when the number of retries by the user exceeds the upper limit At that time, the speaker identification in the third stage of identification is ended, and the irrelevant test sentence in this article is judged to be an imposter.

From the above description, we can see that regarding the deception attack of recording playback, we use the characteristics of Mel Spectral Coefficient (MFSC) to construct the speech recognition model related to this article, which can be combined with speech recognition or model comparison to form the binary classifier. The binary classifier is used to determine whether the relevant test sentence of this article is a spoofing attack for recording playback. For spoofing attacks on synthetic speech, the method of the present invention constructs the dimensionality-reduced hybrid feature to use the other binary classifier to determine whether the irrelevant test sentence is a synthetic speech. Regarding the identification of the target speaker or the impostor, a plurality of classifiers are used to judge the same irrelevant test sentence of the text. Only when the plural judgment results are all reported as the same registered speaker, the text is judged as irrelevant If the test sentence is the target language, otherwise, a maximum number of retries will be given. Therefore, by giving a maximum number of retries, the impostor's blocking rate will not be lost, and at the same time, the target speaker's erroneous blocking can be effectively solved, preventing the target speaker from being mistaken as an imposter, and effectively reducing the target speaker's blocking.

S1: First stage identification

S2: Second stage identification

S3: Third stage identification

S1-1: Prompt speaker identification begins

S1-2: User's sentence input

S1-3: Detect whether it is recording playback

S2-1: Prompt to enter the test sentence

S2-2: User's sentence input

S2-3: Detect whether it is a synthesized voice

a: Extract i-vector features and reduce their dimensionality

b: make a decision

c: Give a maximum number of retries

c1: Determine whether the upper limit of the number of retries has been exceeded

c2: Prompt to enter the test statement

P1: Pre-processing and sound framing

P2-1: Extract CQCC features

P2-2: Short-time Fourier Transform (STFT)

P3: Synthetic hybrid features

10: Input voice

Fig. 1 is a system flow chart of the speaker identification method of the present invention.

Figure 2 is a schematic diagram of hybrid feature vector generation.

FIG. 3 is a schematic diagram of the recognition of the input voice recognition according to the present invention.

Regarding the detailed description and technical content of the present invention, the illustrations are as follows: Please refer to "FIG. 1". In a preferred embodiment, the speaker identification method of the present invention can speak a text to a user Relevant test sentences are identified from an unrelated test sentence in this text. The method steps include: a first-stage identification S1, a second-stage identification S2, and a third-stage identification S3.

The first stage identification S1 includes steps S1-1 to S1-3, and step S1-1 is the start of the identification of the prompt speaker. In this step, a voice message is sent to notify the user that identification can be started. What is being done at this time is the identification of related sentences in this article, but the system does not prompt that this identification is related to or irrelevant to this article. At the same time, for the impersonator, they do not know the content of the related test sentences in this article.

Step S1-2 is the user's sentence input. In this step, the test sentence related to this text is generated in order to wait for the user to speak.

Step S1-3 is to detect whether it is recording playback. In this step, it is to determine whether the relevant test sentence of this article is a spoofing attack of recording playback. It mainly uses the characteristics of Mel Spectrum Coefficient (MFSC) to construct a text-dependent speech recognition model. The speech recognition model chooses to use speech recognition (Speech Recognition) or model matching (Template Matching) to form a text-dependent speech recognition model. Binary classifier, through the binary classifier to determine whether the relevant test sentence of this article is a spoofing attack of recording playback, if the relevant test sentence of this article is not a spoofing attack of recording playback, S1 is identified through the first stage, and Enter the second stage of identification S2; otherwise, reject the relevant test sentence of this text, and end the identification process.

The second stage identification S2 includes steps S2-1 to S2-3, where step S2-1 is to prompt to input a test sentence. In this step, in order to send another voice message to notify the user that they can start Speaking produces irrelevant test sentences for this article. This time, the input of the unrelated test sentence in this article is not limited to the sentence related to this article, but can be a sentence that is spoken at will (this article is irrelevant).

Step S2-2 is the user's sentence input. In this step, the text-independent test sentence is generated to wait for the user to speak.

Step S2-3 is to detect whether it is a synthesized voice. In this step, in order to determine whether the unrelated test sentence in this article is a spoofing attack of synthesized speech, a dimensionality-reduced hybrid feature is constructed for the unrelated test sentence in this article, and the hybrid feature is passed to another binary classifier to determine whether it is Synthetic speech, where the mixed feature is a mixed feature vector constructed by the constant Q cepstral coefficient (CQCC) and spectrogram (Spectrogram) of the irrelevant test sentence in this text. If it is judged that it is not a synthetic speech, the second stage identification S2 , And enter the third stage identification S3, otherwise it will reject the irrelevant test sentence in this article, and end the identification process.

Please also refer to "Figure 2", which is a schematic diagram of hybrid feature vector generation. The irrelevant test sentence in this article first goes through step P1: pre-processing and sound framing, and then separately proceeds to step P2-1: extracting CQCC features and step P2 -2: Short-time Fourier Transform (STFT), the final step: P3: Synthesize the mixed feature, and generate the mixed feature vector accordingly. The CQCC feature is a conventional voice feature, which combines the voice features obtained by Constant Q Transform and the traditional cepstrum process.

The third stage identification S3 is speaker identification, including steps a to c. In this step, in order to determine that the text-independent test sentence is a target speaker or an imposter, the third stage identification S3 is performed by a text-independent speaker identification system with plural registered speakers and plural classifiers. ~Step c.

Among them, step a is to extract the i-vector feature and reduce its dimensionality. Step a is to extract the i-vector features of the irrelevant test sentence of this text, and based on the feature vector after the i-vector dimensionality reduction, let the plural classifiers each determine that the irrelevant test sentence of this text is any one of the plural registrants. As a pretender, And each produces a judgment result. i-vector is a commonly used voice feature dimensionality reduction technology, which is mainly implemented based on the Joint Factor Analysis (JFA) method. It can reduce the dimensionality of voice features of different lengths and fix them to the same length. It is often used in speech recognition. . The number of the complex number classifiers used in the method of the present invention is preferably an odd number. An embodiment of the present invention uses 5, and they are respectively one-model DNN and multi-model deep. Neural network (Multi-model DNN), linear support vector machine (Linear-SVM), kernel support vector machine (Kernel-SVM) and random forest (Random Forest).

Step b is to make a decision. Step b is to make a decision based on the complex judgment result. The present invention adopts an overall learning judgment method to judge the irrelevant test sentence as the target speaker or imposter. In more detail, the overall learning is to integrate different classifiers. In this embodiment, it is to integrate the five classifiers described above, and when the plural judgment results are all judged to be the same registered speaker, the judgment is The irrelevant test sentence in this article is the target language speaker, and the third stage of identification is ended. When the plural judgment result judges that the number of persons with the same registered language is less than half, the irrelevant test sentence in this article is judged to be an imposter, and the third stage of identification is ended. When the plural judgment result judges that the number of persons with the same registered language is not all but not less than half, proceed to step c.

Step c is to give a retry with an upper limit and a positive integer. Step c mainly includes a step c1 and a step c2, and the step c1 is to determine whether the upper limit of the number of retries is exceeded. When the number of retries does not exceed the upper limit, step c2 is continued, and step c2 is a prompt to enter a test sentence. Step c2 allows the user to re-speak another irrelevant test sentence, and then perform step a again. In step c1, when the number of retries of the user exceeds the upper limit, the third stage identification is ended, and the irrelevant test sentence in this text is judged to be an imposter. In addition, for information security considerations, in one embodiment, the chance of retrying is not more than two times, and whenever there is an unrelated test sentence in this text that is judged to be an imposter, the speaker recognition system will send a warning message To the speaker to identify a manager of the system And lock the speaker recognition system. At the same time, unless the user can unlock the speaker, the speaker recognition system will be restarted after a preset period of time.

Please refer to "Figure 3" again, the user speaks an input voice 10, the input voice 10 contains the test sentence related to the text and the test sentence unrelated to the text, the input voice 10 may be the target speaker, deception attack ( (Including recording playback and synthesized speech) or pretenders, and L3, L1, or L2 respectively represent the best state of identification by the method of the present invention. Most of the spoofing attacks on recording playback and synthesized speech will be recognized and eliminated in the first stage identification S1 and the second stage identification S2 respectively, as shown in Figure 3 L1. Most of the impersonators will be identified and excluded in the third stage of identification S3, as shown in Figure 3 L2. Most of the target speakers will be identified through the first stage identification S1, the second stage identification S2, and the third stage identification step S3, as shown in Fig. 3 L3.

In a simulation test of the present invention, 20 target speakers, 20 spoofing attacks, and 67 imposters are included. There are 25 test sentences for each target speaker, 450 sentences for each deception attack, and 65 sentences for each impostor. The simulation test is divided into no retry mechanism (no retry is given), a retry mechanism (given a retry) and a conditional retry mechanism (given a maximum number of retries), which will be fetched after 10,000 times The averaged data is shown in the table below.

Compared with no retry mechanism, if the retry mechanism is not limited (retry until the result converges), the imposter blocking rate will decrease, which means that the imposter is more likely to be misjudged as the target speaker. And compared with no retry mechanism, under the conditional retry mechanism, the blocking rate of imposters and spoofing attacks does not change, but the blocking rate of target speakers has decreased significantly, so that more target speakers will not be misjudged as impersonating By. Therefore The conditional retry mechanism of the present invention can effectively reduce the blocking rate of the target speaker without losing the blocking rate of the impostor and the blocking rate of deception attacks.

In summary, the present invention has the following characteristics:

1. It has a Conditional Retry mechanism (Conditional Retry), and cooperates with the all-round decision strategy, which can effectively reduce the target speaker's blocking rate without losing the imposter's blocking rate.

2. Identify the relevant test sentences in this article and the unrelated test sentences in this article, effectively solving the problem of deception attacks.

S1: First stage identification

S2: Second stage identification

S3: Third stage identification

S1-1: Prompt speaker identification begins

S1-2: User's sentence input

S1-3: Detect whether it is recording playback

S2-1: Prompt to enter the test sentence

S2-2: User's sentence input

S2-3: Detect whether it is a synthesized voice

a: Extract i-vector features and reduce their dimensionality

b: make a decision

c: Give a maximum number of retries

c1: Determine whether the upper limit of the number of retries has been exceeded

c2: Prompt to enter the test statement

Claims (5)

A speaker identification method that identifies a test sentence related to the text and a test sentence unrelated to the text spoken by a user. The steps include: a first stage identification, which is to determine whether the test sentence is related to the text For the deception attack of recording playback, we use the characteristics of Mel Spectrum Coefficient (MFSC) to construct a text-dependent speech recognition model. The speech recognition model can be used in combination with Speech Recognition or Template Matching) to form a binary classifier. Through the binary classifier, it is judged whether the relevant test sentence of this article is a spoofing attack of recording playback. If the irrelevant test sentence of this article is not a spoofing attack of recording playback, the first stage is passed Recognize, otherwise, reject the relevant test sentence of the text; a second-stage recognition, which is to determine whether the irrelevant test sentence of this text is a synthetic speech deception attack, and firstly construct a dimensionality reduction Hybrid feature, the hybrid feature is a hybrid feature vector constructed by the constant Q cepstral coefficient (CQCC) and the spectrogram (Spectrogram) of the irrelevant test sentence in this text. First, let the irrelevant test sentence go through step P1: before Processing and framing, and then proceed separately to step P2-1: extract CQCC features and step P2-2: short-time Fourier transform (STFT), where step P2-1 is to obtain constant Q cepstral coefficients, and step P2-2 In order to obtain the spectrogram, the final step is: P3: Synthesize the mixed feature, that is, generate the mixed feature vector, and pass the mixed feature vector to another binary classifier to determine whether it is a synthetic speech. If it is judged not to be a synthetic speech, pass the The second stage of identification, otherwise it will reject the unrelated test sentence of the text; and a third stage of identification, the third stage of identification is to judge that the unrelated test sentence of this text is the target language person or the imposter, the third stage of identification is by a A speaker identification system with a plural registered speaker and a text-independent plural classifier performs the following steps: Step a is to extract the i-vector features of the irrelevant test sentence in this text, and based on the feature vector after i-vector dimensionality reduction, let the plural classifier individually determine that the irrelevant test sentence in this text is any of the plural registrants Or it is a pretender, and each produces a judgment result; step b, make a decision based on the plural judgment result, when the plural judgment results are all judged to be the same registered speaker, judge that the irrelevant test sentence in this article is the target language person, And end the third stage identification; when the plural judgment result judges that the number of persons with the same registered language is less than half, the irrelevant test sentence in this article is judged to be an imposter, and the third stage identification ends; when the plural judgment result judges If the number of persons who are the same registered language is not all but not less than half, proceed to the next step; and step c, give a maximum number of retries with a positive integer number, when the number of retries does not exceed the maximum number of retries When the user re-speaks another unrelated test sentence, perform step a again, but when the user’s retry exceeds the upper limit, the third stage of identification is ended, and the test is judged to be unrelated to the article The sentence is an imposter. The speaker identification method according to claim 1, wherein whenever there is an unrelated test sentence in the text that is judged to be an imposter, the speaker identification system will send a warning message to a manager of the speaker identification system and lock the The speaker recognition system, and unless the user can unlock it, the speaker recognition system will restart after a preset period of time. The speaker identification method according to claim 1, wherein the retry is not more than two times. The speaker identification method according to claim 1, wherein the number of the plural classifiers is an odd number. The speaker identification method according to claim 4, wherein the number of the plural classifiers is 5, and they are one-model deep neural network (One-model DNN) and multi-model deep neural network (Multi-model DNN). ), Linear-SVM, Kernel-SVM and Random Forest.

Images