CN113345444B - Speaker confirmation method and system - Google Patents

Abstract

The present invention provides a speaker confirmation method and system, including: preprocessing the speaker's audio information, converting the audio information into data in a preset format; inputting data in a preset format corresponding to the speaker's audio information to the trained deep nested residual neural network based on the spatial attention mechanism to obtain the frame-level speaker vector; based on the frame-level speaker vector, the utterance-level speaker vector is generated, and the utterance-level speaker vector is calculated. The cosine similarity between the speaker vector and the target speaker vector is used to determine whether the speaker is the target speaker; the target speaker vector is obtained in advance. The present invention proposes a deep nested residual neural network based on a spatial attention mechanism, which can extract speaker voiceprint features more accurately through the deep neural network.

Description

A speaker verification method and system

technical field

The invention belongs to the field of speaker identification, and more specifically relates to a method and system for speaker identification.

Background technique

Voiceprint is a general term for the voice features contained in the voice, which can represent and mark the speaker, and the voice model established based on these features. As a necessary medium in interpersonal communication, language plays a very important role. It is difficult for a person’s voice to change after adulthood. Due to the difference in vocal organs and pronunciation methods, each person’s voice characteristics in the process of speaking are unique. Even if they are imitated, it is difficult to change the speaker’s most basic pronunciation characteristics and vocal characteristics. Therefore, according to the unique identification and short-term stability of the voiceprint, the present invention can establish a voice model and use it for identity authentication.

Voiceprint recognition, also called speaker recognition, is the process of determining the target speaker based on the voiceprint characteristics of the target speaker to be recognized. Speaker recognition tasks can be fundamentally divided into two categories: speaker recognition and speaker confirmation. Speaker recognition refers to giving a voiceprint sample and matching whether it is the identity of the target speaker from the trained corpus. One-to-one confirmation problem; speaker confirmation refers to confirming which speaker sample in the corpus the identity of the target voiceprint belongs to, and it is a many-to-one selection problem.

Speaker confirmation methods are very important in people's daily work and life, although many domestic and foreign researchers have proposed a series of mature speaker confirmation methods such as GMM-UBM Gaussian mixture-general background model, LSTM long short-term memory Network models and CNN convolutional neural network models, etc., but there are still many problems and room for improvement in research methods, model effects, and application scenarios.

The deficiencies of the existing speaker confirmation methods mainly include the following points: 1. Most of the methods are based on the traditional GMM-UBM Gaussian mixture-universal background model, and the technical means are still in the most traditional model matching stage. With the rapid development of social progress and science and technology, the traditional method reflects the complexity and difficulty in processing a large amount of data information. 2. The speaker's voiceprint contains a lot of effective speaker information, and the speaker's audio is easily affected by various factors such as the environment, recording equipment, and the speaker himself. The existing deep learning speaker confirmation algorithm cannot better extract speaker features, and the overall effect needs to be improved. 3. The speaker confirmation method in different scenes and different languages is the current challenge of voiceprint recognition. The speaker confirmation method in video memory cannot minimize the impact of cross-language data on the model recognition effect, and the effect is not enough in cross-language recognition research. ideal.

SUMMARY OF THE INVENTION

Aiming at the defects of the prior art, the object of the present invention is to provide a speaker confirmation method and system, which aims to solve the problems of low recognition accuracy and large network scale in the existing speaker recognition neural network system.

In order to achieve the above object, in a first aspect, the present invention provides a speaker confirmation method, comprising the following steps:

Preprocessing the audio information of the speaker, converting the audio information into data in a preset format;

Input the data in the preset format corresponding to the speaker's audio information into the trained deep nested residual neural network based on the spatial attention mechanism to obtain the speaker vector at the frame level; the deep embedding based on the spatial attention mechanism The set of residual neural network includes: each layer of four layers contains a residual neural network and a spatial attention mechanism of two nested residual blocks; after the nested residual neural network, a spatial attention mechanism is introduced, and the spatial attention mechanism Introduce average pooling and maximum pooling in the attention module based on the spatial dimension, and combine the two parts of the pooling results to retain useful information to reduce the parameter scale, and use the sigmoid function in the activation layer of the attention module to obtain the frame level speaker vector;

Generate an utterance-level speaker vector based on the frame-level speaker vector, and calculate the cosine similarity between the utterance-level speaker vector and the target speaker vector to determine whether the speaker is a target speaker; The target speaker vectors are pre-fetched.

In an optional example, the preprocessing is performed on the audio information of the speaker, and the audio information is converted into data in a preset format, specifically:

The speaker's WAV format audio file is converted into a flac format file using audio conversion technology, and the flac format file is preprocessed to obtain npy format data containing all information about the speaker.

In an optional example, each nested residual block contains two sub-residual blocks, each sub-residual block contains two units, and each unit is a building block; every two nested residual blocks A convolutional layer is placed in front of the block;

Two nested sub-residual blocks realize the stacking function, and the specific formula is:

H ₁ (x)=F ₁ (x)+x

H ₂ (x)=F ₂ (x)+H ₁ (x)

H(x)＝H ₂ (x)+x

Among them, x represents the input data of the first nested residual block, F ₁ (x) represents the output of the first sub-residual block in the nested residual block, H ₁ (x) represents F ₁ (x) and The combined data of x, F ₂ (x) represents the output of the second sub-residual block in the nested residual block, H ₂ (x) represents the combined data of F ₂ (x) and H ₁ (x), H(x ) denote the output of two nested residual blocks.

In an optional example, the spatial attention mechanism is introduced after the nested residual block, and the sigmoid function is used in the activation layer of the attention module to obtain the speaker vector at the frame level, the specific formula is:

F″=f{avg_pool(V), max_pool(V)}

F'=σ(F")

F=Multiply(V,F')

Among them, V represents the speaker vector output by the nested residual neural network, avg_pool represents the average pooling operation, max_pool represents the maximum pooling operation, and f{} represents the combination of the results of the two pooling operations to obtain a new speaker Vector F″; F′ represents the speaker vector obtained by adding an activation function to F″; F represents the frame-level speaker vector.

In an optional example, the calculation of the cosine similarity between the speaker vector of the utterance level and the target speaker vector to determine whether the speaker is the target speaker is specifically:

A threshold is set for the probability value of the cosine similarity, and when the probability value of the cosine similarity is greater than the threshold, it is judged that the speaker is the target speaker, otherwise it is judged that the speaker is not the target speaker.

In a second aspect, the present invention provides a speaker confirmation system, comprising:

A speaker audio determination unit, configured to preprocess the audio information of the speaker, and convert the audio information into data in a preset format;

The frame-level vector determination unit is used to input the data in the preset format corresponding to the speaker's audio information to the trained deep nested residual neural network based on the spatial attention mechanism to obtain the frame-level speaker vector; The deep nested residual neural network based on the spatial attention mechanism includes: a four-layer nested residual neural network with two nested residual blocks in each layer and a spatial attention mechanism; the spatial attention is introduced after the nested residual neural network Attention mechanism, the spatial attention mechanism introduces average pooling and maximum pooling in the attention module based on the spatial dimension, and merges the two parts of the pooling results to retain useful information and reduce the parameter scale, and in the attention module The sigmoid function is used in the activation layer to obtain the speaker vector at the frame level;

A speaker confirmation unit, configured to generate an utterance-level speaker vector based on the frame-level speaker vector, and calculate the cosine similarity between the utterance-level speaker vector and the target speaker vector to determine the speaker Whether it is the target speaker; the target speaker vector is obtained in advance.

In an optional example, the speaker audio determination unit converts the speaker's WAV format audio file into a flac format file using audio conversion technology, preprocesses the flac format file, and obtains npy containing all information about the speaker format data.

In an optional example, each nested residual block contains two sub-residual blocks, each sub-residual block contains two units, and each unit is a building block; each of the two nested residual blocks Place a convolutional layer in front;

Two nested sub-residual blocks realize the stacking function, and the specific formula is:

H ₁ (x)=F ₁ (x)+x

H ₂ (x)=F ₂ (x)+H ₁ (x)

H(x)＝H ₂ (x)+x

Among them, x represents the input data of the first nested residual block, F ₁ (x) represents the output of the first sub-residual block in the nested residual block, H ₁ (x) represents F ₁ (x) and The combined data of x, F ₂ (x) represents the output of the second sub-residual block in the nested residual block, H ₂ (x) represents the combined data of F ₂ (x) and H ₁ (x), H(x ) denote the output of two nested residual blocks.

In an optional example, the frame-level vector determination unit introduces a spatial attention mechanism after the nested residual block, and uses a sigmoid function in the activation layer of the attention module to obtain a frame-level speaker vector, The specific formula is:

F″=f{avg_pool(V), max_pool(V)}

F'=σ(F")

F=Multiply(V,F')

Among them, V represents the speaker vector output by the nested residual neural network, avg_pool represents the average pooling operation, max_pool represents the maximum pooling operation, and f{} represents the combination of the results of the two pooling operations to obtain a new speaker Vector F″; F′ represents the speaker vector obtained by adding an activation function to F″; F represents the frame-level speaker vector.

In an optional example, the speaker confirmation unit sets a threshold for the probability value of the cosine similarity, and when the probability value of the cosine similarity is greater than the threshold, it is judged that the speaker is the target speaker Otherwise, it is judged that the speaker is not the target speaker.

Generally speaking, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:

The invention provides a speaker confirmation method and system, and proposes a novel nested residual neural network. After demonstrating the efficiency of ResNet in speaker verification, we conduct a series of explorations in this research. This paper proposes for the first time the concept of using residuals in residuals; the present invention proposes the use of spatial attention mechanisms in speaker verification tasks. The invention adopts the method of adding an attention mechanism to obtain more valuable voiceprint information from the voiceprint energy spectrum, which is the first time that the spatial attention used for image processing is used for the processing of voiceprint information. The present invention proposes a speaker confirmation network model with better performance, and proves that the method is superior to the current cutting-edge technology through experiments, and then adds an attention mechanism on the basis of the network model to further improve the system performance.

The present invention provides a speaker confirmation method and system, and proposes a deeply nested residual neural network based on a spatial attention mechanism, which can more accurately extract speaker voiceprint features through the deep neural network. Experimental results show that the method has high learning ability and is able to surpass the performance of previous neural network methods. Compared with other state-of-the-art methods, the experimental results show a 6% improvement in accuracy on the English public dataset LibriSpeech with a similarly low error rate. The present invention also proves that the method has good cross-language adaptability through the training performance on the Chinese data set AISHELL.

Description of drawings

Fig. 1 is a flow chart of a speaker confirmation method provided by an embodiment of the present invention;

Fig. 2 is another kind of speaker confirmation flowchart provided by the embodiment of the present invention;

FIG. 3 is a structural diagram of a nested residual block provided by an embodiment of the present invention;

4 is a structure diagram of a nested residual neural network based on an attention mechanism provided by an embodiment of the present invention;

Fig. 5 is a structural diagram of a speaker's spatial attention mechanism provided by an embodiment of the present invention;

Fig. 6 is a performance diagram of different models provided by the embodiment of the present invention on the English data set;

Fig. 7 is a performance diagram of different models provided by the embodiment of the present invention on the Chinese data set;

Fig. 8 is a performance comparison diagram on different data sets before and after adding the attention mechanism provided by the embodiment of the present invention;

Figure 9 is a comparison of the performance of different models provided by the embodiment of the present invention on the Chinese-English cross-language data set;

Fig. 10 is an architecture diagram of a speaker verification system provided by an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The purpose of the present invention is to extract the fbank voiceprint features of the speaker based on the spatial attention mechanism and the deep nested residual neural network. Frame-level vector representation, and use cosine similarity to judge whether the current speaker is the target speaker.

Fig. 1 is a kind of speaker confirming method flowchart that the embodiment of the present invention provides; As shown in Fig. 1, comprises the following steps:

S101, preprocessing the audio information of the speaker, and converting the audio information into data in a preset format;

S102, input the data in the preset format corresponding to the speaker's audio information to the trained deep nested residual neural network based on the spatial attention mechanism to obtain the speaker vector at the frame level; the spatial attention mechanism-based The deep nested residual neural network includes: a nested residual neural network and a spatial attention mechanism with four layers each containing two nested residual blocks; a spatial attention mechanism is introduced after the nested residual neural network, the The spatial attention mechanism introduces average pooling and maximum pooling in the attention module based on the spatial dimension, and merges the two parts of the pooling results to retain useful information and reduce the parameter scale, and uses the sigmoid function in the activation layer of the attention module , to obtain the frame-level speaker vector;

S103. Generate an utterance-level speaker vector based on the frame-level speaker vector, and calculate the cosine similarity between the utterance-level speaker vector and the target speaker vector, to determine whether the speaker is the target speaker ; The target speaker vector is pre-acquired.

The speaker confirmation method based on spatial attention mechanism and deep nested residual neural network in the present invention is divided into three parts: audio preprocessing, speaker voiceprint feature extraction and speaker confirmation; the overall processing flow is shown in Figure 2 Show. First, the WAV audio files with speaker labels in the public dataset Librispeech are processed into flac files using audio conversion technology, and the flac files are preprocessed to obtain .npy files containing all information about the speakers. Then the .npy file is used as the input of the deep nested residual neural network, and the frame-level vector representation of the speaker is obtained through a series of convolution and pooling operations. During the training process, training is performed in the form of AP and AN sample pairs, where AP stands for anchor-positive, that is, control-positive samples, and AN stands for anchor-negative, that is, control-negative samples. The deep nested neural network improves the performance of the model to the speaker confirmation algorithm by continuously learning the vector similarity in AP and AN.

The present invention proposes nested residual blocks suitable for speaker confirmation tasks. The detailed network structure is shown in Figure 3. Relu represents the linear rectification function, which is a commonly used activation function; NR-block is the structure of the nested residual block proposed in the present invention.

In the architecture of the present invention, two nested residual blocks are stacked, and each nested residual block is formed by stacking two sub-residual blocks. Each subresidual block contains two units, and each unit is a building block. The present invention places a convolutional layer in front of every two nested residual blocks, sets the size of the kernel to 5, and sets the stride to 2, which can perfectly cope with the changes caused by the increase in the number of channels.

In nested residual neural networks, H(x) is considered as the base map fitted to the overlay map, where x represents the input of the first layer in the nested residual block. In the first building block, the present invention assumes that the residual function is formula (1), then formula (2) can be obtained in the second residual block. Therefore, the present invention can realize the stacking function of the nested residual block, and can be used as the input of the next nested residual block. Changing the objective function from H(x) to F(x) can greatly reduce the difficulty of network learning.

H ₁ (x)=F ₁ (x)+x (1)

H ₂ (x) = F ₂ (x) + H ₁ (x) (2)

H(x)＝H ₂ (x)+x (3)

Among them, the subscript 1 represents the first residual block in the proposed nested residual block, and the subscript 2 represents the second residual block in the proposed nested residual block.

Specifically, the nested residual neural network is stacked by four identical network layers containing two nested residual blocks, and each nested residual block contains two residual blocks. In the nested neural network, the output H(X) of the first nested residual block is used as the input of the second nested residual block, continue to execute formula (1)-formula (3), and so on, to get The output of the last nested residual block is also the output of the entire nested residual block in the nested residual neural network, which can be expressed as V, which is the speaker vector below, and V is also the speaker vector at the frame level.

The speaker vector of the nested residual neural network output from the frame-level feature extractor can be expressed as: V = [v ₁ v ₂ ... v _d ]. Among them v _t ∈ R ^D , R ^D belongs to the D-dimensional space vector. The symbol v denotes the vector corresponding to each frame, and the symbol d denotes the input frame dimensions of the layer. By studying the present invention, it can be found that only using a deep neural network may cause gradient explosion and gradient disappearance problems. In the process of transmitting voiceprint information in the form of voiceprint energy spectrum, if all the information is used, it will undoubtedly increase the workload of the system, and this method may be unreliable.

Therefore, FIG. 4 is a structural diagram of a nested residual neural network based on an attention mechanism provided by an embodiment of the present invention; FIG. 5 is a structural diagram of a speaker space attention mechanism provided by an embodiment of the present invention; A spatial attention mechanism is introduced after the block. The present invention introduces average pooling and max pooling in the attention module based on the spatial dimension, and then combines the two together in equation (4), retaining useful information and reducing the scale of parameters. Then, the dimensionality of the features is reduced to a single channel using a 2D convolutional layer. The sigmoid function is used in the activation layer to obtain the speaker's spatial attention features. Finally, Equation (6) adds and multiplies the features and outputs of the attention input to the feature module. So far, the present invention has obtained the speaker frame-level feature vector.

F″=f{avg_pool(V), max_pool(V)} (4)

F'=σ(F") (5)

F = Multiply(V, F') (6)

Among them, V represents the speaker vector output by the nested residual neural network, avg_pool represents the average pooling operation, max_pool represents the maximum pooling operation, and f{} represents the combination of the results of the two pooling operations to obtain a new speaker Vector F"; F' represents the speaker vector obtained by adding an activation function to F"; F represents the final output frame-level speaker vector, which is also the frame-level speaker vector used for cosine similarity judgment later.

Specifically, the frame-level speaker vectors are input into a dimensionality reduction layer, and then averaged in the time dimension to generate utterance-level speaker vectors. The utterance-level speaker vector is generated from the frame-level speaker vector through dimensionality reduction, fully connected layers, averaging, and length normalization.

Fully connected layers project the utterance-level representations into 512-dimensional speaker vectors. We normalize the features by a length normalization operation and use cosine similarity in the objective function:

cos(X, Y)=X ^T Y

where X and Y are two different speaker utterance-level vectors. Then use the cosine similarity formula in the equation to model the probability of similarity between X and Y vectors, and determine whether they are the same speaker by setting a certain threshold for the similarity probability.

In this paper, four experiments are carried out according to different corpora and whether to add attention mechanism. The experimental comparison results show that the method has a high learning ability and can surpass the performance of previous neural network methods.

Among them, in the following Figures 6-9, the ordinate ACC represents the accuracy rate, and the abscissa epoch represents the number of verification rounds.

Figure 6 shows the experimental results of five different network models on the English public dataset LibriSpeech. One of the experiments, baseResNet, serves as a control experiment for this method. On the basis of the control experiment, the present invention fine-tunes the network structure respectively to observe the influence of different structures on the speaker confirmation task. Wherein, the Proposed-NResNet model in Fig. 6 refers to the nested residual neural network model proposed by the present invention. In the control experiment, the residual block is stacked three times, and ResNet-bd is an adjustment for different positions of the residual block in the network. In the present invention, nesting and stacking two residual blocks in NResNet is one of the innovations. From the experimental results, baseResNet, ResNet-bd3 and Proposed-NResNet can achieve ideal performance, but it can be clearly observed that the Proposed-NResNet model proposed by the present invention is more stable and has higher accuracy.

Comprehensive and scientific is the consistent purpose of research. In Figure 7, the present invention selects the best three sets of experiments, and then explores the results on the Chinese public dataset AISHELL. Wherein, the Proposed-NResNet model in FIG. 7 refers to the nested residual neural network model proposed by the present invention. The present invention uses the audio data of 151 people from AISHELL as a training set and the audio data of 40 people as a verification set to verify that the method mentioned in this paper is also feasible in Chinese. First the data must be processed into the format required by the neural network. As shown in the results in Figure 7, the method of the present invention can achieve excellent results on both Chinese and English datasets.

Adding speaker spatial attention mechanism in deep nested neural network is the second innovation of this work. The present invention compares the performance of adding the spatial attention mechanism to NResNet under different language environments through experiments. The present invention observes the effect of each group separately. The specific details can be analyzed in Figure 8. Among them, the Proposed-NResNet+SA model in Fig. 8 refers to the combined model of nested residual neural network and speaker's attention proposed by the present invention. Surprisingly, the accuracy of Proposed-NResNet+SA proposed by the present invention can reach the best performance close to 0.99. There is a significant improvement compared to no attention mechanism added, which further illustrates the high efficiency of the attention mechanism proposed in this paper in the speaker confirmation task.

In addition, the present invention compares the effect of this method on cross-language datasets in FIG. 9 . Wherein, the Proposed-NResNet model in FIG. 9 refers to the nested residual neural network model proposed by the present invention. In this experiment, the present invention uses LibriSpeech's Train-clean-100 as a training set, and the audios of 40 people in the AISHELL Chinese data set are used as a verification set. From the data in the figure, it can be found that the model trained on the English data set by the method proposed by the present invention can also obtain satisfactory results on the Chinese data set.

Table 1 is a comparison table of the accuracy rate and equal error rate of the four neural network models used as a control experiment and the two models proposed in the method of the present invention on the English data set LibriSpeech and the Chinese data set AISHELL respectively. It can be clearly seen from Table 1 that compared with the existing models, the nested residual neural network Proposed-NResNet proposed by the present invention has an improvement of about 3% in the accuracy rate on the English and Chinese data sets, and the equal error rate is even lower. Low; the method Proposed-NResNet+SA, which combines the nested residual neural network and the speaker's attention mechanism proposed by the present invention, has a 3% improvement in accuracy compared with the nested residual neural network alone, and compared with the existing There is a 6% improvement in the model as a whole, and the error rate is nearly doubled.

Table 1 The accuracy rate and equal error rate table of the model

Fig. 10 is an architecture diagram of a speaker confirmation system provided by an embodiment of the present invention, as shown in Fig. 10 , including:

Speaker audio determination unit 1010, configured to preprocess the audio information of the speaker, and convert the audio information into data in a preset format;

The frame-level vector determination unit 1020 is used to input the data in the preset format corresponding to the speaker's audio information to the trained deep nested residual neural network based on the spatial attention mechanism to obtain the frame-level speaker vector; The deep nested residual neural network based on the spatial attention mechanism includes: a nested residual neural network with four layers each containing two nested residual blocks and a spatial attention mechanism; after the nested residual neural network, the Spatial attention mechanism, the spatial attention mechanism introduces average pooling and maximum pooling in the attention module based on the spatial dimension, and merges the two parts of the pooling results to retain useful information and reduce the parameter scale, and in the attention module Use the sigmoid function in the activation layer to obtain the frame-level speaker vector;

The speaker confirmation unit 1030 is configured to generate an utterance-level speaker vector based on the frame-level speaker vector, and calculate the cosine similarity between the utterance-level speaker vector and the target speaker vector to determine the utterance Whether the speaker is the target speaker; the target speaker vector is obtained in advance.

Specifically, for the functions of each unit in FIG. 10 , refer to the introduction in the foregoing method embodiments, and details are not repeated here.

Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, etc., All should be included within the protection scope of the present invention.

Claims (10)

1. a speaker confirmation method, is characterized in that, comprises the steps: Preprocessing the audio information of the speaker, converting the audio information into data in a preset format; Input the data in the preset format corresponding to the speaker's audio information into the trained deep nested residual neural network based on the spatial attention mechanism to obtain the speaker vector at the frame level; the deep embedding based on the spatial attention mechanism The nested residual neural network includes: four layers of nested residual neural network and a spatial attention mechanism containing two nested residual blocks in each layer; a spatial attention mechanism is introduced after the nested residual neural network, and the spatial attention The force mechanism introduces average pooling and maximum pooling in the attention module based on the spatial dimension, and merges the two parts of the pooling results to retain useful information to reduce the parameter scale, and uses the sigmoid function in the activation layer of the attention module to Get frame-level speaker vectors; Generate an utterance-level speaker vector based on the frame-level speaker vector, and calculate the cosine similarity between the utterance-level speaker vector and the target speaker vector to determine whether the speaker is a target speaker; The target speaker vectors are pre-fetched. 2. The speaker confirmation method according to claim 1, wherein the audio information of the speaker is preprocessed, and the audio information is converted into data in a preset format, specifically: The speaker's WAV format audio file is converted into a flac format file using audio conversion technology, and the flac format file is preprocessed to obtain npy format data containing all information about the speaker. 3. The speaker confirmation method according to claim 1, wherein each nested residual block contains two sub-residual blocks, each sub-residual block contains two units, and each unit is a building block ; Place a convolutional layer in front of every two nested residual blocks; Two nested sub-residual blocks realize the stacking function, and the specific formula is: H ₁ (x)=F ₁ (x)+x H ₂ (x)=F ₂ (x)+H ₁ (x) H(x)＝H ₂ (x)+x Among them, x represents the input data of the first nested residual block, F ₁ (x) represents the output of the first sub-residual block in the nested residual block, H ₁ (x) represents F ₁ (x) and The combined data of x, F ₂ (x) represents the output of the second sub-residual block in the nested residual block, H ₂ (x) represents the combined data of F ₂ (x) and H ₁ (x), H(x ) denote the output of two nested residual blocks. 4. The speaker confirmation method according to claim 1, characterized in that a spatial attention mechanism is introduced after the nested residual block, and a sigmoid function is used in the activation layer of the attention module to obtain frame-level speech Or vector, the specific formula is: F″=f{avg_pool(V), max_pool(V)} F'=σ(F") F=Multiply(V,F') Among them, V represents the speaker vector output by the nested residual neural network, avg_pool represents the average pooling operation, max_pool represents the maximum pooling operation, and f{} represents the combination of the results of the two pooling operations to obtain a new speaker Vector F"; F' represents the speaker vector obtained by adding the activation function to F"; F represents the speaker vector at the frame level, and Multiply represents the addition and multiplication operation. 5. The speaker confirmation method according to any one of claims 1 to 4, wherein the calculation of the cosine similarity between the speaker vector of the utterance level and the target speaker vector is to determine the speaker Whether it is the target speaker, specifically: A threshold is set for the probability value of the cosine similarity, and when the probability value of the cosine similarity is greater than the threshold, it is judged that the speaker is the target speaker, otherwise it is judged that the speaker is not the target speaker. 6. A speaker confirmation system, comprising: A speaker audio determination unit, configured to preprocess the audio information of the speaker, and convert the audio information into data in a preset format; The frame-level vector determination unit is used to input the data in the preset format corresponding to the speaker's audio information to the trained deep nested residual neural network based on the spatial attention mechanism to obtain the frame-level speaker vector; The deep nested residual neural network based on the spatial attention mechanism includes: a four-layer nested residual neural network with two nested residual blocks in each layer and a spatial attention mechanism; the spatial attention is introduced after the nested residual neural network Attention mechanism, the spatial attention mechanism introduces average pooling and maximum pooling in the attention module based on the spatial dimension, and merges the two parts of the pooling results to retain useful information and reduce the parameter scale, and in the attention module The sigmoid function is used in the activation layer to obtain the speaker vector at the frame level; A speaker confirmation unit, configured to generate an utterance-level speaker vector based on the frame-level speaker vector, and calculate the cosine similarity between the utterance-level speaker vector and the target speaker vector to determine the speaker Whether it is the target speaker; the target speaker vector is obtained in advance. 7. The speaker confirmation system according to claim 6, wherein the speaker audio determination unit converts the speaker's WAV format audio file into a flac format file using audio conversion technology, and pre-processes the flac format file. Processing to get the npy format data containing all the information of the speaker. 8. The speaker confirmation system according to claim 6, wherein each nested residual block contains two sub-residual blocks, each sub-residual block contains two units, and each unit is a building block ; Place a convolutional layer in front of every two nested residual blocks; Two nested sub-residual blocks realize the stacking function, and the specific formula is: H ₁ (x)=F ₁ (x)+x H ₂ (x)=F ₂ (x)+H ₁ (x) H(x)＝H ₂ (x)+x Among them, x represents the input data of the first nested residual block, F ₁ (x) represents the output of the first sub-residual block in the nested residual block, H ₁ (x) represents F ₁ (x) and The combined data of x, F ₂ (x) represents the output of the second sub-residual block in the nested residual block, H ₂ (x) represents the combined data of F ₂ (x) and H ₁ (x), H(x ) denote the output of two nested residual blocks. 9. The speaker confirmation system according to claim 6, wherein the frame-level vector determination unit introduces a spatial attention mechanism after the nested residual block, and uses a sigmoid function in the activation layer of the attention module , to obtain the speaker vector at the frame level, the specific formula is: F″=f{avg_pool(V), max_pool(V)} F'=σ(F") F=Multiply(V,F') Among them, V represents the speaker vector output by the nested residual neural network, avg_pool represents the average pooling operation, max_pool represents the maximum pooling operation, and f{} represents the combination of the results of the two pooling operations to obtain a new speaker Vector F"; F' represents the speaker vector obtained by adding the activation function to F"; F represents the speaker vector at the frame level, and Multiply represents the addition and multiplication operation. 10. The speaker confirmation system according to any one of claims 6 to 9, wherein the speaker confirmation unit sets a threshold for the probability value of the cosine similarity, when the probability value of the cosine similarity If it is greater than the threshold, it is judged that the speaker is the target speaker, otherwise it is judged that the speaker is not the target speaker.

Images