CN114005452A

CN114005452A - Method and device for extracting voice features, electronic equipment and storage medium

Info

Publication number: CN114005452A
Application number: CN202111272479.0A
Authority: CN
Inventors: 张银辉; 赵情恩; 熊新雷; 陈蓉; 梁芸铭; 周羊; 肖岩
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-10-29
Filing date: 2021-10-29
Publication date: 2022-02-01

Abstract

The disclosure provides a method for extracting voice features, relates to the technical field of artificial intelligence, and particularly relates to the field of voiceprint recognition. The implementation scheme is that the method comprises the following steps: acquiring voice to be processed, and framing the voice to obtain multi-frame voice data; extracting the features of the multi-frame voice data to obtain a first feature vector of the voice, wherein the first feature vector comprises a plurality of elements which respectively correspond to the corresponding features of the multi-frame voice data; obtaining a covariance matrix associated with a plurality of elements in the first eigenvector; acquiring a second feature vector of the voice based on the first feature vector and the covariance matrix; acquiring a parameter vector representing the importance of a plurality of elements in the first feature vector; and splicing the second feature vector with the parameter vector to obtain the voice feature of the voice.

Description

Method and device for extracting voice features, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence technology, in particular to the field of voiceprint recognition, and in particular to a method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product for extracting speech features.

Background

Artificial intelligence is the subject of research that makes computers simulate some human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, machine learning/deep learning, a big data processing technology, a knowledge map technology and the like.

In the voiceprint recognition technology, it is crucial to extract the speech features. Extracting speech features may generally include various modules for extracting front-end features (also referred to as low-level features), for extracting features of the speaker (also referred to as high-level features), and for back-end classification. The voice data can obtain corresponding recognition results after being processed by the modules.

However, in the process of extracting the speech features, the interference from various factors such as the speaker and the environment is often caused, which has a certain influence on the accuracy of extracting the speech features. How to eliminate these influences and extract the features of the voice data more accurately has become a popular research field in the technology of extracting voice features in recent years.

The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, unless otherwise indicated, the problems mentioned in this section should not be considered as having been acknowledged in any prior art.

Disclosure of Invention

The present disclosure provides a method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product for extracting speech features.

According to an aspect of the present disclosure, there is provided a method of extracting a speech feature, including: acquiring voice to be processed, and framing the voice to obtain multi-frame voice data; extracting the features of the multi-frame voice data to obtain a first feature vector of the voice, wherein the first feature vector comprises a plurality of elements which respectively correspond to the corresponding features of the multi-frame voice data; obtaining a covariance matrix associated with a plurality of elements in the first eigenvector; acquiring a second feature vector of the voice based on the first feature vector and the covariance matrix; acquiring a parameter vector representing the importance of a plurality of elements in the first feature vector; and splicing the second feature vector with the parameter vector to obtain the voice feature of the voice.

According to another aspect of the present disclosure, there is provided a method for training a speech feature extraction model, comprising: obtaining sample voice labeled by a sample, and framing the sample voice to obtain multi-frame sample voice data; performing feature extraction on the multi-frame sample voice data to obtain a first sample feature vector of the sample voice, wherein the first sample feature vector comprises a plurality of sample elements which respectively correspond to corresponding features of the multi-frame sample voice data; obtaining a sample covariance matrix associated with a plurality of sample elements in a first sample feature vector; acquiring a second sample feature vector of the sample voice based on the first sample feature vector and the sample covariance matrix; obtaining a sample parameter vector representing the importance of a plurality of sample elements in a first feature vector of a sample; splicing the second sample feature vector with the sample parameter vector to obtain the voice feature of the sample voice; and acquiring network parameters for updating the voice feature extraction model based on the voice features so as to train the voice feature extraction model.

According to another aspect of the present disclosure, there is provided an apparatus for extracting a speech feature, including: a first module configured to: acquiring voice to be processed, and framing the voice to obtain multi-frame voice data; a second module configured to: extracting the features of the multi-frame voice data to obtain a first feature vector of the voice, wherein the first feature vector comprises a plurality of elements which respectively correspond to the corresponding features of the multi-frame voice data; a third module configured to: obtaining a covariance matrix associated with a plurality of elements in the first eigenvector; a fourth module configured to: acquiring a second feature vector of the voice based on the first feature vector and the covariance matrix; a fifth module configured to: acquiring a parameter vector representing the importance of a plurality of elements in the first feature vector; and a sixth module configured to: and splicing the second feature vector with the parameter vector to obtain the voice feature of the voice.

According to another aspect of the present disclosure, there is provided an apparatus for training a speech feature extraction model, including: a first module configured to: obtaining sample voice labeled by a sample, and framing the sample voice to obtain multi-frame sample voice data; a second module configured to: performing feature extraction on the multi-frame sample voice data to obtain a first sample feature vector of the sample voice, wherein the first sample feature vector comprises a plurality of sample elements which respectively correspond to corresponding features of the multi-frame sample voice data; a third module configured to: obtaining a sample covariance matrix associated with a plurality of sample elements in a first sample feature vector; a fourth module configured to: acquiring a second sample feature vector of the sample voice based on the first sample feature vector and the sample covariance matrix; a fifth module configured to: obtaining a sample parameter vector representing the importance of a plurality of sample elements in a first feature vector of a sample; a sixth module configured to: splicing the second sample feature vector with the sample parameter vector to obtain voice features; and a seventh module configured to: network parameters for updating the speech feature extraction model are obtained based on the speech features to train the speech feature extraction model.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-described method.

According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the above method.

According to another aspect of the disclosure, a computer program product is provided, comprising a computer program, wherein the computer program realizes the above method when executed by a processor.

According to one or more embodiments of the present disclosure, features of voice data can be more accurately extracted.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the embodiments and, together with the description, serve to explain the exemplary implementations of the embodiments. The illustrated embodiments are for purposes of illustration only and do not limit the scope of the claims. Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements.

FIG. 1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented, according to an embodiment of the present disclosure;

FIG. 2 shows a flow diagram of a method of extracting speech features according to an embodiment of the present disclosure;

FIG. 3 shows a flow diagram of a method for training a speech feature extraction model according to an embodiment of the present disclosure;

FIG. 4 shows a schematic diagram of a speech extraction model according to an embodiment of the present disclosure;

FIG. 5 shows a block diagram of an apparatus for extracting speech features according to one embodiment of the present disclosure;

FIG. 6 shows a block diagram of an apparatus for extracting speech features according to another embodiment of the present disclosure;

FIG. 7 shows a block diagram of an apparatus for training a speech feature extraction model according to an embodiment of the present disclosure;

fig. 8 shows a block diagram of an electronic device to which the embodiments of the present disclosure can be applied.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, the timing relationship, or the importance relationship of the elements, and such terms are used only to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, based on the context, they may also refer to different instances.

The terminology used in the description of the various described examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the elements may be one or more. Furthermore, the term "and/or" as used in this disclosure is intended to encompass any and all possible combinations of the listed items.

Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

Fig. 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented in accordance with embodiments of the present disclosure. Referring to fig. 1, the system 100 includes one or

more client devices

101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120.

Client devices

101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

In embodiments of the present disclosure, the server 120 may run one or more services or software applications that enable the methods of extracting speech features of the present disclosure to be performed.

In some embodiments, the server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. In certain embodiments, these services may be provided as web-based services or cloud services, for example, provided to users of

client devices

101, 102, 103, 104, 105, and/or 106 under a software as a service (SaaS) model.

In the configuration shown in fig. 1, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof, which may be executed by one or more processors. A user operating a

client device

101, 102, 103, 104, 105, and/or 106 may, in turn, utilize one or more client applications to interact with the server 120 to take advantage of the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from system 100. Accordingly, fig. 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

A user may provide voice data using

client devices

101, 102, 103, 104, 105, and/or 106. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although fig. 1 depicts only six client devices, those skilled in the art will appreciate that any number of client devices may be supported by the present disclosure.

Client devices

101, 102, 103, 104, 105, and/or 106 may include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and so forth. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (e.g., GOOGLE Chrome OS); or include various Mobile operating systems such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular telephones, smart phones, tablets, Personal Digital Assistants (PDAs), and the like. Wearable devices may include head mounted displays and other devices. The gaming system may include a variety of handheld gaming devices, internet-enabled gaming devices, and the like. The client device is capable of executing a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications), Short Message Service (SMS) applications, and may use a variety of communication protocols.

Network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP/IP, SNA, IPX, etc. By way of example only, one or more networks 110 may be a Local Area Network (LAN), an ethernet-based network, a token ring, a Wide Area Network (WAN), the internet, a virtual network, a Virtual Private Network (VPN), an intranet, an extranet, a Public Switched Telephone Network (PSTN), an infrared network, a wireless network (e.g., bluetooth, WIFI), and/or any combination of these and/or other networks.

The server 120 may include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture involving virtualization (e.g., one or more flexible pools of logical storage that may be virtualized to maintain virtual storage for the server). In various embodiments, the server 120 may run one or more services or software applications that provide the functionality described below.

The computing units in server 120 may run one or more operating systems including any of the operating systems described above, as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and/or middle tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

In some implementations, the server 120 may include one or more applications to analyze and consolidate data feeds and/or event updates received from users of the

client devices

101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of

client devices

101, 102, 103, 104, 105, and 106.

In some embodiments, the server 120 may be a server of a distributed system, or a server incorporating a blockchain. The server 120 may also be a cloud server, or a smart cloud computing server or a smart cloud host with artificial intelligence technology. The cloud Server is a host product in a cloud computing service system, and is used for solving the defects of high management difficulty and weak service expansibility in the traditional physical host and Virtual Private Server (VPS) service.

The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The data store 130 may reside in various locations. For example, the data store used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The data store 130 may be of different types. In certain embodiments, the data store used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to the command.

In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key-value stores, object stores, or regular stores supported by a file system.

The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.

Fig. 2 is a flowchart illustrating a method 200 of extracting a speech feature according to an exemplary embodiment of the present disclosure, and as shown in fig. 2, the method 200 may include the steps of:

step S201, obtaining a voice to be processed, and framing the voice to obtain multi-frame voice data;

step S203, extracting the characteristics of the multi-frame voice data to obtain a first characteristic vector of the voice, wherein the first characteristic vector comprises a plurality of elements respectively corresponding to the corresponding characteristics of the multi-frame voice data;

step S205, acquiring covariance matrixes associated with a plurality of elements in the first feature vector;

step S207, acquiring a second feature vector of the voice based on the first feature vector and the covariance matrix;

step S209, obtaining a parameter vector representing the importance of a plurality of elements in the first feature vector; and

step S211, the second feature vector and the parameter vector are spliced to obtain the voice feature of the voice.

According to the method for extracting the speech feature of the present disclosure, in order to more accurately extract the feature of the speech data of the speaker, on the basis of generating the first feature vector of the speech of the speaker, a posterior mean vector of the initial feature vector according to gaussian distribution can be obtained by using a covariance matrix capable of representing the correlation between different feature dimensions of the first feature vector (for example, corresponding feature dimensions related to physiological characteristics of the mouth, vocal cords, etc. of the speaker itself, corresponding feature dimensions related to the environment and background during speaking, etc.). The posterior mean vector can contain richer speech feature information so that the speech features of the speaker can be extracted as much and sufficiently as possible. Meanwhile, on the basis of the first feature vector, a parameter vector can be obtained to represent the importance of each frame data in the multi-frame voice data. Therefore, the speech feature obtained by splicing the posterior mean vector and the parameter vector is an updated feature vector on the basis of the first feature vector, and can be used for accurately extracting the feature of the speech of the speaker, thereby improving the accuracy of speech feature extraction.

The steps of the method 200 for extracting speech features according to the present disclosure will be described in detail below.

In step S201, the speech to be processed may be the speech of an arbitrary speaker. After the voice data of the speaker is obtained, the voice data can be preprocessed, including operations of removing environmental noise, busy tone, color ring tone and the like. And then, performing framing operation on the preprocessed voice data according to a certain time length to obtain multi-frame voice data.

According to some embodiments, in step S203, a plurality of frames of voice data may be input to the first neural network to perform feature extraction by the first neural network.

The first neural network may be any known type of neural network. In some embodiments, the first neural network may be formed of a base network and a fully connected layer, wherein the base network is a 5-layer time-delay neural network layer (TDNN).

Because the neural network can be fitted with any distribution, the distribution required to be met does not need to be preset, and a corresponding calculation result is obtained by depending on the learning capability of the neural network. By means of multiple mappings of the neural network on the feature data, the accuracy of calculating the initial feature vector can be improved.

According to some embodiments, in step S205, the first feature vector may be input to a second neural network to calculate a covariance matrix by the second neural network.

The second neural network may also be any known type of neural network. Preferably, the second neural network is a two-layer feedforward neural network, wherein the size of the first layer is 1500 × 256, and the activation function of the first layer is a RELU activation function; the size of the second layer is 256 × 1500, and the activation function of the second layer is a softplus activation function for logarithmic estimation. In this way, the second neural network takes the first feature vector as input, while auxiliarily outputting the covariance matrix in preparation for a subsequent application of the normalized exponential function (Softmax). Moreover, by means of logarithmic estimation, multiplication calculation can be converted into addition calculation, so that the calculation amount is reduced.

Accordingly, the covariance matrix may have second elements corresponding to the plurality of first elements of the first eigenvector. Since covariance can be used to represent the correlation between data of different dimensions, each second element in the covariance matrix can characterize the correlation between different feature dimensions of the corresponding first element in the initial feature vector. Here, the different feature dimensions of the initial feature vector may refer to, for example, corresponding feature dimensions related to physiological characteristics of the mouth, vocal cords, and the like of the speaker itself, corresponding feature dimensions related to the environment and the background when speaking, and the like. For example, each second element in the covariance matrix may characterize a correlation between the vocal features on the mouth and vocal features on the vocal cords of the corresponding first element in the initial feature vector.

According to some embodiments, in step S207, the second feature vector is a posterior mean vector of the first feature vector according to a gaussian distribution.

Here, the posterior mean vector can be understood as an accurate value obtained based on actual speech feature data of the speaker. In other words, in the case where the speech feature of the speaker is assumed to conform to the gaussian distribution, the prior probability alone does not reflect how the specific form of the gaussian distribution is.

In this way, in the case where the posterior mean vector is obtained as described above, the speech feature of the speaker can be accurately expressed in a gaussian distribution, thereby facilitating accurate extraction of the speech feature of the speaker.

According to some embodiments, a normalized exponent function may be applied to the covariance matrix to obtain a normalized exponent value for the covariance matrix; and multiplying the normalized index value with the first feature vector to obtain a second feature vector.

After applying the normalization index function to the covariance matrix, the obtained normalization index value is multiplied by the first eigenvector, that is, weighted average is performed, so that a posterior mean vector of the first eigenvector according to gaussian distribution can be obtained.

In one example, to obtain the a posteriori mean vector, a priori mean and covariance may also be initialized (e.g., to 0) to add to the weighted average calculation as above.

In this way, since Softmax is similar to the normalization operation, after applying Softmax to the covariance matrix, the obtained Softmax value is multiplied by the first eigenvector, that is, weighted average is performed, and thus a posterior mean vector of the first eigenvector according to a gaussian distribution can be obtained.

According to some embodiments, in step S209, the first feature vector may be input into a third neural network, wherein the third neural network comprises a first sub-neural network and a second sub-neural network, and wherein the first feature vector is input into the first sub-neural network to obtain a normalized weight vector of the first feature vector; and inputting the normalized weight vector and the first feature vector into a second sub-neural network to calculate a parameter vector.

The first and second sub-neural networks may also be any known type of neural network.

Obtaining a normalized weight vector of the first feature vector through the first sub-neural network, wherein the weight vector can be characterized as a score of the importance of each element in the first feature vector, and obtaining a parameter vector by performing operation in the second sub-neural network by using the normalized weight vector, wherein the parameter vector can evaluate the importance of each element in the first feature vector.

According to some embodiments, an initial weight vector of the first feature vector may be obtained by the first sub-neural network; and applying a normalized exponential function to the initial weight vector to obtain a normalized weight vector.

By normalizing the exponential function, each element in the weight vector can be ranged between (0,1), and the sum of all the elements is 1, and the importance of each frame data in the multi-frame voice data is represented in a probability manner.

According to some embodiments, the parameter vector may comprise: at least one of a weighted mean vector and a weighted standard deviation vector.

By obtaining the normalized weight vector and further obtaining the weighted mean vector and the weighted standard deviation vector, the importance of each frame data in the multi-frame voice data can be represented by using the weighted mean vector and the weighted standard deviation vector. By utilizing the two parameter vectors, the capability of the model for selecting frames can be improved, so that the importance of the frames selected by the model is higher.

According to some embodiments, in step S211, the second feature vector has a plurality of dimensions and the parameter vector has a plurality of dimensions, and wherein splicing the second feature vector with the parameter vector to obtain the speech feature of the speech comprises: and generating the voice feature with a target dimension, wherein the target dimension is the sum of the dimensions of the second feature vector and the parameter vector.

According to some examples, the weighted mean vector and the weighted standard deviation vector may have 512 to 1500 dimensions, respectively, and the second feature vector may have 300 to 512 dimensions.

In one example, the second feature vector may have 300 dimensions, the weighted mean vector and the weighted standard deviation vector may have 512 dimensions, respectively, and the speech feature obtained by concatenating the three may have 1324 dimensions.

The parameter vectors and the second feature vectors are spliced together to obtain the voice features, the voice features not only comprise components of the features of the initial voice data, but also comprise components of the importance of each frame data in the multi-frame voice data, and the features of the voice data are well represented.

As described above, according to the method for extracting speech features of the present disclosure, in order to more accurately extract features of speech data of a speaker, a posterior mean vector of an initial feature vector according to a gaussian distribution can be obtained by using a covariance matrix capable of characterizing correlations between different feature dimensions (for example, corresponding feature dimensions related to physiological characteristics such as an oral cavity and vocal cords of the speaker itself, corresponding feature dimensions related to an environment and a background when speaking, and the like) of the first feature vector on the basis of generating the first feature vector of speech of the speaker. The posterior mean vector can contain richer speech feature information so that the speech features of the speaker can be extracted as much and sufficiently as possible. Meanwhile, on the basis of the first feature vector, a parameter vector can be obtained to represent the importance of each frame data in the multi-frame voice data. Therefore, the speech feature obtained by splicing the posterior mean vector and the parameter vector is an updated feature vector on the basis of the initial feature vector, and can be used for accurately extracting the feature of the speech of the speaker, thereby improving the accuracy of speech feature extraction.

There is also provided, in accordance with an embodiment of the present disclosure, a method for training a speech feature extraction model.

FIG. 3 shows a flow diagram of a method 300 for training a speech feature extraction model according to an embodiment of the present disclosure. As shown in fig. 3, the method 300 may include the steps of:

step S301, obtaining sample voice labeled by a sample, and framing the sample voice to obtain multi-frame sample voice data;

step S303, extracting the characteristics of the multi-frame sample voice data to obtain a first sample characteristic vector of the sample voice, wherein the first sample characteristic vector comprises a plurality of sample elements which respectively correspond to the corresponding characteristics of the multi-frame sample voice data;

step S305, obtaining a sample covariance matrix associated with a plurality of sample elements in a first sample feature vector;

step S307, acquiring a second sample feature vector of the sample voice based on the first sample feature vector and the sample covariance matrix;

step S309, obtaining a sample parameter vector representing the importance of a plurality of sample elements in the first feature vector of the sample;

step S311, splicing the second sample feature vector and the sample parameter vector to obtain the voice feature of the sample voice; and

step 313, obtaining network parameters for updating the speech feature extraction model based on the speech features to train the speech feature extraction model.

It is noted that the operations of steps S303 to S311 in the method 300 for training the speech feature extraction model are similar to the operations of steps S203 to S211 in the speech feature extraction method 200 described in conjunction with fig. 2, and are all used for extracting the speech feature of the speaker.

In contrast, in the method 300 for training a speech feature extraction model, the data for training in step S303 is for a predetermined speaker, i.e., sample speech data, so that model training can be performed for a specific speaker. Accordingly, speaker tagging needs to be performed on audio data so that it can be determined for which particular speaker the audio data is intended. And when the sample data is not enough, the data can be enhanced in time domain by aliasing echo, changing the speed and the like, so as to increase the sample data.

In addition, the method 300 for training the speech feature extraction model further includes updating network parameters of the speech feature extraction model in step S313. In other words, the network parameters are updated by using the speech features extracted in step S311, so that multiple rounds of training can be repeated iteratively until the network converges, thereby completing the training of the model. Accordingly, the speech features obtained by splicing in step S311 can be input to, for example, two fully-connected layers (with ReLU activation functions) and further to the output layer, the loss is calculated by CE (cross entropy) criterion, and finally the network parameters are updated inversely according to SGD (random gradient descent), so that the model training is completed by iterating through multiple rounds until the network converges.

FIG. 4 shows a schematic diagram of a speech feature extraction model 400 according to an embodiment of the present disclosure.

The method for extracting speech features and the method for training the speech feature extraction model according to the present disclosure will be further described below with reference to fig. 2 to 4.

As shown in fig. 4, the speech feature extraction model 400 may include an encoder 410, a gaussian a posteriori inference module 420, a third neural network 430, and a decoder 440.

The encoder 410 may include a first neural network (consisting of the TDNN layer 411, the fully-connected layer 412) and a second neural network 413. FIG. 4 schematically shows that the second neural network 413 comprises a neural network layer 413-1 and a neural network layer 413-2. However, the number of layers of the second neural network 413 may be appropriately selected according to the actual situation. The TDNN layer 411 may also be a neural network including multiple layers, which may be selected as appropriate according to the actual situation.

The TDNN layer 411 and the fully-connected layer 412 together may correspond to a first neural network described in connection with fig. 2, and the second neural network 413 may correspond to a second neural network described in connection with fig. 2. As shown in FIG. 4, a plurality of frames of speech data X extracted from audio data about a speaker, which may include data X having a plurality of frames of temporal context, may be input to a speech extraction model 400₁,x₂,…,x_TThe plurality of frame data x₁,x₂,…,x_TMay correspond to T frames after the audio data is framed.

It is noted that in the training phase of the speech feature extraction model 400, the multi-frame speech data X is the sample initial speech feature data corresponding to a specific speaker because the audio data is labeled with the speaker. After the training of the speech extraction model 400 is completed, the audio data may be about any speaker when the speech extraction model 400 is used for speech feature extraction, and thus the multi-frame speech data X is also about any speaker.

As shown in fig. 4, multi-frame voice data X ═ { X ═ X₁,x₂,…,x_TAfter passing through the TDNN layer 411 and the fully-connected layer 412 (the first neural network), a first feature vector z is obtained₁,z₂,…,z_T}. Furthermore, the first feature vector { z }₁,z₂,…,z_TAfter passing through the second neural network 413, the corresponding first feature vector z is obtained₁,z₂,…,z_TCovariance matrix log { L }₁,L₂,…,L_T(As mentioned above, the computation of covariance matrix is done logarithmically for ease of computation).

First feature vector { z₁,z₂,…,z_TEach first element z in₁,z₂,…,z_TTo contain feature information in different feature dimensions. For example, z₁,z₂,…,z_TEach (e.g., in z)₁For example) may contain feature information in a corresponding feature dimension, e.g., related to a physiological characteristic of the speaker's mouth, vocal cords, etc. Thus, the corresponding second element { L } in the covariance matrix₁,L₂,…,L_T} (e.g. correspondingly with L₁For example) may characterize a correlation between vocal features on the mouth and vocal features on the vocal cords, for example.

The operation of the gaussian a posteriori inference module 420 may correspond to generating a second feature vector of the speaker, shown as + s in fig. 4, as described above in connection with fig. 2 and 3. In the Gaussian A posteriori inference module 420, the covariance matrix log { L }₁,L₂,…,L_TSoftmax is performed, followed by a first eigenvector { z }₁,z₂,…,z_TMultiplied to obtain a second eigenvector. By covariance matrix log { L }₁,L₂,…,L_TThe Softmax value of { with the first eigenvector { z }₁,z₂,…,z_TMultiplication, i.e. a weighted averaging operation, is performed, so that a second eigenvector phi is obtained_sThat is, the first feature vector is based on the posterior mean vector of the Gaussian distribution, which is a more accurate representation of the speaker's speech features in the Gaussian distribution.

In one example, the prior mean and covariance (μ) may also be used_p，L_p) Initialized to 0 to add to the weighted average calculation above, i.e., L0-0 and z 0-0.

Meanwhile, in multi-frame voice data X ═ { X ═ X₁,x₂,…,x_TIncludes a plurality of frame speech data { x) having a temporal context relationship₁,x₂,…,x_TIn the case of { z }, the above process of obtaining the second eigenvector φ s can represent the first eigenvector { z }₁,z₂,…,z_TA plurality of first elements z having a temporal context in₁,z₂,…,z_TThe polymerization is carried out in time and can therefore also be referred to as time polymerization.

Note that, in the training phase of the speech extraction model 400, since the input multi-frame speech data X is sample initial speech feature data about a specific speaker, accordingly, the obtained first feature vector { z } is₁,z₂,…,z_TThe covariance matrix log { L }₁,L₂,…,L_TAnd a second eigenvector phi_sAlso a first sample feature vector, a sample covariance matrix, and a second sample feature vector, respectively, for the particular speaker.

The operation of the third neural network 430 may correspond to obtaining a parameter vector characterizing the importance of a plurality of elements in the first feature vector as described above in connection with fig. 2 and 3, such as the weighted mean vector shown in fig. 4

Sum weighted standard deviation vector

The third neural network 430 comprisesA first sub-neural network 431 and a second sub-neural network 432. The initial weight vector α of the first feature vector may be obtained by the first sub-neural network 431_tThe initial weight vector α_tAnd a first feature vector z₁,z₂,…,z_TIs input to a second sub-neural network 432, a weighted mean vector can be obtained

Sum weighted standard deviation vector

Concatenating the second eigenvector φ s, the weighted mean vector

Sum weighted standard deviation vector

Speech features may be obtained.

The decoder 440 may include an embedding layer 441, a fully-connected layer 442, and an output layer 443. The embedding layer 441 may map speech features to generate embedded speech features for the speaker.

Note that during the training phase of the speech extraction model 400, the fully-connected layer 442 and the output layer 443 may be further included. The speech features extracted at the embedding layer 441 can be input to the fully-connected layer 442 and further to the output layer 443 (for speaker classification), the loss calculated by CE (cross-entropy) criterion, and finally the network parameters updated inversely according to SGD (random gradient descent), thus iterating through many rounds to network convergence. In the testing phase, speech features may be extracted at the embedding layer 441, and pairwise similarities may then be calculated by PLDA scoring.

The method for extracting speech features and the method for training the speech feature extraction model of the present disclosure are described above with reference to fig. 2 to 4. In order that this disclosure may be more fully understood, a brief description of the model assumptions on which the principles of this disclosure are based follows.

First, the assumption of a prior distribution of speaker characteristics is made:

1) model: z is a radical of_t＝h+∈_t

2) Latent variables:

3) uncertainty:

wherein z is_tGenerating variables for the model, representing feature vectors of the speaker; h is a hidden variable representing the basic vector of the speaker; e is the same as_tThe residual variable represents the difference vector of different speakers; μ, L are mean and covariance, respectively, in a Gaussian distribution, and z_tH and e_tObey a gaussian distribution.

The basis vector h may reflect the average of the population over several feature attributes, such as several feature dimensions related to the mouth, vocal cords, tongue, lips, etc. Residual variable ∈_tA perturbation vector that may reflect changes in some of these characteristic properties (e.g., the frequency of vibration of the vocal cords). Based on the model, the characteristics of any speaker can be characterized.

Since the basis vector h obeys a gaussian distribution, the posterior probability of the basis vector h can be calculated for a specific population (e.g., 1000 asians), thereby reflecting the specific shape of the population with respect to the gaussian distribution of the basis vector h. That is, the posterior mean vector of the first feature vector according to the gaussian distribution is obtained in the present disclosure.

Given input data about a particular population, the posterior probability distribution of basis vector h can be derived as:

wherein:

here, it can be obtained through a neural network

And

wherein, fenc () and genc () may correspond to the first neural network and the second neural network, respectively, as described above.

Computing the fraction e of each element { z1, z2, …, zT } in the encoder_t：

e_t＝v^Tf(Wz_t+ b) + k (formula 4)

Where f () is the nonlinear activation function tanh, vT and k are the weight and offset of the linear layer, respectively, and W and b are the weight and offset in the nonlinear layer activation function.

E is to be_tNormalized over all frames by the softmax function:

computing a weighted mean vector

Computing a weighted standard deviation vector

Wherein equations 4 and 5 correspond to the first sub-neural network as described above, and equations 6 and 7 correspond to the second sub-neural network as described above.

In addition, the training process of the speech feature extraction model can also be represented by the following pseudo code:

where θ may represent network parameters to be updated, such as weights and offsets. θ may be calculated by multiplying a parameter g by a learning force, wherein the parameter g may reflect an amount of change or update learned from a batch of data (batch) at a time, and the learning force may represent a magnitude of the change. After the parameter g is obtained, it may be added to θ (e.g., weights and offsets) to produce corresponding changes, thereby implementing the training process.

According to another aspect of the present disclosure, a voiceprint feature extraction apparatus is also provided. Fig. 5 shows a block diagram of an apparatus 500 for extracting speech features according to an embodiment of the present disclosure.

As shown in fig. 5, the apparatus 500 may include: a framing module 501 configured to: acquiring voice to be processed, and framing the voice to obtain multi-frame voice data; a first obtaining module 502 configured to: extracting the features of the multi-frame voice data to obtain a first feature vector of the voice, wherein the first feature vector comprises a plurality of elements which respectively correspond to the corresponding features of the multi-frame voice data; a second obtaining module 503 configured to: obtaining a covariance matrix associated with a plurality of elements in the first eigenvector; a third obtaining module 504 configured to: acquiring a second feature vector of the voice based on the first feature vector and the covariance matrix; a fourth obtaining module 505 configured to: acquiring a parameter vector representing the importance of a plurality of elements in the first feature vector; and a stitching module 506 configured to: and splicing the second feature vector with the parameter vector to obtain the voice feature.

According to some embodiments, the multi-frame speech data is extracted from audio data about the speaker, and the multi-frame speech data includes data of a plurality of frames having a temporal context, wherein the data of the plurality of frames corresponds to the plurality of frames of the audio data.

The operations performed by the modules 501 to 506 correspond to the steps S201 to S211 described with reference to fig. 2, and therefore, details of various aspects thereof are not repeated.

Fig. 6 shows a block diagram of a voiceprint feature extraction apparatus 600 according to another embodiment of the present disclosure. Modules 601 to 606 shown in fig. 6 may correspond to modules 501 to 506 shown in fig. 5, respectively. The modules 602 to 606 may include, among other things, further sub-functional modules, as will be explained in more detail below.

According to some embodiments, the first obtaining module comprises 602: a first sub-module 6021 configured to: inputting the plurality of frames of voice data into a first neural network to perform feature extraction through the first neural network.

According to some embodiments, the second obtaining module 603 comprises: a second sub-module 6031 configured to: the first eigenvector is input into a second neural network to compute a covariance matrix through the second neural network.

According to some embodiments, the third obtaining module 604 comprises: a third sub-module 6041 configured to: applying a normalization index function to the covariance matrix to obtain a normalization index value for the covariance matrix; and a fourth sub-module 6042 configured to: the normalized index value is multiplied with the first feature vector to obtain a second feature vector.

According to some embodiments, the fourth obtaining module 605 includes: a fifth submodule 6051 configured to: inputting the first feature vector into a third neural network, wherein the third neural network comprises a first sub-neural network and a second sub-neural network, and wherein a fifth sub-module comprises: a sixth submodule configured to: inputting the first feature vector into a first sub-neural network to obtain a normalized weight vector of the first feature vector; and a seventh sub-module configured to: the normalized weight vector and the first feature vector are input to a second sub-neural network to calculate a parameter vector.

According to some embodiments, the parameter vector comprises: at least one of a weighted mean vector and a weighted standard deviation vector

According to some embodiments, the sixth submodule comprises: an eighth submodule configured to: acquiring an initial weight vector of the first feature vector through a first sub-neural network; and a ninth sub-module configured to: a normalized exponential function is applied to the initial weight vector to obtain a normalized weight vector.

According to some embodiments, the second feature vector has a plurality of dimensions and the parameter vector has a plurality of dimensions, and wherein the stitching module 606 comprises: a tenth submodule 6061 configured to: and generating the voice feature with a target dimension, wherein the target dimension is the sum of the dimensions of the second feature vector and the parameter vector.

According to another aspect of the present disclosure, there is also provided an apparatus for training a speech feature extraction model. FIG. 7 shows a block diagram of an apparatus 700 for training a speech feature extraction model according to an embodiment of the present disclosure.

As shown in fig. 7, the apparatus 700 may include: a sample framing module 701 configured to: obtaining sample voice labeled by a sample, and framing the sample voice to obtain multi-frame sample voice data; a first sample acquisition module 702 configured to: performing feature extraction on the multi-frame sample voice data to obtain a first sample feature vector of the sample voice, wherein the first sample feature vector comprises a plurality of sample elements respectively corresponding to corresponding features of the multi-frame sample voice data; a second sample acquisition module 703 configured to: obtaining a sample covariance matrix associated with the plurality of sample elements in the first sample feature vector; a third sample acquisition module 704 configured to: obtaining a second sample feature vector of the sample voice based on the first sample feature vector and the sample covariance matrix; a fourth sample acquisition module 705 configured to: obtaining a sample parameter vector characterizing the importance of the plurality of sample elements in the sample first feature vector; a sample stitching module 706 configured to: splicing the second sample feature vector with the sample parameter vector to obtain the voice feature of the sample voice; and a parameter adjustment module 707 configured to: and acquiring network parameters for updating a voice feature extraction model based on the voice features so as to train the voice feature extraction model.

According to another aspect of the present disclosure, there is also provided an electronic device comprising at least one processor; and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor, which when executed by the at least one processor, cause the at least one processor to perform the method as described above.

According to another aspect of the present disclosure, there is also provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method as described above.

According to another aspect of the present disclosure, there is also provided a computer program product comprising a computer program, wherein the computer program realizes the method as described above when executed by a processor.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.

Referring to fig. 8, a block diagram of a structure of an electronic device 800, which may be a server or a client of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. Electronic device is intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, an output unit 807, a storage unit 808, and a communication unit 809. The input unit 806 may be any type of device capable of inputting information to the device 800, and the input unit 806 may receive input numeric or character information and generate key signal inputs related to user settings and/or function controls of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and/or a remote control. Output unit 807 can be any type of device capable of presenting information and can include, but is not limited to, a display, speakers, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 808 may include, but is not limited to, a magnetic disk, an optical disk. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and/or chipsets, such as bluetooth (TM) devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and/or the like.

Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The computing unit 801 performs the various methods and processes described above, such as the method 200 and the method 300. For example, in some embodiments, the

methods

200 and 300 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When loaded into RAM 803 and executed by computing unit 801, may perform one or more of the steps of method 200 and method 300 described above. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the method 200 and the method 300 by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be performed in parallel, sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the above-described methods, systems and apparatus are merely exemplary embodiments or examples and that the scope of the present invention is not limited by these embodiments or examples, but only by the claims as issued and their equivalents. Various elements in the embodiments or examples may be omitted or may be replaced with equivalents thereof. Further, the steps may be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced with equivalent elements that appear after the present disclosure.

Claims

1. A method of extracting speech features, comprising:

acquiring voice to be processed, and framing the voice to obtain multi-frame voice data;

extracting features of the multi-frame voice data to obtain a first feature vector of the voice, wherein the first feature vector comprises a plurality of elements which respectively correspond to the corresponding features of the multi-frame voice data;

obtaining a covariance matrix associated with the plurality of elements in the first feature vector;

acquiring a second feature vector of the voice based on the first feature vector and the covariance matrix;

obtaining a parameter vector representing the importance of the plurality of elements in the first feature vector; and

and splicing the second feature vector and the parameter vector to obtain the voice feature of the voice.

2. The method of claim 1, wherein the second eigenvector is a posterior mean vector of the first eigenvector according to a gaussian distribution.

3. The method of claim 1 or 2, wherein obtaining a second feature vector of the speech based on the first feature vector and the covariance matrix comprises:

applying a normalized exponent function to the covariance matrix to obtain a normalized exponent value for the covariance matrix; and

multiplying the normalized index value with the first feature vector to obtain the second feature vector.

4. The method of any of claims 1 to 3, wherein obtaining a parameter vector characterizing the importance of the plurality of elements in the first feature vector comprises:

inputting the first feature vector into a third neural network, wherein the third neural network comprises a first sub-neural network and a second sub-neural network, and wherein,

inputting the first feature vector into the first sub-neural network to obtain a normalized weight vector of the first feature vector; and

inputting the normalized weight vector and the first feature vector into the second sub-neural network to calculate the parameter vector.

5. The method of claim 4, wherein inputting the first feature vector into the first sub-neural network to obtain a normalized weight vector for the first feature vector comprises:

acquiring an initial weight vector of the first feature vector through the first sub-neural network; and

applying a normalized exponential function to the initial weight vector to obtain the normalized weight vector.

6. The method of any of claims 1-5, wherein the parameter vector comprises: at least one of a weighted mean vector and a weighted standard deviation vector.

7. The method of any of claims 1-6, wherein the second feature vector has a plurality of dimensions and the parameter vector has a plurality of dimensions, and wherein stitching the second feature vector with the parameter vector to obtain speech features of the speech comprises:

generating a speech feature having a target dimension, wherein the target dimension is a sum of dimensions of the second feature vector and the parameter vector.

8. A method for training a speech feature extraction model, comprising:

obtaining sample voice labeled by a sample, and framing the sample voice to obtain multi-frame sample voice data;

performing feature extraction on the multi-frame sample voice data to obtain a first sample feature vector of the sample voice, wherein the first sample feature vector comprises a plurality of sample elements respectively corresponding to corresponding features of the multi-frame sample voice data;

obtaining a sample covariance matrix associated with the plurality of sample elements in the first sample feature vector;

obtaining a second sample feature vector of the sample voice based on the first sample feature vector and the sample covariance matrix;

obtaining a sample parameter vector characterizing the importance of the plurality of sample elements in the sample first feature vector;

splicing the second sample feature vector with the sample parameter vector to obtain the voice feature; and

and acquiring network parameters for updating a voice feature extraction model based on the voice features so as to train the voice feature extraction model.

9. An apparatus for extracting speech features, comprising:

a framing module configured to: acquiring voice to be processed, and framing the voice to obtain multi-frame voice data;

a first acquisition module configured to: extracting features of the multi-frame voice data to obtain a first feature vector of the voice, wherein the first feature vector comprises a plurality of elements which respectively correspond to the corresponding features of the multi-frame voice data;

a second acquisition module configured to: obtaining a covariance matrix associated with the plurality of elements in the first feature vector;

a third acquisition module configured to: acquiring a second feature vector of the voice based on the first feature vector and the covariance matrix;

a fourth acquisition module configured to: obtaining a parameter vector representing the importance of the plurality of elements in the first feature vector; and

a stitching module configured to: and splicing the second feature vector and the parameter vector to obtain the voice feature of the voice.

10. The apparatus of claim 9, wherein the second eigenvector is a posterior mean vector of the first eigenvector according to a gaussian distribution.

11. The apparatus of claim 9 or 10, wherein the third obtaining means comprises:

a third sub-module configured to: applying a normalized exponent function to the covariance matrix to obtain a normalized exponent value for the covariance matrix; and

a fourth submodule configured to: multiplying the normalized index value with the first feature vector to obtain the second feature vector.

12. The apparatus of any of claims 9 to 11, wherein the fourth acquisition module comprises:

a fifth submodule configured to: inputting the first feature vector into a third neural network, wherein the third neural network comprises a first sub-neural network and a second sub-neural network, and wherein the fifth sub-module comprises:

a sixth submodule configured to: inputting the first feature vector into the first sub-neural network to obtain a normalized weight vector of the first feature vector; and

a seventh sub-module configured to: inputting the normalized weight vector and the first feature vector into the second sub-neural network to calculate the parameter vector.

13. The apparatus of claim 12, wherein the sixth submodule comprises:

an eighth submodule configured to: acquiring an initial weight vector of the first feature vector through the first sub-neural network; and

a ninth sub-module configured to: applying a normalized exponential function to the initial weight vector to obtain the normalized weight vector.

14. The apparatus of claim 13, wherein the parameter vector comprises: at least one of a weighted mean vector and a weighted standard deviation vector.

15. The apparatus of any of claims 9 to 14, wherein the second feature vector has a plurality of dimensions and the parameter vector has a plurality of dimensions, and wherein the stitching module comprises:

a tenth sub-module configured to: generating a speech feature having a target dimension, wherein the target dimension is a sum of dimensions of the second feature vector and the parameter vector.

16. An apparatus for training a speech feature extraction model, comprising:

a sample framing module configured to: obtaining sample voice labeled by a sample, and framing the sample voice to obtain multi-frame sample voice data;

a first sample acquisition module configured to: performing feature extraction on the multi-frame sample voice data to obtain a first sample feature vector of the sample voice, wherein the first sample feature vector comprises a plurality of sample elements respectively corresponding to corresponding features of the multi-frame sample voice data;

a second sample acquisition module configured to: obtaining a sample covariance matrix associated with the plurality of sample elements in the first sample feature vector;

a third sample acquisition module configured to: obtaining a second sample feature vector of the sample voice based on the first sample feature vector and the sample covariance matrix;

a fourth sample acquisition module configured to: obtaining a sample parameter vector characterizing the importance of the plurality of sample elements in the sample first feature vector;

a sample stitching module configured to: splicing the second sample feature vector with the sample parameter vector to obtain the voice feature of the sample voice; and

a parameter adjustment module configured to: and acquiring network parameters for updating a voice feature extraction model based on the voice features so as to train the voice feature extraction model.

17. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-8.

19. A computer program product comprising a computer program, wherein the computer program realizes the method of any one of claims 1-8 when executed by a processor.