KR102055886B1 - Speaker voice feature extraction method, apparatus and recording medium therefor - Google Patents

Abstract

The present invention relates to a method and apparatus for extracting a feature of a speaker voice, and a recording medium therefor, wherein the method includes a mediation between a Baum-Welch statistical value extracted from an input voice and a distribution of features of a frame unit included in the input voice. Assuming that there is a random concealment variable, a measure of uncertainty and reliability of the input speech signal is extracted to recognize the speaker even for a short or degraded speech signal.

Description

SPEAKER VOICE FEATURE EXTRACTION METHOD, APPARATUS AND RECORDING MEDIUM THEREFOR}

The present invention relates to a method and apparatus for extracting features of a speaker's voice, and a recording medium therefor, and more particularly to extracting a measure of uncertainty and reliability of an input voice signal, and to a speaker for a short or deteriorated voice signal. The present invention relates to a method and apparatus for extracting a feature to recognize the information, and a recording medium therefor.

The contents described in this section merely provide background information on the present embodiment and do not constitute a prior art.

The i-vector is a vector representing the variation of the Gaussian Mixture Model (GMM) supervector created by connecting the Gaussian mean values when the distribution of data is modeled by the Gaussian Mixture Model (GMM). The feature vector proposed in the recognition. Due to the advantage of being able to express various variability existing in the input signal as a small dimension vector, it is used not only for speaker recognition but also for other fields such as video. In this case, the relationship between the Gaussian Mixture Model (GMM) supervector and the eye vector may be expressed as follows.

,

,

와

은 각각 입력에 종속적인 GMM supervector, UBM(universal background model), 전체 변이성 행렬(total variability matrix), 그리고 입력

에 종속적인 아이벡터를 의미한다. From here

,

,

Wow

Are input-dependent GMM supervectors, universal background model (UBM), total variability matrix, and input

The child vector that depends on

The UBM is a GMM trained using several kinds of signals and represents a distribution of speech independent of a specific signal.

를 입력 받으면 데이터

를 생성한다. 두 번째 네트워크는 인코더로, 데이터

를 입력 받으면 이에 해당하는 은닉 변수(latent variable)를 추정한다. 마지막 네트워크인 디스크리미네이터(discriminator)는 인코더 혹은 디코더로부터 은닉 변수과 데이터의 쌍

을 입력으로 받아 인코더로부터 생성되었는지 디코더로부터 생성되었는지를 판단한다. 인코더 및 디코더와 디스크리미네이터를 경쟁하듯 학습함으로써 인코더와 디코더는 각각 실제 분포에서 생성 된 듯한 은닉 변수(latent variable)와 데이터를 출력하도록 학습된다. ALI is a kind of generative model for data generation and consists of three networks. The first network is a decoder, random hidden variable

Data when you enter

Create The second network is an encoder

When the input is received, a corresponding latent variable is estimated. The final network, the discriminator, is a pair of hidden variables and data from an encoder or decoder.

Is inputted to determine whether it is generated from an encoder or a decoder. By competing the encoder and the decoder and the delimiter, the encoder and the decoder are trained to output hidden variables and data that appear to have been generated from the actual distribution, respectively.

In the speaker recognition using the eye vector, only speaker information can be represented in a linear manner, and even in the case of the existing deep learning-based feature extraction technique, even when a random variable is included, a similar distribution to the preset Gaussian distribution is obtained. There was a problem that the loss of the original information is large, because the constraint through the equation to have.

Korea Patent Registration No. 10-1805976

The present invention has been proposed to solve the above-mentioned conventional problems, and has a short or deteriorated voice length regardless of the length of the speech through a delimiter which determines without using a separate mathematical constraint to follow the distribution derived from the real Gaussian distribution. It aims to recognize the speaker by expressing it as a small fixed vector.

In particular, the present invention assumes that there is a hidden random variable that mediates the distribution of Baum-Welch statistical values extracted from the input speech and the distribution of the features of the frame unit included in the input speech. It is an object of the present invention to provide a speaker recognition method and apparatus capable of recognizing a speaker and to recognize a speaker even when a short or degraded signal is given, and a recording medium therefor.

However, the object of the present invention is not limited to the above object, and other objects not mentioned will be clearly understood from the following description.

The present invention provides a speaker speech feature extraction method for extracting features of the speaker speech signal by performing Baum-Welch statistics on a speaker speech signal under a UBM (Universal Background Model) condition.

In the speaker speech feature extraction method according to the present invention, performing Baum-Welch statistics on a speech signal under UBM conditions, and extracting a distribution of a random hidden variable representing a distribution characteristic of the input speech dependent on the input speech signal. ; Calculating a first GMM supervector according to a maximum a posteriori based on Baum-Welch statistical values performed under UBM conditions for the speaker speech signal; Extracting a sampling value for a prior probability distribution of the random hidden variable; Calculating a second GMM supervector using a sampling value for a prior probability distribution of the random concealment variable; The input is sampled by inputting any one of a pair of a distribution of the random hidden variable and a pair of first GMM supervectors, and a pair of sampling values for a prior probability distribution of the random hidden variable and a pair of second GMM supervectors as a delimiter. Classifying whether it is generated from a value or from an input voice; It includes.

In addition, the present invention, as a means for solving the above-described problem, performs a Baum-Welch statistics on the speaker voice signal under UBM (Universal Background Model) conditions, and randomly showing the distribution characteristics of the input voice dependent on the input voice signal An encoder for extracting a distribution of random latent variables; A GMM generator configured to generate a first Gaussian Mixture Model (GMM) supervector according to a maximum a posteriori using Baum-Welch statistical values performed under UBM conditions on the speaker speech signal; A decoder configured to extract a sampling value for a prior probability distribution of the random concealment variable, and calculate a second Gaussian Mixture Model (GMM) supervector using the same; The input is generated from the sampling value by receiving any one of a pair of the random hidden variable and a pair of first GMM supervectors and a sampling value of a prior probability distribution of the random hidden variable and a pair of second GMM supervectors. It provides a speaker speech feature extraction apparatus comprising a; a delimiter for classifying whether or not it is generated from the input voice.

In addition, the makeup speech feature extraction method according to the present invention described above may be implemented as a program and recorded on a computer-readable recording medium.

According to the speaker speech feature extraction method and apparatus of the present invention, and a recording medium therefor, a hidden random variable that mediates the distribution of Baum-Welch statistical values extracted from the input speech and the features of the frame unit included in the input speech. Assuming that is present, a measure of the uncertainty and reliability of the input signal can be extracted, so that a short or degraded signal is given, thereby reducing the performance degradation.

In addition, the present invention, in addition to using a deep neural network (DNN), unlike the conventional i-vector extraction technique, in extracting the random hidden variable, there is no additional mathematical constraint Therefore, speaker information that may not be expressed in a linear projection process of the conventional i-vector may be expressed.

In addition, various effects other than the above-described effects may be directly or implicitly disclosed in the detailed description according to the embodiment of the present invention to be described later.

1 is a block diagram of an apparatus for extracting speaker speech features according to an embodiment of the present invention.
2 is a flowchart illustrating a speaker voice feature extraction process according to an embodiment of the present invention.
3 is a flowchart illustrating a process of extracting an input of a delimiter by an encoder and a GMM generator in the makeup speech feature extraction apparatus according to an embodiment of the present invention.
4 is a flowchart illustrating a process of extracting an input of a delimiter by a decoder in a makeup speech feature extraction apparatus according to an embodiment of the present invention.
5 is a view showing the effect of speaker speech feature extraction according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS To make the features and advantages of the present invention more clear, the present invention will be described in more detail with reference to specific embodiments shown in the accompanying drawings.

However, in the following description and the accompanying drawings, detailed descriptions of well-known functions or configurations that may obscure the subject matter of the present invention will be omitted. In addition, it should be noted that like elements are denoted by the same reference numerals as much as possible throughout the drawings.

The terms or words used in the following description and drawings should not be construed as being limited to the ordinary or dictionary meanings, and the inventors may appropriately define the concept of terms for explaining their own invention in the best way. It should be interpreted as meaning and concept corresponding to the technical idea of the present invention based on the principle that the present invention. Therefore, the embodiments described in the present specification and the configuration shown in the drawings are only the most preferred embodiments of the present invention, and do not represent all of the technical ideas of the present invention, and various alternatives may be substituted at the time of the present application. It should be understood that there may be equivalents and variations.

In addition, terms including ordinal numbers, such as first and second, are used to describe various components, and are used only to distinguish one component from another component, and to limit the components. Not used. For example, without departing from the scope of the present invention, the second component may be referred to as the first component, and similarly, the first component may also be referred to as the second component.

In addition, when a component is referred to as being "connected" or "connected" to another component, it means that it may be connected or connected logically or physically. In other words, although a component may be directly connected or connected to other components, it should be understood that other components may exist in the middle, and may be connected or connected indirectly.

In addition, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In addition, the terms "comprises" or "having" described herein are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or the same. It is to be understood that the present invention does not exclude in advance the possibility of the presence or the addition of other features, numbers, steps, operations, components, parts, or a combination thereof.

Hereinafter, a speaker voice feature extraction method according to an embodiment of the present invention will be described.

의 사후확률 분포로부터 샘플링 된 변수들이 은닉 변수의 사전확률로부터 샘플링 된 실제 은닉 변수와 가까운 현실적인 값을 갖도록 디스크리미네이터를 사용한다.First, the present invention is to extract the characteristics of the speaker voice using an ALI model consisting of an encoder, a decoder, a discriminator, the Baum-Welch statistical value extracted from the input voice It is assumed that there is a random latent variable that mediates between the distribution of the features of the frame unit included in the input speech, and the estimated hidden variable

The delimiter is used to ensure that the variables sampled from the post-probability distribution of P have a realistic value close to the actual hidden variables sampled from the prior probability of the hidden variable.

1 is a block diagram of an apparatus for extracting speaker speech features according to an embodiment of the present invention.

Referring to FIG. 1, the speaker speech feature extraction apparatus according to the present invention includes an encoder 100, a GMM generating unit 200, a decoder 300, and a delimiter 400.

The speaker speech feature extraction apparatus according to the present invention configured as described above performs Baum-Welch statistics on the speaker speech signal x through the encoder 100 under UBM condition in order to extract the feature of the speaker speech signal. Extract the distribution z (x) of the random hidden variable dependent on the signal x.

를 이용하여 GMM슈퍼벡터

를 추출한다.Subsequently, the decoder 300 randomly sampled from N (0,1), which is a prior probability distribution of the hidden variable.

GMM supervector using

Extract

의 입력 음성에 대한 로그 우도(log-likelihood)는 <수학식 1>을 통해 산출할 수 있다. At this time, GMM super vector

The log-likelihood of the input voice of may be calculated by Equation 1.

: 인코더의 파라미터,

: 디코더의 파라미터,

: 입력 음성에 종속적인 랜덤 은닉 변수 (random latent variable),

: GMM 분포 표현에 사용된 가우시안의 개수,

: 입력 음성 및 c번째 가우시안에 대한 1차 Baum-Welch 통계,

: 입력 음성의 프레임 단위 특징의 차원,

: UBM 분포의 c번째 가우시안의 공분산 행렬,

: 입력 음성에 포함된 프레임 수,

: 입력 음성의 l번째 프레임이 UBM의 c번째 가우시안에 포함될 확률,

: 추정한 GMM 분포의 c번째 가우시안의 평균 벡터를 의미한다) (

: Parameter of the encoder,

: The parameters of the decoder,

Random latent variable,

Is the number of Gaussians used to represent the GMM distribution,

: First order Baum-Welch statistics for input voice and c th Gaussian,

: Dimension of the frame unit feature of the input voice,

Is the covariance matrix of the c th Gaussian of the UBM distribution,

: The number of frames included in the input voice,

: Probability that the l th frame of the input speech is included in the c th Gaussian of the UBM,

Is the mean vector of the c-th Gaussian of the estimated GMM distribution.)

에 대한 로그 우도(log-likelihood)이므로 은닉 변수에 대하여 marginalize하면 아래 수학식 2와 같이 근사할 수 있다.Equation 1 is a specific hidden variable

Since it is log-likelihood for, marginalizing the hidden variable can be approximated by Equation 2 below.

는 Monte Carlo 근사에 사용한 샘플의 수이며

는 latent variable의 사전 확률 분포에서 랜덤하게 샘플링한 은닉 변수이다.)(Equation 2

Is the number of samples used to approximate Monte Carlo

Is a hidden variable sampled randomly from the prior probability distribution of the latent variable.)

In addition, in Equation 1, Baum-Welch statistics has information on the distribution pattern of the frame features present in a specific voice when UBM is given, and is defined by Equations 3 and 4 below.

와

는 각각 UBM 의 c 번째 가우시안 요소에서의 0 차 및 1 차 Baum-Welch statistics 를 의미하며,

는 X의 l번째 프레임 특징인

이 UBM의 c 번째 가우시안 요소에서 생성될 확률을, 그리고

는 UBM의 c 번째 가우시안 요소의 평균 벡터를 의미한다. 0 차 Baum-Welch statistics 는 X 의 프레임들 중 c 번째 가우시안 요소에 align 된 수를 나타내며, 1 차 Baum-Welch statistics 는 c번째 가우시안 요소에 align 된 평균 프레임을 의미한다.In Equation 3,4 above

Wow

Are the 0th and 1st Baum-Welch statistics in the c-th Gaussian element of UBM, respectively.

Is the lth frame of X

The probability that it will be generated from the c th Gaussian element of this UBM, and

Denotes the average vector of the c-th Gaussian element of UBM. The 0th order Baum-Welch statistics represent the number of frames aligned with the c-th Gaussian element among X frames, and the 1st order Baum-Welch statistics mean the average frame aligned with the c-th Gaussian element.

Next, the Gaussian Mixture Model (GMM) generation unit 200 estimates the maximum post-postulation process using a extracted z (x) from the input voice x in the encoder 100. GMM supervectors can be extracted.

In this case, the GMM supervector estimated through the maximum a posteriori process may be calculated according to Equation 5.

: 입력 음성,

:

의

번째 프레임 특징이 UBM의

번째 가우시안에 속할 확률,

: 1차 Baum-Welch 통계 값,

: 0차 Baum-Welch 통계 값,

: UBM의

번째 가우시안의 평균 값을 의미한다.)(

: Input voice,

:

of

Second frame features of UBM

Probability of belonging to the first Gaussian,

: 1st Baum-Welch statistical value,

: 0th Baum-Welch statistical value,

UBM's

Mean value of the first Gaussian.)

,

) 또는 (

,

) 중 어느 하나를 입력으로 하여,

를 출력하는 과정을 수행하고, 이 때, 상기

는 시그모이드(sigmoid)함수를 이용한, 심층신경망(Deep Neural Network;DNN) 학습을 이용하여 계산 될 수 있다.Discreminator 400, (

,

) or (

,

) As one of the inputs,

To perform the process of outputting, in this case,

Can be calculated using Deep Neural Network (DNN) learning using sigmoid function.

),

)인 경우, 1을 입력이, (

,

)인 경우, 0을 출력하는 역할을 수행 할 수 있다.In addition, the discriminator 400 has an input function of (Equation 6), and the input is ((

),

If 1, enter 1, (

,

), It can output 0.

은 인코더가 나타내는 은닉 변수와 데이터의 joint distribution을 나타내며,

는 디코더가 나타내는 joint distribution, 그리고

는 디스크리미네이터(400)의 출력 값을 의미한다.)(

Represents the joint distribution of the hidden variables and data represented by the encoder,

Is the joint distribution represented by the decoder, and

Denotes an output value of the delimiter 400.)

)의 로그 우도(log-likelihood)가 최대가 되기 위해, <수학식 7>를 목적함수로 이용할 수 있다.At this time, the delimiter 400 is (

In order to maximize the log-likelihood of), Equation 7 can be used as the objective function.

는

의 랜덤 은닉 변수(random latent variable)와 GMM 분포의 joint distribution,

는

의 랜덤 은닉 변수(random latent variable)와 GMM 분포의 joint distribution,

는 Monte Carlo 추정을 위하여 사용한 샘플의 수,

는 입력 음성이 주어진 경우 랜덤 은닉 변수 (random latent variable)의 분포로부터 샘플된 은닉 변수(latent variable)를 의미한다.here,

Is

Random latent variable and joint distribution of GMM distribution,

Is

Random latent variable and joint distribution of GMM distribution,

Is the number of samples used to estimate Monte Carlo,

Denotes a latent variable sampled from the distribution of a random latent variable given the input negative.

That is, the delimiter 400 of the present invention receives the hidden variable and the voice distribution generated from the encoder 100 or the decoder 300 and generates a random sample from a sample randomly sampled from the prior probability distribution of the hidden variable. Determine if it was created.

In other words, when the actual speech is input to the delimiter 400 encoder 100 and the estimated hidden variable and the distribution of the speech are given, 1 is input to the decoder and a random sampled variable is input from the prior probability distribution of the hidden variable. Given an estimated speech distribution, it is learned to output zero.

In the method for extracting features for existing VAE-based speaker recognition, KL divergence between prior probability distributions of the latent variables so that the probability distribution of the latent variables generated from the encoder can be similar to the Gaussian distribution. Although mathematical constraints have been necessary using the present invention, the present invention can provide stable speaker recognition performance even in short or deteriorated voices by generating variables expressing uncertainty and reliability without these limitations.

2 is a flowchart illustrating a speaker voice feature extraction process according to an embodiment of the present invention.

Referring to FIG. 2, first, a speaker's voice signal is input, and a Baum-Welch statistic is performed on the voice signal under UBM conditions to indicate a random concealment variable indicating a distribution characteristic of the input voice dependent on the input voice signal. Extract the distribution (S101).

Thereafter, the first GMM supervector according to the maximum post-processing process is calculated based on a value obtained by performing Baum-Welch statistics on the input speech signal under UBM condition (S102).

Next, a sampling value for the prior probability distribution of the random hidden variable is extracted (S103).

Thereafter, a second GMM supervector is calculated from the sampling value extracted in step S103 (S104).

Next, any one of the random hidden variable distribution extracted in step S101 and the pair of first GMM supervectors calculated in step S102 or the sampling value extracted in step S103 and the pair of second GMM supervectors calculated in step S104. By using the input as the input of the delimiter 400, a process of classifying whether the input is generated from the sampling value or the input voice is performed (S105).

In the present invention, the delimiter 400 learns that the variables sampled from the posterior probability distribution of the extracted random hidden variable have a realistic value close to the actual hidden variable sampled from the prior probability of the hidden variable.

3 and 4 are flowcharts illustrating in more detail a process for extracting an input of a delimiter according to an embodiment of the present invention.

Referring to FIG. 3, the encoder 100 obtains an average and a variance of a hidden variable by inputting values obtained by performing Baum-Welch statistics (S110) under a UBM (Universal Background Model) condition on a voice signal. It may be (S120). In this case, the variance may be the log variance of the hidden variable.

In addition, the GMM generator 200 extracts the first GMM supervector estimated according to the maximum a posteriori using the Baum-Welch statistical value (S130).

At this time, the first GMM supervector estimated through the maximum post-processing may be calculated according to Equation 5.

[Equation 5]

는 입력 음성,

는

의

번째 프레임 특징이 UBM의

번째 가우시안에 속할 확률,

는 1차 Baum-Welch 통계 값,

는 0차 Baum-Welch 통계 값,

는 UBM의

번째 가우시안의 평균 값을 의미한다.

Input voice,

Is

of

Second frame features of UBM

Probability of belonging to the first Gaussian,

Is the primary Baum-Welch statistic,

Is the 0th order Baum-Welch statistic,

UBM's

The mean value of the first Gaussian.

The reason for calculating in this way is that the speaker voice x cannot be used as it is as the input of the delimiter 400, and a GMM supervector is required.

를 추출한다(S310).Referring to Figure 4, the decoder is randomly sampled from N (0,1), which is the prior probability distribution of the hidden variable.

To extract (S310).

를 이용하여, 제2 GMM 슈퍼벡터 (

)를 추출한다(S320).After this extracted

Using the second GMM supervector (

) Is extracted (S320).

,

,

,

는 디스크리미네이터(400)의 입력으로 이용되어, 입력 신호의 불확실성 및 신뢰도에 대한 척도를 추출할 수 있어, 짧거나 열화 된 신호가 주어진 경우에도 화자를 인식할 수 있도록 한다.3, extracted through 4

,

,

,

Is used as an input of the delimiter 400 to extract a measure of uncertainty and reliability of the input signal, so that the speaker can be recognized even if a short or degraded signal is given.

5 is a view showing the effect of speaker speech feature extraction according to an embodiment of the present invention.

For performance verification, the encoder 100 and the decoder 300 were set as a single hidden layer composed of ReLU active function nodes 4096, respectively. After learning with TIMIT dataset consisting of 630 speakers and 6300 voice samples, performance was verified by TIDIGITS consisting of 326 speakers, speaker generalization by linear discriminant analysis (LDA), and probabilistic linear discriminant analysis (PLDA). . The Gaussians of UBM and GMM were set to 32 and the dimension of the hidden variable was set to 200. First, the differential entropy obtained by the log variance of the extracted hidden variables was calculated as shown in FIG. 5.

In the results of FIG. 5, VAE is a method of optimizing the KL-divergence value between the log likelihood and the posterior probability and the prior probability of the estimated hidden variable without using the delimiter 400, and the ALI does not consider the log likelihood. Method and ALI / NLL are methods that are considered in consideration of both the log likelihood and the delimiter 400 proposed in the present invention. As can be seen from the graph above, as the length of speech becomes longer, the differential entropy calculated by the logarithmic distribution of the hidden variables decreases, and the difference is much more pronounced when the proposed method is used in the present invention. . This means that the feature extracted by the proposed feature extraction technique effectively contains information about the uncertainty according to the voice length.

Furthermore, the speaker recognition performance when using the existing eye vector feature and the hidden variable feature generated by the feature extraction model proposed by the present invention is shown in Table 1 below.

LM LV LM + LV i-vector + LM i-vector + LV i-vector + LM + LV i-vector (200) 3.36 i-vector (400) 2.68 i-vector (600) 2.17 VAE 3.61 4.65 2.03 1.78 1.65 0.97 ALI 4.39 4.56 2.32 1.59 1.55 1.03 ALI / NLL 3.64 3.46 1.91 1.55 1.51 0.94

<Table 1> shows the result of equal error rate (EER), which shows the error when the false recognition rate and the false rejection rate are the same. In the table above, LM, LV, LM + LV, i-vector + LM, i-vector + LV, i-vector + LM + LV connect the mean, log variance, mean and log variance of the latent variables, respectively. Feature that combines i-vector and hidden variable mean in 200 dimensions, feature that links i-vector and logarithmic variable log variance, and mean and logarithm of latent variable It is a feature that links variance with i-vectors. As can be seen from the above results, the log variance and average of the latent variable are used together, which shows that the speaker recognition result is better than the i-vector (i-vector (400)) of the same dimension. Furthermore, we can see that there is a higher performance improvement when used with i-vector.

Table 2 shows the classification error results, which means the error rate when the speaker with the highest PLDA score is selected as the prediction speaker. As can be seen from the above results, when the average and log variance of the latent variables are used together, the performance is higher than that of the i-vector of the same dimension. Showed.

LM LV LM + LV i-vector + LM i-vector + LV i-vector + LM + LV i-vector (200) 12.62 i-vector (400) 7.67 i-vector (600) 5.07 VAE 11.89 17.78 6.94 5.36 4.99 2.75 ALI 16.62 17.31 8.54 5.10 4.79 2.79 ALI / NLL 12.56 12.38 6.76 3.97 4.18 2.49

In the above, the apparatus for extracting the speaker speech feature and the method for extracting the speaker speech feature according to an embodiment of the present invention have been described.

The speaker speech feature extraction method of the present invention as described above may be provided in the form of a computer readable medium suitable for storing computer program instructions and data.

Such computer-readable media suitable for storing computer program instructions and data include, for example, recording media comprising magnetic media, such as hard disks, floppy disks, and magnetic tape, and compact disk read only memory (CD-ROM). , Optical media such as Digital Video Disk (DVD), magneto-optical media such as Floppy Disk, and ROM (Read Only Memory), RAM And a semiconductor memory such as a random access memory, a flash memory, an erasable programmable ROM (EPROM), and an electrically erasable programmable ROM (EEPROM). The processor and memory can be supplemented by or integrated with special purpose logic circuitry.

The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, a functional program for implementing the present invention, codes and code segments associated therewith may be used in consideration of a system environment of a computer that reads a recording medium and executes the program. It may be easily inferred or changed by.

In addition, the computer program recorded on the computer-readable recording medium as described above includes instructions for performing the functions as described above, distributed and distributed through the recording medium to be read and installed on a specific device, a specific computer, and executed. Thus, the above functions can be executed.

Although the specification includes numerous specific implementation details, these should not be construed as limiting to any invention or the scope of the claims, but rather as a description of features that may be specific to a particular embodiment of a particular invention. It must be understood. Certain features that are described in this specification in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable subcombination. Furthermore, while the features may operate in a particular combination and may be initially depicted as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, the claimed combination being a subcombination Or a combination of subcombinations.

Likewise, although the operations are depicted in the drawings in a specific order, it should not be understood that such operations must be performed in the specific order or sequential order shown in order to obtain desirable results or that all illustrated operations must be performed. In certain cases, multitasking and parallel processing may be advantageous. Moreover, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products. It should be understood that it can.

The present invention relates to a technique for extracting features from an input speech signal, and generates a delimiter so that the distribution of the input speech can have a more realistic value by generating a variable representing an uncertainty and a reliability. This enables stable speaker recognition even with short voices.

According to the present invention, it is possible to estimate the uncertainty of the extracted feature due to the short voice, and to extract the feature that effectively expresses the speaker information that cannot be represented in the conventional linear method by using the deep learning structure, When applied to the actual speaker recognition application to deal with can improve the speaker recognition performance. Furthermore, since the i-vector is combined with the input and output of the i-vector, it can be used for speaker adaptation of the speech recognizer and synthesizer that used the i-vector, and the deep learning-based speech recognizer and synthesizer In the case of can be optimized to fit the task through joint training (joint training).

100: encoder 200: GMM generating unit
300: decoder 400: discriminator

Claims (10)

In the speaker speech feature extraction method for extracting the feature of the speaker speech signal by performing Baum-Welch statistics on the speaker speech signal under UBM (Universal Background Model) conditions,
Performing Baum-Welch statistics on the speech signal under UBM conditions, and extracting a distribution of a random hidden variable representing a distribution characteristic of the input speech dependent on the input speech signal;
Calculating a first GMM supervector according to a maximum a posteriori based on Baum-Welch statistical values performed on a speaker speech signal under UBM conditions;
Extracting a sampling value for a prior probability distribution of the random hidden variable;
Calculating a second GMM supervector using a sampling value for a prior probability distribution of the random concealment variable; And
When the distribution of the random concealment variable and the pair of the first GMM supervectors are input as an input value, the input value is classified as being generated based on an input speech, and the sampling value and the first value of the prior probability distribution of the random concealment variable are determined. If a pair of 2 GMM supervectors is input as an input value, classifying the input value as being generated based on a sampling value;
Speaker speech feature extraction method comprising a. The method of claim 1,
The log-likelihood of the input speech of the second Gaussian Mixture Model (GMM) supervector is calculated by Equation 1 below.
[Equation 1]

(

: Parameter of the encoder,

: The parameters of the decoder,

Random latent variable,

Is the number of Gaussians used to represent the GMM distribution,

: First order Baum-Welch statistics for input voice and c th Gaussian,

: Dimension of the frame unit feature of the input voice,

Is the covariance matrix of the c th Gaussian of the UBM distribution,

: The number of frames included in the input voice,

: Probability that the l th frame of the input speech is included in the c th Gaussian of the UBM,

Is the mean vector of the c'th Gaussian in the estimated GMM distribution.) The method of claim 1,
Computing the first GMM supervector,
A speaker speech feature extraction method comprising calculating a first GMM supervector using Equation 2 below.
[Equation 2]

(

: Input voice,

:

of

Second frame features of UBM

Probability of belonging to the first Gaussian,

: 1st Baum-Welch statistical value,

: 0th Baum-Welch statistical value,

UBM's

Mean value of the first Gaussian) The method of claim 1,
The classifying step,
A speaker speech feature extraction method using deep neural network (DNN) learning using a sigmoid function. The method of claim 1,
The classifying step,
Using the following Equation 3, if the input is a pair of the distribution of the random hidden variable and the first GMM supervector, output 1;
And outputting zero when the input is a pair of a sampling value for a prior probability distribution of the random concealment variable and a second GMM supervector.
[Equation 3]

(

Represents the joint distribution of the hidden variables and data represented by the encoder,

Is the joint distribution represented by the decoder, and

Is the output value of the delimiter) The method of claim 1, wherein calculating the second GMM supervector comprises:
A method for extracting speaker speech features using Equation 4 as an objective function in order to maximize the log likelihood of the second GMM supervector.
[Equation 4]

(

:

Random latent variable and joint distribution of GMM distribution,

:

Joint distribution of random hidden variables and GMM distribution of,

= Number of samples used to estimate Monte Carlo,

If the input speech given meaning to the hidden variables (latent variable) sample from a random distribution of the hidden variables (latent random variable)) An encoder for performing Baum-Welch statistics on the speaker speech signal under UBM (Universal Background Model) conditions and extracting a distribution of a random latent variable representing a distribution characteristic of the input speech dependent on the input speech signal;
A GMM generator configured to calculate a first Gaussian Mixture Model (GMM) supervector according to a maximum a posteriori using Baum-Welch statistical values performed under UBM conditions on the speaker speech signal;
A decoder configured to extract a sampling value for a prior probability distribution of the random concealment variable, and calculate a second Gaussian Mixture Model (GMM) supervector using the same; And
When the distribution of the random concealment variable and the pair of the first GMM supervectors are input as an input value, the input value is classified as being generated based on an input speech, and the sampling value and the first value of the prior probability distribution of the random concealment variable are determined. A delimiter for classifying a pair of 2 GMM supervectors as inputs based on a sampling value;
Speaker speech feature extraction apparatus comprising a. The method of claim 7, wherein
The delimiter is an output value of the delimiter

The speaker speech feature extraction apparatus, characterized in that the calculation using the deep neural network (DNN) learning using the sigmoid (sigmoid) function. The method of claim 8,
The discreminator (discriminator),
Using the following formula (3) as the objective function,
Outputs 1 if the input is a pair of random hidden variables and the first GMM supervector,
If the input is a pair of a sampling value for a prior probability distribution of the random hidden variable and a second GMM supervector, output 0;
And using Equation 4 as an objective function to maximize the log likelihood of the second GMM supervector.
[Equation 3]

(

Represents the joint variable of the latent variable and data represented by the encoder,

Is the joint distribution represented by the decoder, and

Is the output value of the delimiter)
[Equation 4]

(

:

Random latent variable and joint distribution of GMM distribution,

:

Random latent variable and joint distribution of GMM distribution,

= Number of samples used to estimate Monte Carlo,

If the input speech given meaning to the hidden variables (latent variable) sample from a random distribution of the hidden variables (latent random variable)) A computer-readable recording medium storing a program for executing the speaker speech feature extraction method according to any one of claims 1 to 6.

Images