KR20210134195A - Method and apparatus for voice recognition using statistical uncertainty modeling - Google Patents

Abstract

The present invention relates to a method and device for recognizing a voice using statistical uncertainty modeling. More specifically, the present invention relates to the method for recognizing the voice comprising: (1) a step of configuring, in consideration of an uncertainty of an input voice deformed by a noise, an uncertainty-aware training (UAT) model based on a deep neural network (DNN) for estimating a phonetic target from the input voice; and (2) a step of processing voice recognition using the UAT model configured in step (1). Therefore, the present invention is capable of showing excellent voice recognition performance.

Description

Speech recognition method and apparatus using statistical uncertainty modeling

The present invention relates to a speech recognition method and apparatus, and more particularly, to a speech recognition method and apparatus utilizing statistical uncertainty modeling.

Recently, deep neural network (DNN)-based voice modeling has made great strides in the field of voice recognition. However, performance degradation when degraded data due to noise is input remains a major challenge in the field of speech recognition.

The existing DNN-based technology generates a deterministic output for a given input without considering the degree of uncertainty arising from various input information sources. Therefore, it is easy to cause problems that do not work properly for data in an environment that is not in the training dataset.

In particular, a lot of research has been done on a DNN-HMM-based speech recognition apparatus, and the existing DNN-HMM-based speech recognition apparatus generates a deterministic output using a given input. Among them, in the case of a voice recognition device that is robust to noise, a network that deterministically maps noisy data to clean data and a voice recognition device learned using only clean data are serially configured or the voice recognition device is trained with noisy data. use the method

However, in both cases, when noise that does not appear in the training data appears, it is easy to cause significant performance degradation due to incorrect mapping. Therefore, it is necessary to develop a technology that can accurately recognize voice even in a noisy environment in consideration of the degree of uncertainty arising from various input information sources.

On the other hand, as prior art related to the present invention, Patent Registration No. 10-1740637 (title of the invention: method and apparatus for voice recognition in a noisy environment using uncertainty, registration date: May 22, 2017) and the like have been disclosed.

The present invention has been proposed to solve the above problems of the previously proposed methods, and while considering the uncertainty of the input speech deformed by noise, speech uncertainty information indicating the distribution of clean speech features and variational inference (Variational Inference) , VI) based on the environmental uncertainty information representing the probability distribution of the latent variable, directly measuring the uncertainty of the input voice, and constructing a trained model to effectively reflect the uncertainty about the transformed input voice. It is an object of the present invention to provide a speech recognition method and apparatus using statistical uncertainty modeling, which exhibits excellent speech recognition performance even when noise that is not present is present.

A speech recognition method using statistical uncertainty modeling according to a feature of the present invention for achieving the above object,

A speech recognition method comprising:

(1) Considering the uncertainty of the input voice deformed by noise, a deep neural network (DNN)-based UAT (Uncertainty-Aware Training) model for estimating a phonetic target from the input voice is constructed. step; and

(2) processing speech recognition using the UAT model constructed in step (1),

The step (1) is,

(1-1) learning CUN (Clean Uncertainty Network) that receives the distorted feature (y _t ) of the input voice _{and estimates the distribution of the clean feature (x t), and outputs voice uncertainty information} to do;

(1-2) Estimating a speech target using the characteristics of the input speech and the speech uncertainty information output in step (1-1), using a Variational Inference (VI) method, Learning the EUN (Environment Uncertainty Network) modeling the probability distribution, outputting environmental uncertainty information;

(1-3) constructing a UAT model including a PN (Prediction Network) for estimating a voice target using the voice uncertainty information and the environmental uncertainty information.

Preferably, in step (1-1),

The mean and log variance of the distribution of the clean speech feature may be output as the speech uncertainty information.

Preferably, the EUN of step (1-2) is,

By transforming the Variational Autoencoder (VAE), the probability distribution of the latent variable output from the encoder can be modeled.

Preferably, in the step (1-2),

The average and variance of the distribution of the latent variable may be output as the environmental uncertainty information.

Preferably, in step (1-3),

The UAT model may be configured by learning an integrated model in which the CUN, EUN, and PN are concatenated to estimate a speech target from the input speech.

More preferably, in step (1-3),

The UAT model with improved performance may be configured by tuning the integrated model using the loss function of the PN.

A speech recognition apparatus utilizing statistical uncertainty modeling according to a feature of the present invention for achieving the above object,

A voice recognition device comprising:

a learning unit configuring a UAT (Uncertainty-Aware Training) model based on a deep neural network (DNN) for estimating a phonetic target from the input voice in consideration of the uncertainty of the input voice deformed by noise; and

A voice recognition unit for processing voice recognition using the UAT model configured in the learning unit,

The UAT model is

a Clean Uncertainty Network (CUN) that receives the distorted feature (y _t ) of the input voice _{, estimates the distribution of a clean feature (x t} ), and outputs voice uncertainty information;

A voice target is estimated using the characteristics of the input voice and the voice uncertainty information output from the CUN, but the probability distribution of the latent variable is modeled in the estimation process by a variational inference (VI) method, and environmental uncertainty information is obtained Output EUN (Environment Uncertainty Network); and

It is characterized in that it is configured to include a PN (Prediction Network) for estimating a voice target using the voice uncertainty information and the environmental uncertainty information.

Preferably, the CUN is

The mean and log variance of the distribution of the clean speech feature may be output as the speech uncertainty information.

Preferably, the EUN is

By transforming the VAE (variational autoencoder), the probability distribution of the latent variable output from the encoder can be modeled.

Preferably, the EUN is

The average and variance of the distribution of the latent variable may be output as the environmental uncertainty information.

Preferably, the UAT model,

As an integrated model in which the CUN, EUN, and PN are concatenated, it may be configured by learning to estimate a phonetic target from the input voice.

More preferably, the UAT model,

The performance may be improved by tuning the integrated model using the loss function of the PN.

According to the speech recognition method and apparatus using statistical uncertainty modeling proposed in the present invention, while considering the uncertainty of the input speech deformed by noise, speech uncertainty information indicating the distribution of clean speech features and variational inference (Variational Inference, VI) Based on the environmental uncertainty information indicating the probability distribution of the latent variable, the uncertainty of the input voice is directly measured, and the trained model is constructed to effectively reflect the uncertainty about the transformed input voice. Even if there is no noise, excellent speech recognition performance may be exhibited.

1 is a diagram illustrating a flow of a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention.
2 is a diagram illustrating the configuration of a UAT model of a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention.
3 is a diagram illustrating a detailed flow of step S100 in a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention.
4 is a diagram illustrating the learning process of step S100 as a functional block in a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention.
5 is a diagram illustrating a learning process of step S100 of a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention.
6 is a flowchart illustrating the configuration of a UAT model in a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention.
7 is a diagram illustrating a configuration of a speech recognition apparatus using statistical uncertainty modeling according to an embodiment of the present invention.
8 is a diagram illustrating a detailed configuration of a learning unit in a speech recognition apparatus using statistical uncertainty modeling according to an embodiment of the present invention.
9 is a view showing the results of a comparative experiment using a CHiME-4 test set for a voice recognition method and apparatus using statistical uncertainty modeling according to an embodiment of the present invention and another voice recognizer.
10 is a view showing the results of a comparative experiment using the AURORA-4 test set for a voice recognition method and apparatus using statistical uncertainty modeling according to an embodiment of the present invention and another voice recognizer.

Hereinafter, preferred embodiments will be described in detail so that those of ordinary skill in the art can easily practice the present invention with reference to the accompanying drawings. However, in describing a preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the same reference numerals are used throughout the drawings for parts having similar functions and functions.

In addition, throughout the specification, when a part is 'connected' with another part, it is not only 'directly connected' but also 'indirectly connected' with another element interposed therebetween. include In addition, "including" a certain component means that other components may be further included, rather than excluding other components, unless otherwise stated.

1 is a diagram illustrating a flow of a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention. As shown in FIG. 1 , the speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention is based on a deep neural network for estimating a speech target from the input speech in consideration of the uncertainty of the input speech deformed by noise. It may be implemented including the step of configuring the UAT (Uncertainty-Aware Training) model 200 ( S100 ) and the step of processing speech recognition using the configured UAT model 200 ( S200 ).

The present invention relates to a speech recognition method utilizing statistical uncertainty modeling, and the speech recognition method utilizing statistical uncertainty modeling according to a feature of the present invention may be composed of software recorded in hardware including a memory and a processor. For example, the voice recognition method utilizing the statistical uncertainty modeling of the present invention may be stored and implemented in a personal computer, a notebook computer, a server computer, a PDA, a smart phone, a tablet PC, and the like. Hereinafter, for convenience of description, a subject performing each step may be omitted.

In step S100, a UAT model based on a deep neural network (DNN) for estimating a phonetic target from the input voice may be constructed in consideration of the uncertainty of the input voice deformed by noise. The detailed flow of step S100 will be described in detail later with reference to FIG. 3 .

In step S200, speech recognition may be processed using the UAT model 200 configured in step S100.

That is, the speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention is an uncertainty that occurs when noisy data is mapped to clean data when modeling pronunciation information of noisy data, and clean data estimated from noisy data. Finally, the UAT model 200 is implemented using a structure for estimating the pronunciation information of noisy data using the uncertainty that occurs when estimating the pronunciation information using the probability distribution of (step S100), and the implemented UAT model 200 is Voice recognition can be performed by using the (step S200).

Hereinafter, in order to describe a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention, uncertainty decoding (UD) and variational autoencoder (VAE) will be first described.

First, uncertainty decoding is a method for constructing a noise-tolerant model. In learning noise-resistant speech recognition, a difference between training data and test data may cause performance degradation. When the training data is limited to a specific environment, if the test data is exposed to an environment that does not exist in the training data, the accuracy of the estimate decreases, and thus the performance of the speech recognizer is deteriorated. The UD may play a role of compensating for such defects in the decoding process.

UD assumes that the mapping from clean features (x _t ) to distorted features (y _t ) by noise is a stochastic process. Assuming that the acoustic model is trained with clean speech data and the observed input is a distorted version of this clean speech data, the likelihood function for the HMM state can be modified as shown in Equation 1 below.

Here, x _t , y _t , and q _t mean the clean feature, noisy feature, and HMM state extracted from frame t, respectively. Since the integral part of Equation 1 is computationally difficult to implement in the UD technology implementation, in the case of the existing GMM-HMM system, p(x _t |q _t ), p(x _t |y _t ), p (x _t) a was estimated using a Gaussian or Gaussian mixtures, for DNN-HMM is p (q _t | x _t) by a calculation of the expected value for the clean feature distribution p | a (q _t y _t) calculated did.

Next, VAE is a kind of autoencoder that reconstructs the input vector from the output, and takes a structure having a hidden layer in the middle as latent variables, which are random variables.

VAE consists of two networks, encoder and decoder. An encoder network takes an input vector and estimates the posterior distribution of the latent variable given the input as a condition. The latent variable sampled from the estimated distribution of the latent variable becomes the input of the decoder, and the input is reconstructed as the output of the decoder.

In VAE learning, an encoder network and a decoder network are trained using an error back-propagation algorithm at a time, and the objective function is defined as in Equation 2 below.

(z|x)는 인코더 네트워크에서 주어진 입력 x로부터 잠재변수 z를 생성할 확률을 의미하고, p_θ(x|z)은 디코더 네트워크에서 잠재변수로부터 입력 x를 재구성할 확률을 의미하며, p_θ(z)는 디코더 네트워크의 파라미터가 주어졌을 때, 잠재변수가 생성될 사전확률을 의미한다. D_KL(q

(z|x)||p_θ(z))은 x가 주어졌을 때 z의 생성 확률 분포와 z의 사전 확률 분포의 차이를 나타내는 Kullback-Leibler divergence를 나타내며, 생성되는 잠재변수의 확률 분포가 최대한 사전 확률 분포에 가깝도록 규제해주는 역할을 한다. 반면 E_q

_{( z | x )}[logpθ(x|z)]는 재구성 오차로, 입력 x가 주어졌을 때 z의 생성 확률 분포와 z로부터 x가 생성되는 확률 분포간의 cross-entropy 오차를 의미한다.q here

( z | x ) means the probability of generating a latent variable z from a given input x in the encoder network _{, p θ} ( x | z ) means the probability of reconstructing the input x from the latent variable in the decoder network _{, p θ} ( z ) means the prior probability that the latent variable will be generated when the parameters of the decoder network are given. D _KL (q

( z | x )||p _θ ( z )) represents the Kullback-Leibler divergence representing the difference between the generation probability distribution of z and the prior probability distribution of z given x , and the probability distribution of the generated latent variable is maximized. It plays a role of regulating to be close to the prior probability distribution. On the other hand, E _q

_{( z | x )} [logpθ( x | z )] is the reconstruction error, which means the cross-entropy error between the probability distribution of the generation of z when the input x is given and the probability distribution of the generation of x from z.

In the speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention, a learning technique that considers the inherent uncertainty of an input speech feature transformed by noise using a deep neural network is proposed.

In order to apply the existing UD method to a deep learning-based acoustic model, an expression for likelihood can be expressed in the form of Equation 3 below.

The conditional probability p(x _t |y _t ) allows us to consider the uncertainty arising from the _{distorted feature (y t} ) of the input speech deformed in likelihood. On the other hand, if the corresponding conditional probability shows a parametric distribution by a specific probability model, it can be seen that this likelihood is determined by the parameters of the distribution. Therefore, _{under the assumption that the distribution of x t} shows a distribution according to a specific probability model, Equation 3 above can be expressed as Equation 4 below for _{ξ xt} , which is a parameter of _{x t .}

Here, g _qt means a mapping function. However, there remains a problem in applying Equation 4 to a deep learning-based acoustic model. Since the conditional distribution of Equation 4 is estimated from the training data, it cannot be perfectly applied when there is a difference between the training data and the test data. Therefore, in the present invention, the latent variable z _t is applied to supplement information that cannot be explained when _{y t} corresponds to the corresponding HMM state q _{t .} When this is introduced into the equation, it is equivalent to the following Equation 5, and is finally expressed by the chain rule.

In the above equation, the new conditional distribution is _{affected by x t} and z _{t . Using} the same principle as before, the likelihood can be expressed as a function of the parameters of the clean speech feature and the latent variable as shown in Equation 6 below.

It is assumed that p(x _t |y _t ) and p(z _t |x _t ,y _t ) follow a Gaussian distribution in which there is no correlation between components.

Under this assumption, the UAT model 200 proposed by the present invention may proceed by combining three independent DNNs and joint training.

2 is a diagram illustrating a configuration of a UAT model 200 of a speech recognition method and apparatus using statistical uncertainty modeling according to an embodiment of the present invention. As shown in FIG. 2 , the UAT model 200 implemented and used in the speech recognition method and apparatus utilizing statistical uncertainty modeling according to an embodiment of the present invention may be composed of a combination of three independent DNNs. have. The first network, the Clean Uncertainty Network (CUN) 210 , can express uncertainty generated in the process of estimating a clean voice. The second network, the Environment Uncertainty Network (EUN) 220 , may represent uncertainty that could not be resolved due to factors not seen in the previous CUN 210 . The last network, the prediction network (PN) 230 , can predict the HMM state through uncertainty information generated from the CUN 210 and the EUN 220 .

More specifically, the UAT model 200 receives a distorted feature (y _t ) of an input voice _{, estimates the distribution of a clean feature (x t} ), and outputs voice uncertainty information (CUN (Clean)) The voice target is estimated using the uncertainty information of the Uncertainty Network 210, the characteristics of the input voice, and the CUN 210, but the probability distribution of the latent variable is modeled in the estimation process using a variational inference (VI) method. and an Environment Uncertainty Network (EUN) 220 for outputting environmental uncertainty information, and a Prediction Network (PN) 230 for estimating a voice target using the voice uncertainty information and the environmental uncertainty information. A detailed flow of configuring the UAT model 200 through learning will be described in detail below with reference to FIG. 3 .

3 is a diagram illustrating a detailed flow of step S100 in a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention. As shown in FIG. 3, step S100 of the speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention is a step of learning CUN (Clean Uncertainty Network) 210 and outputting speech uncertainty information ( S110), a step of outputting environmental uncertainty information by learning the Environment Uncertainty Network (EUN) 220 (S120) and PN (Prediction Network) 230 for estimating a voice target using the voice uncertainty information and the environmental uncertainty information It can be implemented including the step of configuring the UAT model 200 (S130).

4 is a diagram illustrating the learning process of step S100 as a functional block in a speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention. As shown in FIG. 4 , the speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention first learns the CUN 210 and the EUN 220 when learning in step S100 ( S110 and S120 ). , the PN 230 model is trained using the outputs of the learned CUN 210 and EUN 220 . After that, the three networks are finally combined to improve the overall performance by jointy fine tuning using the loss function of the PN 230 (S130).

5 is a diagram illustrating the learning process of step S100 of the speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention, and FIG. 6 is a view illustrating statistical uncertainty modeling according to an embodiment of the present invention. It is a diagram showing the configuration of the UAT model 200 as a flow chart in the speech recognition method utilized. Hereinafter, each step of step S100 of the speech recognition method using statistical uncertainty modeling according to an embodiment of the present invention will be described in detail with reference to FIGS. 5 and 6 .

In step S110, _{the CUN (Clean Uncertainty Network) 210 that receives the distorted feature (y t} ) of the input voice _{and estimates the distribution of the clean feature (x t} ) is learned, and the voice uncertainty information is obtained. can be printed out. More specifically, in step S110, the mean and log variance of the distribution of clean speech features may be output as speech uncertainty information.

That is, the CUN 210 outputs parameter information of a clean speech feature distribution with respect to the transformed input speech feature, and can be learned in a direction to maximize the likelihood of the estimated distribution. More specifically, the CUN 210 is a model that receives a noisy speech feature and outputs a mean (gaussian mean) and log-variance for the distribution of a clean speech feature. Among the outputs of this network, the mean represents the mapped clean speech feature, and the variance represents the uncertainty of the input, respectively. The loss function used for learning is shown in Equation 7 below.

In step S120, the speech target is estimated using the characteristics of the input speech and the speech uncertainty information output in step S110, but using a Variational Inference (VI) method to model the probability distribution of latent variables in the estimation process. By learning the Uncertainty Network 220 , it is possible to output environmental uncertainty information. More specifically, the EUN 220 of step S120 may model a probability distribution of a latent variable output from an encoder by transforming a Variational Autoencoder (VAE). In addition, in step S120, the average and variance of the distribution of the latent variable may be output as environmental uncertainty information.

That is, the EUN 220 is a network for estimating a voice target using the output of the CUN 210 and the characteristics of the input voice. It can be a variational inference (VI) model that models variables.

, 디코더 θ의 입력에 깨끗한 음성 특징 파라미터가 조건부로 입력되는 구조를 갖게 된다. EUN(220)의 손실함수는 VAE의 손실함수와 유사하게 다음 수학식 9을 따를 수 있다.the integral of the x _t as a function of the parameters of ξ _xt x _t can be changed to be represented. If the structure of VAE is redesigned according to this, VAE's encoder

, it has a structure in which a clean speech feature parameter is conditionally input to the input of the decoder θ. The loss function of the EUN 220 may follow Equation 9 similar to the loss function of the VAE.

The mean and variance of the distribution of latent variables output by the EUN 220 may be used as a measure of environmental uncertainty.

In step S130 , the UAT model 200 including a prediction network (PN) 230 for estimating a voice target using the voice uncertainty information and the environmental uncertainty information may be configured. More specifically, in step S130, the UAT model 200 is constructed by learning an integrated model in which the CUN 210, the EUN 220 and the PN 230 are concatenated to estimate a voice target from the input voice. can In addition, in step S130 , the UAT model 200 with improved performance may be configured by tuning the integrated model using the loss function of the PN 230 .

That is, the PN 230 is a model for estimating a voice target using parameters of a probability distribution output from the CUN 210 and the EUN 220 . The PN 230 may generate the final output of the entire network in consideration of data uncertainty. The input vector of the PN 230 is the mean and variance of the clean voice (voice uncertainty information output from the CUN 210), and latent variable parameters (environment uncertainty information output from the EUN 220), through which the UAT model 200 ) can simultaneously reflect the uncertainty of clean voice and environmental information.

The UAT model 200 is configured by combining the above three networks CUN 210 , EUN 220 , and PN 230 in order, and finishing learning by fine-tuning as a cross entropy criterion for the HMM state. can

7 is a diagram illustrating a configuration of a speech recognition apparatus 100 using statistical uncertainty modeling according to an embodiment of the present invention. As shown in FIG. 7 , the speech recognition apparatus 100 using statistical uncertainty modeling according to an embodiment of the present invention considers the uncertainty of the input speech deformed by noise, ), a deep neural network (DNN)-based UAT (Uncertainty-aware training) model for estimating speech recognition is processed using the learning unit 110 and the UAT model 200 configured in the learning unit 110 . It may be configured to include a voice recognition unit 120. The learning unit 110 may perform step S100 and the voice recognition unit 120 may perform step S200, respectively.

May be configured to learn 110 UAT model 200, UAT model 200, as shown in Fig. 2, receives the input speech feature (distorted feature, y _t), a clean speech feature ( Estimate the distribution of clean features, x _t ), and estimate the voice target using the CUN (Clean Uncertainty Network) 210, which outputs voice uncertainty information, the characteristics of the input voice, and the voice uncertainty information output from the CUN 210, EUN (Environment Uncertainty Network) 220, which models the probability distribution of latent variables in the estimation process and outputs environmental uncertainty information, using the variational inference (VI) method, and negative uncertainty information and environmental uncertainty information It may be configured to include a prediction network (PN) 230 for estimating a target. The UAT model 200 is an integrated model in which the CUN 210 , the EUN 220 , and the PN 230 are concatenated, and may be configured by learning to estimate a phonetic target from an input voice.

8 is a diagram illustrating a detailed configuration of the learning unit 110 in the speech recognition apparatus 100 using statistical uncertainty modeling according to an embodiment of the present invention. As shown in FIG. 8 , the learning unit 110 of the speech recognition apparatus 100 using statistical uncertainty modeling according to an embodiment of the present invention performs CUN learning to learn the CUN 210 by performing step S110. Module 111, an EUN learning module 112 for learning the EUN 220 by performing step S120, and a UAT model configuration module 113 for configuring the UAT model 200 by performing step S130. have.

Experiment

In order to verify the performance of the speech recognition method and apparatus using statistical uncertainty modeling according to an embodiment of the present invention, an experiment was performed using Aurora-4 and CHiME-4 datasets.

The two datasets are the datasets used to test the robust speech recognizer. The Aurora-4 dataset is a dataset recorded in various SNR and noise environments, and the CHiME-4 dataset is a dataset recorded in the real environment and reverberated through simulation. It is a dataset with noise added. As the experimental environment, feature extraction of Kaldi toolkit and GMM-HMM training, alignment, and ASR decoding modules were used.

As the baseline of the experiment, five types of existing speech recognizer models were used, and the composition of three models to be mainly contrasted is as follows.

First, the DNN-Conventional model consists of a network that regresses noisy data to clean data without modeling uncertainty, and a network that estimates phonetic targets using the regressed clean data.

In the case of the DNN-ID model, when modeling noisy data as clean data, the Gaussian distribution parameters are output, and the prediction network estimates the voice target using the mean and variance information output from the CUN (210). In this case, the uncertainty about the clean feature is reflected to some extent, but unlike the UAT model 200 proposed in the present invention, information about the uncertainty that occurs when predicting using the uncertain feature is not reflected.

In the case of the VAE-Conventional model, the CUN 210 is modeled using the VAE structure, and the PN 230 estimates a negative target using the distribution of the latent variables of the CUN 210 . Like the DNN-ID model, this model also reflects only the uncertainty of noisy features.

The experimental results were compared with the word error rate (WER) for each model.

9 is a view showing the results of a comparison experiment using a CHiME-4 test set for a voice recognition method and apparatus using statistical uncertainty modeling according to an embodiment of the present invention and another voice recognizer, and FIG. 10 is an embodiment of the present invention. It is a diagram showing the results of comparative experiments using the AURORA-4 test set for a speech recognition method and apparatus using statistical uncertainty modeling according to an embodiment and another speech recognizer. The results of FIGS. 9 and 10 were all expressed as WERs (%). In FIG. 9, datasets A, B, C, and D represent commonly used experimental environments, where A is Clean, matched-channel condition, test condition 1, B is Noisy, matched-channel condition, test conditions 2-7 , C represents Clean, mismatched-channel condition, test condition 8, D represents Noisy, mismatched-channel condition, and test conditions 9-14, respectively.

9 and 10, the DNN-ID and VAE-Conventional models considering the uncertainty of clean features compared to the DNN-Baseline and DNN-Conventional models that do not model the uncertainty in datasets of various environments. It can be seen that the overall performance is excellent. In addition, it can be seen that the UAT model 200 proposed by the present invention shows better performance in various noise environments compared to the rest of the models.

As described above, according to the speech recognition method and apparatus using statistical uncertainty modeling proposed in the present invention, while considering the uncertainty of the input speech deformed by noise, speech uncertainty information and variance representing the distribution of clean speech features Based on Variational Inference (VI), the uncertainty of the input voice is directly measured using the environmental uncertainty information representing the probability distribution of the latent variable, and a trained model is constructed to effectively reflect the uncertainty about the transformed input voice. By doing so, even if noise that is not present in the training data appears, excellent speech recognition performance can be exhibited.

The present invention is a technique for measuring the degree of uncertainty when processing data in which uncertainty exists due to noise or various environmental factors, and using this to mitigate performance degradation caused by various environmental factors. Therefore, the UAT model 200 of the present invention models the uncertainty about voice data in the field of voice recognition based on DNN-HMM, but in recent years, in relation to the field of artificial intelligence, this algorithm is applied to any data processing other than voice such as video and text. expected to be applicable.

Meanwhile, the present invention may include a computer-readable medium including program instructions for performing operations implemented in various communication terminals. For example, the computer-readable medium includes magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD_ROM and DVD, and floppy disks. It may include magneto-optical media and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Such a computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. In this case, the program instructions recorded in the computer-readable medium may be specially designed and configured to implement the present invention, or may be known and used by those skilled in the art of computer software. For example, it may include not only machine language code such as generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

Various modifications and applications of the present invention described above are possible by those skilled in the art to which the present invention pertains, and the scope of the technical idea according to the present invention should be defined by the following claims.

100: speech recognition device
110: study department
111: CUN Learning Module
112: EUN Learning Module
113: UAT model configuration module
120: voice recognition unit
200: UAT model
210: CUN
220: EUN
230: PN
S100: Constructing a deep neural network-based UAT model for estimating a speech target from the input speech in consideration of the uncertainty of the input speech deformed by noise
S110: Step of learning CUN (Clean Uncertainty Network) and outputting voice uncertainty information
S120: A step of learning EUN (Environment Uncertainty Network) and outputting environmental uncertainty information
S130: Constructing a UAT model including a PN (Prediction Network) for estimating a voice target using voice uncertainty information and environmental uncertainty information
S200: processing speech recognition using the configured UAT model

Claims (12)

A speech recognition method comprising:
(1) Considering the uncertainty of the input voice deformed by noise, a deep neural network (DNN)-based UAT (Uncertainty-Aware Training) model for estimating a phonetic target from the input voice is constructed. step; and
(2) processing speech recognition using the UAT model 200 configured in step (1),
The step (1) is,
(1-1) By learning a Clean Uncertainty Network (CUN) 210 that receives the distorted feature (y _t ) of the input voice _{and estimates the distribution of a clean feature (x t ), the voice uncertainty} outputting information;
(1-2) Estimating a speech target using the characteristics of the input speech and the speech uncertainty information output in step (1-1), using a Variational Inference (VI) method, Learning the EUN (Environment Uncertainty Network) 220 for modeling the probability distribution, outputting environmental uncertainty information;
(1-3) Statistical uncertainty, characterized in that it comprises the step of constructing a UAT model 200 including a PN (Prediction Network) 230 for estimating a voice target using the voice uncertainty information and the environmental uncertainty information Speech recognition method using modeling.
The method of claim 1, wherein in step (1-1),
A speech recognition method using statistical uncertainty modeling, characterized in that the mean and log variance of the distribution of the clean speech feature are output as the speech uncertainty information.
According to claim 1, wherein the EUN (220) of step (1-2),
A speech recognition method using statistical uncertainty modeling, characterized in that the VAE (Variational Autoencoder) is transformed to model the probability distribution of the latent variable output from the encoder.
The method of claim 1, wherein in step (1-2),
A speech recognition method using statistical uncertainty modeling, characterized in that the average and variance of the distribution of the latent variable are output as the environmental uncertainty information.
The method of claim 1, wherein in step (1-3),
The UAT model 200 is configured by learning an integrated model in which the CUN 210, EUN 220 and PN 230 are concatenated to estimate a voice target from the input voice, Speech recognition method using statistical uncertainty modeling.
The method of claim 5, wherein in step (1-3),
A speech recognition method using statistical uncertainty modeling, characterized in that the UAT model (200) with improved performance is configured by tuning the integrated model using the loss function of the PN (230).
A voice recognition device 100 comprising:
In consideration of the uncertainty of the input voice deformed by noise, a learning unit ( 110); and
A voice recognition unit 120 for processing voice recognition using the UAT model 200 configured in the learning unit 110,
The UAT model 200,
a Clean Uncertainty Network (CUN) 210 for receiving the distorted feature (y _t ) of the input voice _{, estimating the distribution of a clean feature (x t ), and outputting voice uncertainty information;}
A voice target is estimated using the characteristics of the input voice and the voice uncertainty information output from the CUN 210, and the probability distribution of the latent variable is modeled in the estimation process by a variational inference (VI) method, and the environment EUN (Environment Uncertainty Network) 220 for outputting uncertainty information; and
A speech recognition apparatus 100 using statistical uncertainty modeling, characterized in that it comprises a PN (Prediction Network) 230 for estimating a speech target using the speech uncertainty information and the environmental uncertainty information.
The method of claim 7, wherein the CUN (210),
The speech recognition apparatus 100 using statistical uncertainty modeling, characterized in that the mean and log variance of the distribution of the clean speech feature are output as the speech uncertainty information.
The method of claim 7, wherein the EUN (220),
Speech recognition apparatus 100 using statistical uncertainty modeling, characterized in that the VAE (variational autoencoder) is transformed to model the probability distribution of the latent variable output from the encoder.
The method of claim 7, wherein the EUN (220),
A speech recognition apparatus 100 using statistical uncertainty modeling, characterized in that the average and variance of the distribution of the latent variable are output as the environmental uncertainty information.
According to claim 7, The UAT model 200,
As an integrated model in which the CUN 210, EUN 220 and PN 230 are concatenated, it is characterized in that it is configured by learning to estimate a phonetic target from the input voice, Statistical uncertainty modeling A voice recognition device 100 using
The method of claim 11, wherein the UAT model 200,
The speech recognition apparatus 100 using statistical uncertainty modeling, characterized in that the performance is improved by tuning the integrated model using the loss function of the PN (230).

Images