JP2009145499A - Voice parameter learning apparatus and method therefor, voice recognition apparatus and voice recognition method using them, and their program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice-parameter learning apparatus that does not depend on specific voice emphasis method. <P>SOLUTION: This voice-parameter learning apparatus includes a voice preprocessing section for adaptation, an acoustic model storage section, an adaptation parameter creating section, a voice preprocessing section for recognition, and a dispersion dynamic correcting section. The adaptation parameter creating section creates a dynamic dispersion adaptive parameter depending on a frame as a parameter for dispersion correction, and a static dispersion adaptive parameter independent of the frame. The voice preprocessing section for recognition creates a voice feature amount for each frame of an observation voice signal, and uncertainty showing variation in the voice feature amount. The dispersion dynamic correcting section receives the uncertainty of the voice feature amount, the adaptive parameter, and the acoustic model, and outputs the dispersion of the Gaussian distribution corrected with the adaptive parameter for each frame. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a speech parameter learning method and apparatus, and a speech recognition apparatus and speech recognition method using the apparatus and method for suppressing speech distortion that occurs at the stage of speech preprocessing such as noise suppression and dereverberation. And the programs and recording media.

  When performing speech recognition, the observed speech signal is greatly distorted by external factors such as noise and reverberation. Speech recognition is not good at recognizing such heavily distorted speech. In the speech preprocessing unit, noise can be reduced by performing noise suppression and dereverberation. However, even if audio preprocessing is performed, audio distortion still exists due to distortion newly introduced by the audio preprocessing unit and unerased distortion. Therefore, a method often used is a method of correcting the dispersion parameter of the Gaussian distribution included in the acoustic model for speech recognition. This method is disclosed in Non-Patent Document 1. The functional configuration of a conventional speech recognition apparatus based on this method is shown in FIG. 9, and its operation flow is shown in FIG.

  The speech recognition apparatus 200 includes a speech preprocessing unit 90, an acoustic model storage unit 92, a distributed dynamic correction unit 94, a speech recognition acoustic model storage unit 96, a recognition unit 97, and a pronunciation dictionary model storage unit 98. And a language model storage unit 99.

ただし、差分ｂ_ｔは式（２）に示す様に平均０のガウス分布に従うと仮定する。

The speech preprocessing unit 90 reads the observed speech signal o (t) (step S90) and, for example, the speech feature amount x _t ^ (^ is a figure or an expression estimated by speech enhancement technology such as noise suppression or dereverberation method. Is output for each frame. However, as described above, the speech pre-processing unit 90 cannot completely eliminate the speech distortion, and there is a large mismatch between the estimated speech feature amount x _t ^ and the clean speech feature used when constructing the acoustic model. Exists. This is a major factor that degrades recognition performance. Therefore, it is assumed that the speech feature amount x _t ^ is the sum of the clean speech feature x _t and the difference b _t (formula (1)).

However, it is assumed that the difference b _t follows an average 0 Gaussian distribution as shown in the equation (2).

Here, Σ _xt ^ is a variance of speech feature values. That is, the speech preprocessing unit 90 outputs the variance Σ _xt ^ of the voice feature value together with the estimated voice feature value x _t ^ (step S91). In the speech enhancement method based on GMM, the speech feature amount variance Σ _xt ^ is derived from the dispersion parameters of the mixed Gaussian distribution model of clean speech.

The distributed dynamic correction unit 94 reads the dispersion parameter Σ _{n, m} (n is an HMM state, m is a mixed component) of the acoustic model stored in the acoustic model storage unit 92 (step S92), and the speech preprocessing unit 90 Is corrected using the variance Σ _xt ^ of the voice feature value output by (step S94). Here, the acoustic model will be described. The acoustic model is usually expressed by a hidden Markov model (HMM), and a mixed Gaussian distribution is used as the output distribution of the HMM. Output probability for outputting the speech feature x _t In certain HMM state n is represented by the formula (3).

Here, m is an index of the mixture component of the Gaussian distribution, and M represents the number of mixtures per state. p (m) represents a mixing weight factor. μ _{n, m} and Σ _{n, m} represent the mean parameter and covariance matrix of the Gaussian distribution in the HMM state n and the mixture component m. An ordinary acoustic model often treats a covariance matrix as a diagonal covariance matrix. Therefore, hereinafter, the diagonal component of the covariance matrix may be expressed as the standard deviation σ _{n, m, i} ² using the feature quantity dimension index i.

In general, since the acoustic model parameters are learned using clean speech, for example, the average parameter μ _{n, m} obtained from the data and the speech feature amount x _t ^ estimated by the speech preprocessing unit 90 There is a mismatch. In order to alleviate such mismatch, the distributed dynamic correction unit 94 performs correction so that the dispersion parameter Σ _{n, m} of the acoustic model matches the speech feature amount x _t ^. In order to perform the correction to match the dispersion parameter Σ _{n, m} with the speech feature amount x _t ^, for the output probability p (x _t | n) of the acoustic model in the HMM state n, x _t and x _t and x _t ^ By considering the joint probability of the difference b _t and performing marginalization (integration) on b _t , an output probability p (x _t | n) as shown in Equation (4) can be theoretically derived.

Here, it is assumed that p (b _t | n) ≈p (b _t ). Therefore, the variance dynamic correction unit 94 dynamically corrects the variance parameter Σ _{n, m} of the acoustic model as shown in Expression (5) by using the variance Σ _xt ^ of the voice feature amount for each frame. An output distribution that outputs the estimated speech feature amount x _t ^ can be obtained.

The corrected output distribution is stored in the acoustic model storage unit 96 for speech recognition.
In the recognition unit 97, the acoustic model p (X | n) and the pronunciation dictionary model storage unit 98 for the feature amount set X = [x ₁ ^,..., X _t ^,. Using the pronunciation dictionary model p (n | W) stored in the language model and the language model p (W) stored in the language model storage unit 99, the speech recognition result W is output as shown in equation (6) (step S97). ).

As the score of the acoustic model p (X | n) for the feature quantity set, the acoustic score for each frame t obtained from the output probability p (x _t | n) is accumulated using DP matching (dynamic programming) or the like. It is obtained by doing.

The acoustic score for each frame t obtained from the output probability p (x _t | n) is obtained from the estimated speech feature amount x _t ^ output from the speech preprocessing unit 90 and the distributed dynamic correction unit 94. Using the corrected variance Σ _{n, m} + Σ _xt ^ and other acoustic model parameters, the calculation can be made as shown in Equation (7).

With the above operation, speech recognition is realized in which speech distortion occurring at the stage of performing speech preprocessing such as noise suppression and dereverberation is suppressed.
Deng, L., Droppo, J. and Acero, A., “Dynamic compensation of HMM variances using the feature enhancement uncertainty computed from a parametric model of speech distortion,” IEEE Trans.SAP, vol. 13, no. 3, pp .412-421,2005.

However, in the method described above, it is necessary for the speech preprocessing unit 90 to generate the speech feature amount variance Σ _xt ^ used in the distributed dynamic correction unit 94. The speech preprocessing unit 90 uses a speech enhancement method based on a mixed Gaussian distribution of clean speech, and the variance Σ _xt ^ of the speech feature value is derived from the dispersion parameter of the mixed Gaussian distribution model. In many other speech enhancement methods such as spectral subtraction, speech separation (BSS), and Wiener filter (wiener), it is difficult to directly output the variance of speech features, and the above method is difficult to apply. . That is, the above-described conventional method lacks versatility in that a specific speech enhancement method must be used.

Further, by approximating the square error of the speech feature u _t of the observed speech signal and the speech feature amount x _t ^ estimated by the speech pre-processing unit to the variance of the speech feature amount, the dynamics independent of the speech enhancement method are used. Dispersion correction is possible. However, originally, the variance of the speech feature amount necessary for the distributed dynamic correction is a square error between the clean speech feature x _t and the speech feature amount x _t ^ estimated by the speech pre-processing unit. The accuracy of the automatic dispersion correction is lowered and the performance is deteriorated.

  The present invention has been made in view of the above points, and a speech parameter learning apparatus and method capable of obtaining an appropriate acoustic model using any variance of speech feature values, and speech using them. An object of the present invention is to provide a recognition device, a speech recognition method, a program thereof, and a recording medium.

  The speech parameter learning apparatus according to the present invention includes an adaptive speech preprocessing unit, an acoustic model storage unit, an adaptive parameter generation unit, a recognition speech preprocessing unit, and a distributed dynamic correction unit. The adaptation speech preprocessing unit receives the observed speech signal and generates a speech feature amount of the enhanced speech signal in which the speech feature of each frame of the observed speech signal is emphasized and an uncertainty representing the variation of the speech feature amount. . The acoustic model storage unit stores an acoustic model. The adaptive parameter generation unit receives the set of emphasized speech features, the set of uncertainties, the acoustic model, and the teacher signal as input, and adds them to the frame as adaptive parameters for dispersion correction of the Gaussian distribution in the acoustic model. Generate dependent dynamic distributed adaptation parameters and static distributed adaptive parameters independent of frames. The recognition speech pre-processing unit generates a speech feature amount for each frame of the observed speech signal and an uncertainty representing variation in the speech feature amount. The variance dynamic correction unit outputs the variance of the Gaussian distribution of the acoustic model corrected by the adaptation parameter for each frame with the uncertainty of the speech feature value, the adaptation parameter, and the acoustic model as inputs.

  The speech recognition device according to the present invention includes the speech parameter learning device described above and a recognition unit. The recognizing unit outputs a word string with the speech feature amount output by the speech parameter learning device and the variance of the Gaussian distribution of the acoustic model corrected by the speech parameter learning device as inputs.

  In the speech parameter learning device of the present invention, the adaptive parameter generation unit generates a dynamic dispersion parameter that depends on a frame and a static dispersion parameter that does not depend on a frame from the observed speech signal as parameters for dispersion correction of the acoustic model. To do. That is, since a parameter for dispersion correction can be generated without using the mixed Gaussian distribution method in the speech enhancement unit, a highly versatile speech parameter learning apparatus that can cope with any speech enhancement method can be provided. Further, a speech recognition device using this speech parameter learning device can realize speech recognition with high recognition performance with suppressed speech distortion without depending on a specific speech enhancement method.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

ここで演算子（＋）は行列Σ^Ｓ、Σ^Ｄに対しての、和・積などの演算及びそれらの演算等の組み合わせで表現される２項演算を意味する。 [Basic idea of the invention]
Prior to the description of the embodiments, the basic concept of the method for correcting the variance of the audio feature quantity of the present invention will be described. The present invention, the static component that is independent of the dispersion sigma _'n dispersion correction acoustic model as shown in Equation _{(8), m,} and _t, and a dynamic component matrix sigma ^D that depends on the frame t, in frame t It expressed as a combination of the matrix Σ ^S.

Here, the operator (+) means a binary operation expressed by a combination of operations such as sum and product and the operations on the matrices Σ ^S and Σ ^D.

In _order to correct the variance Σ _{n, m} of the acoustic model, the variance of the speech feature value is used. As shown in the equation (9), a function f that outputs the variance of the feature quantity with the voice feature quantity variance _et and Σ _{n, m} as arguments may be obtained.

When the variance Σ _xt ^ of the speech feature described in the background art is added to the variance of the acoustic model as it is, that is, it is sufficient if Σ _xt ^ is not estimated accurately in Σ _{n, m} + Σ _xt ^ (formula (5)). Unable to get performance. In addition, this method limits the speech enhancement technique. Therefore, in the present invention, as shown in the equation (10), the dynamic component matrix Σ ^D (e _t ) whose acoustic model variance depends on the uncertainty of each speech feature amount in each frame t, and each speech feature amount frame It is expressed as a combination of static component matrices Σ ^S that do not depend on the uncertainty at t. However, it takes the scalar or vector values and matrix values as uncertainty e _t. As the uncertainty of the scalar value, the binary value and reliability output during speech preprocessing such as speech enhancement and speech interval detection (VAD), the reliability calculated by performing speech recognition, and the like can be considered. It is also possible to calculate vector type uncertainty by calculating the uncertainty of the scalar value for each feature dimension. It is also possible to calculate a matrix type uncertainty from a covariance matrix or an autocorrelation matrix.

Further, in order to estimate the variance sigma _{'n, m, t} for the acoustic model of a certain frame t is not only uncertainty e _t of the audio feature, the uncertainty e of speech features, including frame t Information about the set, the speech feature set x _t ^, and the acoustic model Ψ is also useful. Therefore, using these, the acoustic model feature amount variance is expressed as shown in Expression (11).

The acoustic model feature quantity variance Σ ′ _{n, m, t} is a set of emphasized speech feature quantities, for example, as a finite section from t ′ to t ″ including t x = {x _{t ′} ^ ... x _t ^ ... x _Since it depends on _{t ″} ^}, a set of uncertainties of the emphasized speech feature quantity, for example, e = {e _{t ′} ... e _t ... e _{t ″} }, it can be accurately obtained by learning.
An appropriate function system of equation (11) is shown. In general, the more complicated the function system, the larger the amount of learning data and the longer the learning time are required, but the function system can be learned with high accuracy. Conversely, the simpler the function system is, the more the function system can be learned with a small amount of learning data and a short learning time, but the accuracy is generally lower than that of a complex function system. Therefore, the function system described below may be appropriately selected according to application conditions such as the learning data amount and the learning time. In the following, since parameter estimation by learning is assumed, arguments such as e and x are omitted for simplicity. As a simple form of the binary operation (+) of Expression (11), a product expression shown in Expression (12) and a sum expression shown in Expression (13) can be considered.

  The sum expression of the variance in equation (13) can be said to be a theoretically and practically appropriate expression in view of analogy with equation (5). Therefore, hereinafter, the description will be made in the Japanese expression.

When sigma ^S is assumed to depend on the distribution of acoustic models, the feature variance sigma _{'n, m, t} of expressed acoustic model in equation (14) can be expressed by equation (14).

これは特徴量が線形変換された場合の分散の変換式である。
ここでＡ，Ｂ，Ｃ，Ｄは、特徴量次元の正方行列であり、他の部分のＡ〜Ｄとは異なる変数である。行列は任意の形でよい（対称、ブロック、帯、スカラー倍の単位行列）。以降では、分散のバイアス項の影響を無視し(Ｂ＝０,Ｄ＝０)、ＡとＣの対角行列に対しての表現で説明する。ＡとＣのｉ行ｉ列の対角成分を√λ_ｉと√α_ｉと表わすと、音響モデルの特徴量分散Σ′_{ｎ、ｍ、ｔ}の対角成分は式（１７）で表わせる。つまり、音響モデルの分散をパラメトリック表現することができる。

Here, an arbitrary function such as a matrix polynomial is given as a function system of Σ ^S and Σ ^D. As its simplest form, it can be expressed by equations (15) and (16).

This is a dispersion conversion formula when the feature amount is linearly converted.
Here, A, B, C, and D are square matrices of the feature quantity dimension, and are variables different from other parts A to D. The matrix can be in any form (symmetric, block, band, scalar multiple unit matrix). Hereinafter, the influence of the bias term of the dispersion is ignored (B = 0, D = 0), and the description will be made with the expression for the diagonal matrix of A and C. When the diagonal elements of the i-th row i column of A and C represent the √Ramuda _i and √Arufa _i, diagonal elements of the feature quantity distributed Σ _{'n, m, t} for the acoustic model can be expressed by Equation (17). That is, the variance of the acoustic model can be expressed parametrically.

Here, σ _{n, m, i} ² is a diagonal (i × i) component of the covariance matrix of the Gaussian distribution in the acoustic model in the state n and the mixed component m. At this time, the parameters to be estimated by learning are α and λ. It should be noted here that when α = 0, the conventional static dispersion correction method is used. Further, when α = const, λ _i = 1, it is a conventional dynamic dispersion correction method. That is, it can be said that the method of the present invention is a method including both conventional methods. Next, an embodiment of the speech parameter learning apparatus of the present invention based on the above-described idea will be described.

  FIG. 1 shows a schematic functional configuration example of the speech parameter learning apparatus according to the first embodiment of the present invention. The speech parameter learning apparatus 100 includes an adaptive speech preprocessing unit 2, an acoustic model storage unit 4, an adaptive parameter generation unit 6, a recognition speech preprocessing unit 8, and a distributed dynamic correction unit 10. The operation flow is shown in FIG. The speech parameter learning apparatus 100 of this example is realized by reading a predetermined program into a computer composed of, for example, a ROM, a RAM, a CPU, and the like, and executing the program by the CPU.

  The speech parameter learning device 100 estimates α and λ of the parameters described above. The observed speech signal input to the adaptive speech preprocessing unit 2 and the recognition speech preprocessing unit 8 is, for example, a discrete value with a sampling frequency = 8 kHz and a quantization bit number = 16 bits. The adaptation speech pre-processing unit 2 and the recognition speech pre-processing unit 8 process the discrete values, for example, 240 points as one frame.

The adaptive speech preprocessing unit 2 sets a set of emphasized speech feature values { _{xt ′} ^,..., _Xt ^, ..., _{xt ''} ^ that emphasizes the speech characteristics of each frame of the observed speech signal o (t). } And a set of uncertainties {e _{t ′} ,..., E _t ,..., E _{t ″} } representing variations in the emphasized speech feature value (step S2, FIG. 2). The adaptive parameter generation unit 6 includes a set of emphasized speech feature values {x _{t ′} ^,..., X _t ^,..., X _{t ″} ^} and a set of uncertainties representing variations in the emphasized speech feature values {e _{t '} , ..., _et , ..., et _" }, and the acoustic model stored in the acoustic model storage unit 4 and the teacher signal are used as input to generate adaptive parameters for correcting the Gaussian distribution in the acoustic model. (Step S6). The adaptive parameter generation process includes a static distributed adaptation process (step S62) that generates a static distributed adaptive parameter λ that does not depend on a frame (step S62), and a dynamic distributed process that generates a dynamic distributed adaptive parameter α that depends on a frame (step S66). 2). The order of both processes may be either.

Recognition voice pre-processing unit 8, the observed speech signals audio feature amount of each frame of o _(t) x t _^, generates an uncertainty e _t representing the variation of the audio feature amount (step S8). Note that the recognition speech preprocessing unit 8 in this example performs the same processing as the adaptive speech preprocessing unit 2. Distributed dynamic correction unit 10, the adaptive parameter α and λ and uncertainty e _t and, as input an acoustic model stored in the acoustic model storage unit 4, the dispersion sigma _n of the Gaussian distribution of the acoustic model for each _{frame, The} variance Σ ′ _{n, m, t} obtained by correcting _m with the adaptive parameters α and λ is output (step S10).

The adaptive speech preprocessing unit 2, the adaptive parameter generation unit 6, and the distributed dynamic correction unit 10 constitute an adaptive parameter learning unit. Here, the learning of the dispersion parameter of the acoustic model expressed in the parametric manner will be described.
In general, a teacher signal is required for learning. As the teacher signal (hereinafter referred to as a label), label information in each frame is required. The label includes word information, phoneme information, HMM state information, and the like. If the observation audio signal is pre-labeled, it is used as it is. Alternatively, for example, a label may be given using a voice recognizer or a voice section detector (not shown).

  Learning is a method of generating acoustic model parameters using speech data, labels, and the like, and the output of learning is a new acoustic model. The speech recognition apparatus performs speech recognition using the acoustic model. In this example, adaptation is used for dynamic correction. In adaptation, parameters are generated using speech data, labels, and the like. Unlike learning, the output is an adaptation parameter. The adaptive parameter generation unit 6 includes a static variance adaptation unit 62 and a dynamic variance adaptation unit 66, and includes a set of emphasized speech feature amounts, a set of uncertainties of the emphasized speech feature amounts, a label, and an acoustic model. As an input, adaptive parameters for dispersion correction such as α and λ shown in Expression (17) are calculated.

  As a learning standard, for example, likelihood maximization is adopted. Maximum likelihood learning is a norm for learning parameters so that the acoustic model stored in the acoustic model storage unit 4 maximizes the likelihood of outputting learning data. As another learning method, Bayesian learning based on maximization of the posterior probability may be used. However, in that case, it is necessary to set an appropriate conjugate distribution or no information prior distribution for each parameter as the prior distribution. In addition, identification learning using identification criteria such as a speech recognition rate can be raised. By using such a criterion, a cost function having parameters as arguments can be derived.

  A parameter for optimizing the cost function obtained from the learning criterion is estimated. As an optimization method, numerical calculation such as a steepest descent method, a neural network, a sampling method such as Markov chain Monte Carlo, a genetic algorithm, and the like can be considered. In this embodiment, an example using an expected value maximization (EM) algorithm will be described.

The EM algorithm is a method for obtaining a parameter that maximizes the auxiliary function Q (θ | θ ′) defined by the equation (18), not directly maximizing the likelihood.

  θ is a parameter set for dispersion correction, and specifically, α and λ. X is a sequence of clean speech feature values, T is the number of frames, θ ′ is the previous estimated value in each iteration, and θ is a parameter to be estimated in each iteration.

Since the auxiliary function Q (θ | θ ′) matches the increase / decrease relationship in likelihood, θ that maximizes Equation (18) is a local optimal solution. Here, B is a sequence of differential feature values, S is a set of all sequences of HMM states, C is a set of all sequences of mixed components, and N is the number of HMM states. The auxiliary function Q (θ | θ ′) is similar to the auxiliary function of the conventional stochastic matching method, but the logarithm term of the output distribution of the fourth-stage difference vector b _t in equation (18), that is, dynamic correction. The existence of a term is the difference.

  In the expected value step (E-step), the occupancy posterior probability value assigned to the state series and mixed component series for each frame using a data allocation method for hidden variables such as the forward / backward algorithm and the Viterbi algorithm Based on this value, various statistics such as a primary statistic are obtained by calculating the expected value.

In the maximization step (M-step), the parameter θ ^ shown in the equation (19) that maximizes the equation (18) is obtained based on the statistic obtained in the E-step.

  The adaptation parameters α and λ are mutually dependent, and it is difficult to optimize each of them simultaneously. Therefore, the adaptive parameter generation unit 6 estimates the static dispersion parameter λ and the dynamic dispersion parameter α separately. A more specific functional configuration example of the adaptive speech preprocessing unit 2 and the adaptive parameter generation unit 6 is shown in FIG. 3, and the speech parameter learning device 100 will be described in more detail. The operation flow is shown in FIG.

The adaptive speech preprocessing unit 2 includes a speech enhancement unit 20, a feature amount calculation unit 21, an enhanced speech feature amount calculation unit 22, and an uncertainty calculation unit 23. The speech enhancement unit 20 generates an enhanced speech signal o ^ (t) in which speech features for each frame of the input observed speech signal o (t) are enhanced (step S2a). The feature amount calculation unit 21 calculates the feature amount u _t for each frame of the observed audio signal o (t) (step S2b). The emphasized speech feature amount calculation unit 22 calculates the speech feature x _t ^ of the enhanced speech signal as a set of emphasized speech feature amounts {x _{t '} ^, ..., x _t ^, ..., x _t'' ^} (step S2c). The uncertainty calculation unit 23 receives the emphasized speech feature quantity x _t ^ for each frame and the feature quantity u _t of the observed speech signal o (t), and inputs an uncertainty e _t = (x _t ^ _-u ^t) is calculated ^2, the set, for example, _{{e t ', ..., e} t, ..., e t''} outputs the (step S2d). Each set is input to the adaptive parameter generation unit 6.

  The adaptive parameter generation unit 6 includes an occupation probability calculation unit 64, a clean speech variance calculation unit 62a, a scaling factor λ calculation unit 62b, a difference square value calculation unit 66a, and a scaling factor α calculation unit 66b.

The occupancy probability calculation unit 64, a set of enhancement audio feature _{{x t '^, ...,} x t ^, ..., x t''^} and a set of uncertainty _{{e t', ..., e} t, .., E _{t ″} }, the label, and the acoustic model in the acoustic model storage unit 4 are input, and the occupancy probability γ _t (n, m) of the HMM state n and the mixed component m is calculated (step S60). . This occupation probability γ _t (n, m) can be calculated by a data allocation method such as a forward / backward algorithm or a Viterbi algorithm in the E-step of the EM algorithm.

The clean speech variance calculation unit 62a includes a set of emphasized speech feature values {x _{t ′} ^,..., X _t ^, ..., x _{t ″} ^} and a set of uncertainties {e _{t ′} ,..., _Et , .., E _{t ″} } and the acoustic model in the acoustic model storage unit 4 as inputs, and an estimated value A {x _{t, i} , x _t ^, n, m, Ψ, α ′, λ ′} is calculated.

ここで、

クリーンスピーチ分散算出部６２ａと、スケーリング因子λ算出部６２ｂとで静的分散適応手段６２を構成する。 The scaling factor λ calculation unit 62b receives the estimated value A { _{xt, i} , _xt ^, n, m, Ψ, α ′, λ ′} of the clean speech and the occupation probability γ _t (n, m). as, alpha = time const, the scaling factor lambda _i of each feature quantity dimension i, and updates the M-step of the EM algorithm as shown in equation (20) (step S62).

here,

The clean speech variance calculation unit 62a and the scaling factor λ calculation unit 62b constitute a static variance adaptation means 62.

The difference square value calculation unit 66a includes a set of emphasized speech feature values { _{xt '} ^, ..., _xt ^, ..., xt _" ^} and a set of uncertainties { _et' , ..., _et. ,..., E _{t ″} } and the acoustic model in the acoustic model storage unit 4 as inputs, and the expected value E {b ² of the difference b _t ² between the speech feature amount x _t ^ and the clean speech feature x _{t. t, i} | x _t ^, n, m, Ψ, α ′, λ ′} is calculated.

ここで

The scaling factor α calculation unit 66b updates the scaling factor α _i in each feature dimension i as shown in Expression (23) when λ = const (step S66). Equation (23) is obtained by substituting Equation (17) and Equation (2) into Equation (18) and maximizing α _i when λ = const.

here

From equation (23), the scaling factor α _i takes the ratio between the expected value of the square of the difference vector and the uncertainty et _{, i over} all learning data, all HMM states, and all mixture components. Can be interpreted. The difference square value calculation unit 66a and the scaling factor α calculation unit 66b constitute a dynamic dispersion adaptation unit 66.

The distributed dynamic correction unit 10 receives the scaling factors α _i and λ _i , the acoustic model stored in the acoustic model storage unit 4, and the uncertainty e _t for each frame input from the recognition speech preprocessing unit 8. As a result, Gaussian distribution Σ ′ _{n, m, t} of the corrected acoustic model is output. For example, when Σ ′ _{n, m, t} is a diagonal matrix, each diagonal component can be calculated by Expression (26).

[Application example]
The speech recognition device 150 can be configured using the speech model parameter learning device 100 described above. FIG. 5 shows a functional configuration example of the speech recognition apparatus 150. The operation flow is shown in FIG. The speech recognition device 150 is obtained by replacing the speech pre-processing unit 90, the acoustic model storage unit 92, and the distributed dynamic correction unit 94 of the conventional speech recognition device 200 described in the background art with a speech parameter learning device 100. It is. Other configurations are the same as those of the speech recognition apparatus 200. The speech parameter learning device 100 performs the above-described operation for each frame, and the speech feature amount x _t ^ for each frame of the observed speech signal and the variance Σ ′ _n, of the Gaussian distribution of the acoustic model corrected by the adaptive parameter _{m and t} and the average parameter μ _{n, m of} the acoustic model are output (step S10, FIG. 6). The recognizing unit 74 outputs the word string W using the variance Σ ′ _{n, m, t} of the Gaussian distribution of the acoustic model corrected with the adaptive parameter by the same operation as the speech recognition apparatus 200 already described (step S97). ). That is, the speech recognition apparatus 150 can realize speech recognition with suppressed speech distortion without depending on a specific speech enhancement method. Further, as will be described later, a speech recognition device having high recognition performance can be obtained.

Note that since the speech feature amount x _t ^, the variance Σ ′ _{n, m, t} of the Gaussian distribution of the acoustic model corrected with the adaptive parameter, and the average parameter μ _{n, m} are output for each frame, speech recognition is performed. The acoustic model storage unit 96 may not be provided.

〔simulation result〕
The word error rate (WER) of the speech recognition device using the speech parameter learning device of the present invention was evaluated. The recently proposed blind dereverberation method was used for speech enhancement. TI-Digit continuous digit recognition task was used as a speech recognition task. As the acoustic model, a word model was adopted, and an unspecified speaker acoustic model having 16 states per word and 3 Gauss distribution per state was constructed using clean speech. A sampling frequency was 8 kHz, and a 39-dimensional feature vector was used every 10 ms by using a 12-dimensional MFCC, a zeroth-order cepstrum, and their differential components and acceleration components as speech feature values. Note that CMN (Cepstral Mean Normalization) was applied to the voice feature amount.

  Reverberation sound was generated by convolving room transfer characteristics with clean sound. A transfer function measured in a room with a reverberation time of 0.5 seconds was used. The clean voice used the TI-Digit clean set. As test data, 561 utterances spoken by 104 male and female speakers were used. The average length of utterance is 6 seconds.

  FIG. 7 shows the recognition result evaluated by the word error rate. We compared the word error rates of clean speech, reverberation speech, dereverberation speech, distributed dynamic correction (no adaptation), and distributed dynamic correction (Oracle). Here, the oracle is an ideal value calculated from the feature amounts of the clean speech and the dereverberated speech for the feature amount dispersion necessary for the distributed dynamic correction. As shown in FIG. 7, the word error rate is slightly improved by performing dereverberation, but it can be seen that there is a large gap compared to the clean speech recognition result. On the other hand, the recognition performance can be greatly improved by using the conventional distributed dynamic correction, but there is still a big difference compared with the value of Oracle. The goal of this invention is to bring the recognition performance closer to this Oracle value.

  By using the adaptation data of the unspecified speaker, it is possible to adapt to the data emphasized by speech, not to the speaker. The adaptation data uses 520 utterances spoken by the same speaker as the test data. In order to examine the influence of the number of utterances, 2 to 512 utterances were randomly extracted from the adaptation data, and adaptation was performed using the adaptation data. FIG. 8 shows the word error rate by static distributed adaptation (SVA), dynamic distributed adaptation (DVA), and SDVA which is the method of the present invention. The horizontal axis is the number of utterances, and the vertical axis is the word error rate (WER). It can be seen that the recognition performance converges sufficiently with a small amount of utterances of about two utterances. Also, the use of static distributed application improves the word error rate from 31% (FIG. 7) to 15.2%. Even with the use of dynamic distributed application, it is improved to about 15.5%. According to the SDVA that simultaneously performs dynamic distribution application and static distribution application of the present invention, the word error rate can be further improved by about 2%. As a result, the error rate could be reduced to about half or less compared to the speech after dereverberation (31.0%) shown in FIG. Further, for the purpose of further improving the recognition rate, the combination of the distributed adaptation method of the present invention and the adaptation of average parameters by MLLR (Maximum Likelihood Linear Regression) was examined, and a result with a word error rate of 5% was obtained. The word error rate of 5% is close to the clean speech recognition rate (1.2%). Thus, the word error rate can be improved by using the speech parameter learning apparatus according to the present invention.

  The adaptive method described above focuses on the dispersion parameter, but can be combined with an adaptive method corresponding to other parameters such as an average parameter, a state transition rate, and a mixing weight factor.

  Moreover, the apparatus and method of this invention are not limited to the above-mentioned embodiment, It can change suitably in the range which does not deviate from the meaning of this invention. Further, the processes described in the above apparatus and method are not only executed in time series according to the order of description, but also may be executed in parallel or individually as required by the processing capability of the apparatus that executes the process. Good.

  Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

The figure which shows the function structural example of the speech parameter learning apparatus 100 of this invention. The figure which shows the operation | movement flow of the speech parameter learning apparatus 100. The figure which shows the more concrete function structural example of the audio | voice pre-processing part 2 for adaptation, and the adaptive parameter generation part 6. The figure which shows the operation | movement flow of FIG. The figure which shows the function structural example of the speech recognition apparatus 150 using the speech parameter learning apparatus 100. FIG. The figure which shows the operation | movement flow of the speech recognition apparatus 150. The figure which shows the recognition result evaluated by the word error rate. The figure which shows the word error rate by static dispersion | distribution adaptation (SVA), dynamic dispersion | distribution adaptation (DVA), and SDVA which is the method of this invention. The figure which shows the function structural example of the conventional speech recognition apparatus 200. FIG. The figure which shows the operation | movement flow of the speech recognition apparatus 200.

Claims (9)

Adaptive speech preprocessing that takes an observed speech signal as input and generates a set of emphasized speech features that emphasize speech features for each frame of the observed speech signal and a set of uncertainties that represent variations in the emphasized speech features And
An acoustic model storage unit storing an acoustic model;
The set of emphasized speech features, the set of uncertainties, the acoustic model, and the teacher signal are input, and a motion dependent on the frame is used as an adaptive parameter for dispersion correction of the Gaussian distribution in the acoustic model. An adaptive parameter generation unit for generating a static distributed adaptive parameter and a static distributed adaptive parameter independent of the frame,
A speech pre-processing unit for recognition that generates the uncertainty representing the variation of the speech feature amount and the speech feature amount of each frame of the observed speech signal, using the observed speech signal as an input;
A variance dynamic correction unit that receives the uncertainty of the speech feature, the adaptation parameter, and the acoustic model, and outputs a variance of the Gaussian distribution of the acoustic model corrected by the adaptation parameter for each frame;
A speech parameter learning apparatus comprising: The speech parameter learning device according to claim 1,
The adaptive speech preprocessing unit is
A speech enhancement unit that generates an enhanced speech signal in which speech features for each frame of the input observed speech signal are enhanced;
A feature amount calculation unit that calculates a feature amount for each frame of the observed audio signal;
An enhanced speech feature quantity calculating unit that computes an enhanced speech feature quantity for each frame of the enhanced speech signal and generates a set of enhanced speech feature quantities;
An uncertainty calculation unit that calculates an uncertainty representing the variation of the emphasized speech feature amount from the enhanced speech feature amount of the enhanced speech signal and the feature amount of the observed speech signal and generates a set of uncertainties of the enhanced speech feature amount And
The adaptive parameter generation unit
An occupancy probability calculation unit that receives the set of emphasized speech feature values, the set of uncertainties of the emphasized speech feature values, the acoustic model, and the teacher signal, and calculates the occupancy probability of the HMM state n and the mixed component m When,
A set of the emphasized speech feature amount, a set of uncertainties of the emphasized speech feature amount, a clean speech variance calculation unit that calculates the variance of clean speech using the acoustic model as an input, the variance of the clean speech, and the occupation A scaling factor λ calculator for calculating the scaling factor λ as the static variance adaptive parameter,
Using the set of emphasized speech feature values, the set of uncertainties of the emphasized speech feature values, and the acoustic model as input, the expected value of the square value of the difference between the clean speech feature and the speech feature value is calculated. A difference square value calculation unit;
A scaling factor α calculating unit that inputs the occupation probability and the square value of the difference and generates a scaling factor α as the dynamic dispersion adaptive parameter;
A speech parameter learning apparatus characterized by that. The speech parameter learning device according to claim 1 or 2,
A recognition unit that outputs the speech feature value output by the speech parameter learning device and the variance of the Gaussian distribution of the acoustic model corrected in the speech parameter learning device, and outputs a word string;
A speech recognition apparatus comprising: The adaptive speech preprocessing unit receives the observed speech signal as an input, and a set of emphasized speech features that emphasizes the speech features for each frame of the observed speech signal, and a set of uncertainties that represent variations in the enhanced speech feature values A speech preprocessing process for adaptation to generate
The adaptive parameter generation unit receives the set of emphasized speech features, the set of uncertainties, the acoustic model, and the teacher signal as inputs, and uses dynamic distribution depending on the frame as an adaptive parameter for dispersion correction. An adaptive parameter generation process for generating adaptive parameters and the frame-independent static distributed adaptive parameters;
A recognition speech pre-processing unit that receives the observed speech signal as an input and generates a speech feature value for each frame of the observed speech signal and a uncertainty representing a variation in the speech feature value;
A variance dynamic correction unit receives the uncertainty of the speech feature value, the adaptive parameter, and the acoustic model as input, and outputs a variance of the Gaussian distribution of the acoustic model corrected with the adaptive parameter for each frame Dynamic correction process;
A speech parameter learning method including: The speech parameter learning method according to claim 4, wherein
The adaptive speech preprocessing process is
A speech enhancement process in which a speech enhancement unit generates an enhanced speech signal in which speech features for each frame of the input observation speech signal are enhanced;
A feature amount calculating unit for calculating a feature amount for each frame of the observed audio signal;
An enhanced speech feature quantity calculating unit that computes an enhanced speech feature quantity for each frame of the enhanced speech signal to generate a set of enhanced speech feature quantities; and
An uncertainty calculation unit calculates an uncertainty representing the variation of the emphasized speech feature amount from the enhanced speech feature amount of the enhanced speech signal and the feature amount of the observed speech signal, and obtains a set of uncertainties of the enhanced speech feature amount. Including the uncertainty calculation process to generate,
The adaptive parameter generation process is as follows:
An occupancy probability calculation unit calculates the occupancy probability of the HMM state n and the mixed component m with the set of the emphasized speech feature amount, the uncertainties of the emphasized speech feature amount, the acoustic model, and the teacher signal as inputs. Occupancy probability calculation process to
A clean speech variance calculation process in which a clean speech calculation unit calculates the variance of clean speech using the set of the emphasized speech feature amount, the set of uncertainties of the emphasized speech feature amount, and the acoustic model as an input;
A scaling factor λ calculating section for calculating a scaling factor λ from the clean speech variance and the occupation probability;
The difference square value calculation unit receives the set of the emphasized speech feature, the set of uncertainties of the enhanced speech feature, and the acoustic model as a square of the difference between the clean speech feature and the speech feature. A difference square value calculation process for calculating an expected value of the value,
A scaling factor α calculating unit including a scaling factor α calculating step of generating the dynamic dispersion adaptive parameter by inputting the occupation probability, the uncertainty, and the square value of the difference;
A speech parameter learning method characterized by the above. The speech parameter learning method according to claim 4 or 5,
A recognition process in which the recognition unit receives the speech feature amount generated by the speech parameter learning method and the variance of the Gaussian distribution of the corrected acoustic model, and outputs a word string;
A speech recognition method comprising:   A program for causing a computer to function as the speech parameter learning device according to claim 1.   A program for causing a computer to function as the voice recognition apparatus according to claim 3.   A computer-readable recording medium on which the program according to claim 7 is recorded.

Images