JP2017151188A - Vocal tract spectrum estimation device, vocal tract spectrum estimation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely estimate a vocal tract spectrum from a speech signal.SOLUTION: An estimation part 23 estimate a vocal tract spectrum Hof each normalization angular frequency of each spectrum pattern and a weight Uat each time point of each spectrum pattern so as to make small a target function represented using a distance between an observation spectrum Yand a vocal tract spectrogram X.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a vocal tract spectrum estimation device, a vocal tract spectrum estimation method, and a program, and more particularly to a vocal tract spectrum estimation device, a vocal tract spectrum estimation method, and a program for estimating a vocal tract spectrum from a speech signal.

  In general voice processing including voice synthesis or voice conversion, a technique for estimating a vocal tract spectrum from a voice signal is used in many scenes. Assuming that the speech signal for each short interval can be modeled as the output of a linear time-invariant system with a periodic delta function (pulse train) as input, the input and impulse response of this linear system are the vocal cord source signal and vocal tract characteristics, respectively. Corresponding to This assumption is equivalent to representing a speech spectrum in the frequency domain by the product of a vocal cord source spectrum and a vocal tract spectrum represented by a periodic delta function. Therefore, the voice spectrum can be regarded as a periodic sampling of the vocal tract spectrum at basic frequency intervals.

  Based on this viewpoint, methods for estimating the vocal tract spectrum from the speech spectrum have been proposed so far. “STRAIGHT”, which is widely known as one of representative methods, is a method in which a speech signal is cut out with the width of the basic period, and the spectrum of the cut out signal, which is each cut out signal, is used as an estimated value of the vocal tract spectrum ( Non-patent document 1). This corresponds to the fact that in the frequency domain, the peak of each harmonic component is sinc interpolated and regarded as the vocal tract spectrum.

  However, it is known that the vocal tract spectrum estimation value obtained by STRAIGHT changes periodically with time depending on the offset of the cutout frame that cuts out the audio signal with the width of the basic period, even for stationary speech. ing. This is because the harmonic components interfere with each other. Such a fluctuation component of the spectrum which changes periodically with time is inevitably generated by frequency analysis using a finite window for the periodic signal, and should not be included in the vocal tract spectrum estimation value. Therefore, in the estimation of the vocal tract spectrum using STRAIGHT, an improved method has been proposed to remove the fluctuation component from the vocal tract spectrum estimated value (Non-patent Document 2).

As described above, since the voice spectrum can be regarded as a sample of the vocal tract spectrum at the basic frequency (F ₀ ) interval, the higher the voice F _{0, the} fewer clues for estimating the vocal tract spectrum. This suggests that there is an essential limit to independent processing for each frame.

On the other hand, since the same phoneme repeatedly appears in the audio signal, the appearance of similar vocal tract spectra at a plurality of different times is also a clue for estimating the vocal tract spectrum. It can be assumed that multiple frames have a common vocal tract spectrum, and if F ₀ is different in the multiple frames, the actually observable vocal tract spectrum sample points will increase compared to the case of a single frame, It is considered that the estimation accuracy of the vocal tract spectrum is improved.

  Based on this idea, a technique has been proposed for estimating vocal tract spectrum from multiple frames using articulatory motion data recorded simultaneously (Non-patent Document 3). As a similar technique, a vocal tract spectrum estimation method using a factor analysis trajectory hidden Markov model has been proposed (Non-Patent Document 4). In the vocal tract spectrum estimation method based on the factor analysis trajectory hidden Markov model, the context label attached to each frame of the speech signal is used, and in addition to the harmonic component information in multiple frames to which the same context is attached, the dynamic features The vocal tract spectrum is estimated by using the quantity as a clue.

H. Kawahara, I. Masuda-Katsuse, A. de Cheveigne, "Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds," Speech Commun ., 27, pp. 187-207, 1999. H. Kawahara, M. Morise, T. Takahashi, R. Nisimura, T. Irino and H. Banno, "Tandem-STRAIGHT: A temporally stable power spectral representation for periodic signals and applications to interference-free spectrum, F0, and aperiodicity estimation, "Proc. ICASSP, pp. 3933-3936, 2008. Y. Shiga and S. King, "Estimating the spectral envelope of voiced speech using multi-frame analysis," Proc. EUROSPEECH, pp. 1737-1740, 2003. T. Toda, "Statistical approach to vocal tract transfer function estimation based on factor analyzed trajectory HMM," Proc. ICASSP, pp. 3925-3928, 2008.

  However, with the vocal tract spectrum estimation method based on the factor analysis trajectory hidden Markov model, it is possible to estimate the vocal tract spectrum accurately by using the harmonic component information in multiple frames as a clue. There is a problem that enormous effort is required to assign a context label.

  The present invention has been made in view of the above circumstances, and provides a vocal tract spectrum estimation apparatus, a vocal tract spectrum estimation method, and a program capable of accurately estimating a vocal tract spectrum from a speech signal. Objective.

  In order to achieve the above object, the vocal tract spectrum estimation device according to the present invention cuts out time-series data of a speech signal with the width of the basic period, and extracts each time and each normalized angle from the spectrum of each cut-out speech signal. Observation spectrogram estimator that outputs an observation spectrogram representing the frequency component of the observation time of the frequency, the observation spectrogram output by the observation spectrogram estimator, and the vocal tract spectrum that represents the power spectrum of each normalized angular frequency in each spectrum pattern , And each spectral pattern so as to reduce the objective function expressed using the distance from each time and the vocal tract spectrogram representing the time frequency component of each normalized angular frequency obtained from the weight of each spectral pattern at each time The vocal tract spectrum of each normalized angular frequency in And it is configured to include an estimation unit that estimates a weight, the at each time of each spectral pattern.

  A vocal tract spectrum estimation method according to the present invention is a vocal tract spectrum estimation method in a vocal tract spectrum estimation apparatus including an observation spectrogram estimation unit and an estimation unit, wherein the observation spectrogram estimation unit converts time series data of a speech signal. , Cut out with the width of the basic period, and output an observation spectrogram representing the observation time frequency component of each time and each normalized angular frequency from the spectrum of each cut out audio signal, and the estimation unit outputs by the observation spectrogram estimation unit The time spectrum component of each time and each normalized angular frequency obtained from the observed spectrogram, the vocal tract spectrum representing the power spectrum of each normalized angular frequency in each spectrum pattern, and the weight of each spectrum pattern at each time. Expressed using the distance to the representing vocal tract spectrogram That as the objective function to reduce, to estimate the weights at each time of the vocal tract spectrum, and the spectrum pattern of each normalized angular frequency at the respective spectral pattern.

  The program according to the present invention is a program for causing a computer to function as each part of the above vocal tract spectrum estimation apparatus.

  As described above, according to the vocal tract spectrum estimation apparatus, the vocal tract spectrum estimation method, and the program of the present invention, the objective function expressed using the distance between the observation spectrogram and the vocal tract spectrogram is reduced. By estimating the vocal tract spectrum of each normalized angular frequency in each spectrum pattern and the weight of each spectrum pattern at each time, it is possible to accurately estimate the vocal tract spectrum from the speech signal. .

It is the schematic which shows the structure of the vocal tract spectrum estimation apparatus which concerns on embodiment of this invention. It is a flowchart which shows an example of the process of the vocal tract spectrum estimation process routine using the estimation algorithm of the vocal tract spectrum with respect to GMM-NMF. It is a flowchart which shows an example of the process of the vocal tract spectrum estimation processing routine using the vocal tract spectrum estimation algorithm with respect to AR-NMF. It is a figure which shows the evaluation result of the vocal tract spectrum estimated by the vocal tract spectrum estimation apparatus.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<Outline of Embodiment of the Present Invention>
Based on the assumption that the vocal tract spectrogram can be approximated by a low-rank non-negative matrix, without using the context label attached to the speech signal, the information on the harmonic components in multiple frames of the speech signal can be used as a clue. Estimate the road spectrum accurately.

<Estimation of vocal tract spectrum by interpolation of missing data>
<Modeling of vocal tract spectrogram using low rank non-negative matrix>
First, vocal tract spectrogram modeling using a low rank non-negative matrix will be described. The time index is t (t = 0,..., T-1), and the normalized angular frequency corresponding to the frequency index k (k = 0,..., K-1) is represented as ω _k .

Since the types of phonemes in speech are limited, the same vocal tract spectrum appears at a plurality of different times in one speech. This situation can be interpreted as equivalent to being able to represent the vocal tract spectrogram with a low-rank non-negative matrix. Therefore, for example, a non-negative matrix H = (H _{k, r} ) _{k, r in} which R smooth spectral patterns are arranged in the column direction, and a non-negative value weight U = (U _{r, t} ) _{r, r, for} each spectral pattern _{By t} , the vocal tract spectrogram X _{k, t} can be expressed by equation (1).

  R is an index of the spectrum pattern.

H _{k, r that} is smooth and non-negative in the frequency direction can be designed in various _ways, but in this embodiment, two types of H _{k, r} are proposed.

The first proposal defines H _{k, r} with a mixed normal distribution function, as shown in equations (2) and (3).

Here, ω is a normalized angular frequency, N is the number of mixtures, and n (n = 0,..., N−1) is an index of the number of mixtures N. h (ω) is a frequency warping function. If h (ω) = ω, G _n (ω) is a function having the same shape as a normal distribution with mean ρ _n and variance ν ² _{n in} the linear frequency domain. In the present embodiment, the frequency warping function h (ω) is set, for example, as in equation (4) so that a smooth vocal tract spectrum is obtained in the mel frequency region.

Here, f _s is a sampling frequency, and the normalized frequency of [0, π] is mapped to [0,1] by the equation (4). Also, W = (W _{r, n} ) _{r, n} ≧ 0 is the weight of each normal distribution, and Σ _n W _{r, n} for each r in order to eliminate the arbitraryness of the scale regarding the non-negative matrix H and the weight U. = 1.

The second proposal is a method using an all-pole filter often used in a source filter model. Specifically, if the coefficients a _r : = [ _{ar, 0} , a _{r, 1} ,..., A _{r, P} ] ^T of the Pth order all-pole filter are used, the amplitude spectrum H _{k, r} ^(AR) , that is, the vocal tract spectrum H _{k, r} ^(AR) is expressed by equation (5).

  Here, Q (ω) is a (P + 1) × (P + 1) Toeplitz matrix in which the (p, q) component is represented by cos (ω (p−q)).

<Non-negative matrix factorization approach to missing data>
_Given the vocal tract spectrogram Y = (Y _{k, t} ) _{k, t} estimated by STRAIGHT, the estimation problem of the vocal tract spectrogram is the given vocal tract spectrogram Y _{k, t} and the estimated vocal tract spectrogram X _Using the distance D _* (Y _{k, t} ; X _{k, t} ) with _{k, t} , it can be formulated as in equation (6).

Here, Θ is a parameter set, and when H _{k, r} ^(GMM) in the equation (2 ⁾ is used as the vocal tract spectrum H _{k, r} , Θ = {W, U} and the vocal tract spectrum H _When H _{k, r} ^(AR) in the equation (5) is used as _{k, r} , Θ = {{ _ar } _r , U}. Hereinafter, for convenience of explanation, the non-negative matrix factorization (NMF) when H _{k, r} ^(GMM) is used is “GMM-NMF”, and the NMF when H _{k, r} ^(AR) is used. Is called “AR-NMF”. Z _{k, t} ε [0,1] is a parameter representing the reliability of each time frequency component.

Note that if Z _{k, t} = 0 in equation (6), the cost for the vocal tract spectrogram Y _{k, t} estimated by STRAIGHT is not considered, and the time frequency component with a larger Z _{k, t} is more important.

As a simple design method of the reliability Z _{k, t} , Z _{k, t} = 1 is used for the time frequency component corresponding to the fundamental frequency F ₀ and its harmonic frequency at each time, and other time frequency components are used. A method in which Z _{k, t} = ξ (where ξ is a constant between 0 and 1) is conceivable. The value of ξ can be determined experimentally and empirically. In this embodiment, a method of designing using the aperiodicity index A _{k, t} ∈ [0,1] obtained by STRAIGHT will be described. Since the non-periodic index A _{k, t} is the ratio of the non-periodic component included in each time-frequency component, if Z _{k, t} = 1−A _{k, t} for each k, r, the periodic component is It is possible to estimate an important vocal tract spectrogram X _{k, t} .

Examples of the distance D _* (Y _{k, t} ; X _{k, t} ) widely known in NMF include a generalized Kullback-Leibler (KL) divergence D _GKL or a square distance D _EU . The objective function L _GKL (Θ) when using the generalized KL divergence D _GKL is shown in Equation (7), and the objective function L _EU (Θ) when using the square distance D _EU is shown in Equation (8). .

<Parameter estimation algorithm by auxiliary function method>
Next, a parameter estimation algorithm for GMM-NMF and a parameter estimation algorithm for AR-NMF will be described.

<Derivation of iterative algorithm for GMM-NMF>
Since the second term in the right parenthesis in the objective function L _GKL (Θ) when using the generalized KL divergence D _GKL in the above equation (7) includes an addition operation in the logarithmic function, (7) It is difficult to directly solve the optimization problem that minimizes the objective function of the equation.

  However, as is done in many NMF studies, it is possible to iteratively apply the optimization principle called auxiliary function method to an objective function that is difficult to solve the optimization problem directly. It is known that an optimal solution can be obtained.

In the auxiliary function method, an auxiliary variable λ is introduced to the objective function L (Θ) of the parameter Θ, and an auxiliary function L ⁺ (that defines an upper bound satisfying L (Θ) = min _λ L ⁺ (Θ, λ). Θ, λ) is derived. By alternately minimizing the auxiliary function L ⁺ (Θ, λ) with respect to the parameter Θ and the auxiliary variable λ, the objective function L (Θ) can be decreased monotonously in a broad sense.

  Since the logarithmic function is a concave function, the upper bound of the second term in the right parenthesis of the above equation (7) is expressed by equation (9) using Jensen's inequality.

Here, λ _{k, t, r, n} ≧ 0 is an auxiliary variable, and Σ _{r, n} λ _{k, t, r, n} = 1 is satisfied for each k, t. It should be noted that the condition for satisfying the equation (9) is satisfied when the equation (10) is satisfied.

Therefore, the auxiliary function for the objective function L _GKL (Θ) is expressed by equation (11).

Here, it is defined as λ: = {λ _{k, t, r, n} }. By substituting the equation (10) by obtaining a value at which the partial differentiation with respect to W _{r, n} and U _{r, t} of the auxiliary function of the equation (11) is 0, the closure shown in the equations (12) and (13) The form update formula is obtained.

Since the updating formulas of the equations (12) and (13) are all calculated as products of non-negative values, the non-negative values of W _{r, n} and U _{r, t} are natural if the initial values are set to non-negative values. And kept.

Next, an update formula of the objective function L _EU (Θ) when the square distance D _EU is used as the distance D _* (Y _{k, t} ; X _{k, t} ) will be examined. Since equation (8) also includes an addition operation in the second term in the right parenthesis, it is difficult to directly solve the optimization problem that minimizes the objective function of equation (8).

However, since the quadratic function is a convex function, an auxiliary function can be designed in the same manner as the objective function L _GKL (Θ) by using Jensen's inequality for the above equation (8). The design of the auxiliary function is omitted because it is the same as the design method of the auxiliary function in the objective function L _GKL (Θ). However, the update equations for minimizing the equation (8) are the equations (14) and (15). expressed.

<Derivation of iterative algorithm for AR-NMF>
Regarding AR-NMF, a closed-form update formula can be derived in the same way as the iterative algorithm derivation method for GMM-NMF.

First, the auxiliary function L ⁺ _{GKL, AR} (Θ, ξ) for the generalized KL divergence objective function L _GKL (Θ) is examined. The auxiliary function L ⁺ _{GKL, AR} (Θ, ξ) is a non-negative auxiliary variable ξ = {ξ _{k, t, r} } _{k, t, r} satisfying Σ _r ξ _{k, t, r} = 1 for each k, t. Is defined by the equation (16).

Further, the condition for establishing the equation (16) is ξ _{k, t, r} = H ^(AR) _{k, r} (U _{r, t} / X _{k, t} ). The update equation for U _{r, t} is the same as that obtained by replacing H _{k, r} ^(GMM) in equation (13) with H _{k, r} ^(AR) , and is represented by equation (17).

On the other hand, when deriving an iterative algorithm for AR-NMF, a multiplicative update algorithm can be used for updating a _r .

The partial differentiation of the generalized KL divergence objective function L _GKL (Θ) shown in equation (7) with respect to a _r can be expressed by equations (18) to (20).

The first and second terms in the right parenthesis of equation (18) are both positive definite matrices, and the objective function L _GKL (Θ, ξ) is decreased monotonously in a broad sense by using the multiplicative update measurement of equation (21). Can be made.

Although details are omitted, the update formula when the square distance D _EU is used can be derived in the same manner as in the case of the generalized KL divergence. Specifically, the update formula for U _{r, t} is expressed by formula (22) in which H _{k, r} ^(GMM) in formula (15 ⁾ is replaced with H _{k, r} ^(AR), and the update formula for a _r Becomes the equation (23) in the same manner as the equation (21).

<First Embodiment>
<System configuration>
Next, a case where the present invention is applied to a vocal tract spectrum estimation apparatus that estimates the vocal tract spectrum using information on harmonic components in a plurality of frames of a speech signal as a clue is described as a first embodiment of the present invention. Will be described. In the first embodiment, the distance D _* (Y _{k, t} ; X _{k, t} ) is a voice using a vocal tract spectrum estimation algorithm for the GMM-NMF when the generalized KL divergence D _GKL is used. An example of the road spectrum estimation apparatus will be described.

  As shown in FIG. 1, the vocal tract spectrum estimation apparatus according to the first embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program for executing a vocal tract spectrum estimation processing routine described later. And is functionally configured as follows.

  The vocal tract spectrum estimation apparatus 100 includes an input unit 10, a calculation unit 20, a storage unit 30, and an output unit 40.

  The input unit 10 inputs time-series data of a speech signal that is an object of vocal tract spectrum estimation. The storage unit 30 stores time series data of the audio signal input by the input unit 10. In addition, the storage unit 30 stores the result of each process to be described later, and stores the initial value of each parameter used in this process routine.

  The calculation unit 20 includes an observation spectrogram estimation unit 21, an initial setting unit 22, an estimation unit 23, an end determination unit 24, and an output unit 25.

The observation spectrogram estimation unit 21 receives the time series data of the speech signal collected by the input unit 10 and calculates the vocal tract spectrogram Y _{k, t} estimated using STRAIGHT, which is a known vocal tract spectrum estimation method. Further, the calculated vocal tract spectrogram Y _{k, t} is stored in the storage unit 30. More specifically, the observation spectrogram estimation unit 21 cuts out the time-series data of the voice signal collected by the input unit 10 with the width of the basic period, and from each cut-out voice signal, each time t and each normalized angular frequency ω _k. The vocal tract spectrogram Y _{k, t} representing the observed time frequency components of is calculated. The vocal tract spectrogram Y _{k, t} is an example of an observation spectrogram according to the present embodiment.

In addition, when calculating the vocal tract spectrogram Y _{k, t} , the observation spectrogram estimation unit 21 calculates an aperiodic index A _{k, t} for each observation time frequency component, and Z _{k, t} = 1−A _{k, t} To calculate the reliability Z _{k, t} of each time-frequency component. The calculated reliability Z _{k, t} of each time frequency component is stored in the storage unit 30.

The initial setting unit 22 sets initial values of parameters W _{r, n} , U _{r, t} and G _n (ω _k ) used in processing to be described later. Note that W _{r, n} is a non-negative value, and an initial value is set so as to satisfy Σ _n W _{r, n} = 1. U _{r, t is} also a non-negative value, and an initial value is set to an appropriate value using, for example, a random number. Note that r (r = 0,..., R−1) is a spectrum index indicating R spectrum patterns. For G _n (ω _k ), an initial value is set so as to satisfy the above expression (3). At this time, appropriate values may be used for the average ρ _n and the variance ν ² _n . The initial values of the set parameters W _{r, n} , U _{r, t} and G _n (ω _k ) are stored in the storage unit 30.

Based on W _{r, n} and G _n (ω _k ) stored in the storage unit 30 for each of all combinations of (k, r), the estimation unit 23 performs the vocal tract according to the above equation (2). A spectrum H _{k, r} ^(GMM) (hereinafter simply referred to as “H _{k, r} ”) is calculated and stored in the storage unit 30.

Based on H _{k, r} and U _{r, t} stored in the storage unit 30 for each combination of (k, t), the estimation unit 23 performs the vocal tract spectrogram X according to the above equation (1). _{k and t} are calculated and stored in the storage unit 30.

For each of all combinations of (r, n), the estimation unit 23 stores W _{r, n} , Z _{k, t} , X _{k, t} , Y _{k, t} , G _n (ω _k ), U _{r, t} , the normal distribution weight W _{r, n} is updated according to the above equation (12) and stored in the storage unit 30.

With the update of the normal distribution weight W _{r, n} , the estimation unit 23 performs the above (2) based on W _{r, n} and G _n (ω _k ) for all the combinations of (k, r). The vocal tract spectrum H _{k, r} is calculated according to the equation and stored in the storage unit 30.

With the update of the vocal tract spectrum H _{k, r} , the estimator 23 _calculates , based on H _{k, r} and U _{r, t} stored in the storage unit 30 for each combination of (k, t). The vocal tract spectrogram X _{k, t} is calculated according to the above equation (1) and stored in the storage unit 30.

For each of all combinations of (r, t), the estimating unit 23 stores U _{r, t} , Z _{k, t} , X _{k, t} , Y _{k, t} , H ^(GMM) stored in the storage unit 30. _{Based on k, r} , that is, H _{k, r} , the spectrum pattern weights U _{r, t} of the vocal tract spectrogram X _{k , t} are updated according to the above equation (13) and stored in the storage unit 30.

Further, the estimation unit 23 updates the spectral pattern weights U _{r, t} of the vocal tract spectrogram X _{k, t} with the H stored in the storage unit 30 for each of all combinations of (k, t). _{Based on k, r} and U _{r, t} , a vocal tract spectrogram X _{k, t} is calculated according to the above equation (1) and stored in the storage unit 30.

  The end determination unit 24 determines whether or not a predetermined end condition is satisfied. If the end condition is not satisfied, each process of the estimation unit 23 is repeated. When it is determined that the end condition is satisfied, the end determination unit 24 proceeds to processing by the output unit 25.

The output unit 25 outputs the vocal tract spectrogram X _{k, t} stored in the storage unit 30 as a vocal tract spectrogram estimated from the speech signal input to the input unit 10.

  As an end condition, the difference between the value of the objective function (7) with the number of iterations L-1 and the value of the objective function (7) with the number of iterations L is less than a predetermined threshold. What has become smaller can be used. Alternatively, the termination condition may be that the number of repetitions has reached a predetermined upper limit number.

<Operation of vocal tract spectrum estimation device>
Next, the operation of the vocal tract spectrum estimation apparatus 100 according to the first embodiment using the vocal tract spectrum estimation algorithm for GMM-NMF will be described.

  The time-series data of the voice signal acquired by the microphone is input to the vocal tract spectrum estimation apparatus 100 and stored in the storage unit 30. Then, the vocal tract spectrum estimation apparatus 100 executes a vocal tract spectrum estimation processing routine shown in FIG.

First, in step S100, the time series data of the speech signal is read from the storage unit 30, the vocal tract spectrum is estimated by STRAIGHT for the time series data of the speech signal, and each time t and each normalized angular frequency is estimated. The vocal tract spectrogram Y _{k, t} representing the observation time frequency component of ω _k (k = 0,..., K−1) is calculated, and the obtained vocal tract spectrogram Y _{k, t} is stored in the storage unit 30. To do.

In step S102, an aperiodic index A _{k, t} corresponding to the vocal tract spectrogram Y _{k, t} obtained in step S100 is calculated, and the reliability of each time-frequency component is calculated by Z _{k, t} = 1−A _{k, t.} The degree Z _{k, t} is calculated, and the obtained reliability Z _{k, t} is stored in the storage unit 30.

In step S104, initial values of W _{r, n} and U _{r, t} are set using random numbers. Incidentally, W _{r, n} and U _r, the initial value of _t is set to non-negative values, W _r, for the _n, sets the initial value so as to satisfy the _{Σ n W r, n = 1} . For G _n (ω _k ), an initial value is set so as to satisfy the above expression (3). At this time, appropriate values may be used for the average ρ _n and the variance ν ² _n .

The initial values of W _{r, n} , U _{r, t} and G _n (ω _k ) set in this way are stored in the storage unit 30.

Next, in step S106, the vocal tract spectrum H _{k, r} is calculated for each (k, r) according to the above equation (2) based on W _{r, n} and G _n (ω _k ) set in step S104. The combination is calculated and stored in the storage unit 30. In step S106, the vocal tract spectrograms X _{k, t} are each (k) according to the above equation (1) based on U _{r, t} set in step S104 and H _{k, r} calculated in this step. , T) is calculated and stored in the storage unit 30.

In step S108, W _{r, n} , G _n (ω _k ) and U _{r, t} set in step S104, Z _{k, t} calculated in step S102, X _{k, t} calculated in step S106, and Y _{k, t} calculated in step S100, that is, the latest parameters W _{r, n} , Z _{k, t} , X _{k, t} , Y _{k, t} , G _n (ω _k) stored in the storage unit 30. ), U _{r, t} , the normal distribution weight W _{r, n} is calculated for each (r, n) combination according to the above equation (12), and stored in the storage unit 30.

In step S110, as in step S106, the latest parameters W _{r, n} and G _n (ω _k ) stored in the storage unit 30, that is, W _{r, n} calculated in step S108 and step S104 are used. Based on the set G _n (ω _k ), the vocal tract spectrum H _{k, r} is calculated for each (k, r) combination according to the above equation (2) and stored in the storage unit 30. In step S110, the latest parameters U _r in the storage unit 30 is _{stored, t} and H _{k, r,} i.e., H _k calculated in U _{r, t,} and the step set in step S104 _{, r} , the vocal tract spectrogram X _{k, t} is calculated for each (k, t) combination and stored in the storage unit 30 according to the above equation (1).

In step S112, the latest parameters Z _{k, t} , X _{k, t} , Y _{k, t} , H _{k, r} , U _{r, t} stored in the storage unit 30, that is, the U set in step S104. _{Based on r, t} , Z _{k, t} calculated in step S102, X _{k, t} and H _{k, r} calculated in step S110, and Y _{k, t} calculated in step S100, according to the above equation (13), The spectral pattern weights U _{r, t} of the vocal tract spectrogram X _{k, t} are calculated for each (r, t) combination and stored in the storage unit 30.

In step S114, the latest parameters U _r in the storage unit 30 is _{stored, t} and H _{k, r,} i.e., U _r calculated in step _{S112, t,} and H _k calculated in step _{S110, r} Based on the above, the vocal tract spectrogram X _{k, t} is calculated for each (k, t) combination according to the above equation (1) and stored in the storage unit 30.

In the next step S116, based on Y _{k, t} calculated in step S100, Z _{k, t} calculated in step S102, and X _{k, t} calculated in step S114, the objective function L _{GKL is obtained} according to equation (7). The value of (Θ) is calculated and stored in the storage unit 30. Then, the value of the objective function L _GKL (Θ) calculated in the previous step S116 is read from the storage unit 30, and the value of the objective function L _GKL (Θ) calculated in the current step S116 and calculated in the previous step S116. It is determined whether or not the difference from the value of the objective function L _GKL (Θ) is smaller than a predetermined threshold stored in advance in the storage unit 30, and if the difference is greater than or equal to a predetermined threshold If it is determined that the end condition is not satisfied, the process returns to step S108, and the processes of steps S108 to S116 are repeated.

On the other hand, if the difference is less than a predetermined threshold value, it is determined that the end condition is satisfied, and in step S118, the latest vocal tract spectrogram X _{k, t} calculated in step S114 is output. The spectrum estimation processing routine ends.

In the above description, a vocal tract spectrum estimation apparatus that uses a vocal tract spectrum estimation algorithm for GMM-NMF when the generalized KL divergence D _GKL is used as the distance D _* (Y _{k, t} ; X _{k, t} ). However, it is needless to say that the square distance D _EU may be used as the distance D _* (Y _{k, t} ; X _{k, t} ).

In this case, for example, step S108 is replaced with the calculation of the update equation for W _{r, n} shown in equation (14), and step S112 is replaced with the calculation of the update equation for U _{r, t} shown in equation (15).

<Second Embodiment>
<System configuration>
Next, a vocal tract spectrum estimation apparatus according to the second embodiment will be described. In the second embodiment, as a distance D _* (Y _{k, t} ; X _{k, t} ), a vocal tract using a vocal tract spectrum estimation algorithm for AR-NMF when a generalized KL divergence D _GKL is used. An example of a spectrum estimation apparatus will be described. The vocal tract spectrum estimation apparatus according to the second embodiment of the present invention is similar to the system configuration of the vocal tract spectrum estimation apparatus according to the first embodiment shown in FIG. 20, a storage unit 30, and an output unit 40. In addition, the calculation unit 20 includes an observation spectrogram estimation unit 21, an initial setting unit 22, an estimation unit 23, an end determination unit 24, and an output unit 25.

  Since the input unit 10 and the observation spectrogram estimation unit 21 are the same as those in the first embodiment, description thereof is omitted.

The initial setting unit 22 sets initial values of parameters a _r and U _{r, t} used in processing to be described later. Here, the initial value of the P-th order all-pole filter coefficient a _r is set to an appropriate value using, for example, a random number. The setting of each initial value of U _{r, t} is the same as described in (System configuration 1). The initial values of the set parameters a _r and U _{r, t} are stored in the storage unit 30.

Based on a _r stored in the storage unit 30 for each of all combinations of (k, r), the estimation unit 23 performs vocal tract spectrum H _{k, r} ^(AR) ( Hereinafter, it is simply described as “H _{k, r} ”) and stored in the storage unit 30.

Based on H _{k, r} and U _{r, t} stored in the storage unit 30 for each combination of (k, t), the estimation unit 23 performs the vocal tract spectrogram X according to the above equation (1). _{k and t} are calculated and stored in the storage unit 30.

The estimation unit 23 follows the above equation (20) based on a _r , Z _{k, t} , H _{k, r} , X _{k, t} , Y _{k, t} , U _{r, t} stored in the storage unit 30. The all-pole filter coefficient a _r is updated and stored in the storage unit 30.

As the all-pole filter coefficient a _r is updated, the estimation unit 23 calculates the vocal tract spectrum H _{k, r} according to the above equation (5) based on a _r for each combination of (k, r). Calculate and store in the storage unit 30.

With the update of the vocal tract spectrum H _{k, r} , the estimator 23 _calculates , based on H _{k, r} and U _{r, t} stored in the storage unit 30 for each combination of (k, t). The vocal tract spectrogram X _{k, t} is calculated according to the above equation (1) and stored in the storage unit 30.

For each of all combinations of (r, t), the estimation unit 23 stores U _{r, t} , Z _{k, t} , X _{k, t} , Y _{k, t} , H ^(AR) stored in the storage unit 30. _{Based on k, r} , that is, H _{k, r} , the spectrum pattern weights U _{r, t} of the vocal tract spectrogram X _{k , t} are updated according to the above equation (17) and stored in the storage unit 30.

Further, the estimation unit 23 updates the spectral pattern weights U _{r, t} of the vocal tract spectrogram X _{k, t} with the H stored in the storage unit 30 for each of all combinations of (k, t). _{Based on k, r} and U _{r, t} , a vocal tract spectrogram X _{k, t} is calculated according to the above equation (1) and stored in the storage unit 30.

  The end determination unit 24 determines whether or not a predetermined end condition is satisfied. If the end condition is not satisfied, each process of the estimation unit 23 is repeated. When it is determined that the end condition is satisfied, the end determination unit 24 proceeds to processing by the output unit 25.

The output unit 25 outputs the vocal tract spectrogram X _{k, t} stored in the storage unit 30 as a vocal tract spectrogram estimated from the speech signal input to the input unit 10.

<Operation of vocal tract spectrum estimation device>
Next, the operation of the vocal tract spectrum estimation apparatus 100 according to the second embodiment that uses the vocal tract spectrum estimation algorithm for AR-NMF will be described.

  The time-series data of the voice signal acquired by the microphone is input to the vocal tract spectrum estimation apparatus 100 and stored in the storage unit 30. Then, the vocal tract spectrum estimation apparatus 100 executes a vocal tract spectrum estimation processing routine shown in FIG.

  Steps S100, S102, and S118 of the vocal tract spectrum estimation processing routine shown in FIG. 3 are the same as the corresponding steps of the vocal tract spectrum estimation processing routine of FIG.

In step S105, initial values of a _r and U _{r, t} are set using random numbers. At this time, the initial values of a _r and U _{r, t} are set to non-negative values.

The initial values of a _r and U _{r, t} set in this way are stored in the storage unit 30.

Next, in step S107, the vocal tract spectrum H _{k, r} is calculated for each (k, r) combination according to the above equation (5) based on a _r set in step S105, and the storage unit 30 To store. In step S107, the vocal tract spectrograms X _{k, t} are each (k) according to the above equation (1) based on U _{r, t} set in step S105 and H _{k, r} calculated in this step. , T) is calculated and stored in the storage unit 30.

In step S109, Y _{k, t} calculated in step S100, Z _{k, t} calculated in step S102, a _r and U _{r, t} set in step S105, and H _{k, r} calculated in step S107. And X _{k, t} , that is, based on the latest parameters a _r , Z _{k, t} , X _{k, t} , Y _{k, t} , H _{k, r} , U _{r, t} stored in the storage unit 30. The all-pole filter coefficient a _r is calculated according to the above equation (21) and stored in the storage unit 30.

In step S111, the latest a _r in the storage unit 30 is stored, i.e., based on a _r calculated in step S109, according to the above (5), the vocal tract spectrum H _k, each of _r (k, The combination of r) is calculated and stored in the storage unit 30. In step S111, the latest parameters U _r in the storage unit 30 is _{stored, t} and H _{k, r,} i.e., H _k calculated in U _{r, t,} and the steps set in step S105 _{, r} , the vocal tract spectrogram X _{k, t} is calculated for each (k, t) combination and stored in the storage unit 30 according to the above equation (1).

In step S113, the latest parameters Z _{k, t} , X _{k, t} , Y _{k, t} , H _{k, r} , U _{r, t} stored in the storage unit 30, that is, U set in step S105 _{Based on r, t} , Z _{k, t} calculated in step S102, X _{k, t} and H _{k, r} calculated in step S111, and Y _{k, t} calculated in step S100, according to the above equation (17), The spectral pattern weights U _{r, t} of the vocal tract spectrogram X _{k, t} are calculated for each (r, t) combination and stored in the storage unit 30.

In step S115, the latest parameters U _r in the storage unit 30 is _{stored, t} and H _{k, r,} i.e., U _r calculated in step _{S113, t,} and H _k calculated in step S _{111, r} Based on the above, the vocal tract spectrogram X _{k, t} is calculated for each (k, t) combination according to the above equation (1) and stored in the storage unit 30.

In the next step S117, based on Y _{k, t} calculated in step S100, Z _{k, t} calculated in step S102, and X _{k, t} calculated in step S115, the objective function L _{GKL is obtained} according to equation (7). The value of (Θ) is calculated and stored in the storage unit 30. Then, the value of the objective function L _GKL (Θ) calculated in the previous step S117 is read from the storage unit 30, and the value of the objective function L _GKL (Θ) calculated in the current step S117 is calculated in the previous step S117. It is determined whether or not the difference from the value of the objective function L _GKL (Θ) is smaller than a predetermined threshold stored in advance in the storage unit 30, and if the difference is greater than or equal to a predetermined threshold If it is determined that the end condition is not satisfied, the process returns to step S109, and the processes of steps S109 to S117 are repeated.

On the other hand, if the difference is less than the predetermined threshold value, it is determined that the end condition is satisfied, and the latest vocal tract spectrogram X _{k, t} calculated in step S115 is output in step S118, and the vocal tract is calculated. The spectrum estimation processing routine ends.

In the above description, the vocal tract spectrum estimation apparatus using the vocal tract spectrum estimation algorithm for AR-NMF when the generalized KL divergence D _GKL is used as the distance D _* (Y _{k, t} ; X _{k, t} ). However, it is needless to say that the square distance D _EU may be used as the distance D _* (Y _{k, t} ; X _{k, t} ).

In this case, for example, step S109 is replaced with the calculation of the update equation for a _r shown in equation (23), and step S113 is replaced with the calculation of the update equation for U _{r, t} shown in equation (22).

<Voice tract spectrum estimation accuracy evaluation experiment>
<Conditions for evaluation experiment>
Next, the vocal tract spectrum estimated by the proposed method and the vocal tract estimated by STRAIGHT for the purpose of showing the effectiveness of the vocal tract spectrum estimation method (hereinafter referred to as “proposed method”) according to the first embodiment. An evaluation experiment was performed to compare the estimation accuracy of spectra.

From A set of ATR digital speech database, 20 Japanese speech signals (sampling frequency is 16kHz) by one Japanese female speaker are analyzed by STRAIGHT method, fundamental frequency F ₀ , vocal tract spectrum, aperiodicity index A _{k, t} were extracted. The spectrum obtained here is regarded as a correct vocal tract spectrum.

Then, using the correct vocal tract spectrum and F ₀ multiplied by 2 ^−1.0 , 2 ^−0.5 , 2 ^0.0 , 2 ^0.5 , 2 ^1.0 , and 2 ^1.5 , the speech signals were recombined with STRAIGHT.

  Then, the vocal tract spectrum is estimated from the re-synthesized speech signal using STRAIGHT and the proposed method, and the vocal tract spectrum estimation performance of each method is compared using the mel cepstrum distortion between the vocal tract spectrum estimated value and the correct vocal tract spectrum. did. The mel cepstrum distortion is calculated using first to 24th mel cepstrum coefficients. In the estimation of the vocal tract spectrum by STRAIGHT, the frame shift is 5 ms (T = 81761) and the dimension of the vocal tract spectrum is K = 513. It was.

In addition, as a proposed method, evaluation was performed when a generalized KL divergence D _GKL and a square distance D _EU were used in the parameter estimation algorithm for GMM-NMF. Specifically, using the vocal tract spectrum of the re-synthesized speech signal estimated by STRAIGHT and simultaneously using the vocal tract spectrogram Y _{k, t} of 20 utterances for each constant multiple of F ₀ , W _{r, n} and U _{r and t} were estimated. Further, the spectrum pattern R = 90, the number of mixtures N = 100, the average ρ _n = n / (N−1) and the standard deviation ν _n = 1 / (N− with respect to n = 0,. 1). W _{r, n} and U _{r, t} were initialized with non-negative random numbers, and the number of iterations of the vocal tract spectrum estimation algorithm in the proposed method was 100.

<Results of evaluation experiment>
FIG. 4 shows the mel-cepstral distortion of the vocal tract spectrum estimation result by the proposed method and STRAIGHT. The column “GKL” is the evaluation result when using the generalized KL divergence with respect to the magnification x of F ₀ , and the column “EU” is the evaluation result when using the square distance with respect to the magnification x of F _0. , And “STRAIGHT” columns represent evaluation results by STRAIGHT. Each evaluation result is described in the form of [average value ± standard deviation] [dB], and the value in parentheses indicates the evaluation result when the non-periodicity index A _{k, t} is not used. .

, According to the proportion x of F ₀ is increased, since the reduced observable harmonics, the effect due to the use of the harmonic components other than the frame is considered to appear, as shown in FIG. 4, F ₀ The higher the magnification x, the smaller the mel cepstrum distortion in the GKL evaluation result compared to the evaluation result using STRAIGHT, and the harmonic components other than the frame are more effective in estimating the vocal tract spectrum. I can confirm.

Also in the EU evaluation results, the mel cepstrum distortion becomes smaller as the magnification x of F ₀ becomes higher than the evaluation results using STRAIGHT. However, the average mel cepstrum distortion tends to be larger than the GKL evaluation results, and the parameter estimation algorithm for GMM-NMF using generalized KL divergence is more suitable for estimation of vocal tract spectrum. It can be said that

Further, when the non-periodicity index A _{k, t} is uniform for all time frequency components, that is, when Z _{k, t} = 1 for all (k, t) combinations, the estimation of each speech spectrum is performed. In the technique, the value in parentheses is obtained.

The evaluation results of GKL, for even magnification x of any F _0, aperiodic index A _k, is preferable to use a _t, aperiodic index A _k, less than mel-cepstrum distortion in the case of a uniform _t Therefore, it can be confirmed that the non-periodicity index _{Ak, t} contributes to the performance improvement regarding the estimation of the vocal tract spectrum.

Further, when the evaluation results in parentheses in GKL are compared with the evaluation results using STRAIGHT, it can be seen that the mel cepstrum distortion in GKL is small for any F ₀ magnification x. Therefore, it is suggested that the assumption that the vocal tract spectrogram X _{k, t} can be approximated by a low-rank non-negative matrix is useful for vocal tract spectrum estimation.

  As described above, in the proposed method according to the present invention, the vocal tract spectrum is accurately obtained from the audio signal by using the information on the harmonic components in the plurality of frames of the audio signal without using the context label given to the audio signal. It can be estimated well.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the above vocal tract spectrum estimation apparatus has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. Shall be.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Calculation part 21 Observation spectrogram estimation part 22 Initial setting part 23 Estimation part 24 End determination part 25 Output part 30 Storage part 100 Vocal tract spectrum estimation apparatus

Claims (7)

The time-series data of the audio signal is cut out with the width of the basic period, and the observation spectrogram estimation unit that outputs the observation spectrogram representing the observation time frequency component of each time and each normalized angular frequency from the cut out audio signal spectrum,
Each time and each normalization obtained from the observation spectrogram output by the observation spectrogram estimation unit, the vocal tract spectrum representing the power spectrum of each normalized angular frequency in each spectrum pattern, and the weight of each spectrum pattern at each time The vocal tract spectrum of each normalized angular frequency in each spectral pattern and each time of each spectral pattern so as to reduce the objective function expressed using the distance from the vocal tract spectrogram representing the time frequency component of the angular frequency. An estimator for estimating the weight at
A vocal tract spectrum estimation apparatus including: The vocal tract spectrum of the k-th normalized angular frequency ω _k in the r-th spectral pattern is H _{k, r} ^(GMM) expressed by the following equation:
2. The vocal tract according to claim 1, wherein the estimation unit estimates a weight W _{r, n} for each of the n th normal distributions as the vocal tract spectrum of the k th normalized angular frequency ω _k in the r th spectrum pattern. Spectrum estimation device.

Here, G _n (ω) represents a normal distribution with mean ρ _n and variance ν _n ² , and h (ω) represents a frequency warping function. The vocal tract spectrum of the _kth normalized angular frequency ω _{k in} the rth spectrum pattern is represented by H _{k, r} ^(AR) expressed by the following equation:
2. The vocal tract spectrum estimation apparatus according to claim 1, wherein the estimation unit estimates a coefficient a _r of a P-order all-pole filter as the vocal tract spectrum of the k-th normalized angular frequency ω _k in the r-th spectrum pattern. .

However, Q (ω) is a (P + 1) × (P + 1) Toeplitz matrix in which the (p, q) component is represented by cos (ω (p−q)). A vocal tract spectrum estimation method in a vocal tract spectrum estimation apparatus including an observation spectrogram estimation unit and an estimation unit,
The observation spectrogram estimation unit cuts out the time series data of the audio signal with the width of the basic period, and outputs an observation spectrogram representing the observation time frequency component of each time and each normalized angular frequency from the spectrum of the extracted audio signal. And
Each of the estimation units obtained from the observation spectrogram output by the observation spectrogram estimation unit, a vocal tract spectrum representing a power spectrum of each normalized angular frequency in each spectrum pattern, and a weight at each time of each spectrum pattern The vocal tract spectrum of each normalized angular frequency in each spectral pattern, and each of the normalized angular frequencies in each spectral pattern, so as to reduce the objective function represented using the time and the distance from the vocal tract spectrogram representing the time frequency component of each normalized angular frequency, and A vocal tract spectrum estimation method that estimates the weight of a spectrum pattern at each time. The vocal tract spectrum of the k-th normalized angular frequency ω _k in the r-th spectral pattern is H _{k, r} ^(GMM) expressed by the following equation:
The estimation unit estimates a weight W _{r, n} for each of the nth normal distributions as the vocal tract spectrum of the _kth normalized angular frequency ω _{k in} the rth spectrum pattern. The vocal tract spectrum estimation method described.

Here, G _n (ω) represents a normal distribution with mean ρ _n and variance ν _n ² , and h (ω) represents a frequency warping function.
Vocal tract spectrum estimation method. The vocal tract spectrum of the _kth normalized angular frequency ω _{k in} the rth spectrum pattern is represented by H _{k, r} ^(AR) expressed by the following equation:
The voice according to claim 4, wherein the estimation unit estimates a coefficient a _r of a P-th order all-pole filter as the vocal tract spectrum of the k-th normalized angular frequency ω _k in the r-th spectrum pattern. Road spectrum estimation method.

However, Q (ω) is a (P + 1) × (P + 1) Toeplitz matrix in which the (p, q) component is represented by cos (ω (p−q)).
Vocal tract spectrum estimation method.   The program for functioning a computer as each part of the vocal tract spectrum estimation apparatus of any one of Claims 1-3.