JP2014134730A - Fundamental frequency model parameter estimation device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate a parameter of the Fujisaki model by using a restriction relating to non-negativeness of a phrase command and an accent command.SOLUTION: An observation fundamental frequency sequence ^y is extracted from time-series data of voice signals by a fundamental frequency extraction section 2, a degree of inexactness for a fundamental frequency at every time (k) is estimated by a voice/voiceless phase estimation section 3, and an initial value of a command function ^o and an initial value of a parameter group θ are set by an initial value setting section 4. A posteriori probability P(^s|^y, ^o', θ') of a command state sequence ^s is computed by a command state sequence posteriori probability update section 5 and while defining a logarithm posteriori probability logP(^o, θ|^y) as a target function, the command function ^o and the parameter group θ which are non-negative values, are updated by a model parameter update section 6. Until a predetermined settlement condition is satisfied, the computation by the command state sequence posteriori probability update section 5 and updating by the model parameter update section 6 are repeated.

Description

  The present invention relates to a fundamental frequency model parameter estimation device, method, and program, and more particularly, to a fundamental frequency model parameter estimation device, method, and program for estimating parameters of an observed fundamental frequency sequence from a speech signal.

<Fujisaki model>
As a method for analyzing intonation of speech, Fujisaki's fundamental frequency (F ₀ ) pattern generation process model (Fujisaki model) is known (Non-patent Document 1). The Fujisaki model is a mechanical model that explains the F ₀ pattern generation process, focusing on thyroid cartilage movement. The Fujisaki model, the total elongation of the vocal cords with each two independent movement of the thyroid cartilage (translational motion and rotational motion) is interpreted to result temporal variation of F _0, pairs of elongation and F ₀ pattern of the vocal cords The F ₀ pattern is modeled based on the assumption that the numerical value y (t) is proportional. The F ₀ pattern y _p (t) generated by the translational movement of the thyroid cartilage is called a phrase component, and the F ₀ pattern y _a (t) generated by the rotational movement is called an accent component. In the Fujisaki model, the F ₀ pattern y (t) of speech is the sum of these components plus the baseline component y _b determined by the physical constraints of the vocal cords.

It is expressed as These two components are modeled as being the output of a secondary critical braking system,

(* Is a convolution operation related to time t). Here, u _p (t) is called a phrase command function and consists of a sequence of delta functions (phrase commands), and u _a (t) is called an accent command function and consists of a sequence of rectangular waves (accent commands). These command sequences have a constraint that a phrase command occurs at the beginning of an utterance, a phrase command does not occur in succession, and two different commands (phrase command and accent command) do not occur at the same time. . Α and β are the natural angular frequencies of the phrase control mechanism and the accent control mechanism, respectively. It is experiential that α is approximately 3 rad / s and β is 20 rad / s, regardless of the speaker or utterance content. Known to.

<Fujisaki model parameter estimation method 1>
Conventionally, a technique described in Non-Patent Document 2 is known as a technique for estimating the parameters of the Fujisaki model from the F ₀ pattern of a speech signal. In this method first performs pre-processing for smoothing with respect to the observation F ₀ pattern. Specifically, removal of gross error, correction of Microprosody, short after the interpolation silence section and unvoiced, is approximated by a continuous and differentiable piecewise cubic curves everywhere the F ₀ pattern. Next, thus the clue the maximum value and minimum value of the differential values of the obtained smoothed F ₀ pattern, and estimates the position and size of the accent command string. Continue to insert the phrase command to the left-to-right on the basis of the further observation F ₀ pattern from minus the estimated accent component pattern. Finally, the command sequence is slightly changed so that the mean square error between the F ₀ pattern generated from the estimated command sequence and the observed F ₀ pattern is minimized, and the command sequence thus obtained is used as the estimation parameter of the Fujisaki model.

<Fujisaki model parameter estimation method 2>
Conventionally, there are the following other methods for estimating the parameters of the Fujisaki model from the F ₀ pattern of the audio signal (Non-Patent Documents 3 to 5). In this method, a probabilistic model of the F ₀ pattern generation process formulated based on the discrete Fujisaki model is used, and an appropriate parameter is estimated by solving the optimization problem of P (y | θ) according to the model. (y is the observed F ₀ patterns, theta is the parameter of Fujisaki model). In this model, a pair of phrase commands and accent commands that are difficult to handle due to constraints is treated as a value that is stochastically output from the Hidden Markov Model (HMM). In the estimation algorithm, each component is marginalized, and appropriate parameters are estimated by an iterative solution using the EM algorithm.

Hiroya Fujisaki, Sumio Ohno and Wentao Gu, “Physiological and physical mechanisms for fundamental frequency control In some tone languages and a command-response model for generation of their F0 contours,” Proceedings of International Symposium on Tonal Aspects of Languages: Emphasis on Tone Languages , Beijing, pp. 61-64 (2004-3). S. Narusawa, N. Minematsu, K. Hirose, and H. Fujisaki, “A method for automatic extraction of model parameters from fundamental frequency contours of speech,” in Proc. ICASSP, 2002, pp. 509-512. H. Kameoka, J. L. Roux, and Y. Ohishi, “A statistical model of speech F0contours,” in Proc. SAPA, 2010, pp. 43−48. Kota Yoshizato, Hirokazu Kameoka, Daisuke Saito, Shigeki Hiyama, “Estimation of Phrase / Accent Commands from Speech Signals by Statistical Model of F0 Pattern Generation Process,” Proc. Of the Spring Meeting of the Acoustical Society of Japan, 2012, no. 1 -11-9,

pp.311-314.
K. Yoshizato, H. Kameoka, D. Saito, and S. Sagayama, “Statistical approach to fujisaki-model parameter estimation from speech signals and its quantitative evaluation,” in Proc. Speech

Prosody 2012, 2012, pp. 175-178.

The present invention relates to a method for estimating parameters of a Fujisaki model from an F ₀ pattern of speech.

Since this estimation problem is an inverse problem of defect setting, it is difficult to solve analytically. Here, in a non-tone language such as Japanese, there is a restriction that the size of the phrase command and the accent command must be non-negative. Although this non-negativeity is an important constraint for narrowing down solutions, the Fujisaki model parameter estimation methods described in Non-Patent Documents 3 to 5 can directly introduce this constraint as an optimization problem. There wasn't. In the technique described in Non-Patent Document 5 above, an ad hoc method of solving a deconvolution problem with a non-negative constraint when calculating a command sequence from a phrase / accent component is attempted, but the convergence of the algorithm is guaranteed. Whilst there is a problem that a large error between the F ₀ pattern generated from the estimated parameters and the observed F ₀ pattern. This problem of “large error” is a problem also found in the parameter estimation method of the Fujisaki model described in Non-Patent Document 2 above.

  The present invention has been made in view of the above circumstances, and uses a fundamental frequency model parameter apparatus, method, and program capable of estimating parameters of the Fujisaki model using restrictions on non-negativeity of phrase commands and accent commands. The purpose is to provide.

The fundamental frequency model parameter estimation apparatus according to the present invention in order to achieve the object of, as an input audio signal, a command state sequence ^ s made from the state s _k at each time k in the hidden Markov model, at each time k A command comprising _a pair ^ o [k] of a phrase command u _p [k] representing a fundamental frequency pattern generated by translational movement of thyroid cartilage and an accent command u _a [k] representing a fundamental frequency pattern generated by rotational motion of thyroid cartilage Fundamental frequency model parameter estimation that estimates the function ^ o and the parameter group θ representing the amplitude A _p [k] of the phrase command and the amplitude A _a ⁽ⁿ⁾ of each accent command n according to the state s _k at each time k A basic frequency extraction device for extracting an observed fundamental frequency sequence ^ y representing a fundamental frequency at each time k of the voice signal from time series data of the voice signal Means for estimating the degree of uncertainty of the fundamental frequency at each time k according to whether the time series data of the voice signal is a voiced section or an unvoiced section, and the command Based on the initial value setting means for setting the initial value of the function ^ o and the initial value of the parameter group θ, and the command function ^ o 'updated previously or the initial value ^ o' of the command function ^ o, A posteriori probability P (^ s | ^ y, ^ o ', θ') of the command state sequence ^ s given the observed fundamental frequency sequence ^ y, the command function ^ o ', and the parameter group θ' Command state sequence posterior probability update means for calculating the command function ^ o 'or the initial value ^ o' of the command function ^ o updated last time, the observed fundamental frequency sequence ^ y, and the uncertainty at each time k And the posterior probability P ( s | ^ y, ^ o ′, θ ′), the logarithmic posterior probability logP (^ o, θ |) of the command function ^ o and the parameter group θ when the observed fundamental frequency sequence ^ y is given. Model function update means for updating the command function ^ o, which is a non-negative value, and the parameter group θ, so as to increase the objective function with ^ y) as an objective function, and a predetermined convergence condition Up to the first convergence determination means for repeatedly performing the calculation by the command state series posterior probability update means and the update by the model parameter update means.

Fundamental frequency model parameter estimation method according to the present invention, an input audio signal, a command state sequence ^ s made from the state s _k at each time k in the hidden Markov model, caused by the translation movement of the thyroid cartilage at each time k A command function ^ o consisting of a pair ^ o [k] of a phrase command u _p [k] representing a fundamental frequency pattern and an accent command u _a [k] representing a fundamental frequency pattern generated by rotational movement of the thyroid cartilage, and each time k Is a fundamental frequency model parameter estimation method for estimating a phrase command amplitude A _p [k] corresponding to the state s _k and a parameter group θ representing the amplitude A _a ⁽ⁿ⁾ of each accent command n, Means for extracting an observed fundamental frequency sequence ^ y representing a fundamental frequency at each time k of the voice signal from the time series data of the voice signal, The estimation means estimates the degree of uncertainty of the fundamental frequency at each time k according to whether the time series data of the voice signal is a voiced section or an unvoiced section, and the initial value setting means An initial value of the command function ^ o and an initial value of the parameter group θ are set, and the command function ^ o 'or the initial value ^ o of the command function ^ o updated last time by the command state series posterior probability update means ', The posterior probability P (^ s | ^ y, ^ o of the command state series ^ s when the observed fundamental frequency series ^ y, the command function ^ o', and the parameter group θ 'are given. ', Θ'), and the command parameter ^ o 'updated previously or the initial value ^ o' of the command function ^ o, the observed fundamental frequency sequence ^ y, and each time k by the model parameter updating means Uncertainty And the command function ^ o and the parameter group θ when the observed fundamental frequency sequence ^ y is given based on the degree of the posterior probability P (^ s | ^ y, ^ o ', θ') The command function ^ o, which is a non-negative value, and the parameter group θ are updated so that the objective function is increased using the log posterior probability logP (^ o, θ | ^ y) of The calculation by the command state sequence posterior probability update unit and the update by the model parameter update unit are repeatedly performed by the convergence determination unit until a predetermined convergence condition is satisfied.

  The program according to the present invention is a program for causing a computer to function as each unit of the above-described fundamental frequency model parameter estimation apparatus.

  As described above, according to the fundamental frequency model parameter estimation apparatus, method, and program of the present invention, the logarithmic posterior probability logP (of the command function ^ o and the parameter group θ when the observed fundamental frequency sequence ^ y is given. By updating the command function ^ o, which is a non-negative value, and the parameter group θ, so that the objective function is increased with ^ o, θ | ^ y) as the objective function, the non-negativeity of the phrase command and the accent command is related. The effect is that the parameters of the Fujisaki model can be estimated using the constraints.

It is a figure for demonstrating HMM. It is a figure for demonstrating the division | segmentation of a state. It is the schematic which shows the structure of the fundamental frequency model parameter estimation apparatus which concerns on embodiment of this invention. It is a flowchart which shows the content of the fundamental frequency model parameter estimation process routine in the fundamental frequency model parameter estimation apparatus which concerns on embodiment of this invention. It is a figure for demonstrating matching of a command function. It is a figure which shows an experimental result. It is a figure which shows an experimental result.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the method proposed in the present invention, in order to realize parameter estimation of the Fujisaki model with high reproducibility of the observed F ₀ pattern, a probability model of the F ₀ pattern generation process based on the Fujisaki model is formulated and observed based on it. Assume that an F ₀ pattern has occurred. The parameter estimation algorithm of the Fujisaki model is also based on this probability model.

<Probability model of F ₀ pattern generation process>
The principle of the present invention will be described. First, the probability model of the F ₀ pattern generation process will be described.

In order to incorporate the various constraints associated with a command function in the model, phrase command, accent command of the pair ^ o [k] = (u p [k], u a [k]) consider the HMM to output the ^T. In this HMM, the output command function {^ o [k]} _{k = 1} ^K follows a Gaussian distribution at each time,

It is expressed stochastically. Here, {s _k } _{k = 1} ^K is a state series of the HMM, and the mean vector ^ ν _sk [k] and the variance-covariance matrix ^ Υ _sk are values determined as a result of the state transition of the HMM. A specific configuration of the HMM is shown in FIG. A symbol indicating a matrix or a vector is attached with “^”.

In addition, consider dividing each state into several small states in order to parameterize the duration of self-transition. At this time, all the small states have the same output distribution, and the number of small states is set to a sufficiently large value. It shows an example of dividing the state a _n in FIG. For example, as shown in FIG. 2, by setting the state transition probability from a _{n, m} to a _{n, m + 1} to ₁ for all m ≠ 0 _{, the change} from a _{n, 0} to a _{n, m} transition probability corresponds to the probability that state a _n lasts only m step, it becomes possible to flexibly control the persistence length of the accent command. Similarly, by dividing p ₁ , p _0, and a ₀ into small states, it becomes possible to parameterize the distribution of the duration of the phrase command and the length of the interval between commands. Based on these divisions, p ₀ = {p _0,0 , p _0,1 , ...}, a ₀ = {a _0,0 , a _0,1 , ...}, a _n = It is written as {a _{n, 0} , a _{n, 1} , ...}.

  When the proposed HMM configuration is formulated, it can be written as follows.

When a state sequence ^ s = {s _k } _{k = 1} ^K is given, this HMM outputs a pair of _a phrase command function u _p [k] and an accent command function u _a [k]. As shown in Equation (2) and Equation (4), u _p [k] and u _a [k] are convolved in filters G _p [k] and G _a [k], respectively, and the phrase component x _p [ k] and accent component x _a [k] are output. This can be expressed as an expression:

(* Is a convolution operation for discrete time k). At this time, the F ₀ pattern x [k] is

And can be written by superimposing three kinds of components. However, u _b is a baseline component that does not depend on time.

In real speech, a reliable value of the fundamental frequency F ₀ cannot always be observed. For example, an estimated value of the fundamental frequency F ₀ obtained by pitch extraction from speech data is a value that is not reliable at all in the silent section. When estimating the parameters of the Fujisaki model, it is desirable to consider only the F ₀ value of the reliable observation interval and ignore the other intervals. Therefore, the degree of uncertainty v _n ² [k] of the observed F ₀ value at time k is introduced into the proposed model. Specifically, the observed F ₀ value y [k] is superimposed on the true F ₀ value x [k] and the noise components x _n [k] to N (0, v _n ² [k]).

This means that all observation intervals can be handled uniformly regardless of whether they are reliable intervals.

φ _{i ′, i} , u _b , v _{p, sk} ² , v _{a, sk} ² , v _n ² [k], α, β are regarded as constants, and {A _p [k]} _{k = 1} ^K , {A _a ⁽ⁿ⁾ } _{n = 1} ^N is assumed to be uniformly distributed. Then, the probability of ^ y = {y [k]} _{k = 1} ^K when the output value sequence ^ o = {o [k]} _{k = 1} ^K is given by peripheralizing x _n [k] Density function

Is obtained. State sequence ^ s = {s _k } _{k = 1} ^K and parameter group θ = {{A _p [k]} _{k = 1} ^K , {A _a ⁽ⁿ⁾ } _{n = 1} ^N } Output value series ^ o is

Is generated according to P (^ s) is the product of the state transition probabilities

Can be written. Φ _s1 represents the probability that the initial state is s ₁ .

<Fujisaki model parameter estimation algorithm>
In the present invention, the parameter estimation problem of the Fujisaki model is determined by maximizing the a posteriori probability P (^ o, θ | ^ y) of the parameters ^ o, θ when the observation fundamental frequency sequence ^ y is given. Is formulated as a maximum posterior probability (Maximum A Posteriori; MAP) estimation problem, and a local optimal solution of ^ o and θ is obtained by iterative calculation based on the Expectation-Maximization (EM) algorithm with the command state sequence ^ s as a latent variable Explore. The EM algorithm indirectly increases the logarithmic posterior probability logP (^ o, of the parameter logarithmic posterior probability logP (^ o, θ | ^ y) by repeatedly increasing the lower limit function (called Q function). θ | ^ y), and the Q function in this problem is

Can be written. Here, ^c = means matching except for a constant part, and ^ o ′ and θ ′ are values in the previous iteration of ^ o and θ, respectively.

<System configuration>
Next, the embodiment of the present invention will be described with reference to an example in which the present invention is applied to a fundamental frequency model parameter estimation apparatus that analyzes time series data of an observed speech signal and estimates parameters of the Fujisaki model. explain.

  As shown in FIG. 3, the fundamental frequency model parameter estimation apparatus according to the embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program for executing a fundamental frequency model parameter estimation processing routine described later. It is composed of a computer equipped and functionally configured as follows.

  As shown in FIG. 3, the fundamental frequency model parameter estimation apparatus includes a storage unit 1, a fundamental frequency sequence extraction unit 2, a voiced / unvoiced interval estimation unit 3, an initial value setting unit 4, and a command state sequence posterior probability update unit. 5, a model parameter update unit 6, a convergence determination unit 7, a state series calculation unit 8, and an output unit 9.

  The storage unit 1 stores time series data of the observed audio signal.

The basic frequency sequence extraction unit 2 extracts time series data of the basic frequency from the time series data of the audio signal, converts them to be expressed in discrete time k, and uses the time series data of the basic frequency of the audio signal. It is assumed that a certain observation basic frequency sequence ^ y = {F ₀ [k]} (k = 1,..., K). This extraction process of the fundamental frequency can be realized by a well-known technique. For example, Non-Patent Document 6 (H. Kameoka, “Statistical speech spectrum model incorporating all-pole vocal tract model and F0 contour generating process model,” in Tech. Rep. IEICE, 2010, in Japanese.), The fundamental frequency is extracted every 8 ms.

The voiced / unvoiced section estimation unit 3 identifies the voiced section and the unvoiced section from the time-series data of the voice signal, and observes F0 [k] at each discrete time k depending on whether it is a voiced section or an unvoiced section. ] Estimate the degree of value uncertainty v _n ² [k]. In the unvoiced section, the degree of uncertainty is estimated to be large (for example, v _n ² [k] = 10 ¹⁵ ), and in the voiced section, the degree of uncertainty is estimated to be small (for example, v _n ² [k] = 0.22).

The initial value setting unit 4 is the number of accent commands N, the number of iterations M of the EM algorithm, α, β, v _p ² [k], v _a ² [k], u _b , which are parameters used in the processing described later. Is assumed to be a constant and an initial value is set. Set an appropriate value as the initial value. The initial value setting unit 4 determines the number of small states of the HMM and the transition probability φ _{i ′, I} by learning from correct data prepared in advance. The initial value setting unit 4 sets the initial value (non-negative value) of ^ o using the Fujisaki model parameter estimation method described in Non-Patent Document 2. The initial value setting unit 4 sets the linear interpolation of the amplitude of the phrase command function of ^ o as the initial value of A _p [k], and sets an appropriate value as the initial value of A _a ⁽ⁿ⁾ To do.

  In the present embodiment, based on the Q function of the above equation (15), the local optimal solution of the Fujisaki model parameters ^ o and θ includes two steps: a command state sequence posterior probability update unit 5 and a model parameter update unit 6. It is obtained by repeating.

The command state sequence posterior probability updating unit 5 is a step of calculating the posterior probability P (^ s | ^ y, ^ o ', θ') of the command state series (latent variable). Call. If the Forward-Backward algorithm is used, P (s _k = t | ^ y, ^ o ′, θ ′) can be efficiently _obtained for each k and t. In particular,

P (s _k = t | {y [l], ^ o ′ [l], θ ′ [l]} _{l = 1} ^{l = k} )

Can be calculated by solving the recursion formula sequentially (k = 1,2, ..., K), and each P ({y [l], o ′ [l], θ ′ [l]} _{l = k + 1} ^{l = K} | s _k = t) is

Can be calculated by solving the recursion formulas sequentially (k = K, K-1, ..., 1).

As described above, the command state sequence posterior probability update unit 5 sets the command function ^ o ′ or the initial value ^ o ′ updated last time for each combination (k, t) of the time k and the state t. Based on this, by calculating the posterior probability P (s _k = t | ^ y, ^ o ', θ'), the observed fundamental frequency sequence ^ y, the command function ^ o ', and the parameter group θ' are given. The posterior probability P (^ s | ^ y, ^ o ', θ') of the command state sequence ^ s is calculated.

  The model parameter update unit 6 includes an auxiliary variable update unit 61, a command function update unit 62, a convergence determination unit 63, and an average amplitude update unit 64.

  The model parameter updating unit 6 is a step of updating the non-negative command function ^ o and the parameter group θ so as to increase the objective function Q (^ o, θ, ^ o ′, θ ′), and the EM algorithm This is called the M step. The term of logP (^ y | ^ o, θ) is

Can be written. However, G _b [k] = δ [k] (Kronecker delta). Although it is difficult to directly find ^ o that maximizes Equation (21) under the condition that the command functions u _p [k] and u _a [k] are non-negative, Equation (21) is obtained by iterative calculation based on the auxiliary function method. ^ O that locally maximize can be obtained. The auxiliary function method is a technique to increase the objective function by iteratively increasing the lower limit function of the objective function to be maximized as in the EM algorithm, but the lower limit function in Equation (21) is the Jensen inequality.

It is possible to design using the fact that _However, λ _{i, k,} called the auxiliary variable _{l ≧ 0, Σ i Σ l} λ i, k, satisfying the _l = 1. The condition for establishing the equal sign in equation (22) is

It is. Also, the term of Σ _{^ s} P (^ s | ^ y, ^ o ′, θ ′) logP (θ, ^ o | ^ s) is

Can be written. Thus, the inequality

Is the lower limit function of Q (^ o, θ, ^ o ′, θ ′), and is the auxiliary function Q ′ (^ o, θ, ^ o ′, θ ′). Differentiating the auxiliary function Q ′ (^ o, θ, ^ o ′, θ ′) by u _i [l],

It becomes. Therefore, to obtain u _i [l] in the M step, the update formula of the auxiliary function method

It is sufficient to repeat updating λ _{i, k, l} and u _i [l] alternately by using a sufficient number of times.

As described above, the auxiliary variable updating unit 61 determines all combinations (k) of times k and l (l <k) based on the phrase command u _p [k] (or initial value) at each time k updated last time. , L), the auxiliary variable λ _{p, k, l} is calculated and updated according to the above equation (28). In addition, the auxiliary variable update unit 61 performs the above equation (28) for all combinations of (k, l) based on the accent command u _a [k] (or initial value) at each time k updated last time. According to the above, the auxiliary variable λ _{a, k, l} is calculated and updated.

Further, the auxiliary variable updating unit 61 calculates and updates the auxiliary variable λ _{b, k, l} according to the above equation (28) for all combinations of (k, l) based on u _b .

The command function update unit 62 includes the fundamental frequency sequence ^ y, the degree of uncertainty v _n ² [k], and the posterior probability P (^ s | ^) of the command state sequence updated by the command state sequence posterior probability update unit 5. y, ^ o ′, θ ′) and the auxiliary variable λ _{p, k, l} updated by the auxiliary variable updating unit 61 according to the above equation (29), the phrase command at each time l which is a non-negative value Update u _p [l].

The command function update unit 62 also includes the fundamental frequency sequence ^ y, the degree of uncertainty v _n ² [k], and the posterior probability P (^ s of the command state sequence updated by the command state sequence posterior probability update unit 5. | ^ Y, ^ o ′, θ ′) and the auxiliary variable λ _{a, k, l} updated by the auxiliary variable updating unit 61 according to the above equation (29), each time l is a non-negative value. Update the accent command u _a [l].

The command function update unit 62 also includes the fundamental frequency sequence ^ y, the degree of uncertainty v _n ² [k], and the posterior probability P (^ s of the command state sequence updated by the command state sequence posterior probability update unit 5. | Y, ^ o ′, θ ′) and the auxiliary variable λ _{b, k, l} updated by the auxiliary variable updating unit 61, the base component u _b is updated according to the above equation (29).

  The convergence determination unit 63 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, each process of the auxiliary variable update unit 61 and the command function update unit 62 is repeated. If the convergence determining unit 63 determines that the convergence condition is satisfied, the convergence determining unit 63 proceeds to processing by the average amplitude updating unit 64.

  As the convergence condition, it may be used that the number of repetitions s has reached a predetermined number of times S (for example, 20 times). Note that the convergence condition is that the difference between the value of the auxiliary function when the s-1th parameter is used and the value of the auxiliary function when the sth parameter is used is smaller than a predetermined threshold. It may be used as

The average amplitude updating unit 64 continuously updates θ = {{A _p [k]} _{k = 1} ^K and {A _a ⁽ⁿ⁾ } _{n = 1} ^N } as M steps. Differentiating Q ′ (^ o, θ, ^ o ′, θ ′) with A _p [k] and A _a ⁽ⁿ⁾ ,

It becomes. Therefore, the update equation for obtaining A _p [k] and A _a ⁽ⁿ⁾ in the M step is

Can be written.

As described above, the average amplitude updating unit 64, based on the phrase command u _p [k] at each time k updated by the command function updating unit 62, according to the above equation (32), the amplitude of the phrase command at each time k. While updating A _p [k], the accent command u _a [k] at each time k updated by the command function update unit 62 and the posterior probability of the command state sequence updated by the command state sequence posterior probability update unit 5 Based on P (^ s | ^ y, ^ o ', θ'), the parameter group θ is updated by updating the amplitude A _a ⁽ⁿ⁾ of each accent command n according to the above equation (32). .

  The convergence determination unit 7 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, the update value is again substituted into o ′ and θ ′ to repeat The algorithm (each process of the command state sequence posterior probability update unit 5 and the model parameter update unit 6) is repeated. If the convergence determination unit 7 determines that the convergence condition is satisfied, the convergence determination unit 7 proceeds to processing by the state series calculation unit 8.

  As the convergence condition, it may be used that the number of repetitions r has reached a predetermined number R (for example, 20 times). Note that the convergence condition is that the difference between the value of the objective function when the r-1 parameter is used and the value of the objective function when the r parameter is used is smaller than a predetermined threshold. It may be used as

The state sequence calculation unit 8 finally obtains an optimal state sequence ^ s ^* by using the Viterbi algorithm. In particular,

Using δ _t [k] and ψ _t [k] obtained by sequentially solving the recursion formula (k = 1,2, ..., K),

Can be calculated by solving the recursion formulas sequentially (k = K, K-1, ..., 1).

  As described above, the state series calculation unit 8 calculates the state series ^ s according to the above equations (33) to (37) based on the command function ^ o finally updated by the model parameter update unit 6. To do. Then, the output unit 9 outputs the command function ^ o, the parameter group θ, and the state series ^ s.

<Operation of fundamental frequency model parameter estimation device>
Next, the operation of fundamental frequency model parameter estimation apparatus 100 according to the present embodiment will be described. First, time series data of an observed voice signal as an analysis target is input to the fundamental frequency model parameter estimation apparatus 100 and stored in the storage unit 1. Then, the fundamental frequency model parameter estimation apparatus 100 executes a fundamental frequency model parameter estimation processing routine shown in FIG.

First, in step S101, time-series data of an audio signal is read from the storage unit 1, and a fundamental frequency series y consisting of the fundamental frequency F _{0 at} each time k is extracted. In step S102, voiced and unvoiced intervals are specified based on the time-series data of the audio signal, and the degree of uncertainty v _n ² [k] of the fundamental frequency at each time k is estimated.

In the next step S103, appropriate initial values are set for the parameters N, M, α, β, v _p ² [k], v _a ² [k], u _{b and} the number of small states of the HMM The transition probability φ _{i ′, I} is determined by learning from correct data prepared in advance. Further, the command sequence ^ o is estimated by a conventional method and set as an initial value, and an initial value of A _p [k] and an initial value of A _a ⁽ⁿ⁾ are set.

In step S104, based on the initial value of the command sequence ^ o set in step S103 or the command sequence ^ o updated last time in step S105 described later, for all combinations of (k, t), By updating the posterior probability P (s _k = t | ^ y, ^ o ′, θ ′), the posterior probability P (^ s | ^ y, ^ o ′, θ ′) of the command state sequence is updated.

In step S105, the initial value of the command sequence ^ o set in step S103 or the command sequence ^ o updated last time in step S105, the fundamental frequency sequence ^ y calculated in step S101, and the step The degree of uncertainty v _n ² [k] calculated at S102 and the posterior probability P (^ s | ^ y, ^ o ', θ') of the command state sequence updated at Step S104 The command sequence ^ o and the parameter group θ representing the command amplitude are updated so as to increase the objective function Q (^ o, θ, ^ o ′, θ ′) based on

  The step S105 is realized by the processes of the following steps S111 to S114.

In step S111, based on the initial value of the command sequence ^ o set in step S103 or the command sequence ^ o updated last time in step S112 described later, all combinations of (k, l) are described above. According to the equation (28), the auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k, l} are calculated and updated.

In the next step S112, the fundamental frequency sequence ^ y calculated in step S101, the degree of uncertainty v _n ² [k] calculated in step S102, and updated in step S104. A posteriori probability P (^ s | ^ y, ^ o ', θ') of the command state sequence and auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k} updated in step S111 above. _{, l} and _a command sequence ^ o consisting of _a phrase command u _p [l] and an accent command u _a [l] at each time l which is a non-negative value and a base component u _b according to the above equation (29). Update.

In the next step S113, it is determined whether or not the number of repetitions s has reached S as the convergence condition. If the number of repetitions s has not reached S, it is determined that the convergence condition is not satisfied. Then, the process returns to step S111, and the processes of steps S111 to S112 are repeated. On the other hand, when the number of repetitions s reaches S, it is determined that the convergence condition is satisfied, and in step S114, the phrase command u _p [k] and the accent command u _{a at} each time k updated in step S112 above. Based on [k] and the posterior probability P (^ s | ^ y, ^ o ', θ') of the command state sequence updated in step S104, the phrase at each time k according to the above equation (32) The parameter group θ is updated by updating the command amplitude A _p [k] and the accent command amplitude A _a [k] at each position n.

  In step S106, it is determined whether the number of repetitions r has reached R as the convergence condition. If the number of repetitions r has not reached R, it is determined that the convergence condition is not satisfied. In step S107, the command function ^ o and parameter group θ updated in step S105 are substituted into ^ o 'and θ', the process returns to step S104, and the processes in steps S104 to S105 are repeated. . On the other hand, when the number of repetitions r reaches R, it is determined that the convergence condition is satisfied, and in step S108, based on the command function ^ o finally updated in step S105, the above equation (33) The state sequence ^ s is calculated according to the equation (37), and the output unit 9 outputs the command function ^ o, the parameter group θ representing the amplitude of the command, and the state sequence ^ s, and the fundamental frequency model parameter estimation process End the routine.

<Experiment>
An important result in the present embodiment is that the Fujisaki model has been successfully expressed as a probability model. The present inventors believe that a powerful technique for handling prosody can be obtained in the future by incorporating the model proposed in this embodiment into a speech application based on a number of statistical techniques. For that purpose, it is very convenient if the phrase and the accent command function, which are the parameters of the Fujisaki model, can be automatically learned from the speech corpus in the same manner as the spectrum feature amount. In this respect, it can be said that the proposed model and the proposed algorithm of the present embodiment, which are formulated as a probability model, are superior to the non-statistical method as described in Non-Patent Document 2, for example. However, it is not yet clear whether the estimation performance of Fujisaki model parameters from real speech using the proposed algorithm exceeds the performance of existing methods. Therefore, an experiment was performed to quantitatively evaluate the parameter estimation performance of the method proposed in this embodiment.

Detailed experimental conditions are described below. The actual speech data used in this experiment was the B set of the ATR Japanese speech database (Non-Patent Document 7 (A. Kurematsu, K. Takeda, Y. Sagisaka, S. Katagiri, H. Kuwabara, and K. Shikano , "ATR japanese speech database as a tool of speech recognition and synthesis," Speech Communication, vol. 27, pp. 187-207, 1999.)). This is a speech database consisting of 503 phoneme balance sentences, from which one male speaker (MHT) was selected. For the speech database, phrases and accent command functions manually obtained by a prosody research specialist were used as correct answer data. The algorithm described in Non-Patent Document 6 previously proposed by the present inventors was used as a method for extracting the observed F ₀ pattern given as input to the proposed method from the speech data.

For constant parameters, N = 10, discrete time sampling interval t ₀ = 8 ms, EM algorithm iterations M = 20, α = 3.0 rad / s, β = 20.0 rad / s, v _p ² [k] = 0.2 ² , v _a ² [k] = 0.1 ² , v _n ² [k] = 10 ¹⁵ in the unvoiced interval, v _n ² [k] = 0.22 in the voiced interval, and u _b is the minimum of logF ₀ in the voiced interval Each value was set. The number of small states and transition probabilities φ _{i ′, i} of the HMM were determined by learning from 200 sentences of correct data from No. 1 to No. 200 of B set of ATR Japanese speech database. The initial value of ^ o was determined using the method described in Non-Patent Document 2 above. The initial value of A _p [k] is obtained by linear interpolation of the amplitude of the phrase command function of ^ o, and the initial value of A _a ⁽ⁿ⁾ is 0.1n.

The parameter estimation experiment was conducted on 303 sentences from No.201 to No.503. As a method for evaluating the estimated parameters, considering the reproducibility of observed F ₀ pattern, the two linguistic validity. These are generally in a trade-off relationship. For example, can be reproduced very well observed F ₀ pattern if Tatere finely large amount of command to the short interval, thus command functions made can not be said that those linguistically appropriate. Therefore, the purpose of this experiment is to confirm that the reproducibility of the observed F ₀ pattern is very high while the estimated parameters obtained by the method proposed in this embodiment are linguistically adequate. And

The criteria of reproducibility of observed F ₀ pattern, using the observed F ₀ pattern and the mean square error between the reconstructed F ₀ pattern from the estimated command function _{(log F 0 [Hz] RMSE} ), this value is smaller The reproducibility was high. The evaluation rate for linguistic validity used the value of detection rate, and the larger this, the more linguistically valid parameters were assumed. The detection rate is defined as follows. As shown in FIG. 5, the estimated parameter sequence and the correct parameter sequence are compared, and matching is performed in units of commands. The conditions for matching the command and the command are that the two commands are of the same type (phrase commands or accent commands) and that the time difference between the two commands is S = 0.3 seconds or less. However, the accent command was based on the average of the start time and end time. Also, the two matches must not intersect with respect to time. Matching that satisfies these conditions and maximizes the distance can be obtained by dynamic programming, where the distance between the commands with matching is 1 and the distance when it is not is 0. When taking this matching on 303 sentences all used in the estimation experiment, the total number of matching and N _M. Also, the total number of commands in the estimated parameter sequence is N _E , and the total number of commands in the correct parameter sequence is N _A. Here, the insertion error E _I is defined as (N _E -N _M ) / N _A , the dropout error E _D is defined as (N _A -N _M ) / N _A, and the final detection rate D is 1- It was defined as E _I -E _D. Note that the definition of the detection rate does not take into account the amplitude of the command. This is because the amplitude of the phrase and accent commands strongly depends on the value of the baseline component, but the value of the baseline component is greatly different between the proposed method and the correct answer data. Specifically, in the proposed method to set the value u _b baseline component to the minimum value of the log F ₀ in voiced segments, always with correct answer data has been secured to the log 60 Hz, the proposed method u _b This is because it has been confirmed that the estimation performance drops when the value of is fixed.

FIG. 6 summarizes the parameter estimation results using the proposed method and the estimation results using the parameter estimation algorithm (non-statistical method) described in Non-Patent Document 2 selected as the comparison method. As can be seen from this result, the detection rate of the proposed method is comparable to that of the comparative method, while the log F ₀ RMSE value of the proposed method is significantly lower than that of the comparative method. In other words, phrases from real speech using the proposed method, the estimation of the accent command function while satisfying the linguistic validity comparable to existing methods, with performance over existing techniques in reproducibility of observed F ₀ pattern It was confirmed that

FIG. 7 shows the parameter estimation results of the proposed method. The upper graph in this figure is the observed F ₀ pattern (solid line) in the voiced section and the F ₀ pattern (dotted line) reconstructed from the estimated parameters, and the lower graph shows the estimated phrase command function and estimated accent command function. Is. Input F ₀ pattern, as an example, was used as obtained from No.353 of B set of ATR Japanese speech database. As shown in this example, the present invention is observed F ₀ pattern and F ₀ pattern reconstructed from the estimated parameters can be very well matched to such parameters estimation.

  As described above, according to the fundamental frequency model parameter estimation apparatus according to the embodiment of the present invention, logarithmic posterior probabilities logP (of the Fujisaki model parameters ^ o and θ when the observed fundamental frequency sequence ^ y is given. A command function ^ o, which is a non-negative value, and a parameter so that the objective function is increased using the lower limit function Q (^ o, θ, ^ o ', θ') of ^ o, θ | ^ y) as an objective function. By updating the group θ, it is possible to estimate the parameters of the Fujisaki model using the constraint on the non-negativeness of the phrase command and the accent command.

  In the present embodiment, the M step in the EM algorithm is configured by an iterative calculation based on the auxiliary function method (an λ update step and an iterative calculation of ^ o and θ update steps), and Q (^ o, θ, Since it is guaranteed that (^ o ', θ') does not decrease, the convergence of the objective function value (= p (^ o, θ | y)) is guaranteed.

In addition, the estimation algorithm according to the present embodiment, which estimates the parameters of the Fujisaki model using the F ₀ pattern of the speech as an input, can directly introduce non-negative phrase components and accent components, and guarantees convergence. Has been. Specifically, by formulating the probability model representation of the Fujisaki model based on the convolutional mixed hidden Markov model, non-negativeity of the phrase component and the accent component can be directly introduced and convergence can be ensured. As a result, you are possible to reproducibly observed F ₀ patterns to estimate the parameters of very high Fujisaki model.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the fundamental frequency model parameter estimation apparatus described above has a computer system inside, but if the “computer system” uses a WWW system, a homepage providing environment (or display environment) is also available. Shall be included.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 1 Memory | storage part 2 Fundamental frequency sequence extraction part 3 Voiced unvoiced area estimation part 4 Initial value setting part 5 Command state series posterior probability update part 6 Model parameter update part 7 Convergence determination part 8 State series calculation part 61 Auxiliary variable update part 62 Command function Update unit 63 Convergence determination unit 64 Average amplitude update unit 100 Fundamental frequency model parameter estimation device

Claims (5)

As an input voice signal, a command state sequence ^ s made from the state s _k at each time k in the hidden Markov model, phrase command u _p representing the fundamental frequency pattern resulting from the translation movement of the thyroid cartilage at each time k [k] and a command function ^ o of pairs ^ o [k] of the accent command u _a [k] representing the fundamental frequency pattern generated by the rotation movement of the thyroid cartilage, the amplitude a of the phrase command in accordance with the state s _k at each time k a fundamental frequency model parameter estimation device for estimating _p [k] and a parameter group θ representing the amplitude A _a ⁽ⁿ⁾ of each accent command n,
A fundamental frequency extracting means for extracting an observed fundamental frequency sequence ^ y representing a fundamental frequency at each time k of the speech signal from the time series data of the speech signal;
Voiced and unvoiced section estimation means for estimating the degree of uncertainty of the fundamental frequency at each time k depending on whether the time series data of the speech signal is a voiced section or a unvoiced section;
An initial value setting means for setting an initial value of the command function ^ o and an initial value of the parameter group θ;
Based on the previously updated command function ^ o 'or the initial value ^ o' of the command function ^ o, the observed fundamental frequency sequence ^ y, the command function ^ o ', and the parameter group θ' are given. Command state sequence posterior probability update means for calculating the posterior probability P (^ s | ^ y, ^ o ′, θ ′) of the command state sequence ^ s when
The previously updated command function ^ o 'or the initial value ^ o' of the command function ^ o, the observed fundamental frequency sequence ^ y, the degree of uncertainty at each time k, and the posterior probability P (^ s | ^ Y, ^ o ′, θ ′), the logarithmic posterior probability logP (^ o, θ | ^) of the command function ^ o and the parameter group θ when the observed fundamental frequency sequence ^ y is given. model parameter updating means for updating the command function ^ o and the parameter group θ, each of which is a non-negative value so that the objective function is increased with y) as an objective function;
First convergence determination means for repeatedly performing calculation by the command state sequence posterior probability update means and update by the model parameter update means until a predetermined convergence condition is satisfied;
A fundamental frequency model parameter estimation apparatus including: The model parameter update means includes
Based on the initial value of the phrase command u _p [l] at each time l updated last time or the phrase command u _p [l] at each time l, each combination of the times k and l (k, l) is assisted. The variable λ _{p, k, l} is calculated and updated, and based on the initial value of the accent command u _a [k] at each time k updated last time or the accent command u _a [k] at each time k, Auxiliary variable updating means for calculating and updating auxiliary variables λ _{a, k, l} for each combination of times k, l (k, l);
The observed fundamental frequency sequence ^ y, the degree of uncertainty at each time k, the calculated command state sequence posterior probability P (^ s | ^ y, ^ o ', θ'), and the auxiliary variable Based on the auxiliary variables λ _{p, k, l} , λ _{a, k, l} updated by the updating means, the lower limit function Q (^ o, θ, ^ o ′, θ ′) of the objective function is further reduced. Command function updating means for updating the phrase command u _p [l] and the accent command u _a [l] at each time l so that the function is
Second convergence determination means for repeatedly performing the update by the auxiliary variable update means and the update by the command function update means until a predetermined convergence condition is satisfied;
Based on the phrase command u _p [l] at each time l updated by the command function updating means, the amplitude A _p [k] of the phrase command at each time k is updated and updated by the command function updating means. Of each accent command n based on the calculated accent command u _a [l] of each time l and the calculated posterior probability P (^ s | ^ y, ^ o ', θ') of the command state sequence. Average amplitude updating means for updating the parameter group θ by updating the amplitude A _a ⁽ⁿ⁾ ;
The fundamental frequency model parameter estimation apparatus according to claim 1, comprising:   3. The fundamental frequency model parameter estimating apparatus according to claim 1, further comprising a state series calculating unit that calculates the state series ^ s based on the command function ^ o finally updated by the model parameter updating unit. As an input voice signal, a command state sequence ^ s made from the state s _k at each time k in the hidden Markov model, phrase command u _p representing the fundamental frequency pattern resulting from the translation movement of the thyroid cartilage at each time k [k] and a command function ^ o of pairs ^ o [k] of the accent command u _a [k] representing the fundamental frequency pattern generated by the rotation movement of the thyroid cartilage, the amplitude a of the phrase command in accordance with the state s _k at each time k A fundamental frequency model parameter estimation method for estimating _p [k] and a parameter group θ representing the amplitude A _a ⁽ⁿ⁾ of each accent command n,
The fundamental frequency extraction means extracts from the time series data of the speech signal an observed fundamental frequency sequence ^ y representing the fundamental frequency at each time k of the speech signal,
The voiced and unvoiced section estimation means estimates the degree of uncertainty of the fundamental frequency at each time k according to whether the time series data of the voice signal is a voiced or unvoiced section,
By an initial value setting means, an initial value of the command function ^ o and an initial value of the parameter group θ are set,
Based on the command function ^ o 'updated last time or the initial value ^ o' of the command function ^ o by the command state series posterior probability update means, the observed fundamental frequency sequence ^ y, the command function ^ o ', And a posteriori probability P (^ s | ^ y, ^ o ', θ') of the command state sequence ^ s given the parameter group θ '.
The command parameter ^ o 'updated last time or the initial value ^ o' of the command function ^ o, the observed fundamental frequency sequence ^ y, the degree of uncertainty at each time k, and the fact Based on the a posteriori probability P (^ s | ^ y, ^ o ', θ'), the logarithmic posterior probability logP (of the command function ^ o and the parameter group θ when the observed fundamental frequency sequence ^ y is given. Updating the command function ^ o and the parameter group θ each having a non-negative value so that the objective function is increased with ^ o, θ | ^ y) as an objective function,
A fundamental frequency model parameter estimation method in which the calculation by the command state sequence posterior probability update unit and the update by the model parameter update unit are repeatedly performed by the first convergence determination unit until a predetermined convergence condition is satisfied.   The program for functioning a computer as each means of the fundamental frequency model parameter estimation apparatus of any one of Claims 1-3.