JP6665079B2 - Fundamental frequency model parameter estimation device, method, and program - Google Patents

Description

  The present invention relates to a fundamental frequency model parameter estimating device, method, and program, and more particularly to a fundamental frequency model parameter estimating device, method, and program for estimating parameters of an observed fundamental frequency sequence from a speech signal.

  The speech contains various information in addition to linguistic information, and is used for daily communication. We are studying information processing and signal processing for analyzing and synthesizing non-verbal information with the goal of constructing a framework for engineering such non-verbal information.

It is known that the fundamental frequency (F ₀ ) trajectory of speech contains abundant non-verbal information such as speaker characteristics, emotions, and intentions. Thus, modeling of F ₀ trajectory, speech synthesis, speaker recognition, emotion recognition, such as dialogue systems, is very effective in prosody information plays an important role applications. F ₀ locus, a component that varies slowly over the prosodic phrase (a phrase component) constituted by steeply changing component (accent component) in accordance with an accent. These components can be interpreted as corresponding to the translational and rotational motions of the human thyroid cartilage, respectively.Based on this interpretation, a mathematical model expressing the logarithmic F ₀ locus as the sum of these components (hereinafter, referred to as Fujisaki model) has been proposed (Non-Patent Document 1). The Fujisaki model has parameters such as the time and duration of occurrence of phrase accent commands, the size of each command, and the like, and when these are set appropriately, it is known that the measured locus approximates very well. . The validity of the linguistic correspondence of parameters has also been widely confirmed.

Parameters of the foregoing Fujisaki model, because it can efficiently represent prosodic features, it is useful if it is estimated from the F ₀ locus of the measured parameters of Fujisaki model at high speed and with high precision. However, this problem was not always easy because the problem was originally a bad setting problem, and the Fujisaki model had constraints that had to be protected by linguistic knowledge. Previously inventors model the stochastic process of generating F ₀ pattern based Fujisaki model has been proposed a method of estimating the Expectation-Maximization (EM) algorithm the maximum likelihood parameters of Fujisaki model (Non Patent documents 2-4).

H. Fujisaki, O. Fujimura, Ed., “A note on the physiological and physical basis for the phrase and accent components in the Voice fundamental frequency contour,” in Vocal Physiology: Voice Production, Mechanisms and Functions.New York, NY, USA: Raven, 1988. H. Kameoka, J. L. Roux, and Y. Ohishi, “A statistical model of speech F0contours,” in Proc. SAPA, 2010, pp. 43-48. K. Yoshizato, H. Kameoka, D. Saito, and S. Sagayama, “Statistical approach to fujisaki-model parameter estimation from speech signals and its quantitative evaluation,” in Proc. Speech Prosody 2012, 2012, pp. 175-178. K. Yoshizato, H. Kameoka, D. Saito, and S. Sagayama, “Hidden Markov convolutive mixture model for pitch contour analysis of speech,” in Proc.The 13th Annual Conference of the International Speech Communication Association (Interspeech 2012), Sep . 2012.

  The central idea of the above method is that the generation process of the phrase / accent command sequence is represented by a Hidden Markov Model (HMM) .In these methods, 90% or more of the calculation time is a posterior state at each time. Spent on Forward-Backward algorithms to calculate probabilities. In the above method, since the output distribution in all states of the HMM is described by a normal distribution, an exponential calculation and a logarithmic calculation for calculating a sum of products of a large number of probability values in sequential calculation of state posterior probabilities are required, This was the main factor that took time to calculate. If the amount of calculation in this processing unit can be suppressed, the efficiency of the entire algorithm can be increased.

  The present invention has been made in view of the above circumstances, and has as its object to provide a fundamental frequency model parameter estimating apparatus, method, and program capable of estimating the parameters of the Fujisaki model while suppressing the amount of calculation. .

The fundamental frequency model parameter estimation apparatus according to the present invention in order to achieve the object of, as an input audio signal, the state sequence s made from the state s _k at each time k in the hidden Markov model, the thyroid cartilage at each time k a command function o of pairs o [k] of the phrase command representing a fundamental frequency pattern u _p [k] and accent command representing a fundamental frequency pattern generated by the rotation movement of the thyroid cartilage u _a [k] generated by translation motion of , the parameter C ^(p) [k] and estimated the parameters set θ representing parameters C _n ^(a) the state output distributions for each accent command n state output distributions of phrase command in accordance with the state s _k at each time k A fundamental frequency model parameter estimating apparatus, wherein an observation fundamental frequency representing a fundamental frequency at each time k of the audio signal is obtained from time-series data of the audio signal. A fundamental frequency extracting unit for extracting a sequence y, and a voiced estimating unit for estimating a degree of uncertainty of the fundamental frequency at each time k according to whether the time-series data of the audio signal is a voiced section or an unvoiced section. An unvoiced section estimating unit, an initial value setting unit that sets an initial value of the command function o, and the observed fundamental frequency sequence y based on the previously updated command function o or the initial value of the command function o. Using the command function o and the logarithmic joint probability log p (y, o, s) of the state sequence s as an objective function, a state sequence that updates the state sequence s using the Viterbi algorithm so as to increase the objective function An update unit, based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency sequence y, and the degree of the uncertainty at each time k, A command parameter o, each of which is a non-negative value, and a model parameter updating unit that updates the parameter group θ so as to increase the objective function, and updating by the state sequence updating unit until a predetermined convergence condition is satisfied. , And a convergence determining unit that repeats updating by the model parameter updating unit.

Fundamental frequency model parameter estimation method according to the present invention, an input audio signal, the state sequence s made from the state s _k at each time k in the hidden Markov model, the fundamental frequency caused by translation motion of thyroid cartilage at each time k phrase command u _p [k] and the command functions o of pairs o [k] of the accent command u _a representative of the fundamental frequency pattern [k] generated by the rotation movement of the thyroid cartilage, the state s _k at each time k representing the pattern In the fundamental frequency model parameter estimating apparatus for estimating the parameter C ^(p) [k] of the state output distribution of the phrase command corresponding to the parameter C and the parameter group θ representing the parameter C _n ^(a) of the state output distribution of each accent command n A method for estimating a fundamental frequency model parameter, wherein a fundamental frequency extracting unit extracts a time series of the audio signal from time-series data of the audio signal. The voiced and unvoiced section estimation unit extracts the observed fundamental frequency sequence y representing the fundamental frequency of k, and the voiced and unvoiced section estimation unit determines the time-series data of the audio signal at each time k according to whether it is a voiced section or an unvoiced section. Estimating the degree of uncertainty of the fundamental frequency, an initial value setting unit sets an initial value of the command function o, and a state series updating unit sets the initial value of the command function o or the command function o updated last time. Based on the above, the Viterbi algorithm is used to increase the objective function using the observed fundamental frequency sequence y, the command function o, and the logarithmic joint probability log p (y, o, s) of the state sequence s as the objective function. The state sequence s is updated by using the command parameter o or the initial value of the command function o updated last time, the observed fundamental frequency sequence y, and each time. k, the command function o and the parameter group θ, each of which is a non-negative value, are updated so as to increase the objective function based on the degree of uncertainty at k. Until the condition is satisfied, the updating by the state sequence updating unit and the updating by the model parameter updating unit are repeated.

  A program according to the present invention is a program for causing a computer to function as each unit of the above-described fundamental frequency model parameter estimation device.

  As described above, according to the fundamental frequency model parameter estimating apparatus, method, and program of the present invention, the logarithmic joint probability log p (y, o, o,) of the observed fundamental frequency sequence y, the command function o, and the state sequence s. Using the Viterbi algorithm with s) as the objective function, the state sequence s is updated, and the command function o and the parameter group θ, each of which is a non-negative value, are repeatedly updated, thereby reducing the amount of calculation. And the parameters of the Fujisaki model can be estimated.

It is a figure for explaining a Fujisaki model. FIG. 3 is a diagram for explaining an HMM. It is a figure for explaining division of a state. It is a schematic diagram showing the composition of the fundamental frequency model parameter estimating device concerning an embodiment of the invention. 5 is a flowchart showing the contents of a fundamental frequency model parameter estimation processing routine in the fundamental frequency model parameter estimation device according to the embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the method proposed in the present invention, in order to realize observation F ₀ pattern reproducibility is high Fujisaki model parameter estimation, and to formulate a probability model of the F ₀ pattern generation process that is based on Fujisaki model, observations based on it assume F ₀ pattern occurs. The parameter estimation algorithm of the Fujisaki model is also based on this probability model.

<Outline of Embodiment of the Present Invention>
In the models proposed in Non-Patent Documents 2 to 4, the state sequence of the HMM

And output value series

Has as a variable. In Non-Patent Documents 2 and 3, the observed F ₀ pattern

Under given

When

Conditional joint probability of

To

Related to

Conditional probability of

To

It has been shown that the local maximization can be performed by the EM algorithm using as a hidden variable. On the other hand, in Non-Patent Document 4,

To

Related to

Conditional probability of

To

It has been shown that local maximization can be achieved with the EM algorithm using as a hidden variable. The former method is

A non-negative constraint on

Was difficult to incorporate into the estimation process of

Locally optimal under non-negative constraints on

, It is possible to achieve high parameter estimation accuracy. But with the latter method

Given by the Forward-Backward algorithm

A step (E step) of calculating the posterior probability of is required, and this step required a huge amount of calculation. Therefore, in the embodiment of the present invention, the optimization criterion is

And

Instead of

When

Conditional joint probability of

age,

Posterior probability calculation step

We propose a parameter estimation algorithm that replaces the optimal estimation step. That is,

Is optimal under non-negative constraints given

Estimating and

The best under given

By efficiently searching through the Viterbi algorithm

Locally optimal under the non-negative constraint of

When

Can be estimated. The point of this method is that the Forward-Backward algorithm is replaced with the Viterbi algorithm in Non-Patent Document 4. In general, the Viterbi algorithm is faster than the Forward-Backward algorithm, so that the efficiency of the entire calculation is improved. There is expected.

<Probability model of the F ₀ locus>
It will be described probabilistic modeling of F ₀ locus that is described in Non-Patent Document 4.

As shown in FIG. 1, the model (see Non-Patent Document 1) Fujisaki, the sum of the following three components logarithmic F ₀ trajectory y (t)

It is a model represented by. Here, t is time, x _p (t) is a phrase component, x _a (t) is an accent component, and x _b is a time-dependent constant called a baseline component. Furthermore, the phrase component and the accent component are output from the secondary filter of the signal called the phrase command and accent command, respectively.

Is assumed. Here u _p (t) is the pulse train called the phrase command, u _a (t) is a rectangular pulse train called the accent command. At most one of these takes a non-zero value at each time. α and β are angular frequencies representing the response speed of the secondary filter, and are known to take values of approximately α = 3 rad / s and β = 20 rad / s regardless of individuals or utterances.

Hereinafter, until we have this outlines the probabilistic model of the production process of F ₀ locus which is based on Fujisaki model has been developed (Non-Patent Document 4). In the above-described Fujisaki model, the phrase command and the accent command are a delta train and a rectangular pulse train, respectively, and furthermore, it is assumed that these do not overlap each other. A central idea of the methods of Non-Patent Documents 2 to 4 is that the generation process of the phrase / accent instruction sequence is represented by a hidden Markov model (HMM). The index of discrete time is k, and the pair of phrase command u _p [k] and accent command u _a [k]

And If the output distribution of each state is normal distribution, the output series

Is

Obey. Here s _k represents the state at time k. That is, equation (6) is the average

And dispersion

Changes with time as a result of the state transition. The advantage of HMM is that it can flexibly set constraints to be imposed on the sequence to be modeled through the design of the state transition network. The above-mentioned restrictions on the phrase command and the accent command can be expressed by, for example, a state transition network as shown in FIG. In addition, the duration of the self-transition can be parameterized by dividing each state into several smaller states with the same output distribution.

FIG. 2 shows a state transition model of a phrase / accent instruction sequence in a conventional method (see Non-Patent Documents 2 to 4). In the state r ₀ , μ _p [k] and μ _a [k] are both 0. In the state p ₀ , μ _p [k] takes a non-negative value C ^(p) [k], and μ _a [k] becomes 0. Both the state r ₁ in a state r ₀ Similarly mu _p [k] and μ _a [k] is 0. Thus in the process of transition to a state r ₁ via the state p ₁ from the state r ₀ μ _p [k] is a pulse-like sequences. State r ₁ can only transition to states a ₀ ,..., A _N−1 , in which μ _a [k] takes different values C ^(a) _n and μ _p [k] Becomes 0. Directly without passing through the state r ₁ a _n from _{a n '(n ≠ n'} ) a μ by not able to transition to the [k] can be constrained to a rectangular pulse train.

Next, an example of dividing the state a _n a small state in FIG. For example, by setting for all m ≠ 0 as shown in FIG. 3 a _n, from _m a _n, a state transition probability of the _{m + 1} to 1, from a _{n, 0} a _n, to _m transition probability corresponds to the probability that state a _n lasts only m step, it becomes possible to flexibly control the persistence length of the accent command. Similarly p ₁ and p ₀ and a ₀ also be divided into small state, the length of the distribution of intervals between command and persistence length of the phrase command can be parameterized. Based on this division,

Notation. The configuration of the above HMM is as follows.

The command functions u _p [k] and u _a [k] output from the above HMM are convolved with different filters G _p [k] and G _a [k], respectively.

Becomes Here, * represents convolution for discrete time k. G _p [k] and G _a [k] are discrete-time expressions of G _p (t) and G _a (t), respectively. From the above, the discrete-time expression x [k] of the F ₀ trajectory is

Becomes x _b represents the baseline component.

In unvoiced sections, F ₀ may not be observed, or even if observed, it may not be reliable. In some cases, the estimation error in the F ₀ extraction occurs. Therefore observation F ₀ patterns y [k], the above-mentioned F ₀ pattern model x [k] and the noise

The uncertainty of the observed F ₀ pattern can be incorporated through the setting of the variance v ² _n [k]. That is, the observed F ₀ pattern y [k] and

Thus, all observation sections can be treated uniformly regardless of whether they are reliable sections. Here, when x _n [k] is marginalized,

Given

Conditional probability density function of

Is

Becomes From equation (6), the state series

Given

Conditional probability density function of

Is

Given by here,

Represents the series of the mean and variance of the output distribution. State series

Probability distribution of

Is the product of transition probabilities from the assumption of Markov property in HMM.

Given by

<Fujisaki model parameter estimation algorithm>
In Non-Patent Documents 2 and 3, the observation F ₀ series

State sequence when given

Posterior probability of

Maximize

The algorithm for search has been proposed by the EM algorithm, in Non-Patent Document 4, the observed F ₀ sequence

Output sequence when given

Posterior probability of

To

Maximize each element of to be nonnegative

An algorithm has been proposed which searches for EM by the EM algorithm and the auxiliary function method. In contrast, the present invention provides an observation F ₀ series

Output sequence when given

And state series

Simultaneous posterior probability of

Maximize

When

To

After fixing

To maximize

Updating the

After fixing

So that

Is an algorithm for searching by repeating the step of updating under non-negative value constraints.

<State sequence update step>
The state series update step includes:

After fixing

To maximize

Is the step of updating

so

Is dependent on

Because

Maximize

The problem to ask for is

This is the same as the HMM state sequence search problem with the output sequence as. Therefore, it can be solved by using the Viterbi algorithm.

<Status output sequence update step>
The status output series updating step includes:

After fixing

To maximize

Is the step of updating

so

Is dependent on

And

When

Are each

Given by Here, G _b [k] = δ [k] (Kronecker delta). Under the condition that the command functions u _p [k] and u _a [k] are non-negative

Maximize

Is difficult to find directly, but it is locally maximized by iterative calculation based on the auxiliary function method

Can be searched. The auxiliary function method is a method of increasing the objective function by repeatedly increasing the lower function of the objective function to be maximized. The lower bound function of Eq. (12) is Jensen's inequality

Can be designed using the fact that However,

Are called auxiliary variables,

Meet. The condition for the equality of equation (14) is

It is.

Therefore,

Holds, and the right side is an auxiliary function

Call. Differentiating this auxiliary function with respect to u _i [l] gives

So by setting this to 0

Get. From the above, by repeating Equations (15) and (18),

Can be increased.

Also,

Of the state output distribution of the HMM that maximizes

Is

By setting each partial derivative with respect to 0,

Given by However,

Set of k such is a s _k = a _n

Represents

Represents the number of elements in the set.

<System configuration>

  Next, the embodiment of the present invention will be described with reference to an example in which the present invention is applied to a fundamental frequency model parameter estimating apparatus that analyzes time-series data of an observed audio signal and estimates parameters of the Fujisaki model. explain.

  As shown in FIG. 4, a fundamental frequency model parameter estimating apparatus 100 according to an embodiment of the present invention includes a CPU, a RAM, and a ROM storing a program for executing a fundamental frequency model parameter estimating processing routine described later. And is functionally configured as follows.

  As shown in FIG. 4, the fundamental frequency model parameter estimation device 100 includes a storage unit 1, a fundamental frequency sequence extraction unit 2, a voiced unvoiced section estimation unit 3, an initial value setting unit 4, a state sequence update unit 5, , A model parameter updating unit 6, a convergence determining unit 7, and an output unit 9.

  The storage unit 1 stores time-series data of the observed audio signal.

The fundamental frequency sequence extracting unit 2 extracts fundamental frequency time series data from the audio signal time series data, converts them into discrete time k, and converts the fundamental frequency time series data into the fundamental frequency time series data of the audio signal. An observed fundamental frequency sequence

And Extraction of the fundamental frequency, can be achieved by well known techniques, for example, Non-Patent Document 5 (H. Kameoka, "Statistical speech spectrum model incorporating all-pole vocal tract model and F 0 contour generating process model," in Tech. Rep IEICE, 2010, in Japanese.), A fundamental frequency is extracted every 8 ms.

The voiced unvoiced section estimation unit 3 specifies a voiced section and a voiceless section from the time-series data of the audio signal, and determines the observation F ₀ [for each discrete time k according to whether it is a voiced section or a voiceless section. k] Estimate the degree of uncertainty of the value v _n ² [k]. In unvoiced sections, the degree of uncertainty is estimated to be large, and in voiced sections, the degree of uncertainty is estimated to be small.

The initial value setting unit 4 are each a parameter used in the process described later, sets the number N, the initial value regarded as constant u _b accent command. Set an appropriate value as the initial value. Further, the initial value setting unit 4 learns and determines the number of small states of the HMM and the transition probability φ _{i ′, I} from the correct data prepared in advance. Further, the initial value setting unit 4 uses a conventionally known parameter estimation method of the Fujisaki model,

Set the initial value (non-negative value) of. Further, the initial value setting unit 4 sets the initial value of C ^(p) [k] as

Is set by linearly interpolating the amplitude of the phrase command function of, and an appropriate value is set as the initial value of C _n ^(a) .

In the present embodiment, the Fujisaki model parameters

When

Is obtained by repeating the two steps of the state sequence updating unit 5 and the model parameter updating unit 6.

The state sequence updating unit 5 is a command function that has been updated last time.

Or command function

Based on the initial value of

, Directive function

, And state series

Log joint probability of

Is used as an objective function, and the state series is increased using the Viterbi algorithm so as to increase the objective function.

To update. In particular,

Is maximized using the Viterbi algorithm so that

To update.

The model parameter updating unit 6 stores the previously updated command function

Or directive function

Initial value of, observation fundamental frequency series

And the non-negative command functions using the auxiliary function method to increase the objective function based on the degree of uncertainty v _n ² [k] at each time k

, And parameters

To update.

  Specifically, the model parameter updating unit 6 includes an auxiliary variable updating unit 61, a command function updating unit 62, a convergence determining unit 63, and a state output distribution updating unit 64.

Auxiliary variable update unit 61, based on the phrase command u _p at each time k was last updated [k] (or default), time k, l (l <k) all combinations (k, l) of For each of them, the auxiliary variables λ _{p, k, l} are calculated and updated according to the above equation (15). In addition, the auxiliary variable updating unit 61 calculates the above equation (15) for all combinations of (k, l) based on the accent command u _a [k] (or the initial value) at each time k updated last time. , The auxiliary variables λ _{a, k, l} are calculated and updated.

The auxiliary variable update unit 61 based on the u _b, (k, l) for all combinations of, according to the above equation (15), the auxiliary variable lambda _{b, k,} is updated to calculate the _l.

The command function updating unit 62 calculates the fundamental frequency sequence

And the degree of uncertainty v _n ² [k], and the state sequence updated by the state sequence updating unit 5

When the auxiliary variable update section 61 auxiliary variable lambda _{p, k} updated _by, based on the _l, according to the above formula (18), and updates the phrase command u _p at each time l is a non-negative value [l].

In addition, the command function updating unit 62 performs the

And the degree of uncertainty v _n ² [k], and the state sequence updated by the state sequence updating unit 5

Based on the auxiliary variable λ _{a, k, l} updated by the auxiliary variable updating unit 61, the accent command u _a [l] at each time l that is a non-negative value is updated according to the above equation (18).

In addition, the command function updating unit 62 performs the

When a degree of uncertainty v _n ² [k], based auxiliary variable is updated by the auxiliary variable updating unit 61 lambda _{b, k,} to the _l, according to the above formula (18), to update the base component u _b .

  The convergence determination unit 63 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, each process of the auxiliary variable update unit 61 and the command function update unit 62 is repeated. When the convergence determining unit 63 determines that the convergence condition is satisfied, the process proceeds to the processing by the state output distribution updating unit 64.

  As the convergence condition, the fact that the number of repetitions s reaches a predetermined number S (for example, 20 times) may be used. It should be noted that the difference between the value of the auxiliary function when the s-1th parameter is used and the value of the auxiliary function when the sth parameter is used is smaller than a predetermined threshold value. May be used.

State output distributions updating unit 64 on the basis of the phrase command u _p at each time k updated by the command function updater 62 [k], according to the above formula (19), the state output distributions for phrase command at each time k updates the parameter C ^(p) [k], the accent command u _a [k] at each time k updated by the command function updating unit 62, based on the state sequence s that is updated by the state sequence update unit 5 By updating the parameter C _n ^(a) of the state output distribution of each accent command n according to the above equation (20), the parameter group

To update.

The convergence determination unit 7 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, the convergence determination unit 7 renews the update value.

When

And the iterative algorithm (the respective processes of the state sequence updating unit 5 and the model parameter updating unit 6) is repeated. When determining that the convergence condition is satisfied, the convergence determining unit 7 proceeds to the processing by the output unit 9.

  As the convergence condition, a condition that the number of repetitions r reaches a predetermined number R (for example, 20 times) may be used. It should be noted that the difference between the value of the objective function when the r-1 parameter is used and the value of the objective function when the r parameter is used is smaller than a predetermined threshold value. May be used.

Then, the command function is output by the output unit 9.

, Parameter group

, State series

Is output.

<Operation of fundamental frequency model parameter estimation device>
Next, the operation of the fundamental frequency model parameter estimation device 100 according to the present embodiment will be described. First, time-series data of an observed audio signal is input to the fundamental frequency model parameter estimation device 100 as an analysis target, and is stored in the storage unit 1. Then, in the fundamental frequency model parameter estimation device 100, a fundamental frequency model parameter estimation processing routine shown in FIG. 5 is executed.

First, in step S101, from the storage unit 1 reads the time-series data of the audio signal, the fundamental frequency sequence consisting of the fundamental frequency F ₀ of the time k

Is extracted. In step S102, a voiced section and an unvoiced section are specified based on the time-series data of the audio signal, and the degree of uncertainty v _n ² [k] of the fundamental frequency at each time k is estimated.

In the next step S103, appropriate initial values are set for the parameters N and u _b , and the number of small states of the HMM and the transition probability φ _{i ′, I} are determined by learning from the correct data prepared in advance. I do. In addition, the command sequence is

Is estimated and set as an initial value, and an initial value of C ^(p) [k] and an initial value of C _n ^(a) are set.

Then, in step S104, the command sequence set in step S103 is set.

Or the command sequence last updated in step S105 described later.

On the basis of the,

Is maximized using the Viterbi algorithm so that

To update.

In step S105, the initial value of the phrase command u _p [k] at each time k set in the step S103, or in step S106, which will be described later, based on the phrase command u _p [k] at each time k, which was last updated , And for each combination (k, l) of times k, l (l <k), the auxiliary variable λ _{p, k, l} is calculated and updated according to the above equation (15). The initial value of the accent command u _a [k] at each time k set in the step S103, or in step S106, which will be described later, based on the accent command u _a [k] at each time k that was last updated, (k, For all the combinations 1), the auxiliary variables λ _{a, k, l} are calculated and updated according to the above equation (15). The initial value of u _b set in the step S103, or on the basis of a u _b was last updated in step S106 to be described later, for all combinations of (k, l), according to the above equation (15), The auxiliary variables λ _{b, k, l} are calculated and updated.

In the next step S106, the fundamental frequency sequence calculated in the above step S101

And the degree of uncertainty v _n ² [k] at each time k calculated in step S102, and the state sequence updated in step S104

Based on the auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k, l} updated in step S105, each time l which is a non-negative value is calculated according to the above equation (18). phrase command u _p [l] and the command sequence consisting of accent command u _a [l] of

To update the base component u _b.

In the next step S107, it is determined whether the number of repetitions s has reached S as a convergence condition. If the number of repetitions s has not reached S, it is determined that the convergence condition is not satisfied. Then, the process returns to step S105, and the processes of steps S105 to S106 are repeated. On the other hand, if the number of repetitions s reaches S is determined to have been satisfied convergence condition, in step S108, based on the phrase command u _p at each time k updated in step S106 [k], the according to equation (19), and updates the parameter C ^(p) [k] of the state output distributions for phrase command at each time k, accent command u _a [k] at each time k updated in step S106, State series updated in step S104

By updating the parameter C _n ^(a) of the state output distribution of each accent command n based on the above equation (20), the parameter group

To update.

In step S109, it is determined whether or not the number of repetitions r has reached R as a convergence condition. If the number of repetitions r has not reached R, it is determined that the convergence condition is not satisfied. Then, the process returns to step S104, and the processes of steps S104 to S108 are repeated. On the other hand, when the number of repetitions r reaches R, it is determined that the convergence condition is satisfied, and the output unit 9 outputs the command function

, Parameter group

, State series

Is output, and the fundamental frequency model parameter estimation processing routine ends.

<Experiment>
Table 1 shows the calculation time required for each step when the conventional method (Non-Patent Document 4) and the method of the embodiment of the present invention are applied to audio data having an audio data length of 3.62 seconds.

  The implementation environment is as follows.

・ CPU: Core i7-6700K 4.0GHz
・ RAM: 32GB
・ OS: Windows 7 SP1
・ MATLAB R2016a

  The updating step of the state sequence s in the method according to the embodiment of the present invention is approximately 70 times faster than the posterior probability updating step of the state sequence s in the conventional method, and the overall speed can be increased approximately 16 times.

As described above, according to the fundamental frequency model parameter estimation device according to the embodiment of the present invention, the logarithmic joint probability of the observed fundamental frequency sequence, the command function, and the state sequence

By using the Viterbi algorithm to update the state series and repeatedly update the command function and the parameter group θ, the amount of calculation can be suppressed, and the parameters of the Fujisaki model can be estimated with a reduced amount of calculation.

  The present invention is not limited to the embodiment described above, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the above-described fundamental frequency model parameter estimating apparatus has a computer system inside, but the “computer system” also includes a homepage providing environment (or display environment) if a WWW system is used. Shall be included.

  Further, in the specification of the present application, the embodiment is described in which the program is installed in advance. However, the program may be stored in a computer-readable recording medium and provided.

Reference Signs List 1 storage unit 2 fundamental frequency sequence extraction unit 3 voiced unvoiced section estimation unit 4 initial value setting unit 5 state sequence update unit 6 model parameter update unit 7 convergence determination unit 61 auxiliary variable update unit 62 command function update unit 63 convergence determination unit 64 state Output distribution update unit 100 Fundamental frequency model parameter estimation device

Claims (7)

An input audio signal, hiding the state sequence s made from the state s _k at each time k in Markov models, phrase command u _p representing the fundamental frequency pattern resulting from the translation movement of the thyroid cartilage at each time k [k] and thyroid a command function o of pairs o [k] of the accent command representing a fundamental frequency pattern generated by the rotation movement of cartilage u _a [k], the parameter C of the state output distributions for phrase command in accordance with the state s _k at each time k ^(p) A fundamental frequency model parameter estimating device for estimating [k] and a parameter group θ representing ^a parameter C _n ^(a) of ^a state output distribution of each accent command n,
From the time-series data of the audio signal, a fundamental frequency extraction unit that extracts an observation fundamental frequency sequence y representing a fundamental frequency at each time k of the audio signal,
A voiced unvoiced section estimation unit that estimates the degree of uncertainty of the fundamental frequency at each time k according to whether the time series data of the audio signal is a voiced section or an unvoiced section;
An initial value setting unit that sets an initial value of the command function o;
Based on the command function o updated last time or the initial value of the command function o, the logarithmic joint probability log p (y, o, s) of the observed fundamental frequency sequence y, the command function o, and the state sequence s is calculated. A state sequence updating unit that updates the state sequence s using a Viterbi algorithm so as to increase the objective function as an objective function;
Based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency sequence y, and the degree of uncertainty at each time k, the non-negative values are respectively increased so as to increase the objective function. A command parameter o, and a model parameter updating unit that updates the parameter group θ,
A convergence determining unit that repeats updating by the state sequence updating unit and updating by the model parameter updating unit until a predetermined convergence condition is satisfied;
Only including,
The state sequence update unit calculates the logarithm log p (o | s) of the conditional probability density function of the command function o and the probability distribution of the state sequence s, given the state sequence s, according to the following equation. A fundamental frequency model parameter estimating device that updates the state sequence s by using the Viterbi algorithm so as to increase the sum with the logarithm log p (s) .
  The model parameter update unit uses the auxiliary function method based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency sequence y, and the degree of the uncertainty at each time k. The fundamental frequency model parameter estimating apparatus according to claim 1, wherein the command function o and the parameter group θ each having a non-negative value are updated so as to increase the objective function. The model parameter update unit,
Based on the previous initial value of the phrase command u _p updated each time l [l] or phrase command u _p [l] at each time l, the time k, for each of the combinations of l (k, l), the auxiliary updates to calculate the variable lambda _{p, k, l,} and, based on the initial value of the accent command u _a of each time k was last updated [k] or accent command u _a of each time k [k], time k, for each of the combinations of l (k, l), the auxiliary variable lambda _{a, k,} and update calculated _l, and, last updated baseline component u _b or baseline initial value of the component u _b An auxiliary variable updating unit that calculates and updates an auxiliary variable λ _{b, k, l} for each combination (k, l) of times k and l based on
The observed fundamental frequency sequence y, the degree of uncertainty at each time k, and the auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k,} updated by the auxiliary variable updating unit based on the _l, functions as a lower bound of the objective function as an auxiliary function, so to increase the auxiliary function, the phrase command u _p at each time l [l] and accent command u _a [l], the base a command function updating unit that updates the line component u _b,
Until a predetermined convergence condition is satisfied, a convergence determination unit that repeats updating by the auxiliary variable updating unit and updating by the command function updating unit,
On the basis of the command phrase command u _p at each time l updated by the function updating unit [l], to update the parameter C ^(p) [k] of the state output distributions for phrase command at each time k, and the The parameter C _n ^(a) of the state output distribution of each accent command n is updated based on the accent command u _a [l] at each time l updated by the command function updating unit and the updated state sequence s. A state output distribution updating unit that updates the parameter group θ,
The fundamental frequency model parameter estimating device according to claim 2, comprising: An input audio signal, hiding the state sequence s made from the state s _k at each time k in Markov models, phrase command u _p representing the fundamental frequency pattern resulting from the translation movement of the thyroid cartilage at each time k [k] and thyroid a command function o of pairs o [k] of the accent command representing a fundamental frequency pattern generated by the rotation movement of cartilage u _a [k], the parameter C of the state output distributions for phrase command in accordance with the state s _k at each time k ^(p) A fundamental frequency model parameter estimating method in a fundamental frequency model parameter estimating apparatus for estimating [k] and a parameter group θ representing ^a parameter C _n ^(a) of ^a state output distribution of each accent command n,
A fundamental frequency extracting unit extracts, from the time-series data of the audio signal, an observation fundamental frequency sequence y representing a fundamental frequency at each time k of the audio signal,
The voiced unvoiced section estimation unit estimates the degree of uncertainty of the fundamental frequency at each time k according to whether the time-series data of the audio signal is a voiced section or an unvoiced section,
An initial value setting unit sets an initial value of the command function o,
A state sequence updating unit, based on the previously updated command function o or the initial value of the command function o, has a logarithmic joint probability log p (y of the observed fundamental frequency sequence y, the command function o, and the state sequence s. , O, s) as an objective function, update the state sequence s using the Viterbi algorithm so as to increase the objective function,
A model parameter updating unit increases the objective function based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency sequence y, and the degree of the uncertainty at each time k. In such a manner, the command function o, which is each a non-negative value, and the parameter group θ are updated,
The convergence determination unit repeats the update by the state sequence update unit and the update by the model parameter update unit until a predetermined convergence condition is satisfied.
Including
The update by the state sequence update unit includes the logarithm log p (o | s) of the conditional probability density function of the command function o and the state sequence s given by the state sequence s according to the following equation. A method for estimating a fundamental frequency model parameter that updates the state sequence s by using the Viterbi algorithm so as to increase the sum of the probability distribution and the logarithm log p (s) .
  By updating the model parameter updating unit, based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency sequence y, and the degree of the uncertainty at each time k, 5. The method of estimating a fundamental frequency model parameter according to claim 4, wherein the command function o and the parameter group θ each having a non-negative value are updated so as to increase the objective function by using a functional method. By the model parameter updating unit updating,
Auxiliary variable updating unit, based on the initial value of the phrase command u _p at each time l was last updated [l] or phrase command u _p at each time l [l], the time k, a combination of l (k, l for each), auxiliary variables lambda _{p, k,} is updated to calculate the _l, and, last accent command u _a of each time k is updated [k] or at each time k accent command u _a in [k] based on the initial value, the time k, for each of the combinations of l (k, l), the auxiliary variable lambda _{a, k,} and update calculated _l, and baseline components u _b or baseline was last updated based on the initial value of the component u _b, time k, for each of the combinations of l (k, l), the auxiliary variable lambda _{b, k,} to calculate the _l update,
A command function update unit updates the observed fundamental frequency sequence y, the degree of uncertainty at each time k, and the auxiliary variables λ _{p, k, l} , λ _{a, k, l} updated by the auxiliary variable update unit. , lambda _{b, k,} based on the _l, functions as a lower bound of the objective function as an auxiliary function, so to increase the auxiliary function, phrase command u _p [l] and accent command u _a of each time l and [l], to update the baseline component u _b,
A convergence determination unit causes the update by the auxiliary variable update unit and the update by the command function update unit to be repeated until a predetermined convergence condition is satisfied,
State power distribution updating unit, on the basis of the phrase command at each time l updated by the command function updating unit u _p [l], the parameters of the state output distributions for phrase command at each time k C ^(p) [k] , And based on the accent command u _a [l] at each time l updated by the command function updating unit and the updated state sequence s, the parameter of the state output distribution of each accent command n The fundamental frequency model parameter estimation method according to claim 5, wherein the parameter group θ is updated by updating C _n ^(a) .   A program for causing a computer to function as each unit of the fundamental frequency model parameter estimation device according to any one of claims 1 to 3.