JP6553584B2 - Basic frequency model parameter estimation apparatus, method, and program - Google Patents

Description

  The present invention relates to a fundamental frequency model parameter estimation device, method, and program, and more particularly to a fundamental frequency model parameter estimation device, method, and program for estimating parameters of an observed fundamental frequency sequence from a speech signal.

  Speech contains various information in addition to linguistic information, and is used for daily communication. We are conducting research on information processing and signal processing for analysis and synthesis of non-verbal information with the goal of constructing a framework for engineering such non-verbal information.

It is known that the fundamental frequency (F ₀ ) trajectory of speech contains a large amount of non-verbal information such as speaker nature, emotion and intention. Thus, modeling of F ₀ trajectory, speech synthesis, speaker recognition, emotion recognition, such as dialogue systems, is very effective in prosody information plays an important role applications. F ₀ locus, a component that varies slowly over the prosodic phrase (a phrase component) constituted by steeply changing component (accent component) in accordance with an accent. These components can be interpreted as corresponding to the translational motion and rotational motion of human thyroid cartilage, respectively, but based on this interpretation, a mathematical model that represents the logarithm F ₀ trajectory as the sum of these components (hereinafter, Fujisaki model has been proposed (Non-Patent Document 1). The Fujisaki model has parameters such as the occurrence time and duration of the phrase / accent command, the size of each command, etc., and when these parameters are set appropriately, it is known to approximate the measured trajectory very well. . In addition, the relevance of linguistic correspondence of parameters is also widely confirmed.

Since the parameters of the Fujisaki model described above can express prosodic features efficiently, it is useful if the parameters of the Fujisaki model can be estimated at high speed and with high accuracy from the measured F ₀ trajectory. However, this problem is not necessarily easy because it is originally a defect setting problem and that the Fujisaki model has constraints to be protected by linguistic knowledge. Previously inventors model the stochastic process of generating F ₀ pattern based Fujisaki model has been proposed a method of estimating the Expectation-Maximization (EM) algorithm the maximum likelihood parameters of Fujisaki model (Non Patent documents 2 to 4).

H. Fujisaki, O. Fujimura, Ed., “A note on the physical and physical basis for the phrase and accent components in the Voice fundamental frequency contour,” in Vocal Physiology: Voice Production, Mechanisms and Functions. New York, NY, NY, NY USA: Raven, 1988. H. Kameoka, J. L. Roux, and Y. Ohishi, “A statistical model of speech F0contours,” in Proc. SAPA, 2010, pp. 43−48. K. Yoshizato, H. Kameoka, D. Saito, and S. Sagayama, “Statistical approach to fujisaki-model parameter estimation from speech signals and its quantitative evaluation,” in Proc. Speech Prosody 2012, 2012, pp. 175-178. K. Yoshizato, H. Kameoka, D. Saito, and S. Sagayama, “Hidden Markov convolutive mixture model for pitch contour analysis of speech,” in Proc. . 2012.

  In the case of Japanese, it is assumed that the accent instruction of the Fujisaki model occurs at a time corresponding to the position of the accent. This means that the rise and fall times of the accent command tend to be the time of the syllable or mora boundary. Therefore, the estimation of the accent command sequence is assumed by introducing the assumption that the change of the spectral feature tends to become large at the time when the accent command rises, because the spectrum feature usually has a sudden change at the syllable or mora boundary. Improvement in accuracy can be expected.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a fundamental frequency model parameter estimation apparatus, method, and program capable of accurately estimating the parameters of the Fujisaki model.

The fundamental frequency model parameter estimation apparatus according to the present invention in order to achieve the object of, as an input audio signal, the state sequence s made from the state s _k at each time k in the hidden Markov model, the thyroid cartilage at each time k And a command function o consisting of a phrase command u _p [k] representing a fundamental frequency pattern generated by a parallel movement motion of a and an accent command u _a [k] representing a fundamental frequency pattern generated by a rotational motion of thyroid cartilage , The parameter C ^(p) [k] of the state output distribution of the phrase command corresponding to the state s _k at each time k and the parameter group θ representing the parameter C _n ^(a) of the state output distribution of each accent command n are estimated. A fundamental frequency model parameter estimation device that performs an observation feature vector representing a feature vector at each time k of the speech signal from time-series data of the speech signal. A feature vector sequence extracting unit for extracting a sequence v, a fundamental frequency extracting unit for extracting an observed fundamental frequency sequence y representing a fundamental frequency at each time k of the speech signal from the time series data of the speech signal, and the speech A voiced and unvoiced section estimation unit for estimating the degree of uncertainty of the fundamental frequency at each time k depending on whether the time series data of the signal is a voiced or unvoiced section, and an initial value of the command function o The observation fundamental frequency series y, the observation feature vector series v, the instruction function o, and the state, based on an initial value setting unit that sets a and the initial value of the instruction function o or the instruction function o updated previously. A state sequence updating unit for updating the state sequence s so as to increase the objective function with the log joint probability log p (y, v, o, s) of the sequence s as an objective function; In order to increase the objective function on the basis of the commanded function o or the initial value of the commanded function o, the observed fundamental frequency sequence y, and the degree of the uncertainty at each time k, each has a nonnegative value. Convergence determination for repeating updating by the state series updating unit and updating by the model parameter updating unit until the command function o and the model parameter updating unit for updating the parameter group θ satisfy the predetermined convergence condition It is comprised including a part.

Fundamental frequency model parameter estimation method according to the present invention, an input audio signal, the state sequence s made from the state s _k at each time k in the hidden Markov model, the fundamental frequency caused by translation motion of thyroid cartilage at each time k A command function o comprising a pair o [k] of a phrase command u _p [k] representing a pattern and an accent command u _a [k] representing a fundamental frequency pattern generated by the rotational movement of the thyroid cartilage, and a state s _{k at} each time _k In the fundamental frequency model parameter estimating device for estimating the parameter C ^(p) [k] of the state output distribution of the phrase command according to the parameter group θ representing the parameter C _n ^(a) of the state output distribution of each accent command n A fundamental frequency model parameter estimation method, wherein a feature vector sequence extraction unit extracts the audio signal from time-series data of the audio signal. An observation feature vector sequence v representing a feature vector at each time k is extracted, and a fundamental frequency extraction unit extracts an observation fundamental frequency sequence y representing a fundamental frequency at each time k of the speech signal from time series data of the speech signal. The voiced and unvoiced section estimation unit estimates the degree of uncertainty of the fundamental frequency at each time k according to whether the time series data of the voice signal is a voiced or unvoiced section. A value setting unit sets an initial value of the command function o, and a state series update unit, based on the command function o updated last time or the initial value of the command function o, the observation fundamental frequency sequence y, The state sequence s is increased such that the objective function is increased with the observation feature vector sequence v, the command function o, and the log joint probability log p (y, v, o, s) of the state sequence s as an objective function. The model function updating unit is configured to calculate the objective function based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency series y, and the degree of uncertainty at each time k. The command function o and the parameter group θ which are nonnegative values are updated so as to increase the value, and updating by the state series update unit until the convergence determination unit satisfies a predetermined convergence condition, and Update by the model parameter update unit is repeated.

  A program according to the present invention is a program for causing a computer to function as each unit of the fundamental frequency model parameter estimation apparatus.

  As described above, according to the fundamental frequency model parameter estimation device, method, and program of the present invention, from the time-series data of the audio signal, the observed feature vector sequence v representing the feature vector of each time k of the audio signal And the state series s is updated using the observed fundamental frequency series y, the observation feature vector series v, the command function o, and the logarithmic simultaneous probability log p (y, v, o, s) of the state series s as an objective function. By repeating updating of the command function o and the parameter group θ, which are respectively nonnegative values, it is possible to obtain an effect that the parameters of the Fujisaki model can be accurately estimated.

It is a figure for demonstrating the Fujisaki model. It is a figure for demonstrating HMM. It is a figure for demonstrating the division | segmentation of a state. It is the schematic which shows the structure of the fundamental frequency model parameter estimation apparatus which concerns on embodiment of this invention. It is a flowchart which shows the content of the fundamental frequency model parameter estimation process routine in the fundamental frequency model parameter estimation apparatus which concerns on embodiment of this invention. It is a figure which shows an experimental result. It is a figure which shows an experimental result.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the method proposed in the present invention, in order to realize parameter estimation of the Fujisaki model with high reproducibility of the observed F ₀ pattern, a probability model of the F ₀ pattern generation process based on the Fujisaki model is formulated and observed based on it. Assume that the F ₀ pattern has occurred. The parameter estimation algorithm of the Fujisaki model is also based on this probability model.

<Overview of the embodiment of the present invention>
The central idea of the above method is that the generation process of the phrase and accent command sequence is represented by a Hidden Markov Model (HMM). Therefore, in a state corresponding to the rise time and fall time of the accent command, the feature amount (hereinafter, delta feature amount) representing not only the value of the accent command but also the time fluctuation of the spectrum is simultaneously generated. By this extension, the state sequence estimation (phrase · accent command estimation) of the HMM can be performed using not only the fundamental frequency pattern but also the delta feature of the spectrum as a clue.

<Probability modeling of F ₀ trajectory>
The probability modeling of the F ₀ trajectory described in Non-Patent Document 4 will be described.

As shown in FIG. 1, the Fujisaki model (see Non-Patent Document 1) calculates the logarithm F ₀ trajectory y (t) as the sum of the following three components:

It is a model represented by. Here, t is a time, x _p (t) is a phrase component, x _a (t) is an accent component, and u _b is a constant not depending on time called a baseline component. Furthermore, the phrase component and accent component are output from the secondary filter of the signal called phrase command and accent command, respectively.

It is assumed that Here, u _p (t) is a pulse train called a phrase command, and u _a (t) is a rectangular pulse train called an accent command. Of these, at most one takes a non-zero value at each time. α and β are angular frequencies representing the response speed of the secondary filter, respectively, and are known to take values of approximately α = 3 rad / s and β = 20 rad / s regardless of the individual or speech.

Hereinafter, until we have this outlines the probabilistic model of the production process of F ₀ locus which is based on Fujisaki model has been developed (Non-Patent Document 4). In the Fujisaki model described above, it is assumed that the phrase command and the accent command are a delta train and a rectangular pulse train, respectively, and that these do not overlap each other. The central idea of the methods of Non Patent Literatures 2 to 4 lies in that the process of generating a phrase / accent command sequence is represented by a Hidden Markov Model (HMM). _Let k be the index of discrete time, and a pair of phrase command u _p [k] and accent command u _a [k]

And When the output distribution of each state is a normal distribution, the output series

Is

Obey. Here, s _k represents the state at time k. That is, equation (6) is average

And distributed

Means that changes with time as a result of state transitions. The advantage of HMMs is that they can flexibly set the constraints to be imposed on the series that they want to model through the design of the state transition network. The above-mentioned restrictions on the phrase command and the accent command can be expressed by a state transition network as shown in FIG. In addition, the duration of self-transition can be parameterized by dividing each state into several small states with the same output distribution.

FIG. 2 shows a state transition model of the phrase / accent command sequence in the conventional method (see Non-Patent Documents 2 to 4). In the state r ₀ , μ _p [k] and μ _a [k] are both 0. In the state p ₀ , μ _p [k] takes a non-negative value C ^(p) [k], and μ _a [k] becomes 0. In the state r ₁ , both μ _p [k] and μ _a [k] are ₀ as in the state r ₀ . Thus in the process of transition to a state r ₁ via the state p ₁ from the state r ₀ μ _p [k] is a pulse-like sequences. State r ₁ can only transition to states a ₀ ,..., A _N-1 and in these states μ _a [k] takes different values C ^(a) _n and μ _p [k] Will be 0. Directly without passing through the state r ₁ a _n from _{a n '(n ≠ n'} ) a μ by not able to transition to the [k] can be constrained to a rectangular pulse train.

Next, an example of dividing the state a _n a small state in FIG. For example, by setting the state transition probability from a _{n, m} to an _{n, m + 1} to ₁ for all m に_{対 し て} 0 as shown in FIG. 3, from a _{n, 0} to a _{n, m} transition probability corresponds to the probability that state a _n lasts only m step, it becomes possible to flexibly control the persistence length of the accent command. Similarly, by dividing p ₁ , p _0, and a ₀ into small states, it becomes possible to parameterize the distribution of the duration of the phrase command and the length of the interval between the commands. Based on these divisions, after that,

It is written as The configuration of the above HMM is as follows.

Phrase components and accent components are the convolution of different filters G _p [k] and G _a [k] into the command functions u _p [k] and u _a [k] output from the HMM.

It becomes. However, * represents the convolution regarding discrete time k. Also, G _p [k] and G _a [k] are discrete-time representations of G _p (t) and G _a (t), respectively. From the above, the discrete-time expression x [k] of the F ₀ locus is

It becomes. u _b represents a baseline component.

In the unvoiced interval, F ₀ may not be observed, or may not be reliable even if it is observed. In addition, an estimation error may occur in F ₀ extraction. Therefore, the observation F ₀ pattern y [k] and the above-mentioned F ₀ pattern model x [k] and noise

The uncertainty of the observed F ₀ pattern can be incorporated through the setting of the variance v ² _n [k]. That is, the observation F ₀ pattern y [k]

In this way, all observation intervals can be handled in a unified manner regardless of whether they are reliable intervals. Here, if x _n [k] is marginalized,

Under the given

Conditional probability density function of

Is

It becomes. From Equation (6), the state series

Under the given

Conditional probability density function of

Is

Given in. here,

Represents the series of mean and variance of the output distribution. State series

Probability distribution

Is the product of the transition probabilities according to the Markov assumption in HMM

Given in.

<Conventional parameter estimation algorithm>
In Non-Patent Documents 2 and 3, the observed F ₀ series

State sequence when given

Posterior probability of

Maximize

Has been proposed by using the EM algorithm. In Non-Patent Document 4, the observation F ₀ sequence is proposed.

State output sequence when given

Posterior probability of

The

Maximize so that each element of is non-negative

An algorithm that searches for EM by EM algorithm and auxiliary function method has been proposed. Also observed F ₀ series

State output sequence when given

And state series

Simultaneous posterior probability

Maximize

When

The

After fixing

To maximize

A step of updating

After fixing

To increase

It is also possible to search by repeating the step of updating under a non-negative constraint.

<Proposed parameter estimation algorithm>
In the algorithm proposed in the embodiment of the present invention, the F ₀ pattern

In addition to the spectral feature vector sequence

Becomes observation data,

Is newly considered as the state output distribution of the HMM described above. Spectral feature

For example, an absolute value of a delta feature value that is a temporal variation of a phoneme feature value such as a mel frequency cepstrum coefficient (MFCC) or a line spectrum pair (LSP) is used. Where G ₀ and G ₁ are parameters

Represents a random probability distribution model described by (for example, Gaussian mixture model (GMM)), and is assumed to be used as a probability distribution of spectral features in phoneme boundary and non-phoneme boundary sections, respectively. . Since states a _{n, 0} and r _1,0 indicate the timing of the start and end of the accent command, respectively,

Corresponds to the assumption that it follows the spectral feature distribution G _{0 at the} phoneme boundary, and otherwise follows the spectral feature distribution G ₁ at the non-phoneme boundary at the time. Parameters of G ₀ and G ₁

In advance, learning speech data is divided into phoneme boundary sections and non-phoneme boundary sections using forced phoneme alignment or phoneme segmentation, and learning is performed using spectral feature amounts in the respective sections. The objective function of the proposed Fujisaki model parameter estimation algorithm is

It is.

However,

Are respectively given by the equations (11), (13) and (6). Also,

It is. At this time,

Maximize locally

Can be searched by the following algorithm.

(Pre-learning step)
1. Divide learning speech data into phoneme boundary and non-phoneme boundary using forced phoneme alignment or phoneme segmentation.
2. Parameters of G ₀ and G ₁ using spectral features in phone boundary and non-phone boundary intervals, respectively.

To learn.

(Step 1: State series update step)
1. By the Viterbi algorithm

To be the largest

Update

(Step 2: Status output sequence update step)
1.

To increase

Update

<State series update step>
The state series update step is

After fixing

To be the largest

Is a step of updating

so

Terms that depend on

Because

Maximize

The question of asking is

It is isomorphic to the HMM's state series search problem with an output series. Therefore, it can be solved using the Viterbi algorithm.

<Status output sequence update step>
The state output sequence update step is

After fixing

To be the largest

Is a step of updating

so

Terms that depend on

And

When

Are each

Given in. Where G _b [k] = δ [k] (Kronecker's delta). Under the condition that the command function u _p [k], u _a [k] is nonnegative

Maximize

Is difficult to find directly, but locally maximized by iterative calculation based on the auxiliary function method

Can be explored. The auxiliary function method is a method of increasing the objective function by repeatedly increasing the lower bound function of the objective function to be maximized. The lower bound function of equation (14) is the Jensen inequality

It is possible to design using the fact that However,

Is called an auxiliary variable,

Meet. The equality condition of equation (16) is

It is.

Therefore,

And the right hand side is the auxiliary function

Call it When this auxiliary function is partially differentiated with respect to u _i [l],

So if you set this to 0,

Get As mentioned above, by repeating Formula (17) and Formula (20)

Can be increased.

Also,

HMM state output distribution parameter that maximizes

Is

By setting each partial derivative of x as 0

Given in. However,

Is the set of k such that s _k = a _n

Represents

Represents the number of elements in the set.

<System configuration>
Next, an embodiment of the present invention will be described by taking as an example a case where the present invention is applied to a fundamental frequency model parameter estimating device that analyzes time-series data of an observed speech signal and estimates parameters of the Fujisaki model. explain.

  As shown in FIG. 4, a fundamental frequency model parameter estimation apparatus 100 according to an embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program for executing a fundamental frequency model parameter estimation processing routine described later. And is functionally configured as follows.

  As shown in FIG. 4, the fundamental frequency model parameter estimation apparatus 100 includes a storage unit 10, a prior learning unit 11, a feature vector sequence extraction unit 1, a fundamental frequency sequence extraction unit 2, and a voiced and unvoiced section estimation unit 3. An initial value setting unit 4, a state series update unit 5, a model parameter update unit 6, a convergence determination unit 7, and an output unit 9 are provided.

  The storage unit 10 stores time series data of the observed voice signal and time series data of the learning voice signal. In the time series data of the speech signal for learning, a label indicating whether it is a section of a phoneme boundary or a section of a non-phoneme boundary is attached in advance to each time.

The feature vector sequence extraction unit 1 generates a spectrogram feature vector from the time-series data of the learning speech signal stored in the storage unit 10.

Spectrogram feature vector sequence which is a sequence of

Extract

In addition, the feature vector series extraction unit 1 calculates a spectrogram feature quantity vector from the time series data of the observed speech signal stored in the storage unit 10.

Series of

To extract.

The pre-learning unit 11 extracts a spectrumgram feature vector extracted from time-series data of a speech signal for learning

Parameters of the distribution G ₀ and G ₁ of the spectrogram feature vector in each of the section of the phoneme boundary and the section of the non-phoneme boundary based on and the label attached to the time-series data of the speech signal for learning

To learn

The fundamental frequency series extraction unit 2 extracts the time series data of the fundamental frequency from the time series data of the observed speech signal, converts them so as to express them at discrete time k, and when the fundamental frequency of the speech signal Observation fundamental frequency series which is series data

And This fundamental frequency extraction process can be realized by a well-known technique. For example, Non-Patent Document 5 (H. Kameoka, “Statistical speech spectrum model incorporating all-pole vocal tract model and F ₀ contour generating process model,” in Tech. Rep. IEICE, 2010, in Japanese.) The fundamental frequency is extracted every 8 ms.

The voiced / voiceless section estimation unit 3 identifies a voiced section and a voiceless section from the time-series data of the voice signal, and observes F ₀ [ k] Estimate the degree of uncertainty v _n ² [k]. In the unvoiced section, the degree of uncertainty is estimated to be large, and in the voiced section, the degree of uncertainty is estimated to be small.

The initial value setting unit 4 are each a parameter used in the process described later, sets the number N, the initial value regarded as constant u _b accent command. Set an appropriate value as the initial value. Further, the initial value setting unit 4 learns and determines the number of small states of the HMM, the transition probability φ _{i ′, I} from the correct answer data prepared in advance. Further, the initial value setting unit 4 uses the parameter estimation method of the conventionally known Fujisaki model,

Set the initial value (non-negative value) of. Further, the initial value setting unit 4 sets the initial value of C ^(p) [k] as:

The linear interpolation of the amplitude of the phrase command function of is set, and an appropriate value is set as the initial value of C _n ^(a) .

In this embodiment, Fujisaki model parameters

When

Is obtained by repeating the two steps of the state series update unit 5 and the model parameter update unit 6.

The state series update unit 5 is a command function updated last time.

Or command function

Parameters of feature vector distribution in each of the initial values of and the learned phoneme boundary sections and non-phoneme boundary sections

Based on the observed fundamental frequency series

, Spectogram feature vector series

, Command function

And state series

Logarithmic joint probability of

Using the Viterbi algorithm to increase the objective function with the objective function as

Update. In particular,

Using Viterbi algorithm to maximize the state sequence

Update.

However,

Is a parameter of the distribution of feature vectors in each of the learned phoneme boundary section and the non-phoneme boundary section

It is represented by the above formula (13) using

In the state corresponding to the segment of the phoneme boundary in, the state sequence according to the distribution G ₀ of the feature vector in the segment of the phoneme boundary

In the state corresponding to the section of the non-phoneme boundary, the state series follows the feature vector distribution G ₁ in the section of the non-phoneme boundary.

Is updated.

The model parameter update unit 6 uses the command function updated last time.

Or command function

Initial value, observed fundamental frequency series

, And the degree of uncertainty v _n ² [k] at each time k, using the auxiliary function method to increase the objective function, each with a non-negative command function

, And parameter group

Update
Specifically, the model parameter update unit 6 includes an auxiliary variable update unit 61, a command function update unit 62, a convergence determination unit 63, and a state output distribution update unit 64.

The auxiliary variable updating unit 61 performs all combinations (k, l) of times k and l (l <k) based on the phrase command u _p [k] (or initial value) at each time k updated last time. For each, the auxiliary variable λ _{p, k, l} is calculated and updated according to equation (17) above. In addition, the auxiliary variable update unit 61 performs the above equation (17) for all combinations of (k, l) based on the accent command u _a [k] (or initial value) at each time k updated last time. According to the above, the auxiliary variable λ _{a, k, l} is calculated and updated.

Further, the auxiliary variable updating unit 61 calculates and updates the auxiliary variables λ _{b, k, l} according to the above equation (17) for all combinations of (k, l) based on u _b .

The command function update unit 62 has a basic frequency sequence.

And the degree of uncertainty v _n ² [k] and the state series updated by the state series update unit 5

Based on the auxiliary variable λ _{p, k, l} updated by the auxiliary variable updating unit 61, the phrase command u _p [l] at each time l, which is a non-negative value, is updated according to the above equation (20).

The command function update unit 62 also has a basic frequency sequence.

And the degree of uncertainty v _n ² [k] and the state series updated by the state series update unit 5

Then, based on the auxiliary variables λ _{a, k, l} updated by the auxiliary variable update unit 61, the accent command u _a [l] at each time l which is a non-negative value is updated according to the above equation (20).

The command function update unit 62 also has a basic frequency sequence.

Based on the degree of uncertainty v _n ² [k] and the auxiliary variable λ _{b, k, l} updated by the auxiliary variable updating unit 61, the base component u _b is updated according to the above equation (20) .

  The convergence determination unit 63 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, each process of the auxiliary variable update unit 61 and the command function update unit 62 is repeated. If the convergence determination unit 63 determines that the convergence condition is satisfied, the convergence determination unit 63 proceeds to processing by the state output distribution update unit 64.

  As the convergence condition, it may be used that the number of repetitions s has reached a predetermined number of times S (for example, 20 times). The convergence condition is that the difference between the value of the auxiliary function when using the s-1st parameter and the value of the auxiliary function when using the sth parameter has become smaller than a predetermined threshold. It may be used as

Based on the phrase command u _p [k] at each time k updated by the command function updating unit 62, the state output distribution updating unit 64 calculates the state output distribution of the phrase command at each time k according to the above equation (21). The parameter C ^(p) [k] is updated, and based on the accent command u _a [k] at each time k updated by the command function update unit 62 and the state sequence s updated by the state sequence update unit 5. Parameter group _n by updating the parameter C _n ^(a) of the state output distribution of each accent command n according to the above equation (22)

Update

The convergence determination unit 7 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, the update value is updated again.

When

And the iterative algorithm (each process of the state series update unit 5 and the model parameter update unit 6) is repeated. If the convergence determination unit 7 determines that the convergence condition is satisfied, the convergence determination unit 7 proceeds to processing by the output unit 9.

  As the convergence condition, it may be used that the number of repetitions r has reached a predetermined number R (for example, 20 times). Note that the convergence condition is that the difference between the value of the objective function when using the r-1st parameter and the value of the objective function when using the rth parameter is smaller than a predetermined threshold. It may be used as

Then, the output function 9 causes the command function

, Parameter group

, State series

Output

<Operation of fundamental frequency model parameter estimation device>
Next, the operation of the fundamental frequency model parameter estimation device 100 according to the present embodiment will be described. First, time-series data of a speech signal for learning is input to the fundamental frequency model parameter estimation device 100 and stored in the storage unit 10. Then, in the fundamental frequency model parameter estimation apparatus 100, the spectrogram feature quantity vector is obtained from the time series data of the speech signal for learning stored in the storage unit 10.

Spectralgram feature vector sequence that is a sequence of

Are extracted, and the parameters of the distributions G ₀ and G ₁ of the spectrogram feature vectors in each of the phoneme boundary section and the non-phoneme boundary section are parameters.

Is learned.

  Then, time-series data of the observed speech signal is input to the fundamental frequency model parameter estimation device 100 as an analysis target, and stored in the storage unit 10. Then, the fundamental frequency model parameter estimation apparatus 100 executes a fundamental frequency model parameter estimation processing routine shown in FIG.

First, in step S100, the time series data of the observed audio signal is read from the storage unit 10, and the spectrogram feature vector at each time k is read.

Spectogram feature vector series consisting of

To extract.

In step S101, time series data of the observed audio signal is read from the storage unit 10, and a basic frequency series including the basic frequency F _{0 at} each time k is read.

Extract In step S102, voiced sections and unvoiced sections are identified based on time-series data of the speech signal, and the degree v _n ² [k] of uncertainty of the fundamental frequency at each time k is estimated.

In the next step S103, the parameters N, sets the appropriate initial value for u _b, the number of the small state of the HMM, the transition probabilities phi _{i ',} the _I, by learning from the correct answer data prepared previously determined Do. In addition, the command sequence by the conventional method

Is estimated and set as an initial value, and an initial value of C ^(p) [k] and an initial value of C _n ^(a) are set.

Then, in step S104, the command sequence set in step S103 is

Initial value, or the command sequence previously updated in step S105 described later

And the spectrogram feature vector sequence extracted in step S100.

And the parameters G ₀ and G ₁ of the distribution of the spectrogram feature vector in each of the phoneme boundary section and the non-phoneme boundary section obtained by the prior learning

And based on

State sequence using the Viterbi algorithm so that

Update

In step S105, the initial value of the phrase command u _p [k] at each time k set in the step S103, or in step S106, which will be described later, based on the phrase command u _p [k] at each time k, which was last updated The auxiliary variables λ _{p, k, l} are calculated and updated according to the above equation (17) for each of all combinations (k, l) at time k, l (l <k). Based on the initial value of the accent command u _a [k] at each time k set in step S103 above or the accent command u _a [k] at each time k previously updated in step S106 described later, (k, For all combinations of l), the auxiliary variable λ _{a, k, l} is calculated and updated according to the above equation (17). The initial value of u _b set in the step S103, or on the basis of a u _b it was last updated in step S106 to be described later, for all combinations of (k, l), according to the above equation (17), The auxiliary variable λ _{b, k, l} is calculated and updated.

In the next step S106, the fundamental frequency sequence calculated in the above step S101

And the degree of uncertainty v _n ² [k] calculated at step S102 and the state series updated at step S104

Based on the auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k, l} updated in step S105, each time l which is a non-negative value according to the above equation (20) Command series consisting of the phrase command u _p [l] and the accent command u _a [l]

And update the base component u _b .

In the next step S107, it is determined whether the number of repetitions s has reached S as a convergence condition, and if the number of repetitions s has not reached S, it is determined that the convergence condition is not satisfied. Then, the process returns to step S105, and the processes of steps S105 to S106 are repeated. On the other hand, when the number of repetitions s has reached S, it is determined that the convergence condition is satisfied, and in step S108, based on the phrase command u _p [k] at each time k updated in step S106 above. According to Equation (21), the parameter C ^(p) [k] of the phrase command state output distribution at each time k is updated, and the accent command u _a [k] at each time k updated in step S106, State series updated in step S104

And the parameter group C _n ^(a) of the state output distribution of each accent command n according to the above equation (22).

Update

Then, in step S109, it is determined whether the number of repetitions r has reached R as a convergence condition, and if the number of repetitions r has not reached R, it is determined that the convergence condition is not satisfied. Then, the process returns to step S104, and the processes of steps S104 to S108 are repeated. On the other hand, when the number of repetitions r reaches R, it is determined that the convergence condition is satisfied, and the output function 9

, Parameter group

, State series

Is output and the fundamental frequency model parameter estimation processing routine is terminated.

<Experiment>
In order to verify the parameter estimation accuracy of the proposed method described in the embodiment of the present invention, the speech data of set B of ATR digital speech database (503 speech read by one male Japanese speaker (MHT)) is used Evaluation experiments were conducted.

The correct data for the phrase / accent command was manually labeled by an expert. Estimation of F ₀ tracks from the audio data is carried out by using an existing method, the value of the baseline frequency u _b is the value of the minimum value of F ₀ in voiced segments for each reading sentence. The initial value of each phrase / accent command sequence was set to the value obtained by the conventional method. The number of iterations was 20 times. F ₀ pattern model composed of observed F ₀ patterns and estimated phrase / accent commands

The mean square error (RMSE) in the voiced interval between and the phrase / accent command dropout rate and error insertion rate were used as evaluation indices for parameter estimation accuracy.

  Figures 6 and 7 show the results of RMSE and erroneous insertion / dropout rates for each method. Compared to the conventional method, the RMSE value improved by about 5% in all the variations of the proposed method. As for the misinsertion / dropping rate, the most remarkable improvement was seen when using ΔMFCC as the spectral feature, and the error could be reduced by about 3%.

As described above, according to the fundamental frequency model parameter estimation device according to the embodiment of the present invention, a spectral feature vector sequence is extracted from time series data of an audio signal, and an observation fundamental frequency sequence, a spectral feature vector sequence, Logarithmic joint probability of command function and state series

The parameters of the Fujisaki model can be estimated with high accuracy by repeating updating the state sequence and updating the command function and the parameter group with the objective function as the objective function.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the scope of the present invention.

  For example, although the above-described fundamental frequency model parameter estimation device has a computer system inside, the "computer system" is also a homepage providing environment (or display environment) if it is using a WWW system. Shall be included.

  Furthermore, although the present invention has been described as an embodiment in which the program is installed in advance, it is also possible to provide the program by storing the program in a computer readable recording medium.

1 feature vector sequence extraction unit 2 fundamental frequency sequence extraction unit 3 voiced unvoiced section estimation unit 4 initial value setting unit 5 state sequence update unit 6 model parameter update unit 7 convergence determination unit 10 storage unit 11 prior learning unit 61 auxiliary variable update unit 62 Command function update unit 63 Convergence determination unit 64 State output distribution update unit 100 Basic frequency model parameter estimation device

Claims (7)

An input audio signal, hiding the state sequence s made from the state s _k at each time k in Markov models, phrase command u _p representing the fundamental frequency pattern resulting from the translation movement of the thyroid cartilage at each time k [k] and thyroid A command function o composed of a pair o [k] of accent commands u _a [k] representing a fundamental frequency pattern generated by the rotational motion of the cartilage, and a parameter C of the phrase command state output distribution according to the state s _k at each time k ^(p) A fundamental frequency model parameter estimating apparatus for estimating [k] and a parameter group θ representing ^a parameter C _n ^(a) of ^a state output distribution of each accent command n,
A feature vector sequence extraction unit that extracts an observation feature vector sequence v representing a feature vector at each time k of the audio signal from the time series data of the audio signal;
A fundamental frequency extraction unit for extracting an observed fundamental frequency sequence y representing a fundamental frequency at each time k of the speech signal from the time series data of the speech signal;
About the time-series data of the speech signal, a voiced / unvoiced section estimation unit that estimates the degree of uncertainty of the fundamental frequency at each time k, depending on whether it is a voiced section or an unvoiced section;
An initial value setting unit for setting an initial value of the command function o;
Based on the previously updated command function o or the initial value of the command function o, the logarithmic simultaneous probability log p (of the observed fundamental frequency sequence y, the observed feature vector sequence v, the command function o, and the state sequence s ( a state sequence update unit that updates the state sequence s so that the objective function is increased with y, v, o, s) as an objective function;
Based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency series y, and the degree of uncertainty at each time k, each non-negative value is increased. A model parameter update unit for updating the command function o and the parameter group θ,
A convergence determination unit that repeats the update by the state series update unit and the update by the model parameter update unit until a predetermined convergence condition is satisfied,
A fundamental frequency model parameter estimation apparatus including: A pre-learning unit that learns the distribution of feature vectors in each of the phoneme boundary section and the non-phoneme boundary section based on the feature vector at each time extracted from the time-series data of the speech signal for learning;
The state series update unit
In the state sequence s, based on the previously updated command function o or the initial value of the command function o, and the distribution of feature vectors in each of the section of the phoneme boundary and the section of the non-phoneme boundary In the state corresponding to the phoneme boundary section, according to the feature vector distribution in the phoneme boundary section, in the state sequence s in the state corresponding to the nonphoneme boundary section, the feature vector of the nonphoneme boundary section The fundamental frequency model parameter estimation apparatus according to claim 1, wherein the state series s is updated so as to follow a distribution.   The state series update unit, based on the initial value of the command function o or the command function o updated last time, and the distribution of feature vectors in each of the phoneme boundary section and the non-phoneme boundary section, The fundamental frequency model parameter estimation apparatus according to claim 2, wherein the state sequence s is updated using a Viterbi algorithm so as to increase log p (v | s) + log p (o | s) + log p (s). An input audio signal, hiding the state sequence s made from the state s _k at each time k in Markov models, phrase command u _p representing the fundamental frequency pattern resulting from the translation movement of the thyroid cartilage at each time k [k] and thyroid A command function o composed of a pair o [k] of accent commands u _a [k] representing a fundamental frequency pattern generated by the rotational motion of the cartilage, and a parameter C of the phrase command state output distribution according to the state s _k at each time k ^(p) A fundamental frequency model parameter estimation method in a fundamental frequency model parameter estimation device for estimating a parameter group θ representing [k] and a parameter C _n ^(a) of ^a state output distribution of each accent command n,
A feature vector series extraction unit extracts an observed feature vector series v representing a feature vector at each time k of the voice signal from the time series data of the voice signal;
A fundamental frequency extraction unit extracts an observed fundamental frequency sequence y representing a fundamental frequency at each time k of the audio signal from the time series data of the audio signal;
A voiced unvoiced section estimation unit estimates the degree of uncertainty of the fundamental frequency at each time k according to which of a voiced section and an unvoiced section the time series data of the voice signal;
An initial value setting unit sets an initial value of the command function o,
The state series update unit is configured to select one of the observed fundamental frequency series y, the observed feature vector series v, the command function o, and the state series s based on the previously updated command function o or the initial value of the command function o. Updating the state sequence s so as to increase the objective function with the logarithmic joint probability log p (y, v, o, s) as an objective function;
A model parameter updating unit increases the objective function based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency series y, and the degree of uncertainty at each time k And update the command function o and the parameter group θ, each of which is a non-negative value,
The basic frequency model parameter estimation method, wherein the convergence determination unit repeats the update by the state series update unit and the update by the model parameter update unit until the convergence condition is determined in advance. The pre-learning unit learns the distribution of the feature vector in each of the phoneme boundary section and the non-phoneme boundary section based on the feature vector at each time extracted from the time series data of the speech signal for learning. In addition,
By updating the state series update unit,
In the state sequence s, based on the previously updated command function o or the initial value of the command function o, and the distribution of feature vectors in each of the section of the phoneme boundary and the section of the non-phoneme boundary In the state corresponding to the phoneme boundary section, according to the feature vector distribution in the phoneme boundary section, in the state sequence s in the state corresponding to the nonphoneme boundary section, the feature vector of the nonphoneme boundary section 5. The fundamental frequency model parameter estimation method according to claim 4, wherein the state series s is updated so as to follow a distribution.   The state series update unit updates the command function o updated last time or the initial value of the command function o, the distribution of feature vectors in each of the phoneme boundary section and the non-phoneme boundary section, 6. The fundamental frequency model according to claim 5, wherein the state sequence s is updated using a Viterbi algorithm so as to increase log p (v | s) + log p (o | s) + log p (s) based on Parameter estimation method.   The program for functioning a computer as each part of the fundamental frequency model parameter estimation apparatus of any one of Claims 1-3.