JP2018097115A - Fundamental frequency model parameter estimation device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to accurately estimate a Fujisaki-model parameter.SOLUTION: A characteristic vector series extraction unit 1 is configured to extract an observation characteristic vector series from time-series data on a voice signal, and a fundamental frequency extraction unit 2 is configured to extract an observation fundamental frequency series from the time-series data on the voice signal. A vocal/voiceless segment estimation unit 3 is configured to estimate a degree of uncertainty of the fundamental frequency in each time k. A state series update unit 5 is configured to update a state series s, using a Viterbi algorithm with a logarithm concurrent probability log p(y,o,s) determined as an object function. A model parameter update unit 6 is configured to update a command function o and a parameter group θ, which are non-negative values, so as to increase the object function. A calculation by the state series update unit 5 and updating by the model parameter update unit 6 are repetitively conducted until a predetermined convergence condition is satisfied.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a fundamental frequency model parameter estimation device, method, and program, and more particularly, to a fundamental frequency model parameter estimation device, method, and program for estimating parameters of an observed fundamental frequency sequence from a speech signal.

  Voices contain various information in addition to language information and are used for daily communication. We are researching information processing and signal processing for analyzing and synthesizing non-linguistic information with the goal of constructing a framework for engineering these non-linguistic information.

It is known that the fundamental frequency (F ₀ ) trajectory of speech contains abundant non-linguistic information such as speaker characteristics, emotions and intentions. For this reason, F ₀ trajectory modeling is extremely effective in applications where prosodic information plays an important role, such as speech synthesis, speaker recognition, emotion recognition, and dialogue systems. The F ₀ locus is composed of a component (phrase component) that changes gently over the entire prosodic phrase and a component (accent component) that changes sharply according to the accent. These components can be interpreted as corresponding to the translational motion and rotational motion of human thyroid cartilage, respectively, but based on this interpretation, a mathematical model that represents the logarithm F ₀ trajectory as the sum of these components (hereinafter, Fujisaki model) has been proposed (Non-Patent Document 1). The Fujisaki model has parameters such as the occurrence time and duration of the phrase / accent command, the size of each command, etc., and when these parameters are set appropriately, it is known to approximate the measured trajectory very well. . In addition, the validity of the linguistic correspondence of parameters has been widely confirmed.

Since the parameters of the above-mentioned Fujisaki model can express prosodic features efficiently, it is useful if the parameters of the Fujisaki model can be estimated at high speed and with high accuracy from the measured F ₀ trajectory. However, this problem was originally a failure setting problem, and the Fujisaki model was not always easy because there were restrictions that should be observed based on linguistic knowledge. Previously inventors model the stochastic process of generating F ₀ pattern based Fujisaki model has been proposed a method of estimating the Expectation-Maximization (EM) algorithm the maximum likelihood parameters of Fujisaki model (Non Patent Documents 2 to 4).

H. Fujisaki, O. Fujimura, Ed., “A note on the physiological and physical basis for the phrase and accent components in the Voice fundamental frequency contour,” in Vocal Physiology: Voice Production, Mechanisms and Functions.New York, NY, USA: Raven, 1988. H. Kameoka, J. L. Roux, and Y. Ohishi, “A statistical model of speech F0contours,” in Proc. SAPA, 2010, pp. 43−48. K. Yoshizato, H. Kameoka, D. Saito, and S. Sagayama, “Statistical approach to fujisaki-model parameter estimation from speech signals and its quantitative evaluation,” in Proc. Speech Prosody 2012, 2012, pp. 175-178. K. Yoshizato, H. Kameoka, D. Saito, and S. Sagayama, “Hidden Markov convolutive mixture model for pitch contour analysis of speech,” in Proc. The 13th Annual Conference of the International Speech Communication Association (Interspeech 2012), Sep . 2012.

  The Fujisaki model's accent command is assumed to occur at the time corresponding to the accent position in Japanese. This means that the rise time and fall time of the accent command tend to be the time of the syllable or mora boundary. Therefore, the estimation of the accent command sequence is assumed by introducing the assumption that the change of the spectral feature tends to become large at the time when the accent command rises, because the spectrum feature usually has a sudden change at the syllable or mora boundary. Improvement in accuracy can be expected.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a fundamental frequency model parameter estimation apparatus, method, and program capable of accurately estimating the parameters of the Fujisaki model.

The fundamental frequency model parameter estimation apparatus according to the present invention in order to achieve the object of, as an input audio signal, the state sequence s made from the state s _k at each time k in the hidden Markov model, the thyroid cartilage at each time k A command function o consisting of a pair o [k] of a phrase command u _p [k] representing a fundamental frequency pattern generated by the translational motion of the centroid and an accent command u _a [k] representing a fundamental frequency pattern generated by the rotational motion of the thyroid cartilage , The parameter C ^(p) [k] of the state output distribution of the phrase command corresponding to the state s _k at each time k and the parameter group θ representing the parameter C _n ^(a) of the state output distribution of each accent command n are estimated. A fundamental frequency model parameter estimation device that performs an observation feature vector representing a feature vector at each time k of the speech signal from time-series data of the speech signal. A feature vector sequence extracting unit for extracting a sequence v, a fundamental frequency extracting unit for extracting an observed fundamental frequency sequence y representing a fundamental frequency at each time k of the speech signal from the time series data of the speech signal, and the speech A voiced and unvoiced section estimation unit for estimating the degree of uncertainty of the fundamental frequency at each time k depending on whether the time series data of the signal is a voiced or unvoiced section, and an initial value of the command function o Based on the initial value setting unit that sets the command function o or the initial value of the command function o updated last time, the observation basic frequency series y, the observation feature vector series v, the command function o, and the state A state series update unit for updating the state series s so as to increase the objective function using a logarithmic simultaneous probability log p (y, v, o, s) of the series s as an objective function, and a previous update The command function o or the initial value of the command function o, the observed fundamental frequency sequence y, and the degree of uncertainty at each time k are each non-negative so as to increase the objective function. A model parameter update unit that updates the command function o and the parameter group θ, and a convergence determination that repeats the update by the state series update unit and the update by the model parameter update unit until a predetermined convergence condition is satisfied. Part.

Fundamental frequency model parameter estimation method according to the present invention, an input audio signal, the state sequence s made from the state s _k at each time k in the hidden Markov model, the fundamental frequency caused by translation motion of thyroid cartilage at each time k A command function o composed of a pair o [k] of a phrase command u _p [k] representing a pattern and an accent command u _a [k] representing a fundamental frequency pattern generated by the rotational motion of the thyroid cartilage, and a state s _{k at} each time _k In the fundamental frequency model parameter estimation apparatus for estimating the parameter C ^(p) [k] of the state output distribution of the phrase command according to the parameter group and the parameter group θ representing the parameter C _n ^(a) of the state output distribution of each accent command n A fundamental frequency model parameter estimation method, wherein a feature vector sequence extraction unit extracts the audio signal from time-series data of the audio signal. An observed feature vector sequence v representing a feature vector at each time k is extracted, and a fundamental frequency extraction unit obtains an observed fundamental frequency sequence y representing a fundamental frequency at each time k of the speech signal from the time series data of the speech signal. The voiced and unvoiced section estimation unit estimates the degree of uncertainty of the fundamental frequency at each time k according to whether the time series data of the voice signal is a voiced or unvoiced section. A value setting unit sets an initial value of the command function o, and a state series update unit, based on the command function o updated last time or the initial value of the command function o, the observation fundamental frequency sequence y, Using the observed feature vector series v, the command function o, and the logarithmic simultaneous probability log p (y, v, o, s) of the state series s as the objective function, the state series s is increased. A new model parameter update unit determines whether the objective function is based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency sequence y, and the degree of uncertainty at each time k. To update the command function o and the parameter group θ, each of which is a non-negative value, until the convergence determination unit satisfies a predetermined convergence condition, the update by the state series update unit, and Updates by the model parameter update unit are repeated.

  A program according to the present invention is a program for causing a computer to function as each unit of the fundamental frequency model parameter estimation apparatus.

  As described above, according to the fundamental frequency model parameter estimation apparatus, method, and program of the present invention, the observed feature vector sequence v representing the feature vector at each time k of the speech signal from the time series data of the speech signal. And the state series s is updated using the observed fundamental frequency series y, the observation feature vector series v, the command function o, and the logarithmic simultaneous probability log p (y, v, o, s) of the state series s as an objective function. By repeatedly updating the non-negative command function o and the parameter group θ, it is possible to obtain the effect that the parameters of the Fujisaki model can be estimated with high accuracy.

It is a figure for demonstrating the Fujisaki model. It is a figure for demonstrating HMM. It is a figure for demonstrating the division | segmentation of a state. It is the schematic which shows the structure of the fundamental frequency model parameter estimation apparatus which concerns on embodiment of this invention. It is a flowchart which shows the content of the fundamental frequency model parameter estimation process routine in the fundamental frequency model parameter estimation apparatus which concerns on embodiment of this invention. It is a figure which shows an experimental result. It is a figure which shows an experimental result.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the method proposed in the present invention, in order to realize parameter estimation of the Fujisaki model with high reproducibility of the observed F ₀ pattern, a probability model of the F ₀ pattern generation process based on the Fujisaki model is formulated and observed based on it. Assume that an F ₀ pattern has occurred. The parameter estimation algorithm of the Fujisaki model is also based on this probability model.

<Outline of Embodiment of the Present Invention>
The central idea of the above method is that the generation process of the phrase / accent command sequence is expressed by a hidden Markov model (HMM). Therefore, in a state corresponding to the rise time and fall time of the accent command, it is expanded so that not only the value of the accent command but also a feature amount (hereinafter referred to as a delta feature amount) representing a temporal variation of the spectrum is generated. With this extension, it is possible to perform HMM state sequence estimation (phrase / accent command estimation) using not only the fundamental frequency pattern but also the delta feature quantity of the spectrum as a clue.

<Probability modeling of F ₀ trajectory>
The probability modeling of the F ₀ locus described in Non-Patent Document 4 will be described.

As shown in FIG. 1, the Fujisaki model (see Non-Patent Document 1) uses a logarithm F ₀ trajectory y (t) of the following three components:

It is a model represented by. Here, t is a time-dependent constant called x time (time), x _p (t) is a phrase component, x _a (t) is an accent component, and u _b is a baseline component. Furthermore, the phrase component and accent component are output from the secondary filter of the signal called phrase command and accent command, respectively.

It is assumed that Here, u _p (t) is a pulse train called a phrase command, and u _a (t) is a rectangular pulse train called an accent command. Of these, at most one takes a non-zero value at each time. α and β are angular frequencies representing the response speed of the secondary filter, respectively, and are known to take values of approximately α = 3 rad / s and β = 20 rad / s regardless of the individual or speech.

The following outlines the stochastic model (Non-patent Document 4) of the F ₀ locus generation process based on the Fujisaki model that has been developed by the inventors. In the above-mentioned Fujisaki model, the phrase command and the accent command are a delta train and a rectangular pulse train, respectively, and further, it is assumed that they do not overlap each other. The central idea of the methods of Non-Patent Documents 2 to 4 is that the generation process of the phrase / accent command sequence is expressed by a hidden Markov model (HMM). The discrete time index is k, and the phrase command u _p [k] and accent command u _a [k] are paired.

And When the output distribution of each state is a normal distribution, the output series

Is

Follow. Here, s _k represents the state at time k 1. That is, equation (6) is average

And distributed

Means that changes with time as a result of state transitions. The advantage of HMM is that you can flexibly set the constraints to be imposed on the sequence you want to model through the design of the state transition network. The above-mentioned restrictions on the phrase command and the accent command can be expressed by a state transition network as shown in FIG. In addition, the duration of self-transition can be parameterized by dividing each state into several small states with the same output distribution.

FIG. 2 shows a state transition model of the phrase / accent command sequence in the conventional method (see Non-Patent Documents 2 to 4). In state r ₀ , μ _p [k] and μ _a [k] are both 0. In the state p ₀ , μ _p [k] takes a non-negative value C ^(p) [k], and μ _a [k] becomes 0. In the state r ₁ , both μ _p [k] and μ _a [k] are ₀ as in the state r ₀ . Therefore, μ _p [k] becomes a pulse-like sequence in the process of transition from the state r ₀ to the state r ₁ via the state p ₁ . State r ₁ can only transition to states a ₀ , ..., a _N−1 , in which μ _a [k] takes different values C ^(a) _n and μ _p [k] Becomes 0. Directly without passing through the state r ₁ a _n from _{a n '(n ≠ n'} ) a μ by not able to transition to the [k] can be constrained to a rectangular pulse train.

Next, an example of dividing the state a _n a small state in FIG. For example, as shown in FIG. 3, by setting the state transition probability from a _{n, m} to a _{n, m + 1} to ₁ for all m ≠ 0 _{, the change} from a _{n, 0} to a _{n, m} transition probability corresponds to the probability that state a _n lasts only m step, it becomes possible to flexibly control the persistence length of the accent command. Similarly, by dividing p ₁ , p _0, and a ₀ into small states, it becomes possible to parameterize the distribution of the duration of the phrase command and the length of the interval between commands. Based on these divisions, after that,

Is written. The configuration of the above HMM is as follows.

Phrase components and accent components are the convolution of different filters G _p [k] and G _a [k] into the command functions u _p [k] and u _a [k] output from the HMM.

It becomes. However, * represents the convolution regarding discrete time k. G _p [k] and G _a [k] are discrete time representations of G _p (t) and G _a (t), respectively. From the above, the discrete time representation x [k] of the F ₀ trajectory is

It becomes. u _b represents a baseline component.

In the silent section, F ₀ may not be observed, and even if it is observed, it may not be reliable. In addition, an estimation error may occur in F ₀ extraction. Therefore, the observed F ₀ pattern y [k], the above F ₀ pattern model x [k] and noise

, The uncertainty of the observed F ₀ pattern can be incorporated through the setting of the variance v ² _n [k]. That is, the observed F ₀ pattern y [k]

This means that all observation intervals can be handled uniformly regardless of whether they are reliable intervals. Here, if x _n [k] is marginalized,

Under the given

Conditional probability density function of

Is

It becomes. From Equation (6), the state series

Under the given

Conditional probability density function of

Is

Given in. here,

Represents the series of mean and variance of the output distribution. State series

Probability distribution

Is the product of transition probabilities based on the Markov assumption in HMM.

Given in.

<Conventional parameter estimation algorithm>
In Non-Patent Documents 2 and 3, the observed F ₀ series

State sequence when given

Posterior probability of

Maximize

Has been proposed by using the EM algorithm. In Non-Patent Document 4, the observation F ₀ sequence is proposed.

State output sequence when given

Posterior probability of

The

Maximize so that each element of is non-negative

An algorithm that searches for EM by EM algorithm and auxiliary function method has been proposed. Also observed F ₀ series

State output sequence when given

And state series

Simultaneous posterior probability

Maximize

When

The

After fixing

To maximize

A step of updating

After fixing

To increase

It is also possible to search by repeating the step of updating under a non-negative constraint.

<Proposed parameter estimation algorithm>
In the algorithm proposed in the embodiment of the present invention, the F ₀ pattern

In addition to spectral feature vector series

Becomes observation data,

Is newly considered as the state output distribution of the HMM described above. Spectral feature

For example, an absolute value of a delta feature value that is a temporal variation of a phoneme feature value such as a mel frequency cepstrum coefficient (MFCC) or a line spectrum pair (LSP) is used. Where G ₀ and G ₁ are parameters

Represents a random probability distribution model described by (for example, Gaussian mixture model (GMM)), and is assumed to be used as a probability distribution of spectral features in phoneme boundary and non-phoneme boundary sections, respectively. . Since states a _{n, 0} and r _1,0 indicate the timing of the start and end of the accent command, respectively,

Corresponds to the assumption that it follows the spectral feature distribution G _{0 at the} phoneme boundary, and otherwise follows the spectral feature distribution G ₁ at the non-phoneme boundary at the time. G ₀ and G ₁ parameters

In advance, learning speech data is divided into phoneme boundary sections and non-phoneme boundary sections using forced phoneme alignment or phoneme segmentation, and learning is performed using spectral feature amounts in the respective sections. The objective function of the proposed Fujisaki model parameter estimation algorithm is

It is.

However,

Are given by equations (11), (13), and (6), respectively. Also,

It is. At this time,

Maximize locally

Can be searched by the following algorithm.

(Pre-learning step)
1. Divide learning speech data into phoneme boundary and non-phoneme boundary using forced phoneme alignment or phoneme segmentation.
2. Parameters of G ₀ and G ₁ using spectral features in phone boundary and non-phone boundary intervals, respectively.

To learn.

(Step 1: State series update step)
1. By Viterbi algorithm

To maximize

Update.

(Step 2: State output series update step)
1.

To increase

Update.

<State series update step>
The state series update step

After fixing

To maximize

Is a step of updating.

so

The term that depends on

Because

Maximize

The problem

Is the same form as the HMM state sequence search problem. Therefore, it can be solved using the Viterbi algorithm.

<Status output sequence update step>
The status output series update step

After fixing

To maximize

Is a step of updating.

so

The term that depends on

And

When

Each

Given in. However, G _b [k] = δ [k] (Kronecker delta). Under the condition that the command function u _p [k], u _a [k] is non-negative

Maximize

Is difficult to find directly, but is locally maximized by iterative calculation based on the auxiliary function method

Can be explored. The auxiliary function method is a method of increasing the objective function by repeatedly increasing the lower bound function of the objective function to be maximized. The lower bound of equation (14) is the Jensen inequality

It is possible to design using the fact that However,

Is called an auxiliary variable,

Meet. The condition for the equality in equation (16) is

It is.

Therefore,

And the right side is the auxiliary function

Call it. When this auxiliary function is partially differentiated with respect to u _i [l],

So if you set this to 0,

Get. From the above, by repeating Equation (17) and Equation (20)

Can be increased.

Also,

HMM state output distribution parameter that maximizes

Is

By substituting 0 for each partial derivative

Given in. However,

Is the set of k such that s _k = a _n

Represents

Represents the number of elements in the set.

<System configuration>
Next, the embodiment of the present invention will be described with reference to an example in which the present invention is applied to a fundamental frequency model parameter estimation apparatus that analyzes time series data of an observed speech signal and estimates parameters of the Fujisaki model. explain.

  As shown in FIG. 4, a fundamental frequency model parameter estimation apparatus 100 according to an embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program for executing a fundamental frequency model parameter estimation processing routine described later. And is functionally configured as follows.

  As shown in FIG. 4, the fundamental frequency model parameter estimation apparatus 100 includes a storage unit 10, a prior learning unit 11, a feature vector sequence extraction unit 1, a fundamental frequency sequence extraction unit 2, and a voiced and unvoiced section estimation unit 3. , An initial value setting unit 4, a state series update unit 5, a model parameter update unit 6, a convergence determination unit 7, and an output unit 9.

  The storage unit 10 stores time series data of the observed voice signal and time series data of the learning voice signal. In the time-series data of the speech signal for learning, a label indicating whether it is a phoneme boundary section or a non-phoneme boundary section is assigned in advance to each time.

The feature vector series extraction unit 1 calculates a spectrogram feature quantity vector from the time series data of the speech signal for learning stored in the storage unit 10.

Spectralgram feature vector sequence that is a sequence of

To extract.

In addition, the feature vector series extraction unit 1 calculates a spectrogram feature quantity vector from the time series data of the observed speech signal stored in the storage unit 10.

Series

To extract.

The pre-learning unit 11 extracts a spectrumgram feature vector extracted from time-series data of a speech signal for learning

And the parameters of the distributions G ₀ and G ₁ of the spectrogram feature vectors in each of the phoneme boundary section and the non-phoneme boundary section based on the labels given to the time-series data of the learning speech signal

To learn.

The fundamental frequency series extraction unit 2 extracts time series data of the fundamental frequency from the observed time series data of the audio signal, converts the time series data so as to be expressed in the discrete time k, and outputs the time of the fundamental frequency of the audio signal. Observational fundamental frequency series that is series data

And This fundamental frequency extraction process can be realized by a well-known technique. For example, Non-Patent Document 5 (H. Kameoka, “Statistical speech spectrum model incorporating all-pole vocal tract model and F ₀ contour generating process model,” in Tech. Rep. IEICE, 2010, in Japanese.) The fundamental frequency is extracted every 8 ms.

The voiced / voiceless section estimation unit 3 identifies a voiced section and a voiceless section from the time-series data of the voice signal, and observes F ₀ [ k] Estimate the degree of uncertainty v _n ² [k]. In the unvoiced section, the degree of uncertainty is estimated to be large, and in the voiced section, the degree of uncertainty is estimated to be small.

The initial value setting unit 4 are each a parameter used in the process described later, sets the number N, the initial value regarded as constant u _b accent command. Set an appropriate value as the initial value. The initial value setting unit 4 determines the number of small states of the HMM and the transition probability φ _{i ′, I} by learning from correct data prepared in advance. Further, the initial value setting unit 4 uses a conventionally known parameter estimation method of the Fujisaki model,

Set the initial value (non-negative value) of. Moreover, the initial value setting unit 4 uses the initial value of C ^(p) [k] as

Is set by linearly interpolating the amplitude of the phrase command function, and an appropriate value is set as the initial value of C _n ^(a) .

In this embodiment, Fujisaki model parameters

When

Is obtained by repeating the two steps of the state series update unit 5 and the model parameter update unit 6.

The state series update unit 5 is a command function updated last time.

Or command function

Parameters of feature vector distribution in each of the initial values of and the learned phoneme boundary sections and non-phoneme boundary sections

Based on the observed fundamental frequency series

, Spectogram feature vector series

, Command function

And state series

Logarithmic joint probability of

Using the Viterbi algorithm to increase the objective function with the objective function as

Update. In particular,

Using Viterbi algorithm to maximize the state sequence

Update.

However,

Is a parameter of the distribution of feature vectors in each of the learned phoneme boundary section and the non-phoneme boundary section

It is represented by the above formula (13) using

In the state corresponding to the phoneme boundary section, the state series according to the feature vector distribution G ₀ in the phoneme boundary section

In the state corresponding to the section of the non-phoneme boundary, the state series follows the feature vector distribution G ₁ in the section of the non-phoneme boundary.

Is updated.

The model parameter update unit 6 uses the command function updated last time.

Or directive function

Initial value, observed fundamental frequency series

, And a non-negative command function using the auxiliary function method to increase the objective function based on the degree of uncertainty v _n ² [k] at each time k

And parameters

Update.
Specifically, the model parameter update unit 6 includes an auxiliary variable update unit 61, a command function update unit 62, a convergence determination unit 63, and a state output distribution update unit 64.

The auxiliary variable updating unit 61 performs all combinations (k, l) of times k and l (l <k) based on the phrase command u _p [k] (or initial value) at each time k updated last time. For each, the auxiliary variable λ _{p, k, l} is calculated and updated according to equation (17) above. In addition, the auxiliary variable update unit 61 performs the above equation (17) for all combinations of (k, l) based on the accent command u _a [k] (or initial value) at each time k updated last time. According to the above, the auxiliary variable λ _{a, k, l} is calculated and updated.

Further, the auxiliary variable updating unit 61 calculates and updates the auxiliary variable λ _{b, k, l} according to the above equation (17) for all combinations of (k, l) based on u _b .

The command function update unit 62 has a basic frequency sequence.

And the degree of uncertainty v _n ² [k] and the state series updated by the state series update unit 5

Based on the auxiliary variable λ _{p, k, l} updated by the auxiliary variable updating unit 61, the phrase command u _p [l] at each time l, which is a non-negative value, is updated according to the above equation (20).

The command function update unit 62 also has a basic frequency sequence.

And the degree of uncertainty v _n ² [k] and the state series updated by the state series update unit 5

Based on the auxiliary variable λ _{a, k, l} updated by the auxiliary variable updating unit 61, the accent command u _a [l] at each time l, which is a non-negative value, is updated according to the above equation (20).

The command function update unit 62 also has a basic frequency sequence.

Then, based on the degree of uncertainty v _n ² [k] and the auxiliary variable λ _{b, k, l} updated by the auxiliary variable updating unit 61, the base component u _b is updated according to the above equation (20). .

  The convergence determination unit 63 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, each process of the auxiliary variable update unit 61 and the command function update unit 62 is repeated. If the convergence determination unit 63 determines that the convergence condition is satisfied, the convergence determination unit 63 proceeds to processing by the state output distribution update unit 64.

  As the convergence condition, it may be used that the number of repetitions s has reached a predetermined number of times S (for example, 20 times). Note that the convergence condition is that the difference between the value of the auxiliary function when the s-1th parameter is used and the value of the auxiliary function when the sth parameter is used is smaller than a predetermined threshold. It may be used as

Based on the phrase command u _p [k] at each time k updated by the command function update unit 62, the state output distribution updating unit 64 performs the phrase command state output distribution at each time k according to the above equation (21). The parameter C ^(p) [k] is updated, and based on the accent command u _a [k] at each time k updated by the command function update unit 62 and the state sequence s updated by the state sequence update unit 5. Then, by updating the parameter C _n ^(a) of the state output distribution of each accent command n according to the above equation (22), the parameter group

Update.

The convergence determination unit 7 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, the update value is updated again.

When

And the iterative algorithm (each process of the state series update unit 5 and the model parameter update unit 6) is repeated. If the convergence determination unit 7 determines that the convergence condition is satisfied, the convergence determination unit 7 proceeds to processing by the output unit 9.

  As the convergence condition, it may be used that the number of repetitions r has reached a predetermined number R (for example, 20 times). Note that the convergence condition is that the difference between the value of the objective function when the r-1 parameter is used and the value of the objective function when the r parameter is used is smaller than a predetermined threshold. It may be used as

Then, the output function 9 causes the command function

, Parameters

, State series

Is output.

<Operation of fundamental frequency model parameter estimation device>
Next, the operation of fundamental frequency model parameter estimation apparatus 100 according to the present embodiment will be described. First, time series data of a speech signal for learning is input to the fundamental frequency model parameter estimation apparatus 100 and stored in the storage unit 10. Then, in the fundamental frequency model parameter estimation apparatus 100, the spectrogram feature quantity vector is obtained from the time series data of the speech signal for learning stored in the storage unit 10.

Spectralgram feature vector sequence that is a sequence of

Are extracted, and the parameters of the distributions G ₀ and G ₁ of the spectrogram feature vectors in each of the phoneme boundary section and the non-phoneme boundary section are parameters.

Is learned.

  Then, time series data of the observed speech signal is input to the fundamental frequency model parameter estimation device 100 as an analysis target and stored in the storage unit 10. Then, the fundamental frequency model parameter estimation apparatus 100 executes a fundamental frequency model parameter estimation processing routine shown in FIG.

First, in step S100, the time series data of the observed audio signal is read from the storage unit 10, and the spectrogram feature vector at each time k is read.

Spectogram feature vector series consisting of

To extract.

In step S101, time series data of the observed audio signal is read from the storage unit 10, and a basic frequency series including the basic frequency F _{0 at} each time k is read.

To extract. In step S102, voiced and unvoiced intervals are specified based on the time-series data of the audio signal, and the degree of uncertainty v _n ² [k] of the fundamental frequency at each time k is estimated.

In the next step S103, the parameters N, sets the appropriate initial value for u _b, the number of the small state of the HMM, the transition probabilities phi _{i ',} the _I, by learning from the correct answer data prepared previously determined To do. In addition, the command sequence by the conventional method

Is estimated and set as an initial value, and an initial value of C ^(p) [k] and an initial value of C _n ^(a) are set.

In step S104, the command sequence set in step S103 is set.

Initial value or a command series updated last time in step S105 described later

And the spectrogram feature vector sequence extracted in step S100

And the parameters of the spectrogram feature vectors G ₀ and G _{1 in} the phoneme boundary section and the non-phoneme boundary section obtained by the prior learning

And based on

Using Viterbi algorithm to maximize the state sequence

Update.

In step S105, the initial value of the phrase command u _p [k] at each time k set in the step S103, or in step S106, which will be described later, based on the phrase command u _p [k] at each time k, which was last updated For each combination (k, l) of times k and l (l <k), the auxiliary variable λ _{p, k, l} is calculated and updated according to the above equation (17). Based on the initial value of the accent command u _a [k] at each time k set in step S103 or the accent command u _a [k] at each time k updated last time in step S106 described later, (k, For all combinations of l), the auxiliary variable λ _{a, k, l} is calculated and updated according to the above equation (17). The initial value of u _b set in the step S103, or on the basis of a u _b it was last updated in step S106 to be described later, for all combinations of (k, l), according to the above equation (17), The auxiliary variable λ _{b, k, l} is calculated and updated.

In the next step S106, the fundamental frequency sequence calculated in step S101 above.

And the degree of uncertainty v _n ² [k] calculated at step S102 and the state series updated at step S104

Based on the auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k, l} updated in step S105, each time l which is a non-negative value according to the above equation (20) Command sequence consisting of _a phrase command u _p [l] and an accent command u _a [l]

To update the base component u _b.

In the next step S107, it is determined whether or not the number of repetitions s has reached S as the convergence condition. If the number of repetitions s has not reached S, it is determined that the convergence condition is not satisfied. Then, the process returns to step S105, and the processes of steps S105 to S106 are repeated. On the other hand, if the number of repetitions s reaches S is determined to have been satisfied convergence condition, in step S108, based on the phrase command u _p at each time k updated in step S106 [k], the According to Equation (21), the parameter C ^(p) [k] of the phrase command state output distribution at each time k is updated, and the accent command u _a [k] at each time k updated in step S106, State series updated in step S104

And updating the parameter C _n ^(a) of the state output distribution of each accent command n according to the above equation (22), the parameter group

Update.

In step S109, it is determined whether the number of repetitions r has reached R as the convergence condition. If the number of repetitions r has not reached R, it is determined that the convergence condition is not satisfied. Then, the process returns to step S104, and the processes of steps S104 to S108 are repeated. On the other hand, when the number of repetitions r reaches R, it is determined that the convergence condition is satisfied, and the output function 9

, Parameters

, State series

Is output and the fundamental frequency model parameter estimation processing routine is terminated.

<Experiment>
In order to verify the parameter estimation accuracy of the proposed method described in the embodiment of the present invention, speech data of Set B of the ATR digital speech database (503 spoken speeches by one male Japanese speaker (MHT)) was used. An evaluation experiment was conducted.

The correct data for the phrase / accent command was manually labeled by an expert. Estimation of F ₀ tracks from the audio data is carried out by using an existing method, the value of the baseline frequency u _b is the value of the minimum value of F ₀ in voiced segments for each reading sentence. The initial value of each phrase / accent command sequence was set to the value obtained by the conventional method. The number of iterations was 20 times. F ₀ pattern model composed of observed F ₀ patterns and estimated phrase / accent commands

The mean square error (RMSE) in the voiced interval between and the phrase / accent command dropout rate and error insertion rate were used as evaluation indices for parameter estimation accuracy.

  Figures 6 and 7 show the results of RMSE and erroneous insertion / dropout rates for each method. Compared with the conventional method, the RMSE value improved by about 5% in all variations of the proposed method. In addition, the error insertion / dropout rate improved most significantly when ΔMFCC was used as the spectral feature, and the error was reduced by about 3%.

As described above, according to the fundamental frequency model parameter estimation device according to the embodiment of the present invention, the spectral feature vector sequence is extracted from the time series data of the speech signal, the observed fundamental frequency sequence, the spectral feature vector sequence, Logarithmic joint probability of command function and state series

Using the objective function as an objective function, it is possible to accurately estimate the parameters of the Fujisaki model by repeatedly updating the state series and updating the command function and the parameter group.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the fundamental frequency model parameter estimation apparatus described above has a computer system inside, but if the “computer system” uses a WWW system, a homepage providing environment (or display environment) is also available. Shall be included.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 1 Feature vector series extraction part 2 Fundamental frequency series extraction part 3 Voiced unvoiced area estimation part 4 Initial value setting part 5 State series update part 6 Model parameter update part 7 Convergence determination part 10 Storage part 11 Prior learning part 61 Auxiliary variable update part 62 Command function update unit 63 Convergence determination unit 64 State output distribution update unit 100 Fundamental frequency model parameter estimation device

Claims (7)

An input audio signal, hiding the state sequence s made from the state s _k at each time k in Markov models, phrase command u _p representing the fundamental frequency pattern resulting from the translation movement of the thyroid cartilage at each time k [k] and thyroid A command function o composed of a pair o [k] of accent commands u _a [k] representing a fundamental frequency pattern generated by the rotational motion of the cartilage, and a parameter C of the phrase command state output distribution according to the state s _k at each time k ^(p) A fundamental frequency model parameter estimating apparatus for estimating [k] and a parameter group θ representing ^a parameter C _n ^(a) of ^a state output distribution of each accent command n,
A feature vector sequence extraction unit that extracts an observation feature vector sequence v representing a feature vector at each time k of the audio signal from the time series data of the audio signal;
A fundamental frequency extraction unit for extracting an observed fundamental frequency sequence y representing a fundamental frequency at each time k of the speech signal from the time series data of the speech signal;
About the time-series data of the speech signal, a voiced / unvoiced section estimation unit that estimates the degree of uncertainty of the fundamental frequency at each time k, depending on whether it is a voiced section or an unvoiced section;
An initial value setting unit for setting an initial value of the command function o;
Based on the previously updated command function o or the initial value of the command function o, the logarithmic simultaneous probability log p (of the observed fundamental frequency sequence y, the observed feature vector sequence v, the command function o, and the state sequence s ( a state sequence update unit that updates the state sequence s so that the objective function is increased with y, v, o, s) as an objective function;
Based on the previously updated command function o or the initial value of the command function o, the observed fundamental frequency series y, and the degree of uncertainty at each time k, each non-negative value is increased. A model parameter update unit for updating the command function o and the parameter group θ,
A convergence determination unit that repeats the update by the state series update unit and the update by the model parameter update unit until a predetermined convergence condition is satisfied,
A fundamental frequency model parameter estimation apparatus including: A pre-learning unit that learns the distribution of feature vectors in each of the phoneme boundary section and the non-phoneme boundary section based on the feature vector at each time extracted from the time-series data of the speech signal for learning;
The state series update unit
Based on the previously updated initial value of the command function o or the command function o and the distribution of feature vectors in each of the phoneme boundary section and the non-phoneme boundary section, the state series s In the state corresponding to the phoneme boundary section, according to the feature vector distribution in the phoneme boundary section, in the state sequence s in the state corresponding to the nonphoneme boundary section, the feature vector of the nonphoneme boundary section The fundamental frequency model parameter estimation apparatus according to claim 1, wherein the state series s is updated so as to follow a distribution.   The state series update unit, based on the initial value of the command function o or the command function o updated last time, and the distribution of feature vectors in each of the phoneme boundary section and the non-phoneme boundary section, The fundamental frequency model parameter estimation apparatus according to claim 2, wherein the state sequence s is updated using a Viterbi algorithm so as to increase log p (v | s) + log p (o | s) + log p (s). An input audio signal, hiding the state sequence s made from the state s _k at each time k in Markov models, phrase command u _p representing the fundamental frequency pattern resulting from the translation movement of the thyroid cartilage at each time k [k] and thyroid A command function o composed of a pair o [k] of accent commands u _a [k] representing a fundamental frequency pattern generated by the rotational motion of the cartilage, and a parameter C of the phrase command state output distribution according to the state s _k at each time k ^(p) A fundamental frequency model parameter estimating method in a fundamental frequency model parameter estimating apparatus for estimating [k] and a parameter group θ representing ^a parameter C _n ^(a) of ^a state output distribution of each accent command n,
A feature vector series extraction unit extracts an observed feature vector series v representing a feature vector at each time k of the voice signal from the time series data of the voice signal;
A fundamental frequency extraction unit extracts an observed fundamental frequency sequence y representing a fundamental frequency at each time k of the audio signal from the time series data of the audio signal;
The voiced and unvoiced section estimation unit estimates the degree of uncertainty of the fundamental frequency at each time k, depending on whether the time series data of the voice signal is a voiced or unvoiced section,
An initial value setting unit sets an initial value of the command function o,
Based on the command function o or the initial value of the command function o that was updated last time, the state series update unit updates the observation basic frequency series y, the observation feature vector series v, the command function o, and the state series s. Updating the state sequence s so as to increase the objective function with the logarithmic joint probability log p (y, v, o, s) as an objective function;
The model parameter updating unit increases the objective function based on the command function o updated last time or the initial value of the command function o, the observed fundamental frequency series y, and the degree of uncertainty at each time k. Updating the command function o and the parameter group θ, each of which is a non-negative value,
A fundamental frequency model parameter estimation method in which a convergence determination unit repeats the update by the state series update unit and the update by the model parameter update unit until a predetermined convergence condition is satisfied. The pre-learning unit learns the distribution of the feature vector in each of the phoneme boundary section and the non-phoneme boundary section based on the feature vector at each time extracted from the time series data of the speech signal for learning. In addition,
By updating the state series update unit,
Based on the previously updated initial value of the command function o or the command function o and the distribution of feature vectors in each of the phoneme boundary section and the non-phoneme boundary section, the state series s In the state corresponding to the phoneme boundary section, according to the feature vector distribution in the phoneme boundary section, in the state sequence s in the state corresponding to the nonphoneme boundary section, the feature vector of the nonphoneme boundary section 5. The fundamental frequency model parameter estimation method according to claim 4, wherein the state series s is updated so as to follow a distribution.   The state series update unit updates the command function o updated last time or the initial value of the command function o, the distribution of feature vectors in each of the phoneme boundary section and the non-phoneme boundary section, 6. The fundamental frequency model according to claim 5, wherein the state sequence s is updated using a Viterbi algorithm so as to increase log p (v | s) + log p (o | s) + log p (s) based on Parameter estimation method.   The program for functioning a computer as each part of the fundamental frequency model parameter estimation apparatus of any one of Claims 1-3.