JP2017151224A - Basic frequency pattern prediction device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate an optimum Fpattern corresponding to a spectrum feature amount sequence by taking into consideration the restriction of a physical generation process of the Fpattern.SOLUTION: A learning part 30 learns a parameter of a first probability distribution that models a relationship between a spectrum feature amount vector of each time of a day of the source speech and a basic frequency of each time of a day of a target speech, and learns the parameter of a second probability distribution that models a basic frequency pattern generation process. A conversion processing part 50 predicts the basic frequency of each time of a day of the target speech corresponding to the source speech of a prediction object from the spectrum feature amount vector of each time of a day extracted from the time series data of the source speech of the prediction object so that the standard shown by using the first probability distribution and the second probability distribution becomes large.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a fundamental frequency pattern prediction apparatus, method, and program, and more particularly, to a fundamental frequency pattern prediction apparatus, method, and program for predicting a fundamental frequency pattern of a target voice from a source voice.

  In communication with others, voice is a convenient means, but sometimes physical barriers inevitably cause various barriers. For example, if even one of the vocal organs does not operate normally, it suffers from serious vocal disturbances and hinders voice communication. Moreover, the physical action of voice generation is not suitable for highly confidential communication and is vulnerable to ambient noise. In order to eliminate these barriers, voices can be generated by moving the vocal organs beyond physical constraints, voices can be generated by specifying an appropriate pronunciation, or when voices are so fine that it is difficult to hear It is necessary to expand the speech generation function beyond physical and physical constraints, such as generating normal speech from the vocal organ movements.

For example, for laryngectomy patients who have lost their larynx due to laryngeal cancer, etc., voice assist technology has been proposed to convert less natural speech generated by alternative vocalization methods using residual organs into more natural speech. (See Non-Patent Document 1 to Non-Patent Document 3). In addition to this, a technique for converting a non-audible murmur voice into a natural voice has been proposed, and application as a call technique with excellent secrecy is expected. All of the above-mentioned techniques are common in that they deal with the problem of predicting the fundamental frequency (F ₀ ) pattern of natural speech from the spectral feature quantity sequence of speech, and are composed of learning processing and conversion processing. In the learning process, the same speech data of the target voice (electric voice in the former case, non-audible murmur voice in the latter case) and normal voice is used. First, at each discrete time (hereinafter referred to as a frame), the spectral feature of the target speech obtained from several frames before and after, the logarithm F _{0 of the} normal speech and its dynamic component (time differential or time difference) are extracted, and the spectral distance measure. A combined vector corresponding to these is obtained by dynamic time expansion and contraction based on. This is called parallel data. Using the parallel data of each frame, the spectral probability of the target speech and the combined probability density function of the static and dynamic components of the logarithm F ₀ of the normal speech are expressed by a mixed normal distribution model (GaMMian Mixture Model; GMM). GMM parameters can be learned using the Expectation-Maximization algorithm. In the conversion process, the learned GMM can be used to convert the spectral feature quantity sequence of the target speech into the F ₀ pattern of the normal speech by the maximum likelihood sequence conversion method considering intra-sequence variation.

Keigo Nakamura, Tomoki Toda, Hiroshi Saruwatari, Kiyohiro Shikano, "Speaking-aid systems using GMM-based voice conversion for electrolaryngeal speech," Speech Communication, vol. 54, no. 1, pp. 134-146, 2012. Kou Tanaka, Tomoki Toda, Graham Neubig, Sakriani Sakti, Satoshi Nakamura, "A hybrid approach to electrolaryngeal speech enhancement based on noise reduction and statistical excitation generation," IEICE Transactions on Information and Systems, vol.E97-D, no. 6, pp. 1429-1437, Jun. 2014. Kou Tanaka, Tomoki Toda, Graham Neubig, Sakriani Sakti, Satoshi Nakamura, "Direct F0 controlof an electrolarynx based on statistical excitation feature prediction and its evaluation through simulation," Proc. INTERSPEECH, pp. 31-35, Sep. 2014. Hirokazu Kameoka, Jonathan Le Roux, Yasunori Ohishi, "A statistical model of speech F0 contours," ISCA Tutorial and Research Workshop on Statistical And Perceptual Audition (SAPA 2010), pp. 43-48, Sep. 2010. Kota Yoshizato, Hirokazu Kameoka, Daisuke Saito, Shigeki Sagayama, "Hidden Markov convolutive mixture model for pitch contour analysis of speech," in Proc.The 13th Annual Conference of the International Speech Communication Association (Interspeech 2012), Sep. 2012.

In the prior art, a model that takes into account the physical generation process of the voice F ₀ pattern was not used in the learning process or conversion process, so an unnatural F ₀ pattern that could not be physically spoken by humans was used. It could happen. To solve this problem, by performing prediction in consideration of the physical process of generating F ₀ pattern, it may be possible to generate a more natural F ₀ pattern.

The F ₀ pattern is generated by the movement of the thyroid cartilage that gives tension to the vocal cords. In Non-Patent Documents 4 and 5, parameters related to the movement of the thyroid cartilage called phrase / accent command are based on the probability model of the control mechanism. An estimation technique has been proposed. In this technology, the time series generation process of the phrase / accent command is expressed by a hidden Markov model (HMM). One of the points is the linguistics about the command sequence through the design of the topology of the HMM and learning of the transition probability. Or a priori knowledge can be incorporated into the parameter estimation.

The present invention has been made in view of the above circumstances, the fundamental frequency can be estimated optimal F ₀ pattern corresponding to the spectral feature amount sequence taking into account the constraints of the physical process of generating F ₀ pattern An object is to provide a pattern prediction apparatus, method, and program.

  In order to achieve the above object, a fundamental frequency pattern predicting apparatus according to the present invention receives, as input, parallel data composed of time series data of source speech of a learning sample and time series of target speech, and the time series of the source speech. Based on the spectral feature vector at each time extracted from the data and the fundamental frequency at each time extracted from the time-series data of the target speech, the spectral feature vector at each time of the source speech, A first model parameter learning unit that learns a parameter of a first probability distribution that models a relationship between the fundamental frequency at each time of the target speech, and a fundamental frequency pattern based on the fundamental frequency at each time of the target speech A second model parameter learning unit for learning a parameter of a second probability distribution modeling the generation process; Using the time series data of the source speech to be measured as an input, the spectral feature quantity vector of each time extracted from the time series data of the source speech to be predicted and the first model learned by the first model parameter learning unit Based on the parameters of the probability distribution and the parameters of the second probability distribution learned by the second model parameter learning unit, the criterion expressed using the first probability distribution and the second probability distribution is increased. And a fundamental frequency predicting unit that predicts a fundamental frequency at each time of the target speech corresponding to the source speech to be predicted.

  A fundamental frequency pattern prediction method according to the present invention is a fundamental frequency pattern prediction method in a fundamental frequency pattern prediction apparatus including a first model parameter learning unit, a second model parameter learning unit, and a fundamental frequency prediction unit, The first model parameter learning unit receives the parallel data composed of the time series data of the source speech and the target speech of the learning sample as input, and the spectral feature quantity at each time extracted from the time series data of the source speech Based on a vector and a fundamental frequency at each time of the source speech based on a fundamental frequency at each time extracted from the time series data of the target speech and a fundamental frequency at each time of the target speech Learning the parameters of the first probability distribution modeling the relationship of A learning unit learns a parameter of a second probability distribution obtained by modeling a fundamental frequency pattern generation process based on a fundamental frequency at each time of the target speech, and the fundamental frequency predicting unit is a source speech to be predicted. Using the series data as an input, the spectral feature vector at each time extracted from the time series data of the source speech to be predicted, the parameters of the first probability distribution learned by the first model parameter learning unit, Based on the parameters of the second probability distribution learned by the second model parameter learning unit, the prediction target is increased so as to increase the criterion expressed using the first probability distribution and the second probability distribution. The basic frequency at each time of the target speech corresponding to the source speech of is predicted.

  A program according to the present invention is a program for causing a computer to function as each unit of the above-described basic frequency pattern prediction apparatus.

As described above, according to the fundamental frequency pattern predicting apparatus, method, and program of the present invention, the relationship between the spectrum feature vector at each time of the source speech and the fundamental frequency at each time of the target speech is modeled. Of the first probability distribution obtained by learning, the parameter of the second probability distribution modeling the fundamental frequency pattern generation process is learned, and the spectral feature amount at each time extracted from the time series data of the source speech to be predicted F ₀ is predicted from the vector by predicting the fundamental frequency at each time of the target speech corresponding to the source speech to be predicted so as to increase the criterion expressed using the first probability distribution and the second probability distribution. it is possible to estimate the optimal F ₀ pattern corresponding to the spectral feature amount sequence taking into account the constraints of the physical process of generating patterns, called Results can be obtained.

It is a figure for demonstrating an example of the state transition network of HMM. It is a figure for demonstrating an example of the state transition network of HMM. It is a figure for demonstrating an example of the state transition network of HMM. It is a figure for demonstrating the division | segmentation of a state. It is the schematic which shows the structure of the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is the schematic which shows the structure of the learning part of the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is the schematic which shows the structure of the conversion process part of the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the learning process routine in the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the fundamental frequency pattern prediction process routine in the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is the schematic which shows the structure of the conversion process part of the fundamental frequency pattern prediction apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the content of the learning process routine in the fundamental frequency pattern prediction apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the content of the fundamental frequency pattern prediction process routine in the fundamental frequency pattern prediction apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows the _F0 pattern of the audio | voice used for experiment data. It is a figure which shows an experimental result.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The technique proposed in the present invention predicts a fundamental frequency pattern from a feature sequence of speech and replaces the fundamental frequency pattern of the original speech with the predicted fundamental frequency pattern to improve speech naturalness. Processing technology.

<Related technology 1: F ₀ pattern prediction method from spectral feature sequence>
First, the F ₀ pattern prediction method from the spectral feature quantity sequence will be described.

Non-Patent Document 1 to Non-Patent Document 3 propose a method for predicting an F ₀ pattern from a spectrum feature amount sequence. The conventional method consists of the process of converting from a given spectral feature amount sequence in F ₀ pattern using the model trained with the processing for learning the parameters of the joint probability distribution model of the spectral feature amount sequence and F ₀ pattern.

<Learning process>
It is assumed that parallel data of a source sound (for example, electric sound) and a target sound (for example, natural sound) is given. The spectral feature vector of the source speech is c [k], and the combined vector (called F ₀ feature) of the logarithm F _{0 of the} target speech and its dynamic component (time differential or time difference) is q [k] = (y [k]; Δy [k]) ^T. Where k is an index of discrete time. As the speech feature value c [k], for example, a vector obtained by performing dimension compression by principal component analysis on a vector obtained by connecting a series of mel cepstrum (vector) for several frames around the time k is used. In this method, the joint probability distribution (Gaussian Mixture Model; GMM) of the joint probability distribution of c [k] and q [k] is used.

In the learning process, the GMM parameters (weight, mean, variance-covariance matrix) of the GMM are learned from given parallel data {c [k]; q [k]} ^K _{k = 1} . However, N (x; μ, Σ) means that the probability density function of x is given by a normal distribution having an average of μ and a variance-covariance matrix of Σ.

GMM parameters can be estimated by the Expectation-Maximization (EM) algorithm. If the learned GMM parameter is γ, the conditional distribution P (q [k] | c [k], γ) is the distribution for predicting the F ₀ feature q [k] from the spectrum feature c [k]. Can see,

As in P (c [k], q [k] | γ), it is given by GMM. However,

And e ^{(q | c)} _m and D ^{(q | c)} _m are

Given in.

<Conversion processing>
In the conversion process. A given spectral feature series

Maximum likelihood F ₀ pattern under

Is obtained by the following equation (8).

  However,

W is a transformation matrix (constant) describing the relationship between y and q. Here, P (q | c, γ) is a GMM given by the parameter γ learned by the learning process,

Given in. Here, m = (m ₁ ,..., M _K ), and m _k represents the component index of the GMM at time k. Where P (q [k] | c [k], γ) is

Can be approximated by From equation (12), ^ m _k means a normal distribution index having the highest probability of generating data c [k]. Therefore, from equation (9), P (q | c, m, ^ γ) is for all k.

It is given by taking the product of By approximation of equation (11), P (q | c, γ) is

The vector e ^{(q | c)} concatenated with

Distribution with block diagonal matrix D ^{(q | c)} as a variance-covariance matrix

It becomes. Substituting q = Wy into this and normalizing to be the distribution of y,

So that

A conditional distribution of y is obtained as follows. Therefore, the solution of equation (8) is

It becomes.

<Related technology 2: F ₀ pattern generation process model>
Next, a probability model of the F ₀ pattern generation process will be described.

Fujisaki's fundamental frequency (F ₀ ) pattern generation process model (Fujisaki model) is known as a model that describes the F ₀ pattern generation process of speech (Non-patent Document 6).

[Non-Patent Document 6]: H. Fujisaki, “In Vocal Physiology: Voice Production, Mechanisms and Functions,” Raven Press, 1988.

The Fujisaki model is a physical model describing the process of generating F ₀ pattern due to the motion of the thyroid cartilage. The Fujisaki model, the total elongation of the vocal cords with each two independent movement of the thyroid cartilage (translational motion and rotational motion) is interpreted to result temporal variation of F _0, pairs of elongation and F ₀ pattern of the vocal cords The F ₀ pattern is modeled on the assumption that the numerical value y (t) is proportional. The F ₀ pattern x _p (t) generated by the translational motion of the thyroid cartilage is called a phrase component, and the F ₀ pattern x _a (t) generated by the rotational motion is called an accent component. In the Fujisaki model, the F ₀ pattern y (t) of speech is the sum of these components plus the baseline component b determined by the physical constraints of the vocal cords.

It is expressed as These two components are assumed to be the output of the second-order critical braking system,

(* Is a convolution operation with respect to time t). Here, u _p (t) is called a phrase command function and consists of a sequence of delta functions (phrase commands), and u _a (t) is called an accent command function and consists of a sequence of rectangular waves (accent commands). These command sequences have a constraint condition that a phrase command occurs at the beginning of an utterance, phrase commands do not occur twice in succession, and two different commands do not occur at the same time. Α and β are the natural angular frequencies of the phrase control mechanism and the accent control mechanism, respectively. It is experiential that α is approximately 3 rad / s and β is 20 rad / s, regardless of the speaker or utterance content. Known to.

<Related technology 3: F ₀ pattern generation process model parameter estimation method>
The above-mentioned Fujisaki model can be described by the following probability model (see Non-Patent Documents 4, 5, and 7).

[Non-patent document 7]: Tatsuma Ishihara, Kota Yoshizato, Hirokazu Kameoka, Daisuke Saito, Shigeki Hatakeyama, \ Statistical vocabulary model of the Fujisaki model command sequence of the speech fundamental frequency, "Lecture at the 2013 Acoustical Conference of the Acoustical Society of Japan Proceedings, 1-7-9, pp. 283-286, Mar. 2013.

First phrase, a pair o of accent command function _{[k] = (u p [} k], u a [k]) consider the HMM to output the ^T. Here, k represents an index of discrete time. The state output distribution is a normal distribution.

The command function pair o [k] is generated by Here, {s _k } ^K _{k = 1} is an HMM state sequence, and the average vector ρ [k] is a value determined as a result of the state transition of the HMM. Examples of specific HMM configurations are shown in FIGS.

In the example of the state transition network of the HMM shown in FIG. 1, both μ ^(p) _t [k] and μ ^(a) _t are ₀ in the state t = r ₀ . The state t = r ₀ can only transition to the state p ₀ , and in the state t = p ₀ , μ ^(p) _t [k] takes a non-negative value A ^(p) [k], and μ ^(a) _t Becomes 0. The transition after state t = p ₀ is allowed only to state r ₁ . Similar to state t = r ₀ , μ ^(p) _t [k] and μ ^(a) _t are both 0 in state t = r ₁ . A transition from state t = r ₁ to only _one of states a ₀ , ..., a _N-1 is possible, and in state t = a _n μ ^(a) _t is a non-negative value A ^(a) _n Therefore, μ ^(p) _t [k] is 0. Next state t = a _n is allowed to transition only in the state r ₀ or r _1. This guarantees that μ _a [k] is a rectangular pulse train.

In the example of the state transition network of the HMM shown in FIG. 2, both μ ^(p) _t [k] and μ ^(a) _t are ₀ in the state t = r ₀ . A transition from state t = r ₀ to only _one of states p ₀ , ..., p _M-1 is possible, and μ ^(p) _t is a non-negative value A ^(p) _m in state t = p _m Therefore, μ ^(a) _t becomes 0. Next state t = p _m is allowed to transition only in state r _1. Similar to state t = r ₀ , μ ^(p) _t [k] and μ ^(a) _t are both 0 in state t = r ₁ . A transition from state t = r ₁ to only _one of states a ₀ , ..., a _N-1 is possible, and in state t = a _n μ ^(a) _t is a non-negative value A ^(a) _n Therefore, μ ^(p) _t becomes 0. Next state t = a _n is allowed to transition only in the state r ₀ or r _1. This guarantees that μ _a [k] is a rectangular pulse train.

The example of the state transition network of the HMM shown in FIG. 3 is composed of a plurality of Left-to-Right type HMMs in which respective end points and start points are connected. Similar to Figure 1 and 2, both the state t = r _l in μ ^(p) _t [k] and mu ^(a) _t is 0. The state t = In ^p _m μ ^(p) _t takes a nonnegative value ^{_{A (p) m, μ (}} a) t is 0. State t = in ^a _n μ ^(a) _t takes a nonnegative value ^{_{A (a) n, μ (}} p) t is zero.

  In the example of FIG. 1, ρ [k] is expressed by the following equation (24).

  2 and 3, ρ [k] is expressed by the following equation (25).

In either example, as shown in FIG. 4, the time for each state to stay in the same state by dividing it into several small states with the same output distribution and restricting the left-to-right state transition path Long probabilities can be parameterized. Figure 4 is an example of dividing the state a _n. For example, by setting the state transition probability from a _{n, n ′} to a _{n, n ′ + 1} to ₁ for all m ≠ 0 as shown in this figure, a _{n, 0} to a _{n, n ′} transition probabilities to correspond to the probability that state a _n lasts only n 'step, it becomes possible or to learn to set the probability of persistence length of accent command. Similarly, by dividing p _m and r _l into small states, it becomes possible to parameterize the distribution of the duration of the phrase command and the length of the interval between commands. After that, the state set

Is written. The configuration of the above HMM can be written as follows.

When the state sequence s = {s _k } ^K _{k = 1} is given, the HMM outputs a pair of _a phrase command function u _p [k] and an accent command function u _a [k]. As shown in Eqs. (18) and (20), g _p [k] and g _a [k] are convolved with u _p [k] and u _a [k], respectively, and the phrase component x _p [k] And the accent component x _a [k] are output. This can be expressed as an expression:

(* Is a convolution operation for discrete time k). At this time, the F ₀ pattern x [k] is

And can be written by superimposing three kinds of components. Where b is a baseline component that does not depend on time.

In real speech, reliable F ₀ values are not always observable. When estimating the parameters of the Fujisaki model, it is desirable to consider only the F ₀ value of the reliable observation interval and ignore the other intervals. For example, in a voiceless section, periodic coarse / fine waves associated with normal vocal cord vibration are not observed, so even if some value is obtained as an estimated value of F ₀ from the voiceless section by automatic pitch extraction, its value Is not considered to be the value of F ₀ of a signal emitted from the vocal cords. Therefore, the degree of uncertainty v ² _n [k] at time k of the observed F ₀ value is introduced into the proposed model. Specifically, the observed F ₀ value y [k], the true F ₀ value x [k] and the noise component

With overlay

This means that all observation intervals can be handled uniformly regardless of whether they are reliable intervals.

Probability density function of y = {y [k]} ^K _{k = 1} given the output value sequence o = {o [k]} ^K _{k = 1} by marginalizing x _n [k]

Is obtained. State series s = {s _k } ^K _{k = 1} and parameter indicating command amplitude

And the transition probability matrix φ = (φ _{i, j} ) _{I × I} , the output value sequence o is

Is generated according to P (s | φ) is the product of the state transition probabilities.

Can be written. However,

Represents the probability that the initial state is s ₁ . From equations (30), (32) and (33), P (y, o, s | θ, φ) is

Can be written. If this is marginalized with respect to o,

Is obtained. However,

It is. On the other hand, if you marginalize s

Is obtained. However, Σ _s means an operation for taking a sum for all state sequences.

<Parameter estimation algorithm 1>
By regarding y and o as complete data, s and θ that locally maximize Equation (35) can be searched by the Expectation-Maximization algorithm. Although derivation is omitted,

Updating s and θ so that becomes large, and using the updated s and θ

And R

(35) can be monotonously increased by repeating the updating step according to the above (refer to Non-Patent Document 4 above for details).

  Specifically, the following initial setting, E step, and M step are executed.

(Initial setting)
s and θ are initialized.

(E step)
Conditional expected values for phrase, accent, and baseline components

And the conditional covariance matrix R

Update with However,

It is. Also in R

The block diagonal component corresponding to

And

  That is,

(* Is a block component not used later).

(M step)
A state series s = (s ₁ ,..., S _K ) that maximizes Q (s, θ) is searched. γ _p and γ _a are diagonal

Since it is a column,

Are both

Can be written in the form of the sum of terms for every k. Therefore, Q (s, θ) is aside from terms that do not depend on s.

Can be written. Therefore, the state sequence s = (s ₁ ,..., S _K ) that maximizes Q (s, θ) can be obtained by the Viterbi algorithm (refer to Non-Patent Document 4 for details). Here, [·] _{k, k} represents a k-by-k element of the matrix, and [·] _k represents a k-th element of the vector.

  Subsequently, θ is updated to maximize Q (s, θ). Θ that maximizes Q (s, θ) can be obtained by setting the partial differential for each variable of Q (s, θ) to 0 (for details, see Non-Patent Document 4 above).

  Further, a state transition probability φ is obtained from the estimated state sequence s.

[First Embodiment]
<Outline of Embodiment of the Present Invention>
The technique according to the embodiment of the present invention includes a learning process and a conversion process as in the F ₀ pattern prediction method from the spectral feature quantity sequence of the related technique 1 described above, but instead of the equation (8), the related process described above is used. By using as a criterion the product of the probability distribution of the F ₀ pattern prediction method from the spectral feature amount sequence of Technology 1 and the probability distribution of the F ₀ pattern generation process model parameter estimation method of Related Technology 3 described above, the above-described F _{0 is used.} This is a technique that makes it possible to statistically predict the F ₀ pattern that is as close as possible to the pattern generation process model from the spectral feature amount.

In the learning process, the parallel data learning sample {c [k], q [k]} ^K _{k = 1}

Γ is learned so that becomes as large as possible. Also, θ and φ are learned so that P (y, s | θ, φ) becomes as large as possible under the basic frequency pattern {y [k]} ^K _{k = 1 of} the learning sample.

On the other hand, in the conversion process, P (q | c, γ) P (y, s | θ, φ) or a probability density function approximating these is given under the condition that {c [k]} ^K _{k = 1} of the input speech is given. Y is determined so that becomes as large as possible.

<System configuration>
Next, an embodiment of the present invention will be described by taking as an example a case where the present invention is applied to a fundamental frequency pattern prediction apparatus that predicts a fundamental frequency pattern of a target speech from a spectral feature quantity sequence of a source speech.

  As shown in FIG. 5, the fundamental frequency pattern prediction apparatus according to the first embodiment of the present invention includes a CPU, a RAM, a learning processing routine, and a program for executing a fundamental frequency pattern prediction processing routine, which will be described later. The computer is provided with a ROM that stores the above, and is functionally configured as follows.

  As shown in FIG. 5, the fundamental frequency pattern prediction apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 90.

  The input unit 10 accepts parallel data composed of time-series data of the source sound (for example, electric sound) of the learning sample and time-series data of the target sound (for example, natural sound). In addition, the input unit 10 receives time-series data of the source audio to be predicted.

  The calculation unit 20 includes a learning unit 30, a parameter storage unit 40, and a conversion processing unit 50.

  As shown in FIG. 6, the learning unit 30 includes a feature amount extraction unit 32, a fundamental frequency series extraction unit 34, a first model parameter learning unit 36, and a second model parameter learning unit 38.

  The feature quantity extraction unit 32 extracts the source speech spectogram feature quantity vector c [k] from the time series data of the source speech of the learning sample received by the input unit 10. Where k is an index of discrete time. For example, as in Non-Patent Documents 1 to 3, c [k] is obtained by performing dimensional compression by principal component analysis on a vector obtained by concatenating a series of mel cepstrum (vector) for several frames around the time k. Used as

The fundamental frequency series extraction unit 34 extracts the fundamental frequency y [k] at each time k of the target speech from the time series data of the target speech of the learning sample received by the input unit 10, and y = (y [1], ..., y [K]) ^T.

  This fundamental frequency extraction process can be realized by a well-known technique. For example, Non-Patent Document 8 (H. Kameoka, “Statistical speech spectrum model incorporating all-pole vocal tract model and F0 contour generating process model,” in Tech. Rep. IEICE, 2010, in Japanese.), The fundamental frequency is extracted every 8 ms.

Also, let q [k] = (y [k], Δy [k]) ^T be the combined vector (called F ₀ feature) of y and its dynamic component (time derivative or time difference).

As described above, data {c [k], q [k]} ^K _{k = 1} is obtained.

  The first model parameter learning unit 36 combines the spectral feature vector c [k] at each time k extracted by the feature extraction unit 32 and the fundamental frequency at each time k extracted by the fundamental frequency sequence extraction unit 34. Based on the vector q [k], a mixed normal representing a joint probability distribution of the spectral feature vector c [k] at each time of the source speech and the combined vector q [k] of the fundamental frequency at each time k of the target speech A parameter of the first probability distribution which is a distribution is learned.

Specifically, the first model parameter learning unit 36 learns the GMM parameter γ in Expression (1) in the same manner as the learning process of the F ₀ pattern prediction method from the spectral feature quantity sequence described above. Let the learned GMM parameter be ^ γ.

  The second model parameter learning unit 38, based on the fundamental frequency y [k] at each time k extracted by the fundamental frequency sequence extraction unit 34, each of the fundamental frequency y [k] at each time k and the hidden Markov model. A parameter of the second probability distribution, which is a probability distribution of a combination with the state series s composed of time states, is learned.

Specifically, the second model parameter learning unit 38 learns the parameters θ and φ of the F ₀ pattern generation process model according to the parameter estimation algorithm 1 of the F ₀ pattern generation process model parameter estimation method of the related technique 3 described above. .

If data o = {o _k } ^K _{k = 1} is available for the phrase command sequence and accent command sequence in the learning sample, θ and φ may be learned from o (for normal learning in HMM). Equivalent). Let the parameters of the learned F ₀ pattern generation process model be ^ θ and ^ φ.

  The conversion processing unit 50 receives the time-series data of the source speech to be predicted as an input, the spectral feature vector at each time extracted from the time-series data of the source speech, and the first model parameter learning unit 36 learned the first time. Based on the parameter γ of one probability distribution and the parameters θ and φ of the second probability distribution learned by the second model parameter learning unit 38, the product of the first probability distribution and the second probability distribution is expressed. The basic frequency y at each time is estimated by estimating the basic frequency y at each time and the state sequence s consisting of the state at each time so as to increase the criterion. To do.

  Here, the principle of predicting the fundamental frequency y at each time of the target speech corresponding to the source speech to be predicted will be described.

Using the parameters ^ γ, ^ θ, ^ φ learned in the learning process, and the feature amount sequence c = {c [k]} ^K _{k = 1} of the source speech to be predicted, Equations (9) and (35) Product of

Y and s are estimated so that becomes as large as possible. Where q = Wy and

It is. The ω is a non-negative constant mean or put how much consideration model of F ₀ pattern generation process in the prediction of F ₀ pattern.

  less than,

The algorithm for increasing Similar to the F ₀ pattern prediction method from the spectral feature amount sequence of Related Technique 1 described above,

And y and s can be estimated by the following iterative process (the execution order of steps 1 and 2 is arbitrary).

(Step 1) y is initially set by the conversion process of the F ₀ pattern prediction method from the spectral feature quantity sequence of Related Technique 1 described above.

(Step 2) ^ m is obtained by using equation (87) using c.

(Step 3) y is fixed, and s is estimated by the parameter estimation algorithm 1 of the F0 pattern generation process model parameter estimation method of the related technique 3 described above.

(Step 4) s and ^ m are fixed, y is updated by the following formula, and the process returns to Step 3.

(GγG ^T ) ⁻¹ is an inverse matrix of a large matrix, but can be efficiently calculated in the following manner. GγG ^T

G ⁻¹ _p and G ⁻¹ _a are

(GγG ^T ) ⁻¹ can be approximated by a lower triangular band matrix such as

And then Woodbury's official

(GγG ^T ) ^-1 by applying Eq.

Can be written. Also from Woodbury's official

 Is

Can be written. From equations (93) and (94)

Are both banded

The calculation of the form of can be calculated efficiently by Cholesky decomposition. Where a is an arbitrary vector and A is an arbitrary matrix.

  In order to realize the principle described above, in the present embodiment, as shown in FIG. 7, the conversion processing unit 50 includes a feature amount extraction unit 52, a basic frequency sequence prediction unit 54, and a normal distribution sequence prediction unit 56. A state sequence estimation unit 58, a fundamental frequency sequence update unit 60, and a convergence determination unit 62.

  Similar to the feature quantity extraction unit 32, the feature quantity extraction unit 52 obtains the spectrogram feature quantity vector c [k] of the source speech at each time k from the time series data of the prediction target source voice received by the input unit 10. Extract.

The fundamental frequency sequence prediction unit 54 includes the parameter γ of the first probability distribution learned by the first model parameter learning unit 36, the spectrum feature quantity vector c [k] at each time k extracted by the feature quantity extraction unit 52, and In the same manner as the conversion processing of the F ₀ pattern prediction method described above, the fundamental frequency y [k] at each time k is estimated by estimating the fundamental frequency y [k] at each time k according to the above equation (16). ] Is initialized.

The normal distribution series prediction unit 56 includes the parameter γ of the first probability distribution learned by the first model parameter learning unit 36, the spectral feature quantity vector c [k] at each time k extracted by the feature quantity extraction unit 52, and Based on the above, according to the above equation (87), the index ^ m _k of the normal distribution having the highest probability that the spectral feature vector c [k] at each time k is generated is estimated.

The state sequence estimation unit 58 fixes the fundamental frequency y [k] at each time k that is initially set by the basic frequency sequence prediction unit 54 or updated last time by the state sequence estimation unit 58, and the related technique 3 described above. Similarly to the parameter estimation algorithm 1 of the F ₀ pattern generation process model parameter estimation method, the state sequence s that locally maximizes the above equation (35), the amplitude of the phrase command according to the state at each time k, and the accent command The state series s is estimated by searching the parameter θ representing the amplitude by the EM algorithm.

Based on the state sequence s estimated by the state sequence estimation unit 58 and the normal distribution index ^ m _{k at} each time estimated by the normal distribution sequence prediction unit 56, the fundamental frequency sequence update unit 60 88), the fundamental frequency y [k] at each time k is updated.

  The convergence determination unit 62 repeats each process by the state sequence estimation unit 58 and the fundamental frequency sequence update unit 60 until a predetermined convergence determination condition is satisfied. The convergence determination condition is, for example, reaching a predetermined number of repetitions.

  When the convergence determination condition is satisfied, the fundamental frequency y [k] finally obtained at each time k is output as a prediction result of the fundamental frequency at each time of the target speech corresponding to the source speech to be predicted. 90 for output.

<Operation of fundamental frequency pattern prediction device>
Next, the operation of basic frequency pattern prediction apparatus 100 according to the present embodiment will be described. First, when parallel data composed of time series data of source speech and target speech of a learning sample is input to the fundamental frequency pattern prediction device 100, the fundamental frequency pattern prediction device 100 performs the learning process shown in FIG. The routine is executed.

  First, in step S101, input time-series data of the source sound is read,

A spectral feature vector c [k] at each time k is extracted. In step S102, the time-series data of the input target speech is read, the fundamental frequency y [k] at each time k of the target speech is extracted, and the combined vector q of the fundamental frequency y [k] and its dynamic component is extracted. Extract [k].

  In step S103, based on the spectral feature vector c [k] at each time k extracted in step S101 and the combined vector q [k] of the fundamental frequency at each time k extracted in step S102. Thus, the GMM parameter γ in the above equation (1) is learned.

  In step S104, the state series s and the parameter θ representing the amplitude of the phrase command and the amplitude of each accent command corresponding to the state at each time are initialized.

  In step S105, the conditional expected value ￣x of the phrase component, the accent component, and the baseline component and the conditional variance-covariance matrix R are updated according to the above formulas (49) and (50).

In the next step S106, the parameter θ initially set in step S104 or updated last time in step S107, which will be described later, and the conditional expected values of the phrase component, accent component, and baseline component updated in step S105. Based on ￣x and the conditional covariance matrix R, the state sequence s = (s ₁ ,..., S _K , which maximizes Q (s, θ), using the above equation (63). ) Is obtained by the Viterbi algorithm, and the state sequence s is updated.

  In step S107, the state series s updated in step S106, the conditional expected value ￣x of the phrase component, accent component, and baseline component updated in step S105, and the conditional variance-covariance matrix R are included. Based on this, the parameter θ that maximizes Q (s, θ) is obtained by setting the partial differential for each variable of Q (s, θ) to 0, and the amplitude of the phrase command corresponding to the state at each time and The parameter θ representing the amplitude of each accent command is updated.

  In step S108, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S105. On the other hand, if the convergence determination condition is satisfied, in step S109, the parameter γ learned in step S103 and the parameter θ finally obtained in step S107 are stored in the parameter storage unit 40. Further, the state transition probability φ is obtained from the state series s finally obtained in step S106 and stored in the parameter storage unit 40.

  Next, when the time-series data of the source speech to be predicted is input to the fundamental frequency pattern predicting apparatus 100, the fundamental frequency pattern predicting process routine shown in FIG.

  First, in step S121, the input time-series data of the target speech to be predicted is read, and the spectrum feature vector c [k] at each time k is extracted. In step S122, based on the parameter γ stored in the parameter storage unit 40 and the spectral feature quantity vector c [k] at each time extracted in step S121, according to the above equation (16), at each time k. By estimating the fundamental frequency y [k], the fundamental frequency y [k] at each time k is initialized.

In step S123, based on the parameter γ stored in the parameter storage unit 40 and the spectral feature quantity vector c [k] at each time k extracted in step S121, according to the equation (87), Estimate the normal distribution index ^ m _k with the highest probability of generating the spectral feature vector c [k] at time k.

  In step S124, a state sequence that is initially set in step S122 or that is updated last time in step S125, which will be described later, is fixed at the fundamental frequency y [k], and the above equation (35) is locally maximized. The state series s is estimated by searching for s and the parameter θ representing the amplitude of the phrase command and the amplitude of each accent command according to the state at each time k using the EM algorithm.

In step S125, based on the state series s estimated in step S124 and the normal distribution index ^ m _k of each time estimated in step S123, the basics of each time k are calculated according to the above equation (88). Update the frequency y [k].

  In step S126, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S124. On the other hand, if the convergence determination condition is satisfied, in step S127, the fundamental frequency y [k] at each time k finally obtained in step S125 is changed to each of the target speech corresponding to the source speech to be predicted. As a result of prediction of the fundamental frequency of time, the output unit 90 outputs the result, and the fundamental frequency pattern prediction processing routine is finished.

As described above, according to the fundamental frequency pattern predicting apparatus according to the first embodiment, the relationship between the spectrum feature vector at each time of the source speech and the fundamental frequency at each time of the target speech is modeled. Learning the parameter γ of the first probability distribution P (q [k], c [k] | γ), which is a generalized GMM, and modeling the fundamental frequency pattern generation process, the second probability distribution P (y, s | θ , Φ) parameters θ and φ are learned, and the first probability distribution P (q [k], c [k] | is obtained from the spectral feature quantity vector at each time extracted from the time series data of the source speech to be predicted. The basic frequency at each time of the target speech corresponding to the source speech to be predicted is set so as to increase the criterion expressed using the product of γ) and the second probability distribution P (y, s | θ, φ). by predicting, Do given the constraints of the physical process of generating F ₀ pattern It is possible to estimate the optimal F ₀ pattern corresponding to Luo spectral feature amount sequence.

[Second Embodiment]
Next, a fundamental frequency pattern prediction apparatus according to the second embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected about the part which becomes the same structure as 1st Embodiment, and description is abbreviate | omitted.

  In the second embodiment, the method for estimating the second probability distribution and parameters and the method for predicting the fundamental frequency at each time are different from those in the first embodiment.

  The principle of learning parameters by the second model parameter learning unit 38 of the learning unit 30 of the fundamental frequency pattern prediction apparatus according to the second embodiment will be described.

First, the parameter estimation algorithm of the F ₀ pattern generation process model parameter estimation method of the related technique 3 will be described.

<Parameter estimation algorithm 2>
Given an observation F ₀ sequence y, an iterative algorithm for obtaining a local optimal solution of the posterior probabilities P (o, θ | y) of model parameters θ and o is shown below. The state series s is a hidden variable, and the posterior probability P (o, θ | y) is

Note that the Q function Q (o, θ, o ′, θ ′) is

I can put it. However,

Represents equality except for the constant term. G _b [k] = δ [k] (Kronecker delta). Therefore, the step of calculating P (s _k = t | y, o ′, θ ′) by the Forward-Backward algorithm and the step of increasing Q (o, θ, o ′, θ ′) for o and θ are repeated. Thus, a solution in which P (o, θ | y) is locally maximum can be obtained. Since o is a phrase / accent command sequence pair, in the step of increasing Q (o, θ, o ′, θ ′), it is necessary to consider the non-negative constraint of o. An update rule that increases Q (o, θ, o ′, θ ′) while satisfying the non-negative constraint of o can be derived as follows. First, the lower bound of Q (o, θ, o ′, θ ′) is from Jensen's inequality.

Can be designed as follows. I, k, and l are

 Any variable that satisfies So the lower bound of the Q function is

It is expressed. Q (o, θ, o ′, θ ′) can be increased by alternately repeating the step of maximizing the lower bound function with respect to λ _{i, k, l} ≧ 0 and the step of maximizing with respect to o 1. The update rule for any step can be determined analytically,

It is represented by After the above iterations converge, θ is continuously updated. In the case of FIG.

  In the case of FIGS.

It is. After substituting these update values for o ′ and θ ′, P (s _k = t | y, o ′, θ ′) is updated again, and thereafter the same processing is repeated to thereby determine the posterior probability P (o , Θ | y) can be monotonously increased.

  After the above iterative algorithm has converged, the optimum s obtained by the Viterbi algorithm of the parameter estimation algorithm 1 described above is taken as the state sequence estimation result.

  Further, a state transition probability φ is obtained from the estimated state sequence s.

In accordance with the principle described above, the second model parameter learning unit 38 determines the fundamental frequency y [k] at each time k based on the fundamental frequency y [k] at each time k extracted by the fundamental frequency sequence extraction unit 34. , A command composed of a pair of a phrase command u _p [k] representing a fundamental frequency pattern generated by parallel movement of thyroid cartilage at each time k and an accent command u _a [k] representing a fundamental frequency pattern generated by rotational motion of thyroid cartilage The parameters θ and φ of the second probability distribution, which is a probability distribution in combination with the function o [k], are learned.

  The conversion processing unit 50 according to the second embodiment receives the time-series data of the source speech to be predicted as an input, the spectral feature quantity vector c [k] at each time k extracted from the time-series data of the source speech, Based on the parameter γ of the first probability distribution learned by the first model parameter learning unit 36 and the parameters θ and φ of the second probability distribution learned by the second model parameter learning unit 38, the first probability distribution and In order to increase the criterion expressed using the second probability distribution, a fundamental frequency y [k] at each time k and a command function o [k] composed of a pair of phrase command and accent command at each time k To estimate the fundamental frequency y [k] at each time k of the target speech corresponding to the source speech to be predicted.

  Here, the principle of predicting the fundamental frequency y [k] at each time k of the target sound corresponding to the source sound to be predicted will be described.

<Conversion processing>
Using the parameters {circumflex over (γ)}, {circumflex over (θ)}, {circumflex over (ψ)}, and the feature amount sequence c = {c [k]} ^K _{k = 1} of the prediction target source, Product of

Y and o are estimated so that becomes as large as possible. Where q = Wy and

It is.

  next,

The algorithm for increasing Similar to the F ₀ pattern prediction method from the spectral feature amount sequence of Related Technique 1 described above,

And y and o can be estimated by the following iterative process (the execution order of steps 1 and 2 is arbitrary).

(Step 1) y is initially set by the conversion process of the F ₀ pattern prediction method from the spectral feature quantity sequence of Related Technique 1 described above.

(Step 2) Using c, ^ m is obtained by equation (100).

(Step 3) y is fixed, and o is estimated by the parameter estimation algorithm 2 of the F ₀ pattern generation process model parameter estimation method of the related technique 3 described above.

(Step 4) Fix o and ^ m, update y by the following formula, and return to Step 3.

  In order to realize the principle described above, in the second embodiment, as shown in FIG. 10, the conversion processing unit 50 includes a feature quantity extraction unit 52, a fundamental frequency sequence prediction unit 54, and a normal distribution sequence prediction. Unit 256, command sequence estimation unit 258, fundamental frequency sequence update unit 260, and convergence determination unit 62.

  The normal distribution series prediction unit 256 converts the first probability distribution parameter γ learned by the first model parameter learning unit 36 and the spectral feature quantity vector c [k] at each time extracted by the feature quantity extraction unit 52. Based on the above equation (100), the normal distribution index ^ mk having the highest probability of generating the spectral feature vector c [k] at each time k is estimated.

  The command sequence estimator 258 fixes the fundamental frequency y [k] at each time k that is initially set by the fundamental frequency sequence predictor 54 or updated last time by the state sequence estimator 58, and the related technique 3 described above. The command sequence o that locally maximizes the posterior probability P (o, θ | y) is estimated in the same manner as the parameter estimation algorithm 2 of the F0 pattern generation process model parameter estimation method.

  Based on the command sequence o estimated by the command sequence estimation unit 258 and the index ^ mk of the normal distribution at each time estimated by the normal distribution sequence prediction unit 56, the fundamental frequency sequence update unit 260 ), The basic frequency y [k] at each time k is updated.

  Convergence determining unit 62 repeats the processes by command sequence estimating unit 258 and fundamental frequency sequence updating unit 260 until a predetermined convergence determination condition is satisfied. The convergence determination condition is, for example, reaching a predetermined number of repetitions.

  When the convergence determination condition is satisfied, the fundamental frequency y [k] finally obtained at each time k is output as a prediction result of the fundamental frequency at each time of the target speech corresponding to the source speech to be predicted. 90 for output.

<Operation of fundamental frequency pattern prediction device>

  Next, the operation of the fundamental frequency pattern prediction apparatus 100 according to the second embodiment will be described. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  First, when parallel data composed of time series data of source speech and target speech of a learning sample is input to the fundamental frequency pattern prediction device 100, the fundamental frequency pattern prediction device 100 performs the learning process shown in FIG. The routine is executed.

  First, in step S101, input time series data of source speech is read, and a spectral feature vector c [k] at each time k is extracted. In step S102, the time-series data of the input target voice is read, the fundamental frequency y [k] at each time k of the target voice is extracted, and the fundamental frequency y [k] at each time k and its dynamic component are extracted. Extract the coupling vector q [k].

  In step S103, based on the spectral feature vector c [k] at each time k extracted in step S101 and the combined vector q [k] of the fundamental frequency at each time k extracted in step S102. Thus, the GMM parameter γ in the above equation (1) is learned.

In step S200, the command series o and the parameter θ representing the amplitude of the phrase command and the amplitude of each accent command according to the state at each time are initialized. Further, based on the time series data of the target speech, the voiced and unvoiced sections are specified, and the degree of uncertainty v _n ² [k] of the fundamental frequency at each time k is estimated.

In step S201, the posterior probabilities are calculated for all combinations of (k, t) based on the initial value of the command sequence o set in step S200 or the command sequence o updated last time in step S203 described later. Update P (s _k = t | y, o ′, θ ′).

In step S202, based on the initial value of the command sequence o set in step S200 or the command sequence o updated last time in step S203, which will be described later, all the combinations of (k, l) 71), the auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k, l} are calculated and updated.

In the next step S203, the fundamental frequency sequence y extracted in step S102, the degree of uncertainty v _n ² [k] calculated in step S200, and updated in step S201 are updated. Posterior probability P (s _k = t | y, o ′, θ ′) and auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k, l} updated in step S202 Based on the above, the command sequence o composed of the phrase command u _p [l] and the accent command u _a [l] at each time l which is a non-negative value and the base component u _b are updated according to the above equation (72).

In the next step S204, it is determined whether or not the number of repetitions s has reached S as the convergence condition. If the number of repetitions s has not reached S, it is determined that the convergence condition is not satisfied. Then, the process returns to step S202. On the other hand, when the number of repetitions s reaches S, it is determined that the convergence condition is satisfied, and in step S205, the phrase command u _p [k] and the accent command u _{a at} each time k updated in step S203 above. Based on [k] and the posterior probability P (s _k = t | y, o ′, θ ′) updated in step S201, the above equation (73), equation (74), or equation (75) In accordance with the equation (76), the amplitude A ^(p) [k] of the phrase command at each time k and the amplitude A _a ^(a) of the accent command at each position n are updated to correspond to the state at each time. The parameter θ representing the amplitude of the phrase command and the amplitude of each accent command is updated.

  In step S206, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S201. On the other hand, when the convergence determination condition is satisfied, in step S207,

Based on the command sequence o finally updated in step S203, the state sequence s is estimated by the Viterbi algorithm. Further, the state transition probability φ is obtained from the estimated state sequence s.

  In step S208, the parameter γ learned in step S103, the parameter θ finally obtained in step S205, and the state transition probability φ obtained in step S106 are stored in the parameter storage unit 40. To do.

  Next, when time series data of the source speech to be predicted is input to the fundamental frequency pattern prediction apparatus 100, the fundamental frequency pattern prediction apparatus 100 executes a fundamental frequency pattern prediction processing routine shown in FIG.

  First, in step S121, the input time-series data of the target speech to be predicted is read, and the spectrum feature vector c [k] at each time k is extracted. In step S122, the basic frequency y [k] at each time is initialized based on the parameter γ stored in the parameter storage unit 40 and the spectrum feature vector c [k] at each time extracted in step S121. Set.

In step S123, based on the parameter γ stored in the parameter storage unit 40 and the spectral feature vector c [k] at each time extracted in step S121, the index ^ of the normal distribution at each time k Estimate m _k .

  In step S221, the basic frequency y [k] at each time k initially set in step S122 or updated last time in step S125 described later is fixed, and the posterior probability is the same as in steps S201 to S206. A command sequence o that locally maximizes P (o, θ | y) is estimated.

Then, in step S222, based on the command sequence o estimated in step S221 and the normal distribution index ^ m _k of each time estimated in step S123, each time k Update the fundamental frequency y [k].

  In step S126, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S221. On the other hand, if the convergence determination condition is satisfied, in step S127, the fundamental frequency y [k] at each time k finally obtained in step S222 is set to each of the target speech corresponding to the source speech to be predicted. As a result of prediction of the fundamental frequency of time, the output unit 90 outputs the result, and the fundamental frequency pattern prediction processing routine is finished.

As described above, according to the fundamental frequency pattern predicting apparatus according to the second embodiment, the relationship between the spectral feature vector at each time of the source speech and the fundamental frequency at each time of the target speech is modeled. Learning the parameter γ of the first probability distribution P (q [k], c [k] | γ), which is a generalized GMM, and modeling the fundamental frequency pattern generation process, the second probability distribution P (y, o | θ , Φ) parameters θ and φ are learned, and the first probability distribution P (q [k], c [k] | is obtained from the spectral feature quantity vector at each time extracted from the time series data of the source speech to be predicted. The basic frequency at each time of the target speech corresponding to the source speech to be predicted is increased so that the criterion expressed using the product of γ) and the second probability distribution P (y, o | θ, φ) is increased. by predicting, Do given the constraints of the physical process of generating F ₀ pattern It is possible to estimate the optimal F ₀ pattern corresponding to Luo spectral feature amount sequence.

<Experiment>

For the F ₀ pattern audio data shown in FIG. 13, the F ₀ pattern prediction method from the spectral feature quantity sequence of Related Technique 1 as the conventional method described above and the method according to the first embodiment of the present invention are used. An experiment was conducted to predict the F ₀ pattern from the spectral feature series. FIG. 14 shows the F ₀ pattern predicted by both methods. In FIG. 14, the solid line shows an example of the prediction result of the F ₀ pattern from the speech feature amount sequence by the conventional method, and the dotted line shows the F ₀ pattern from the speech feature amount sequence by the method according to the first embodiment. An example of the prediction result is shown.

The proximity to the F ₀ pattern in FIG. 13 is an indicator of the goodness of the F ₀ pattern. Therefore, the F ₀ patterns obtained by each method, was measured the cosine distance (meaning closer closer to 1) the F ₀ pattern of FIG. 13, the conventional method is 0.55, the first embodiment This method was 0.59. From this, the superiority of the technique of the first embodiment of the present invention over the conventional technique was shown.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, although the basic frequency pattern prediction apparatus described above has a computer system inside, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. Shall be.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Operation part 30 Learning part 32 Feature-value extraction part 34 Fundamental frequency series extraction part 36 1st model parameter learning part 38 2nd model parameter learning part 40 Parameter memory | storage part 50 Conversion process part 52 Feature-value extraction part 54 Fundamental frequency Sequence prediction unit 56 Normal distribution sequence prediction unit 58 State sequence estimation unit 60 Fundamental frequency sequence update unit 62 Convergence determination unit 90 Output unit 100 Fundamental frequency pattern prediction device 256 Normal distribution sequence prediction unit 258 Command sequence estimation unit 260 Basic frequency sequence update unit

Claims (8)

Using parallel data composed of time-series data of source speech and target speech of the learning sample as input, a spectral feature vector at each time extracted from the time-series data of the source speech, and the time of the target speech Based on the fundamental frequency at each time extracted from the sequence data, the first modeled the relationship between the spectral feature quantity vector at each time of the source speech and the fundamental frequency at each time of the target speech. A first model parameter learning unit for learning parameters of the probability distribution;
A second model parameter learning unit that learns a parameter of a second probability distribution obtained by modeling a fundamental frequency pattern generation process based on a fundamental frequency at each time of the target speech;
Using the time series data of the source speech to be predicted as an input, the spectral feature vector at each time extracted from the time series data of the source speech to be predicted, and the first model parameter learning unit learns the first Based on the parameters of the probability distribution and the parameters of the second probability distribution learned by the second model parameter learning unit, the criterion expressed using the first probability distribution and the second probability distribution is increased. A fundamental frequency prediction unit that predicts a fundamental frequency at each time of the target speech corresponding to the source speech to be predicted;
A fundamental frequency pattern prediction apparatus including: The first probability distribution is a mixed normal distribution representing a joint probability distribution of a spectral feature vector at each time of the source speech, a fundamental frequency at each time of the target speech, and a dynamic component of the fundamental frequency,
The second probability distribution is
The fundamental frequency of each time,
From a series of states consisting of states at each time of the Hidden Markov Model, or a pair of phrase commands representing the fundamental frequency pattern generated by the translational movement of the thyroid cartilage at each time and an accent command representing the fundamental frequency pattern generated by the rotational movement of the thyroid cartilage The fundamental frequency pattern prediction apparatus according to claim 1, wherein the probability distribution is a combination with a command function. The criteria
A function corresponding to a combination of the fundamental frequency at each time and the state series, represented by a product of the first probability distribution and the second probability distribution, or the first probability distribution and the second probability The fundamental frequency pattern prediction apparatus according to claim 2, wherein the function is a function corresponding to a combination of a fundamental frequency at each time and a command function at each time, which is expressed using a product of the distribution.   The first model parameter learning unit uses a EM (Expectation-Maximization) algorithm to determine the likelihood of the spectrum feature vector at each time of the source speech and the fundamental frequency at each time of the target speech obtained from the first probability distribution. The fundamental frequency pattern prediction apparatus according to any one of claims 1 to 3, wherein a parameter of the first probability distribution is learned so that becomes larger. The second model parameter learning unit uses a EM (Expectation-Maximization) algorithm to determine the likelihood of a fundamental frequency at each time and a state sequence including a state at each time of a hidden Markov model, obtained from the second probability distribution. Or a parameter group representing the state transition probability in the state series, the amplitude of the phrase command and the amplitude of each accent command according to the state at each time, as a parameter of the second probability distribution, Or, when a basic frequency at each time is given, a command function consisting of a pair of phrase command and accent command at each time and a logarithmic posterior probability of the parameter group as an objective function, the objective function is increased. A command function and the parameter group are learned as parameters of the second probability distribution. The fundamental frequency pattern prediction apparatus according to any one of claims 4. The fundamental frequency prediction unit
The function corresponding to the combination of the fundamental frequency at each time and the state sequence composed of the state at each time of the hidden Markov model, expressed using the product of the first probability distribution and the second probability distribution, is increased. As described above, by estimating the fundamental frequency at each time and the state series, the fundamental frequency at each time of the target speech corresponding to the source speech to be predicted is predicted, or the first probability distribution and the The function corresponding to the combination of the basic frequency at each time, the command function consisting of the phrase command and the accent command pair at each time, expressed using the product with the second probability distribution, is increased. The basic frequency at each time of the target speech corresponding to the prediction target source speech is predicted by estimating the fundamental frequency and the command function at each time. The fundamental frequency pattern prediction apparatus according to any one of 5. A fundamental frequency pattern prediction method in a fundamental frequency pattern prediction apparatus including a first model parameter learning unit, a second model parameter learning unit, and a fundamental frequency prediction unit,
Spectral characteristics at each time extracted from the time series data of the source speech, with the first model parameter learning unit receiving parallel data composed of the time series data of the source speech and the target speech of the learning sample. A spectral feature vector at each time of the source speech and a fundamental frequency at each time of the target speech based on a quantity vector and a fundamental frequency at each time extracted from the time-series data of the target speech. Learn the parameters of the first probability distribution that models the relationship between
The second model parameter learning unit learns a parameter of a second probability distribution obtained by modeling a fundamental frequency pattern generation process based on a fundamental frequency at each time of the target speech;
The fundamental frequency prediction unit receives the time-series data of the source speech to be predicted as an input, the spectral feature quantity vector at each time extracted from the time-series data of the source speech to be predicted, and the first model parameter learning unit The first probability distribution and the second probability distribution are used on the basis of the parameter of the first probability distribution learned by the parameter and the parameter of the second probability distribution learned by the second model parameter learning unit. A basic frequency pattern prediction method for predicting a fundamental frequency at each time of the target speech corresponding to the source speech to be predicted so as to increase a criterion expressed as follows.   The program for functioning a computer as each part of the fundamental frequency pattern prediction apparatus of any one of Claims 1-6.