JP2017151225A - Basic frequency pattern prediction device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate an optimum Fpattern corresponding to a spectrum feature amount sequence, while taking into consideration the restriction of the physical generation process of the Fpattern.SOLUTION: A learning part 30 learns to make the standard large, which is shown by using a parameter of a first probability distribution that models a relationship between a spectrum feature amount vector of each time of a day of a source speech and a basic frequency of each time of a day of a target speech, and the parameter of a second probability distribution that models a basic frequency pattern generation process. The conversion processing part 30 predicts the basic frequency of each time of a day of the target speech corresponding to the source speech of the prediction object from the spectrum feature amount vector of each time of a day extracted from time series data of the source speech of the prediction target so that the standard shown by the first probability distribution and the second probability distribution becomes large.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a fundamental frequency pattern prediction apparatus, method, and program, and more particularly, to a fundamental frequency pattern prediction apparatus, method, and program for predicting a fundamental frequency pattern of a target voice from a source voice.

  In communication with others, voice is a convenient means, but sometimes physical barriers inevitably cause various barriers. For example, if even one of the vocal organs does not operate normally, it suffers from serious vocal disturbances and hinders voice communication. Moreover, the physical action of voice generation is not suitable for highly confidential communication and is vulnerable to ambient noise. In order to eliminate these barriers, voices can be generated by moving the vocal organs beyond physical constraints, voices can be generated by specifying an appropriate pronunciation, or when voices are so fine that it is difficult to hear It is necessary to expand the speech generation function beyond physical and physical constraints, such as generating normal speech from the vocal organ movements.

For example, for laryngectomy patients who have lost their larynx due to laryngeal cancer, etc., voice assist technology has been proposed to convert less natural speech generated by alternative vocalization methods using residual organs into more natural speech. (See Non-Patent Document 1 to Non-Patent Document 3). In addition to this, a technique for converting a non-audible murmur voice into a natural voice has been proposed, and application as a call technique with excellent secrecy is expected. All of the above-mentioned techniques are common in that they deal with the problem of predicting the fundamental frequency (F ₀ ) pattern of natural speech from the spectral feature quantity sequence of speech, and are composed of learning processing and conversion processing. In the learning process, the same speech data of the target voice (electric voice in the former case, non-audible murmur voice in the latter case) and normal voice is used. First, at each discrete time (hereinafter referred to as a frame), the spectral features of the target speech obtained from several frames before and after, the logarithm F _{0 of the} normal speech and its dynamic component (time differential or time difference) are extracted, and the spectral distance measure. A combined vector corresponding to these is obtained by dynamic time expansion and contraction based on. This is called parallel data. Using the parallel data of each frame, the spectral probability of the target speech and the combined probability density function of the static and dynamic components of the logarithm F ₀ of the normal speech are expressed by a mixed normal distribution model (GaMMian Mixture Model; GMM). GMM parameters can be learned using the Expectation-Maximization algorithm. In the conversion process, the learned GMM can be used to convert the spectral feature quantity sequence of the target speech into the F ₀ pattern of the normal speech by the maximum likelihood sequence conversion method considering intra-sequence variation.

Keigo Nakamura, Tomoki Toda, Hiroshi Saruwatari, Kiyohiro Shikano, "Speaking-aid systems using GMM-based voice conversion for electrolaryngeal speech," Speech Communication, vol. 54, no. 1, pp. 134-146, 2012. Kou Tanaka, Tomoki Toda, Graham Neubig, Sakriani Sakti, Satoshi Nakamura, "A hybrid approach to electrolaryngeal speech enhancement based on noise reduction and statistical excitation generation," IEICE Transactions on Information and Systems, vol.E97-D, no. 6, pp. 1429-1437, Jun. 2014. Kou Tanaka, Tomoki Toda, Graham Neubig, Sakriani Sakti, Satoshi Nakamura, "Direct F0 controlof an electrolarynx based on statistical excitation feature prediction and its evaluation through simulation," Proc. INTERSPEECH, pp. 31-35, Sep. 2014. Hirokazu Kameoka, Jonathan Le Roux, Yasunori Ohishi, "A statistical model of speech F0 contours," ISCA Tutorial and Research Workshop on Statistical And Perceptual Audition (SAPA 2010), pp. 43-48, Sep. 2010. Kota Yoshizato, Hirokazu Kameoka, Daisuke Saito, Shigeki Sagayama, "Hidden Markov convolutive mixture model for pitch contour analysis of speech," in Proc.The 13th Annual Conference of the International Speech Communication Association (Interspeech 2012), Sep. 2012.

In the prior art, a model that takes into account the physical generation process of the voice F ₀ pattern was not used in the learning process or conversion process, so an unnatural F ₀ pattern that could not be physically spoken by humans was used. It could happen. To solve this problem, by performing prediction in consideration of the physical process of generating F ₀ pattern, it may be possible to generate a more natural F ₀ pattern.

The F ₀ pattern is generated by the movement of the thyroid cartilage that gives tension to the vocal cords. In Non-Patent Documents 4 and 5, parameters related to the movement of the thyroid cartilage called phrase / accent command are based on the probability model of the control mechanism. An estimation technique has been proposed. In this technology, the time series generation process of the phrase / accent command is expressed by a hidden Markov model (HMM). One of the points is the linguistics about the command sequence through the design of the topology of the HMM and learning of the transition probability. Or a priori knowledge can be incorporated into the parameter estimation.

The present invention has been made in view of the above circumstances, the fundamental frequency can be estimated optimal F ₀ pattern corresponding to the spectral feature amount sequence taking into account the constraints of the physical process of generating F ₀ pattern An object is to provide a pattern prediction apparatus, method, and program.

  In order to achieve the above object, a fundamental frequency pattern predicting apparatus according to the present invention receives, as input, parallel data composed of time series data of source speech of a learning sample and time series of target speech, and the time series of the source speech. Based on the spectral feature vector at each time extracted from the data and the fundamental frequency at each time extracted from the time-series data of the target speech, the spectral feature vector at each time of the source speech, A parameter of a first probability distribution that models the relationship between the fundamental frequencies at each time of the target speech, and a parameter of a second probability distribution that models the fundamental frequency pattern generation process are the first probability distribution and the first probability distribution. A learning unit that learns to increase a criterion expressed using a two-probability distribution, and a time series of a source speech to be predicted Data, the spectral feature vector at each time extracted from the time-series data of the source speech to be predicted, the parameters of the first probability distribution and the second probability distribution learned by the learning unit And a conversion processing unit that predicts a fundamental frequency at each time of the target speech corresponding to the source speech to be predicted so as to increase the criterion based on the parameter.

  A fundamental frequency pattern prediction method according to the present invention is a fundamental frequency pattern prediction method in a fundamental frequency pattern prediction apparatus including a learning unit and a conversion processing unit, wherein the learning unit is time-series data of source speech of a learning sample. And time data extracted from the time-series data of the target sound and the time-series data of the target sound, and the time-series data of the target sound. Based on the fundamental frequency of the source speech, a parameter of the first probability distribution modeling the relationship between the spectral feature quantity vector at each time of the source speech and the fundamental frequency at each time of the target speech, and a fundamental frequency pattern The parameters of the second probability distribution modeling the generation process are the first probability distribution and the second probability distribution. Each time extracted from the time-series data of the prediction target source speech, using the time-series data of the source speech to be predicted as an input, the conversion processing unit learns to increase the criterion expressed using Corresponding to the source speech to be predicted so as to increase the criterion on the basis of the spectral feature quantity vector and the parameters of the first probability distribution and the second probability distribution learned by the learning unit. The fundamental frequency at each time of the target speech to be predicted is predicted.

  A program according to the present invention is a program for causing a computer to function as each unit of the above-described basic frequency pattern prediction apparatus.

As described above, according to the fundamental frequency pattern predicting apparatus, method, and program of the present invention, the relationship between the spectral feature vector at each time of the source speech and the fundamental frequency at each time of the target speech is modeled. The criterion expressed using the first probability distribution and the second probability distribution is increased for the first probability distribution parameter and the second probability distribution parameter obtained by modeling the fundamental frequency pattern generation process. The prediction target is learned so as to increase the criterion expressed using the first probability distribution and the second probability distribution from the spectral feature quantity vector at each time extracted from the time series data of the source speech to be predicted. spectrum of by predicting the fundamental frequency of the target time sound corresponding to the source voice, taking into account the constraints of the physical process of generating F ₀ pattern It is possible to estimate the optimal F ₀ pattern corresponding to Le feature amount sequence, the effect is obtained that.

It is a figure for demonstrating an example of the state transition network of HMM. It is a figure for demonstrating an example of the state transition network of HMM. It is a figure for demonstrating an example of the state transition network of HMM. It is a figure for demonstrating the division | segmentation of a state. It is the schematic which shows the structure of the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is the schematic which shows the structure of the learning part of the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is the schematic which shows the structure of the conversion process part of the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the learning process routine in the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the fundamental frequency pattern prediction process routine in the fundamental frequency pattern prediction apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the learning process routine in the fundamental frequency pattern prediction apparatus which concerns on the 2nd Embodiment of this invention. It is the schematic which shows the structure of the learning part of the fundamental frequency pattern prediction apparatus which concerns on the 3rd Embodiment of this invention. It is a flowchart which shows the content of the learning process routine in the fundamental frequency pattern prediction apparatus which concerns on the 3rd Embodiment of this invention. It is a figure which shows an experimental result.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The technique proposed in the present invention predicts a fundamental frequency pattern from a feature sequence of speech and replaces the fundamental frequency pattern of the original speech with the predicted fundamental frequency pattern to improve speech naturalness. Processing technology.

<Related technology 1: F ₀ pattern prediction method from spectral feature sequence>
First, the F ₀ pattern prediction method from the spectral feature quantity sequence will be described.

Non-Patent Document 1 to Non-Patent Document 3 propose a method for predicting an F ₀ pattern from a spectrum feature amount sequence. The conventional method consists of the process of converting from a given spectral feature amount sequence in F ₀ pattern using the model trained with the processing for learning the parameters of the joint probability distribution model of the spectral feature amount sequence and F ₀ pattern.

<Learning process>
It is assumed that parallel data of a source sound (for example, electric sound) and a target sound (for example, natural sound) is given. The spectral feature vector of the source speech is c [k], and the combined vector (called F ₀ feature) of the logarithm F _{0 of the} target speech and its dynamic component (time differential or time difference) is q [k] = (y [k]; Δy [k]) ^T. Where k is an index of discrete time. As the speech feature value c [k], for example, a vector obtained by performing dimension compression by principal component analysis on a vector obtained by connecting a series of mel cepstrum (vector) for several frames around the time k is used. In this method, the joint probability distribution (Gaussian Mixture Model; GMM) of the joint probability distribution of c [k] and q [k] is used.

In the learning process, the GMM parameters (weight, mean, variance-covariance matrix) of the GMM are learned from given parallel data {c [k]; q [k]} ^K _{k = 1} . However, N (x; μ, Σ) means that the probability density function of x is given by a normal distribution having an average of μ and a variance-covariance matrix of Σ.

GMM parameters can be estimated by the Expectation-Maximization (EM) algorithm. If the learned GMM parameter is γ, the conditional distribution P (q [k] | c [k], γ) is a distribution for predicting the F ₀ feature q [k] from the spectrum feature c [k]. Can see,

As in P (c [k], q [k] | γ), it is given by GMM. However,

And e ^{(q | c)} _m and D ^{(q | c)} _m are

Given in.

<Conversion processing>
In the conversion process. A given spectral feature series

Maximum likelihood F ₀ pattern under

Is obtained by the following equation (8).

  However,

W is a transformation matrix (constant) describing the relationship between y and q. Here, P (q | c, γ) is a GMM given by the parameter γ learned by the learning process,

Given in. Here, m = (m ₁ ,..., M _K ), and m _k represents the component index of the GMM at time k. Where P (q [k] | c [k], γ) is

Can be approximated by From equation (12), ^ m _k means a normal distribution index having the highest probability of generating data c [k]. Therefore, from equation (9), P (q | c, m, ^ γ) is for all k.

It is given by taking the product of By approximation of equation (11), P (q | c, γ) is

The vector e ^{(q | c)} concatenated with

Distribution with block diagonal matrix D ^{(q | c)} as a variance-covariance matrix

It becomes. Substituting q = Wy into this and normalizing to be the distribution of y,

So that

A conditional distribution of y is obtained as follows. Therefore, the solution of equation (8) is

It becomes.

<Related technology 2: F ₀ pattern generation process model>
Next, a probability model of the F ₀ pattern generation process will be described.

Fujisaki's fundamental frequency (F ₀ ) pattern generation process model (Fujisaki model) is known as a model that describes the F ₀ pattern generation process of speech (Non-patent Document 6).

[Non-Patent Document 6]: H. Fujisaki, “In Vocal Physiology: Voice Production, Mechanisms and Functions,” Raven Press, 1988.

The Fujisaki model is a physical model describing the process of generating F ₀ pattern due to the motion of the thyroid cartilage. The Fujisaki model, the total elongation of the vocal cords with each two independent movement of the thyroid cartilage (translational motion and rotational motion) is interpreted to result temporal variation of F _0, pairs of elongation and F ₀ pattern of the vocal cords The F ₀ pattern is modeled on the assumption that the numerical value y (t) is proportional. The F ₀ pattern x _p (t) generated by the translational motion of the thyroid cartilage is called a phrase component, and the F ₀ pattern x _a (t) generated by the rotational motion is called an accent component. In the Fujisaki model, the F ₀ pattern y (t) of speech is the sum of these components plus the baseline component b determined by the physical constraints of the vocal cords.

It is expressed as These two components are assumed to be the output of the second-order critical braking system,

(* Is a convolution operation with respect to time t). Here, u _p (t) is called a phrase command function and consists of a sequence of delta functions (phrase commands), and u _a (t) is called an accent command function and consists of a sequence of rectangular waves (accent commands). These command sequences have a constraint condition that a phrase command occurs at the beginning of an utterance, phrase commands do not occur twice in succession, and two different commands do not occur at the same time. Α and β are the natural angular frequencies of the phrase control mechanism and the accent control mechanism, respectively. It is experiential that α is approximately 3 rad / s and β is 20 rad / s, regardless of the speaker or utterance content. Known to.

<Related technology 3: F ₀ pattern generation process model parameter estimation method>
The above-mentioned Fujisaki model can be described by the following probability model (see Non-Patent Documents 4, 5, and 7).

[Non-Patent Document 7]: Tatsuma Ishihara, Kota Yoshizato, Hirokazu Kameoka, Daisuke Saito, Shigeki Hiyama, \ Statistical vocabulary model of the Fujisaki model command sequence of the fundamental speech frequency, "Lecture at the 2013 Acoustical Society of Japan Proceedings, 1-7-9, pp. 283-286, Mar. 2013.

First phrase, a pair o of accent command function _{[k] = (u p [} k], u a [k]) consider the HMM to output the ^T. Here, k represents an index of discrete time. The state output distribution is a normal distribution.

The command function pair o [k] is generated by Here, {s _k } ^K _{k = 1} is an HMM state sequence, and the average vector ρ [k] is a value determined as a result of the state transition of the HMM. Examples of specific HMM configurations are shown in FIGS.

In the example of the state transition network of the HMM shown in FIG. 1, both μ ^(p) _t [k] and μ ^(a) _t are ₀ in the state t = r ₀ . The state t = r ₀ can only transition to the state p ₀ , and in the state t = p ₀ , μ ^(p) _t [k] takes a non-negative value A ^(p) [k], and μ ^(a) _t Becomes 0. The transition after state t = p ₀ is allowed only to state r ₁ . Similar to state t = r ₀ , μ ^(p) _t [k] and μ ^(a) _t are both 0 in state t = r ₁ . A transition from state t = r ₁ to only _one of states a ₀ , ..., a _N-1 is possible, and in state t = a _n μ ^(a) _t is a non-negative value A ^(a) _n Therefore, μ ^(p) _t [k] is 0. Next state t = a _n is allowed to transition only in the state r ₀ or r _1. This guarantees that μ _a [k] is a rectangular pulse train.

In the example of the state transition network of the HMM shown in FIG. 2, both μ ^(p) _t [k] and μ ^(a) _t are ₀ in the state t = r ₀ . A transition from state t = r ₀ to only _one of states p ₀ , ..., p _M-1 is possible, and μ ^(p) _t is a non-negative value A ^(p) _m in state t = p _m Therefore, μ ^(a) _t becomes 0. Next state t = p _m is allowed to transition only in state r _1. Similar to state t = r ₀ , μ ^(p) _t [k] and μ ^(a) _t are both 0 in state t = r ₁ . A transition from state t = r ₁ to only _one of states a ₀ , ..., a _N-1 is possible, and in state t = a _n μ ^(a) _t is a non-negative value A ^(a) _n Therefore, μ ^(p) _t becomes 0. Next state t = a _n is allowed to transition only in the state r ₀ or r _1. This guarantees that μ _a [k] is a rectangular pulse train.

The example of the state transition network of the HMM shown in FIG. 3 is composed of a plurality of Left-to-Right type HMMs in which respective end points and start points are connected. Similar to Figure 1 and 2, both the state t = r _l in μ ^(p) _t [k] and mu ^(a) _t is 0. The state t = In ^p _m μ ^(p) _t takes a nonnegative value ^{_{A (p) m, μ (}} a) t is 0. In the state t = a _n μ ^(a) _t takes a non-negative value A ^(a) _n and μ ^(p)

_t is 0.

  In the example of FIG. 1, ρ [k] is expressed by the following equation (24).

  2 and 3, ρ [k] is expressed by the following equation (25).

In either example, as shown in FIG. 4, the time for each state to stay in the same state by dividing it into several small states with the same output distribution and restricting the left-to-right state transition path Long probabilities can be parameterized. Figure 4 is an example of dividing the state a _n. For example, by setting the state transition probability from a _{n, n ′} to a _{n, n ′ + 1} to ₁ for all m ≠ 0 as shown in this figure, a _{n, 0} to a _{n, n ′} transition probabilities to correspond to the probability that state a _n lasts only n 'step, it becomes possible or to learn to set the probability of persistence length of accent command. Similarly, by dividing p _m and r _l into small states, it becomes possible to parameterize the distribution of the duration of the phrase command and the length of the interval between commands. After that, the state set

Is written. The configuration of the above HMM can be written as follows.

When the state sequence s = {s _k } ^K _{k = 1} is given, the HMM outputs a pair of _a phrase command function u _p [k] and an accent command function u _a [k]. As shown in Eqs. (18) and (20), g _p [k] and g _a [k] are convolved with u _p [k] and u _a [k], respectively, and the phrase component x _p [k] And the accent component x _a [k] are output. This can be expressed as an expression:

(* Is a convolution operation for discrete time k). At this time, the F ₀ pattern x [k] is

And can be written by superimposing three kinds of components. Where b is a baseline component that does not depend on time.

In real speech, reliable F ₀ values are not always observable. When estimating the parameters of the Fujisaki model, it is desirable to consider only the F ₀ value of the reliable observation interval and ignore the other intervals. For example, in a voiceless section, periodic coarse / fine waves associated with normal vocal cord vibration are not observed, so even if some value is obtained as an estimated value of F ₀ from the voiceless section by automatic pitch extraction, its value Is not considered to be the value of F ₀ of a signal emitted from the vocal cords. Therefore, the degree of uncertainty v ² _n [k] at time k of the observed F ₀ value is introduced into the proposed model. Specifically, the observed F ₀ value y [k], the true F ₀ value x [k] and the noise component

With overlay

This means that all observation intervals can be handled uniformly regardless of whether they are reliable intervals.

Probability density function of y = {y [k]} ^K _{k = 1} given the output value sequence o = {o [k]} ^K _{k = 1} by marginalizing x _n [k]

Is obtained. State series s = {s _k } ^K _{k = 1} and parameter indicating command amplitude

And the transition probability matrix φ = (φ _{i, j} ) _{I × I} , the output value sequence o is

Is generated according to P (s | φ) is the product of the state transition probabilities.

Can be written. However,

Represents the probability that the initial state is s ₁ . From equations (30), (32) and (33), P (y, o, s | θ, φ) is

Can be written. If this is marginalized with respect to o,

Is obtained. However,

It is. On the other hand, if you marginalize s

Is obtained. However, Σ _s means an operation for taking a sum for all state sequences.

<Parameter estimation algorithm 1>
By regarding y and o as complete data, s and θ that locally maximize Equation (35) can be searched by the Expectation-Maximization algorithm. Although derivation is omitted,

Updating s and θ so that becomes large, and using the updated s and θ

And R

(35) can be monotonously increased by repeating the updating step according to the above (refer to Non-Patent Document 4 above for details).

  Specifically, the following initial setting, E step, and M step are executed.

(Initial setting)
s and θ are initialized.

(E step)
Conditional expected values for phrase, accent, and baseline components

And the conditional covariance matrix R

Update with However,

It is. Also in R

The block diagonal component corresponding to

And

  That is,

(* Is a block component not used later).

(M step)
A state series s = (s ₁ ,..., S _K ) that maximizes Q (s, θ) is searched. γ _p and γ _a are diagonal

Since it is a column,

Are both

Can be written in the form of the sum of terms for every k. Therefore, Q (s, θ) is aside from terms that do not depend on s.

Can be written. Therefore, the state sequence s = (s ₁ ,..., S _K ) that maximizes Q (s, θ) can be obtained by the Viterbi algorithm (refer to Non-Patent Document 4 for details). Here, [·] _{k, k} represents a k-by-k element of the matrix, and [·] _k represents a k-th element of the vector.

  Subsequently, θ is updated to maximize Q (s, θ). Θ that maximizes Q (s, θ) can be obtained by setting the partial differential for each variable of Q (s, θ) to 0 (for details, see Non-Patent Document 4 above).

  Further, a state transition probability φ is obtained from the estimated state sequence s.

[First Embodiment]
<Outline of Embodiment of the Present Invention>
The technique according to the embodiment of the present invention includes a learning process and a conversion process as in the related technique 1 described above. Instead of the expression (8), the probability distribution of the related technique 1 and the probability distribution of the related technique 2 are expressed as By using “Product-of-Experts” (see Non-Patent Document 9), it is possible to statistically predict F ₀ patterns from spectral features that are as close as possible to the F0 pattern generation process model of Related Technology 2. Technology.

[Non-Patent Document 9]: G. E. Hinton, “Training Products of Experts by Minimizing Contrastive Divergence,” Neural Computation, no. 14, no. 8, pp. 1771-1800, 2002.

  In the learning process and the conversion process, a joint distribution P (c, y, m, s | γ, θ, φ) of c and y described later is used as a common criterion.

In the learning process, a learning distribution {c [k], q [k]} ^K _{k = 1} of the parallel data is given, and the joint distribution P (c, y, m, s | γ, θ, φ of c, y is given. ) Is learned so as to be as large as possible. In the present embodiment, γ, θ, and φ are learned separately. For example, given the F ₀ pattern {y [k]} ^K _{k = 1 of} the learning sample, P (y, s | θ, φ) can be as much as possible using the parameter estimation algorithm 1 of the related technique 3 described above. Learn θ and φ to be large. Then, θ and φ are fixed, and γ is learned so that P (c, y, m, s | γ, θ, φ) becomes as large as possible.

  Under the condition that the phrase / accent command data o of the learning sample is given, the related technique 3 described above may be used to learn θ and φ so that P (o | θ, φ) is as large as possible. .

<System configuration>
Next, an embodiment of the present invention will be described by taking as an example a case where the present invention is applied to a fundamental frequency pattern prediction apparatus that predicts a fundamental frequency pattern of a target speech from a spectral feature quantity sequence of a source speech.

  As shown in FIG. 5, the fundamental frequency pattern prediction apparatus according to the first embodiment of the present invention includes a CPU, a RAM, a learning processing routine, and a program for executing a fundamental frequency pattern prediction processing routine, which will be described later. The computer is provided with a ROM that stores the above, and is functionally configured as follows.

  As shown in FIG. 5, the fundamental frequency pattern prediction apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 90.

  The input unit 10 accepts parallel data composed of time-series data of the source sound (for example, electric sound) of the learning sample and time-series data of the target sound (for example, natural sound). In addition, the input unit 10 receives time-series data of the source audio to be predicted.

  The calculation unit 20 includes a learning unit 30, a parameter storage unit 40, and a conversion processing unit 50.

  As shown in FIG. 6, the learning unit 30 includes a feature amount extraction unit 32, a fundamental frequency series extraction unit 34, a first model parameter learning unit 36, and a second model parameter learning unit 38.

  The feature quantity extraction unit 32 extracts the source speech spectogram feature quantity vector c [k] from the time series data of the source speech of the learning sample received by the input unit 10. Where k is an index of discrete time. For example, as in Non-Patent Documents 1 to 3, a vector obtained by performing dimension compression by principal component analysis on a vector obtained by concatenating a series of mel cepstrum (vectors) for several frames around the time k is used as c [k ] Is used.

The fundamental frequency series extraction unit 34 extracts the fundamental frequency y [k] at each time k of the target speech from the time series data of the target speech of the learning sample received by the input unit 10, and y = (y [1], ..., y [K]) ^T.

This fundamental frequency extraction process can be realized by a well-known technique. For example, Non-Patent Document 8 (H. Kameoka, “Statistical speech spectrum model incorporating all-pole vocal tract model and F ₀ contour generating process model,” in Tech. Rep. IEICE, 2010, in Japanese.) The fundamental frequency is extracted every 8 ms.

Also, let q [k] = (y [k], Δy [k]) ^T be the combined vector (called F ₀ feature) of y and its dynamic component (time derivative or time difference).

As described above, data {c [k], q [k]} ^K _{k = 1} is obtained.

  The second model parameter learning unit 38, based on the fundamental frequency y [k] at each time k extracted by the fundamental frequency sequence extraction unit 34, each of the fundamental frequency y [k] at each time k and the hidden Markov model. A parameter of the second probability distribution, which is a probability distribution of a combination with the state series s composed of time states, is learned.

Specifically, the second model parameter learning unit 38 learns the parameters θ and φ of the F ₀ pattern generation process model according to the parameter estimation algorithm 1 of the F ₀ pattern generation process model parameter estimation method of the related technique 3 described above. .

If data o = {o _k } ^K _{k = 1} is available for the phrase command sequence and accent command sequence in the learning sample, θ and φ may be learned from o (for normal learning in HMM). Equivalent). Let the parameters of the learned F ₀ pattern generation process model be ^ θ and ^ φ.

  The first model parameter learning unit 36 combines the spectral feature vector c [k] at each time k extracted by the feature extraction unit 32 and the fundamental frequency at each time k extracted by the fundamental frequency sequence extraction unit 34. Based on the vector q [k] and the parameters θ and φ learned by the second model parameter learning unit 38, the combined distribution P (c, y, m, s | γ, θ, φ) of c and y is obtained. A mixed normal distribution representing a joint probability distribution of the spectral feature vector c [k] at each time of the source speech and the combined vector q [k] of the fundamental frequency at each time k of the target speech so as to be as large as possible. A parameter γ of one probability distribution is learned.

<Learning / Conversion Criteria>
Here, a standard common to the learning process and the conversion process will be described.

  The joint distribution P (c, y, m, s | γ, θ, φ) of c and y is given in the following form.

However,

As described above, the combined distribution P (c, y, m, s | γ, θ, φ) of c and y has normalized the product of the probability distributions of the GMM-based F ₀ pattern prediction model and the F ₀ pattern generation process model. Λ and Γ are matrices representing the magnitudes of the contributions of both models. Both are arbitrary diagonal matrices (constants). Other variables are the same as those described in Related Technology 1 and Related Technology 3 described above. In addition, in the above, the superscripts (m) and (s) are given to variables depending on the GMM component index sequence and the HMM state sequence for the sake of simplicity.

<Unified algorithm for learning and conversion>
Next, a unified algorithm for learning / conversion processing will be described.

  Optimization problem using the same criterion for both learning and transformation

Thus, the Expectation-Maximization algorithm can be applied by treating q and u as latent variables (hidden variables). If y, c, q, and u are complete data, the likelihood function for the complete data can be written as follows.

However,

  When the equation (92) is squarely completed, P (c, y, m, s | γ, θ, φ) is expressed as follows.

  In step E, the expected value is calculated according to the following equations (100) and (101).

  In M steps,

LogP (c, y, m, s | γ, θ, φ) can be increased by updating each variable so that the maximum value is obtained. Therefore, the optimization problem can be obtained by repeating E step and M step. Can be obtained.

  In accordance with the principle described above, the first model parameter learning unit 36 fixes the parameters θ and φ learned by the second model parameter learning unit 36, and the spectral features at each time k extracted by the feature amount extraction unit 32. Based on the quantity vector c [k] and the combined vector q [k] of the fundamental frequency at each time k extracted by the fundamental frequency sequence extraction unit 34, an E step and an M are performed by an EM (Expectation-Maximization) algorithm. By repeating the steps, the GMM parameter γ of the first probability distribution is learned so as to increase logP (c, y, m, s | γ, θ, φ). Let the learned GMM parameter be ^ γ.

  The conversion processing unit 50 receives the time-series data of the source speech to be predicted as an input, the spectral feature vector at each time extracted from the time-series data of the source speech, and the first model parameter learning unit 36 learned the first time. Based on the parameter γ of one probability distribution and the parameters θ and φ of the second probability distribution learned by the second model parameter learning unit 38, the E step and the M step are repeated by an EM (Expectation-Maximization) algorithm. In order to increase P (c, y, m, s | γ, θ, φ), which is a criterion common to the learning process, the spectrum feature vector at each time k and the spectral feature vector at each time k is generated. The target sound corresponding to the source sound to be predicted is estimated by estimating the normal distribution index m having the highest probability and the state series s composed of the states at each time. To predict the fundamental frequency y of each time.

  In the present embodiment, as illustrated in FIG. 7, the conversion processing unit 50 includes a feature amount extraction unit 52, an expected value calculation unit 54, a variable update unit 56, and a convergence determination unit 58.

  Similar to the feature quantity extraction unit 32, the feature quantity extraction unit 52 obtains the spectrogram feature quantity vector c [k] of the source speech at each time k from the time series data of the prediction target source voice received by the input unit 10. Extract.

  The expected value calculation unit 54 includes the parameter γ of the first probability distribution learned by the first model parameter learning unit 36, the parameters θ and φ of the second probability distribution learned by the second model parameter learning unit 38, and the feature The spectral feature vector c [k] at each time k extracted by the quantity extraction unit 52, the basic frequency y at each time last updated by the variable update unit 56, the index m of the normal distribution at each time k, and each Based on the state series s composed of time states, the expected value is calculated according to the above formulas (100) and (101).

  Based on the expected value calculated by the expected value calculation unit 54, the variable update unit 56 sets the basic frequency y at each time and the index m of the normal distribution at each time k so that the formula (102) becomes maximum. And the state series s composed of the states at the respective times are updated.

  The convergence determination unit 58 repeats each process performed by the expected value calculation unit 54 and the variable update unit 56 until a predetermined convergence determination condition is satisfied. The convergence determination condition is, for example, reaching a predetermined number of repetitions.

  When the convergence determination condition is satisfied, the fundamental frequency y [k] finally obtained at each time k is output as a prediction result of the fundamental frequency at each time of the target speech corresponding to the source speech to be predicted. 90 for output.

<Operation of fundamental frequency pattern prediction device>
Next, the operation of basic frequency pattern prediction apparatus 100 according to the present embodiment will be described. First, when parallel data composed of time series data of source speech and target speech of a learning sample is input to the fundamental frequency pattern prediction device 100, the fundamental frequency pattern prediction device 100 performs the learning process shown in FIG. The routine is executed.

  First, in step S101, input time-series data of the source sound is read,

A spectral feature vector c [k] at each time k is extracted. In step S102, the time-series data of the input target speech is read, the fundamental frequency y [k] at each time k of the target speech is extracted, and the combined vector q of the fundamental frequency y [k] and its dynamic component is extracted. Extract [k].

  In step S104, the state series s and the parameter θ representing the amplitude of the phrase command and the amplitude of each accent command corresponding to the state at each time are initialized.

  In step S105, the conditional expected value ￣x of the phrase component, the accent component, and the baseline component and the conditional variance-covariance matrix R are updated according to the above formulas (49) and (50).

In the next step S106, the parameter θ initially set in step S104 or updated last time in step S107, which will be described later, and the conditional expected values of the phrase component, accent component, and baseline component updated in step S105. Based on ￣x and the conditional covariance matrix R, the state sequence s = (s ₁ ,..., S _K , which maximizes Q (s, θ), using the above equation (63). ) Is obtained by the Viterbi algorithm, and the state sequence s is updated.

  In step S107, the state series s updated in step S106, the conditional expected value ￣x of the phrase component, accent component, and baseline component updated in step S105, and the conditional variance-covariance matrix R are included. Based on this, the parameter θ that maximizes Q (s, θ) is obtained by setting the partial differential for each variable of Q (s, θ) to 0, and the amplitude of the phrase command corresponding to the state at each time and The parameter θ representing the amplitude of each accent command is updated.

  In step S108, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S105. On the other hand, when the convergence determination condition is satisfied, the process proceeds to step S109.

  In step S109, the spectral feature vector c [k] at each time k extracted in step S101, the combined vector q [k] of the fundamental frequency at each time k extracted in step S102, and the above step S107. On the basis of the parameter θ finally obtained in step S10, the state transition probability φ obtained from the state sequence s finally obtained in step S106, and the parameter γ previously updated in step S110 described later. The expected value is calculated according to the equations (100) and (101).

  In step S110, the parameter γ is updated based on the expected value calculated in step S109 so that the equation (102) becomes maximum.

  In step S112, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S109. On the other hand, when the convergence determination condition is satisfied, in step S113, the state transition probability φ obtained from the parameter θ finally obtained in step S107 and the state sequence s finally obtained in step S106. And the parameter γ finally obtained in step S110 is stored in the parameter storage unit 40.

  Next, when the time-series data of the source speech to be predicted is input to the fundamental frequency pattern predicting apparatus 100, the fundamental frequency pattern predicting process routine shown in FIG.

  First, in step S121, the input time-series data of the target speech to be predicted is read, and the spectrum feature vector c [k] at each time k is extracted.

In step S122, based on the parameter γ stored in the parameter storage unit 40 and the spectral feature quantity vector c [k] at each time extracted in step S121, according to the above equation (16), at each time k. By estimating the fundamental frequency y [k], the fundamental frequency y [k] at each time k is initialized, and an initial value is set to the coupling vector q [k] of the fundamental frequency at each time k. In addition, initial values are set in the normal distribution index ^ m _k having the highest probability that the spectral feature vector c [k] at each time k is generated and the state sequence s.

  In step S123, the spectral feature vector c [k] at each time k extracted in step S121, the fundamental frequency y [k] at each time k, and the fundamental frequency y [k] at each time k are obtained. Based on the combination vector q [k] of the fundamental frequency at time k, the parameter θ, the state transition probability φ, and the parameter γ stored in the parameter storage unit 40, according to the above equations (100) and (101), Calculate the expected value.

In step S124, based on the expected value calculated in step S123, the fundamental frequency y [k] at each time k and the normal distribution index ^ m at each time k so that the equation (102) is maximized. _k and state series s are updated.

  In step S125, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S123. On the other hand, when the convergence determination condition is satisfied, in step S126, the fundamental frequency y [k] at each time k finally obtained in step S124 is set to each of the target speech corresponding to the source speech to be predicted. As a result of prediction of the fundamental frequency of time, the output unit 90 outputs the result, and the fundamental frequency pattern prediction processing routine is finished.

As described above, according to the fundamental frequency pattern prediction apparatus according to the first embodiment, the parameters θ and φ of the second probability distribution modeling the fundamental frequency pattern generation process are learned, and each time of the source speech The parameter P of the first probability distribution that models the relationship between the spectral feature vector of the target speech and the fundamental frequency at each time of the target speech is expressed as a criterion P expressed using the first probability distribution and the second probability distribution. Learning to increase (c, y, m, s | γ, θ, φ), and from the spectral feature quantity vector at each time extracted from the time series data of the source speech to be predicted, the first probability distribution and The basic frequency at each time of the target speech corresponding to the source speech to be predicted so as to increase the criterion P (c, y, m, s | γ, θ, φ) expressed using the second probability distribution by predicting the physical of F ₀ pattern It is possible to estimate the optimal F ₀ pattern corresponding to the spectral feature amount sequence taking into account the constraints of the production process

[Second Embodiment]
Next, a fundamental frequency pattern prediction apparatus according to the second embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected about the part which becomes the same structure as 1st Embodiment, and description is abbreviate | omitted.

  In the second embodiment, the second probability distribution is P (y, o | θ, φ), and the method for estimating the parameters θ and φ is different from that in the first embodiment.

  The principle of learning parameters by the second model parameter learning unit 38 of the learning unit 30 of the fundamental frequency pattern prediction apparatus according to the second embodiment will be described.

First, the parameter estimation algorithm of the F ₀ pattern generation process model parameter estimation method of the related technique 3 will be described.

<Parameter estimation algorithm 2>
Given an observation F ₀ sequence y, an iterative algorithm for obtaining a local optimal solution of the posterior probabilities P (o, θ | y) of model parameters θ and o is shown below. The state series s is a hidden variable, and the posterior probability P (o, θ | y) is

Note that the Q function Q (o, θ, o ′, θ ′) is

I can put it. However,

Represents equality except for the constant term. G _b [k] = δ [k] (Kronecker delta). Therefore, the step of calculating P (s _k = t | y, o ′, θ ′) by the Forward-Backward algorithm and the step of increasing Q (o, θ, o ′, θ ′) for o and θ are repeated. Thus, a solution in which P (o, θ | y) is locally maximum can be obtained. Since o is a phrase / accent command sequence pair, in the step of increasing Q (o, θ, o ′, θ ′), it is necessary to consider the non-negative constraint of o. An update rule that increases Q (o, θ, o ′, θ ′) while satisfying the non-negative constraint of o can be derived as follows. First, the lower bound of Q (o, θ, o ′, θ ′) is from Jensen's inequality.

Can be designed as follows. I, k, and l are

 Any variable that satisfies So the lower bound of the Q function is

It is expressed. Q (o, θ, o ′, θ ′) can be increased by alternately repeating the step of maximizing the lower bound function with respect to λ _{i, k, l} ≧ 0 and the step of maximizing with respect to o 1. The update rule for any step can be determined analytically,

It is represented by After the above iterations converge, θ is continuously updated. In the case of FIG.

  In the case of FIGS.

It is. After substituting these update values for o ′ and θ ′, P (s _k = t | y, o ′, θ ′) is updated again, and thereafter the same processing is repeated to thereby determine the posterior probability P (o , Θ | y) can be monotonously increased.

  After the above iterative algorithm has converged, the optimum s obtained by the Viterbi algorithm of the parameter estimation algorithm 1 described above is taken as the state sequence estimation result.

  Further, a state transition probability φ is obtained from the estimated state sequence s.

In accordance with the principle described above, the second model parameter learning unit 38 determines the fundamental frequency y [k] at each time k based on the fundamental frequency y [k] at each time k extracted by the fundamental frequency sequence extraction unit 34. , A command composed of a pair of a phrase command u _p [k] representing a fundamental frequency pattern generated by parallel movement of thyroid cartilage at each time k and an accent command u _a [k] representing a fundamental frequency pattern generated by rotational motion of thyroid cartilage The parameters θ and φ of the second probability distribution, which is a probability distribution in combination with the function o [k], are learned.

  As in the first embodiment, the first model parameter learning unit 36 fixes the parameters θ and φ learned by the second model parameter learning unit 36 and extracts each of the features extracted by the feature amount extraction unit 32. Based on the spectral feature vector c [k] at time k and the combined vector q [k] of the fundamental frequency at each time k extracted by the fundamental frequency sequence extraction unit 34, an EM (Expectation-Maximization) algorithm is used. By repeating the E step and the M step, the GMM parameter γ of the first probability distribution is learned so as to increase logP (c, y, m, s | γ, θ, φ). Let the learned GMM parameter be ^ γ.

  Similarly to the first embodiment, the conversion processing unit 50 receives the time-series data of the source speech to be predicted as an input, the spectral feature quantity vector at each time extracted from the time-series data of the source speech, and the first Based on the parameter γ of the first probability distribution learned by the model parameter learning unit 36 and the parameters θ and φ of the second probability distribution learned by the second model parameter learning unit 38, an EM (Expectation-Maximization) algorithm Thus, by repeating the E step and the M step, the basic frequency y at each time is set so as to increase P (c, y, m, s | γ, θ, φ), which is a standard common to the learning process, By estimating a normal distribution index m having the highest probability of generating a spectral feature vector at each time k and a state series s consisting of states at each time, the source to be predicted To predict the fundamental frequency y at each time of the target sound corresponding to the voice.

<Operation of fundamental frequency pattern prediction device>

  Next, the operation of the fundamental frequency pattern prediction apparatus according to the second embodiment will be described. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  First, when parallel data including time series data of source speech and target speech of a learning sample is input to the fundamental frequency pattern prediction device, the learning processing routine shown in FIG. Executed.

  First, in step S101, input time-series data of the source sound is read,

A spectral feature vector c [k] at each time k is extracted. In step S102, the time-series data of the input target speech is read, the fundamental frequency y [k] at each time k of the target speech is extracted, and the combined vector q of the fundamental frequency y [k] and its dynamic component is extracted. Extract [k].

In step S200, the command series o and the parameter θ representing the amplitude of the phrase command and the amplitude of each accent command according to the state at each time are initialized. Further, based on the time series data of the target speech, the voiced and unvoiced sections are specified, and the degree of uncertainty v _n ² [k] of the fundamental frequency at each time k is estimated.

In step S201, the posterior probabilities are calculated for all combinations of (k, t) based on the initial value of the command sequence o set in step S200 or the command sequence o updated last time in step S203 described later. Update P (s _k = t | y, o ′, θ ′).

In step S202, based on the initial value of the command sequence o set in step S200 or the command sequence o updated last time in step S203, which will be described later, all the combinations of (k, l) 71), the auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k, l} are calculated and updated.

In the next step S203, the fundamental frequency sequence y extracted in step S102, the degree of uncertainty v _n ² [k] calculated in step S200, and updated in step S201 are updated. Posterior probability P (s _k = t | y, o ′, θ ′) and auxiliary variables λ _{p, k, l} , λ _{a, k, l} , λ _{b, k, l} updated in step S202 Based on the above, the command sequence o composed of the phrase command u _p [l] and the accent command u _a [l] at each time l which is a non-negative value and the base component u _b are updated according to the above equation (72).

In the next step S204, it is determined whether or not the number of repetitions s has reached S as the convergence condition. If the number of repetitions s has not reached S, it is determined that the convergence condition is not satisfied. Then, the process returns to step S202. On the other hand, when the number of repetitions s reaches S, it is determined that the convergence condition is satisfied, and in step S205, the phrase command u _p [k] and the accent command u _{a at} each time k updated in step S203 above. Based on [k] and the posterior probability P (s _k = t | y, o ′, θ ′) updated in step S201, the above equation (73), equation (74), or equation (75) In accordance with the equation (76), the amplitude A ^(p) [k] of the phrase command at each time k and the amplitude A _a ^(a) of the accent command at each position n are updated to correspond to the state at each time. The parameter θ representing the amplitude of the phrase command and the amplitude of each accent command is updated.

  In step S206, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S201. On the other hand, if the convergence determination condition is satisfied, in step S207, the state sequence s is estimated by the Viterbi algorithm based on the command sequence o finally updated in step S203. Further, the state transition probability φ is obtained from the estimated state sequence s.

  In step S109, the spectral feature vector c [k] at each time k extracted in step S101, the combined vector q [k] of the fundamental frequency at each time k extracted in step S102, and the above Based on the parameter θ finally obtained in step S205, the state transition probability φ obtained in step S207, and the parameter γ updated last time in step S110 described later, the above equations (100), ( 101), an expected value is calculated.

  In step S110, the parameter γ is updated based on the expected value calculated in step S109 so that the equation (102) becomes maximum.

  In step S112, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S109. On the other hand, when the convergence determination condition is satisfied, in step S113, the parameter θ finally obtained in step S205, the state transition probability φ obtained in step S207, and finally in step S110. The obtained parameter γ is stored in the parameter storage unit 40.

  Next, when time-series data of source speech to be predicted is input to the fundamental frequency pattern prediction apparatus 100, the fundamental frequency pattern prediction apparatus 100 executes the fundamental frequency pattern prediction processing routine shown in FIG. The output unit 90 outputs the prediction result of the fundamental frequency at each time of the target sound corresponding to the target source sound.

As described above, according to the fundamental frequency pattern predicting apparatus according to the second embodiment, the parameters θ and φ of the second probability distribution modeling the fundamental frequency pattern generation process are learned, and each time of the source speech The parameter P of the first probability distribution that models the relationship between the spectral feature vector of the target speech and the fundamental frequency at each time of the target speech is expressed as a criterion P expressed using the first probability distribution and the second probability distribution. Learning to increase (c, y, m, s | γ, θ, φ), and from the spectral feature quantity vector at each time extracted from the time series data of the source speech to be predicted, the first probability distribution and The basic frequency at each time of the target speech corresponding to the source speech to be predicted so as to increase the criterion P (c, y, m, s | γ, θ, φ) expressed using the second probability distribution by predicting the physical of F ₀ pattern It is possible to estimate the optimal F ₀ pattern corresponding to the spectral feature amount sequence taking into account the constraints of the production process

[Third Embodiment]
Next, a fundamental frequency pattern prediction apparatus according to the third embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected about the part which becomes the same structure as 1st Embodiment, and description is abbreviate | omitted.

  In the third embodiment, the method for estimating the parameters γ, θ, and φ is different from that in the first embodiment.

In the third embodiment, P (c, y, m, s | γ, θ, φ) is as large as possible under the F ₀ pattern {y [k]} ^K _{k = 1} of the learning sample. Thus, the parameters γ, θ, and φ are learned simultaneously.

  As shown in FIG. 11, the learning unit 30 of the fundamental frequency pattern prediction apparatus according to the third embodiment includes a feature amount extraction unit 32, a fundamental frequency sequence extraction unit 34, and a model parameter learning unit 336. Yes.

  The model parameter learning unit 336 combines the spectral feature vector c [k] at each time k extracted by the feature extraction unit 32 and the fundamental frequency combination vector q at each time k extracted by the fundamental frequency sequence extraction unit 34. Based on [k], the spectral feature vector c [k of the source speech at each time so that the combined distribution P (c, y, m, s | γ, θ, φ) of c and y is as large as possible. ] And the parameter γ of the first probability distribution, which is a mixed normal distribution representing the joint probability distribution of the fundamental frequency combined vector q [k] at each time k of the target speech, and the fundamental frequency y [k] at each time k Then, the parameters θ and φ of the second probability distribution, which is a probability distribution of a combination with the state series s consisting of the state at each time of the hidden Markov model, are learned.

  Specifically, the spectral feature quantity vector c [k] at each time k extracted by the feature quantity extraction unit 32 and the combined vector q [k] of the fundamental frequency at each time k extracted by the fundamental frequency series extraction unit 34. The first probability to increase logP (c, y, m, s | γ, θ, φ) by repeating the E step and the M step using an EM (Expectation-Maximization) algorithm. The GMM parameter γ of the distribution and the parameters θ and φ of the second probability distribution are learned.

  Similarly to the first embodiment, the conversion processing unit 50 receives the time series data of the source speech to be predicted as an input, the spectral feature quantity vector at each time extracted from the time series data of the source speech, and the model parameter Based on the first probability distribution parameter γ and the second probability distribution parameters θ and φ learned by the learning unit 336, learning is performed by repeating the E step and the M step by an EM (Expectation-Maximization) algorithm. Probability of generating the fundamental frequency y at each time and the spectral feature vector at each time k so as to increase P (c, y, m, s | γ, θ, φ), which is a standard common to the processing Is a normal frequency index m having the highest frequency, and a state sequence s composed of states at each time to estimate the fundamental frequency at each time of the target speech corresponding to the source speech to be predicted To predict.

<Operation of fundamental frequency pattern prediction device>
Next, the operation of the fundamental frequency pattern prediction apparatus according to the third embodiment will be described. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  First, when parallel data including time series data of source speech and target speech of a learning sample is input to the fundamental frequency pattern prediction device, the learning process routine shown in FIG. Executed.

  First, in step S101, input time series data of source speech is read, and a spectral feature vector c [k] at each time k is extracted. In step S102, the time-series data of the input target speech is read, the fundamental frequency y [k] at each time k of the target speech is extracted, and the combined vector q of the fundamental frequency y [k] and its dynamic component is extracted. Extract [k].

  In step S300, the state series s, the parameter φ obtained from the state series s, the parameter θ representing the amplitude of the phrase command and the amplitude of each accent command according to the state at each time, and the parameter γ are initialized.

  In step S301, the spectral feature vector c [k] at each time k extracted in step S101, the basic frequency y [k] at each time k extracted in step S102, and the basic at each time k. The combined vector q [k] of the fundamental frequency at each time k obtained from the frequency y [k], the parameter θ, the state transition probability φ, and the parameter γ that are initialized or updated last time in step S302 described later, Based on the above, the expected value is calculated according to the above formulas (100) and (101).

  In step S302, based on the expected value calculated in step S301, the parameter θ, the state sequence s, the state transition probability φ obtained from the state sequence s, and the parameter γ are set so that the equation (102) becomes maximum. Update.

  In step S303, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S301. On the other hand, when the convergence determination condition is satisfied, in step S113, the parameter θ finally obtained in step S302, the state transition probability φ obtained from the state sequence s, and the parameter γ are set to the parameter storage unit. 40.

  Next, when time-series data of source speech to be predicted is input to the fundamental frequency pattern prediction apparatus 100, the fundamental frequency pattern prediction apparatus 100 executes the fundamental frequency pattern prediction processing routine shown in FIG. The output unit 90 outputs the prediction result of the fundamental frequency at each time of the target sound corresponding to the target source sound.

As described above, according to the fundamental frequency pattern predicting apparatus according to the third embodiment, the relationship between the spectrum feature vector at each time of the source speech and the fundamental frequency at each time of the target speech is modeled. The first probability distribution parameter γ and the second probability distribution parameters θ and φ modeling the fundamental frequency pattern generation process are expressed by a criterion P (c , y, m, s | γ, θ, φ), the first probability distribution and the second probability distribution are obtained from the spectral feature quantity vector at each time extracted from the time series data of the source speech to be predicted. The basic frequency at each time of the target speech corresponding to the source speech to be predicted is predicted so as to increase the criterion P (c, y, m, s | γ, θ, φ) expressed using the probability distribution. by physical raw F ₀ pattern It is possible to estimate the optimal F ₀ pattern corresponding to the spectral feature amount sequence taking into account the constraints of the process

<Experiment>
Spectral feature series, F ₀ pattern, and phrase / accent command are extracted from the speech signal, and the above model parameters (GMM parameters) are learned by learning using pair data of the spectral feature series and phrase / accent command series. and the after, and conducted an experiment to convert the spectral feature amount sequence phrase ascent command sequence by the conversion process, the converted phrase ascent command sequence to confirm whether the possible extent restore the original F ₀ pattern. FIG. 13 shows an example of the result. The dotted line is the F ₀ pattern estimated from the speech signal, and the broken line is the F ₀ pattern obtained from the phrase ascent command sequence converted from the spectral feature amount sequence. It was confirmed that the original F ₀ pattern was reconstructed in spite of the fact that the spectral feature quantity did not contain much F ₀ information.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, although the basic frequency pattern prediction apparatus described above has a computer system inside, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. Shall be.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Calculation part 30 Learning part 32 Feature-value extraction part 34 Fundamental frequency series extraction part 36 1st model parameter learning part 38 2nd model parameter learning part 40 Parameter storage part 50 Conversion process part 52 Feature-value extraction part 54 Expected value Calculation unit 56 Variable update unit 58 Convergence determination unit 90 Output unit 100 Fundamental frequency pattern prediction device 336 Model parameter learning unit

Claims (8)

Using parallel data composed of time-series data of source speech and target speech of the learning sample as input, a spectral feature vector at each time extracted from the time-series data of the source speech, and the time of the target speech Based on the fundamental frequency at each time extracted from the sequence data, the first modeled the relationship between the spectral feature quantity vector at each time of the source speech and the fundamental frequency at each time of the target speech. Learning for learning a parameter of a probability distribution and a parameter of a second probability distribution obtained by modeling a fundamental frequency pattern generation process so as to increase a criterion expressed using the first probability distribution and the second probability distribution. And
Using the time series data of the source speech to be predicted as an input, the spectral feature quantity vector at each time extracted from the time series data of the source speech to be predicted, and the parameters of the first probability distribution learned by the learning unit And a conversion processing unit that predicts a fundamental frequency at each time of the target speech corresponding to the source speech to be predicted so as to increase the criterion based on the parameters of the second probability distribution,
A fundamental frequency pattern prediction apparatus including: The first probability distribution is a mixed normal distribution representing a joint probability distribution of a spectral feature vector at each time of the source speech, a fundamental frequency at each time of the target speech, and a dynamic component of the fundamental frequency,
The second probability distribution is
The fundamental frequency of each time,
From a series of states consisting of states at each time of the Hidden Markov Model, or a pair of phrase commands representing the fundamental frequency pattern generated by the translational movement of the thyroid cartilage at each time and an accent command representing the fundamental frequency pattern generated by the rotational movement of the thyroid cartilage The fundamental frequency pattern prediction apparatus according to claim 1, wherein the probability distribution is a combination with a command function. The fundamental frequency pattern prediction apparatus according to claim 2, wherein the criterion is represented by the following expression.
Where c is a spectral feature vector at each time of the source speech, y is a fundamental frequency at each time of the target speech, and m is a probability that a spectral feature vector at each time has been generated. The index of the highest normal distribution, s is the state series, γ is a parameter of the first probability distribution, and θ is the amplitude of the phrase command and the accent command according to the state at each time. Is a parameter representing the amplitude, φ is the state transition probability, q is the combined vector of the fundamental frequency and dynamic component at each time of the target speech, u is the phrase command and accent command at each time is there. The learning unit increases the likelihood of the fundamental frequency at each time and the state sequence including the state at each time of the hidden Markov model, obtained from the second probability distribution by an EM (Expectation-Maximization) algorithm. In addition, as a parameter of the second probability distribution, a parameter indicating the state transition probability of the state series, the phrase command amplitude according to the state at each time, and the amplitude of each accent command is learned, or the basic of each time When the frequency is given, the second probability is increased so that the objective function is increased with the logarithmic posterior probability of the command function including the phrase command and the accent command at each time and the parameter representing the amplitude as the objective function. As a distribution parameter, a state transition probability of the state series and a parameter representing the amplitude are set as the second probability component. The fundamental frequency pattern prediction apparatus according to claim 1, wherein learning is performed as a cloth parameter.   The learning unit fixes the learned second probability distribution parameter, and learns the first probability distribution parameter by an EM (Expectation-Maximization) algorithm so that the criterion is increased. 4. The fundamental frequency pattern prediction apparatus according to 4. The conversion processing unit
Based on the spectral feature vector at each time extracted from the time-series data of the source speech to be predicted, the parameters of the first probability distribution and the parameters of the second probability distribution learned by the learning unit, A normal distribution index with the highest probability of generating a fundamental frequency at each time of the target speech corresponding to the source speech to be predicted and a spectral feature vector at each time so as to increase the criterion by the EM algorithm The basic frequency pattern prediction apparatus according to claim 2, wherein the fundamental frequency at each time of the target speech corresponding to the prediction target source speech is predicted by estimating the state sequence. A fundamental frequency pattern prediction method in a fundamental frequency pattern prediction apparatus including a learning unit and a conversion processing unit,
The learning unit receives parallel data consisting of time series data of source speech and target speech time series data of a learning sample as input, and a spectral feature vector at each time extracted from the time series data of the source speech, Based on the fundamental frequency at each time extracted from the time series data of the target speech, the relationship between the spectral feature quantity vector at each time of the source speech and the fundamental frequency at each time of the target speech is The criterion expressed by using the first probability distribution and the second probability distribution is increased for the modeled first probability distribution parameter and the second probability distribution parameter modeling the fundamental frequency pattern generation process. To learn and
The conversion processing unit receives time series data of the source speech to be predicted as an input, a spectral feature quantity vector at each time extracted from the time series data of the source speech to be predicted, and the learning unit Based on the parameter of the first probability distribution and the parameter of the second probability distribution, the fundamental frequency for predicting the fundamental frequency at each time of the target speech corresponding to the prediction target source speech so as to increase the criterion Pattern prediction method.   The program for functioning a computer as each part of the fundamental frequency pattern prediction apparatus of any one of Claims 1-6.