JP6468518B2 - Basic frequency pattern prediction apparatus, method, and program - Google Patents

Description

  The present invention relates to a fundamental frequency pattern prediction apparatus, method, and program, and more particularly, to a fundamental frequency pattern prediction apparatus, method, and program for predicting a fundamental frequency pattern of a target voice from a source voice.

  In communication with others, voice is a convenient means, but sometimes physical barriers inevitably cause various barriers. For example, if even one of the vocal organs does not operate normally, it suffers from serious vocal disturbances and hinders voice communication. Moreover, the physical action of voice generation is not suitable for highly confidential communication and is vulnerable to ambient noise. In order to eliminate these barriers, voices can be generated by moving the vocal organs beyond physical constraints, voices can be generated by specifying an appropriate pronunciation, or when voices are so fine that it is difficult to hear It is necessary to expand the speech generation function beyond physical and physical constraints, such as generating normal speech from the vocal organ movements.

  For example, for laryngectomy patients who have lost their larynx due to laryngeal cancer, etc., voice assist technology has been proposed to convert less natural speech generated by alternative vocalization methods using residual organs into more natural speech. (See Non-Patent Documents 1 to 3).

In addition to this, a technique for converting a non-audible murmur voice to a natural voice has been proposed, and application as a call technique with excellent secrecy is expected (see Non-Patent Document 4). All of the above-described techniques are common in that they deal with the problem of predicting the fundamental frequency (F ₀ ) pattern of natural speech from the spectral feature quantity sequence of speech, and are composed of learning processing and conversion processing. In the learning process, the same speech data of the target voice (electric voice in the former case, non-audible murmur voice in the latter case) and normal voice is used. First, at each discrete time (hereinafter referred to as a frame), a spectral feature quantity of the target speech obtained from several frames before and after, a logarithm F _{0 of the} normal speech and its dynamic component (time differential or time difference) are extracted, and a spectral distance measure A combined vector corresponding to these is obtained by dynamic time expansion and contraction based on. This is called parallel data. Using the parallel data of each frame, the combined probability density function of the static and dynamic components of the target speech's spectral features and the logarithm F ₀ of the normal speech is a mixed normal distribution model (hereinafter referred to as GMM). Express. GMM parameters can be learned by an Expectation-Maximization algorithm. In the conversion process, using the learned GMM, the spectral feature quantity sequence of the target speech can be converted into the F ₀ pattern of the normal speech by the maximum likelihood sequence conversion method considering intra-sequence variation.

Keigo Nakamura, Tomoki Toda, Hiroshi Saruwatari, Kiyohiro Shikano, "Speaking-aid systems using1 GMM-based voice conversion for electrolaryngeal speech," Speech Communication, vol. 54, no. 1, pp. 134 {146, 2012. Kou Tanaka, Tomoki Toda, Graham Neubig, Sakriani Sakti, Satoshi Nakamura, "A hybrid approach to electrolaryngeal speech enhancement based on noise reduction and statistical excitation generation," IEICE Transactions on Information and Systems, vol.E97-D, no. 6, pp. 1429-1437, Jun. 2014. Kou Tanaka, Tomoki Toda, Graham Neubig, Sakriani Sakti, Satoshi Nakamura, "Direct F0 control of an electrolarynx based on statistical excitation feature prediction and its evaluation through simulation," Proc. INTERSPEECH, pp. 31 {35, Sep. 2014. Tomoki Toda, Kiyohiro Shikano, "NAM-to-speech conversion with Gaussian mixture models," Proc. INTERSPEECH, pp. 244-247, 2005.

In the prior art, since the model considering the physical process of generating speech F ₀ pattern in the learning process and the conversion process has not been used, physically human unnatural F ₀ pattern as not be uttered It could happen. To solve this problem, by performing prediction in consideration of the physical process of generating F ₀ pattern, it may be possible to generate a more natural F ₀ pattern.

The F ₀ pattern is generated by the movement of the thyroid cartilage that gives tension to the vocal cords. In Non-Patent Document 5, a parameter related to the movement of the thyroid cartilage called a phrase / accent command is estimated based on the probability model of the control mechanism. Technology has been proposed. In this technology, one of the points is that the time series generation process of the phrase / accent command is expressed by a hidden Markov model (hereinafter referred to as HMM). Linguistic or a priori knowledge about the sequence can be incorporated into the parameter estimation.

[Non-Patent Document 5]: Hirokazu Kameoka, Jonathan Le Roux, Yasunori Ohishi, "A statistical model of speech F ₀ contours," ISCA Tutorial and Research Workshop on Statistical And Perceptual Audition (SAPA2010), pp. 43-48, Sep. 2010.

The present invention has been made to solve the above problems, by estimating the optimal F ₀ pattern corresponding to the spectral feature amount sequence taking into account the constraints of the physical process of generating F ₀ pattern An object of the present invention is to provide a fundamental frequency pattern prediction apparatus, method, and program capable of performing the above.

  In order to achieve the above object, a fundamental frequency pattern predicting apparatus according to a first aspect of the present invention uses a source speech time-series data as an input, and obtains a fundamental frequency pattern generated by translational movement of thyroid cartilage at each time of a target speech. A basic frequency pattern predicting device for predicting a command sequence consisting of a pair of a phrase command to represent and an accent command representing a basic frequency pattern generated by the rotational motion of thyroid cartilage, wherein the time series data of the source speech of the learning sample and the time of the target speech With parallel data consisting of series data as input, the spectral feature vector at each time extracted from the time series data of the source speech and the time of the hidden Markov model estimated from the time series data of the target speech , Indicates a pair of the phrase command and the accent command A learning unit that learns parameters of a simultaneous generation model of a combination of a spectral feature vector at each time of the source speech and the state sequence of a target speech, based on a state sequence composed of state indices; Based on the time series data of the source speech as input, based on the spectral feature vector at each time extracted from the time series data of the source speech to be predicted, and the parameters of the simultaneously generated model learned by the learning unit And a conversion processing unit that predicts the command sequence of the target speech corresponding to the source speech to be predicted.

Further, the fundamental frequency pattern predicting device according to the first invention, the hidden Markov model includes a plurality of states p _m that the phrase command is to occur, a plurality of states a _n that the accent command is occurring, the phrase command and and a state r _0, r ₁ none of the accent command not causing transitions in transition from the state r ₀ to one of the plurality of states p _m in the state r _1, the state r it may be each state is connected to ₁ from the transition to one of the plurality of states a _n transitions to the state r _0.

  In the fundamental frequency pattern prediction apparatus according to the first invention, the learning unit may learn the parameter λ of the simultaneously generated model so as to increase an objective function expressed by the following equation.

Where c is a spectral feature vector c [k] at each time k of the source speech, and s is the state series consisting of states s _{k at} each time k,

Is a parameter of the state output distribution P (c [k] | s _k ) of the state s _k , φ _{i, i ′} is the transition probability between the states i, _{i ′} of the hidden Markov model, and φ _i is , The probability that the initial state is state i.

  In the fundamental frequency pattern prediction apparatus according to the first aspect of the present invention, the conversion processing unit estimates the state sequence so as to increase the objective function using a Viterbi algorithm, and calculates the state sequence from the estimated state sequence. The command sequence of the target speech corresponding to the source speech to be predicted may be predicted.

  The basic frequency pattern prediction method according to the second invention is the phrase command representing the basic frequency pattern generated by the translational movement of the thyroid cartilage at each time of the target speech and the rotation of the thyroid cartilage with the time series data of the source speech as input. A basic frequency pattern prediction method in a basic frequency pattern prediction apparatus for predicting a command sequence consisting of a pair of accent commands representing a basic frequency pattern generated by exercise, wherein a learning unit uses time-series data of a source sound of a learning sample and a target sound Each of the hidden Markov models estimated from the spectral feature vector at each time extracted from the time series data of the source speech and the time series data of the target speech. The phrase command and the accent of time Learning a parameter of a simultaneous generation model of a combination of a spectrum feature vector at each time of the source speech and the state sequence of a target speech based on a state sequence consisting of a state index indicating a pair of commands; The conversion processing unit receives the time-series data of the source speech to be predicted as an input, the spectral feature quantity vector at each time extracted from the time-series data of the source speech to be predicted, and the learning learned by the learning unit And executing the step of predicting the command sequence of the target speech corresponding to the source speech to be predicted based on a parameter of a simultaneously generated model.

Further, the fundamental frequency pattern prediction method according to the second invention, the hidden Markov model includes a plurality of states p _m that the phrase command is to occur, a plurality of states a _n that the accent command is occurring, the phrase command and and a state r _0, r ₁ none of the accent command not causing transitions in transition from the state r ₀ to one of the plurality of states p _m in the state r _1, the state r it may be each state is connected to ₁ from the transition to one of the plurality of states a _n transitions to the state r _0.

  In the fundamental frequency pattern prediction method according to the second invention, the learning unit may learn the parameter λ of the simultaneous generation model so as to increase an objective function expressed by the following equation.

Where c is a spectral feature vector c [k] at each time k of the source speech, and s is the state series consisting of states s _{k at} each time k,

Is a parameter of the state output distribution P (c [k] | s _k ) of the state s _k , φ _{i, i ′} is the transition probability between the states i, _{i ′} of the hidden Markov model, and φ _i is , The probability that the initial state is state i.

  A program according to the third invention is a program for causing a computer to function as each part of the fundamental frequency pattern prediction apparatus according to the first invention.

According to the fundamental frequency pattern prediction apparatus, method, and program of the present invention, the spectral feature vector at each time extracted from the time series data of the source speech, the phrase command and the accent command at each time of the hidden Markov model Based on a state sequence consisting of a state index indicating a pair, the parameters of the simultaneous generation model of the combination of the spectral feature vector at each time of the source speech and the state sequence of the target speech are learned, and the source to be predicted Based on the spectral feature vector at each time extracted from the time series data of the speech and the parameters of the learned simultaneous generation model, by predicting the target speech command sequence corresponding to the source speech to be predicted, F ₀ pattern spectrum feature amount sequence taking into account the constraints of the physical process of generating It is possible to estimate the optimum F ₀ pattern corresponding to the above.

It is a figure which shows an example of a structure of HMM. It is a figure which shows an example at the time of dividing _| segmenting state an. FIG. 10 is a diagram illustrating an example of state transition when the state p ₀ in the state set is increased in the configuration of the HMM. It is a block diagram which shows the structure of the fundamental frequency pattern prediction apparatus which concerns on embodiment of this invention. 3 is a block diagram illustrating a configuration of a learning unit 30. FIG. 3 is a block diagram illustrating a configuration of a conversion processing unit 50. FIG. It is a flowchart which shows the learning process routine in the fundamental frequency pattern prediction apparatus which concerns on embodiment of this invention. It is a flowchart which shows the fundamental frequency pattern prediction process routine in the fundamental frequency pattern prediction apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the effect of the experiment of embodiment of this invention. It is a figure which shows an example of HMM which connected the state of the start point and end point of several Left-to-Right type HMM.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The technology proposed in the embodiments of the present invention belongs to the technical field of signal processing, and predicts a fundamental frequency pattern from a feature sequence of speech and replaces the fundamental frequency pattern of the original speech with the predicted fundamental frequency pattern. This is a speech processing technology that aims to improve the naturalness of speech.

  Here, related techniques 1 to 3 in the embodiment of the present invention will be described.

<Related technology 1: F ₀ pattern prediction method from speech feature sequence>

First, a description will be given F ₀ pattern prediction method from the audio feature amount sequence.

In Non-Patent Document 1 to Non-Patent Document 3, a method for predicting an F ₀ pattern from a speech feature amount sequence is proposed. The method includes processing for learning parameters of a speech feature quantity sequence and a F ₀ pattern simultaneous probability distribution model, and processing for converting a given speech feature quantity sequence into a F ₀ pattern using the learned model.

<Learning process>

It is assumed that parallel data of learning processing source speech (for example, electrical speech) and target speech (for example, natural speech) is given. The feature vector of the source speech is c [k], and the combined vector of the logarithm F _{0 of the} target speech and its dynamic component (time derivative or time difference) is q [k] = (y [k], Δy [k]. ) ^T. Here, k is an index of discrete time. As the speech feature value c [k], for example, a vector obtained by dimensional compression by principal component analysis is used for a vector obtained by connecting a series of mel cepstrum (vector) for several frames around the time k. In this method, the joint probability distribution of c [k] and q [k] is modeled by a mixed normal distribution model (Gaussian Mixture Model; GMM), and given parallel data {c [k], q [k]} _k in the learning process. _{= 1} The parameter of the GMM is learned from ^K. The parameters of the GMM can be estimated by an Expectation-Maximization (EM) algorithm.

<Conversion processing>

In the conversion process, y = (y, which is the maximum likelihood F ₀ pattern under a given speech feature quantity sequence c = (c [1] ^T ,..., C [K] ^T ) ^{T in} the conversion process. [1] ^T , ..., y [K] ^T ) ^T

Calculated by Here, q = (q [1] ^T ,..., Q [K] ^T ) ^T , and W is a transformation matrix representing the relationship between y and q.

<Related technology 2: _{F 0} pattern generation process model>

Next, a description will be given F ₀ pattern generation process model.

Fujisaki's fundamental frequency (F ₀ ) pattern generation process model (Fujisaki model) is known as a model that describes the generation process of speech F ₀ patterns (see Non-Patent Document 6).

[Non-Patent Document 6]: H. Fujisaki, “In Vocal Physiology: Voice Production, Mechanisms and Functions,” Raven Press, 1988.

The Fujisaki model is a physical model describing the process of generating F ₀ pattern due to the motion of the thyroid cartilage. The Fujisaki model, the total elongation of the vocal cords with each two independent movement of the thyroid cartilage (translational motion and rotational motion) is interpreted to result temporal variation of F _0, pairs of elongation and F ₀ pattern of the vocal cords The F ₀ pattern is modeled based on the assumption that the numerical value y (t) is proportional. The F ₀ pattern x _p (t) generated by the translational movement of the thyroid cartilage is called a phrase component, and the F ₀ pattern x _a (t) generated by the rotational movement is called an accent component. In the Fujisaki model, the speech F ₀ pattern y (t) is the sum of these components plus the baseline component b determined by the physical constraints of the vocal cords.

It is expressed. These two components are assumed to be the output of the second-order critical braking system,

(* Is a convolution operation with respect to time t). Here, u _p (t) is called a phrase command function and consists of a sequence of delta functions (phrase commands), and u _a (t) is called an accent command function and consists of a sequence of rectangular waves (accent commands). These command sequences have a constraint condition that a phrase command occurs at the beginning of an utterance, phrase commands do not occur twice in succession, and two different commands do not occur at the same time. Further, α and β are natural angular frequencies of the phrase control mechanism and the accent control mechanism, respectively, and it is empirically known that α is about 3 rad / s and β is about 20 rad / s regardless of the speaker and the content of the utterance. It has been.

<Related technology 3: F ₀ pattern generation process model parameter estimation method>

Next, the F ₀ pattern generation process model parameter estimation method will be described.

  The above-mentioned Fujisaki model can be described by the following probability model (see Non-Patent Document 5 and Non-Patent Document 7).

[Non-Patent Document 7]: Kota Yoshizato, Hirokazu Kameoka, Daisuke Saito, Shigeki Sagayama, "Hidden Markov convolutive mixture model for pitch contour analysis of speech," in Proc. The 13th Annual Conference of the International Speech Communication Association (Interspeech 2012), Sep. 2012.

First pair _o of phrase accent command function [k] = (u p [ k], u a [k]) consider the HMM to output the ^T. Here, k represents an index of discrete time. The state output distribution is a normal distribution.

It is assumed that a command function pair o [k] is generated by Where {s _k } _{k = 1} ^K is the state sequence of the HMM, and the average vector

Is a value determined as a result of state transition of the HMM. A specific configuration of the HMM is shown in FIG.

The duration of self-transition can be parameterized by dividing each state into several small states. At this time, all the small states have the same output distribution, and the number of small states is set to a sufficiently large value. It shows an example of dividing the state a _n in FIG. For example, as shown in FIG. 2, by setting the state transition probability from an _{, m} to an _{, m + 1} to ₁ for all m ≠ 0, the transition probability from an _{, 0} to an _{, m} is It corresponds to the probability that state a _n lasts only m step, it becomes possible to flexibly control the persistence length of the accent command. Similarly, by dividing p, r ₀ and r ₁ into small states, it becomes possible to parameterize the distribution of the duration of the phrase command and the length of the interval between commands. Based on this division, after that,

Is written. The configuration of the above HMM can be written as follows.

When a state sequence s = {s _k } ^K _{k = 1} is given, the HMM outputs a pair of _a phrase command function u _p [k] and an accent command function u _a [k]. Equation (3) and (5) as shown _{in, u} p [k] and _u a [k], respectively _g p [k] and _g a [k] is convolved phrase component _x p [k] And the accent component x _a [k] are output. This can be expressed as an expression:

(* Is a convolution operation related to the discrete time k). At this time, the F ₀ pattern x [k] is

And can be written by superimposing three kinds of components. However, b is a baseline component which does not depend on time.

In real speech, a reliable F ₀ value is not always observable. In estimating the parameters of the Fujisaki model, it is desirable to take into account only the F ₀ value of the reliable observation interval and ignore the other intervals. For example, since the periodic coarse / dense wave associated with the normal vocal cord vibration is not observed in the voiceless section, even if an estimated value of F ₀ is obtained from the voiceless section by automatic pitch extraction, the value is It is not appropriate to consider the value of F ₀ of a signal emitted from a vocal cord. Therefore, the degree of uncertainty v _n ² [k] at time k of the observed F ₀ value is introduced into the proposed model. Specifically, the observed F ₀ value y [k], the true F ₀ value x [k], and the noise component

With overlay

This means that all observation intervals can be handled uniformly regardless of whether they are reliable intervals. Also,

Is regarded as a constant,

Is assumed to be uniformly distributed. Then, the probability density function of y = {y [k]} _{k = 1} ^K when the output value series o = {o [k]} _{k = 1} ^K is given by peripheralizing x _n [k].

Is obtained. State sequence s = {s [k]} _{k = 1} ^K and a parameter representing the amplitude of the command

Is given, the output value sequence o is

Is generated according to P (s) is a product of state transition probabilities.

Can be written. However, φs ₁ represents the probability that the initial state is s ₁ .

<Outline of Embodiment of the Present Invention>

The technology according to the embodiment of the present invention combines the probability models described in Non-Patent Document 5 and Non-Patent Document 7 and the probability models described in Non-Patent Documents 1 to 3, and a phrase accent from a speech spectrum feature quantity sequence. by predicting the instruction is a technique that makes it possible to estimate the optimal F ₀ pattern corresponding to the spectral feature amount sequence while satisfying the constraints of physical generation process of F ₀ pattern.

<Principle according to the embodiment of the present invention>

  In order to learn model parameters, first, the state transition network of the HMM in the related technique 3 described above is modified.

P 0, the state p 0 in the state set as in FIG. . . , P M−1 so that the average of each state output distribution does not depend on time k. Further, similarly to FIG. 2, p m is also divided into small state in the state output distributions, p m = {p m, 0, p m, 1. . . ,}. Thus, the p m of different m, different intensity values A (m) p correspond. Here, unlike the related art 3, the average A (m) p of the power distribution in the state p m to note that has a variable that is independent of k. r 1 = {r i, j | i ≧ 1, j ≧ 0}, r i, j = {r i, j, l | l ≧ 0}, a i = {a i, j | j ≧ 0}, By writing as a i, j = {a i, j, l | l ≧ 0}, the above example can be described as an HMM having the following configuration.

Next, using the related technique 3 in which the HMM is replaced with the HMM having the above configuration, the phrase / accent command (state series s) is estimated in advance for the target speech data. Also, let the feature vector of the source audio be c [k]. Here, k is an index of discrete time. For example, as in Non-Patent Documents 1 to 3, c [k] is obtained by performing dimension compression by principal component analysis on a vector obtained by concatenating a series of mel cepstrum (vector) for several frames around the time k. Used as As described above, the pair data of the feature vector of the source speech {c [k], s _k } _{k = 1} ^K and the state series of the phrase / accent command are obtained.

The simultaneous generation model of this pair of data {c [k], s _k } _{k = 1} ^K is described by an HMM in which the output symbol is c [k] instead of o [k] in the above HMM (FIG. 3). To do. Further, the state output distribution is modified to a mixed normal distribution (GaMMian Mixture Model; GMM). In the learning process, the GMM parameters (mixing weight, average of each normal distribution and variance-covariance matrix) are parameters to be learned. That is, logP (c, s | λ) is

Given in. However, λ _i = {α _{m, i} , μ _{m, i} , Σ _{m, i} | m = 1,. . , M} represent output distribution parameters (GMM mixing weights, average of each normal distribution and variance-covariance matrix), and λ = {λ _i } ^I _{i = 1} is a parameter to be estimated in the learning process. . φ _{i, i ′} represents the probability of transition from state i to state _{i ′ and} is a constant.

  In the learning process, a given spectrum feature quantity sequence c = (c [1],..., C [K]) and a state sequence s = (s [1],. Λ is estimated so that logP (c, s | λ) has a local maximum. This can be realized by an Expectation-Maximization algorithm.

  The conversion process from the feature amount sequence of the source speech to the phrase / accent command is performed under the given spectrum feature amount sequence c = (c [1],. Therefore, s = (s [1],... S [K]) is obtained by a normal Viterbi algorithm, and the average sequence at that time is set as a phrase / accent command sequence.

<System configuration>

  Next, an embodiment of the present invention will be described by taking as an example a case where the present invention is applied to a fundamental frequency pattern prediction apparatus that predicts a fundamental frequency pattern of a target speech from a spectral feature quantity sequence of a source speech.

  As shown in FIG. 4, the fundamental frequency pattern prediction apparatus according to the embodiment of the present invention stores a CPU, a RAM, a learning processing routine described later, and a program for executing a fundamental frequency pattern prediction processing routine. It is comprised by the computer provided with ROM, and is comprised as shown below functionally.

  As shown in FIG. 4, the fundamental frequency pattern prediction apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 90.

  The input unit 10 accepts parallel data composed of time-series data of the source sound (for example, electric sound) of the learning sample and time-series data of the target sound (for example, natural sound). The input unit 10 accepts input of parallel data composed of time-series data of the source sound (for example, electric sound) of the learning sample and time-series data of the target sound (for example, natural sound). In addition, the input unit 10 receives time-series data of the source audio to be predicted.

  The calculation unit 20 includes a learning unit 30, a parameter storage unit 40, and a conversion processing unit 50.

  The learning unit 30 extracts the spectral feature quantity at each time extracted from the time series data of the source voice from the parallel data composed of the time series data of the source voice and the target voice of the learning sample received by the input unit 10. Spectral feature quantity of each time of source speech based on vector and state sequence consisting of state index indicating phrase command and accent command pair at each time of HMM estimated from time series data of target speech The parameter of the simultaneous generation model of the combination of the vector and the state sequence of the target speech is learned.

  As illustrated in FIG. 5, the learning unit 30 includes a feature amount extraction unit 32, a fundamental frequency sequence extraction unit 34, a state sequence estimation unit 36, and a model parameter learning unit 38.

  The feature amount extraction unit 32 extracts a source speech spectrogram feature amount vector c [k] from the time series data of the source speech of the learning sample received by the input unit 10. Here, k is an index of discrete time. For example, as in Non-Patent Documents 1 to 3, c [k] is obtained by performing dimension compression by principal component analysis on a vector obtained by concatenating a series of mel cepstrum (vector) for several frames around the time k. Used as

The fundamental frequency series extraction unit 34 extracts the fundamental frequency y [k] at each time k of the target speech from the time series data of the target speech of the learning sample received by the input unit 10, and y = (y [1], ..., y [K]) ^T.

  This fundamental frequency extraction process can be realized by a well-known technique. For example, Non-Patent Document 8 (H. Kameoka, “Statistical speech spectrum model incorporating all-pole vocal tract model and F0 contour generating process model,” in Tech. Rep. IEICE, 2010, in Japanese.), The fundamental frequency is extracted every 8 ms.

Further, a combined vector (referred to as F ₀ feature amount) of y and its dynamic component (time differential or time difference) is set to q [k] = (y [k], Δy [k]) ^T.

Based on the fundamental frequency y [k] extracted by the fundamental frequency sequence extraction unit 34, the state sequence estimation unit 36 estimates the state sequence s composed of the states s _{k at} each time k in the HMM.

Here HMM, as shown in principle and Figure 3 according to an embodiment of the present invention described above, a plurality of states p _m where phrase command is to occur, a plurality of states a _n accent command occurring phrases both the command and the accent command and a state r _0, r ₁ does not occur, a transition to state r ₁ transitions from state r ₀ to one of a plurality of states p _m, from the state r ₁ more each state is connected so as to transition to state r ₀ transitions to either state a _n.

From the above, data of {c [k], s _k } _{k = 1} ^K is obtained.

  The model parameter learning unit 38 uses the existing EM algorithm based on the spectral feature vector c [k] at each time k extracted by the feature extraction unit 32 and the state sequence s estimated by the state sequence estimation unit 36. Accordingly, the parameter λ of the simultaneous generation model is learned so as to increase the objective function represented by logP (c, s) in the above equation (16).

  The conversion processing unit 50 receives time-series data of the prediction target source speech from the input unit 10, and learns the spectral feature quantity vector at each time extracted from the time-series data of the prediction target source speech and the learning unit 30. Based on the parameter λ of the generated simultaneous model, a command sequence of the target speech corresponding to the source speech to be predicted is predicted. Here, the estimation of the command sequence is performed by estimating the state sequence using the Viterbi algorithm so as to increase the objective function of the above equation (16), and from the estimated state sequence, the target corresponding to the source speech to be predicted Predict the command sequence of speech.

  As illustrated in FIG. 6, the conversion processing unit 50 includes a feature amount extraction unit 52, a state sequence estimation unit 54, and a command sequence update unit 56.

  Similar to the feature quantity extraction unit 32, the feature quantity extraction unit 52 obtains a spectrogram feature quantity vector c [k] at each time k of the source voice from the time series data of the source speech to be predicted received by the input unit 10. Extract.

  Based on the parameter λ of the simultaneously generated model learned by the learning unit 30 and the spectrogram feature quantity vector c [k] extracted by the feature quantity extraction unit 52, the state series estimation unit 54 performs the above-described process according to the Viterbi algorithm. The state series s is estimated so as to increase the objective function of the equation (16).

  The command sequence update unit 56 calculates an average sequence of the state sequence s as a phrase / accent command sequence based on the state sequence s estimated by the state sequence estimation unit 54, and outputs it to the output unit 90.

<Operation of fundamental frequency pattern prediction device>

  Next, the operation of the fundamental frequency pattern prediction apparatus 100 according to the embodiment of the present invention will be described. First, when the input unit 10 receives parallel data composed of time series data of the source speech and target speech of the learning sample, the fundamental frequency pattern prediction apparatus 100 executes a learning processing routine shown in FIG.

  First, in step S100, input time-series data of source speech is read, and a spectral feature quantity vector c [k] at each time k is extracted.

  Next, in step S102, the time-series data of the input target voice is read, the fundamental frequency y [k] at each time k of the target voice is extracted, and the fundamental frequency y [k] and its dynamic components are extracted. A combined vector q [k] is extracted.

  In step S104, the state sequence s in the HMM is estimated based on the fundamental frequency y [k] extracted in step s102.

  In step S106, the parameter λ of the simultaneous generation model is initialized.

  In step S108, an expected value is calculated according to the parameter λ of the simultaneous generation model that is initially set in step S106 or updated last time in step S110.

  In step S110, based on the spectral feature vector c [k] at each time k extracted in step S100, the state series s estimated in step S104, and the expected value calculated in step S108, the above ( The parameter λ of the simultaneous generation model is updated so as to increase the objective function shown in the equation (16).

  In step S112, it is determined whether or not a predetermined convergence determination condition is satisfied. If the convergence determination condition is not satisfied, the process returns to step S108. On the other hand, when the convergence determination condition is satisfied, the parameter λ learned in step S110 is stored in the parameter storage unit 40 in step S114.

  Next, when the time-series data of the source speech to be predicted is input to the fundamental frequency pattern prediction apparatus 100, the fundamental frequency pattern prediction processing routine shown in FIG.

  First, in step S200, the input time-series data of the target speech to be predicted is read, and the spectrum feature vector c [k] at each time k is extracted.

  In step S202, the state sequence s is estimated according to the Viterbi algorithm based on the parameter λ of the simultaneously generated model learned by the learning unit 30 and the spectrogram feature quantity vector c [k] extracted in step S200.

  In step S204, based on the state series s estimated in step S202, an average series corresponding to the state series s is calculated as a phrase / accent command sequence and output to the output unit 90.

<Effect of experiment of this embodiment>

Extracting a spectral feature amount sequence and F ₀ pattern and phrase accent command from the audio signal, learning the model parameters (parameters of GMM) by the learning process using the paired data of the spectral feature amount sequence and phrase accent command sequence and the after, and conducted an experiment to convert the spectral feature amount sequence phrase ascent command sequence by the conversion process, the converted phrase ascent command sequence to confirm whether the possible extent restore the original F ₀ pattern. FIG. 9 shows an example of the result. The dotted line is the F ₀ pattern estimated from the speech signal, and the broken line is the F ₀ pattern obtained from the phrase ascent command sequence converted from the spectral feature amount sequence. It can be confirmed that the original F ₀ pattern can be substantially restored even though the spectrum feature amount does not contain much F ₀ information.

As described above, according to the fundamental frequency pattern predicting apparatus according to the embodiment of the present invention, the spectral feature quantity vector at each time extracted from the time series data of the source speech, and each time of the hidden Markov model, Based on a state sequence consisting of a state index indicating a pair of a phrase command and an accent command, learning parameters of a simultaneous generation model of a combination of a spectral feature vector at each time of the source speech and the state sequence of the target speech Then, based on the spectral feature vector at each time extracted from the time series data of the source speech to be predicted and the parameters of the learned simultaneous generation model, the target speech command sequence corresponding to the source speech to be predicted by predicting, spectrum taking into account the constraints of the physical process of generating F ₀ pattern It is possible to estimate the optimum _F0 pattern corresponding to the toll feature amount series.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

For example, in the above-described embodiment, the HMM is an HMM having a state transition network as shown in FIG. 3. However, the present invention is not limited to this, and a plurality of Left-to-Right HMMs as shown in FIG. It is good also as HMM which connected the state of the start point and end point of. Corresponding to a plurality of states _{p m} where phrase command is to occur has a plurality of Left-to-Right type HMM. Similar to the HMM example of FIG. 3, the states p ₀ ,. . . , P _M−1 state output distribution average does not depend on time k. Each state in FIG. 10 is divided into small states as in FIG.

DESCRIPTION OF SYMBOLS 10 Input part 20 Calculation part 30 Learning part 32 Feature-value extraction part 34 Fundamental frequency series extraction part 36, 54 State series estimation part 38 Model parameter learning part 40 Parameter storage part 50 Conversion process part 52 Feature-value extraction part 56 Command sequence update part 90 Output Unit 100 Fundamental Frequency Pattern Prediction Device

Claims (8)

From the source voice time series data as input, from a pair of phrase commands that represent the basic frequency pattern generated by the translational movement of the thyroid cartilage at each time of the target voice and the accent commands that represent the basic frequency pattern generated by the rotational movement of the thyroid cartilage A basic frequency pattern predicting device for predicting a command sequence comprising:
Using parallel data composed of time-series data of source speech and target speech of the learning sample as input, a spectral feature vector at each time extracted from the time-series data of the source speech, and the time of the target speech Based on a state sequence consisting of an index of a state indicating a pair of the phrase command and the accent command at each time of the hidden Markov model estimated from the sequence data, a spectral feature quantity vector at each time of the source speech, A learning unit for learning a parameter of a simultaneous generation model of a combination of the target speech and the state sequence;
Using time series data of the source speech to be predicted as input, a spectral feature vector at each time extracted from the time series data of the source speech to be predicted, and the parameters of the simultaneously generated model learned by the learning unit, And a conversion processing unit that predicts the instruction sequence of the target speech corresponding to the source speech to be predicted,
A fundamental frequency pattern prediction apparatus including: The Hidden Markov Model includes a plurality of states p _m that the phrase command is to occur, a plurality of states a _n that the accent command is occurring, the phrase state r ₀ the command and any of the accent command not causing, r have _one and, in the transition from the state r ₀ to one of the plurality of states p _m transitions to the state r _1, the transition from the state r ₁ to any one of the plurality of states a _n The fundamental frequency pattern prediction apparatus according to claim 1, wherein the states are connected so as to transition to the state r ₀ . The fundamental frequency pattern prediction apparatus according to claim 1, wherein the learning unit learns a parameter λ of the simultaneous generation model so as to increase an objective function represented by the following expression.
Where c is a spectral feature vector c [k] at each time k of the source speech, and s is the state series consisting of states s _{k at} each time k,
Is a parameter of the state output distribution P (c [k] | s _k ) of the state s _k , φ _{i, i ′} is the transition probability between the states i, _{i ′} of the hidden Markov model, and φ _i is , The probability that the initial state is state i.   The conversion processing unit estimates the state sequence so as to increase the objective function using a Viterbi algorithm, and the target speech corresponding to the source speech to be predicted is estimated from the estimated state sequence. 4. The fundamental frequency pattern prediction apparatus according to claim 3, wherein the command sequence is predicted. From the source voice time series data as input, from a pair of phrase commands that represent the basic frequency pattern generated by the translational movement of the thyroid cartilage at each time of the target voice and the accent commands that represent the basic frequency pattern generated by the rotational movement of the thyroid cartilage A fundamental frequency pattern prediction method in a fundamental frequency pattern prediction apparatus that predicts a command sequence comprising:
The learning unit receives parallel data consisting of time-series data of source speech and target speech of a learning sample as input, and a spectral feature quantity vector at each time extracted from the time-series data of the source speech, A spectrum of each time of the source speech based on a state sequence consisting of an index of a state indicating a pair of the phrase command and the accent command at each time of the hidden Markov model estimated from time series data of the target speech Learning a parameter of a simultaneous generation model of a combination of a feature vector and the state sequence of a target speech;
The conversion processing unit receives time series data of the source speech to be predicted as input, and a spectral feature vector at each time extracted from the time series data of the source speech to be predicted, and the simultaneous learning learned by the learning unit Predicting the command sequence of the target speech corresponding to the source speech to be predicted based on parameters of the generation model;
A basic frequency pattern prediction method including: The Hidden Markov Model includes a plurality of states p _m that the phrase command is to occur, a plurality of states a _n that the accent command is occurring, the phrase state r ₀ the command and any of the accent command not causing, r have _one and, in the transition from the state r ₀ to one of the plurality of states p _m transitions to the state r _1, the transition from the state r ₁ to any one of the plurality of states a _n fundamental frequency pattern predicting method according to claim 5 wherein each state is connected to transition to the state r _0. The fundamental frequency pattern prediction method according to claim 5 or 6, wherein the learning unit learns the parameter λ of the simultaneous generation model so as to increase an objective function represented by the following expression.
Where c is a spectral feature vector c [k] at each time k of the source speech, and s is the state series consisting of states s _{k at} each time k,
Is a parameter of the state output distribution P (c [k] | s _k ) of the state s _k , φ _{i, i ′} is the transition probability between the states i, _{i ′} of the hidden Markov model, and φ _i is , The probability that the initial state is state i.   The program for functioning a computer as each part of the fundamental frequency pattern prediction apparatus of any one of Claims 1-4.