JP6137477B2 - Basic frequency model parameter estimation apparatus, method, and program - Google Patents

Description

  The present invention relates to a fundamental frequency model parameter estimation device, method, and program, and more particularly, to a fundamental frequency model parameter estimation device, method, and program for estimating parameters of an observed fundamental frequency sequence from a speech signal.

  Voices contain various information in addition to language information and are used for daily communication. We are researching information processing and signal processing for analyzing and synthesizing non-linguistic information with the goal of constructing a framework for engineering these non-linguistic information.

It is known that the fundamental frequency (F ₀ ) trajectory of speech includes abundant non-linguistic information such as speaker characteristics, emotions, intentions, and the like. For this reason, modeling of the F ₀ trajectory is extremely effective in applications in which prosodic information plays an important role, such as speech synthesis, speaker recognition, emotion recognition, and a dialogue system. The F ₀ trajectory is composed of a component (phrase component) that changes gently over the entire prosodic phrase and a component (accent component) that changes sharply according to the accent. These components can be interpreted as corresponding to the translational motion and rotational motion of human thyroid cartilage, respectively, but based on this interpretation, a mathematical model representing the logarithmic F ₀ trajectory as the sum of these components (hereinafter, Called the Fujisaki model). The Fujisaki model has parameters such as the occurrence time and duration of the phrase / accent command, the size of each command, etc., and when these parameters are set appropriately, it is known to approximate the measured trajectory very well. . In addition, the validity of the linguistic correspondence of parameters has been widely confirmed.

Since the parameters of the aforementioned Fujisaki model can express prosodic features efficiently, it is a very important problem to estimate the parameters of the Fujisaki model from the actually measured F ₀ trajectory (see Non-Patent Document 1). However, this problem was originally a failure setting problem, and the Fujisaki model was not always easy because there were restrictions that should be observed based on linguistic knowledge. Until now, the inventors modeled the stochastic generation process of the F ₀ locus based on the Fujisaki model (see Non-Patent Document 2), and reduced the parameter estimation problem of the Fujisaki model to the maximum likelihood estimation problem based on the EM algorithm. We have been successful in developing effective parameter estimation algorithms.

Shuichi Narusawa, Nobuaki Himatsu, Hiroyoshi Hirose, Hiroya Fujisaki, “Automatic parameter extraction method of fundamental frequency pattern generation process model of speech,” IPSJ Journal, vol. 43, no. 7, pp. 2155-2168, 2002. K. Yoshizato, H.C. Kameoka, D.H. Saito, and S.K. Sagayama, “Hidden Markov voluntary mix model for pitch control analysis of speech,” in Proc. The 13th Annual Conference of the International Speck Communication Association (Interspec 2012), Sep. 2012.

  The central idea of the above method is that the generation process of the phrase / accent command sequence is expressed by a Hidden Markov Model (HMM). Any command sequence could be generated, allowing the generation of command sequences that are not necessarily linguistically valid.

  If the possible range of the command sequence can be appropriately limited based on linguistic constraints, Fujisaki model parameters should be estimated effectively using the proposed model.

  The present invention has been made in view of the above circumstances, and by incorporating linguistic a priori knowledge into the model through the design of the state transition topology of the HMM, it is possible to accurately estimate the parameters of the Fujisaki model. An object of the present invention is to provide a frequency model parameter apparatus, method, and program.

In order to achieve the above object, the fundamental frequency model parameter estimating apparatus according to the first invention receives a speech signal as an input, and a phrase command u _p [representing a fundamental frequency pattern generated by parallel movement of thyroid cartilage at each time k. k] and _a command sequence ^ o consisting of a pair ^ o [k] of an accent command u _a [k] representing a fundamental frequency pattern generated by the rotational motion of the thyroid cartilage, and the phrase command and each phrase k at each time k of the hidden Markov model a command state sequence ^ s made of index s _k of the state indicating the accent command pair, the state i of the hidden Markov model ', each of the transition probability between i phi _i', the parameter group ^ theta comprising _i Is a fundamental frequency model parameter estimation device that estimates the fundamental frequency at each time k of the speech signal from the time-series data of the speech signal The fundamental frequency extraction unit that extracts the main frequency sequence ^ y and the degree of uncertainty of the fundamental frequency at each time k according to whether the time series data of the audio signal is a voiced segment or an unvoiced segment. A voiced / unvoiced section estimation unit to be estimated; an initial value of the command sequence ^ o; an initial value setting unit for setting an initial value of the parameter group ^ θ; and the command sequence ^ o ′ or the command sequence updated last time Based on the initial value ^ o 'of ^ o, for each combination (k, t) of time k and state t, the observed fundamental frequency sequence ^ y, the command string ^ o', and the parameter group ^ θ ' A state posterior probability update unit that calculates the posterior probability P (s _k = t | ^ y, ^ o ', ^ θ') when given by the Forward-Backward algorithm, and the previously updated Command sequence ^ o 'or Initial value ^ o 'of the command sequence ^ o, the observed fundamental frequency sequence ^ y, the degree of uncertainty at each time k, and the posterior probability of each of the combinations (k, t) of time k and state t Based on P (s _k = t | ^ y, ^ o ′, ^ θ ′), a model parameter update unit that updates the command sequence ^ o and the parameter group ^ θ, and a predetermined convergence condition Based on the convergence determination unit that repeatedly performs the calculation by the state series posterior probability update unit and the update by the model parameter update unit until the condition is satisfied, and the command sequence ^ o that is finally updated by the model parameter update unit, Viterbi A state sequence calculation unit that calculates the state sequence ^ s using an algorithm, wherein the hidden Markov model represents the sequence of states corresponding to a plurality of templates, and is constant A plurality of Left-to-Right HMMs whose state transitions in a direction, wherein a state of a starting point in each of the plurality of Left-to-Right HMMs is connected to a specific state, and the plurality of Left-to-Right HMMs The state of the end point in each of the HMMs is connected to the specific state, and the parameter group ^ θ is, for each of the plurality of templates n, the output average μ _p ⁽ⁿ⁾ in the state corresponding to the phrase command in the template n. The output average μ _a ^{(n, m)} of each state corresponding to each accent command m in the template n is further included.

The fundamental frequency model parameter estimation method according to the second invention is based on the phrase command u _p [k] representing the fundamental frequency pattern generated by the translational motion of the thyroid cartilage at each time k, and the rotational motion of the thyroid cartilage. A command sequence ^ o consisting of a pair ^ o [k] of accent commands u _a [k] representing a fundamental frequency pattern generated by the above, and a state indicating a pair of the phrase command and the accent command at each time k of the hidden Markov model The fundamental frequency model parameter estimation for estimating the command state sequence ^ s consisting of the indices s _k and the parameter group ^ θ including the transition probabilities φ _{i ', i} between the states i ′ and i of the hidden Markov model In this method, the fundamental frequency extraction unit observes the fundamental frequency at each time k of the speech signal from the time series data of the speech signal. This frequency series ^ y is extracted, and the voiced and unvoiced section estimation unit determines the uncertainty of the fundamental frequency at each time k according to whether the time series data of the speech signal is a voiced or unvoiced section. The degree is estimated, the initial value setting unit sets the initial value of the command sequence ^ o and the initial value of the parameter group ^ θ, and the state sequence posterior probability update unit updates the command sequence ^ o last time. 'Or the initial value ^ o' of the command sequence ^ o, for each combination of time k and state t (k, t), the observed fundamental frequency sequence ^ y, the command sequence ^ o ', and the A posteriori probability P (s _k = t | ^ y, ^ o ', ^ θ') when a parameter group ^ θ 'is given is calculated using the Forward-Backward algorithm, and the model parameter update unit Further The new command sequence ^ o 'or the initial value ^ o' of the command sequence ^ o, the observed fundamental frequency sequence ^ y, the degree of uncertainty at each time k, and the combination of time k and state t (k , T), the command sequence ^ o and the parameter group ^ θ are updated based on the a posteriori probability P (s _k = t | The state series posterior probability update unit and the model parameter update unit repeatedly perform the calculation until the predetermined convergence condition is satisfied, and finally the state parameter calculation unit causes the model parameter update unit to Calculating the state sequence ^ s using a Viterbi algorithm based on the updated command sequence ^ o, wherein the hidden Markov model corresponds to a plurality of templates, A plurality of Left-to-Right HMMs representing a row and having a state transition in a certain direction, wherein a state of a starting point in each of the plurality of Left-to-Right HMMs is connected to a specific state, and The state of the end point in each of the plurality of Left-to-Right HMMs is connected to the specific state, and the parameter group ^ θ is set to a state corresponding to the phrase command in the template n for each of the plurality of templates n. The output average μ _p ⁽ⁿ⁾ and the output average μ _a ^{(n, m)} of each state corresponding to each accent command m in the template n are further included.

The fundamental frequency model parameter estimating apparatus according to the third invention is the phrase command u _p [k] representing the fundamental frequency pattern generated by the translational motion of the thyroid cartilage at each time k, and the rotational motion of the thyroid cartilage, with the speech signal as an input. A command sequence ^ o consisting of a pair ^ o [k] of accent commands u _a [k] representing a fundamental frequency pattern generated by the above, and a state indicating a pair of the phrase command and the accent command at each time k of the hidden Markov model Is a fundamental frequency model parameter estimation device for estimating a command state sequence ^ s consisting of an index s _k of an observed fundamental frequency representing a fundamental frequency at each time k of the speech signal from time series data of the speech signal The fundamental frequency extracting unit for extracting the sequence ^ y, and the time-series data of the voice signal, in either the voiced section or the unvoiced section Depending on whether there is a voiced and unvoiced section estimation unit that estimates the degree of uncertainty of the fundamental frequency at each time k, an initial value setting unit that sets an initial value of the command sequence ^ o, and the last updated 'initial value or the command string ^ o ^ o' command string ^ o and the state i of the hidden Markov model ', each of the transition probability between i phi _i', previously obtained parameter group including _i ^ When the observation fundamental frequency sequence ^ y, the command sequence ^ o ', and the parameter group ^ θ' are given for each of the combinations (k, t) of time k and state t based on θ ' A state sequence posterior probability update unit that calculates the posterior probability P (s _k = t | ^ y, ^ o ′, ^ θ ′) using the Forward-Backward algorithm, and the previously updated command sequence ^ o ′. Or the initial value ^ o 'of the command sequence ^ o, the observation basic Wavenumber series ^ y, the degree of the uncertainty at each time k, and time k, the probability after each of the previous article of a combination of state t (k, t) P ( s k = t | ^ y, ^ o ', ^ based on θ ′), a model parameter update unit that updates the command sequence ^ o, a calculation by the state sequence posterior probability update unit, and an update by the model parameter update unit until a predetermined convergence condition is satisfied. A convergence determination unit that is repeatedly performed, and a state sequence calculation unit that calculates the state sequence ^ s using a Viterbi algorithm based on a command sequence ^ o that is finally updated by the model parameter update unit, The hidden Markov model is a plurality of Left-to-Right HMMs representing a sequence of the states corresponding to a plurality of templates and transitioning in a certain direction. The start point state in each of the plurality of Left-to-Right HMMs is connected to a specific state, and the end point state in each of the plurality of Left-to-Right HMMs is connected to the specific state, and the parameter group ^ Θ is, for each of the plurality of templates n, an output average μ _p ⁽ⁿ⁾ in a state corresponding to the phrase command in the template n, and an output average μ in each state corresponding to each accent command m in the template n. _a ^{(n, m)} is further included.

A fundamental frequency model parameter estimation method according to a fourth aspect of the present invention is a phrase command u _p [k] representing a fundamental frequency pattern generated by translational motion of thyroid cartilage at each time k, and a rotational motion of thyroid cartilage, with a speech signal as an input. A command sequence ^ o consisting of a pair ^ o [k] of accent commands u _a [k] representing a fundamental frequency pattern generated by the above, and a state indicating a pair of the phrase command and the accent command at each time k of the hidden Markov model Is a fundamental frequency model parameter estimation method for estimating a command state sequence ^ s consisting of indices s _{k of the} speech signal from the time series data of the speech signal by the fundamental frequency extraction unit. The observed fundamental frequency sequence ^ y representing the frequency is extracted, and the voiced and unvoiced interval estimation unit extracts the time series data of the speech signal. And estimating the degree of uncertainty of the fundamental frequency at each time k depending on whether it is a voiced or unvoiced section, and setting an initial value of the command sequence ^ o by an initial value setting unit, The state sequence posterior probability update unit updates the command sequence ^ o 'or the initial value ^ o' of the command sequence ^ o and the transition probability φ between the states i 'and i of the hidden Markov model. Based on the parameter group ^ θ 'obtained in advance including _{i' and i,} for each combination (k, t) of time k and state t, the observed fundamental frequency series ^ y and the command string ^ o ' , And the posterior probability P (s _k = t | ^ y, ^ o ', ^ θ') given the parameter group ^ θ 'is calculated using the Forward-Backward algorithm, and the model parameter update unit The finger that was last updated by Each of the sequence ^ o 'or the initial value ^ o' of the command sequence ^ o, the observed fundamental frequency sequence ^ y, the degree of uncertainty at each time k, and the combination (k, t) of time k and state t The command sequence ^ o is updated based on the posterior probability P (s _k = t | ^ y, ^ o ', ^ θ') until a predetermined convergence condition is satisfied by the convergence determination unit. , The calculation by the state series posterior probability update unit and the update by the model parameter update unit are repeated, and the state series calculation unit performs Viterbi based on the command sequence ^ o finally updated by the model parameter update unit. Calculating the state series ^ s using an algorithm, wherein the hidden Markov model represents the series of states corresponding to a plurality of templates, and the state transitions in a certain direction. A plurality of Left-to-Right HMMs, wherein a state of a start point in each of the plurality of Left-to-Right HMMs is connected to a specific state, and an end point in each of the plurality of Left-to-Right HMMs The state is connected to the specific state, and the parameter group ^ θ is, for each of the plurality of templates n, an output average μ _p ⁽ⁿ⁾ in a state corresponding to the phrase command in the template n, It further includes an output average μ _a ^{(n, m)} in each state corresponding to the accent command m.

  A program according to the present invention is a program for causing a computer to function as each unit of the fundamental frequency model parameter estimation apparatus.

  As described above, according to the fundamental frequency model parameter estimation apparatus, method, and program of the present invention, a hidden Markov model that is a plurality of Left-to-Right HMMs representing a sequence of states corresponding to a plurality of templates is obtained. By using the linguistic a priori knowledge into the model through the design of the state transition topology of the HMM by estimating the command sequence ^ o, the command state sequence ^ s, and the parameter group ^ θ, the Fujisaki model The effect that the parameter can be estimated with high accuracy is obtained.

It is a figure for demonstrating the Fujisaki model. It is a figure for demonstrating HMM in a conventional method. It is a figure for demonstrating HMM in embodiment of this invention. It is a figure for demonstrating the division | segmentation of a state. It is the schematic which shows the structure of the fundamental frequency model parameter estimation apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the fundamental frequency model parameter estimation process routine in the fundamental frequency model parameter estimation apparatus which concerns on the 1st Embodiment of this invention. It is the schematic which shows the structure of the fundamental frequency model parameter estimation apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the content of the fundamental frequency model parameter estimation process routine in the fundamental frequency model parameter estimation apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows an experimental result.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the method proposed in the present invention, in order to realize parameter estimation of the Fujisaki model with high reproducibility of the observed F ₀ pattern, a probability model of the F ₀ pattern generation process based on the Fujisaki model is formulated and observed based on it. Assume that an F ₀ pattern has occurred. The parameter estimation algorithm of the Fujisaki model is also based on this probability model.

<Outline of the invention>
First, an outline of the present invention will be described.

  In normal speech, the types of intonation type are limited. In the case of Japanese, the pitch accent is expressed by binary values of high and low, and the number of mora included in one accent phrase is limited. For example, the accent patterns of “every reality” and “Tomorrow is a wheelchair” are the same, so the intonation is almost the same. This suggests that the Fujisaki model command string pair may be represented by connecting finite types of templates. Therefore, a finite type of Left-to-Right HMM corresponding to the template of the command sequence is considered, and the above-described template-based command sequence generation model is established by the HMM capable of transitioning between templates. It is a point. Specifically, it is realized by the following (1) to (3).

(1) Based on the product of the probability that a logarithmic fundamental frequency locus is generated when a phrase / accent command sequence and a base component are determined, and the probability that a phrase / accent command sequence (HMM output sequence) is generated, The HMM state transition probability, the HMM output sequence, the posterior probability of the HMM state sequence, and the output distribution parameter are updated so as to increase the criterion.

(2) In (1) above, the HMM is such that each start point of the Left-to-Right HMM for each phrase / accent command sequence is connected to a specific state, and each end point is connected to the specific state. .

(3) Using the model of (2) above, with the HMM state transition probability and output distribution parameters fixed, the HMM output sequence and HMM state sequence are estimated to obtain a phrase / accent command sequence .

<Probability modeling of F ₀ trajectory>
Next, probability modeling of the F ₀ locus will be described.

In the Fujisaki model, it is assumed that the logarithm F ₀ trajectory y (t) is represented by the sum of three components as follows (see FIG. 1).

Here, t is a time-independent constant called time, x _p (t) is a phrase component, x _a (t) is an accent component, and x _b is a baseline component. Furthermore, it is assumed that the phrase component and the accent component are the outputs of secondary filters of signals called phrase commands and accent commands, respectively.

Here, u _p (t) is a delta train called a phrase command, and u _a (t) is a rectangular pulse train called an accent command. Of these, at most one takes a non-zero value at each time. α and β are angular frequencies representing the speed of response of the secondary filter, respectively, and are known to take values of approximately α = 3 rad / s and β = 20 rad / s regardless of an individual or an utterance.

The following outlines a stochastic model (see Non-Patent Document 2) of the F ₀ trajectory generation process based on the Fujisaki model that has been developed by the inventors. In the above-mentioned Fujisaki model, the phrase command and the accent command are a delta train and a rectangular pulse train, respectively, and further, it is assumed that they do not overlap each other. In order to successfully describe a command sequence that satisfies these constraints as a form of a probability model, the inventors have a pair of phrase commands u _p [k] and accent commands u _a [k] ^ o [k] = ( u _p [k], u _a [k]) A model for expressing ^T as the output of the HMM was devised. When the output distribution of each state is a normal distribution, the output sequence {^ o [k]} ^K _{k = 1} is

Follow. Here, s _k represents a state at time k. That is, the equation (6) is obtained by calculating the mean μ [k] = (μ _p [k], μ _a [k]) ^T = ^ c _sk and variance ^ Σ [k] = ^ Υ _sk = diag (v _{p, k} , It means that v _{a, k} ) changes with time as a result of the state transition. A symbol indicating a matrix or a vector is attached with “^”. The configuration of the above HMM is as follows.

Phrase components and accent components are obtained by convolving different filters G _p [k] and G _a [k] with the command functions u _p [k] and u _a [k] output from the HMM.

It becomes. However, * represents the convolution regarding discrete time k. G _p [k] and G _a [k] are discrete time representations of G _p (t) and G _a (t), respectively. From the above, the discrete time representation x [k] of the F ₀ trajectory is

It becomes. _xb represents a baseline component.

In the unvoiced section, F ₀ may not be observed, and it is often unreliable even if it is observed. In addition, an estimation error may occur in the F ₀ extraction. Therefore, the observation F ₀ trajectory y [k] is expressed as the sum of the above F ₀ trajectory model x [k] and the noises x _n [k] to N (0, v ² _n [k]). _Zero series of uncertainties can be incorporated through the setting of the variance v ² _n [k]. Therefore, the observed F ₀ series y [k] is

It is expressed. Here, if x _n [k] is marginalized, the conditional probability of ^ y = {y [k]} ^K _{k = 1} with ^ o = {^ o [k]} ^K _{k = 1.} The density function P (^ y | ^ o) is

It becomes. From Equation (6), the conditional probability density function P (^ o | ^ s, where {^ o [k]} ^K _{k = 1} under the state sequence ^ s = {s _k } ^K _{k = 1} . ^ theta) is P (^ o | ^ s, ^ θ) = Π K k = 1 N (^ o [k]; given by _{c sk [k], ^ Υ} sk). Here, ^ θ represents a series of mean and variance of the output distribution. State sequence ^ s probability distribution P (^ s) than Markov assumption in HMM, the transition probabilities of the product _{P (^ s) = φ s1} Π K k = 2 φ sk, given by _sk-1.

FIG. 2 shows a state transition model of the phrase / accent command sequence in the conventional method. In the state r ₀ , μ _p [k] and μ _a [k] are zero. In the state p ₁ , μ _p [k] can take a non-negative value A _p [k], and μ _a [k] is zero. Self-transition in the state p ₁ is prohibited. In the state _{r 1} μ _p [k] and μ _a [k] is also limited only to zero. This state guarantees that μ _p [k] becomes a pulse train. State r ₀ is a state a ₁ ,. . . , A _N only, and μ _a [k] can take different values A ⁽ⁿ⁾ _a in these states, but μ _p [k] is limited to zero. To r ₁ transitions directly from a _n without passing through the a _{n 'is} prohibited. This is to ensure that μ _a [k] is a rectangular pulse train.

<Phrase / accent command vocabulary model>
An important idea in the above model is that the constraints of the Fujisaki model are expressed in the state transition topology of the HMM. Under the state transition topology so far, any command sequence that satisfies the constraints of the Fujisaki model can be used. It was possible to generate a command sequence that was not necessarily linguistically valid. If the range that the command sequence can take can be appropriately limited based on linguistic a priori knowledge, it should be possible to effectively estimate the Fujisaki model parameters using the proposed model. From the above, the essential point of the idea of the present invention is to incorporate linguistic a priori knowledge into the model through the design of the state transition topology of the HMM.

In normal speech, the types of intonation type are limited. This is because, in Japanese, the pitch accent is expressed by a binary value of high and low, and the number of mora included in one accent phrase is limited. For example, the accent patterns of “every reality” and “Tomorrow is a wheelchair” are the same, so the intonation is almost the same. This suggests that the Fujisaki model command string pair may be represented by connecting finite types of templates. Therefore, by considering a finite type of Left-to-Right HMM corresponding to a command sequence template and considering an HMM capable of transitioning between templates, a template-based command sequence generation model as described above can be established. That's right. The state transition topology of the phrase / accent command sequence based on the vocabulary model of the pitch pattern template can be represented by, for example, an HMM as shown in FIG. This HMM is referred to as a phrase / accent command string vocabulary model. Here, in order to flexibly handle how much time expansion / contraction of each template is allowed, each state (except for p ₁ , p ₂ ,...) Is the same as shown in FIG. It was decided to divide into small states constrained to have an output distribution. Thereby, it is possible to individually parameterize the stopping time in each state.

FIG. 4 shows a case where the state a _1,1 is divided into four substates a _1,1,0 , a _1,1,1 , a _1,1,2 , a _1,1,3 . Transition probabilities φ _{a1,1,0, a1,1,1} correspond to the probability that state a _1,1 lasts four times.

<Optimization algorithm>
Next, an iterative algorithm for obtaining a local optimal solution of the posterior probability P (^ o, ^ θ | ^ y) of the model parameters ^ θ and ^ o, given the observation F ₀ series ^ y, an EM algorithm and an auxiliary Derived based on the functional method. The state sequence ^ s is a hidden variable, and the posterior probability P (^ o, ^ θ | ^ y) is P (^ o, ^ θ, ^ s | ^ y) ∝P (^ y | ^ o) P (^ o Note that | ^ s, ^ θ) P (^ s) is obtained by marginalizing about ^ s, the Q function Q (^ o, ^ θ, ^ o ', ^ θ') is

I can put it. Where ^c = is equal except for constant terms. Therefore, the step of calculating P (^ s | ^ y, ^ o ', ^ θ') by the Forward-Backward algorithm, and Q (^ o, ^ θ, ^ o ', ^ θ') for ^ o and ^ θ By repeating the step of increasing the value, a solution in which P (^ o, ^ θ | ^ y) is locally maximum can be obtained. Since ^ o is a pair of Fujisaki model command functions, it is necessary to consider the non-negative constraint of o in the step of increasing Q (^ o, ^ θ, ^ o ′, ^ θ ′). An update rule that increases Q (^ o, ^ θ, ^ o ', ^ θ') while satisfying the nonnegative constraint of ^ o can be derived from the same idea as in Non-Patent Document 2 above. From Non-Patent Document 2 above, the lower bound of Q (^ o, ^ θ, ^ o ', ^ θ') is the Jensen inequality.

Can be used to design. Here, G _b [k] = δ [k] (Kronecker delta). Moreover, lambda _{i, k, l} is an arbitrary variable satisfying _{0 <λ i, k, l} <1, Σ i Σ l λ i, k, l = 1. In summary, the lower bound of the Q function is

It is expressed. Q (^ o, ^ θ, ^ o ', ^ θ') is increased by alternately repeating the step of maximizing the lower bound function with respect to λ _{i, k, l} ≧ 0 and the step of maximizing with respect to o. be able to. The update rule for any step can be determined analytically,

It is represented by In order to operate the algorithm stably in the above update, instead of Q (^ o, ^ θ, ^ o ', ^ θ'), a sparse regularization term for u _p [k] and u _a [k] The function Q (^ o, ^ θ, ^ o ′, ^ θ ′) + Λ to which Λ is added can be targeted for optimization. here,

It is. The update rule with this regularization term is inequality

It is possible to design using the fact that Here, w _k , d _k , and m _k are auxiliary variables. The conditions for establishing equality in equations (22) and (23) are

It is. Thus, the inequality

Is the lower limit function of Q (^ o, ^ θ, ^ o ′, ^ θ ′) + Λ. Differentiating Q ″ (^ o, ^ θ, ^ o ′, ^ θ ′) by u _i [l],

It becomes. Therefore, the update rule alternately performs the step of maximizing the lower limit function with respect to λ _{i, k, l} ≧ 0, d _k , m _k and the step of maximizing with respect to ^ o, so that Q (^ o , ^ Θ, ^ o ′, ^ θ ′) + Λ can be increased. Each update formula

It is.

  After the above iterations converge, ^ θ is continuously updated. The update formula can be obtained analytically,

Can be written. These updated values are newly substituted into o 'and θ' and proceed to the next iteration. The update of the transition probability will be described later.

  After the above iterative algorithm has converged, the optimum ^ s obtained by the Viterbi algorithm is used as the command string estimation.

<System configuration>
Next, the embodiment of the present invention will be described with reference to an example in which the present invention is applied to a fundamental frequency model parameter estimation apparatus that analyzes time series data of an observed speech signal and estimates parameters of the Fujisaki model. explain.

  The fundamental frequency model parameter estimation apparatus according to the first embodiment of the present invention is a computer including a CPU, a RAM, and a ROM that stores a program for executing a fundamental frequency model parameter estimation processing routine described later. It is configured and functionally configured as follows.

  As shown in FIG. 5, the fundamental frequency model parameter estimation apparatus 100 includes a storage unit 1, a fundamental frequency sequence extraction unit 2, a voiced / unvoiced interval estimation unit 3, an initial value setting unit 4, and a state sequence posterior probability update unit. 5, a model parameter update unit 6, a convergence determination unit 7, a state series calculation unit 8, and an output unit 9.

  The storage unit 1 stores time series data of the observed audio signal. The storage unit 1 also stores data representing each state of the HMM in which the state transition topology is designed in advance based on linguistic a priori knowledge. The HMM described above is an HMM that includes a plurality of Left-to-Right HMMs for a plurality of templates and that can transition between templates.

The basic frequency sequence extraction unit 2 extracts time series data of the basic frequency from the time series data of the audio signal, converts them to be expressed in discrete time k, and uses the time series data of the basic frequency of the audio signal. It is assumed that a certain observation basic frequency sequence ^ y = {F ₀ [k]} (k = 1,..., K). This fundamental frequency extraction process can be realized by a well-known technique. For example, Non-Patent Document 3 (H. Kameoka, "Statistical speech spectrum model incorporating all-pole vocal tract model and F0 contour generating process model," in Tech. Rep. IEICE, 2010, in Japanese.), The fundamental frequency is extracted every 8 ms.

The voiced / voiceless section estimation unit 3 identifies a voiced section and a voiceless section from the time-series data of the voice signal, and observes F ₀ [ k] Estimate the degree of uncertainty v _n ² [k]. In the voiced section, the degree of uncertainty is estimated to be large (for example, v _n ² [k] = 10 ¹⁵ ), and in the unvoiced section, the degree of uncertainty is estimated to be small (for example, v _n ² [k] = 0.1 ² ).

The initial value setting unit 4 regards the number of iterations M, α, β, v _p ² [k], v _a ² [k], v _b ² of the EM algorithm, which are parameters used in the processing described later, as constants, Set the initial value. Set an appropriate value as the initial value.

The initial value setting unit 4 uses the parameter estimation method of Fujisaki model described in Non-Patent Document _{1, ^ o [k] =} (u p [k], u a [k]) T initial value of , U _b Set the initial value.

Further, the initial value setting unit 4 sets the initial values of μ _p ⁽ⁿ⁾ and μ _a ^{(n, m)} as 0 as the maximum values of u _p [k] and u _a [k] set as initial values, respectively. It is given by a variable output from a uniform random number from 5 to 1 times. In addition, among the HMM states whose state transition topology is designed in advance based on linguistic a priori knowledge, the initial probability φ _i of each state i with i∈r ⁰ is expressed as a uniform distribution with respect to i. give. For other states i, the initial probability φ _i is initialized to zero. Further, based on the state transition topology of the HMM, when the transition from the state i ′ to the state i is impossible, the transition probability φ _{i ′, i} is set to 0, and the possible transition probability φ _{i ′, i} of the state transition is set. , I give initial values with uniform distribution.

  In the present embodiment, based on the above Q function, the local optimal solution of Fujisaki model parameters ^ o and θ is obtained by repeating the two steps of the state sequence posterior probability update unit 5 and the model parameter update unit 6.

The state series posterior probability update unit 5 is a step of calculating the posterior probability P (^ s | ^ y, ^ o ', θ') of the command state series (latent variable), which is called an E step in the EM algorithm. . If the Forward-Backward algorithm is used, P (s _k = t | ^ y, ^ o ′, θ ′) can be efficiently _obtained for each k and t. In particular,

Each P (s _k = t | {y [l], ^ o ′ [l], ^ θ ′ [l]} _{l = 1} ^{l = k} ) is

Can be calculated by solving the recursion formulas sequentially (k = 1,2, ..., K), and each P ({^ y [l], ^ o '[l], ^ θ' [l]} _{l = k + 1} ^{l = K} | s _k = t)

Can be calculated by solving the recursion formulas sequentially (k = K, K-1, ..., 1).

Thus, the state sequence posterior probability update unit 5 uses the Forward-Backward algorithm for each combination (k, t) of the time k and the state t to update the command sequence ^ o ′. Or, by calculating the posterior probability P (s _k = t | ^ y, ^ o ', ^ θ') based on the initial value ^ o ', the observed fundamental frequency sequence ^ y, the command string ^ o', and The posterior probability P (^ s | ^ y, ^ o ', ^ θ') of the command state sequence ^ s when the parameter group ^ θ 'is given is calculated.

  The model parameter update unit 6 includes an auxiliary variable update unit 61, a command function update unit 62, a convergence determination unit 63, and an average amplitude update unit 64.

  The model parameter update unit 6 is a step of updating the command sequence ^ o and the parameter group ^ θ so as to increase the objective function Q (^ o, θ, ^ o ′, ^ θ ′). Is called M step. When considering the sparse constraint term, ^ o and ^ θ are updated so as to increase Q (^ o, ^ θ, ^ o ′, ^ θ ′) + Λ.

The auxiliary variable updating unit 61 performs all combinations (k, l) of times k and l (l <k) based on the phrase command u _p [k] (or initial value) at each time k updated last time. For each, the auxiliary variable λ _{p, k, l} is calculated and updated according to equation (19) above. In addition, the auxiliary variable update unit 61 performs the above equation (19) for all combinations of (k, l) based on the accent command u _a [k] (or initial value) at each time k updated last time. According to the above, the auxiliary variable λ _{a, k, l} is calculated and updated.

When considering the sparse constraint terms, the auxiliary variables w _k , d _k , and m _k are further updated by the equations (33) to (35).

The command function update unit 62 includes the fundamental frequency sequence ^ y, the degree of uncertainty v _n ² [k], and the posterior probability P (^ s | ^ y of the command state sequence updated by the state sequence posterior probability update unit 5. , ^ O ′, ^ θ ′) and the auxiliary variable λ _{p, k, l} updated by the auxiliary variable updating unit 61 according to the above equation (20), the phrase command u _p [k ] Is updated. When considering the sparse constraint term, u _p [k] is updated using Equation (36) instead.

The command function update unit 62 also includes the fundamental frequency sequence ^ y, the degree of uncertainty v _n ² [k], and the posterior probability P (^ s | of the command state sequence updated by the state sequence posterior probability update unit 5. (^ Y, ^ o ′, ^ θ ′) and the auxiliary variable λ _{a, k, l} updated by the auxiliary variable updating unit 61 according to the above equation (20), the accent command u _{a at} each time k [K] is updated. When considering the sparse constraint term, u _a [k] is updated using Equation (36) instead.

  The convergence determination unit 63 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, each process of the auxiliary variable update unit 61 and the command function update unit 62 is repeated. If the convergence determining unit 63 determines that the convergence condition is satisfied, the convergence determining unit 63 proceeds to processing by the average amplitude updating unit 64.

  As the convergence condition, it may be used that the number of repetitions s has reached a predetermined number S (for example, 500 times). Note that the convergence condition is that the difference between the value of the auxiliary function when the s-1th parameter is used and the value of the auxiliary function when the sth parameter is used is smaller than a predetermined threshold. It may be used as

The average amplitude updating unit 64 continues as M steps as follows: ^ θ = {{μ _p (n)} n _{= 1} ^N , {μ _a ^{(n, m)} } _{n = 1} ^N _{m = 1} ^M , {φ _{i ', i} }} is updated.

The average amplitude update unit 64 includes the phrase command u _p [k] at each time k updated by the command function update unit 62 and the posterior probability P (s _k = t | ^ updated by the state series posterior probability update unit 5. Based on y, ^ o ′, ^ θ ′), for each of the plurality of templates n, the output average μ _p ⁽ⁿ⁾ in a state corresponding to the phrase command in the template n is updated according to the above equation (42). The average amplitude updating unit 64 also includes the accent command u _a [k] at each time k updated by the command function updating unit 62 and the posterior probability P (s _k = t updated by the state series posterior probability updating unit 5. | Y, ^ o ′, ^ θ ′), for each of a plurality of templates n, according to the above equation (43), the output average μ _a ^(n, Update ^m) .

Further, the average amplitude update unit 64 calculates P (s _k = i ′ | {y [l], o ′ [l], ^ θ ′ [l]} _{l = k} calculated by the state sequence posterior probability update unit 5. ₊₁ ^{l = K} ), P ({y [l], o ′ [l], ^ θ ′ [l]} _{l = k + 1} ^{l = K} | _sk = i), the state i of the HMM For each pair of ', i, the transition probability φ _{i', i} between states _{i ', i} is updated according to the following equation.

  The convergence determination unit 7 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, the update value is substituted into ^ o ′ and ^ θ ′. The iterative algorithm (each process of the state series posterior probability update unit 5 and the model parameter update unit 6) is repeated. If the convergence determination unit 7 determines that the convergence condition is satisfied, the convergence determination unit 7 proceeds to processing by the state series calculation unit 8.

  As the convergence condition, it may be used that the number of repetitions r has reached a predetermined number R (for example, 20 times). Note that the convergence condition is that the difference between the value of the objective function when the r-1 parameter is used and the value of the objective function when the r parameter is used is smaller than a predetermined threshold. It may be used as

  The state sequence calculation unit 8 finally obtains an optimal state sequence ^ s * by using the Viterbi algorithm. In particular,

Using δ _t [k] and ψ _t [k] obtained by sequentially solving the recursion formula (k = 1,2, ..., K),

Can be calculated by solving the recursion formulas sequentially (k = K, K-1, ..., 1).

  As described above, the state series calculation unit 8 calculates the state series ^ s according to the above equations (49) to (53) based on the command sequence ^ o finally updated by the model parameter update unit 6. To do. Then, the output unit 9 outputs a command sequence ^ o, a parameter group ^ θ, and a state sequence ^ s.

<Operation of fundamental frequency model parameter estimation device>
Next, the operation of the fundamental frequency model parameter estimation apparatus 100 according to the first embodiment will be described. First, time series data of an observed voice signal as an analysis target is input to the fundamental frequency model parameter estimation apparatus 100 and stored in the storage unit 1. Then, in the fundamental frequency model parameter estimation apparatus 100, a fundamental frequency model parameter estimation processing routine shown in FIG. 6 is executed.

First, in step S101, time-series data of an audio signal is read from the storage unit 1, and a fundamental frequency sequence ^ y composed of the fundamental frequency F _{0 at} each time k is extracted. In step S102, voiced and unvoiced intervals are specified based on the time-series data of the audio signal, and the degree of uncertainty v _n ² [k] of the fundamental frequency at each time k is estimated.

In the next step S103, appropriate initial values are set for the parameters M, α, β, v _p ² [k], v _a ² [k], and v _b ² . Further, the command sequences ^ o and u _b are estimated by the conventional method and set as initial values. In step S104, initial values of μ _p ⁽ⁿ⁾ and μ _a ^{(n, m)} are set based on the command sequence ^ o set in step S103. Further, based on the state transition topology of the HMM designed in advance, the initial probability φ _i of each state i and the transition probability φ _{i ′, i} between the states i and i ′ are set.

In step S105, based on the initial value of the command sequence ^ o set in step S103 or the command sequence ^ o updated last time in step S106 described later, for all combinations of (k, t), Update posterior probability P (^ s | ^ y, ^ o ', ^ θ') of command state sequence by updating posterior probability P (s _k = t | ^ y, ^ o ', ^ θ') To do.

In step S106, the initial value of the command sequence ^ o set in step S103 or the command sequence ^ o updated last time in step S106, the fundamental frequency sequence ^ y calculated in step S101, and the step Uncertainty level v _n ² [k] calculated at S102 at each time k, and posterior probability P (^ s | ^ y, ^ o ', ^ θ') of the command state sequence updated at Step S104 above And the command sequence ^ o to increase the objective function Q (^ o, θ, ^ o ′, ^ θ ′) or Q (^ o, θ, ^ o ′, ^ θ ′) + Λ , Update parameter group ^ θ representing command amplitude and state transition probability

  The step S106 is realized by the processes of the following steps S111 to S114.

In step S111, based on the initial value of the command sequence ^ o set in step S103 or the command sequence ^ o updated last time in step S112 described later, all combinations of (k, l) are described above. The auxiliary variables λ _{p, k, l} and λ _{a, k, l} are calculated and updated according to the equation (19). When considering the sparse constraint terms, the auxiliary variables w _k , d _k , and m _k are further updated by the equations (33) to (35).

In the next step S112, the fundamental frequency sequence ^ y calculated in step S101, the degree of uncertainty v _n ² [k] calculated in step S102, and updated in step S104. Based on the posterior probability P (^ s | ^ y, ^ o ', ^ θ') of the command state sequence and the auxiliary variables λ _{p, k, l} and λ _{a, k, l} updated in step S111 above. In accordance with the above equation (20), the command sequence ^ o consisting of the phrase command u _p [k] and the accent command u _a [k] at each time k is updated. When considering the sparse constraint term, the command sequence ^ o consisting of the phrase command u _p [k] and the accent command u _a [k] at each time k is updated instead using the equation (36).

In the next step S113, it is determined whether or not the number of repetitions s has reached S as the convergence condition. If the number of repetitions s has not reached S, it is determined that the convergence condition is not satisfied. Then, the process returns to step S111, and the processes of steps S111 to S112 are repeated. On the other hand, when the number of repetitions s reaches S, it is determined that the convergence condition is satisfied, and in step S114, the phrase command u _p [k] and the accent command u _{a at} each time k updated in step S112 above. Based on [k] and the posterior probability P (^ s | ^ y, ^ o ', ^ θ') of the command state sequence updated in step S104,

For each of the plurality of templates n, the output average μ _p ⁽ⁿ⁾ in a state corresponding to the phrase command in the template n is updated according to the above equation (42), and for each of the plurality of templates n, the above equation (43) Accordingly, the output average amplitude μ _a ^{(n, m)} in the state corresponding to each accent command m in the template n is updated.

Further, P (s _k = i ′ | {y [l], o ′ [l], ^ θ ′ [l]} _{l = k + 1} ^{l = K} ), P ({y [l], o ′ [l], ^ θ ′ [l]} _{l = k + 1} ^{l = K} | _sk = i), for each pair of HMM states i ′, i 49) to (51), the transition probability φ _{i ′, i} between states _{i ′ and i} Update.

  In step S107, it is determined whether the number of iterations r has reached R as the convergence condition. If the number of iterations r has not reached R, it is determined that the convergence condition is not satisfied. In step S108, the command sequence ^ o and parameter group θ updated in step S106 are substituted into ^ o 'and ^ θ', and the process returns to step S105, and the processes in steps S105 to S106 are performed. repeat. On the other hand, when the number of repetitions r reaches R, it is determined that the convergence condition is satisfied, and in step S108, based on the command sequence ^ o finally updated in step S106, the above equation (49) The state sequence ^ s is calculated according to the equation (53), and the output unit 9 outputs the command sequence ^ o, the parameter group ^ θ representing the amplitude of the command, and the state sequence ^ s to estimate the fundamental frequency model parameters. The processing routine ends.

  As described above, according to the fundamental frequency model parameter estimation apparatus according to the first embodiment, a hidden Markov model including a plurality of Left-to-Right HMMs representing a sequence of states corresponding to a plurality of templates is obtained. By using the linguistic a priori knowledge into the model through the design of the state transition topology of the HMM by estimating the command sequence ^ o, the command state sequence ^ s, and the parameter group ^ θ, the Fujisaki model The parameter can be estimated with high accuracy.

[Second Embodiment]
<System configuration>
Next, a second embodiment will be described. In addition, about the part which becomes the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The second embodiment is different from the first embodiment in that the command sequence ^ o and the state sequence ^ s are estimated while being fixed to model parameters obtained in advance at the learning stage. .

  As shown in FIG. 7, the fundamental frequency model parameter estimation apparatus 100 according to the second embodiment includes a storage unit 1, a fundamental frequency sequence extraction unit 2, a voiced / unvoiced interval estimation unit 3, and an initial value setting unit 4. A state sequence posterior probability update unit 5, a model parameter update unit 6, a convergence determination unit 7, a state sequence calculation unit 8, and an output unit 9.

The storage unit 1 stores time series data of the observed audio signal. In addition, the storage unit 1 stores the model parameters ^ θ = {{μ _p (n)} n _{= 1} ^N , {μ _a ⁽ ) estimated by the fundamental frequency model parameter estimation apparatus described in the first embodiment. ^{n, m)} } _{n = 1} ^N _{m = 1} ^M , {φ _{i ′, i} }} is stored.

The initial value setting unit 4 sets the values of μ _p ⁽ⁿ⁾ and μ _a ^{(n, m) and} the value of the transition probability φ _{i ′, i to} the values stored in the storage unit 1. The initial value setting unit 4 sets initial values of other parameters in the same manner as in the first embodiment.

The model parameter update unit 6 includes an auxiliary variable update unit 61, a command function update unit 62, and a convergence determination unit 63. In the model parameter updating unit 6, model parameters ^ θ = {{μ _p (n)} n _{= 1} ^N , {μ _a ^{(n, m)} } _{n = 1} ^N _{m = 1} ^M , {φ _{i ′, i} } } Value is not updated.

<Operation of fundamental frequency model parameter estimation device>
Next, the operation of the fundamental frequency model parameter estimation apparatus 100 according to the second embodiment will be described. First, model parameters ^ θ = {{μ _p (n)} n _{= 1} ^N , {μ _a ^{(n, m)} estimated by the fundamental frequency model parameter estimation apparatus by the method described in the first embodiment. } _{n = 1} ^N _{m = 1} ^M , {φ _{i ′, i} }} is input to the fundamental frequency model parameter estimation apparatus 100 and stored in the storage unit 1. In addition, time series data of the observed speech signal is input to the fundamental frequency model parameter estimation apparatus 100 as a recognition target and stored in the storage unit 1.

  Then, the fundamental frequency model parameter estimation apparatus 100 executes a fundamental frequency model parameter estimation processing routine shown in FIG. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

First, in step S101, time-series data of an audio signal is read from the storage unit 1, and a fundamental frequency sequence ^ y composed of the fundamental frequency F _{0 at} each time k is extracted. In step S102, voiced and unvoiced intervals are specified based on the time-series data of the audio signal, and the degree of uncertainty v _n ² [k] of the fundamental frequency at each time k is estimated.

In the next step S103, appropriate initial values are set for the parameters M, α, β, v _p ² [k], v _a ² [k], and v _b ² . Further, the command sequences ^ o and u _b are estimated by the conventional method and set as initial values. Then, in step S204, the storage unit 1, μ _p ^(n), reads the value of ^{_{μ a (n, m),}} μ p (n), it is set to μ _a ^{(n, m).} Further, the value of the transition probability φ _{i ′, i} is read from the storage unit 1 and set to the transition probability φ _{i ′, i} .

In step S105, the posterior probability P (s _k = t | ^ y, ^ o ', ^ θ') is updated for all the combinations of (k, t), whereby the posterior probability P of the command state sequence is updated. (^ S | ^ y, ^ o ', ^ θ') is updated.

  In step S106, the command sequence ^ o is increased so as to increase the objective function Q (^ o, θ, ^ o ', ^ θ') or Q (^ o, θ, ^ o ', ^ θ') + Λ. Update parameter group θ representing command amplitude and state transition probability

  The step S106 is realized by the processes of the following steps S111 to S113.

In step S111, auxiliary variables λ _{p, k, l} and λ _{a, k, l} are calculated and updated for all combinations of (k, l) according to the above equation (19). When considering the sparse constraint terms, the auxiliary variables w _k , d _k , and m _k are further updated by the equations (33) to (35).

In the next step S112, the command sequence ^ o composed of the phrase command u _p [k] and the accent command u _a [k] at each time k is updated according to the above equation (20). When considering the sparse constraint term, the command sequence ^ o consisting of the phrase command u _p [k] and the accent command u _a [k] at each time k is updated instead using the equation (36).

  In the next step S113, it is determined whether or not the number of repetitions s has reached S as a convergence condition. When the number of repetitions s reaches S, it is determined that the convergence condition is satisfied, and the process proceeds to step S107.

  Then, in step S107, it is determined whether or not the number of repetitions r has reached R as a convergence condition. If the number of repetitions r has not reached R, it is updated in step S108 in step S106. The command sequence ^ o and parameter group θ are substituted into ^ o ′ and ^ θ ′, and the process returns to step S105. On the other hand, when the number of repetitions r reaches R, it is determined that the convergence condition is satisfied, and in step S108, the state sequence ^ s is calculated according to the above equations (49) to (53), and the output unit 9 outputs the command sequence ^ o, the parameter group ^ θ representing the command amplitude, and the state series ^ s, and the basic frequency model parameter estimation processing routine is terminated.

  As described above, according to the fundamental frequency model parameter estimation apparatus according to the second embodiment, a hidden Markov model including a plurality of Left-to-Right HMMs representing a sequence of states corresponding to a plurality of templates is obtained. By using the presumed command sequence ^ o and the command state sequence ^ s to incorporate linguistic a priori knowledge into the model through the design of the state transition topology of the HMM, the parameters of the Fujisaki model can be estimated accurately. can do.

<Experiment>
Since the estimation accuracy of the command sequence depends on the size of the vocabulary model (number of intonation templates), the estimation accuracy of the command sequence was evaluated for vocabulary models of different sizes.

In the learning stage, model parameters were estimated from the first 50 sentences of the ATR phoneme balance sentence read by the male speaker (MHT) using the F ₀ trajectory extracted using the fundamental frequency estimation technique as learning data. The constant parameters were set as follows. t ₀ = 8 ms, α = 3.0 rad / s, β = 20.0 rad / s, v ² _p [k] = 32, v ² _a [k] = 0.032, v ² _b = 10 ⁻⁸ , voiced ^{_{v 2 n [k] = 10}} 15 in the interval, and _v 2n [k] = 0.1 ² and set in the unvoiced interval. Further, μ _b is set to the lowest value of all values of log F ₀ in the voiced interval.

  In the recognition stage, the command sequence is estimated by fixing the model parameters from the last 53 sentences of the ATR phoneme balance sentence read out by the same speaker used in the learning stage, and the estimation accuracy is evaluated for the result. did.

  Evaluation was performed for 5, 10, and 15 templates, respectively. As the initial value of the model parameter Θ, a value obtained using the method described in Non-Patent Document 1 was set.

The framework of evaluation is the same as the conventional research by the inventors (see Non-Patent Document 2). In order to evaluate the estimation accuracy of the command sequence, matching with correct data was calculated by matching for each command based on dynamic programming and used for evaluation. In the phrase command, it is defined that the command whose position difference is S or less is matched. In the accent command, it is defined that a set of commands in which the average of the difference between the command start positions and the command difference between the command end positions is S or less is matched. Information on the size of the command was not used for matching. This is because the magnitude of the command is affected by the value of the baseline component, but the setting method differs between the proposed method and the correct answer data. N _E and N _A are each as Fujisaki model command number estimated by the proposed method to be commanded number of columns of correct data, N _M is is the command number matched with the estimation command string and correct command sequence And Let N _Esum, N _Asum, and N _Msum be sums over the full text of N _E , N _A , and N _M , respectively. From these values, the insertion error E _I is defined by (N _Esum −N _Msum ) / N _Asum , the dropout error E _D is defined by (N _Asum −N _Msum ) / N _Asum , and the accuracy A is 1-E defined in _I -E _D.

  FIG. 9 shows the evaluation result at S = 0.3 s. The left, center, and right columns in FIG. 9 represent the estimation accuracy of the phrase command and the accent command, the estimation accuracy of only the phrase command, and the estimation accuracy of only the accent command, respectively. The “initial value” line is the estimation accuracy of the initial value of the EM algorithm (thus, according to the method of Non-Patent Document 1), and the “T = N” line is the estimation accuracy when the number of templates is N in the proposed algorithm. The “conventional method” line is the estimation accuracy of a method using a probability model that does not incorporate the vocabulary model of this embodiment (Non-patent Document 2). The overall estimation accuracy is improved at T = 5 and T = 10, and the estimation accuracy of the phrase command is improved in all cases.

  Thus, since the proposed method can use information on the position, size, and number of accent commands to estimate the position of the phrase command, it is considered that the detection accuracy of the phrase command has been greatly improved. On the other hand, it was found that the estimation accuracy of accent commands tended to decrease with the increase of vocabulary models due to overlearning.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the fundamental frequency model parameter estimation apparatus described above has a computer system inside, but if the “computer system” uses a WWW system, a homepage providing environment (or display environment) is also available. Shall be included.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 1 Memory | storage part 2 Fundamental frequency sequence extraction part 3 Voiced unvoiced area estimation part 4 Initial value setting part 5 State sequence posterior probability update part 6 Model parameter update part 7 Convergence determination part 8 State series calculation part 9 Output part 61 Auxiliary variable update part 62 Command function update unit 63 Convergence determination unit 64 Average amplitude update unit 100 Fundamental frequency model parameter estimation device

Claims (7)

With an audio signal as an input, a phrase command u _p [k] representing a fundamental frequency pattern generated by translational movement of thyroid cartilage at each time k and an accent command u _a [k] representing a fundamental frequency pattern generated by rotational movement of thyroid cartilage a command string ^ o consisting of a pair ^ o [k] of, and hidden at each time k of the Markov model, consisting of the index s _k of state indicating the phrase command and said accent command of the pair instruction state series ^ s, the the state i of the hidden Markov model ', each of the transition probability between i phi _i', a fundamental frequency model parameter estimation device for estimating a parameter group ^ theta including _i,
A fundamental frequency extraction unit that extracts an observed fundamental frequency sequence ^ y representing a fundamental frequency at each time k of the speech signal from the time series data of the speech signal;
About the time-series data of the speech signal, a voiced / unvoiced section estimation unit that estimates the degree of uncertainty of the fundamental frequency at each time k according to whether it is a voiced section or an unvoiced section;
An initial value setting unit for setting an initial value of the command sequence ^ o and an initial value of the parameter group ^ θ;
For each combination (k, t) of time k and state t, based on the updated value of the command sequence ^ o updated last time or the command sequence ^ o ' that is the initial value of the command sequence ^ o A posteriori probability when a fundamental frequency series ^ y, the command sequence ^ o ', and an updated value of the parameter group ^ θ updated last time or a parameter group ^ θ' which is an initial value of the parameter group ^ θ are given. A state sequence posterior probability updater that calculates P (s _k = t | ^ y, ^ o ′, ^ θ ′) using the Forward-Backward algorithm;
The command sequence ^ o ', the observed fundamental frequency sequence ^ y, the degree of uncertainty at each time k, and the a posteriori probability P (s _k =) of each combination (k, t) of the time k and the state t. t | ^ y, ^ o ', ^ θ'), a model parameter update unit that updates the command sequence ^ o and the parameter group ^ θ;
A convergence determination unit that repeatedly performs calculation by the state sequence posterior probability update unit and update by the model parameter update unit until a predetermined convergence condition is satisfied,
A state sequence calculation unit that calculates the command state sequence ^ s using the Viterbi algorithm based on the command sequence ^ o that is finally updated by the model parameter update unit;
Including
The hidden Markov model is a plurality of Left-to-Right HMMs representing a sequence of the states corresponding to a plurality of templates and having a state transition in a certain direction, wherein the plurality of Left-to-Right HMMs A starting point state in each of the plurality of Left-to-Right HMMs is connected to the specific state, and an end point state in each of the plurality of Left-to-Right HMMs is connected to the specific state;
The parameter group { circumflex over ( θ ^{)} is} an output average μ _p ⁽ⁿ⁾ in a state corresponding to the phrase command in the template n and a state in each state corresponding to each accent command m in the template n. A fundamental frequency model parameter estimation device further including an output average μ _a ^{(n, m)} . Wherein the model parameter updating unit, before Symbol command string ^ o ', the observation fundamental frequency sequence ^ y, the degree of the uncertainty at each time k, and time k, before each combination of state t (k, t) Based on the post-article probability P (s _k = t | ^ y, ^ o ', ^ θ'), the command sequence ^ o and the parameter group ^ θ when the observed fundamental frequency sequence ^ y is given. log posterior probability logP (^ o, ^ θ | ^ y) as the objective function, to increase the objective function, the basic of claim 1, wherein updating the directive columns ^ o, and the parameter group ^ theta Frequency model parameter estimation device. With an audio signal as an input, a phrase command u _p [k] representing a fundamental frequency pattern generated by translational movement of thyroid cartilage at each time k and an accent command u _a [k] representing a fundamental frequency pattern generated by rotational movement of thyroid cartilage of the pair ^ o a command string ^ o consisting of a [k], at each time k of the hidden Markov model, and a command status series ^ s made from the index s _k of state indicating the phrase command and said accent command of the pair A fundamental frequency model parameter estimating apparatus for estimating,
A fundamental frequency extraction unit that extracts an observed fundamental frequency sequence ^ y representing a fundamental frequency at each time k of the speech signal from the time series data of the speech signal;
About the time-series data of the speech signal, a voiced / unvoiced section estimation unit that estimates the degree of uncertainty of the fundamental frequency at each time k according to whether it is a voiced section or an unvoiced section;
An initial value setting unit for setting an initial value of the command sequence ^ o;
Each of the transition probabilities φ _{i ′} between the state i ′ and i of the hidden Markov model and the instruction sequence ^ o ′ which is the updated value of the instruction sequence ^ o updated last time or the initial value of the command string ^ o. _{, i} and the previously obtained parameter group ^ θ ', for each combination of time k and state t (k, t), the observed fundamental frequency sequence ^ y, the command string ^ o', and A state sequence posterior probability update unit for calculating the posterior probability P (s _k = t | ^ y, ^ o ′, ^ θ ′) when the parameter group ^ θ ′ is given using the Forward-Backward algorithm; ,
Before Symbol command string ^ o ', the observation fundamental frequency series ^ y, the degree of the uncertainty at each time k, and time k, a combination of state t (k, t) each of the posterior probability P (s _k = T | ^ y, ^ o ', ^ θ'), a model parameter update unit that updates the command sequence ^ o;
A convergence determination unit that repeatedly performs calculation by the state sequence posterior probability update unit and update by the model parameter update unit until a predetermined convergence condition is satisfied,
A state sequence calculation unit that calculates the command state sequence ^ s using the Viterbi algorithm based on the command sequence ^ o that is finally updated by the model parameter update unit;
Including
The hidden Markov model is a plurality of Left-to-Right HMMs representing a sequence of the states corresponding to a plurality of templates and having a state transition in a certain direction, wherein the plurality of Left-to-Right HMMs A starting point state in each of the plurality of Left-to-Right HMMs is connected to the specific state, and an end point state in each of the plurality of Left-to-Right HMMs is connected to the specific state;
The parameter group {circumflex over (θ)} is, for each of the plurality of templates n, an output average μ _p ⁽ⁿ⁾ in a state corresponding to the phrase command in the template n and a state in each state corresponding to each accent command m in the template n. A fundamental frequency model parameter estimation device further including an output average μ _a ^{(n, m)} . With an audio signal as an input, a phrase command u _p [k] representing a fundamental frequency pattern generated by translational movement of thyroid cartilage at each time k and an accent command u _a [k] representing a fundamental frequency pattern generated by rotational movement of thyroid cartilage a command string ^ o consisting of a pair ^ o [k] of, and hidden at each time k of the Markov model, consisting of the index s _k of state indicating the phrase command and said accent command of the pair instruction state series ^ s, the the state i of the hidden Markov model ', each of the transition probability between i phi _i', a fundamental frequency model parameter estimation method for estimating a parameter group ^ theta including _i,
The fundamental frequency extraction unit extracts the observed fundamental frequency sequence ^ y representing the fundamental frequency at each time k of the speech signal from the time series data of the speech signal,
The voiced and unvoiced section estimation unit estimates the degree of uncertainty of the fundamental frequency at each time k according to whether the time series data of the voice signal is a voiced or unvoiced section,
An initial value setting unit sets an initial value of the command sequence ^ o and an initial value of the parameter group ^ θ,
The state series posterior probability update unit, based on a command string ^ o 'is the initial value of the updated value or the command string ^ o of the instruction sequence ^ o was last updated, the time k, a combination of state t (k, For each of t), the observed fundamental frequency sequence ^ y, the command sequence ^ o ', and the parameter group ^ θ' which is the updated value of the parameter group ^ θ updated last time or the initial value of the parameter group ^ θ. A posteriori probability P (s _k = t | ^ y, ^ o ′, ^ θ ′) is given using the Forward-Backward algorithm,
The posterior of each of the command string ^ o ', the observed fundamental frequency sequence ^ y, the degree of uncertainty at each time k, and the combination (k, t) of the time k and the state t by the model parameter update unit Based on the probability P (s _k = t | ^ y, ^ o ', ^ θ'), the command sequence ^ o and the parameter group ^ θ are updated,
The convergence determination unit repeatedly performs the calculation by the state series posterior probability update unit and the update by the model parameter update unit until a predetermined convergence condition is satisfied,
Calculating a command state sequence ^ s using a Viterbi algorithm based on a command sequence ^ o finally updated by the model parameter update unit by a state series calculation unit;
The hidden Markov model is a plurality of Left-to-Right HMMs representing a sequence of the states corresponding to a plurality of templates and having a state transition in a certain direction, wherein the plurality of Left-to-Right HMMs A starting point state in each of the plurality of Left-to-Right HMMs is connected to the specific state, and an end point state in each of the plurality of Left-to-Right HMMs is connected to the specific state;
The parameter group {circumflex over (θ)} is, for each of the plurality of templates n, an output average μ _p ⁽ⁿ⁾ in a state corresponding to the phrase command in the template n and a state in each state corresponding to each accent command m in the template n. A fundamental frequency model parameter estimation method further including an output average μ _a ^{(n, m)} . With an audio signal as an input, a phrase command u _p [k] representing a fundamental frequency pattern generated by translational movement of thyroid cartilage at each time k and an accent command u _a [k] representing a fundamental frequency pattern generated by rotational movement of thyroid cartilage of the pair ^ o a command string ^ o consisting of a [k], at each time k of the hidden Markov model, and a command status series ^ s made from the index s _k of state indicating the phrase command and said accent command of the pair A fundamental frequency model parameter estimation method for estimation, comprising:
The fundamental frequency extraction unit extracts the observed fundamental frequency sequence ^ y representing the fundamental frequency at each time k of the speech signal from the time series data of the speech signal,
The voiced and unvoiced section estimation unit estimates the degree of uncertainty of the fundamental frequency at each time k according to whether the time series data of the voice signal is a voiced or unvoiced section,
An initial value setting unit sets an initial value of the command sequence ^ o,
By the state series posterior probability update unit, the previously updated value of the command sequence ^ o or the initial value of the command sequence ^ o and the state i ', i of the hidden Markov model For each combination (k, t) of time k and state t, based on a previously determined parameter group {circumflex over (θ)} including each transition probability φ _{i ′, i} . A posteriori probability P (s _k = t | ^ y, ^ o ', ^ θ') when the command sequence ^ o 'and the parameter group ^ θ' are given is calculated using the Forward-Backward algorithm. And
The model parameter updating unit, each of the previous article before Symbol command string ^ o ', the observation fundamental frequency sequence ^ y, the degree of the uncertainty at each time k, and time k, a combination of state t (k, t) Based on the post-probability P (s _k = t | ^ y, ^ o ', ^ θ'), the command sequence ^ o is updated,
The convergence determination unit repeatedly performs the calculation by the state series posterior probability update unit and the update by the model parameter update unit until a predetermined convergence condition is satisfied,
Calculating a command state sequence ^ s using a Viterbi algorithm based on a command sequence ^ o finally updated by the model parameter update unit by a state series calculation unit;
The hidden Markov model is a plurality of Left-to-Right HMMs representing a sequence of the states corresponding to a plurality of templates and having a state transition in a certain direction, wherein the plurality of Left-to-Right HMMs A starting point state in each of the plurality of Left-to-Right HMMs is connected to the specific state, and an end point state in each of the plurality of Left-to-Right HMMs is connected to the specific state;
The parameter group {circumflex over (θ)} is, for each of the plurality of templates n, an output average μ _p ⁽ⁿ⁾ in a state corresponding to the phrase command in the template n and a state in each state corresponding to each accent command m in the template n. A fundamental frequency model parameter estimation method further including an output average μ _a ^{(n, m)} .   The program for functioning a computer as each part of the fundamental frequency model parameter estimation apparatus of Claim 1 or 2.   The program for functioning a computer as each part of the fundamental frequency model parameter estimation apparatus of Claim 3.