JP2018138972A - Instruction sequence estimating device, state sequence estimation model learning device, its method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an instruction sequence estimating device etc. which improves the estimation accuracy of the Fujisaki model instruction sequence estimation, by estimating the Fujisaki model instruction sequence from an observation Flocus and language feature quantity considering the language characteristic quantity.SOLUTION: The instruction sequence estimating device includes an instruction sequence estimating part to estimate the corresponding Fujisaki model instruction sequence by using a state sequence estimation model for an observation Flocus and the language feature quantity sequence corresponding to the observation Flocus as input, and the state sequence estimation model includes a state estimation DNN and a state prior distribution model, the state estimation DNN is a DNN which estimates the posterior probability of the corresponding HMM state number at each time from a language feature quantity, and the state prior distribution model is a model which keeps the value of the prior distribution about each HMM state.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a signal processing technique for estimating parameters of a fundamental frequency pattern generation process model from speech fundamental frequency patterns.

In addition to linguistic information, various information (hereinafter also referred to as non-linguistic information) is included in speech and is used for daily communication. Information processing and signal processing for analyzing and synthesizing non-linguistic information have been studied by building a framework for engineering non-linguistic information. It is known that the trajectory of the fundamental frequency (hereinafter also referred to as F ₀ ) of speech contains abundant non-linguistic information such as speaker characteristics, emotions, and intentions. For this reason, F ₀ trajectory modeling is extremely effective in applications where prosodic information plays an important role, such as speech synthesis, speaker recognition, emotion recognition, and dialogue systems. The F ₀ locus is composed of a component (phrase component) that changes gently over the entire prosodic phrase and a component (accent component) that changes sharply according to the accent. These components can be interpreted as corresponding to the translational motion and rotational motion of human thyroid cartilage, respectively, but based on this interpretation, a mathematical model that represents the logarithm F ₀ trajectory as the sum of these components (hereinafter, (Also called the Fujisaki model) has been proposed. The Fujisaki model has parameters such as the occurrence time and duration of phrase commands and accent commands, and the size of each command, and is known to approximate the measured F ₀ trajectory very well when these parameters are set appropriately. It has been. In addition, the validity of the linguistic correspondence of parameters has been widely confirmed. Since the parameters of the above-mentioned Fujisaki model can express prosodic features efficiently, it is useful if the parameters of the Fujisaki model can be estimated with high accuracy from the measured F ₀ trajectory. Until now, the stochastic generation process of the F ₀ pattern based on the Fujisaki model has been modeled, and the maximum likelihood parameter of the Fujisaki model is expressed by the Expectation-Maximization (EM) algorithm (see Non-Patent Documents 1-3) and auxiliary functions. Methods have been proposed for estimation using the method (see Non-Patent Document 4). The command sequence estimation device 90 of the prior art estimates the corresponding Fujisaki model command sequence from the observed F ₀ trajectory using the estimated parameters, and outputs it as an estimated Fujisaki model command sequence (see FIG. 1).

H. Kameoka, J. L. Roux, and Y. Ohishi, "A statistical imodel of speech F0 contours", in Proc.SAPA, 2010, pp. 43-48. K. Yoshizato, H. Kameoka, D. Saito, and S. Sagayama, "Statistical approach to fujisaki-model parameter estimation from speech signals and its quantitative evaluation", in Proc. Speech Prosody 2012, 2012, pp. 175-178. K. Yoshizato, H. Kameoka, D. Saito, and S. Sagayama, "Hidden Markov convolutive mixture model for pitch contour analysis of speech", in Proc. The 13th Annual Conference of the International Speech Communication Association (Interspeech 2012), Sep . 2012. Ryotaro Sato, Hirokazu Kameoka, Kunio Kanno, “Prosody Prosody Sequence Estimation by Simultaneous Generation Model of Fundamental Frequency Pattern and Phoneme Feature Sequence”, Research Report Spoken Language Information Processing (SLP), 2016, pp.1-6.

It has been confirmed that the Fujisaki model command sequence is information associated with language feature quantities (phonemes, accent types, etc.). For example, there is a tendency that a phrase command is likely to stand near the start time of a phrase phrase. Also, there is a tendency that an accent command is likely to stand near the accent nucleus. Therefore, when estimating the Fujisaki model command sequence from the observation F ₀ locus, if the language feature amount corresponding to the observation F ₀ locus is obtained, in order to improve the accuracy of Fujisaki model command sequence estimation, language feature value is valid It can be a clue.

However, the conventional method (see Non-Patent Documents 1-4) is to estimate the Fujisaki model command sequence only from the observed F ₀ trajectory, and does not consider the language feature.

The present invention considers a language feature amount and estimates a Fujisaki model command sequence from the observed F ₀ trajectory and the language feature amount, thereby improving the estimation accuracy of the Fujisaki model command sequence estimation, and the command sequence An object of the present invention is to provide a state sequence estimation model learning device, a method, and a program for learning a state sequence estimation model used in the estimation device.

In order to solve the above-described problem, according to one aspect of the present invention, the command sequence estimation device receives an observation F ₀ trajectory and a language feature amount sequence corresponding to the observation F ₀ trajectory as input, and state sequence estimation It includes a command sequence estimator that estimates the corresponding Fujisaki model command sequence using the model, the state sequence estimation model includes the state estimation DNN and the state prior distribution model, and the state estimation DNN corresponds to each time from the language feature This is a DNN that estimates the posterior probability of the HMM state number, and the state prior distribution model is a model that holds the value of the prior distribution for each HMM state.

In order to solve the above-described problem, according to another aspect of the present invention, a command sequence estimation method executed by a command sequence estimation apparatus includes an observation F ₀ trajectory and a language feature amount sequence corresponding to the observation F ₀ trajectory. And a command sequence estimation step for estimating the corresponding Fujisaki model command sequence using the state sequence estimation model. The state sequence estimation model includes a state estimation DNN and a state prior distribution model, and the state estimation DNN is a language. The DNN estimates the posterior probability of the corresponding HMM state number at each time from the feature quantity, and the state prior distribution model is a model that holds the value of the prior distribution for each HMM state.

  According to the present invention, it is possible to improve the estimation accuracy of the Fujisaki model command sequence estimation.

The functional block diagram of the command sequence estimation apparatus which concerns on a prior art. The functional block diagram of the command sequence estimation apparatus which concerns on 1st embodiment. The figure which shows the example of the processing flow of the command sequence estimation apparatus which concerns on 1st embodiment. Log F ₀ locus and phrase component of Fujisaki model diagram showing the relationship between the accent component. The figure which shows the state transition network expressing the restrictions with respect to a phrase command and an accent command. The figure which shows the example which divided | segmented the state of HMM. The functional block diagram of the state series estimation model learning apparatus which concerns on 1st embodiment. The figure which shows the example of the processing flow of the state series estimation model learning apparatus which concerns on 1st embodiment. The functional block diagram of the state series estimation model learning apparatus which concerns on 1st embodiment. The figure which shows the example of the processing flow of the state series estimation model learning apparatus which concerns on 2nd embodiment.

  Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, constituent parts having the same function and steps for performing the same process are denoted by the same reference numerals, and redundant description is omitted. In the following description, it is assumed that processing performed for each element of a vector or matrix is applied to all elements of the vector or matrix unless otherwise specified.

<Points of first embodiment>
In the present embodiment, the F ₀ trajectory generation model is formulated using language information (language feature amount). Thereby, the Fujisaki model command sequence can be estimated using the observed F ₀ trajectory and the language feature. By considering not only the observed F ₀ trajectory but also the linguistic feature quantity, it is possible to estimate the Fujisaki model command sequence with higher accuracy.

A new model of linguistic feature series generation process from HMM state is newly added to the probability model of F ₀ pattern generation process in the previous research, and integrated modeling is performed. As a model for the generation process of a language feature quantity sequence from an HMM state, a DNN is used in which a language feature quantity sequence is input at each time and a posterior probability of an HMM state number is output. From the above formulation, it is possible to estimate the Fujisaki model command sequence considering not only the observed F ₀ trajectory but also the linguistic features, and at the same time, as in the previous study (see Non-Patent Document 4), the Viterbi algorithm and auxiliary function method The parameter estimation algorithm using can be derived. As a result, the estimation accuracy of the Fujisaki model command sequence is improved.

<Command Sequence Estimation Device 110 According to the First Embodiment>
FIG. 2 is a functional block diagram of the command sequence estimation apparatus 110 according to the present embodiment, and FIG. 3 shows an example of the processing flow.

  The command sequence estimation device 110 is configured by a computer including a CPU, a RAM, and a ROM that stores a program for executing the following processing, and is functionally configured as follows.

  Command sequence estimation apparatus 110 includes a command sequence estimation unit 111.

  Before describing the command sequence estimation unit 111, terms are first described.

(Explanation of terms used)
Observation F ₀ locus: F ₀ is information represented by a real number for each frame corresponding to the pitch (pitch) of the voice. When the number of frames of input speech is K, it is expressed as y = (y [0], y [1],..., Y [K-1]). For example, it is obtained by performing signal processing on the input speech waveform.
Estimated phrase command sequence: This is an estimation result of the phrase command sequence of the Fujisaki model output by command sequence estimation. u _p [0], u _p [1], ..., u _p [K-1].
Estimated accent command sequence: an estimation result of the phrase command sequence of the Fujisaki model that is output by command sequence estimation. u _a [0] _, u _a [1], ..., u _a [K-1].
Estimated Fujisaki model command sequence: an estimation result of the Fujisaki model command sequence output by command sequence estimation. It consists of an estimated phrase command sequence and an estimated accent command sequence. u _i = (u _i [0], u _i [1],... _, u _i [K−1]) (i = p, a).
Language feature series: Information such as pronunciation corresponding to the observed F ₀ trajectory. Includes phoneme information and accent information. Information on the start time and end time of each phoneme is stored. In addition to this, part of speech information and syntactic structure information may be included. w = (w [0], w [1], ..., w [K-1]).
Language feature value data: Data that holds each language feature value series for a plurality of utterances. When the number of utterances in the data is N, it is expressed as {w ₀ , w ₁ ,..., W _N−1 }.
HMM state series: A series of state numbers of the Fujisaki model state at each time for an utterance corresponding to a language feature quantity series. When the number of frames is K, it is expressed as s = (s [0], s [1], ..., s [K-1]). Here, the total number of states of the HMM is I, and s [k] = i (i = 0, 1,..., I-1).
HMM state data: The state number of the Fujisaki model state at each time is stored for a plurality of utterances corresponding to language feature data. When the number of utterances in the data is N, it is expressed as {s ₀ , s ₁ ,..., S _N−1 }.
State series estimation model: It consists of a state estimation DNN and a state prior distribution model.
State estimation DNN: DNN for estimating the posterior probability of the corresponding HMM state number at each time from the language feature. Used to model the posterior probability p (s [k] | w [k]).
State prior distribution model: A model in which the value of the prior distribution p (s) is held for each HMM state s = 0, 1,..., I-1. Used to model p (s [k]).

  Hereinafter, the processing content of the command sequence estimation unit 111 will be described.

<Command sequence estimation unit 111>
The command sequence estimation unit 111 receives the state series estimation model prior to the estimation of the Fujisaki model command sequence. The command sequence estimation unit 111 receives the observation F ₀ trajectory y and the language feature amount sequence w as inputs, estimates a corresponding Fujisaki model command sequence using the state sequence estimation model (S111), and estimates the estimation result as an estimated Fujisaki model Output as command sequence u _i .

(Probability modeling of F ₀ trajectory (see Non-Patent Documents 1-4))
First, the probability model of the F ₀ trajectory used in this embodiment is formulated. The Fujisaki model uses the logarithm F ₀ trajectory y (t) as the sum of the following three components:

It is a model represented by. Here, t is a time, x _p (t) is a phrase component, x _a (t) is an accent component, and x _b is a time-independent constant called a baseline component. FIG. 4 shows the relationship between the logarithmic F ₀ trajectory y (t) of the Fujisaki model, the phrase component x _p (t), and the accent component x _a (t). Furthermore, the phrase component x _p (t) and the accent component x _a (t) are the second-order filters G _p (t) and G of the signal called the phrase command u _p (t) and the accent command u _a (t), respectively. _a (t) output

It is assumed that Here, the phrase command u _p (t) is a pulse train, and the accent command u _a (t) is a rectangular pulse train (see FIG. 4). For example, the phrase command u _p (t) is expressed by information indicating the start time and amplitude of the pulse train, and the accent command u _a (t) is information indicating the disclosure time, end time (or duration) and amplitude of the rectangular pulse train. And may be expressed as Of these phrase commands u _p (t) and accent commands u _a (t), at most one takes a non-zero value at each time. α and β are angular frequencies representing the response speed of the secondary filter, respectively, and are known to take values of about α = 3 rad / s and β = 20 rad / s regardless of the individual or speech.

In the above-mentioned Fujisaki model, it is assumed that the phrase command u _p (t) and the accent command u _a (t) are the delta sequence and the rectangular pulse sequence, respectively, and that they do not overlap each other. The central idea of the methods of Non-Patent Documents 1-4 is that the generation process of the phrase command u _p (t) and the accent command u _a (t) is expressed by a hidden Markov model (HMM). . The index of the frame time and k, phrase command u _p [k], accent command u _a [k] output value o the pair of _{[k] = (u p [} k], u a [k]) and ^T. However, ^T represents transposition. When the output distribution of each state is a normal distribution, the output sequence {o [k]} ^K _{k = 1} is

Follow. Here, s [k] represents a state at time k. In other words, equation (6) is expressed as follows: mean ρ [k] = (μ _p [k], μ _a [k]) ^T = c _{s [k]} [k] and variance Σ [k] = Υ _{s [k]} = diag It means that (σ _{p, k} ² , σ _{a, k} ² ) changes with time as a result of state transition. μ _p [k], σ _{p, k} ² are the mean and variance of the output distribution (normal distribution) of the phrase command u _p [k], respectively, and μ _a [k], σ _{a, k} ² are the accent commands u _a Average and variance of the output distribution (normal distribution) of [k]. The advantage of the HMM is that the constraints to be imposed on the sequence to be modeled through the design of the state transition network can be flexibly set. The above-mentioned restrictions on the phrase command u _p [k] and the accent command u _a [k] can be expressed by, for example, a state transition network as shown in FIG. 5 (see Non-Patent Document 4). State p ₀ is a state in which only phrase command u _p [k] is activated, a _n (n = 0,1, ..., N-1) is a state in which only accent command u _a [k] is activated, r _i (i = 0,1) represents the state in which neither command is activated. The phrase command u _p [k] is activated in an impulse shape and the accent command u _a [k] is activated in a rectangular pulse train due to the restriction of the path represented by the arrows in the figure. In addition, the duration of self-transition can be parameterized by dividing each state into several small states with the same output distribution. Example of dividing the state a _n in FIG. 6 are indicated (see Non-Patent Document 4). For example, as shown in this figure, by setting the state transition probability from a _{n, m} to a _{n, m + 1} to ₁ for all m ≠ 0, the transition from a _{n, 0} to a _{n, m} probability corresponds to the probability that state a _n lasts only m step, it becomes possible to flexibly control the persistence length of the accent command. Similarly, by dividing p ₀ into small states, it is possible to parameterize the distribution of the duration of the phrase command and the length of the interval between commands.

  The formulation of the HMM so far is the same as in the conventional research (see Non-Patent Document 4). In the present embodiment, the generation probability p (w | s) of the language feature quantity sequence w is further modeled and integrated from the HMM state sequence s.

(Integration with language feature model)
In the present embodiment, the process of generating the language feature amount series w from the state series s is formulated assuming that it is independent at each time k.

Here, p (w [k]) is a constant. p (s [k] | w [k]) can be modeled by DNN. Specifically, a model is constructed by configuring a DNN that inputs w [k] at each time k and outputs the posterior probability p (s [k] | w [k]) of the HMM state number s [k]. Can be p (s [k]) is a prior distribution of HMM state numbers. For example, the relative frequency of each state number in the HMM state data is used.

  From the above, the configuration of the HMM in the present embodiment is as follows.

O [k] = (u _p [k], u _a [k]) output from the above HMM, different filters G _p [k] for the command functions u _p [k], u _a [k] included in ^T ] And G _a [k] are convolved with the phrase component and accent component

It becomes. However, * represents the convolution regarding the frame time k. G _p [k] and G _a [k] are discrete time representations of G _p (t) and G _a (t), respectively. From the above, the discrete time representation x [k] of the F ₀ trajectory is

It becomes. x _b represents a baseline component. In unvoiced intervals, F ₀ may not be observed or may not be reliable even if it is observed. In addition, an estimation error may occur in F ₀ extraction. Therefore, the observation F ₀ pattern y [k] is represented as the sum of the above F ₀ pattern model x [k] and the noise x _n [k] to N (0, v ² _n [k]), thereby observing F _0. The uncertainty of the pattern y [k] can be incorporated through the setting of the variance v ² _n [k]. That is, the observed F ₀ pattern y [k]

This means that all observation intervals can be handled in a unified manner regardless of whether they are reliable intervals.

Here, when the noise x _n [k] is marginalized, the observed F ₀ trajectory y = {y [k]} ^K- with the output sequence o = {o [k]} ^K-1 _{k = 0} The conditional probability density function p (y | o) for ¹ _{k = 0} is

It becomes. From Equation (6), the conditional probability density function of the output sequence o = {o [k]} ^K-1 _{k = 0} given the state sequence s = {s [k]} ^K-1 _{k = 0} p (o | s) is

Given in. The probability distribution p (s) of the state sequence s is the product of the transition probabilities φ _{s [k-1] and s [k] based on} the Markov assumption in the HMM.

Given in. Note that π _{s [0]} is an initial state probability.

  From the above, the proposed model is

It can be expressed in the form of p (y | o), p (o | s), and p (s) are formulated by the same formulation as the conventional F ₀ locus probability modeling (see Non-Patent Document 4), and p (w | s ), A model is learned from language feature data and HMM state data by a state sequence estimation model learning device 120 described later and used.

(Fujisaki model parameter estimation algorithm)
In the present embodiment, state output when an observation F ₀ sequence y = {y [k]} ^K−1 _{k = 0} and a language feature sequence w = {w [k]} ^K−1 _{k = 0} is given. Sequence o = {o [k]} ^K-1 _{k = 0} and state sequence s = {s [k]} ^K-1 _{k = 0}

The Fujisaki model parameters can be estimated by obtaining the state output series o and the state series s that maximize. Algorithm, logp, fix the status output sequence _{o (y, o, w,} s) and updating the state sequence s such that the maximum, logp (y, fix the state sequence _s, o _, w _, s) is increased by repeating the step of updating the state output sequence o under non-negative constraints.

The algorithm is as follows.
1. The initial value of the state output series o is obtained from the observed F ₀ series y. Various methods are conceivable as a method for obtaining the initial value. For example, the initial value of the state output series o can be obtained by the methods of Non-Patent Document 1 and Non-Patent Document 4. As for the initial value acquisition method, any existing technique may be used, and an optimal method may be selected as appropriate in accordance with the use environment and the like.
2. For each time k and for each state s [k], the output probability distribution p (w [k] | s [k]) of w is calculated by equation (9).

3. Update the state sequence s according to equations (17) to (24) described later.
4. Update the state output sequence (command sequence sequence) o according to equations (25) to (28) described later.
5. Repeat steps 3 and 4 above a certain number of times, and obtain the command sequence sequence o = ({u _p [0] _, u _p [1] _, … _, u _p [K-1]} _, {u _a [0] _, u _a [1] _, … _, u _a [K-1]}) are output as the estimated Fujisaki model command sequence.

  Hereinafter, a method for updating the state series s and the state output series o will be described.

(Update step of status series s)
In this step, the state sequence s is updated so that the logp (y _, o _, w _, s) becomes maximum after fixing the state output sequence o. logp (y _, o _, w _, s) = logp (y | o) + logp (o | s) + logp (w | s) + logp (s) where the term that depends on the state sequence s is logp (o | s ) + logp (w | s) + logp (s), so the problem of finding the state sequence s that maximizes logp (y _, o _, w _, s) is the state of the HMM with o _, w as the output sequence It is the same type as the sequence search problem. Therefore, it can be solved using the Viterbi algorithm. The algorithm is shown below.

(Viterbi algorithm)
1.Initialization

2 derivation process

3.Result

4.Backtrack of state series

The obtained {s [k] ^* } is updated as the state sequence s ^* .

(Update step of status output series o)
In this step, the state output sequence o is updated so that logp (y _, o _, s) becomes maximum after fixing the state sequence s. Since this step is the same as Non-Patent Document 4, the derivation process is omitted and only the update formula is described.

Here, l = 0, 1,..., K−1, and C ^(p) [k] _and C ^(a) _n are average parameters of the state output distribution (normal distribution) of the phrase command and the accent command, respectively. Furthermore, T _n is, s [k] = the set of k such that a _n T _n = | represents {k s [k] = a n}, | T n | denotes the number of elements in the set.

  Next, the posterior probability p (s [k] | w [k]) used when calculating p (w [k] | s [k]) used in the equations (19), (20), etc. (equation (7) The state sequence estimation model learning device 120 that learns the DNN that outputs (see (8)) will be described.

<State Sequence Estimation Model Learning Device 120 according to First Embodiment>
FIG. 7 is a functional block diagram of the state sequence estimation model learning device 120, and FIG. 8 is a diagram illustrating an example of a processing flow thereof.

  The state sequence estimation model learning device 120 includes a CPU, a RAM, and a computer including a ROM that stores a program for executing the following processing, and is functionally configured as follows.

  The state series estimation model learning device 120 includes a model learning unit 121.

<Model learning unit 121>
The model learning unit 121 includes language feature data {w ₀ , w ₁ ,..., W _N-1 } (also referred to as a plurality of learning language feature sequences w) and HMM state data {s ₀ , s ₁ ,. s _N-1 } (also referred to as a plurality of learning HMM state sequences s) is input, and using these data, a state sequence estimation model is learned (S121) and output. Among the state series estimation models, the state estimation DNN includes language feature data {w ₀ , w ₁ , ..., w _N-1 } and HMM state data {s ₀ , s ₁ , ..., s _N-1 }. Using the set, a DNN that estimates the posterior probability p (s | w) of the HMM state number s at each time k is learned from the language feature value w. For example, the learning algorithm can be the same as the learning algorithm of identification DNN generally used in speech recognition or the like. As the HMM state data {s ₀ , s ₁ ,..., S _N-1 }, a correct answer label for manually estimated Fujisaki model command sequence is given and used. Alternatively, a result automatically estimated from the observed F ₀ trajectory data {y ₀ , y ₁ ,..., Y _N−1 } by the algorithm of Non-Patent Document 1-4 may be used.

Among the state series estimation models, in order to obtain a state prior distribution model, for example, using the HMM state data {s ₀ , s ₁ ,..., S _N-1 }, The relative frequency of each HMM state can be defined as p (s = i).

Here, the HMM state number of the kth time frame during the nth utterance is represented by s _{n, k} ,

It was.

<Effect>
With such a configuration, the state sequence estimation model learning device 120 configures a DNN that receives language feature quantities such as reading and accent as input and outputs the posterior probability of the HMM state in Non-Patent Document 4. The command sequence estimator 110 estimates the Fujisaki model command sequence sequence considering both the observed F ₀ trajectory and the language feature sequence by using the posterior probability sequence output by this DNN during HMM state decoding in the conventional research. Thus, the estimation accuracy of the Fujisaki model command sequence estimation can be improved.

<Points of second embodiment>
In the first embodiment, for example, as a method of preparing HMM state data, there is a method of using a manual correct label. However, manual assignment of correct answer labels requires work by an expert and is expensive. Therefore, it is conceivable to perform DNN learning by using the HMM state sequence corresponding to the Fujisaki model command sequence estimated from the observed F ₀ trajectory as teacher data by conventional research (see Non-Patent Documents 1-3). On the other hand, an estimation error may be included in the estimation result of the Fujisaki model command sequence by the conventional research (see Non-Patent Documents 1-3). When data with errors in the estimation results (HMM state sequences corresponding to the estimated Fujisaki model command sequence) is used as the DNN learning teacher data, the accuracy of the posterior probability sequence output from the DNN learned from the teacher data Generally decreases. Therefore, the Fujisaki model command sequence estimation accuracy according to the first embodiment may also decrease.

  From the above, one of the factors for implementing the first embodiment at low cost and improving the accuracy of instruction sequence estimation of the Fujisaki model is the learning of DNN with high accuracy. Teacher data is required.

In the second embodiment, to obtain training data for more accurate DNN learning, in the conventional study is estimated only from the observed F ₀ locus (see Non-Patent Documents 1 to 3) without observing F ₀ locus and linguistic feature quantity The method of the first embodiment for performing estimation using both of the above is used. Thereby, DNN with higher accuracy can be learned from more accurate learning data, and the final Fujisaki model command sequence estimation accuracy can be improved.

<Second embodiment>
A description will be given centering on differences from the first embodiment.

  The configuration of the state sequence estimation model learning device is different from that of the first embodiment.

  Before describing the state sequence estimation model learning device 220, the terms will be described again.

  The configuration of the command sequence estimation device 110 is the same as that of the first embodiment, but is different from the first embodiment in that a relearning state sequence estimation model described later is used instead of the state sequence estimation model.

(Explanation of terms used)
Observation F ₀ trajectory data: data holding each observation F ₀ value for a plurality of utterances corresponding to language feature data. When the number of utterances in the data is N, it is expressed as {y _0, y _1, ... _, y _N-1 }.
Estimated HMM state data: For each utterance in the observed F ₀ trajectory data, the command sequence estimation of the first embodiment is performed, and the HMM state sequence corresponding to the estimated Fujisaki model command sequence is held. That is, the state series s is obtained by repeating the update of the state series s and the state output series o a certain number of times. When N is the number of utterances in the data, it is expressed as {s ^e _0, s ^e _1, ... _, S ^e _N−1 }.
Re-learning state series estimation model: using estimated HMM state data {s ^e _0, s ^e _1, … _, s ^e _N-1 } and language feature data {w ₀ , w ₁ ,…, w _N-1 } The state series estimation model learned again.

<State Sequence Estimation Model Learning Device 220 according to Second Embodiment>
FIG. 9 is a functional block diagram of the state sequence estimation model learning device 220, and FIG. 10 is a diagram illustrating an example of a processing flow thereof.

  The state sequence estimation model learning device 220 includes a command sequence estimation unit 211 and a model learning unit 121.

The command sequence estimation unit 211 has the same configuration as the command sequence estimation unit 111 of the first embodiment. However, the language feature amount sequence w, observation F ₀ each language feature data instead of the trajectory _{y {w 0, w 1,} ..., w N-1}, observation F ₀ locus data _{{y 0, y 1, ...} , y _N-1 } (also referred to as multiple learning observation F ₀ trajectories) as input, and instead of the estimated Fujisaki model command sequence u _i , estimated HMM state data {s ^e _0, s ^e _1, … _, s ^e _{N -1} } is output. Therefore, the command sequence estimating unit 211, the language feature data _{{w 0, w 1, ...} , w N-1} and the observed F ₀ locus data _{{y 0, y 1, ...} , y N-1} as input Then, the corresponding Fujisaki model command sequence is estimated using the state series estimation model (S211), and the estimated HMM state data {s ^e corresponding to the estimated Fujisaki model command sequence (in other words, at the time of estimating the estimated Fujisaki model command sequence) _0, s ^e _1, … _, s ^e _N-1 } are output.

The model learning unit 121 has the same configuration as the model learning unit 121 of the first embodiment, and performs the same processing (S121). However, HMM state data _{{s 0, s 1, ...} , s N-1} instead of the estimated HMM state data ^{_{{s e 0, s e 1}} , ..., s e N-1} is that it uses different.

With such a configuration, the command string estimation of the first embodiment is performed on the observation F ₀ trajectory {y _0, y _1, ... _, Y _N-1 } of each utterance of the observation F ₀ trajectory data, and the estimation result from, HMM state number s ^e _n at each _{time, k (n = 0,1, ...} , n-1, k = 0,1, ..., K-1) obtained. These are stored as estimated HMM state data {s ^e _0, s ^e _1, ... _, S ^e _N−1 }, and are used by the model learning unit 121 for state sequence estimation model learning.

<Effect>
With such a configuration, DNN with higher accuracy can be learned from more accurate learning data, and the final Fujisaki model command sequence estimation accuracy can be increased.

<Other variations>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
In addition, various processing functions in each device described in the above embodiments and modifications may be realized by a computer. In that case, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its storage unit. When executing the process, this computer reads the program stored in its own storage unit and executes the process according to the read program. As another embodiment of this program, a computer may read a program directly from a portable recording medium and execute processing according to the program. Further, each time a program is transferred from the server computer to the computer, processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program includes information provided for processing by the electronic computer and equivalent to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In addition, although each device is configured by executing a predetermined program on a computer, at least a part of these processing contents may be realized by hardware.

Claims (7)

Wherein the observation F ₀ locus, and inputs the linguistic feature amount sequence corresponding to the observation F ₀ locus, using the state series estimation model, a command sequence estimating unit that estimates a corresponding Fujisaki model command string,
The state series estimation model includes a state estimation DNN and a state prior distribution model, the state estimation DNN is a DNN that estimates a posterior probability of the corresponding HMM state number at each time from a language feature, and the state prior distribution model is It is a model that holds the value of the prior distribution for each HMM state.
Command sequence estimation device. A state sequence estimation model learning device for learning the state estimation DNN used in claim 1,
A model learning unit that learns the state sequence estimation model using a plurality of learning language feature amount sequences and a plurality of learning HMM state sequences,
State sequence estimation model learning device. The state sequence estimation model learning device according to claim 2,
Using the plurality of learning language feature amount sequences and the plurality of learning observation F ₀ trajectories as inputs, using the second state sequence estimation model, estimating a plurality of corresponding Fujisaki model command sequences, and estimating a plurality of estimated Fujisaki models A second command sequence estimator for obtaining a plurality of estimated HMM state sequences that are a plurality of HMM state sequences corresponding to the command sequence;
The plurality of learning HMM state sequences are a plurality of the estimated HMM state sequences,
The second state sequence estimation model includes a second state estimation DNN and a second state prior distribution model, and the second state estimation DNN is a DNN for estimating a posterior probability of a corresponding HMM state number at each time from a language feature. The second state prior distribution model is a model that holds the value of the prior distribution for each HMM state.
State sequence estimation model learning device. Wherein the observation F ₀ locus, and inputs the linguistic feature amount sequence corresponding to the observation F ₀ locus, using the state series estimation model, a command sequence estimation step of estimating a corresponding Fujisaki model command string,
The state series estimation model includes a state estimation DNN and a state prior distribution model, the state estimation DNN is a DNN that estimates a posterior probability of the corresponding HMM state number at each time from a language feature, and the state prior distribution model is It is a model that holds the value of the prior distribution for each HMM state.
A command sequence estimation method executed by the command sequence estimation apparatus. A state sequence estimation model learning method for learning the state estimation DNN used in claim 4,
A model learning step of learning the state sequence estimation model using a plurality of learning language feature amount sequences and a plurality of learning HMM state sequences;
A state sequence estimation model learning method executed by the state sequence estimation model learning device. The state sequence estimation model learning method according to claim 5,
Using the plurality of learning language feature amount sequences and the plurality of learning observation F ₀ trajectories as inputs, using the second state sequence estimation model, estimating a plurality of corresponding Fujisaki model command sequences, and estimating a plurality of estimated Fujisaki models A second command sequence estimation step for obtaining a plurality of estimated HMM state sequences that are a plurality of HMM state sequences corresponding to the command sequence;
The plurality of learning HMM state sequences are a plurality of the estimated HMM state sequences,
The second state sequence estimation model includes a second state estimation DNN and a second state prior distribution model, and the second state estimation DNN is a DNN for estimating a posterior probability of a corresponding HMM state number at each time from a language feature. The second state prior distribution model is a model that holds the value of the prior distribution for each HMM state.
State series estimation model learning method.   A program for causing a computer to function as the command sequence estimation device according to claim 1 or the state series estimation model learning device according to claim 2 or claim 3.

Images