JP6618453B2 - Database generation apparatus, generation method, speech synthesis apparatus, and program for speech synthesis - Google Patents

Description

  The present invention relates to a database generation technique used in a speech synthesizer.

  Non-Patent Document 1 discloses speech synthesis based on a Hidden Markov Model (HMM). In the speech synthesis based on the HMM, modeling based on the HMM is performed in advance for each phoneme (which is different from the phoneme defined by phonetic definition) based on the learning data (hereinafter, the phoneme based on the HMM). This model is simply called a phoneme model.) A database is created. FIG. 1 shows an HMM of a phoneme. In FIG. 1, modeling is performed as an HMM in five states S1 to S5, and speech synthesis parameters determined in advance by learning are output in each state. For speech synthesis, for example, the speech synthesizer determines the arrangement of phonemes from input text information, and the next state after the final state of the HMM of the preceding phonemic model is the first of the HMMs of the subsequent phonemic model. The HMM is connected so as to be in the state, and the HMM for one utterance is configured. Then, a speech signal is generated by controlling a filter parameter, a fundamental frequency of speech, and the like based on speech synthesis parameters output by each state of the HMM for one utterance. Note that the speech synthesis parameter output by the HMM state is given in the form of a probability density function. In general, an appropriate distribution (for example, a normal distribution) is assumed for the probability density function, and the HMM state includes parameters of the distribution (for example, one-dimensional normal distribution is average and variance, multi-dimensional normal distribution). If there are, the values of the mean vector and the variance-covariance matrix are stored. Hereinafter, this is referred to as a speech synthesis distribution parameter. That is, the speech synthesis parameter output by the HMM state is determined by a function having the speech synthesis distribution parameter as a variable. Hereinafter, this function is referred to as a speech synthesis parameter output function.

  This database contains decision trees. FIG. 2 shows a decision tree that is a binary tree included in the database. The circles (nodes of the decision tree) in FIG. 2 correspond to the rules for dividing the phoneme space, and these rules are called “questions”. The question is, for example, whether the central phoneme is / a / or whether it is the first state of the HMM, and the answer is “Yes” or “No”. The number of questions for constructing the decision tree is, for example, about several hundred to several thousand, and those for constructing the database are prepared in advance. A set of many questions prepared in advance is called a “question set”. The leaves of the decision tree (triangles in FIG. 2) correspond to the speech synthesis distribution parameters and are associated with the state of the HMM in FIG. As a result, for any type of phoneme, by applying the node questions in order from the root of the decision tree to the leaf, a corresponding set of distribution parameters for speech synthesis is determined, and this parameter determines each state of the phoneme model. A speech synthesis parameter output function corresponding to is determined. That is, any kind of phoneme model can be obtained from this database. Note that the number of leaves of the decision tree is usually smaller than the total number of HMM states of the phoneme model to be learned. For example, the number of phonemes included in the learning data is 50000, and when these phonemes are modeled by an HMM of 5 states, the total number of states of the phoneme model to be learned is 250,000. The number is usually less than 250,000. That is, the correspondence between one leaf of the decision tree and the state of the phoneme model to be learned is generally one-to-many, and a plurality of states of the phoneme model to be learned are represented by speech synthesis parameters. Share one probability density function for.

  The database generation process in the database generation apparatus is divided into two processes: a decision tree construction process and an HMM learning process. As the processing order, a decision tree construction process is performed, and subsequently, an HMM learning process based on the decision tree determined in the decision tree construction process is repeated a predetermined number of times to generate a database.

  The decision tree construction process is a process for selecting a question corresponding to each node of the decision tree in FIG. 2 from the questions in the question set. Specifically, the generating device places one cluster corresponding to the entire phoneme space as its initial state, and sequentially divides the cluster by the questions in the question set, and finally corresponds to the tree structure and its leaves. Determine which cluster to use. At that time, which question is applied at which position in the question set is determined by the evaluation value. The evaluation value is a log likelihood of the speech synthesis parameter output function associated with the state of the phoneme model corresponding to the cluster with respect to the learning data. Specifically, in the decision tree construction process, the generating device divides the cluster into two for each question in the question set for the cluster having the smallest evaluation value at that time, and evaluates two clusters. Find the sum of values. Then, the question having the maximum evaluation value is set as a question to be applied to the division. This process is repeated. For example, when the amount of change before and after the total evaluation value division is less than the threshold value, the division process is stopped. Thus, a speech synthesis distribution parameter that describes a decision tree having a question as a node and a speech synthesis parameter output function corresponding to a leaf is determined. The decision tree construction process is not stopped by the amount of change in the total evaluation value before and after the division, but the depth from the root in the tree structure (the number of divisions from the entire phoneme space until the cluster is generated) May be configured to stop the decision tree construction process when the upper limit is reached.

  The HMM learning process corresponds to the state transition probabilities between the HMM states of the phoneme model (250,000 states if there are 50000 phoneme models to be learned and each is a 5-state HMM), and each state of the HMM Is a process of adjusting the distribution parameter for speech synthesis based on the learning data. For example, a Baum-Welch algorithm is used. Also in this case, log likelihood is used as an evaluation criterion for the speech synthesis distribution parameter. However, since the probability density function is shared between different HMM states as described above, generally, one set of distribution parameters for speech synthesis is determined from a plurality of corresponding HMM states.

Takakatsu Yoshimura, Keiichi Tokuda, Takashi Masuko, Takao Kobayashi, Tadashi Kitamura, “Simultaneous Modeling of Spectrum, Pitch, and Duration in HMM-Based Speech Synthesis”, IEICE Transactions (D-II), J83-D -II, 11, pp. 2099-2107, November 2000

  In general, in order to perform natural speech synthesis, a database used for speech synthesis requires a phoneme model that is finely classified. For example, an audio signal corresponding to Japanese “a” is affected by the audio signals before and after that. That is, even when the same “A” is written on the characters, the sound signal waveform corresponding to “A” varies depending on the signals (phonemes) before and after that. Therefore, the phonemes in the database used for speech synthesis do not correspond one-on-one with the characters in Japanese notation, and even if the characters in Japanese notation are the same, they differ depending on the phonemes before and after that. Managed as phonemes.

  Further, in the generation of the database, a technique called connected learning using a speech signal generated continuously as speech information for learning is used. Here, in the connected learning, the part corresponding to a certain phoneme in the input speech signal is probabilistically probable to the HMMs connected in the order of phonemes without using the information on the start time and end time of each phoneme. Since it is determined as the position of a new HMM state, a time position shift may occur between the phoneme to be modeled by the HMM and the corresponding portion of the input speech signal waveform. FIG. 3A shows an audio signal uttered as / i /, / a / ("A" followed by "A"), and a phoneme preceding the phoneme / i + a / (/ a / recognized by the generator. The relationship between the section of (i / meaning) and the section of phoneme / ia / (meaning phoneme / a / following / i /) is shown. The number of states of the HMM is 5. In FIG. 3A, the rear part of the signal waveform of / i / is recognized as the head part of phoneme / ia /. Similarly, FIG. 3 (B) shows a speech signal uttered as “/ u /, / a / (“ a ”followed by“ a ””) and a phoneme / u + a / (/ a / ”recognized by the generator. The relationship between the phoneme / u / meaning) and the phoneme / u−a / (meaning phoneme / a / following / u /) is shown. In FIG. 3B, a part of the front side of the signal waveform of / a / is recognized as a rear end part of phoneme / u + a /. Such a phenomenon occurs because the boundary position between two phonemes is information (in the case of HMM learning) for voice signals uttered as / i /, / a /, / u /, / a /. That is, it is not given as a restriction at the time of HMM learning. In general, the temporal phoneme position in the speech signal is determined completely separately from the learning of the HMM by visual observation, listening, or another statistical method, and connected learning is performed rather than learning the HMM using it as a constraint. This is more advantageous because there are few restrictions from the viewpoint of modeling speech for learning by HMM. However, speech synthesis is usually a process of synthesizing speech in a phoneme order that is different from the speech for learning, and a phoneme model that includes time-positional deviations when viewed with another standard (for example, visual observation or listening). HMMs are connected, and speech of one utterance is synthesized based on the connected HMMs. Therefore, if the consistency between the phoneme type and the time position shift cannot be maintained as a result of the connected learning, the quality of synthesized speech deteriorates.

  The above problem is more likely to occur as the number of phoneme spaces divided in the decision tree is increased as compared to the amount of learning data used for HMM learning. This is because, for example, the kana in Japanese notation is the same but the phoneme classification in the speech synthesis is different, so there is no sharing of the probability density function, and the common feature between these phonemes ( That is, each phoneme model is optimized independently without taking into account the phoneme space macroscopic distribution of the same phonetic kana in the phoneme space). However, even if the number of divisions of the phoneme space is reduced in order to prevent the above problem, the fine features of each phoneme cannot be expressed by the HMM, and the quality of synthesized speech deteriorates.

  The present invention provides a technique for generating a database that improves the quality of synthesized speech.

  According to one aspect of the present invention, a database generation device used in a speech synthesizer that synthesizes speech based on a plurality of phoneme models, each phoneme model having one or more internal states, each internal state Is associated with a speech synthesis parameter output function, and the generating device determines, based on learning data and a question of a question set prepared in advance, a node whose node corresponds to any question of the question set. A construction means for constructing a tree, wherein each of the leaves of the first decision tree corresponds to a speech synthesis parameter output function generated based on the learning data, and a plurality of based on the learning data. Selecting means for selecting one of the speech feature parameter output functions; and a speech synthesis parameter corresponding to each leaf of the first decision tree based on the learning data. Adjustment means for performing adjustment processing for adjusting a meter output function and a voice feature parameter output function selected by the selection means, wherein the adjustment means is configured to adjust each of the first decision trees in the adjustment processing. A speech synthesis parameter output function corresponding to a leaf; a speech feature parameter output function selected by the selection means; a likelihood of the speech synthesis parameter output function for the learning data; and the learning data of the speech feature parameter output function. It adjusts based on the likelihood with respect to and the weighted sum of it.

  According to the present invention, a database that improves the quality of synthesized speech can be generated.

The figure which shows HMM. The figure which shows a decision tree. Explanatory drawing of the subject by connection learning. The figure which shows the question set by one Embodiment. The figure which shows HMM by one Embodiment. The block diagram of the production | generation apparatus by one Embodiment.

  Hereinafter, exemplary embodiments of the present invention will be described with reference to the drawings. In addition, the following embodiment is an illustration and does not limit this invention to the content of embodiment. In the following drawings, components that are not necessary for the description of the embodiments are omitted from the drawings.

  In the generation apparatus according to the present invention, as in the conventional generation apparatus, the database is generated by repeating the decision tree construction process and the HMM learning process a predetermined number of times. However, in this embodiment, two question sets are provided as shown in FIG. Here, the questions in the second question set are prepared in such a manner that less phoneme spaces are divided compared to the questions in the first question set. Specifically, the questions in the second question set are less likely to divide only the leaves connected to a specific node during the decision tree construction process compared to the questions in the first question set. Be prepared. In the decision tree construction process, the generation device constructs a decision tree based on the first question set (first decision tree) and a decision tree based on the second question set (second decision tree). Since the phoneme space cannot be finely divided in the second question set, the number of phoneme spaces divided by the second decision tree is usually smaller than the number of phoneme spaces divided by the first decision tree. That is, the number of leaves of the second decision tree is smaller than the number of leaves of the first decision tree. The leaves of the first decision tree and the second decision tree respectively correspond to the speech synthesis parameter distribution, as in FIG. In the following description, the speech synthesis parameter corresponding to the leaf of the second decision tree is referred to as a speech feature parameter for distinction, and the speech feature parameter distribution parameter corresponds to the leaf of the first decision tree. In order to distinguish from the distribution parameters for speech synthesis, they are called “adjustment distribution parameters”.

  Therefore, as shown in FIG. 5, in this embodiment, the distribution parameter for speech synthesis and the distribution parameter for adjustment are respectively associated with one state of the HMM. In the HMM learning process, the generation apparatus according to the present embodiment uses a logarithmic likelihood A of a speech synthesis parameter output function determined by a speech synthesis distribution parameter for learning data and a speech feature parameter output function determined by an adjustment distribution parameter for learning data. HMM learning is performed so that nA + mB, which is a weighted sum, is maximized. Here, n and m are weights, respectively, n + m = 1, and m is a value greater than 0 and less than 1. If m is 0, it is the same as the conventional HMM learning process using only the first decision tree, and if n is 0, it is the same as the conventional HMM learning using only the second decision tree.

  The generation apparatus performs the HMM learning process in this manner, and stores the first decision tree and the speech synthesis distribution parameters corresponding to the leaves of the first decision tree in the database. Then, the speech synthesizer performs speech synthesis as in the conventional case. That is, in the present invention, the second decision tree and the adjustment distribution parameter are not used for speech synthesis.

  As described above, for accurate speech synthesis, it is necessary to express the detailed features of the corresponding phoneme by the HMM of the phoneme model. For this purpose, the number of phoneme space divisions using a large first decision tree Need to be more. However, when the large first decision tree is used, the problem described with reference to FIG. 3 occurs. On the other hand, when HMM learning is performed using only a small second decision tree, the number of phoneme space divisions is reduced and the synthesized speech quality deteriorates. In this embodiment, a large first decision tree and a second decision tree smaller than the first decision tree are used, and the HMM learning process takes into account the distribution of speech synthesis parameters corresponding to the leaves of the two decision trees. Then, a speech synthesis distribution parameter (speech synthesis parameter distribution parameter corresponding to the leaf of the first decision tree) used in speech synthesis is determined. As a result, in the HMM concatenation learning, the shared structure of the probability density function determined by the second decision tree smaller than the first decision tree is also taken into consideration, and thus only the first decision tree that is large in the conventional concatenation learning is used. It is possible to suppress a time positional shift of the phoneme modeling target by the HMM, which has occurred in the method. Further, since the number of divisions of the phoneme space is determined by the first decision tree, it is possible to suppress deterioration of the synthesized speech quality.

  In the above description, the first question set and the second question set are different from each other. However, only one first question set may be used. In this case, the stop condition for the construction of the first decision tree by the first question set, that is, the threshold value of the change amount of the log likelihood before and after the division is smaller than the stop condition for the construction of the second decision tree by the first question set. Therefore, the construction process of the second decision tree is stopped earlier than the construction process of the first decision tree. In this case, the first decision tree is obtained by further extending the leaves of the second decision tree in place of the nodes.

  Instead of constructing the second decision tree by the second question set, the second decision tree can be configured to be determined in advance by the user. For example, all phonemes are classified into a total of six types of five vowels (/ a /, / i /, / u /, / e /, / o /) and other phonemes, It is also possible to use a second decision tree composed of six leaves corresponding to all states of the phoneme model HMM. As a result, it is possible to prevent the HMM corresponding to the vowel from being influenced by the phonemes adjacent to each other at the beginning and the end and becoming different for each phoneme model. Alternatively, in the case of using five-state HMMs per phoneme, each of the five vowels corresponds to the second to fourth states of the HMM (that is, 1 corresponding to the second to fourth states of / a /). 5 leaves) (one leaf, one leaf corresponding to the second to fourth states of / i /) and one leaf corresponding to all other states of the HMM, a total of 6 A second decision tree consisting of two leaves can also be used. As a result, the influence of adjacent phonemes can be concentrated in the first state and the fifth state, and can be prevented from affecting the second state to the fourth state. Furthermore, in speech synthesis by HMM, silence is also treated as one type of phoneme, but the leaf is such that the entire state of the HMM of the silent phoneme model and the entire state of the HMM of the other phonemic model are leaves. Can also use two second decision trees. As a result, it is possible to prevent a phenomenon in which the speech boundary before and after the utterance is shifted in time position due to the connected learning and the silent HMM corresponds to a speech section that is not silent.

  In the above description, the decision tree determines the parameters of the distribution of the speech synthesis parameters for each state of the HMM of the phoneme model from the phoneme by the decision tree, but the present invention is not limited to this. It is only necessary to determine the probability density function of the speech synthesis parameter of each state of the HMM corresponding to each phoneme, and the determination method is not limited to the method using a decision tree. For example, as described above, in the case where there are only five types of vowels and a total of six distributions, the HMM of the corresponding phoneme model can be applied to any type of phonemes without using the second decision tree. The distribution of each state can be easily determined. The same applies to the first decision tree.

  The types of parameters that are actually output by the speech synthesis parameter output function determined by the speech synthesis distribution parameter determined by the first decision tree and the speech feature parameter output function determined by the adjustment distribution parameter determined by the second decision tree are as follows: The same type or different types may be used. As an example of using different types of parameters, in modeling the spectral characteristics of speech, the distribution determined by the speech synthesis distribution parameters is targeted for line spectrum pair (LSP) coefficients widely used in speech synthesis and speech coding. The distribution determined by the adjustment distribution parameter may be a cepstrum coefficient widely used in speech recognition. Since the speech feature parameter output function that is determined by the distribution parameter for adjustment is not used in speech synthesis, it is possible to learn HMM considering parameters that are difficult to use for speech synthesis and parameter distribution functions (for example, mixed normal distribution). It is.

  Furthermore, the present invention is not limited to a speech synthesizer based on HMM in a strict sense, but includes speech synthesis based on a model similar to HMM. For example, for a hidden semi-Markov model (HSMM), which is a model in which the distribution of the self-transition probability regarding the number of staying hours is added to the HMM as a constraint, the speech synthesis based on this HSMM is similar to the series of methods based on the previous explanation It is applicable to. More generally, the present invention is directed to a speech synthesis scheme based on a model of a structure in which each phoneme model has one or more internal states, and each internal state is associated with a speech synthesis parameter output function. can do.

  FIG. 6 is a configuration diagram of the generation apparatus according to the present embodiment. In the question set holding unit 3, a question set including a plurality of questions prepared in advance is prepared. As described above, the question set for constructing the first decision tree and the question set for constructing the second decision tree may be the same or different. Moreover, the question set for constructing the first decision tree and the question set for constructing the second decision tree may share the same question partially.

  The decision tree construction unit 1 performs the above-described decision tree construction process to construct a first decision tree and a second decision tree. In addition, the learning processing unit 2 performs the above-described HMM learning processing, and performs speech synthesis parameter adjustment processing corresponding to each leaf of the first decision tree. The decision tree construction process and the subsequent HMM learning process are performed a predetermined number of times, and when the learning processing unit 2 performs the last HMM learning process, the first decision tree and the first decision tree at that time The speech synthesis parameter corresponding to the leaf is stored in the database 4. A speech synthesizer (not shown) performs speech synthesis using information stored in the database 4.

  The decision tree construction unit 1 may be configured to construct only the first decision tree, and the user decides and sets the second decision tree in the decision tree construction unit 1.

  The generation apparatus according to the present invention can be realized by a program that causes a computer to operate as the generation apparatus. These computer programs can be stored in a computer-readable storage medium or distributed via a network.

  1: Decision tree construction unit, 2: Learning processing unit, 3: Question set holding unit, 4: Database

Claims (13)

A database generator for use in a speech synthesizer that synthesizes speech based on a plurality of phoneme models,
Each phoneme model has one or more internal states, each internal state being associated with a speech synthesis parameter output function,
The generator is
Based on learning data and a question of a question set prepared in advance, the node is a construction means for constructing a first decision tree corresponding to any question of the question set, the leaf of the first decision tree Each of the construction means corresponding to a speech synthesis parameter output function generated based on the learning data;
Selecting means for selecting one of a plurality of speech feature parameter output functions based on the learning data;
Adjustment means for performing adjustment processing for adjusting a speech synthesis parameter output function corresponding to each leaf of the first decision tree based on the learning data and a speech feature parameter output function selected by the selection means; ,
In the adjustment process, the adjustment unit is configured to convert a speech synthesis parameter output function corresponding to each leaf of the first decision tree and a speech feature parameter output function selected by the selection unit into the speech synthesis parameter output function. A generation apparatus that adjusts based on a weighted sum of likelihood for learning data and likelihood of the speech feature parameter output function for the learning data. Each of the speech synthesis parameter output functions corresponding to the leaves of the first decision tree corresponds to one or more internal states of each phoneme model,
2. The generation apparatus according to claim 1, wherein each of the speech feature parameter output functions selected by the selection unit corresponds to one or more internal states of each phoneme model. The selecting means selects a speech feature parameter output function based on a second decision tree having a smaller number of leaves than the first decision tree;
The construction means constructs the second decision tree used by the selection means, whose node corresponds to any question of the question set based on the learning data and the questions of the prepared question set. The generating apparatus according to claim 1 or 2, wherein   The selection means selects each of a plurality of vowel phoneme models and a speech feature parameter output function of one phoneme model corresponding to other than the plurality of vowels. The generating device described in 1.   4. The generating apparatus according to claim 1, wherein the selection unit selects a phoneme model of silence and a speech feature parameter output function of a phoneme model other than the silence. 5.   The adjustment unit stores the first decision tree and the speech synthesis parameter output function after the adjustment process corresponding to each leaf of the first decision tree in a database. The generation device according to any one of 5. A database generator for use in a speech synthesizer that synthesizes speech based on a plurality of phoneme models,
Each phoneme model has one or more internal states, each internal state being associated with a speech synthesis parameter output function,
The generator is
First selection means for selecting one of a plurality of speech synthesis parameter output functions based on the learning data;
Second selection means for selecting one of a plurality of speech feature parameter output functions based on the learning data;
Adjusting means for adjusting a speech synthesis parameter output function selected by the first selection means based on the learning data and a speech feature parameter output function selected by the second selection means;
In the adjustment process, the adjustment unit is configured to convert the speech synthesis parameter output function selected by the first selection unit and the speech feature parameter output function selected by the second selection unit into the learning of the speech synthesis parameter output function. A generation apparatus that adjusts based on a weighted sum of likelihood for data and likelihood of the speech feature parameter output function for the learning data.   8. The generating apparatus according to claim 7, wherein the number of speech feature parameter output functions selected by the first selection unit is greater than the number of speech feature parameter output functions selected by the second selection unit.   9. The generating apparatus according to claim 7, wherein the adjustment unit stores a speech synthesis parameter output function after the adjustment processing of the speech synthesis parameter output function selected by the first selection unit in a database.   The speech synthesis parameter output function after the adjustment processing corresponding to each of the first leaves of the first decision tree and the plurality of first leaves of the first decision tree stored in the database of the generation device according to claim 6. A speech synthesizer that performs speech synthesis based on the above.   The speech synthesis is performed based on the speech synthesis parameter output function after the adjustment processing of the speech synthesis parameter output function selected by the first selection unit, which is stored in the database of the generation device according to claim 9. A speech synthesizer.   A program that causes a computer to function as the generation device according to claim 1. A generation method for generating in a computer a database used in a speech synthesizer that synthesizes speech based on a plurality of phoneme models,
Each phoneme model has one or more internal states, each internal state being associated with a speech synthesis parameter output function,
The generation method is:
Based on learning data and a question of a prepared question set, a construction step in which the node constructs a first decision tree corresponding to any question of the question set, the leaf of the first decision tree Each of the construction steps corresponding to a speech synthesis parameter output function generated based on the learning data;
A selection step of selecting one of a plurality of speech feature parameter output functions based on the learning data;
Adjusting a speech synthesis parameter output function corresponding to each leaf of the first decision tree based on the learning data and a speech feature parameter output function selected in the selection step;
In the adjustment step, the speech synthesis parameter output function corresponding to the leaf of the first decision tree and the speech feature parameter output function selected in the selection step are represented by the likelihood of the speech synthesis parameter output function for the learning data. And a step of adjusting based on a weighted sum of the speech feature parameter output function and the likelihood of the learning data with respect to the learning data.

Images