JP2017194510A - Acoustic model learning device, voice synthesis device, methods therefor and programs - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an acoustic model learning technology for learning an acoustic model for enabling voice synthesis taking a speaking intention into account, and a technology capable of performing voice synthesis taking the speaking intention into account.SOLUTION: An acoustic model learning device comprises: a context data storage part 11 storing context data; a language feature amount vector extraction part 13 for extracting a language feature amount vector of each of context data while using each of context data read from the context data storage part; an intention information vector storage part 16 storing an intention information vector representing a speaking intention of each of context data; and an acoustic model learning part 17 for generating an acoustic model by performing acoustic model learning while using the extracted language feature amount vector of each of context data, voice data corresponding to each of context data and the intention information vector of each of context data read from the intention information vector storage part.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a voice synthesis technique and a technique for learning an acoustic model used for voice synthesis.

  There is a technique based on DNN (Deep Neural Network) as a technique for learning a speech synthesis model from speech data and generating synthesized speech (see, for example, Non-Patent Document 1). An outline of this technique is shown in FIGS.

  Conventionally, as exemplified in FIGS. 13 to 14, a DNN acoustic model has been learned based on speech data and a language feature vector generated based on context data. Also, speech parameters were generated from the context obtained by text analysis of the text to be synthesized and the learned DNN acoustic model, and synthesized speech waveforms were obtained from the obtained speech parameters by generating speech waveforms. .

Zen et al., "Statistical parametric speech synthesis using deep neural networks", Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on.IEEE, 2013, pp. 7962-7966.

  However, in the case of human utterance, the intention may be transmitted by uttering properly using prosody etc. according to the intention, instead of uttering only according to information such as reading of text to be uttered and accents. On the other hand, in conventional speech synthesis, speech parameters are generated based only on context information such as reading of text to be synthesized and accents, and the intention is not taken into consideration. Therefore, in the conventional speech synthesis, speech that is incompatible with the intention corresponding to the text is synthesized, and there is a possibility that misunderstanding occurs or the synthesized speech is felt unnatural.

  An object of the present invention is to provide an acoustic model learning device that learns an acoustic model for enabling speech synthesis in consideration of speech intention, a speech synthesis device that enables speech synthesis in consideration of speech intention, and a method and a program thereof. Is to provide.

  An acoustic model learning device according to an aspect of the present invention extracts a language feature vector of each context data using a context data storage unit storing each context data and each context data read from the context data storage unit Language feature vector extraction unit, intention information vector storage unit storing intention information vectors representing utterance intention of each context data, and corresponding language feature vector and each context data of each extracted context data An acoustic model learning unit that generates an acoustic model by performing acoustic model learning using the voice data to be performed and the intention information vector of each context data read from the intention information vector storage unit.

  A speech synthesizer according to an aspect of the present invention includes a text analysis unit that analyzes input text to obtain a context, a language feature vector extraction unit that extracts a language feature vector of each context, and an input utterance intention A speech parameter generation unit that generates speech parameters using the intention information vector representing the speech model, the acoustic model generated by the acoustic model learning device according to claim 1, and the extracted language feature vector; and the generated speech A speech waveform generation unit that generates synthesized speech using parameters.

  It is possible to learn an acoustic model for enabling speech synthesis in consideration of the utterance intention. Speech synthesis can be performed in consideration of the utterance intention.

The block diagram for demonstrating the example of the acoustic model learning apparatus of 1st embodiment. The flowchart for demonstrating the example of the acoustic model learning method. The flowchart for demonstrating the example of a process of the intention information vector preparation part 15. FIG. The block diagram for demonstrating the example of the speech synthesizer of 1st embodiment. The flowchart for demonstrating the example of the speech synthesis method. The block diagram for demonstrating the example of the acoustic model learning apparatus of 2nd embodiment. The block diagram for demonstrating the example of the intention class learning part 19. FIG. The block diagram for demonstrating the example of the speech synthesizer of 2nd embodiment. The block diagram for demonstrating the example of the acoustic model learning apparatus of 3rd embodiment. The block diagram for demonstrating the example of the speech synthesizer of 3rd embodiment. The flowchart for demonstrating the example of the speech synthesis method of 3rd embodiment. The flowchart for demonstrating the example of the speech synthesis method of 4th embodiment. The block diagram for demonstrating the example of the conventional acoustic model learning apparatus. The block diagram for demonstrating the example of a speech synthesizer.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[First embodiment]
(Acoustic model learning apparatus and method)
As illustrated in FIG. 1, the acoustic model learning device according to the first embodiment includes a context data storage unit 11, a voice data storage unit 12, a language feature vector extraction unit 13, an intention data storage unit 14, and an intention information vector creation unit. 15, an intention information vector storage unit 16, an acoustic model learning unit 17, and an acoustic model storage unit 18 are provided.

  The acoustic model learning method according to the first embodiment is realized by causing each unit of the acoustic model learning device to execute the processes of steps S13 to S17 described below with reference to FIG.

  The acoustic model learning apparatus and method learns an acoustic model using speech data, context data, and intention information corresponding to the intention of each utterance.

<Context data storage unit 11>
Each context data is stored in the context data storage unit 11. The total number of context data stored in the context data storage unit 11 is I, for example, where I is a positive integer. The context data is information such as pronunciation given to each utterance in the voice data stored in the voice data storage unit 12. One context data is given to each utterance in the voice data. The context data includes phoneme information (pronunciation information) and accent information (accent type, accent phrase length), for example. In addition to this, the part of speech information may be included in the context data.

<Audio data storage unit 12>
The voice data storage unit 12 stores voice data used for acoustic model learning. This audio data is data such as audio parameters (pitch parameters (basic frequency (F0), etc.), spectrum parameters (cepstrum, mel cepstrum, etc.)) obtained as a result of performing signal processing on the audio signal, for example. .

  When the total number of context data stored in the context data storage unit 11 is I, I audio data respectively corresponding to the I context data is stored in the audio data storage unit 12.

<Language feature vector extraction unit 13>
The language feature vector extraction unit 13 extracts the language feature vector of each context data using each context data read from the context data storage unit 11 (step S13). The extracted language feature vector data is output to the acoustic model learning unit 17.

  The language feature vector represents context data as a numerical vector. For example, as in Non-Patent Document 1, phoneme information and accent information are each expressed in 1-of-K, and are further set as numerical vectors obtained by concatenating numerical information such as sentence length.

<Intent data storage unit 14>
It is assumed that intention data is stored in the intention data storage unit 14. The intention data is data that holds intention information given to each utterance included in the voice data and context data.

The intention information is information that is given to each utterance and represents the intention of the utterance. Assume that N is an integer of 2 or more, and all intention information is composed of N types, and one intention information is associated with one utterance. N types of intention information are represented by {c ₁ , c ₂ ,..., C _N }. For example, using the dialogue action information consisting of all 33 types (N = 33) in Table 6 of Reference 1, {c ₁ = "greeting", c ₂ = "information provision", ..., c ₃₃ = "other" } Is a character string corresponding to each dialogue act.

  [Reference 1] Toyomi Meguro, et al. "Analysis of listening dialogue and construction of dialogue control unit based on analysis.", Transactions of Information Processing Society of Japan, 53.12, 2012, pp.2787-2801.

The intention data is expressed, for example, as {d ₁ , d ₂ ,..., D _I } using the total number of utterances I in the voice data and context data. When the intended information one each for each utterance corresponds, for example sentence number i n th intention information about speech corresponding configured by a d _i = c _n.

<Intention information vector creation unit 15>
The intention information vector creation unit 15 uses the intention data read from the intention data storage unit 14 to create an intention information vector representing the utterance intention of each context data (step S15). The created intention information vector is stored in the intention information vector storage unit 16.

The intention information vector is a representation of each intention information as a K-dimensional numerical vector, where K is a positive integer. The intention information vector v _i is determined based on d _i for each context data i (i = 1, 2,..., I: I is the total number of context data). Intention information vector data, which is a set of intention information vectors corresponding to each context data, is expressed as V = {v ₁ , v ₂ ,..., V _I }.

For example, the intention information vector creation unit 15 sets the dimension of the intention information vector v _i corresponding to the context data i to N (K = N), and v _i = [v ₁ ⁱ , v ₂ ⁱ ,..., V _N ⁱ ]. When expressed, the intention information vector v corresponding to the context data i can be obtained by using the following 1-of-K representation of the intention information for the input of the intention information c _n corresponding to the context data i. Create _i .

  Here, the index expressing the dimension of the intention information vector is set to n ′ = 1, 2,.

  An example of processing of the intention information vector creation unit 15 is shown in FIG.

<Intention information vector storage unit 16>
The intention information vector storage unit 16 stores an intention information vector representing the utterance intention of each context data.

  The intention information vector is created by the intention information vector data creation unit 16 and stored in the intention information vector storage unit 16 as described above. The creation of the intention information vector may be performed in advance before the acoustic model learning process is performed.

<Acoustic model learning unit 17>
The acoustic model learning unit 17 stores a language feature vector of each context data extracted by the language feature vector extraction unit 13, voice data corresponding to each context data read from the voice data storage unit 12, and intention information vector storage. An acoustic model is generated by performing acoustic model learning using the intention information vector of each context data read from the unit 16 (step S17). The audio data corresponding to each context data read from the audio data storage unit 12 is, for example, an audio parameter. The generated acoustic model is stored in the acoustic model storage unit 18.

  The acoustic model learning apparatus and method are different from conventional ones in that an intention information vector is used when performing acoustic model learning.

  For example, acoustic model learning is performed from speech data, language feature vector data, and intention information vector data, and a DNN acoustic model that receives a language feature vector and intention information vector and outputs corresponding speech parameters is learned. Regarding the configuration of the DNN acoustic model, the intention information vector may be simply connected to the language feature vector and utilized as the DNN input vector. Alternatively, the intention information vector may be input to one or more intermediate layers of the DNN and learned by a configuration similar to the model, such as the model of Reference 2 in the speech recognition field. As for the learning algorithm, a conventional general DNN learning algorithm such as error back-propagation and stochastic gradient descent can be used as in Non-Patent Document 1.

[Reference 2]
Xue, Shaofei, et al. "Fast adaptation of deep neural network based on discriminant codes for speech recognition.", Audio, Speech, and Language Processing, IEEE / ACM Transactions on 22.12 (2014), pp.1713-1725.

(Speech synthesizer and method)
As illustrated in FIG. 4, the speech synthesizer of the first embodiment includes a text analysis unit 21, a language feature vector extraction unit 22, an intention information vector creation unit 23, a speech parameter generation unit 24, and a speech waveform generation unit 25. I have.

  The speech synthesis method according to the first embodiment is realized by causing each unit of the speech synthesizer to execute the processes of steps S21 to S25 described below with reference to FIG.

  The speech synthesizer and method obtains synthesized speech from an input text, intention information corresponding to the input text, an acoustic model, and an acoustic model obtained by the acoustic model learning unit 17.

  In the speech synthesis apparatus and method, synthesized speech is generated from text to be synthesized and intention information. An example of the processing procedure is as follows.

  The intention information is designated by the user and input by an input means such as a keyboard or a mouse. An intention information estimator may be prepared in advance, automatically estimated from the input text, and input. Moreover, you may input based on the information which can be acquired from other systems, such as using the dialog action information obtained from the dialog system using the technique of the reference document 1 as intention information. The intention information vector extraction used in the speech synthesis apparatus and method is assumed to be the same as the intention information vector extraction used in the acoustic model learning apparatus and method.

<Text analysis unit 21>
The text analysis unit 21 performs text analysis on the input text and obtains a context that is information such as reading of the synthesized text and accents (step S21). The obtained context is output to the language feature vector extraction unit 22.

<Language feature vector extraction unit 22>
The language feature vector extraction unit 22 extracts a language feature vector corresponding to the input context (step S22). The extracted language feature vector is output to the speech parameter generation unit 24.

  Since the process of the language feature vector extraction unit 22 is the same as the process of the language feature vector extraction unit 13, duplicate description is omitted here.

<Intention information vector creation unit 23>
Intention information vector generating unit 23 generates the intention information vector corresponding to the intended information c _n input (step S23). The created intention information vector is output to the voice parameter generation unit 24.

  Since the process of the intention information vector creating unit 23 is the same as the process of the intention information vector creating unit 15, duplicate description is omitted here.

<Acoustic model storage unit 18>
The acoustic model storage unit 18 stores an acoustic model generated by the acoustic model learning apparatus and method.

<Audio parameter generation unit 24>
The speech parameter generation unit 24 includes the language feature vector obtained by the language feature vector extraction unit 22, the intention information vector created by the intention information vector creation unit 23, and the acoustic model read from the acoustic model storage unit 18. Is used to generate voice parameters (step S24). The generated speech parameter is output to the speech waveform generation unit 25.

  For example, the speech parameter generation unit 24 inputs a language feature vector and an intention information vector to the acoustic model, and generates a speech parameter by forward propagation.

<Audio waveform generation unit 25>
The voice waveform generation unit 25 obtains synthesized voice by voice waveform generation from the voice parameters generated by the voice parameter generation unit 24 (step S25).

  Prior to speech waveform generation, for example, a speech parameter sequence smoothed in the time direction may be obtained using a maximum likelihood generation (MLPG) algorithm (see, for example, Reference 3). Further, for example, a technique described in Reference Document 4 may be used for generating a speech waveform.

[Reference 3] Masuko et al., "HMM-based speech synthesis using dynamic features", IEICE, vol.J79-D-II, no.12, pp.2184-2190, Dec. 1996.
[Reference 4] Imai et al., “Mel Log Spectrum Approximation (MLSA) Filter for Speech Synthesis”, IEICE Transactions A Vol.J66-A No.2 pp.122-129, Feb. 1983.

  Thus, the intention information corresponding to the intention of each context is utilized. That is, as an input to the speech synthesizer, in addition to conventional contexts such as reading and accent, the acoustic model is configured to utilize intention information and output speech parameters reflecting the corresponding intention information. Thereby, it is possible to reflect the tendency of the voice parameter corresponding to each intention in the voice parameter generated from the acoustic model. In this way, by synthesizing speech that matches the intention of the text to be synthesized, it is possible to prevent the misunderstanding or the unnatural feeling of the synthesized speech from being expressed by the speech.

[Second Embodiment]
In the first embodiment, a plurality of pieces of intention information may correspond to similar voice expression. For example, when the dialogue action information as in Reference 1 is used as the intention information, the dialogue action such as information provision, self-disclosure_facts, etc. does not express the voice strongly, and the voice close to the normal reading tone is obtained. There is a possibility of being uttered. Therefore, in the first embodiment, a classification having an excessively large number of classes may be used for expressing an intention by voice. As the number of classes increases, the number of dimensions of the input context increases, and the number of parameters of the acoustic model (eg, DNN acoustic model) increases. In general, an acoustic model with a large number of parameters is likely to cause overlearning of learning data, which lowers the quality of synthesized speech or reduces the expressiveness of intentional expression by speech. Alternatively, a large amount of speech data and context data is required to obtain sufficient synthesized speech quality and expressive power of intention expression by speech, and the cost for learning the speech synthesizer and method increases.

  Therefore, in the second embodiment, for example, intention information clustering is performed based on voice parameters to obtain intention class information. By expressing multiple intentions with similar voice parameter trends in one intention class and using them as contexts, learning with an acoustic model with a small number of parameters prevents over-learning and improves the quality of synthesized speech. Increase the expressive power of intention expression. In addition, the speech synthesizer can be learned from a small amount of data, and the cost is reduced.

  Hereinafter, parts different from the first embodiment will be mainly described. A duplicate description of the same parts as in the first embodiment is omitted.

(Acoustic model learning apparatus and method)
The acoustic model learning apparatus according to the second embodiment further includes an intention class learning unit 19 and an intention class classification information storage unit 110 as illustrated in FIG. The intention class learning unit 19 includes, for example, an intention feature vector extraction unit 191 and an intention clustering unit 192, as illustrated in FIG.

<Intention feature vector extraction unit 191>
For each intention information, the intention feature vector extraction unit 191 obtains an intention feature vector representing the characteristics of the intention information from the speech data of the corresponding utterance. The obtained intention feature vector is output to the intention clustering unit 192.

For example, as the intention feature vector, first, F0, speech rate, and power average / standard deviation of each intention information are obtained and used as the intention feature vector. At this time, the intention information _{c n (n = 1,2, ...} , N: N is the total intended number) intended feature vector w _n of is defined as follows, for example.

Here, mnF0 _n, stdF0 _n is F0 mean and standard deviation of the respective intention information c _n, mnPow _n, mean and standard deviation of the power of each StdPow _n intention information c _n, mnSr _n, stdSr _n are each the average value of the utterance speed of the intention information c _n and the standard deviation. Alternatively, a spectral feature quantity such as a cepstrum feature quantity may be used as the intention feature vector. In addition, for F0 of the ending one mora, statistical processing regarding a specific time interval may be performed instead of the entire utterance such as calculating and using the average / standard deviation of the time difference coefficient, and may be used as the intention feature vector.

<Intention clustering unit 192>
The intention clustering unit 192 uses the N intention feature vectors obtained by the intention feature vector extraction unit 191 to perform clustering so as to be divided into arbitrary M (M is an integer of 2 or more and less than N). Get intention class classification information. The obtained intention class classification information is stored in the intention class classification information storage unit 110.

  As the clustering algorithm, a general clustering algorithm such as k-means method or hierarchical clustering can be used.

The intention class classification information is information regarding which intention class information each intention information belongs to. For example, when each intention information c _n (n = 1, 2,..., N: N is the total number of intention information) is clustered into intention class information e _in (1 ≦ i _n ≦ M), its index is listed. Data of the format is held as I = [i ₁ , i ₂ ,..., I _N ]. “In” _in “e _in ” means “i _n ” which is a subscript n of i. This data I is an example of intention class classification information.

The intention class information is information representing the result of clustering the intention information. When the total number of classes is M (M is an integer of 2 or more and less than N), for example, {e ₁ , e ₂ , ..., e _M } It is expressed as follows.

<Intention information vector creation unit 15>
When an intention information is input, the intention information vector creation unit 15 of the second embodiment outputs a corresponding intention information vector based on the intention class classification information.

First, the intention information vector generating unit 15, when the intended information c _n corresponding to a certain context data is input, based on the intended classification information, obtain intention class information e _in corresponding to the intention information c _n input . The point which outputs intention class information based on this intention class classification information is a different part from 1st embodiment.

Then, the intention information vector creation unit 15 outputs an intention information vector v _i corresponding to the intention class information e _in .

For example, as in the first embodiment, the dimension of the intention information vector v _i corresponding to the context data i is M (K = M), and v _i = [v ₁ ⁱ , v ₂ ⁱ ,..., V _M ⁱ ] when expressed as intention information to the input of the intention class information e _m corresponding to the context data i, by using the 1-of-K expression intent information as described below, corresponding to the context data i Create vector v _i .

  Here, the index expressing the dimension of the intention information vector is m ′ = 1, 2,.

  Thus, in the second embodiment, the intention information vector corresponding to each context data is a vector representing the intention class information corresponding to the utterance intention of each context data.

  Other processes of the acoustic model learning apparatus and method are the same as those in the first embodiment.

(Speech synthesizer and method)
The acoustic model learning device of the second embodiment further includes an intention class classification information storage unit 110 as illustrated in FIG.

  The intention information vector creation unit 23 performs the same processing as the intention information vector creation unit 15 of the second embodiment.

In other words, the intention information vector generating unit 23 of the second embodiment performs the intended information d _i for each utterance, the intention information vector extracted from the intended classification information read from the intended classification information storage unit 110, intended information vector v _i is output.

  Other processes of the speech synthesis apparatus and method are the same as those in the first embodiment.

[Third embodiment]
In the second embodiment, in order to realize high expressive power of intention expression by speech, it is only necessary to learn an intention class that maximizes the likelihood of an acoustic model (for example, DNN acoustic model) and parameters of the DNN acoustic model. In the model learning of the second embodiment, the intention class learning unit in the previous stage determines the intention class of each utterance, and the acoustic model learning unit in the subsequent stage uses the intention class determined in the previous stage to calculate the likelihood of the acoustic model. The parameter of the acoustic model that maximizes is determined. However, since the intention class and acoustic model parameters are optimized in multiple stages, the obtained intention class and acoustic model parameters fall into a local solution, and the likelihood of the DNN acoustic model may not be sufficiently large. Therefore, there is a possibility that the expressive power of intention expression by voice cannot be sufficiently improved.

  Therefore, in the third embodiment, the intention class and acoustic model parameters that achieve a larger likelihood of the acoustic model are estimated by an algorithm that simultaneously learns the intention class and acoustic model parameters that maximize the acoustic model likelihood. It is possible to improve the expressive power of intention expression by voice.

  Hereinafter, the parts different from the first embodiment and the second embodiment will be mainly described. A duplicate description of the same parts as those in the first embodiment and the second embodiment is omitted.

(Acoustic model learning apparatus and method)
The acoustic model learning device of the third embodiment further includes an intention class determination unit 111 and a likelihood reference intention class classification information storage unit 112 as illustrated in FIG.

<Acoustic model learning unit 17>
The acoustic model learning unit 17 of the third embodiment reads the speech data read from the speech data storage unit 12, the language feature vector data extracted by the language feature vector extraction unit 13, and the intention information vector storage unit 16. From the intention information vector data, the intention class probability, which is the probability that each utterance intention belongs to each intention class, and the parameters of the acoustic model are simultaneously estimated, and the acoustic model and the intention class classification information are output (step S17). For example, a generalized EM (GEM) algorithm (see, for example, Reference 5) that uses intention class information corresponding to each intention information as a hidden variable and applies a gradient method to M steps of the EM algorithm is used. In the GEM algorithm, appropriate initial values are given for acoustic model parameters and intention class probabilities, and both are updated alternately.

  The intention class classification information obtained by the intention clustering unit 192 of the second embodiment may be used to output an intention information vector similar to that of the second embodiment, and may be used as the initial value of the intention class probability. Random numbers can be set as the initial values of the parameters of the acoustic model, as in Non-Patent Document 1.

  [Reference 5] Masami Miyakawa, "EM algorithm and its surroundings", Applied statistics 16.1, 1987, pp.1-21

  In this way, the acoustic model learning unit 17 reads the language feature vector of each context data extracted by the language feature vector extraction unit 13, the voice data corresponding to each context data, and the intention information vector storage unit 16. Using the intention information vector of each context data, the acoustic model and each utterance intention are assigned to each intention class by performing acoustic model learning based on the initial probability of each utterance intention belonging to each intention class. Probability of belonging to

  The generated acoustic model is stored in the acoustic model storage unit 18. The probability that each generated utterance intention belongs to each intention class is output to the intention class determination unit 111.

<Intention class determination unit 111>
The intention class determination unit 111 determines likelihood reference intention class classification information from the intention class probability (step S111, see FIG. 11). For example, for each intention information c _n (n = 1, 2,..., N: N is the total number of intentions), the index i _n = argmax _m p _nm of the intention class having the maximum intention class probability is output, and the list The format data is stored as I = [i ₁ , i ₂ ,..., I _N ].

The intention class probability is that each intention information c _n (n = 1, 2,..., N: N is the total number of intentions) is each intention class information e _m (m = 1, 2,..., M: M is all the intention classes. The probability p _nm belonging to (number).

  The determined likelihood criterion intention class classification information is stored in the likelihood criterion intention class classification information storage unit 112.

Likelihood reference intention class classification information is determined by intention class probability and intention class determination as described above. Similar to the intention class classification information, for example, each intention information c _n (n = 1,..., N: N is the total number of intentions) is intended class information e _in (1 ≦ i _n ≦ M: M is all intention classes) When the data is clustered into (number), the index is held as list format data I = [i ₁ , i ₂ ,..., I _N ].

  In this way, the intention class determination unit 111 determines the intention class that maximizes the probability that each utterance intention belongs to each intention class as the intention class to which each utterance intention belongs.

  Other processes of the acoustic model learning apparatus and method are the same as those in the first embodiment.

(Speech synthesizer and method)
As illustrated in FIG. 10, the speech synthesizer according to the third embodiment includes a likelihood reference intention class classification information storage unit 112.

<Intention information vector creation unit 23>
The intention information vector creation unit 23 of the third embodiment is not the intention class classification information read from the intention class classification information storage unit 110, but the likelihood reference intention class classification information read from the likelihood reference intention class classification information storage unit 112. Intention information vector extraction is performed using (step S23).

  Other processes of the speech synthesis apparatus and method are the same as those in the first embodiment.

[Fourth embodiment]
When an algorithm having an initial value dependency such as the GEM algorithm is used in the algorithm used in the acoustic model / intention class learning of the third embodiment, the likelihood of the acoustic model (for example, DNN acoustic model) is sufficiently increased. In order to sufficiently improve the expressiveness of intention expression by voice, it is preferable to set an appropriate initial value.

  Therefore, in the fourth embodiment, the initial value of the intention class probability based on the likelihood reference intention class reclassification information and the updating of the likelihood reference intention class reclassification information by intention class probability calculation and acoustic model learning are repeated. To do. Likelihood reference intention class reclassification information obtained by intention class probability calculation / acoustic model learning at each iteration step is learned based on a criterion that maximizes the likelihood of an acoustic model. It is expected that an acoustic model having a higher likelihood can be learned by setting the initial value of the probability and executing the intention class probability calculation / acoustic model learning again. Therefore, by repeatedly setting the initial value of the intention class probability based on the likelihood reference intention class reclassification information and updating the likelihood reference intention class reclassification information through intention class probability calculation and acoustic model learning, iteratively The likelihood of the acoustic model can be increased. Thereby, the expressive power of intention expression by voice is further improved.

  Hereinafter, parts different from the third embodiment will be mainly described. A duplicate description of the same parts as in the third embodiment is omitted.

(Acoustic model learning apparatus and method)
<Acoustic model learning unit 17>
The acoustic model learning unit 17 according to the fourth embodiment includes a language feature vector of each context data extracted by the language feature vector extraction unit 13, voice data corresponding to each context data, and an intention information vector storage unit 16. The acoustic model and each utterance intention are obtained by performing acoustic model learning based on the initial value of the probability that each predetermined utterance intention belongs to each intention class using the intention information vector of each read context data. A probability belonging to the intention class is generated (step S17).

  The probability that each generated utterance intention belongs to each intention class is output to the intention class determination unit 111. The intention class determination unit 111 determines the intention class to which each utterance intention belongs by the same method as that described in the third embodiment.

  In the fourth embodiment, the probability that each utterance intention belongs to the intention class to which each utterance intention belongs determined by the intention class determination unit 111 is 1, and the probability that each utterance intention belongs to other intention classes is 0. Then, the acoustic model learning unit 17 and the intention class determining unit 111 are repeatedly processed as an initial value of the probability that each predetermined speech intention in the acoustic model learning unit 17 belongs to each intention class.

The acoustic model likelihood data s is assumed to be data for recording the likelihood of the acoustic model at each step in which the acoustic model learning unit 17 repeats the setting of the initial value of the intention class probability and the acoustic model learning. For example, when the acoustic model likelihood in the j-th step is s _j , the acoustic model likelihood data s is as follows: s = [s ₁ , s ₂ , ..., s _J ] (J is the total number of steps) It is expressed in

  At this time, in order to perform repetitive processing, for example, the acoustic model learning unit 17 first initializes acoustic model likelihood data. That is, the acoustic model learning unit 17 initializes the acoustic model likelihood data s.

  Then, the acoustic model learning unit 17 uses the speech data read from the speech data storage unit 12, the language feature vector extracted from the language feature vector extraction unit 13, the learned acoustic model, and the intention class probability. The acoustic model likelihood is calculated, and the acoustic model likelihood data s is updated. For example, the acoustic model likelihood is added to the end of the list format data s.

The acoustic model learning unit 17 outputs “end” or “not finished” of the learning step from the acoustic model likelihood data s. As a criterion, whether the number of iteration steps of learning has reached a predetermined value or whether the difference between the last two terms of the acoustic model likelihood data s in the list format is lower than a threshold value s _th (s _j -s _j-1 <s _th ) or a combination thereof can be used.

  As illustrated in FIG. 12, the processes of the acoustic model learning unit 17 and the intention class determining unit 111 are repeatedly performed until “end” is determined in the end determination of the acoustic model learning unit 17. When it is determined as “end” in the end determination of the acoustic model learning unit 17, the last generated acoustic model is stored in the acoustic model storage unit 18 as a final acoustic model.

(Speech synthesizer and method)
Since the speech synthesizer and method of the fourth embodiment are the same as the speech synthesizer and method of the third embodiment, redundant description is omitted here.

[Program and recording medium]
When each process in the acoustic model learning device or the speech synthesizer is realized by a computer, the processing contents of functions that the acoustic model learning device or the speech synthesizer should have are described by a program. Then, by executing this program on a computer, each process is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  Each processing means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

[Modification]
The processing described in the acoustic model learning device and the speech synthesis device and these methods are not only executed in time series according to the order of description, but also in parallel or individually as required by the processing capability of the device that executes the processing. May be executed.

  For example, the process of step S15 may be performed before the process of step S13, or the process of step S13 and the process of step S15 may be performed in parallel. Further, for example, the process of step S23 may be performed before the processes of step S21 and step S22, or the process of step S21 and step S22 and the process of step S23 may be performed in parallel.

  Needless to say, other modifications are possible without departing from the spirit of the present invention.

Claims (8)

A context data storage unit storing each context data;
Using each context data read from the context data storage unit, a language feature vector extraction unit that extracts a language feature vector of each context data;
An intention information vector storage unit storing an intention information vector representing the utterance intention of each context data;
Acoustic model learning is performed using the language feature vector of each extracted context data, the speech data corresponding to each context data, and the intention information vector of each context data read from the intention information vector storage unit. An acoustic model learning unit for generating an acoustic model by performing,
An acoustic model learning device. The acoustic model learning device according to claim 1,
For each utterance intention, it is assumed that intention class information that is information about the intention class to which each utterance intention belongs is predetermined.
The intention information vector of each context data is a vector representing intention class information corresponding to the utterance intention of each context data.
Acoustic model learning device. The acoustic model learning device according to claim 1,
The acoustic model learning unit includes a language feature vector of each of the extracted context data, speech data corresponding to each of the context data, and an intention information vector of each of the context data read from the intention information vector storage unit. And generating an acoustic model and a probability that each utterance intention belongs to each intention class by performing acoustic model learning based on a predetermined initial value of the probability that each utterance intention belongs to each intention class,
An intention class determining unit that determines an intention class that maximizes a probability that each utterance intention belongs to each intention class as an intention class to which each utterance intention belongs;
Acoustic model learning device. In the acoustic model learning device according to claim 3,
The probability that the probability that the utterance intention belongs to the intention class to which the utterance intention belongs determined by the intention class determination unit is 1 and the probability that the utterance intention belongs to another intention class is 0 is determined in advance. As an initial value of the probability that each utterance intention belongs to each intention class, the processing of the acoustic model learning unit and the intention class determination unit is repeated.
Acoustic model learning device. A text analysis unit that parses input text and obtains context;
A language feature vector extraction unit for extracting a language feature vector of each context;
A speech parameter generation unit that generates speech parameters using the intention information vector representing the input speech intention, the acoustic model generated by the acoustic model learning device according to claim 1, and the extracted language feature vector. When,
A speech waveform generation unit that generates synthesized speech using the generated speech parameters;
A speech synthesizer. A language feature vector extraction unit that extracts a language feature vector of each context data using each context data read from a context data storage unit in which each context data is stored;
An intention information vector in which an acoustic model learning unit stores a language feature vector of each of the extracted context data, speech data corresponding to each of the context data, and an intention information vector representing the utterance intention of each context data An acoustic model learning step of generating an acoustic model by performing acoustic model learning using the intention information vector of each context data read from the storage unit;
Acoustic model learning method including A text analysis step in which a text analysis unit analyzes input text to obtain a context;
A language feature vector extraction step in which a language feature vector extraction unit extracts a language feature vector of each context;
The speech parameter generation unit uses the intention information vector representing the input utterance intention, the acoustic model generated by the acoustic model learning device according to claim 1, and the extracted language feature vector to generate speech parameters. An audio parameter generation step to generate;
A speech waveform generation step in which a speech waveform generation unit generates synthesized speech using the generated speech parameters;
A speech synthesis method including:   The program for functioning a computer as each part of the acoustic model learning apparatus in any one of Claim 1 to 4, or the speech synthesizer of Claim 5.

Images