JP6680933B2 - Acoustic model learning device, speech synthesis device, acoustic model learning method, speech synthesis method, program - Google Patents

Description

  The present invention relates to an acoustic model learning device that learns a deep neural network acoustic model from speech data, a speech synthesis device that generates synthetic speech using the learned deep neural network acoustic model, an acoustic model learning method, a speech synthesis method, and a program. Regarding

  There is a technique based on DNN (Deep Neural Network) as a method of generating a synthesized voice of a target speaker from voice data of the target speaker (Non-Patent Document 1). Hereinafter, configurations and operations of the acoustic model learning device and the speech synthesis device of Non-Patent Document 1 will be described with reference to FIGS. 1 and 2. FIG. 1 is a block diagram showing a configuration of an acoustic model learning device 91 of the document. FIG. 2 is a block diagram showing the configuration of the speech synthesizer 92 of the document.

  As shown in FIG. 1, the acoustic model learning device 91 of Non-Patent Document 1 includes a speaker voice database 911, an acoustic model learning unit 913, and an acoustic model storage unit 914. The speaker voice database 911 includes a voice data storage unit 9111 and a context data storage unit 9112. The voice data storage unit 9111 stores in advance voice data (voice parameters) of the target speaker. The context data storage unit 9112 stores in advance context data corresponding to the voice data of the target speaker. Although details will be described later, it is assumed that the context data includes at least phoneme information and accent information of the voice data.

  The acoustic model learning unit 913 learns the acoustic model of the target speaker by DNN (deep neural network) using the speech data and context data of the target speaker, and the learned acoustic model (hereinafter, the DNN acoustic model, Or, referred to as a deep neural network acoustic model) is stored in the acoustic model storage unit 914.

  As shown in FIG. 2, the voice synthesis device 92 of Non-Patent Document 1 includes a text analysis unit 921, a voice parameter generation unit 922, and a voice waveform generation unit 923.

  The text analysis unit 921 analyzes the input text (text data for speech synthesis purpose) and acquires the context data described above. The voice parameter generation unit 922 uses the deep neural network acoustic model stored in the acoustic model storage unit 914 to generate a voice parameter from the context data. The voice waveform generation unit 923 generates a voice waveform using the generated voice parameter.

Zen et al., "Statistical parametric speech synthesis using deep neural networks." Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013 pp. 7962-7966.

  In order to achieve high-quality speech synthesis by the DNN acoustic model, the acoustic model learning unit 913 needs a large amount of speech data and context data of the target speaker. Also, only one speaker's voice could be synthesized from one DNN acoustic model.

  Therefore, in order to achieve voices of a plurality of speakers by voice synthesis based on DNN, a large amount of voice data and context data are required for each of the plurality of speakers, and learning costs are high.

  Further, in order to obtain the synthesized speech of a plurality of speakers, it is necessary to hold as many DNN acoustic models as the number of speakers, and the number of memories used increases as the number of speakers increases.

  Therefore, it is an object of the present invention to provide an acoustic model learning device that can learn a DNN acoustic model that can generate synthetic speech of a small size and multiple speakers at low cost.

  The acoustic model learning device of the present invention determines the voice data of a plurality of speakers, the context data of a plurality of speakers including at least phoneme information and accent information of the voice data, the data for specifying the speaker, or the characteristics of the speaker. Using the represented data, a pitch parameter, which is necessary for speech waveform synthesis, and an acoustic model learning unit that learns a deep neural network acoustic model for generating a speech parameter including a spectrum parameter. It is characterized in that context data of a plurality of speakers including at least phoneme information and accent information of voice data, data specifying a speaker or data representing characteristics of the speaker is input to the input layer.

  According to the acoustic model learning device of the present invention, it is possible to learn a DNN acoustic model having a small size and capable of generating synthetic speech of a plurality of speakers at low cost.

The block diagram which shows the structure of the acoustic model learning apparatus of nonpatent literature 1. FIG. 3 is a block diagram showing the configuration of the speech synthesis device of Non-Patent Document 1. 3 is a block diagram showing the configuration of an acoustic model learning device of Example 1. FIG. 3 is a flowchart showing the operation of the acoustic model learning device of the first embodiment. 3 is a block diagram showing the configuration of the speech synthesizer of Embodiment 1. FIG. 3 is a flowchart showing the operation of the speech synthesizer of the first embodiment. 3 is a block diagram showing the configuration of an acoustic model learning device of Example 2. FIG. 5 is a flowchart showing the operation of the acoustic model learning device of the second embodiment. 3 is a block diagram showing the configuration of a speech synthesizer of Example 2. FIG. 7 is a flowchart showing the operation of the speech synthesizer according to the second embodiment. 7 is a block diagram showing the configuration of an acoustic model learning device of Example 3. FIG. 9 is a flowchart showing the operation of the acoustic model learning device of the third embodiment. 6 is a block diagram showing the configuration of a speech synthesizer according to a third embodiment. FIG. 9 is a flowchart showing the operation of the speech synthesizer according to the third embodiment.

  Hereinafter, embodiments of the present invention will be described in detail. It should be noted that components having the same function are denoted by the same reference numeral, and redundant description will be omitted.

  Hereinafter, the configuration and operation of the acoustic model learning device according to the first embodiment will be described with reference to FIGS. 3 and 4. FIG. 3 is a block diagram showing the configuration of the acoustic model learning device 11 of this embodiment. FIG. 4 is a flowchart showing the operation of the acoustic model learning device 11 of this embodiment. The difference from the acoustic model learning device 91 of Non-Patent Document 1 is that the acoustic model learning device 11 of the present embodiment utilizes data for identifying a speaker.

  As shown in FIG. 3, the acoustic model learning device 11 according to the present exemplary embodiment includes a multi-speaker speech database 111, an acoustic model learning unit 113, and an acoustic model storage unit 914. The multi-speaker voice database 111 stores, for each of a plurality of speakers (N is an integer of 2 or more, N speakers), voice data storage units 1111-1, ... 1111-N, and context data storage units 1112-1, ..., 1112-N that store context data corresponding to the voice data of each speaker. The voice data is data of voices produced by a plurality of N speakers who are targeted for learning the DNN acoustic model for voice synthesis. The context data is information such as pronunciation that is given one by one for each utterance in the voice data. The context data holds utterance information of voice data and includes at least phoneme information (pronunciation information) and accent information (accent type, accent phrase length). In addition to this, the context data may include part-of-speech information and the like. The acoustic model storage unit 914 is the same as the constituent element of the same name in the acoustic model learning device 91 of Non-Patent Document 1 described above.

  The acoustic model learning unit 113 uses, in addition to the voice data of a plurality of speakers and the corresponding context data, the data that specifies the speaker to generate a DNN acoustic model for generating a voice parameter required for voice waveform synthesis. The learned DNN acoustic model is stored in the acoustic model storage unit 914 (S113). The data for specifying the speaker is information (data) for specifying the speaker who read a certain voice data. It is possible to use, for example, a speaker code in which the data specifying the speaker is expressed as a numerical vector. The speaker code can be a vector in which information that identifies which speaker of N speakers is uttered is expressed by a 1-of-K expression. The 1-of-K expression is an expression in which only one element of the vector is 1, and all other elements are 0.

  That is, the acoustic model learning unit 113 receives the language feature vector expressing the context data as a numerical vector and the speaker code connected, and outputs the DNN acoustic model that outputs the voice parameters corresponding to the speaker and the context data. Is learned (S113).

  The configuration and operation of the speech synthesizer 12 of this embodiment will be described below with reference to FIGS. 5 and 6. FIG. 5 is a block diagram showing the configuration of the speech synthesizer 12 of this embodiment. FIG. 6 is a flowchart showing the operation of the speech synthesizer 12 of this embodiment. The difference from the speech synthesizer 92 of Non-Patent Document 1 is that the speech synthesizer 12 of the present embodiment utilizes data for specifying a speaker.

As shown in FIG. 5, the speech synthesis device 12 of this embodiment includes a text analysis unit 921, a speech parameter generation unit 122, and a speech waveform generation unit 923. The text analysis unit 921 and the speech waveform generation unit 923 operate in the same manner as the components of the same name in the speech synthesis device 92 of Non-Patent Document 1 described above. The speech parameter generation unit 122 uses the DNN acoustic model stored in the acoustic model storage unit 914 to analyze the context data obtained by analyzing the input text, and the data (speaker that specifies the speaker input together with the input text. The voice parameter is generated from the person code) (S122). The voice parameters include pitch parameters (fundamental frequency F0, etc.) and spectrum parameters (cepstrum, mel cepstrum, etc.). Specifically, the voice parameter generation unit 122 connects the context data and the speaker code to obtain an input vector to the DNN acoustic model. The voice parameter generation unit 122 inputs the input vector to the DNN acoustic model and generates a voice parameter by forward propagation (S122). The voice waveform generation unit 923 obtains a synthetic voice by voice waveform generation from voice parameters, as in Non-Patent Document 1 (S923). The speech waveform generation unit 923 may obtain a speech parameter sequence smoothed in the time direction using, for example, a maximum likelihood generation (MLPG) algorithm (Reference Non-Patent Document 1) before generating the speech waveform. For example, (Reference Non-Patent Document 2) may be used for the voice waveform generation.
(Reference Non-Patent Document 1: Mashiko et al., "HMM-based speech synthesis using dynamic features", IEICE, vol.J79-D-II, no.12, pp.2184-2190, Dec. 1996. )
(Reference Non-Patent Document 2: Imai et al., “Mel-Log Spectral Approximation (MLSA) Filter for Speech Synthesis”, IEICE Transactions A Vol.J66-A No.2 pp.122-129, Feb. 1983. .)

  According to the acoustic model learning apparatus 11 of the present embodiment, since the data (speaker code) that specifies the speaker is utilized in addition to the context data, the DNN that outputs the corresponding context data and the voice parameter reflecting the speaker characteristic. Can learn acoustic models.

  In the present embodiment, it is assumed that the voice parameter includes a component that characterizes the speaker and a component that is common to both speakers as Japanese voice. Specifically, a speaker code, which is a 1-of-K expression of each speaker, is used as an input corresponding to a component that characterizes the speaker, and an input corresponding to a component that is common among speakers as a Japanese voice. Is used as the context data. A parameter estimator corresponding to each component is learned inside the DNN by giving a voice signal composed of a component characterizing the speaker and a component common to the speakers as a teacher signal. This enables speech synthesis corresponding to each speaker used for learning with a single DNN acoustic model.

  Since Japanese voice has various voice parameter expressions for various contexts, a large amount of voice data is usually required to accurately estimate voice parameters for various contexts. . However, in the present embodiment, since it is assumed that the voice parameter includes a component that characterizes the speaker and a component that is common among the speakers as Japanese voice, there is a sufficient amount of voice data across multiple speakers. All that is required is not to prepare a large amount of voice data for a single speaker. That is, since the voice data of a plurality of speakers are efficiently used and one DNN acoustic model is learned, the voice data required for learning can be reduced. Further, since voice synthesis that reflects a plurality of speaker characteristics is realized with one acoustic model, a voice synthesis system that handles a large number of speakers can be realized with a smaller memory usage.

  When the speaker code (1-of-K expression) is used as in the first embodiment, it is not possible to perform voice synthesis for speakers other than the speakers included in the multi-speaker voice database 111. Therefore, in the second embodiment, the feature of the spectrum information of the reference utterance of the target speaker is extracted and used for model learning and voice synthesis, thereby enabling the voice synthesis for an arbitrary target speaker who can obtain the reference utterance. . Hereinafter, the configuration and operation of the acoustic model learning device 21 according to the second embodiment will be described with reference to FIGS. 7 and 8. FIG. 7 is a block diagram showing the configuration of the acoustic model learning device 21 of this embodiment. FIG. 8 is a flowchart showing the operation of the acoustic model learning device 21 of this embodiment. The difference from the acoustic model learning device 11 according to the first exemplary embodiment is that the acoustic model learning device 21 according to the present exemplary embodiment utilizes data (speaker spectrum characteristic vector) representing the characteristics of the speaker.

  As shown in FIG. 7, the acoustic model learning device 21 according to the present embodiment includes a multi-speaker speech database 111, a spectrum feature extraction unit 212, an acoustic model learning unit 213, and an acoustic model storage unit 914. The human voice database 111 and the acoustic model storage unit 914 are the same as the constituent requirements of the same name in the first embodiment.

The spectrum feature extraction unit 212 extracts the reference utterance of each speaker from the voice data storage units 1111-1 to 1111-N of each speaker, and extracts the speaker spectrum feature vector of each speaker from the reference utterance of each speaker. Generate (S212). Here, the reference utterance is an utterance by a speaker used at the time of learning or a target speaker at the time of voice synthesis, and is characterized in that transcription is unnecessary and a short sentence utterance is sufficient. The speaker spectrum feature vector is a numerical vector expressing the features of the spectrum information found in the voice spoken by the speaker. For example, i-vector may be used to generate the speaker spectrum feature vector. For the spectral feature extraction unit 212, for example, the knowledge of Reference Non-Patent Document 3 may be used and an i-vector extractor may be used.
(Reference Non-Patent Document 3: Dehak, Najim, et al. "Front-end factor analysis for speaker verification." Audio, Speech, and Language Processing, IEEE Transactions on 19.4.
(2011): 788-798.)

  Next, the acoustic model learning unit 213 uses the voice data of the plurality of speakers, the context data of the plurality of speakers, and the speaker spectrum feature vector that is the data representing the features of the speakers, and the DNN acoustic model. Is learned and the learned DNN acoustic model is stored in the acoustic model storage unit 914 (S213).

  The configuration and operation of the speech synthesizer 22 of this embodiment will be described below with reference to FIGS. 9 and 10. FIG. 9 is a block diagram showing the configuration of the speech synthesizer 22 of this embodiment. FIG. 10 is a flowchart showing the operation of the speech synthesizer 22 of this embodiment. The difference from the speech synthesizer 12 of the first embodiment is that the speech synthesizer 22 of the present embodiment utilizes data (speaker spectrum feature vector) representing the characteristics of the speaker.

  As shown in FIG. 9, the speech synthesis device 22 of this embodiment includes a text analysis unit 921, a spectrum feature extraction unit 221, a speech parameter generation unit 222, and a speech waveform generation unit 923. The text analysis unit 921 and the voice waveform generation unit 923 are the same as in the first embodiment. The spectrum feature extraction unit 221 extracts the above-described speaker spectrum feature vector from the reference utterance input together with the text for speech synthesis (S221). As described above, the reference utterance is the utterance by the target speaker.

  The speech parameter generation unit 222 uses the DNN acoustic model stored in the acoustic model storage unit 914 to analyze the input text, the context data obtained by analyzing the input text, and the speaker spectrum feature vector extracted from the reference utterance. Is generated (S222).

  Since the voice synthesizer 12 of the first embodiment uses the speaker code, the target speaker not included in the multi-speaker voice database 111 used during acoustic model learning is unknown during acoustic model learning. , I cannot synthesize voice. In order to solve this problem, in the present embodiment, a vector (speaker spectrum) that expresses the characteristics of the spectrum information of the voice uttered by the speaker, such as i-vector used in the fields of voice recognition and speaker identification. Feature vector). As a result, even if the target speaker is not included in the multi-speaker voice database 111, the voice of the speaker acoustically similar to the voice of the target speaker is modeled in the acoustic model. If the reference utterance of the speaker is acquired, it is possible to synthesize a voice having spectral characteristics close to the target speaker. Therefore, even a target speaker who is not included in the multi-speaker voice database 111 can generate a synthesized voice. As described above, for example, i-vector can be used to generate the speaker spectrum feature vector, but the method of realizing step S212 is not limited to this.

  In the method of the second embodiment, i-vector, which is a typical method for extracting the speaker information vector from the utterance, is proposed for the purpose of speaker adaptation of the model in the speaker identification field and the voice recognition field. It has come. In these fields, it was important that the individuality of the spectrum information, among the individualities appearing in the voice, be represented by a vector. On the other hand, in the field of speech synthesis, when the speaker information vector is extracted to realize the speech synthesis of the target speaker, among the individualities appearing in the voice, not only the individuality of the spectrum information but also the individuality of the prosodic information. Is also important, which is considered to be different from the speech recognition problem. Therefore, in the acoustic model learning device 31 of the third embodiment, the data representing the characteristics of the speaker also includes F0 information. Hereinafter, the configuration and operation of the acoustic model learning device 31 according to the third embodiment will be described with reference to FIGS. 11 and 12. FIG. 11 is a block diagram showing the configuration of the acoustic model learning device 31 of this embodiment. FIG. 12 is a flowchart showing the operation of the acoustic model learning device 31 of this embodiment. The difference from the acoustic model learning device 21 of the second embodiment is that the acoustic model learning device 31 of the present embodiment utilizes not only the speaker spectrum feature vector but also the speaker prosody feature vector as data representing the features of the speaker. Is.

  As shown in FIG. 11, the acoustic model learning device 31 of the present exemplary embodiment includes a multi-speaker speech database 111, a spectrum feature extraction unit 212, a prosody feature extraction unit 312, an acoustic model learning unit 313, and an acoustic model storage. The multi-speaker voice database 111, the spectrum feature extraction unit 212, and the acoustic model storage unit 914 including the unit 914 are the same as the constituent elements of the same name in the second embodiment.

  The prosody feature extraction unit 312 extracts the reference utterance of each speaker from the voice data storage units 1111-1 to 1111-N of each speaker and extracts the speaker prosody feature vector of each speaker from the reference utterance of each speaker. It is generated (S312). The speaker prosody feature vector is a vector expressing the individuality of the prosody information among the individualities appearing in the voice. More specifically, the speaker prosody feature vector is a numerical vector expressing the features of the prosody information among the acoustic features found in the speech uttered by the speaker.

  The prosody feature extraction unit 312 may calculate, for example, the average and variance of the F0 sequence analyzed from the reference utterance, and extract the F0 feature information as a speaker prosody feature vector. The prosody feature extraction unit 312 may perform more detailed prosody feature modeling using the method of Reference Non-Patent Document 4. (Reference Non-Patent Document 4: Dehak, Najim, Pierre Dumouchel, and Patrick Kenny. "Modeling prosodic features with joint factor analysis for speaker verification." Audio, Speech, and Language Processing, IEEE Transactions on 15.7 (2007): 2095-2103 .)

  Next, the acoustic model learning unit 313 learns a DNN acoustic model by using voice data of a plurality of speakers, context data of a plurality of speakers, a speaker spectrum feature vector, and a speaker prosody feature vector. Then, the learned DNN acoustic model is stored in the acoustic model storage unit 914 (S313).

  The configuration and operation of the speech synthesizer 32 of this embodiment will be described below with reference to FIGS. 13 and 14. FIG. 13 is a block diagram showing the configuration of the speech synthesizer 32 of this embodiment. FIG. 14 is a flowchart showing the operation of the speech synthesizer 32 of this embodiment. The difference from the voice synthesizer 22 of the second embodiment is that the voice synthesizer 32 of the present embodiment utilizes not only the speaker spectrum feature vector but also the speaker prosody feature vector as data representing the feature of the speaker. .

  As shown in FIG. 13, the speech synthesis device 32 of this embodiment includes a text analysis unit 921, a spectrum feature extraction unit 221, a prosody feature extraction unit 321, a speech parameter generation unit 322, and a speech waveform generation unit 923. Including. The text analysis unit 921, the spectrum feature extraction unit 221, and the voice waveform generation unit 923 are the same as in the first embodiment. The prosody feature extraction unit 321 extracts the above-mentioned speaker prosody feature vector from the reference utterance input together with the text for speech synthesis (S321). As described above, the reference utterance is the utterance by the target speaker.

  The voice parameter generation unit 322 uses the DNN acoustic model stored in the acoustic model storage unit 914 to analyze the context data obtained by analyzing the input text, the speaker spectrum feature vector, and the speaker prosody feature vector, and outputs the voice. Parameters are generated (S322).

  The acoustic characteristics of a speaker can be classified into spectral characteristics and prosody characteristics. When the speaker spectrum feature vector is used as in the second embodiment, of the features of the speaker, the features of the spectrum are reflected in the synthesized voice, and the features of the prosody of the target speaker are not reflected. In the present embodiment, by using a vector that also represents the prosody information of the target speaker, it is possible to synthesize a voice that also reflects the prosody features of the speakers not included in the multi-speaker voice database 111. Becomes

  The acoustic model learning device and the speech synthesizing device described in the above embodiments may be implemented as the acoustic model learning unit and the speech synthesizing unit, respectively, and the present invention may be realized as independent hardware having these components as constituent elements.

  In addition, since the speaker code, the speaker spectrum feature vector, the speaker prosody feature vector, etc. described in the above-mentioned embodiment have a common term that they are vectors expressing the information of the speaker, these are the speaker information vectors. May be collectively referred to.

<Additional notes>
The device of the present invention is, for example, as a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating with the outside of the hardware entity. Connectable communication unit, CPU (Central Processing Unit, may include a cache memory or register, etc.), RAM or ROM as memory, external storage device as hard disk, and their input unit, output unit, communication unit , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged among external storage devices. If necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above-described functions and data necessary for processing of this program (not limited to the external storage device, for example, the program is read). It may be stored in a ROM that is a dedicated storage device). In addition, data and the like obtained by the processing of these programs are appropriately stored in the RAM, the external storage device, or the like.

  In the hardware entity, each program stored in an external storage device (or ROM, etc.) and data necessary for the processing of each program are read into the memory as necessary, and interpreted and executed / processed by the CPU as appropriate. . As a result, the CPU realizes a predetermined function (each constituent element represented by the above, ... Unit, ... Means, etc.).

  The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention. Further, the processes described in the above embodiments are not only executed in time series in the order described, but may be executed in parallel or individually according to the processing capability of the device that executes the processes or as necessary. .

  As described above, when the processing functions of the hardware entity (the device of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on the computer, the processing functions of the hardware entity are realized on the computer.

The program describing the processing contents can be recorded in a computer-readable recording medium. The computer-readable recording medium may be any recording medium such as a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape or the like is used as a magnetic recording device, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), or a CD-ROM (Compact Disc Read Only) is used as an optical disc. Memory), CD-
R (Recordable) / RW (ReWritable) or the like can be used as a magneto-optical recording medium, MO (Magneto-Optical disc) or the like, and semiconductor memory can be an EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) or the like. .

  The distribution of this program is performed by selling, transferring, or lending a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, the program may be stored in a storage device of a server computer and transferred from the server computer to another computer via a network to distribute the program.

  A computer that executes such a program first stores, for example, the program recorded on a portable recording medium or the program transferred from the server computer in its own storage device. Then, when executing the processing, this computer reads the program stored in its own recording medium and executes the processing according to the read program. As another execution form of this program, a computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to this computer. Each time, the processing according to the received program may be sequentially executed. Further, the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by executing the execution instruction and acquiring the result without transferring the program from the server computer to the computer. May be It should be noted that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to a computer but has the property of defining computer processing).

  Further, in this embodiment, the hardware entity is configured by executing a predetermined program on the computer, but at least a part of these processing contents may be implemented by hardware.

Claims (10)

Using voice data of a plurality of speakers, context data of a plurality of speakers including at least phoneme information and accent information of the voice data, data specifying the speaker or data representing characteristics of the speaker An acoustic model learning unit that learns a deep neural network acoustic model for generating a voice parameter including a pitch parameter and a spectrum parameter required for voice waveform synthesis,
In the input layer of the neural network, context data of a plurality of speakers including at least phoneme information and accent information of the voice data, and data specifying the speakers or data representing characteristics of the speakers are input. Acoustic model learning device. The acoustic model learning device according to claim 1, wherein
The acoustic model learning unit,
The deep neural network acoustic model is learned using voice data of a plurality of speakers, context data of a plurality of speakers, and data for specifying the speaker represented by a 1-of-K expression vector. ,
An acoustic model learning device, wherein context data of a plurality of speakers and data for specifying the speakers represented by a 1-of-K expression vector are input to an input layer of the neural network. The acoustic model learning device according to claim 1, wherein
The acoustic model learning unit,
Speech data of a plurality of speakers, context data of a plurality of speakers, a speaker spectrum feature vector represented by i-vector as data representing the features of the speakers, and a speaker prosody feature vector representing F0 feature information. And learn the deep neural network acoustic model using
In the input layer of the neural network, context data of a plurality of speakers, a speaker spectrum feature vector represented by i-vector as data representing the features of the speakers, and a speaker prosody feature vector representing F0 feature information. An acoustic model learning device characterized by inputting ,. A text analysis unit that analyzes the input text and acquires context data including at least phoneme information and accent information,
Deep neural network acoustic model learned using voice data of a plurality of speakers, context data of voice data of the plurality of speakers, and data specifying the speaker or data representing characteristics of the speaker. By using the context data obtained by analyzing the input text, and from the data that specifies the speaker or the data representing the characteristics of the speaker, which is input together with the input text, a pitch parameter and a spectrum. A voice parameter generation unit for generating voice parameters including parameters,
A voice waveform generating unit that generates a voice waveform using the generated voice parameter,
In the input layer of the neural network, context data of a plurality of speakers including at least phoneme information and accent information of the voice data, and data specifying the speakers or data representing characteristics of the speakers are input. A speech synthesizer. The voice synthesizer according to claim 4,
The deep neural network acoustic model is
Learned using voice data of a plurality of speakers, context data of a plurality of speakers, and data specifying the speaker represented by a 1-of-K expression vector,
A voice synthesizer, wherein context data of a plurality of speakers and data for specifying the speakers expressed by a 1-of-K expression vector are input to an input layer of the neural network. The voice synthesizer according to claim 4,
The deep neural network acoustic model is
Speech data of a plurality of speakers, context data of a plurality of speakers, a speaker spectrum feature vector represented by i-vector as data representing the features of the speakers, and a speaker prosody feature vector representing F0 feature information. And are learned using
In the input layer of the neural network, context data of a plurality of speakers, a speaker spectrum feature vector represented by i-vector as data representing the features of the speakers, and a speaker prosody feature vector representing F0 feature information. A voice synthesizer characterized by inputting ,. An acoustic model learning method executed by an acoustic model learning device,
Using voice data of a plurality of speakers, context data of a plurality of speakers including at least phoneme information and accent information of the voice data, data specifying the speaker or data representing characteristics of the speaker , A step of learning a deep neural network acoustic model for generating a voice parameter including a pitch parameter and a spectrum parameter required for voice waveform synthesis,
In the input layer of the neural network, context data of a plurality of speakers including at least phoneme information and accent information of the voice data, and data specifying the speakers or data representing characteristics of the speakers are input. Acoustic model learning method. A voice synthesis method executed by a voice synthesizer, comprising:
Analyzing the input text to obtain context data including at least phoneme information and accent information;
Deep neural network acoustic model learned using voice data of a plurality of speakers, context data of voice data of the plurality of speakers, and data specifying the speaker or data representing characteristics of the speaker. By using the context data obtained by analyzing the input text, and from the data that specifies the speaker or the data representing the characteristics of the speaker, which is input together with the input text, a pitch parameter and a spectrum. Generating voice parameters including parameters,
Generating a voice waveform using the generated voice parameters,
In the input layer of the neural network, context data of a plurality of speakers including at least phoneme information and accent information of the voice data, and data specifying the speakers or data representing characteristics of the speakers are input. Speech synthesis method.   A program that causes a computer to function as the acoustic model learning device according to claim 1.   A program that causes a computer to function as the voice synthesizer according to claim 4.

Images