JP6626052B2 - Acoustic model generation method, speech synthesis method, acoustic model generation device, speech synthesis device, program - Google Patents

Description

  The present technology relates to a speech synthesis technology based on DNN, and relates to an acoustic model generation method, a speech synthesis method, an acoustic model generation device, a speech synthesis device, and a program.

  As a technique for learning a model for speech synthesis from speech data and generating a synthesized speech, there is a technique based on DNN (Non-Patent Document 1). In addition, as a method for efficiently utilizing learning data of a plurality of domains and synthesizing high-quality speech for each domain, there is a multi-domain modeling technique of DNN. The domains include, for example, speakers (Non-Patent Documents 2 and 3) and dialogue action information (Non-Patent Document 4). As a multi-domain modeling technique, there is a method utilizing a model configuration such as a speaker code (Non-Patent Document 2) and a shared hidden layer (SHU) (Non-Patent Document 3). An outline of this method is shown in FIGS.

  As shown in FIG. 1, the acoustic model learning unit 913 of the acoustic model generation device 91 includes a multi-domain voice DB stored in advance in the multi-domain voice DB storage unit 911 and a plurality of voice databases stored in the multi-domain context DB storage unit 912 in advance. Using the domain context DB, acoustic model learning is performed, a multi-domain acoustic model is obtained, and stored in the multi-domain acoustic model storage unit 914 (S913).

  As shown in FIG. 2, the text analysis unit 921 of the speech synthesizer 92 performs text analysis on the input text to obtain a context (S921). The voice parameter generation unit 922 of the voice synthesis device 92 inputs a domain number to be synthesized with the context into the multiple domain acoustic model stored in the multiple domain acoustic model storage unit 914, and generates a voice parameter (S922). The voice waveform generation unit 923 of the voice synthesizer 92 obtains a synthesized voice by generating a voice waveform from the obtained voice parameters (S923).

Zen et al., “Statistical parametric speech synthesis using deep neural networks,” Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on.IEEE, 2013 pp. 7962-7966. N. Hojo, Y. Ijima, and H. Mizuno, “An investigation of DNN-based speech synthesis using speaker codes,” Interspeech 2016, pp.2278-2282, 2016. Y. Fan, Y. Qian, FK Soong, and L. He, “Multi-speaker modeling and speaker adaptation for DNN-based tts synthesis,” 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.4475 -4479, IEEE, 2015. Nobukatsu Hojo et al., “Speech synthesis capable of expressing dialogue information”, Japan Society for Artificial Intelligence 2016 National Convention, 2016. Nose, Takashi, and Akinori Ito. “Analysis of spectral enhancement using global variance in HMM-based speech synthesis.” INTERSPEECH. 2014. Tomoki, Toda, and Keiichi Tokuda. “A speech parameter generation algorithm considering global variance for HMM-based speech synthesis.” IEICE TRANSACTIONS on Information and Systems 90.5 (2007): 816-824. Takamichi, Shinnosuke, et al. “A postfilter to modify the modulation spectrum in HMM-based speech synthesis.” Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on. IEEE, 2014.

  The maintenance of the multi-domain voice DB requires the cost of voice recording and the like, and the maintenance of the multi-domain context DB requires the cost of annotation such as phonemes and accent types. For example, due to these costs, the data amount of the multiple domain DB may be uneven depending on the domain. Here, the ratio of the term related to each domain in the objective function at the time of learning the multiple domain DNN is proportional to the data amount of each domain. Therefore, when model learning is performed without considering the bias (heterogeneity) of the data amount, the modeling accuracy of the domain in which only a small amount of data is obtained (the speech parameter estimated for the input context, the correct speech parameter, and the Mean square error) has less effect on the objective function than other domains. For this reason, a domain from which only a small amount of data can be obtained is not accurately modeled, and there is a possibility that voice quality is degraded.

  Therefore, an object of the present invention is to provide an acoustic model generation method for generating an acoustic model that realizes high-quality speech synthesis even for a domain in which only a small amount of learning data is obtained.

  For the voice data, the voice parameters of the voice included in the DNN learning database are analyzed and held, and the context data is analyzed for the voice of the voice included in the DNN learning database, and the context of the utterance is analyzed. The domain is assumed to be stored, information other than the context included in the voice is represented by a category, and the multi-domain voice DB is configured to hold the voice data of the voice of the plurality of domains, and the multi-domain context is stored. Assume that the DB holds context data of utterances of voices of a plurality of domains.

  The acoustic model generation method of the present invention includes three steps.

  The first step is to generate a multi-domain homogenized context DB by adding a context to context data of a domain in which the total number of frames in the multi-domain context DB is not the maximum, and to generate a pseudo-domain for each context data in the multi-domain homogenized context DB. The voice data is generated, and a multiple domain homogenized pseudo voice DB is generated.

  The second step is to integrate the multi-domain audio DB and the multi-domain homogenized pseudo speech DB to generate a multi-domain homogenous speech DB, and to integrate the multi-domain context DB and the multi-domain homogenization context DB to obtain a multi-domain homogenous context DB. Generate a DB.

  In the third step, an acoustic model is learned using a multi-domain homogeneous speech DB and a multi-domain homogeneous context DB as learning data.

  According to the acoustic model generation method of the present invention, it is possible to generate an acoustic model that realizes high-quality speech synthesis even for a domain in which only a small amount of learning data is obtained.

FIG. 2 is a block diagram illustrating a configuration of a conventional acoustic model generation device. FIG. 2 is a block diagram showing a configuration of a conventional speech synthesizer. FIG. 1 is a block diagram illustrating a configuration of an acoustic model generation device according to a first embodiment. 5 is a flowchart illustrating the operation of the acoustic model generation device according to the first embodiment. FIG. 2 is a block diagram illustrating a configuration of a multiple domain homogenization DB generation unit according to the first embodiment. 5 is a flowchart illustrating the operation of the multiple domain homogenization DB generation unit according to the first embodiment. FIG. 2 is a block diagram illustrating a configuration of a multiple domain homogeneous DB generation unit according to the first embodiment. 5 is a flowchart illustrating the operation of a multiple domain homogeneous DB generation unit according to the first embodiment. FIG. 1 is a block diagram illustrating a configuration of a speech synthesis device according to a first embodiment. 5 is a flowchart illustrating the operation of the speech synthesis device according to the first embodiment. FIG. 9 is a block diagram illustrating a configuration of an acoustic model generation device according to a second embodiment. 9 is a flowchart illustrating the operation of the acoustic model generation device according to the second embodiment. FIG. 10 is a block diagram showing a configuration of a multiple domain homogenization post-filter pseudo-sound DB generator according to a second embodiment. FIG. 9 is a block diagram illustrating a configuration of a multiple domain homogeneous DB generation unit according to the second embodiment.

  Hereinafter, embodiments of the present invention will be described in detail. Note that components having the same functions are given the same numbers, and overlapping descriptions are omitted.

≪ Explanation of terms ≪
<Audio parameters>
It represents F0 information (pitch), spectral envelope information (cepstrum, mel-cepstrum, etc.) of each utterance obtained as a result of performing signal processing on a certain audio signal.

<Context>
Represents information such as pronunciation given to a certain utterance. The context needs to include phoneme information (pronunciation information) and accent information (accent type, accent phrase length). The context may also include part of speech information and the like. Further, information on the start time and end time of each phoneme (phoneme segmentation information) may be stored.

<Audio data>
For speech included in the DNN learning database, the speech parameters are analyzed and stored.

<Context data>
For the speech included in the DNN learning database, the context of the utterance is analyzed and the speech is retained.

<Domain>
Represents information other than the context included in the audio, expressed by category. For example, speakers ('speaker 1', ..., 'speaker N'), emotions ('happy', 'angry', 'sad', 'easy', ...), dialogue acts ('exclamation', 'apology) ',…),etc.

<Domain number>
For domains used for speech synthesis, numbers are assigned to each domain. The number of domains is N, and n = 1,..., N. For example, n = 1: 'speaker 1', ..., n = N: represents 'speaker N'.

<Multiple domain context DB>
Represents voices of a plurality of domains that hold context data of the utterance. When the number of utterances included in the domain n is K _n (n = 1,..., N),

Expressed by Here, the amount of learning data (total number of frames) corresponding to each domain n generally does not always match (Kn ≠ Kn 'when _n ≠ _n ') due to constraints such as the cost of data maintenance.

<Multi-domain audio DB>
For voices of a plurality of domains, the voice data is stored. When the number of utterances included in the domain n is K _n (n = 1,..., N),

Expressed by As with the multiple domain context DB, the amount of learning data (total number of frames) corresponding to each domain n does not always match.

<Multi-domain acoustic model>
This represents a DNN acoustic model for speech synthesis that has been modeled and learned so that speech parameters of a plurality of domains can be synthesized by one model. For example, a multi-speaker acoustic model using a speaker code (Non-Patent Document 2), a multi-speaker acoustic model using a shared hidden layer (SHU) (Non-Patent Document 3), a multi-dialogue action acoustic model (Non-Patent Document 4) Use

<Multi-domain homogenization context DB>
A context database used to homogenize the amount of learning data (total number of frames) corresponding to each domain included in the database. It is used as an input vector of DNN for generating pseudo data. When the number of utterances included in domain n is K ′ _n (n = 1,..., N),

Expressed by

<Multi-domain homogenized pseudo speech DB>
A speech database used to homogenize the amount of training data (total number of frames) corresponding to each domain included in the database. It is created by generating pseudo audio data from the multiple domain homogenization context DB. When the number of utterances included in domain n is K ′ _n (n = 1,..., N),

Expressed by The frames of the multi-domain homogenization context DB and the multi-domain homogenization pseudo speech DB correspond one-to-one. Therefore, the total number of frames in each domain of the two DBs is the same.

<Multi-domain homogeneous context DB>
A context database obtained by integrating the multi-domain context DB and the multi-domain homogenization context DB. Used for acoustic model learning.

<Multi-domain homogeneous voice DB>
An audio database obtained by integrating the multiple domain audio DB and the multiple domain homogenized pseudo audio DB. Used for acoustic model learning.

<Multi-domain homogeneous acoustic model>
An acoustic model obtained by acoustic model learning using a multi-domain homogeneous context DB and a multi-domain homogeneous speech DB.

≫End of terminology explanation≫
Hereinafter, the configuration of the acoustic model generation device 11 according to the first embodiment will be described with reference to FIG. As shown in the figure, the acoustic model generation device 11 of the present embodiment includes a multi-domain homogenized DB generation unit 111, a multi-domain homogenous DB generation unit 112, an acoustic model learning unit 913, and a multi-domain homogenous acoustic model storage unit. 114.

  Hereinafter, the operation of the acoustic model generation device 11 of the present embodiment will be described with reference to FIG.

[Multi-domain homogenization DB generation unit 111]
The multi-domain homogenization DB generation unit 111 generates a multi-domain homogenization context DB by adding a context to context data of a domain in which the total number of frames in the multi-domain context DB is not the maximum, and generates a multi-domain homogenization context DB. Pseudo voice data is generated for the context data, and a multiple domain homogenized pseudo voice DB is generated (S111).

[Multi-domain homogeneous DB generation unit 112]
The multi-domain homogenous DB generation unit 112 integrates the multi-domain audio DB and the multi-domain homogenized pseudo audio DB to generate a multi-domain homogenous audio DB, and integrates the multi-domain context DB and the multi-domain homogenized context DB to generate A domain homogeneous context DB is generated (S112).

[Acoustic model learning unit 913]
The acoustic model learning unit 913 performs acoustic model learning in the same manner as in the related art. However, the acoustic model learning unit 913 uses a multi-domain homogeneous speech DB instead of the multi-domain speech DB and a multi-domain homogeneous context DB instead of the multi-domain context DB as learning data. That is, the acoustic model learning unit 913 learns an acoustic model using the multi-domain homogeneous speech DB and the multi-domain homogeneous context DB as learning data, and stores the acoustic model in the multi-domain homogeneous acoustic model storage unit 114 (S913). The acoustic model learned by the acoustic model learning unit 913 is called a multi-domain homogeneous acoustic model.

  Hereinafter, the configurations and operations of the multi-domain homogenized DB generator 111 and the multi-domain homogenized DB generator 112 will be described in more detail with reference to FIGS. 5, 6, 7, and 8.

[Multi-domain homogenization DB generation unit 111]
As shown in FIG. 5, the multi-domain homogenization DB generation unit 111 includes a context addition unit 1111, a multi-domain homogenization context DB storage unit 1112, a speech parameter generation unit 922, and a multi-domain homogenization pseudo speech DB storage unit 1113. including. The context adding unit 1111 executes, for example, step S1111 including the following sub-steps (a) and (b) to generate a multi-domain homogenized context DB.

(A) The context adding unit 1111 calculates, for each domain n, the domain n ^{* in} which the total number of frames included in the multiple domain context DB is the maximum and the maximum frame number _{Fn *} .

Here, the number of frames of the k-th (k = 1,..., K _n ) utterance of the domain n is defined as f _k ⁽ⁿ⁾ .

(B) For each domain n other than n ^* , the context adding unit 1111 adds a context to the context data of each domain until the total number of frames of the domain becomes F ^′ _n = F _{n *,} and the context addition unit 1111 performs multi-domain homogenization. Generate the Context DB. Context adding unit 1111, the F _^'n

It is preferable to reduce the amount of data used for pseudo data generation, reduce the amount of computer memory required for speech parameter generation and acoustic model learning, and reduce the cost of calculation time by setting appropriate values in the range . At this time, for example, the context to be added is a context other than the domain n. The context adding unit 1111 stores and holds the generated multi-domain homogenized context DB in the multi-domain homogenized context DB storage unit 1112.

  The voice parameter generation unit 922 repeats the process of generating voice parameters using the corresponding domain number and the multiple domain acoustic model for each context of the multiple domain homogenized context DB, and generates pseudo voice data corresponding to each context data. Generated and used as a multiple domain homogenized pseudo audio DB (S922). At this time, for example, a multi-domain acoustic model learned by a conventional technique is used as the multi-domain acoustic model. The voice parameter generation unit 922 stores and holds the generated multi-domain homogenized pseudo voice DB in the multi-domain homogenized pseudo voice DB storage unit 1113.

[Multi-domain homogeneous DB generation unit 112]
As shown in FIG. 7, the multi-domain homogeneous DB generation unit 112 includes a speech DB integration unit 1121, a multi-domain homogeneous speech DB storage unit 1122, a context DB integration unit 1123, and a multi-domain homogeneous context DB storage unit 1124. Configuration.

  The audio DB integration unit 1121 integrates the multiple domain audio DB and the multiple domain homogenized pseudo audio DB, and stores and holds the multiple domain homogeneous audio DB in the multiple domain homogeneous audio DB storage unit 1122 (S1121).

  The context DB integration unit 1123 integrates the multi-domain context DB and the multi-domain homogenization context DB, and stores and holds the multi-domain homogenization context DB in the multi-domain homogenous context DB storage unit 1124 (S1123).

[Speech synthesizer 12]
As shown in FIG. 9, the speech synthesizer 12 according to the present embodiment includes a text analysis unit 921, a speech parameter generation unit 922, a speech waveform generation unit 923 similar to the related art, and a multi-domain homogenization unit different from the related art. An acoustic model storage unit 114 is included. As shown in FIG. 10, the speech synthesizer 12 according to the present embodiment executes steps 921, S922, and S923 to obtain a synthesized speech in the same manner as in the related art. However, it differs from the prior art in that a multi-domain homogeneous acoustic model is used instead of the conventional multi-domain acoustic model.

  As described above, the acoustic model generation device 11 of the present embodiment uses the acoustic model learned using the heterogeneous multi-domain DB, generates the pseudo-speech data, and converts the pseudo-speech data to the multi-domain speech DB. By adding the same to the multiple domain context DB, the data amount of each domain included in the learning data can be made uniform. Accordingly, acoustic model learning can be performed from training data of a uniform data amount for each domain, and high-quality synthesized speech can be obtained even for a domain where only a small amount of data can be obtained.

  It is known that voice parameters generated by an acoustic model tend to be excessively smoothed as compared to voice parameters of actual human speech (natural speech). In the first embodiment, since the excessively smoothed speech parameters are added to the learning data, the speech parameters generated from the learned acoustic model may be further smoothed. Thus, in the present embodiment, post-filter processing is performed on the pseudo-generated speech parameters to bring the over-smoothed speech parameters closer to those of a naturally uttered speech. This makes it possible to avoid smoothing of the speech parameters generated from the learned acoustic model.

≪ Explanation of terms ≪
<Multi-domain homogenized post-filter pseudo-speech DB>
For each voice data included in the multiple domain homogenized pseudo voice DB, a post-filtering process is performed to make the tendency of the voice parameter closer to a natural utterance.

<Multi-domain homogeneous post-filter audio DB>
An audio database obtained by integrating the multi-domain homogenized post-filter pseudo audio DB and the multi-domain audio DB.

<Multi-domain homogeneous post-filter acoustic model>
An acoustic model obtained by performing acoustic model learning from a multi-domain homogeneous post-filter speech DB and a multi-domain homogeneous context DB.

≫End of terminology explanation≫
As shown in FIG. 11, the acoustic model generation device 21 of the present embodiment includes a multi-domain homogenization DB generation unit 111 similar to the first embodiment, and a multi-domain homogenization post-filter pseudo speech DB generation different from the first embodiment. Unit 211, a multi-domain homogeneous DB generation unit 212 different from the first embodiment, an acoustic model learning unit 913 similar to the first embodiment and the related art, and a multi-domain homogeneous post-filter acoustic model storage unit different from the first embodiment. 214.

  Hereinafter, the operation of the acoustic model generation device 21 of the present embodiment will be described with reference to FIG.

[Multi-domain homogenization DB generation unit 111]
Step S111 is executed as in the first embodiment.

[Multi-domain homogenization post-filter pseudo-sound DB generator 211]
The multi-domain homogenized post-filter pseudo-speech DB generator 211 acquires the multi-domain homogenized post-filter pseudo-speech DB from the multi-domain homogenized pseudo-speech DB by post-filtering (S211).

[Multi-domain homogeneous DB generation unit 212]
This is the same as the first embodiment except that a multi-domain homogenized post-filter pseudo audio DB is used instead of the multi-domain homogenized pseudo audio DB. However, the obtained voice DB is referred to as a multi-domain homogeneous post-filter voice DB to distinguish it from the first embodiment. That is, the multi-domain homogenous DB generation unit 212 generates the multi-domain homogenous post-filter audio DB by integrating the multi-domain audio DB and the multi-domain homogenized post-filter pseudo audio DB. The DB and the multi-domain homogenized context DB are integrated to generate a multi-domain homogenous context DB (S212).

[Acoustic model learning unit 913]
The acoustic model learning unit 913 executes Step S913 as in the first embodiment. However, the acoustic model learning unit 913 uses a multi-domain homogeneous post-filter speech DB instead of the multi-domain homogeneous speech DB as learning data. That is, the acoustic model learning unit 913 learns the acoustic model using the multi-domain homogeneous post-filter speech DB and the multi-domain homogeneous context DB as the learning data, and stores the acoustic model in the multi-domain homogeneous post-filter acoustic model storage unit 214. (S913). The acoustic model that the acoustic model learning unit 913 learns is referred to as a multi-domain homogeneous post-filter acoustic model to distinguish it from the first embodiment.

[Speech synthesizer (not shown)]
This is the same as the first embodiment except that a multi-domain homogeneous post-filter acoustic model is used instead of the multi-domain homogeneous acoustic model.

  Hereinafter, the configurations of the multi-domain homogenized post-filter pseudo-speech DB generator 211 and the multi-domain homogenous DB generator 212 will be described in more detail with reference to FIGS.

[Multi-domain homogenized post-filter pseudo-sound DB generator 211]
As shown in FIG. 13, the multi-domain homogenized post-filter pseudo-speech DB generator 211 includes a post-filter 2111 and a multi-domain homogenized post-filter pseudo-speech DB storage 2113. The post filter 2111 performs the above-described post filter processing. As the post-filter processing, for example, dispersion guarantee processing for cepstrum feature amounts (Non-Patent Document 5), Global Variance guarantee processing (Non-Patent Document 6), modulation spectrum guarantee processing (Non-Patent Document 7), and the like may be used. The post filter 2111 stores the acquired multi-domain homogenized post-filter pseudo-speech DB in the multi-domain homogenized post-filter pseudo-speech DB storage unit 2113.

[Multi-domain homogeneous DB generation unit 212]
As illustrated in FIG. 14, the multi-domain homogeneous DB generation unit 212 includes a speech DB integration unit 1121 similar to the first embodiment, a multi-domain homogeneous post-filter speech DB storage unit 2122 different from the first embodiment, A similar context DB integration unit 1123 and a multi-domain homogeneous context DB storage unit 1124 similar to the first embodiment are included. The audio DB integration unit 1121 stores the multi-domain homogeneous post-filter audio DB obtained by the integration in the multi-domain homogeneous post-filter audio DB storage unit 2122.

  As described above, it is known that voice parameters generated by the acoustic model tend to be excessively smoothed in the time direction as compared to voice parameters of actual human utterances (natural utterances). In the first embodiment, an excessively smoothed speech parameter is added to the learning data, which may affect the acoustic model learning.

  The acoustic model generation device 21 of the present embodiment performs post-filter processing on the pseudo-generated speech parameters to bring the over-smoothed speech parameters closer to those of spontaneous speech. As a result, it is possible to avoid the influence of excessive smoothing on the learning of the acoustic model, and it is possible to improve speech quality.

<Supplementary note>
The device of the present invention includes, as a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) that can communicate outside the hardware entity. A communication unit, which may include a central processing unit, a cache memory and a register, a RAM or a ROM as a memory, an external storage device as a hard disk, and an input unit, an output unit, and a communication unit thereof. , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. As a physical entity having such hardware resources, there is a general-purpose computer or the like.

  The external storage device of the hardware entity stores a program necessary to realize the above-described functions, data necessary for processing the program, and the like. It may be stored in a ROM that is a dedicated storage device). Data and the like obtained by the processing of these programs are appropriately stored in a RAM, an external storage device, or the like.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing of each program are read into a memory as needed, and interpreted and executed / processed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (each of the constituent elements described as the above-described section, means, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately modified without departing from the spirit of the present invention. Further, the processing described in the above embodiment may be performed not only in chronological order according to the order described, but also in parallel or individually according to the processing capability of the apparatus that executes the processing or as necessary. .

  As described above, when the processing function of the hardware entity (the device of the present invention) described in the above embodiment is implemented by a computer, the processing content of the function that the hardware entity should have is described by a program. Then, by executing this program on a computer, the processing functions of the hardware entities are realized on the computer.

  A program describing this processing content can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape, or the like is used as a magnetic recording device, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), or a CD-ROM (Compact Disc Read Only) is used as an optical disk. Memory), CD-R (Recordable) / RW (ReWritable), magneto-optical recording media, MO (Magneto-Optical disc), EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) as semiconductor memory, etc. Can be used.

  The distribution of the program is performed by, for example, selling, transferring, lending, or the like, a portable recording medium such as a DVD or a CD-ROM on which the program is recorded. Further, the program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, when executing the process, the computer reads the program stored in its own recording medium and executes the process according to the read program. Further, as another execution form of the program, the computer may directly read the program from the portable recording medium and execute processing according to the program, and further, the program may be transferred from the server computer to the computer. Each time, the processing according to the received program may be sequentially executed. A configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by executing the program and acquiring the result without transferring the program from the server computer to the computer. It may be. It should be noted that the program in the present embodiment includes information used for processing by the computer and which is similar to the program (data that is not a direct command to the computer but has characteristics that define the processing of the computer).

  Further, in this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

  Each step described in the description and the claims corresponds to each step of a method for generating various information. The various types of information referred to here correspond to programs and the like (programs and other information used for processing by electronic computers and equivalent to the programs) prescribed in Article 2, Paragraph 4 of the Patent Act. The information in (1) corresponds to the provisions of Article 2, Paragraph 3, Item 1 of the Patent Act. Therefore, it goes without saying that the method of generating various types of information described in the description and the claims corresponds to the method of producing a product specified in Article 2, Paragraph 3, Item 3 of the Patent Act. Absent.

Claims (9)

Speech data is obtained by analyzing and retaining speech parameters of speech included in a database for DNN learning,
As for the context data, it is assumed that the context of the utterance of the voice included in the DNN learning database is analyzed and held,
The domain is assumed that information other than the context included in the voice is represented by a category,
It is assumed that the multiple domain audio DB holds the audio data of the audio of the plurality of domains,
A multi-domain context DB holds the context data of the utterances for the voices of the plurality of domains,
The context is added to the context data of the domain in which the total number of frames in the multi-domain context DB is not the maximum to generate a multi-domain homogenized context DB, and the context data of the multi-domain homogenized context DB is simulated. A first step of generating audio data to generate a multiple domain homogenized pseudo audio DB;
The multi-domain voice DB and the multi-domain homogenized pseudo voice DB are integrated to generate a multi-domain homogenous voice DB, and the multi-domain context DB and the multi-domain homogenization context DB are integrated to form a multi-domain homogenous context DB. Generating a second step;
A third step of learning an acoustic model using the multi-domain homogeneous speech DB and the multi-domain homogeneous context DB as learning data;
An acoustic model generation method including: Speech data is obtained by analyzing and retaining speech parameters of speech included in a database for DNN learning,
As for the context data, it is assumed that the context of the utterance of the voice included in the DNN learning database is analyzed and held,
The domain is assumed that information other than the context included in the voice is represented by a category,
It is assumed that the multiple domain audio DB holds the audio data of the audio of the plurality of domains,
A multi-domain context DB holds the context data of the utterances for the voices of the plurality of domains,
The context is added to the context data of the domain in which the total number of frames in the multi-domain context DB is not the maximum to generate a multi-domain homogenized context DB, and the context data of the multi-domain homogenized context DB is simulated. A first step of generating audio data to generate a multiple domain homogenized pseudo audio DB;
A second step of obtaining a multi-domain homogenized post-filter pseudo-speech DB from the multi-domain homogenized pseudo-speech DB by post-filtering;
The multi-domain voice DB and the multi-domain homogenized post-filter pseudo voice DB are integrated to generate a multi-domain homogenous post-filter voice DB, and the multi-domain context DB and the multi-domain homogenized context DB are integrated to form a multi-domain A third step of generating a homogeneous context DB;
A fourth step of learning an acoustic model using the multi-domain homogeneous post-filter speech DB and the multi-domain homogeneous context DB as learning data;
An acoustic model generation method including: The acoustic model generation method according to claim 1 or 2,
In the first step,
The context is set such that the total number of frames of the domain in which the total number of frames in the multiple domain context DB is not the maximum is equal to the total number of frames of the domain in which the total number of frames in the multiple domain context DB is the maximum. The acoustic model generation method to be added.   A speech synthesis method for acquiring a synthesized speech using an acoustic model generated by the acoustic model generation method according to claim 1. Speech data is obtained by analyzing and retaining speech parameters of speech included in a database for DNN learning,
As for the context data, it is assumed that the context of the utterance of the voice included in the DNN learning database is analyzed and held,
The domain is assumed that information other than the context included in the voice is represented by a category,
It is assumed that the multiple domain audio DB holds the audio data of the audio of the plurality of domains,
A multi-domain context DB holds the context data of the utterances for the voices of the plurality of domains,
The context is added to the context data of the domain in which the total number of frames in the multi-domain context DB is not the maximum to generate a multi-domain homogenized context DB, and the context data of the multi-domain homogenized context DB is simulated. A multi-domain homogenized DB generation unit that generates audio data and generates a multi-domain homogenized pseudo audio DB;
The multi-domain voice DB and the multi-domain homogenized pseudo voice DB are integrated to generate a multi-domain homogenous voice DB, and the multi-domain context DB and the multi-domain homogenization context DB are integrated to form a multi-domain homogenous context DB. A multi-domain homogenous DB generation unit to generate;
An acoustic model learning unit that learns an acoustic model using the multi-domain homogeneous speech DB and the multi-domain homogeneous context DB as learning data;
An acoustic model generation device including: Speech data is obtained by analyzing and retaining speech parameters of speech included in a database for DNN learning,
As for the context data, it is assumed that the context of the utterance of the voice included in the DNN learning database is analyzed and held,
The domain is assumed that information other than the context included in the voice is represented by a category,
It is assumed that the multiple domain audio DB holds the audio data of the audio of the plurality of domains,
The multi-domain context DB holds the context data of the utterances of the voices of the plurality of domains,
The context is added to the context data of the domain in which the total number of frames in the multi-domain context DB is not the maximum to generate a multi-domain homogenized context DB, and the context data of the multi-domain homogenized context DB is simulated. A multi-domain homogenized DB generation unit that generates audio data and generates a multi-domain homogenized pseudo audio DB;
A multi-domain homogenized post-filter pseudo-speech DB generator that acquires a multi-domain homogenized post-filter pseudo-speech DB from the multi-domain homogenized pseudo-speech DB by post-filter processing;
The multi-domain voice DB and the multi-domain homogenized post-filter pseudo voice DB are integrated to generate a multi-domain homogenous post-filter voice DB, and the multi-domain context DB and the multi-domain homogenized context DB are integrated to form a multi-domain A multi-domain homogeneous DB generator for generating a homogeneous context DB;
An acoustic model learning unit that learns an acoustic model using the multi-domain homogeneous post-filter audio DB and the multi-domain homogeneous context DB as learning data;
An acoustic model generation device including:   A speech synthesizer for acquiring a synthesized speech using an acoustic model generated by the acoustic model generator according to claim 5.   A program for causing a computer to execute the acoustic model generation method according to claim 1.   A program for causing a computer to execute the speech synthesis method according to claim 4.

Images