JP6806662B2 - Speech synthesis system, statistical model generator, speech synthesizer, speech synthesis method - Google Patents

Description

The present invention relates to a speech synthesis system, a statistical model generator, a speech synthesizer, and a speech synthesis method.

In recent years, speech synthesis technology has been introduced in various fields such as automatic answering of telephone calls, announcements of public transportation and local governments, and reading of information by applications of smartphones and personal computers. In recent years, technologies such as voice recognition, machine translation, and dialogue generation have been dramatically improved, and practical application to voice translation, service robots, and the like is rapidly being promoted.

Oord, Aaron van den; Dieleman, Sander; Zen, Heiga; Simonyan, Karen; Vinyals, Oriol; Graves, Alex; Kalchbrenner, Nal; Senior, Andrew; Kavukcuoglu, Koray (2016-09-12). "WaveNet: A Generative Model for Raw Audio ". Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, et al. 2017. Tacotron: A fully end-toend text-to-speech synthesis model. "Unit selection with Hierarchical Cascaded Long Short Term Memory Bidirectional Recurrent Neural Nets", Vincent Pollet, Enrico Zovato, Sufian Irhimeh, Pier Batzu, Interspeech 2017.

There are various speech synthesis methods, but the text speech synthesis method is attracting attention as one of the important technologies. A general text-to-speech synthesis method includes a front-end process for converting text into a phonetic symbol string and a back-end process for generating a speech waveform from an intermediate language. In recent years, those to which statistical methods such as DNN (Deep Neural Network) are applied to front-end processing and back-end processing have been put into practical use.

Recently, a so-called ETE (End-To-End) type speech synthesis method has also appeared, in which speech is directly generated from input text without going through an intermediate language. In the ET-type speech synthesis method, the relationship between the language features of the spoken text of the speech corpus and the acoustic features of the speech waveform is prepared in advance as a statistical model by a statistical method such as DNN, and statistics are obtained during speech synthesis. Based on the model, a series having acoustic features corresponding to the text to be voice-synthesized is generated and the voice is synthesized.

By the way, in the field of using speech synthesis technology, for example, there is a considerable need to perform speech synthesis with the reading (pronunciation) specified by the user for a person's name, a proper noun, etc., and the user also has an ET-type speech synthesis method. There are many needs for dictionary functions. However, the statistical model in the ETE-type speech synthesis method is configured as one large model as a whole (DNN model, etc.), and a large amount of computers are used for re-learning the ETE-type speech synthesis method using a huge amount of data. It requires resources, and it is not always practical to retrain the statistical model every time the contents of the user dictionary are added to the training data or new words are added.

The present invention has been made in view of such a background, and an object of the present invention is to provide a speech synthesis system based on an ET-type speech synthesis method capable of performing speech synthesis with a reading (pronunciation) specified by a user. To do.

One of the present inventions for solving the above problems is a speech synthesis system configured by using an information processing device, which is used for speech synthesis based on learning data in which spoken text and speech data are associated with each other. A model learning unit that generates a statistical model to be used, and a text replacement unit that generates data in which all or part of the spoken text is replaced with a phonetic symbol string and the speech data is associated with the speech data. , A storage unit that stores a user dictionary, which is data containing information corresponding to a word and a pronunciation symbol string for the word, and a target text, which is a text to be voice-synthesized, are included in the target text. A user dictionary application unit that replaces a word included in the user dictionary with the pronunciation symbol string for the word in the user dictionary, and a speech synthesis process based on the statistical model for the target text after the replacement. A speech synthesis processing unit that generates a synthetic speech by performing the above is provided.

According to the present invention, it is possible to provide a speech synthesis system based on an ET-type speech synthesis method capable of performing speech synthesis with a reading (pronunciation) specified by a user.

It is a figure which shows the schematic structure of the speech synthesis system. It is a block diagram of an information processing apparatus shown as an example of the hardware used for the realization of a speech synthesis system. It is a figure explaining the structure of the speech synthesis system. It is a figure explaining the detail of the text replacement part. It is a figure explaining the detail of the replacement word extraction part. It is a figure explaining the detail of the voice feature amount extraction part. It is a figure explaining the detail of the user dictionary application part.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings.

FIG. 1 shows a schematic configuration of a system (hereinafter, referred to as a speech synthesis system 1) that synthesizes speech by an ETE (End-To-End) type speech synthesis method described in the present embodiment.

The speech synthesis system 1 learns the relationship between the language features of the speech text 51 of the speech corpus 50 and the acoustic features of the speech data 52 by a statistical method (machine learning or the like) such as DNN (Deep Neural Network). A statistical model 60 is generated in advance, and a series having acoustic features corresponding to a text (sentence or phrase) (hereinafter referred to as an input text 700 (target text)) to be voice-synthesized is a statistical model 60. Generate based on and synthesize voice.

In the present embodiment, DNN will be described as an example of the above statistical method, but the statistical method is not necessarily limited, and for example, another statistical method such as a Hidden Markov Model (HMM) may be used. You may use it. In the following, the case where the language described as text is Japanese will be described as an example, but the language described as text may be another language, and the text contains a mixture of multiple languages. May be good.

As shown in the figure, the speech synthesis system 1 generates a speech corpus 50, a statistical model generation unit 100 that generates a statistical model 60 based on the speech corpus 50, and a sequence having acoustic features corresponding to the input text 700. A voice synthesis unit 200, which synthesizes voice waveforms based on the generated sequence and generates (outputs) a synthetic voice 800, is included.

The voice corpus 50 includes the utterance text 51 and voice data 52 (voice waveform data, encoded voice data, etc.) associated with the utterance text 51. The voice corpus 50 is used as learning data when the statistical model generation unit 100 generates the statistical model 60.

The speech synthesizer 200 uses the statistical model 60 to describe the pronunciation and utterance styles (for example, conversational tone, emotional muffled, etc.), voice intonation, and loudness specified for the input text 700. , Vocalization characteristics characterized by factors such as rhythm, speed, and length of interval), etc.) are synthesized.

FIG. 2 is a block diagram of an information processing device 10 (computer, computer resource) shown as an example of hardware used for realizing the speech synthesis system 1. As shown in the figure, the information processing device 10 includes a processor 11, a main storage device 12, an auxiliary storage device 13, an input device 14, an output device 15, and a communication device 16. These are communicably connected to each other via a communication means such as a bus (not shown).

It should be noted that the information processing apparatus 10 does not necessarily have all the configurations realized by hardware, and for example, a part or all of the configurations are realized by virtual resources such as a cloud server of a cloud system. You may.

The processor 11 is configured by using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or the like. When the processor 11 reads and executes the program stored in the main storage device 12, various functions of the speech synthesis system 1 are realized.

The main storage device 12 is a device that stores programs and data, and is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile semiconductor memory (NVRAM (Non Volatile RAM)), and the like.

The auxiliary storage device 13 includes, for example, a hard disk drive, SSD (Solid State Drive), optical storage device (CD (Compact Disc), DVD (Digital Versatile Disc), etc.), storage system, IC card, SD memory card, FD ( A reading / writing device for a recording medium such as a flexible disk, a storage area for a cloud server, and the like. Programs and data stored in the auxiliary storage device 13 are read into the main storage device 12 at any time. The data managed by the voice synthesis system 1 such as the voice corpus 50 is managed in, for example, a database of a DBMS (DataBase Management System) that uses the auxiliary storage device 13 as a data storage area.

The input device 14 is an interface (or user interface) for inputting the voice corpus 50 and the input text 700, and is, for example, a keyboard, a mouse, a touch panel, a card reader, a microphone, an amplifier, and the like. The information processing device 10 may be configured to accept input of information from another device via the communication device 16.

The output device 15 is an interface (or user interface) that outputs various types of information, and includes an audio output device (speaker, amplifier, etc.) that outputs synthetic speech. The information processing device 10 includes an interface (for example, an audio output device (speaker, etc.)), a screen display device (liquid crystal monitor, LCD (Liquid Crystal Display), graphic card) that provides the user with various information such as processing progress and processing results. Etc.), a printing device, etc.) may be further provided as the output device 15. Further, the information processing device 10 may be configured to output information to and from another device via the communication device 16.

The communication device 16 is a wired or wireless communication interface that realizes communication with another device via a communication means such as LAN or the Internet. For example, a NIC (Network Interface Card) and various wireless communication modules. , USB (Universal Serial Interface) module, serial communication module, modem, etc.

The statistical model generation unit 100, the voice synthesis unit 200, the voice corpus 50, and the statistical model 60 do not have to be realized by common hardware, and a plurality of hardware connected so as to be communicable. It may be distributed and arranged in. For example, the voice corpus 50 and the statistical model generation unit 100, and the statistical model 60 and the voice synthesis unit 200 may be configured by independent resources. The statistical model 60 and the voice synthesizer 200 are incorporated into devices such as car navigation devices, smartphones, mobile phones, and personal computers.

Further, the components described later of the statistical model generation unit 100 do not necessarily have to be realized by common hardware, and may be distributed and arranged in a plurality of communicably connected hardware. Further, the components described later of the voice synthesis unit 200 do not necessarily have to be realized by common hardware, and may be distributed and arranged in a plurality of hardware connected so as to be communicable.

Further, the voice corpus 50 and the statistical model 60 are arranged in resources on a communication network such as a cloud server, and the statistical model generation unit 100 and the voice synthesis unit 200 are used in the voice corpus 50 and the statistical model 60 through a wired or wireless communication network. You may want to access it.

Further, the statistical model generation unit 100 and the speech synthesis unit 200 are arranged on independent hardware, and the statistical model 60 generated by the statistical model generation unit 100 is stored on a physical recording medium (optical storage device (CD (Compact Disc)). ), DVD (Digital Versatile Disc), etc.), hard disk drive, SSD, IC card, SD memory card, etc.), or may be provided to the speech synthesizer 200 via a wired or wireless communication network.

FIG. 3 is a diagram for explaining the configuration of the speech synthesis system 1 shown in FIG. 1 in detail. As shown in the figure, the statistical model generation unit 100 has the functions of the text replacement unit 110 and the model learning unit 120.

The text replacement unit 110 replaces a part or all of the utterance text 51 of the voice corpus 50 with a phonetic symbol string in the text data format. More specifically, the text replacement unit 110 replaces a part or all of the utterance text 51 with a phonetic symbol string in the text data format.

FIG. 4 is a diagram illustrating details of the text replacement unit 110 of FIG. As shown in the figure, the text replacement unit 110 includes a replacement word extraction unit 111, a voice feature amount extraction unit 112, a phonetic symbol string generation unit 113, and a text replacement processing unit 115.

The replacement word extraction unit 111 extracts a word (for example, a proper noun) to be replaced with a phonetic symbol string from the utterance text 51 of the voice corpus 50. In the following description, the term "word" includes all aspects that can be registered in the user dictionary 212, which will be described later, such as one character, one word, a block of a plurality of words, a phrase, a clause, and a sentence.

Here, the words to be replaced are extracted from the utterance text 51 in this way. When all the words in all the utterance text 51 of the voice corpus 50 are replaced with the pronunciation symbol strings, the voice by the voice synthesis unit 200 is used. This is because speech synthesis based on text that does not contain a phonetic symbol string (hereinafter referred to as normal text) cannot be performed during synthesis. Therefore, for example, when the purpose is to realize the function of the user dictionary, the replacement word extraction unit 111 extracts, for example, a word that may be registered in the user dictionary as a word to be replaced.

FIG. 5 is a diagram for explaining the details of the replacement word extraction unit 111 shown in FIG. As shown in the figure, the replacement word extraction unit 111 has the functions of the morphological analysis unit 1111 and the word extraction unit 1112.

Further, the replacement word extraction unit 111 stores the word extraction data 106. The word extraction data 106 includes information that serves as a reference for determining whether or not the word extraction unit 1112 extracts as a word to be replaced with a phonetic symbol string. The content of the word extraction data 106 is set by the user, for example. The replacement word extraction unit 111 may automatically extract words by using, for example, a statistical method (machine learning or the like).

The morphological analysis unit 1111 performs morphological analysis and divides the utterance text 51 into the smallest unit of the language (morpheme in the case of Japanese). For example, when the utterance text 51 is "foreigner's suffrage", the morphological analysis unit 1111 divides it into four words "foreign", "person", "suffrage", and "right".

The word extraction unit 1112 extracts a word to be replaced with a phonetic symbol string from the words obtained by being divided by the morphological analysis unit 2101. Here, the above extraction method is not necessarily limited, but for example, a rule-based method or a statistics-based method can be used. For example, when constructing a speech synthesis system 1 for railway broadcasting, it is assumed that the station name is registered in the user dictionary 212 described later, and the replacement word extraction unit 111 extracts the station name from the utterance text 51. Specifically, for example, the input text 700 is "Mr. Shinjuku got off the train in Shinjuku", and the word extraction data 106 is "notation = Shinjuku, reading = Shinjuku, attribute = station name". In this case, the word extraction unit 2102 extracts only words in which "Shinjuku" is used as the "station name". However, if only the station name is replaced in this way, the accent type may be biased. Therefore, for example, the replacement word extraction unit 111 uses the prosodic balance of the word (the phoneme connection of the selected word is biased) so that the words that may be registered in the user dictionary can be read out with high quality as a whole. Select words from the spoken text 51 of the entire voice corpus 50 in consideration of the balance of prosody (accent type, position in the sentence, etc.) (extract words other than the station name if necessary). It may be targeted).

FIG. 6 is a diagram for explaining the details of the voice feature amount extraction unit 112 shown in FIG. The voice feature amount extraction unit 112 extracts, for example, pronunciation (phonemes), utterance style, prosody, etc. from the voice data 52 of the voice corpus 50 as voice feature amounts.

As shown in the figure, the voice feature extraction unit 112 has a pronunciation (phoneme) extraction unit 1121, an utterance style extraction unit 1122, and a prosodic feature extraction unit 1123. The pronunciation (phoneme) extraction unit 1121 extracts pronunciation (phoneme) information from the voice data 52 of the voice corpus 50. The utterance style extraction unit 1122 extracts utterance style information from the voice data 52 of the voice corpus 50. The prosodic feature extraction unit 1123 extracts prosodic feature information from the voice data 52 of the voice corpus 50. By extracting information on not only pronunciation (phonemes) but also speech style and prosody and incorporating them into phonetic symbol strings in this way, it is possible to tune the speech style and intonation of synthetic speech. The voice feature amount extraction unit 112 may extract still another information as the voice feature amount 104. The processing performed by the voice feature amount extraction unit 112 can be automatically performed by the information processing device using, for example, voice recognition technology or text analysis technology. In addition, a part or all of the above-mentioned processing may be performed manually.

The pronunciation (phoneme) information can be extracted from the spoken text 51 by a language processing technique, for example. For example, whether the word "tomorrow" is pronounced as "tomorrow" or "tomorrow", etc. In some cases, the pronunciation may not be uniquely determined by the language processing technique. In that case, for example, a speech recognition technique may be used to extract an accurate pronunciation based on the speech data 52. For example, when the utterance text 51 corresponding to a certain voice data 52 is "Tomorrow is sunny", the voice feature amount extraction unit 112 pronounces "Aswahaledes" (phoneme) by applying the voice recognition technology to the voice data 52. ) Extract information.

Returning to FIG. 4, the phonetic symbol string generation unit 113 generates the phonetic symbol string 105 based on the phonetic feature amount 104 extracted by the voice feature amount extraction unit 112 (the phonetic feature amount 104 is expressed by the phonetic symbol string). The description method of the phonetic symbol string 105 is not necessarily limited, but in the case of Japanese, for example, in the case of Japanese, the phonetic symbol string generation unit 113 is based on the voice feature quantity 104 extracted by the voice feature quantity extraction unit 112, and is JIETA (general corporation electronic). Generates Japanese text-speech synthesis symbols (JEITA IT-4006) specified by the Japan Electronics and Information Technology Industries Association.

By using a code (symbol code, special code, etc.) that is not included in the character code used for general text, the accuracy of reading (pronunciation) designation can be improved. For example, the phonetic symbol string generation unit 113 uses the pronunciation "Aswahalledes" extracted by the voice feature extraction unit 112 as a code that is not included in the character code used in the general text "Aswahalle'des%." Convert to a phonetic symbol string containing. In addition to the pronunciation (phoneme) information, information such as accent information, prosodic boundary information, and pose information may be added to improve the quality of reading designation.

The text replacement processing unit 115 uses the phonetic symbol strings generated by the phonetic symbol string generation unit 113 corresponding to the words extracted by the replacement word extraction unit 111 (extracted words 103 in the figure) of the utterance text 51. The replaced utterance text (hereinafter, referred to as the utterance text 102 including the phonetic symbol string) is generated.

Returning to FIG. 3, the text replacement processing unit 115 adds the utterance text 102 including the phonetic symbol string generated by the text replacement unit 110 to the voice corpus 50 as new learning data in combination with the corresponding voice data 52. To do. As a result, learning data is added to the voice corpus 50 each time the text replacement unit 110 generates the utterance text 102 including the phonetic symbol string.

To give a specific example, for example, the utterance text 51 is "The next stop is Shinjuku.", And the replacement word extraction unit 111 extracts the word "Shinjuku" (extracted word 103) from the utterance text 51. If so, the text replacement processing unit 115 generates the utterance text 102 including the pronunciation symbol string, which says "The next stop is Shinjuku." The text replacement processing unit 115 adds the utterance text 102 including the generated phonetic symbol string to the voice corpus 50 in combination with the corresponding voice data 52.

In addition, the text replacement processing unit 115 newly adds a combination of the utterance text 102 including the pronunciation symbol string generated by the text replacement unit 110 and the corresponding voice data 52 to the voice corpus 50 as described above. Alternatively, the utterance text 51 in the combination of the utterance text 51 and the voice data 52 before being replaced with the utterance symbol string of the voice corpus 50 may be replaced with the utterance text 102 including the utterance symbol string (. That is, it updates the existing combination of voice corpora 50).

The model learning unit 120 shown in FIG. 3 learns the combinations of all the utterance texts 51 and the voice data 52 (including the combinations newly added by the text replacement processing unit 115) included in the voice corpus 50. (Machine learning, etc.) is performed as a result, and a statistical model 60 is generated. Since the training data includes the utterance text 102 including the phonetic symbol string in this way, by using the generated statistical model 60, not only the normal text but also the text including the phonetic symbol string is voice-synthesized. It can be performed.

Subsequently, the voice synthesis unit 200 shown in FIG. 3 will be described. As shown in the figure, the voice synthesis unit 200 has the functions of the user dictionary application unit 210 and the voice synthesis processing unit 220.

When the user dictionary application unit 210 detects a word registered in the user dictionary in the input text 700 during speech synthesis based on the input text 700, the detected word is pronounced as specified in the user dictionary 212 for the word. Replace with the symbol string and generate the text 203 including the phonetic symbol string. The function of the user dictionary application unit 210 can be realized by a simple character string replacement algorithm, but the input text 700 is described in a language such as Japanese in which there is no clear split character between words. If so, a simple string replacement algorithm may not be able to perform the above replacement correctly. For example, if the user dictionary 212 contains the data "notation = carrot, reading (phonetic symbol string) = carrot", the simple character string replacement algorithm replaces "foreign carrot government" with "foreign carrot government". It may be done. Therefore, the user dictionary application unit 210 of the present embodiment generates the text 203 including the phonetic symbol string as follows.

FIG. 7 is a diagram illustrating details of the user dictionary application unit 210. As shown in the figure, the user dictionary application unit 210 includes a morphological analysis unit 2101, a word extraction unit 2102, and a phonetic symbol string replacement unit 2103.

Further, the user dictionary application unit 210 stores the word extraction data 211 and the user dictionary 212. The word extraction data 211 includes information that serves as a reference for determining whether or not the word extraction unit 2102 extracts as a word to be replaced. The user dictionary 212 includes information in which a word (notation) and a reading (phonetic symbol string) are associated with each other.

The morphological analysis unit 2101 performs morphological analysis and divides the input text 700 into the smallest unit of the language (morpheme in the case of Japanese). For example, when "foreign suffrage" is input as the input text 700, the morphological analysis unit 2101 divides it into four words "foreign", "person", "suffrage", and "right". Therefore, for example, even if the content such as "notation = carrot, reading (phonetic symbol string) = carrot" is registered in the user dictionary 212, it will not be replaced by mistake.

The word extraction unit 2102 extracts a word (for example, a proper noun) to be replaced with a phonetic symbol string from the words divided by the morphological analysis unit 2101. The extraction method is not necessarily limited, but for example, a rule-based method or a statistics-based method is used. It is the same as the word extraction unit 1112 described above, but for example, the input text 700 is "Mr. Shinjuku got off the train in Shinjuku", and the word extraction data 211 contains "Notation = Shinjuku, Reading = Shinjuku". When ", attribute = station name" is registered, the word extraction unit 2102 extracts a word in which "Shinjuku" is used as the "station name".

The phonetic symbol string replacement unit 2103 replaces the words registered in the user dictionary 212 among the words extracted by the word extraction unit 2102 with the readings (phonetic symbol strings) registered in the user dictionary 212 for the words. In the above example, for example, when "notation = Shinjuku, reading (phonetic symbol string) = syndicate" is registered in the user dictionary 212, the phonetic symbol string replacement unit 2103 says "Mr. Shinjuku got off the train in Shinjuku. Replace the input text 700 with the text "Shinjuku-san got off the train at the dictionary."

As the morphological analysis unit 2101 and the word extraction unit 2102, the same ones as the morphological analysis unit 1111 and the word extraction unit 1112 in the replacement word extraction unit 111 of the statistical model generation unit 100 described above (those having a common algorithm) are used. Is preferable. Further, as the phonetic symbol string replacement unit 2103, it is preferable to use the same one as the text replacement processing unit 115 of the text replacement unit 110 described above (one having a common algorithm). In this way, the quality of the synthesized voice can be improved by using a common algorithm between the statistical model generation unit 100 and the voice synthesis unit 200.

Returning to FIG. 3, the speech synthesis processing unit 220 uses the statistical model 60 to generate the synthetic speech 800 from the input text 700. The voice synthesis processing unit 220, for example, has a method of directly generating a voice waveform as in Non-Patent Document 1, a method of generating voice parameters for each frame as in Non-Patent Document 2, and a method of generating voice in Non-Patent Document 3. The synthesized voice 800 is generated by a method of synthesizing voice by connecting voice elements selected by DNN as described above.

For example, when the voice corpus 50 does not include the utterance text 51 corresponding to the input text 700, the statistical model 60 may be generated by analogy with the model learning unit 120 statistical method (machine learning or the like). Good. For example, when the voice corpus 50 does not include the utterance text 102 including the phonetic symbol string including "Shinjuk", the model learning unit 120 analogizes with "Shinjuk" and the voice data 52 by a statistical method. A statistical model 60 including the correspondence of is generated.

As described in detail above, the speech synthesis system 1 of the present embodiment includes learning data in which the speech text 51 in which some words of the speech text 51 are replaced with a pronunciation symbol string and the speech data 52 are associated with each other. The statistical model 60 is learned in advance by the voice corpus 50, and for the input text 700, the words included in the user dictionary 212 among the words included in the input text 700 are associated with the words in the user dictionary 212. A synthesized voice is generated by replacing the replaced input text 700 with the utterance symbol string and performing a voice synthesis process based on the statistical model 60. Therefore, it is possible to perform speech synthesis with the reading (pronunciation) specified by the user without re-learning the statistical model 60. As described above, according to the present embodiment, it is possible to realize a practical speech synthesis system by the ET type speech synthesis method capable of performing speech synthesis in the reading (pronunciation) specified by the user.

Although the present invention has been specifically described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist thereof. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to add / delete / replace a part of the configuration of the above embodiment with another configuration.

Further, each of the above configurations, functional units, processing units, processing means and the like may be realized by hardware by designing a part or all of them by, for example, an integrated circuit. Further, each of the above configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be placed in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

Further, in each figure, control lines and information lines are shown as necessary for explanation, and not all control lines and information lines in mounting are necessarily shown. For example, in practice almost all configurations may be considered interconnected.

Further, the arrangement form of various functional units, various processing units, and various databases of the information processing apparatus described above is only an example. The arrangement form of various function units, various processing units, and various databases can be changed to the optimum arrangement form for each information processing device from the viewpoints of hardware and software performance, processing efficiency, communication efficiency, and the like.

1 Speech synthesis system, 10 Information processing device, 50 Speech corpus, 51 Speech text, 52 Speech data, 60 Statistical model, 100 Statistical model generator, 102 Text including pronunciation symbol string, 103 Extracted words, 104 Speech features, 105 pronunciation symbol string, 106 word extraction data, 110 text replacement unit, 111 replacement word extraction unit, 1111 morphological analysis unit, 1112 word extraction unit, 112 speech feature extraction unit, 1121 pronunciation (sound element) extraction unit, 1122 speech style Extraction unit, 1123 Prosodic feature extraction unit, 113 pronunciation symbol string generation unit, 115 text replacement processing unit, 120 model learning unit, 200 speech synthesis unit, 203 text including pronunciation symbol string, 202 text including pronunciation symbol string, 210 users Dictionary application unit, 211 word extraction data, 212 user dictionary, 2101 morphological analysis unit, 2102 word extraction unit, 2103 pronunciation symbol string replacement unit, 700 input text, 800 synthetic speech

Claims (15)

A model learning unit that generates a statistical model used for speech synthesis based on learning data that associates utterance text with speech data.
A text replacement unit that generates data in which the utterance text in which some words of the utterance text are replaced with phonetic symbol strings and the voice data are associated with the learning data
A storage unit that stores a user dictionary, which is data containing information corresponding to a word and a phonetic symbol string for the word, and a storage unit.
Statistical model generator with
Regarding the target text, which is the text to be voice-synthesized, the phonetic symbol string in which the word included in the user dictionary among the words included in the target text is associated with the word in the user dictionary. User dictionary application part to be replaced with
A speech synthesis processing unit that generates synthetic speech by performing speech synthesis processing based on the statistical model for the target text after the replacement,
With a voice synthesizer
Speech synthesis system equipped with. The voice synthesis system according to claim 1.
The text replacement unit extracts a voice feature amount from the voice data, generates a pronunciation symbol string based on the extracted voice feature amount, and replaces a part of the words in the speech text with the speech symbol string. The data associated with the voice data is generated as the training data.
Speech synthesis system. The voice synthesis system according to claim 2.
The text replacement unit generates the phonetic symbol string by performing voice recognition on the voice data.
Speech synthesis system. The voice synthesis system according to claim 2.
The speech feature is at least one of pronunciation (phoneme), speech style, and prosody.
Speech synthesis system. The voice synthesis system according to claim 1.
The text replacement unit divides the utterance text into a plurality of words by an algorithm of morphological analysis.
The user dictionary application unit divides the target text into a plurality of words by an algorithm common to the algorithm of the morphological analysis.
Speech synthesis system. The voice synthesis system according to claim 1.
The text replacement unit extracts a word to be replaced with a phonetic symbol string from the utterance text by a word extraction algorithm.
The user dictionary application unit extracts a word to be replaced with a phonetic symbol string from the target text by an algorithm common to the word extraction algorithm.
Speech synthesis system. The voice synthesis system according to claim 1.
The learning data is a voice corpus,
Speech synthesis system. The voice synthesis system according to claim 7.
The text replacement unit extracts a word to be replaced with a phonetic symbol string from the spoken text of the voice corpus by a word extraction algorithm considering the phonological or prosodic balance of the extracted word.
Speech synthesis system. The voice synthesis system according to claim 1.
The model learning unit generates the statistical model by DNN (Deep Neural Network).
Speech synthesis system. The voice synthesis system according to claim 1.
ETE (End-To-End) type speech synthesis system,
Speech synthesis system. The statistical model generator in the speech synthesis system according to claim 1.
The model learning unit that generates a statistical model used for speech synthesis based on the learning data that associates the utterance text with the speech data.
The text replacement unit that generates data in which the utterance text in which some words of the utterance text are replaced with phonetic symbol strings and the voice data are associated with each other as the learning data.
A statistical model generator equipped with. The statistical model generator according to claim 11.
The learning data is a voice corpus,
Statistical model generator. The statistical model generator according to claim 11.
The model learning unit generates the statistical model by DNN (Deep Neural Network).
Statistical model generator. The voice synthesizer in the voice synthesis system according to claim 1.
The storage unit that stores a user dictionary, which is data including information corresponding to a word and a phonetic symbol string for the word,
Regarding the target text, which is the text to be voice-synthesized, the phonetic symbol string in which the word included in the user dictionary among the words included in the target text is associated with the word in the user dictionary. The user dictionary application part to be replaced with
The speech synthesis processing unit that generates synthetic speech by performing speech synthesis processing based on the statistical model for the target text after the replacement,
A voice synthesizer equipped with. Information processing device
Steps to generate a statistical model used for speech synthesis based on learning data that associates spoken text with speech data,
A step of generating as the training data data in which the utterance text in which some words of the utterance text are replaced with phonetic symbol strings and the voice data are associated with each other.
A step of storing a user dictionary, which is data containing information corresponding to a word and a phonetic symbol string for the word.
Regarding the target text, which is the text to be voice-synthesized, the phonetic symbol string in which the word included in the user dictionary among the words included in the target text is associated with the word in the user dictionary. Steps to replace with, and
A step of generating a synthesized voice by performing a voice synthesis process based on the statistical model on the target text after the replacement.
A speech synthesis method that executes.

Images