JP4964695B2 - Speech synthesis apparatus, speech synthesis method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To naturally combine a natural voice and a synthesized voice by detecting a natural voice section partially matching a synthesized voice section and by providing the synthesized voice with prosody (intonation/rhythm) information of the natural voice section. <P>SOLUTION: The voice synthesis device includes a recorded voice storage means, an input text analysis means, a recorded voice selection means, a connection boundary calculation means, a rule synthesis means, and a connection synthesis means. The voice synthesis device further includes a natural voice prosody section determination means determining a section partially matching the recorded natural voice in the synthesized voice section, a natural voice prosody extraction means extracting the natural voice prosody of the matching part, and a hybrid prosody generation means generating prosody information of the entire synthesized voice section using the extracted natural voice prosody. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to a speech synthesizer and a speech synthesis method. In particular, the present invention relates to a hybrid speech synthesizer that creates announcement speech by using synthesized speech and real voice together.

  Systems that provide information using voice, such as automatic announcements at railways and public facilities, or telephone information provision systems at banks and securities companies, are becoming widespread. Voice messages used in these application fields are characterized by many fixed expressions. For example, in an example of broadcasting on a railway, the announcement voice “Soon to go to Tokyo on line 5” is often used in various variations by changing the part of “line 5” and “Tokyo”.

  For this reason, in such an announcement system used in the automatic announcement field with many fixed expressions, a fixed expression part is prepared as a part of a real voice, and an announcement sound is created by appropriately combining them. For example, in the above-described announcement example, a technique of creating an announcement voice as a sentence by combining voice components “soon”, “to 5th line”, “to go to Tokyo”, “we will go” is adopted. This is called a recording / editing system, and is a system (system) which is currently mainstream in the announcement field as described above.

  In this recording / editing system, since the voice parts are combined, the quality in terms of real voice is high. However, since the audio components are cut into pieces, it is difficult to match the inflection and the rhythm, and the quality in terms of the naturalness of the audio falls. Furthermore, since the audio parts must be recorded in advance, re-recording is required when new words are added, which is a method for cost and convenience.

  On the other hand, by synthesizing speech using a rule synthesis method called speech synthesis or TTS (Text-to-Speech) technology, it is possible to generate speech data that reads an arbitrary sentence. The rule synthesis method is described in detail in “Digital Speech Processing” (Sadaaki Furui, Tokai University Press) and “Progress in Speech Synthesis” (VAN SANTEN et al., Springer). However, this method can convert an arbitrary sentence into speech, but it is not as good as the recording and editing method in terms of the real voice and naturalness.

  Therefore, in order to solve the problems of the above-described recording and editing method, a hybrid method that combines the recording and editing method and the rule composition method has been considered. This is a typical expression part, for example, "Soon" and "I will continue" in the above sentence example, using recorded voice parts, the contents may change " The “Tokyo bound” part is a method of creating a voice message by combining the synthesized voice parts generated by the TTS technology. As a result, it is possible to have the flexibility of the TTS technology that can cope with an arbitrary phrase while maintaining the real voice feeling of the recording and editing method.

  However, since the problem of the recording and editing system that the naturalness of inflection and rhythm is low still remains in this hybrid system, the following hybrid system at a variable position is conceivable as a method for solving this problem. This is because the position where the synthesized speech part and the real voice part are combined is not a word / phrase unit as in the hybrid system example above, but a voiced consonant or a phoneme position with low power is dynamically searched. In this case, the coupling position between the audio components is made inconspicuous by determining the coupling position more freely. Furthermore, the overall naturalness can be improved by adjusting the inflection and rhythm of the synthesized voice part according to the front and rear real voice parts.

Further, from the opposite viewpoint, a technique for improving the real voice and naturalness of the speech synthesis method when there are many typical expressions has been invented. For example, the invention disclosed in Patent Document 1 discloses a technique that uses information acquired from the real voice itself as prosodic (intonation / rhythm) information of a typical expression part when performing speech synthesis. By using this technology, although it is a speech synthesis method, the naturalness of the inflection / rhythm is almost the same as that of the real voice in the typical expression part.
JP-A-11-249677

  By using the above-mentioned variable position hybrid method, the coupling position between the real voice part and the synthesized voice part becomes inconspicuous, and the naturalness is also improved by adjusting the inflection and rhythm of the synthesized voice part to match the real voice. It will be. However, since the adjustment technique is still insufficient, it is found that the synthesized speech portion is synthesized speech, and as a result, there is a problem that the quality is greatly lowered.

  In view of this, it is conceivable to solve the problem by applying the technique disclosed in Patent Document 1 to the speech synthesis part and making its naturalness almost equivalent to the real voice. However, it cannot be solved by simply combining the two methods. This is only when a voice message is created in a system that implements the above-mentioned variable position hybrid method, and the same content as the section in which the synthesized speech technology is used, more precisely, when the real voice of the same phoneme is recorded, This is because the technique of Patent Document 1 can be applied only to the same phoneme part. In other words, if the same content is not recorded, the application itself is impossible. Considering that the section in which the synthesized speech is used is an arbitrary phrase, it must be assumed that the possibility that the same voice is recorded is quite low.

  The present invention has been made in view of the above problems, and detects a real voice segment that partially matches a synthesized speech segment when the variable position hybrid method is implemented, and prosody (inflection / rhythm) of the real voice segment. The problem is solved by adding information to the synthesized speech. It is another object of the present invention to provide a hybrid speech synthesizer capable of creating a voice message with a higher sense of natural voice and naturalness by combining the real voice and the synthesized voice more naturally than in the prior art.

The present invention includes an input text analysis unit for converting the accepted text pronunciation text-to-speech, and recording audio storage unit for storing the text and recorded real voice data sentence section that is set in advance by the real voice in advance, the A recorded voice selection unit that compares the sentence stored in the recorded voice storage unit with the pronunciation text and selects the voice data and sentences used for speech synthesis from the recorded voice storage unit, and the pronunciation text and the selected sentence A connection boundary calculation unit for determining a boundary between a synthesized voice section for generating voice by voice synthesis and a real voice section for generating voice from the real voice data, and a voice set in advance based on the determined synthesized voice section A rule synthesizer that generates speech synthesis data using segments and prosodic models, real voice data corresponding to the real voice segment, and the generated speech synthesis data In a hybrid speech synthesizer comprising: a connection synthesizer that generates a synthesized speech sentence corresponding to a subsequently input text, the real voice in a synthesized speech section that uses the synthesized speech determined by the connection boundary calculation unit A real voice prosody section determining unit that determines a real voice prosody section that uses data prosody, a real voice prosody extraction unit that extracts the prosody of the section determined by the real voice prosody section determiner from the selected real voice data, and the extraction prosodic by human voice, which is a hybrid prosody generation unit for generating prosody information for the entire synthetic speech segment from the prosodic model, wherein the rule-based synthesis unit, with the synthesized speech segment, the prosody and the speech segment generates speech synthesis data by the information, the recorded speech selection unit compares the pronunciation text and the sentence of the recorded speech storage unit, the sentence Among them, the voice data and sentence having the largest number of syllables whose pronunciation coincides with the pronunciation text are output, and the connection synthesis unit generates prosody information in the real voice segment, the real voice prosody segment, and the hybrid prosody generation unit The real voice prosody section determination unit uses the prosody of the real voice data based on the longest match in syllable units when comparing the pronunciation text and the selected sentence. that determine the location.

  Therefore, according to the present invention, in the hybrid speech synthesizer capable of combining the voice data and the synthesized speech to create a voice message with a high sense of natural voice and naturalness, further improving the naturalness of the synthesized speech section. Can do. This makes it possible to create a voice message with higher naturalness and higher quality.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is a block diagram of a speech synthesizer showing a first embodiment of the present invention. The voice synthesizer 1 includes a CPU (processor) 3 that performs arithmetic processing, a memory 2 that stores data and programs, a storage device 5 that stores data and programs, a display device 4 that displays calculation results, and a voice. Is included. The memory 2 is loaded with a text generator 7 as a program for generating text and a speech synthesizer 8 for converting the text output from the text generator 7 into speech, and is executed by the CPU 3. In the voice synthesizer 8, the text is converted into voice data and output, and the CPU 3 transmits the voice data to the voice reproduction device 6 to instruct voice output.

  As will be described later, the text generation unit 7 functions as a software module that generates guidance information in a car navigation device or the like as text. The storage device 5 stores various data used by the speech synthesizer 8 as will be described later.

  FIG. 2 is a functional block diagram of the speech synthesizer 8 shown in FIG.

  In FIG. 2, the basic configuration of the speech synthesizer 1 and the speech synthesis method of the present invention uses a real voice component (a real voice recording voice data) recorded by analyzing the content of the text 10 input from the text generator 7. An input text analysis unit 20 that generates a pronunciation text 21 for separating a part and a part that uses a synthesized voice part, and a recorded voice storage unit 30 that stores a large number of sentence recording voices (= real voice recording voice data) used for voice synthesis. Recording voice selection unit 40 for determining a recorded sentence that can be used as a material of a real voice part (= part or all of the voice recorded voice data), and a synthesized voice section and a voice recording by voice synthesis from the selected recorded sentence A connection boundary calculation unit 50 that determines a boundary (boundary) between the voice data and the voice segment, and voice recording voice data in the synthesized voice section using the synthesized voice A real voice prosody section determining unit 60 for determining a portion where the prosody can be used as it is, a real voice prosody extracting unit 70 for extracting prosody information corresponding to the portion determined using the prosody of the real voice recorded voice data from the real voice recorded voice data, Based on the extracted prosodic information, a hybrid prosody generating unit 80 for generating hybrid prosodic information for the entire section for speech synthesis by complementing the prosody of a part to be newly synthesized with speech, and a speech element based on the generated hybrid prosodic information A rule synthesizing unit 100 that performs speech synthesis using data in the fragment database 110 and the prosodic model 120, and a connection that generates a synthesized speech sentence corresponding to the input text by connecting the real voice component section and the synthesized speech section. It is comprised with the synthetic | combination part 90. FIG.

  Next, in FIG. 2 which is the basic configuration of the present invention, what kind of device should be specifically configured for each element will be described. Here, a specific description will be given of the case where the present speech synthesizer 1 is implemented as a device that synthesizes a guidance speech in a car navigation system.

  First, the input text 10 is input using, for example, a digital pen such as a keyboard or a touch panel, and passed to the input text analysis unit 20 as electronic data. In addition to these, various input devices such as a character recognition device (OCR) can be considered. In addition, input text to be synthesized in advance may be stored in the database of the storage device 5, or new text may be dynamically generated using a text processing device or text processing method (not shown). .

  In the case of the first embodiment, the processing flow will be described on the assumption that the input text 10 shown in FIG. 3 is “Soon after the Shibuya Minami Bypass is right turn” data is input to the speech synthesizer 1.

  Next, the recorded voice storage unit 30 is a database that stores a large number of recorded voice data that can be used as a real voice part for a text to be synthesized. The recorded voice storage unit 30 can be easily realized by using various database devices and data storage techniques, also called a voice corpus.

  In the present embodiment, the voice recording voice data is obtained by recording in advance a voice (sentence recording voice) in which a human being speaks a predetermined sentence. As an example of the recorded voice storage unit 30, for example, data can be stored in the form of a table (or relational database) having the structure shown in FIG. Here, the real voice recording voice data itself is stored in advance in the voice file 220 in the recording voice storage unit 30. Further, the recorded voice storage unit 30 stores the recorded voice voice data of the voice file 220 converted to text as the recorded voice text 210, and information obtained by converting the pronunciation of the recorded voice text 210 into text is recorded voice pronunciation text. 230. Also, the audio file 220 corresponding to the recorded audio text 210 and the recorded audio pronunciation text 230 are associated as a series of data by the ID 200. The recorded voice text 210, the voice file 220, and the recorded voice pronunciation text 230 corresponding to these IDs 200 are recorded voice information.

  In the recorded voice pronunciation text 230 shown in FIG. 4, symbols indicating accent positions and phrase delimiters are deleted, and only syllables (units composed of vowels and consonants such as a, ka, and gi) are included. Although only the kana characters shown are described, they can be stored in a format of phonetic symbol string text including symbols such as accents. In addition, there is enough information to indicate what phoneme (unit consisting of vowels such as A, K, G, and I and consonants) is composed, such as a phoneme character string or a phoneme ID string. It only has to be stored. In the following, the syllable unit, phoneme unit, etc. will be divided into more abstract phonemes.

  Next, the input text analysis unit 20 performs an analysis process called a text analysis process or a natural language analysis process on the text 10 input from the text generation unit 7, and later a recorded voice selection unit 40 or a connection boundary. An object is to extract or convert information in a form that can be easily used by the calculation unit 50 or the like. The specific processing here depends on the implementation such as what kind of real voice prosody section is selected. In the case of the first embodiment, a process of converting the input text 10 shown in FIG. 3 into a pronunciation text 21 shown in FIG. 5 is performed. The method of converting the input text 10 in FIG. 3 into the pronunciation text 21 in FIG. 5 can be realized by performing natural language analysis processing, specifically, morphological analysis using word dictionary data. This method is disclosed in, for example, “Natural Language Processing” (Masao Nagao, Iwanami Shoten). As another technique, it is possible to use a pattern matching technique that does not require dictionary data. In this case, the recorded voice storage unit 30 can be realized by having a matching pattern as shown in FIG. 6 instead of the recorded voice pronunciation text 230 of FIG. The matching pattern in FIG. 6 includes a recorded voice text 2101 corresponding to the ID 2001 and a matching pattern 2102 including a main part of the recorded voice text 2101. In this case, by applying a character string matching process (disclosed widely in the above document etc.), the symbol “*” is regarded as a wild card (can be matched with any character string) and is best (wild card This is a process of searching for the recorded audio information that matches (the part is the shortest).

  Next, based on the information analyzed by the input text analysis unit 20 (the pronunciation text 21 in FIG. 5), the recorded voice selection unit 40 is the closest to the input text 10, that is, the real voice recording that contains many real voice parts having the same contents. This is processing for selecting audio data (audio file 220) from the recorded audio storage unit 30.

  This processing can be realized by counting the number of syllables included in common in the pronunciation text 21 and the recorded voice pronunciation text 230. In the case of the first embodiment, the input text analysis result (pronunciation text 21) of FIG. 5 is compared with the recorded voice pronunciation text 230 of each real voice recording voice data stored in the recorded voice storage unit 30.

  As a result of this comparison, the number of coincident syllables as shown in FIG. 7 can be calculated for each recorded voice data. FIG. 7 shows, as a table, the number of syllables 240 whose pronunciation coincides with the pronunciation text 21 in the recorded voice pronunciation text 230 corresponding to the recorded voice text 210 of the recorded voice storage unit 30. Of these, the recorded voice information including the real voice recorded voice data having the largest number of coincident syllables 240 may be used as the output result of the recorded voice selection unit 40. In this syllable comparison process, the order of syllable comparison must not be changed. For example, FIG. 8A shows a comparison result between the recorded voice pronunciation text 230 with ID = 2 and the pronunciation text 21 shown in FIG. In FIG. 8A, “mamonaku” corresponds one-to-one, there is no matching syllable for “shi”, “bu”, and “ya”, and after “minami” there is a corresponding one-to-one speech. Therefore, the comparison process can be easily performed. However, depending on the pronunciation text, there is a case where the number of matching syllables in the latter half is reduced by matching the syllable on the left side of the text with priority. For such a case, for example, the character string comparison method is the leftmost shortest matching method, or a plurality of matching patterns are generated and a matching pattern having the largest number of matching syllables is selected among all the matching patterns. It is possible to select a more appropriate matching voice by, for example.

  FIG. 8B shows a comparison result between the recorded voice pronunciation text 230 and the pronunciation text 21 that are partly different from ID = 2. As in this example, the number of coincidence syllables is greater than in FIG. 8A, but there also appears a case where a non-matching portion, that is, a section where a synthesized speech component is generated by speech synthesis processing becomes extremely short later. Depending on the speech synthesis process, there is a method that is not suitable for generating a short synthesized speech. In such a case, priority is given to the matching pattern in FIG. It is also conceivable to do. As a method for determining the priority order of the matching patterns in FIG. 8A and FIG. 8B, for example, the pronunciation text 21 obtained from the input text 10 and the recorded syllable pronunciation text 230 are compared and recorded. With reference to the mismatch cost shown in FIG. 16, the total mismatch cost as shown in FIG. 15 can be calculated for each number of characters in the phonetic text 230 where there was a mismatch. By comparing the mismatch costs, it is possible to give priority to the match pattern in FIG. 15A having a smaller mismatch cost even if the number of matching syllables is small. FIG. 16 is a table in which the relationship between the number of mismatch characters and the mismatch cost is set in advance.

  Here, the discrepancy cost setting method shown in FIG. 8 is based on the input text 10 (pronunciation text 21) in the recorded voice information (the real voice recording voice data corresponding to the recorded voice pronunciation text 230 shown in FIG. 15) in the present invention. In consideration of the fact that only the part that matches the syllable is used as it is, it is necessary to set it according to the length of the synthesized speech section combined between the real voices (the real voice recording voice data). In the example of FIG. 8, the mismatch cost is simply defined according to only the number of mismatch characters, but the mismatch cost is set according to the phoneme environment such as the type of mismatch characters and the phonemes before and after the mismatch character. You can also. If such a setting method is adopted, when combined in a silent section of real voice recording voice data, even if the number of mismatched characters is extremely small, a smoother matching pattern is prioritized by reducing the mismatching cost. It is also possible.

  This ordering change method is performed when the speech synthesis unit (rule synthesis unit 100) determines what characteristics to adopt, that is, when the system of the present invention is implemented, and when the recorded speech is used. The processing in the selection unit 40 can be performed not in syllable units but in finer phoneme units, and the processing method is the same as in syllables.

  Here, whether processing is performed in syllable units or phoneme units depends on how small the speech synthesis unit (rule synthesis unit 100) supports speech synthesis. If the rule synthesizing unit 100 only supports speech synthesis in units of syllables, hybrid prosody generation from the recorded speech storage unit 30 associated with the recorded speech selection unit 40 and the subsequent connection boundary calculation unit 50 is performed. Everything up to section 80 must be processed in syllable units.

  On the other hand, if the rule synthesizing unit 100 supports speech synthesis in phoneme units, the processing from the recorded speech storage unit 30 to the hybrid prosody generation unit 80 can select either syllable units or phoneme units. It is. If the purpose of the present invention is to combine the real voice (real voice recording voice data) and the synthesized voice more smoothly, it is desirable to base the processing on a phoneme unit, which is a more detailed unit. .

  Next, the connection boundary calculation unit 50 uses which part of the real voice recording voice data (voice file 220) selected by the voice recording selection unit 40 as a real voice part as it is, and which part is synthesized voice processing. Decide whether to use the synthesized speech component generated in. As the simplest method, as a result of the syllable comparison process executed by the recorded voice selection unit 40, the real voice of the recorded voice data (speech file 220) is used for the matched syllable part, and other non-matching parts. For, a method of using a synthesized speech component generated by speech synthesis can be used.

  However, since the actual voice (real voice) is a voice in which the syllables are smoothly connected, the real voice and the synthesized voice cannot simply be smoothly combined between all syllables. As a method for solving this problem, there is a method to be called a variable position hybrid system described below.

  According to this hybrid method, the connection cost indicating the ease of connection (ease of combining real voice and synthesized speech) between all syllables or all phonemes is calculated, and the connection cost is the highest. The process of extending the length of the synthesized speech component is performed so that connection (combination of real voice and synthesized speech) is performed at a small location.

  More specifically, it is possible to select a pause position at the beginning of the unvoiced consonant or a phoneme boundary where the speech power is sufficiently low, and perform a process of expanding the synthesized speech part to this phoneme boundary. In other words, the combined position of the real voice and the synthesized voice is not always constant, and the combined position is dynamically changed according to the content.

  For example, in the case of the first embodiment, consider a case where the real voice recording voice data ID2 is selected by the comparison of FIG. 8A (FIG. 9). In this case, the real voice usage part determined only by the phoneme / syllable coincidence comparison process is “mamonaku” and “minami bypass ootsudesde”, and “shibuya” in the middle uses the synthesized speech component generated by speech synthesis. . However, “Yu” of “Shibuya” and “Mi” of “Minami Bypass” are both voiced voices, and noise is generated when voices are combined between them.

  Therefore, a process of expanding the synthesized voice part to a silent section or a part with a small voice power is performed. In the case of the example in FIG. 9, the coupling position does not change because it is a silent section immediately before “Shibuya”. On the other hand, on the back side of “Shibuya”, when a silent section or a portion with low voice power is searched for next, a “pa” portion of “bypass” is found. A plosive phoneme “p” exists at the head of the syllable “pa”, and here, a silent section in which the speech signal is zero occurs once. When the silent voice (real voice recording voice data) and the synthesized voice are combined in the silent part, no noise is generated. As a result, from the connection boundary calculation unit 50, as shown in FIG. 10, the selected recording voice ID = 2, “mamonaku” and “pasno sakiusetsudes” which are sections using the real voice component, and synthesis “Shibuyamanamibai”, which is a section that uses audio parts, is output.

  Next, the real voice prosody section determination unit 60 performs a process of determining a section in which prosodic information of the original real voice recording voice data can be used among syllable sections that are subjected to voice synthesis processing as a synthesized voice component. This processing is a basic part of the present invention, and the technique (such as the above-mentioned conventional variable position hybrid method) is used to achieve a smooth connection between the real voice (real voice recording voice data) and the synthesized voice (connection boundary calculation unit 50). (B) Basically, a process for identifying a portion where prosodic information extracted from real voice recording voice data can be used from the expanded synthesized voice section.

  Hereinafter, the case of the first embodiment will be specifically described. The connection boundary calculation unit 50 reduces the matching syllable part (upper and lower solid lines) shown in FIG. 8A to the matching syllable part shown in FIG. 9 by the real voice prosody section determination unit 60. In other words, the synthesized speech portion which is a non-matching portion is expanded from “Shibuya” to “Shibuyamanamibai”.

  Here, when the processing result information as shown in FIG. 10 is input to the real voice prosody section determining unit 60, the synthesized voice section “Shibuyamanamibai” and the section “Nakanonamimi” of the real voice recording voice data corresponding to this synthesized voice section. Comparison processing with “buy” is performed. In this comparison as well, the matching part can be determined using a character string matching method such as the leftmost shortest matching described above.

  The real voice prosody section determination unit 60 uses the longest match method in units of syllables to indicate a section where the original recorded voice data and phonology (syllables) coincide with each other in the synthesized speech section “Shibuyamanamibai” with a broken line in FIG. As shown, it can be determined to be “minamibai”.

  From the above processing, the real voice prosody section determination unit 60 outputs the real voice prosody section using the prosody of the real voice recording voice data from the synthesized voice section “Shibuyama Minamibi” as “Minamibi” as shown in FIG. That is, the real voice prosody section determination unit 60 adds information on the real voice prosody section that uses only the prosody of the real voice recording voice data in the section that uses the synthetic voice in addition to the real voice section and the synthetic voice section.

  Next, the real voice prosody extraction unit 70 performs a prosody information extraction process for a real voice recording voice section corresponding to the synthesized voice section output from the real voice prosody section determination unit 60. Prosodic information refers to information indicating the fundamental frequency of speech, the duration of phonemes / syllables, and the temporal change in speech power. This prosody extraction process can be realized, for example, by determining what phoneme or syllable constitutes the input speech and its position by automatic segmentation processing using speech recognition technology. The fundamental frequency and audio power can be realized by using a general F0 (basic frequency) extraction process or power calculation process used in the audio signal processing technology. Alternatively, the above-mentioned prosodic information can be extracted from the entire real voice recording voice data in advance, and the real voice prosody extraction unit 70 can extract the prosodic information part corresponding to the synthesized voice section. An example of information output from the real voice prosody extraction unit 70 in the case of the first embodiment is shown in FIG. Here, prosodic information (starting point and end point of fundamental frequency, duration length) of the section “Nakanonamibai” of the voice recording voice data corresponding to the synthesized speech section “Shibuyamanamibai” is extracted for each syllable.

  Next, the hybrid prosody generation unit 80 generates prosody information for the synthesized speech section based on the prosody information for the partial section of the real voice recorded speech data output from the real voice prosody extraction unit 70. This processing uses the information in the prosody information extracted by the real voice prosody extraction unit 70 for the part that matches the recorded voice data and the synthesized voice, and ignores the extracted information for the part that does not match, or A process of generating prosodic information of a section corresponding to the synthesized speech is performed with reference to the extracted information.

  The case of the first embodiment will be specifically described. In the hybrid prosody generation unit 80, when the prosody information for the real voice recorded voice data section “Nakanonamibai” shown in FIG. 12 is input from the real voice prosody extraction section 70, the synthesized voice section “Shibuya” corresponding to the real voice recorded voice data section is input. The portion in which the prosodic information extracted in “southern” can be used is determined. The determination process in the hybrid prosody generation unit 80 can also be realized by using the character string matching process that has been used in the various processes described above. In the case of this example, since the syllabary of the “southern” part matches, the prosody information extracted from the recorded voice data can be used as the prosody information of the part. On the other hand, for the section “Shibuya” where the syllables do not match, the prosody information for “Shibuya” may be newly generated using the prosody generation processing included in the rule synthesis unit, or the prosodic information of “Nakano” Therefore, it can be generated by a prosody conversion process (for example, a process of uniformly expanding / contracting and moving the fundamental frequency and phoneme continuation length so as to be continuous with the preceding and succeeding portions).

  FIG. 13 shows an example of a syllable string with prosodic information in which the prosodic generation unit 80 performs prosody generation processing to generate prosodic information for “Shibuya”. This prosody generation processing is disclosed in “Digital Speech Processing” (Sadaaki Furui, Tokai University Press), “Progress in Speech Synthesis” (VAN SANTEN et al., Springer) and the like.

  Next, the rule synthesizing unit 100 uses the syllable string with prosodic information (FIG. 13) output from the hybrid prosody generating unit 80 as input, and performs speech synthesis processing so that the prosody specified in the syllable string with prosodic information is realized. Do. At that time, conversion to synthesized speech is performed with reference to the speech segment database 110 and the prosodic model 120 which are components of the synthesized speech. Since this rule composition processing is also widely disclosed in the above-mentioned documents and the like, description thereof is omitted. In the case of the first embodiment, as a result of this processing, a synthesized speech component “Shibuyamanamibi” realizing the prosody of FIG. 13 is generated by speech synthesis processing as shown in FIG.

  Finally, the connection synthesis unit 90 connects and synthesizes (combines) the parts of the real voice recording voice data output from the recorded voice selection unit 40 and the connection boundary calculation unit 50 and the synthesized voice component output from the rule synthesis unit 100. It is processed and output as hybrid synthesized speech 130. This connection synthesis processing can also be realized by simply combining synthesized speech, and waveform superimposition signal processing such as TD-PSOLA (Time Domain Pitch Synchronous Overlap Add) is used for the combined portion. It is also possible to join more smoothly.

  In the case of the first embodiment, the synthesized voice component “Shibu Minami” output from the rule synthesizing unit 100 is added to the components “mamonak” and “pasno akitsutsudes” of the real voice recording voice data output from the connection boundary calculating unit 50. “Bi” is combined, and a hybrid synthesized speech corresponding to “mamonaku” “shibuyamaminamibai” “pasnosakiousetsudesu” is output.

  In the hybrid synthesized speech output here, the sections of “mamonaku” and “pasnosakiousetetsudes” are completely recorded voice data, and the section of “shibuyamanamibai” is synthesized speech, but “minamibai” Since the part is a synthesized voice that directly realizes the prosody of the recorded voice data, it is possible to realize a synthetic voice that is prosodic in nature and has the prosody continuously connected to the latter half of “Pasano Sakiusetsudes”. As described above, according to the present invention, a real voice segment partially matching the synthesized speech segment when the above-described variable position hybrid synthesis method is implemented is detected, and the prosodic (intonation / rhythm) information of the real voice segment is synthesized. It is possible to provide a hybrid speech synthesizer capable of creating a voice message having both a higher feeling of natural voice and naturalness by adding to the voice and combining the voice and the synthesized voice more naturally than in the prior art.

  In the above, the real voice prosody section determining unit 60 uses the prosody of the real voice recorded voice data for the pronunciation text 21 converted from the input text 10 and the recorded voice pronunciation text 230 based on the longest match in syllable units. However, the longest match in phoneme units may be used for comparison between the pronunciation text 21 and the recorded voice pronunciation text 230.

<Embodiment 2>
Next, an embodiment when the present invention is specialized for a car navigation system will be described.

  FIG. 17 is a configuration diagram when the present invention is implemented as a car navigation system. The input text 10 in FIG. 2 of the first embodiment is transferred from the car navigation device (particularly the utterance content determination unit) 310 in FIG. Also, the hybrid synthesized speech 130 output from the connection synthesis unit 90 in FIG. 2 is directly output from the audio reproduction device 320 such as a speaker (including an amplifier) in FIG. 17 of the second embodiment. Become. The rest of the configuration is the same as in FIG. 2 of the first embodiment, and the contents of processing in each processing unit and the flow of those processing are basically the same as those described in the first embodiment.

  Therefore, here, the processing flow in the speech synthesizer 8 of the present invention will be described with reference to the flowchart of FIG.

  First, when the hybrid synthesizing process shown in FIG. 18 is started, the process waits until the text to be read (spoken) determined by the utterance content determination unit of the car navigation apparatus is input. If a read-out text is input, the input text 10 is transferred to an input text analysis process 410 and converted into an internal representation format for a speech synthesis process. Details of this processing are as described in the input text analysis unit 20 of the first embodiment.

  Subsequently, the internal representation data is transferred to the recorded voice selection processing 420, and the recorded voice pronunciation text 230 is input text 10 (from the real voice recorded voice data (voice file 220) stored in the recorded voice storage unit 30. The recorded voice data (voice information) having the pattern that best matches the pronunciation text 21) is selected. The details of this selection processing are as described in the recording voice selection unit 40 of the first embodiment.

  If an appropriate matching pattern cannot be selected in the recorded voice selection process 420, the input text 10 and the internal expression data are passed to the rule synthesis process 430, and the entire input text is converted into a synthesized voice and output. Is done. That is, when there is no appropriate matching pattern, all the reading (speech) text is output as synthesized speech.

  On the other hand, when an appropriate matching pattern can be selected from the recorded voice pronunciation text 230 in the recorded voice selection process 420, the matched recorded voice information (recorded voice information in the horizontal row of data shown in FIG. 4) is used to calculate the connection boundary. Passed to processing. The details of this process are as described in the connection boundary calculation unit 50 of the first embodiment.

  Subsequently, the real voice prosody section determination process 440 is activated. In this process, for all connection boundaries determined by the connection boundary calculation process (boundaries in selected real voice recording voice data), synthesis using real voice prosody or synthesized voice is used. The process of determining whether it is a voice segment is repeatedly executed. Details of this processing are as described in the real voice prosody section determining unit 60 of the first embodiment.

  Subsequently, the real voice prosody extraction processing 450 is started. In this process, the prosody extraction process is repeatedly executed for all sections determined as the real voice prosody section in the real voice prosody section determination process 440. The details of this processing are as described in the real voice prosody extraction unit 70 of the first embodiment.

  Subsequently, the hybrid prosody generation process 460 is activated. In this process, prosody information generation processing is repeatedly executed for all sections determined as synthesized speech sections in real voice prosodic section determination processing 440 and all phonemes in the sections. Details of this processing are as described in the hybrid prosody generation unit 80 of the first embodiment.

  Subsequently, the rule composition process 470 is activated. In this process, all synthetic speech sections are converted into synthesized speech according to the prosodic information generated in the hybrid prosody generation process. Details of this processing are as described in the rule composition unit 100 of the first embodiment.

  Subsequently, the real voice segment cutout process 480 is activated. This process divides the real voice recording voice data data (speech file 220) selected from the voice storage unit in good agreement with the input text, and a portion corresponding to the real voice prosody section determined by the real voice prosody section determination processing 440. Performs processing to cut out and output only the voice recording voice data data.

  Finally, connection composition processing 490 is activated. In this process, the synthesized voice data corresponding to the synthesized voice section and the real voice recording voice data data corresponding to the real voice prosody section output from the rule synthesis process 470 and the real voice section cutout process 480, respectively. In accordance with the order, the process of connecting and outputting sequentially is repeated. As a result, the final output from this process is hybrid synthesized speech data (partly synthesized speech and partly real voice recording speech data) corresponding to the input text.

<Embodiment 3>
Next, an embodiment when the present invention is implemented as a hybrid synthesized speech editing tool having a user interface will be described with reference to FIG.

  FIG. 19 is a configuration in which a text input unit 510, a user input unit 520, and an information display unit 530 are added to the basic configuration of the present invention shown in FIGS.

  Here, the text input unit 510 is an input device for inputting a read-out (speech) text to the speech synthesizer 1 of the present invention. For example, a conventional user interface device such as a keyboard can be used. .

  When text is input by the text input unit 510, the processing described in the first embodiment or the second embodiment is executed, and the hybrid synthesized speech 130 is output.

  However, in the third embodiment, the information of the results processed by each processing unit from the recorded voice selection unit 40 to the hybrid prosody generation unit 80 is separately passed to the information display unit 530 (dotted arrow) ) And can be presented to the user. Similarly, by passing the information designated by the user through the user input unit 520 to each processing unit from the recording voice selection unit 40 to the hybrid prosody generation unit 80, the information output by each unit is changed to a specific content. Is possible.

  The information display unit 530 is a device for presenting various types of information to the user, and for example, a graphical display device such as a display device can be used. For example, the information display unit 530 may use the display device 4 shown in FIG. 1 of the first embodiment. An example of information displayed on the information display unit 530 is shown in FIG.

  In FIG. 20, the reading text input in the upper input text field 531 is analyzed by the analysis result information (pronunciation text 21) as a result of passing the input text to the input text analysis unit 20, and the recorded voice selection unit 40 automatically determines the match. The text of the recorded voice ID2 selected (recorded voice pronunciation text 230) is displayed. In addition, the degree of phoneme coincidence, which is the basis for coincidence determination, is displayed by the number of corresponding lines. In this way, the recorded voice selection unit 40 can display what kind of recorded voice is selected under what match determination.

  Further, in FIG. 20, the section determined to use the real voice recorded voice data as a result of the connection boundary calculation unit 50 and the real voice prosody section determination unit 60 is displayed in italic characters. As described above, it is possible to graphically display which section of the input text is the synthesized voice and which section is the real voice recording voice data. In addition to this, various display methods such as dividing by color or enclosing by a rectangle or a rounded rectangle can be considered.

  Further, in the central part of FIG. 20, prosody information for the real voice segment extracted by the real voice prosody extraction unit 70 and hybrid prosody information 532 for the synthesized speech segment generated by the hybrid prosody generation unit 80. Thus, by displaying as a graph of time and frequency (F0), it is possible to intuitively indicate what kind of sound the hybrid synthesized speech that is output becomes.

  On the other hand, the user input unit 520 is a user interface device. For example, the user inputs and designates information (for example, a recording voice ID to be selected by the recording voice selection unit) through a mouse, a keyboard, and the like. A process of passing to an appropriate processing unit (for example, the recording voice ID to the recording voice selection unit 40 and the connection boundary information to the connection boundary calculation unit 50) is performed. Here, the information that can be specified by the user is a recording voice ID to be output instead of the information selected by the recording voice selection unit 40, or is determined by the connection boundary calculation unit 50 or the real voice prosody section determination unit 60. It may be a section of a real voice-synthesized speech section to be output instead of a thing. For example, by clicking the recorded voice text at the bottom of the screen with the mouse, the alternative recorded voice text is displayed in the menu, and the user is allowed to specify what should be actually selected. Make it possible.

  Similarly, an interaction such as clicking or dragging with a mouse each of a real voice section (part displayed in italics) and a synthesized voice section (part normally displayed) shown in the analysis result of the information display unit 530 in FIG. It is possible to allow the user to specify which part is the synthesized voice and which part is the real voice.

  Furthermore, on the graph display screen of the prosodic information on the screen, the prosodic information output by the hybrid prosody generating unit 80 (the curve displayed with a dotted line in the graph) is moved by dragging the mouse with respect to the synthesized speech section. Thus, the user can directly specify the prosodic information to be generated.

  As described above, the information directly designated by the user is transferred to the corresponding processing unit, and is output instead of the processing result automatically calculated by each processing unit. By adopting such a configuration, the user can directly specify the contents of the hybrid synthesized speech through the information display unit 530 and the user input unit 520.

  In each of the above embodiments, the real voice prosody section determining unit 60 converts the pronunciation text 21 converted from the input text 10 and the recorded voice pronunciation text 230 into the prosody of the real voice recorded voice data based on the longest match in syllable units. In the example shown in FIG. 1, the section using the prosody of the recorded voice data may be determined based on the coincidence of the linguistic information associated with each phoneme or syllable. Furthermore, as the language information, an interval using the prosody of the real voice recording voice data may be determined using an accent nucleus (accent lowering position) included in the recorded voice pronunciation text 230 as the position information. The linguistic information including the accent nucleus can be acquired as intermediate information for analysis processing in the input text analysis unit 20. In addition, the voice data (FIG. 4) stored in the recorded voice storage unit 30 is stored by adding the language information obtained as a result of applying the text analysis process in advance. It becomes possible to determine the real voice prosody section by determining the coincidence by the language information including the above accent kernel.

  As described above, according to the present invention, a real voice segment that partially matches the synthesized voice segment when the variable position hybrid synthesis method is implemented is detected, and the prosodic (intonation / rhythm) information of the real voice segment is obtained. It is possible to provide a hybrid speech synthesizer capable of creating a voice message having both a higher feeling of natural voice and naturalness by giving it to synthesized speech and combining natural voice and synthesized speech more naturally than in the prior art. In particular, the present invention can be applied to a car navigation device that performs guidance by voice and a device that performs guidance by voice.

The block diagram which shows the 1st Embodiment of this invention and shows the system configuration | structure of a speech synthesizer. Similarly, the block diagram which shows the 1st Embodiment of this invention and shows the flow of a process of a speech synthesizer. It is explanatory drawing which shows an example of an input text. It is explanatory drawing which shows an example of the data stored in a sound recording storage part. It is explanatory drawing which shows an example of the pronunciation text which converted the input text. It is explanatory drawing which shows an example of a matching pattern. It is a table which shows the number of syllables whose pronunciation coincides with the pronunciation text 21 in the recorded voice pronunciation text. It is explanatory drawing which shows the number of coincidence syllables of pronunciation text and sound recording sound pronunciation text, (a) shows the comparison result of sound recording sound text and sound recording text of ID = 2, (b) is another sound recording sound pronunciation text and The comparison result of pronunciation text is shown. It is explanatory drawing which shows the number of coincidence syllables of a pronunciation text and a sound recording voice pronunciation text, and shows the process which expands a synthetic | combination voice part to a silence area and a location with small voice power. It is explanatory drawing which shows an example of the sound recording which a connection boundary calculation part outputs, a real voice area, and a synthetic | combination voice area. It is explanatory drawing which shows a mode that the area in which the original real voice recording sound and a phoneme (syllable) correspond in a synthetic speech area is determined. It is explanatory drawing which shows an example of the sound recording which a real voice prosody section determination part outputs, a real voice section, a synthetic | combination voice area, and a real voice prosody. syllable. It is a table which shows the analysis result of fundamental frequency (Hz) and duration (msec). It is explanatory drawing which shows the output result in a hybrid prosody generation part. It is explanatory drawing which shows the number of coincidence syllables of pronunciation text and sound recording sound pronunciation text, (a) shows the comparison result of sound recording sound text and sound recording text of ID = 2, (b) is another sound recording sound pronunciation text and The comparison result of pronunciation text is shown. It is a table which shows the relationship between the number of mismatch characters and mismatch cost. The block diagram which shows the 2nd Embodiment of this invention and shows the flow of a process of a speech synthesizer. Similarly, it is a flowchart which shows 2nd Embodiment and shows an example of the process in a speech synthesizer. The block diagram which shows the 3rd Embodiment of this invention and shows the flow of a process of a speech synthesizer. Similarly, it is explanatory drawing which shows 3rd Embodiment and shows an example of the information displayed on an information display part.

Explanation of symbols

20 Input text analysis unit 30 Recorded speech storage unit 40 Recorded speech selection unit 50 Connection boundary calculation unit 60 Real voice prosody section determination unit 70 Real voice prosody extraction unit 80 Hybrid prosody generation unit 90 Connection synthesis unit 100 Rule synthesis unit 110 Speech segment database 120 Prosodic model 130 Hybrid synthetic speech 200 Recording speech ID
210 Recorded voice text 220 Recorded voice file 230 Recorded voice pronunciation text

Claims (16)

An input text analysis unit that accepts text to be converted into speech and converts it into pronunciation text;
And recording the voice storage unit for storing the text and recorded real voice data a preset sentence chapter by real voice in advance,
A recorded voice selection unit that compares the pronunciation text with the sentence stored in the recorded voice storage unit, and selects the voice data and sentence used for speech synthesis from the recorded voice storage unit;
A connection boundary calculation unit for determining a boundary between a synthesized speech section for generating speech by speech synthesis from the pronunciation text and the selected sentence and a real voice section for generating speech from the real voice data;
Based on the determined synthesized speech section, a rule synthesizer that generates speech synthesis data based on a preset speech segment and a prosodic model;
In a hybrid speech synthesizer comprising a connection synthesizer that generates a synthesized speech sentence corresponding to a text inputted by connecting the real voice data corresponding to the real voice interval and the generated speech synthesis data,
A real voice prosody section determination unit that determines a real voice prosody section that uses the prosody of the real voice data in a synthetic voice section that uses the synthesized speech determined by the connection boundary calculation unit;
A real voice prosody extraction unit that extracts the prosody of the section determined by the real voice prosody section determination unit from the selected real voice data;
A prosody based on the extracted real voice, and a hybrid prosody generation unit that generates prosody information of the entire synthesized speech section from the prosody model,
The recorded voice selection unit
Compare the sentence and the pronunciation text in the recording voice storage unit, and output the voice data and sentence with the largest number of syllables whose pronunciation coincides with the pronunciation text in the sentence,
The rule composition unit includes:
For the synthesized speech segment, it generates a speech synthesis data by said prosodic information and the speech unit,
The connection composition unit
Connecting the real voice section, the real voice prosody section, and the synthesized voice section in which the prosody information is generated by the hybrid prosody generation unit;
The real voice prosody section determining unit
Wherein in performing pronunciation text and the comparison of the selected sentence, hybrid speech synthesis apparatus characterized that you decide where to use the prosody of the human voice data based on the longest match in syllable. An input text analysis unit that accepts text to be converted into speech and converts it into pronunciation text;
Real voice data in which a preset sentence is recorded in real voice, and a recording voice storage unit that stores the sentence in advance,
A recorded voice selection unit that compares the pronunciation text with the sentence stored in the recorded voice storage unit, and selects the voice data and sentence used for speech synthesis from the recorded voice storage unit;
A connection boundary calculation unit for determining a boundary between a synthesized speech section for generating speech by speech synthesis from the pronunciation text and the selected sentence and a real voice section for generating speech from the real voice data;
Based on the determined synthesized speech section, a rule synthesizer that generates speech synthesis data based on a preset speech segment and a prosodic model;
In a hybrid speech synthesizer comprising a connection synthesizer that generates a synthesized speech sentence corresponding to a text inputted by connecting the real voice data corresponding to the real voice interval and the generated speech synthesis data,
A real voice prosody section determination unit that determines a real voice prosody section that uses the prosody of the real voice data in a synthetic voice section that uses the synthesized speech determined by the connection boundary calculation unit;
A real voice prosody extraction unit that extracts the prosody of the section determined by the real voice prosody section determination unit from the selected real voice data;
A prosody based on the extracted real voice, and a hybrid prosody generation unit that generates prosody information of the entire synthesized speech section from the prosody model,
The recorded voice selection unit
Compare the sentence and the pronunciation text in the recording voice storage unit, and output the voice data and sentence with the largest number of syllables whose pronunciation coincides with the pronunciation text in the sentence,
The rule composition unit includes:
For the synthesized speech section, generate speech synthesis data from the speech segments and the prosodic information,
The connection composition unit
Connecting the real voice section, the real voice prosody section, and the synthesized voice section in which the prosody information is generated by the hybrid prosody generation unit;
The real voice prosody section determining unit
Wherein in performing pronunciation text and the comparison of the selected sentence, the longest matching feature and be Ruha hybrid speech synthesizer determining where to use the prosody of the human voice data based on the phoneme units. The real voice prosody section determining unit
The hybrid speech synthesizer according to claim 1 or 2 , wherein a section using a prosody of real voice data is determined based on a match of language information associated with the syllable or phoneme . 4. The hybrid speech synthesizer according to claim 3 , wherein the language information is accent kernel position information . The connection boundary calculation unit
The hybrid speech synthesizer according to any one of claims 1 to 4, wherein information on a boundary between the synthesized speech section and the real voice section is received, and the boundary is determined based on the information . The connection boundary calculation unit
Receiving information on the boundary between the synthesized speech section and the real voice section, and determining the boundary based on the information ;
The real voice prosody section determining unit
To any one of claims 1 to claim 4, characterized in that determining the real voice prosody section based on the information accepting information about the real voice prosody interval using the prosody of the human voice data in the synthesized speech segment The hybrid speech synthesizer described. The hybrid prosody generation unit includes:
Among the synthesized speech segments, the prosody by the speech unit and the extracted real voice is set for the real voice prosody segment, and the prosody of the prosody model is set for the synthesized speech segment excluding the real voice prosody segment, 3. The hybrid speech synthesizer according to claim 1, wherein prosody information of the entire synthesized speech section is generated . Input text analysis processing that accepts text to be converted to speech and converts it to pronunciation text;
  The recorded voice storage unit stores the real voice data and the sentence used for voice synthesis by comparing the sentence and the pronunciation text of the recorded voice storage unit storing the sentence in advance with the real voice data recording the preset sentence by the real voice Recording voice selection process to select from,
  A connection boundary calculation process for determining a boundary between a synthesized speech section for generating speech by speech synthesis from the pronunciation text and the selected sentence and a real voice section for generating speech from the real voice data;
  Based on the determined synthesized speech section, a rule synthesis process for generating speech synthesis data using a predetermined speech segment and a prosodic model;
  A synthesized speech sentence by executing, by a computer, connection synthesis processing for generating synthesized speech sentences corresponding to input text by connecting the real voice data corresponding to the real voice section and the generated speech synthesis data. In a hybrid speech synthesis method for generating
  A real voice prosody section determination process for determining a real voice prosody section that uses the prosody of the real voice data in a synthetic voice section using the synthesized speech determined in the connection boundary calculation process;
  A real voice prosody extraction process for extracting the prosody of the section determined by the real voice prosody section determination process from the selected real voice data;
  Including the prosody by the extracted real voice, and a hybrid prosody generation process for generating prosody information of the entire synthesized speech section from the prosody model,
  The recorded voice selection process is:
  Compare the sentence and the pronunciation text in the recording voice storage unit, and output the voice data and sentence with the largest number of syllables whose pronunciation coincides with the pronunciation text in the sentence,
  The rule composition process is:
  For the synthesized speech section, generate speech synthesis data from the speech segments and the prosodic information,
  The connection composition process is as follows:
  Connecting the real voice segment, the real voice prosody segment, and the synthesized speech segment generated prosodic information in the hybrid prosody generation process,
  The real voice prosody section determination process includes:
  A hybrid speech synthesizing method characterized in that, when comparing the pronunciation text and the selected sentence, a portion using the prosody of the real voice data is determined based on the longest match in syllable units. Input text analysis processing that accepts text to be converted to speech and converts it to pronunciation text;
The recorded voice storage unit stores the real voice data and the sentence used for voice synthesis by comparing the sentence and the pronunciation text of the recorded voice storage unit storing the sentence in advance with the real voice data recording the preset sentence by the real voice Recording voice selection process to select from,
A connection boundary calculation process for determining a boundary between a synthesized speech section for generating speech by speech synthesis from the pronunciation text and the selected sentence and a real voice section for generating speech from the real voice data;
Based on the determined synthesized speech section, a rule synthesis process for generating speech synthesis data using a predetermined speech segment and a prosodic model;
A synthesized speech sentence by executing, by a computer, connection synthesis processing for generating synthesized speech sentences corresponding to input text by connecting the real voice data corresponding to the real voice section and the generated speech synthesis data. In a hybrid speech synthesis method for generating
A real voice prosody section determination process for determining a real voice prosody section that uses the prosody of the real voice data in a synthetic voice section using the synthesized speech determined in the connection boundary calculation process;
A real voice prosody extraction process for extracting the prosody of the section determined by the real voice prosody section determination process from the selected real voice data;
Including the prosody by the extracted real voice, and a hybrid prosody generation process for generating prosody information of the entire synthesized speech section from the prosody model,
The recorded voice selection process is:
Compare the sentence and the pronunciation text in the recording voice storage unit, and output the voice data and sentence with the largest number of syllables whose pronunciation coincides with the pronunciation text in the sentence,
The rule composition process is:
For the synthetic speech segment, it viewed including the process of generating the speech synthesis data by said prosodic information and the speech unit,
The connection composition process is as follows:
Connecting the real voice segment, the real voice prosody segment, and the synthesized speech segment generated prosodic information in the hybrid prosody generation process,
The real voice prosody section determination process includes:
When comparing the pronunciation text with the selected text, maximum features and to Ruha hybrid speech synthesis method to determine where to use the prosody of the human voice data based on a match with phoneme. The real voice prosody section determination process includes:
The hybrid speech synthesis method according to claim 8 or 9, wherein a section using a prosody of real voice data is determined based on a match of linguistic information associated with the syllable or phoneme . The hybrid speech synthesis method according to claim 10 , wherein the language information is position information of an accent nucleus . The connection boundary calculation process includes:
The hybrid speech synthesis method according to any one of claims 8 to 11, wherein information on a boundary between the synthesized speech section and the real voice section is received, and the boundary is determined based on the information . The connection boundary calculation process includes:
Receiving information on the boundary between the synthesized speech section and the real voice section, and determining the boundary based on the information;
The real voice prosody section determination process includes:
12. The real voice prosody section is received according to information on a real voice prosody section that uses the prosody of the real voice data in the synthesized speech section, and the real voice prosody section is determined based on the information. The described hybrid speech synthesis method. The hybrid prosody generation process includes:
Among the synthesized speech segments, the prosody by the speech unit and the extracted real voice is set for the real voice prosody segment, and the prosody of the prosody model is set for the synthesized speech segment excluding the real voice prosody segment, hybrid speech synthesis method according to claim 8 or claim 9 you and generates a prosodic information of the entire synthesized speech segment. A program that converts received text into synthesized speech,
  Input text analysis processing that accepts text to be converted to speech and converts it to pronunciation text;
  The recorded voice storage unit stores the real voice data and the sentence used for voice synthesis by comparing the sentence and the pronunciation text of the recorded voice storage unit storing the sentence in advance with the real voice data recording the preset sentence by the real voice Recording voice selection process to select from,
  A connection boundary calculation process for determining a boundary between a synthesized speech section for generating speech by speech synthesis from the pronunciation text and the selected sentence and a real voice section for generating speech from the real voice data;
  A real voice prosody section determination process for determining a real voice prosody section that uses the prosody of the real voice data in a synthetic voice section using the synthesized speech determined in the connection boundary calculation process;
  A real voice prosody extraction process for extracting the prosody of the section determined by the real voice prosody section determination process from the selected real voice data;
  The prosody by the extracted real voice, and a hybrid prosody generation process for generating prosody information of the entire synthesized speech section from the prosody model;
  Based on the determined synthesized speech section, a rule synthesis process for generating speech synthesis data from a predetermined speech segment and the prosodic information;
  A connection synthesis process for generating a synthesized speech sentence corresponding to the input text by connecting the real voice segment, the real voice prosody segment, and the synthesized speech segment for which prosody information is generated by the hybrid prosody generation process; Let the calculator work,
  The recorded voice selection process is:
  Compare the sentence and the pronunciation text in the recording voice storage unit, and output the voice data and sentence with the largest number of syllables whose pronunciation coincides with the pronunciation text in the sentence,
  The real voice prosody section determination process includes:
  When comparing the pronunciation text and the selected sentence, a program is used to determine a location where the prosody of the real voice data is used based on the longest match in syllable units. A program that converts received text into synthesized speech,
Input text analysis processing that accepts text to be converted to speech and converts it to pronunciation text;
The recorded voice storage unit stores the real voice data and the sentence used for voice synthesis by comparing the sentence and the pronunciation text of the recorded voice storage unit storing the sentence in advance with the real voice data recording the preset sentence by the real voice Recording voice selection process to select from,
A connection boundary calculation process for determining a boundary between a synthesized speech section for generating speech by speech synthesis from the pronunciation text and the selected sentence and a real voice section for generating speech from the real voice data;
A real voice prosody section determination process for determining a real voice prosody section that uses the prosody of the real voice data in a synthetic voice section using the synthesized speech determined in the connection boundary calculation process;
A real voice prosody extraction process for extracting the prosody of the section determined by the real voice prosody section determination process from the selected real voice data;
The prosody by the extracted real voice, and a hybrid prosody generation process for generating prosody information of the entire synthesized speech section from the prosody model;
Based on the determined synthesized speech section, a rule synthesis process for generating speech synthesis data from a predetermined speech segment and the prosodic information;
A connection synthesis process for generating a synthesized speech sentence corresponding to the input text by connecting the real voice segment, the real voice prosody segment, and the synthesized speech segment for which prosody information is generated by the hybrid prosody generation process ; Let the calculator work ,
The recorded voice selection process is:
Compare the sentence and the pronunciation text in the recording voice storage unit, and output the voice data and sentence with the largest number of syllables whose pronunciation coincides with the pronunciation text in the sentence,
The real voice prosody section determination process includes:
When comparing the pronunciation text and the selected sentence, a program is used to determine a location where the prosody of the real voice data is used based on the longest match in phoneme units.

Images