JP4829605B2 - Speech synthesis apparatus and speech synthesis program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice synthesizer and a voice synthesizing program capable of suppressing voice quality degradation in synthesized voice without connecting phonemes in which a voice wave form does not continue well. <P>SOLUTION: The voice synthesizer 1 for synthesizing voice of an input text data, comprises: a rhythm data base 7 for accumulating rhythm data; a phoneme data base 11 for accumulating phoneme data; a language analysis section 3 for analyzing language of the text data; a rhythm generating section 5 for generating rhythm information; a phoneme searching section 9 for searching the phoneme data; and a phoneme synthesizing section 13 for connecting the phoneme data. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

  The present invention relates to a speech synthesizer and a speech synthesis program for performing speech synthesis from text data.

  Conventionally, as a method of synthesizing input text data, the text data is subjected to language analysis, and the phonetic inflection, amplitude, and duration value are determined as absolute values (so-called target costs) from the results of the language analysis. A method of handling and searching speech units stored in a database and connecting the searched speech units is disclosed (for example, Patent Documents 1 and 2).

  In the method disclosed here, a relatively small database (small amount of data) is used, and the inflection and amplitude of each phoneme are calculated based on the small number of accumulated speech segments. If the duration value is not treated as an absolute value, it is not possible to search for speech units that can be connected smoothly, and the synthesized speech that combines the speech units may sound unnatural. Met.

  By the way, in recent years, the database used for speech synthesis has become enormous (stored data has increased dramatically), and many natural combinations of various speech segments have been stored and input. It is not rare that speech segments corresponding to some or all of the text data are stored in the database as they are.

As a result, as another method for synthesizing the input text data, prosodic information is generated from the text data, and based on this prosodic information, the speech of the speech unit is generated from the speech units stored in the database. A method is disclosed in which a waveform (voice waveform data) is searched for smoothly connected, and the searched speech segments are connected (for example, Patent Document 3). In the method disclosed herein, a relatively large database (a large amount of data) is used.
Japanese Patent Laid-Open No. 2001-100777 JP 11-338488 A JP 2004-139033 A

  However, in the conventional method of generating prosodic information and performing speech synthesis, a speech unit that matches the prosodic information is preferentially selected from speech units stored in the database. Since speech units derived from information are handled in a fixed manner, a search is made for a speech waveform having inferior continuity (smoothness) when the speech unit is connected. There is a problem that the sound quality of the sound quality deteriorates.

  Therefore, the present invention provides a speech synthesizer and a speech synthesizer program that can solve the above-described problems and can suppress deterioration in the quality of synthesized speech without connecting speech segments with inferior speech waveform continuity. For the purpose.

  In order to solve the above-mentioned problem, the speech synthesizer according to claim 1 is a speech synthesizer for synthesizing input text data, and includes prosodic data storage means, speech segment data storage means, and language analysis. Means, prosodic information generation means, speech segment data search means, and speech segment data synthesis means.

According to this configuration, the speech synthesizer includes a prosody data storage unit that stores prosody data that associates phonemes and information about prosody, and a speech element that stores speech unit data that associates phonemes and speech waveforms. One piece data storage means is provided in advance. Prosodic information includes parts of speech, accent phrases, presence of accent kernels, word boundaries, phrase boundaries, clause boundaries, exhalation paragraph boundaries, dependency, focus (emphasis, eg, distinguish important words in a sentence from other words) Therefore, it is information on whether there is a large, slow and intentional gap). The speech synthesizer performs language analysis on the text data by means of language analysis means, and decomposes the text data into phoneme strings. The speech synthesizer includes an allowable range that is a change direction and a change range of a fundamental frequency of a phoneme located before and after an arbitrary phoneme in a phoneme sequence decomposed by the language analysis unit by the prosodic information generation unit , Prosody information consisting of transition probabilities that are the frequency of transition from phonemes to any phoneme included in the allowable range is generated using prosodic data stored in the prosody data storage means.

  Then, the speech synthesizer searches the speech unit data stored in the speech unit data storage unit based on the prosody information generated by the prosody information generation unit by the speech unit data search unit, and determines the connection cost. And a combination of the speech segment data that minimizes the prosodic cost. After that, the speech synthesizer synthesizes and outputs the combination of the speech unit data output by the speech unit data search unit by the speech unit data synthesis unit.

  The speech synthesizer according to claim 2 is the speech synthesizer according to claim 1, wherein the language analysis unit is responsible for phoneme or part-of-speech types and syntactic analysis when the text data is language-analyzed. It is characterized by performing language analysis on at least one of distance, presence / absence of punctuation marks, presence / absence of accent core, clause in exhalation paragraph, word position, and number of mora of the word.

  According to such a configuration, the speech synthesizer uses the language analysis means to determine the type of phoneme or part of speech, the distance of dependency of syntactic analysis, the presence or absence of punctuation marks, the presence or absence of an accent nucleus, the phrase within the expiratory paragraph, the position of the word, By performing linguistic analysis on at least one mora number of words, the reliability of the prosodic information generated by the prosodic information generating means is improved.

  The speech synthesizer according to claim 3 is the speech synthesizer according to claim 1 or 2, wherein the prosody information generation means generates information about the prosody within a preset range when the prosody information is generated. This is performed using the prosodic data having

  According to such a configuration, the speech synthesizer narrows down information related to the prosody within the preset range for the prosody data used by the prosody information generation unit, so that the speech unit data searched by the speech unit data search unit can be searched. For single data, irregular voice segment data connections can be removed. In addition, the amount of speech unit data searched by the speech unit data search means is reduced, and the synthesis speed can be improved.

  The speech synthesis device according to claim 4 is the speech synthesis device according to any one of claims 1 to 3, wherein the transition probability distribution is generated when the prosodic information generation unit generates the prosodic information. The ratio of the degree of influence between the prosodic cost and the connection cost is calculated according to the above.

  According to this configuration, when the speech synthesizer generates prosody information, for example, when there is no peak in the transition probability, the ratio of the degree of influence between the prosody cost and the connection cost according to the distribution of the transition probability. By calculating, it is possible to ensure an appropriate connection between phonemes.

  The speech synthesis program according to claim 5, the speech prosthesis data storage means for storing prosody data in which phonemes and information related to prosody are associated with each other in order to synthesize input text data, and the phonemes and speech waveforms A computer comprising speech unit data storage means for storing speech unit data associated with the speech unit data function as language analysis means, prosodic information generation means, speech segment data search means, speech unit data synthesis means The configuration.

According to this configuration, the speech synthesis program performs language analysis on the text data by the language analysis unit, decomposes the text data into phoneme sequences, and in the phoneme sequence decomposed by the language analysis unit by the prosody information generation unit, It consists of the direction of change in the fundamental frequency of a phoneme located before and after an arbitrary phoneme , an allowable range that is a changing range, and a transition probability that is the frequency of transition from a phoneme to any phoneme included in the allowable range. Prosodic information is generated using prosodic data stored in the prosodic data storage means. Then, the speech synthesis program searches the speech unit data stored in the speech unit data storage unit based on the prosodic information generated by the prosody information generation unit by the speech unit data search unit, and determines the connection cost. The speech unit data combination that minimizes the prosody cost is output, and the speech unit data synthesis unit synthesizes and outputs the speech unit data combination output by the speech unit data search unit.

  According to the first and fifth aspects of the present invention, in a phoneme string obtained by decomposing text data, a prosody comprising an allowable range of fundamental frequencies, amplitudes and durations of phonemes located before and after an arbitrary phoneme and a transition probability. Since the information is generated and the speech segment data is searched based on the prosodic information, it is possible to suppress the deterioration of the sound quality of the synthesized speech without connecting speech segments having inferior speech waveform continuity.

  According to the invention described in claim 2, by analyzing the text data based on various elements, the reliability of the generated prosodic information can be improved, and the quality of the synthesized speech can be kept high. .

  According to the third aspect of the present invention, the information about prosodic data is narrowed down within a preset range with respect to prosodic data, whereby irregular speech element data connections are made for the searched speech element data. Can be excluded.

  According to the invention described in claim 4, by calculating the ratio of the degree of influence between the prosody cost and the connection cost according to the distribution of transition probabilities, an appropriate connection between phonemes can be ensured, and the synthesis The sound quality of the voice can be kept high.

Next, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.
<Configuration of speech synthesizer>
FIG. 1 is a block diagram of a speech synthesizer. As shown in FIG. 1, a speech synthesizer 1 synthesizes speech from input text data, and includes a language analysis unit (language analysis unit) 3, a prosody generation unit (prosody information generation unit) 5, a prosody database. (Prosodic data storage means) 7, speech segment search unit (speech segment data search means) 9, speech segment database (speech segment data storage means) 11, and speech unit synthesizer (speech segment data) Synthesizing means) 13.

  This speech synthesizer 1 performs waveform-connected speech synthesis using a large-scale database (a variety of data types and a large capacity) of a prosody database 7 and a speech segment database 11. When performing this waveform connection type speech synthesis, it is necessary to smoothly connect speech unit data in order to improve the sound quality of the synthesized speech. However, when connecting speech unit data, it is preferable that prosody correction by signal processing is not performed on the speech waveform of the speech unit data as much as possible. In other words, when considering the continuity of the speech waveform of speech unit data, the range of combinations of the speech unit data is expanded (selected from various combinations of speech unit data), and the originally stored speech By using the fragment data as it is, it naturally sounds natural. That is, the sound quality of the synthesized speech is improved when the connected speech element data is not subjected to prosody correction by signal processing.

  Therefore, this speech synthesizer 1 does not use the absolute value of the phoneme feature value as in the conventional case, but prosody information (details will be described later) generated by the prosody generation unit 5, that is, the permissible phonemes to be connected. Based on the range and the transition probability, a combination of speech element data that can be heard more naturally is searched and connected.

  The language analysis unit 3 performs language analysis on the input text data, divides the text data into phoneme strings (search units), and outputs them to the prosody generation unit 5. In the language analysis by the language analysis unit 3, in addition to the phoneme string obtained by dividing the text data, the word boundary, the word part of speech, the accent phrase boundary, the accent nucleus position, the phrase boundary, and the phrase boundary The relationship between the boundary and the dependency of each clause, whether the sentence ends in question, whether there is a range of quotes for the words included in the text data, and the words included in the text data The focus (important word) is sought for whether there is a conflict or parallel relationship.

  In the phoneme string, at least three phonemes are continuous. However, the first phoneme of the text data has no front phoneme, and the last phoneme of the text data has no rear phoneme. However, even if there is no phoneme, that is, silence, “sil” (silent Is assumed to have three phonemes at the beginning and end of the sentence.

  In addition, the language analysis unit 3 may determine the importance of the word and the characteristics of the sentence such as the assertion sentence / question sentence as necessary. For example, in an interactive language (spoken language) exchanged in daily conversations, there are question sentences (for example, “What?”, “Where?”) With only nouns, and these are input as text data. Since the words do not have normal accents and intonations, it is necessary to obtain sentence features by the language analysis unit 3 and separate them (handle them as different words).

The prosody generation unit 5 includes an allowable range of fundamental frequencies (F ₀ ), amplitudes and durations of phonemes located before and after an arbitrary phoneme in the phoneme sequence divided by the language analysis unit 3, and transition probabilities thereof. Prosody information consisting of

The allowable range is the direction of change in the fundamental frequency (F ₀ ), amplitude, and duration (when increasing or decreasing over time) when changing from the current phoneme to the next phoneme, and these The range of change (the possible range). The changing range is, for example, the width of the fundamental frequency (for example, 30 Hz to 40 Hz) if the phoneme is high. The transition probability is a frequency obtained by changing a certain phoneme to any phoneme included in the allowable range as a probability for each phoneme.

  The prosody database 7 stores prosody data in which phonemes and information about prosody are associated with each other, and is configured by a general recording medium. Information on prosody includes part of speech, accent phrases, presence / absence of accent kernel, word boundary, phrase boundary, phrase boundary, expiratory paragraph boundary, dependency, information on presence / absence of focus, and the like.

In this prosody database 7, acoustic analysis results (fundamental frequency (F ₀ ), spectrum, amplitude, phoneme duration length, etc.) and linguistic analysis of voices produced by a plurality of sentences prepared in advance are analyzed. Analysis results (parts of speech, word boundaries, syntax structure, presence of focus, etc.) are stored as prosodic data.

  Here, the prosody generation unit 5 and the prosody database 7 will be described with reference to FIG. FIG. 2 is a schematic diagram schematically showing the prosody data stored in the prosody database 7 and schematically showing the prosody information generated by the prosody generation unit 5. As shown in FIG. 2, the prosody generation unit 5 clusters the prosody data stored in the prosody database 7 based on the linguistic analysis result obtained by performing linguistic analysis on the input text data. Prosody information consisting of That is, when prosodic information is generated, information related to prosody is narrowed down to a preset range.

Here, as an example of clustering, as shown in FIG. 2A, a case where attention is paid to the direction of change of the fundamental frequency (F ₀ ) is shown. In this clustering, the fundamental frequency (F ₀ ), amplitude, and duration are calculated in mora units (units consisting of consonant phonemes and vowel phonemes), and classified so that their linguistic features are similar. It is. As a result of clustering (hereinafter referred to as a cluster), phonemes in words, A. If there is no accent nucleus in the word and is in the middle of an accent phrase, B. If there is an accent nucleus in the word, C. The permissible range and transition probability differ depending on the case of the exhalation paragraph boundary.

In FIG. 2 (a), circles schematically indicate phonemes. When the direction of an arrow from the circle is upward, the fundamental frequency (F ₀ ) of the connected phonemes increases. When the direction of the arrow is downward, the fundamental frequency (F ₀ ) of the phoneme to be connected is decreased. The range indicated by this arrow is the allowable range of the fundamental frequency (F ₀ ) of the phoneme to be connected.

  These A. B. FIG. 2B shows the direction of change (increase or decrease) and its transition probability in C. Then, as shown in FIG. B. The prosody generation unit 5 obtains the median value of C and a possible range.

Returning to FIG. 1, the description of the configuration of the speech synthesizer 1 will be continued.
The speech segment search unit 9 searches the speech segment data stored in the speech segment database 11 based on the prosodic information generated by the prosody generation unit 5, and the speech with the minimum connection cost and prosodic cost is obtained. A combination of segment data is output.

The connection cost is obtained by calculating a connection point between speech unit data, a fundamental frequency (F ₀ ) near the connection point, and a spectrum error. In general, the connection cost is a numerical value of the connection between search units (speech segment data in this case) for speech synthesis. If the numerical value is low, the connection is good and the numerical value is high. If this is the case, the connection will be bad.

The prosodic cost is the transition probability (probability to increase) in each phoneme pair, that is, the direction of change (increase or decrease) of the fundamental frequency (F ₀ ) in each speech element data pair (each continuous speech element data) Or the probability of decrease) and the transition probability (probability of becoming longer or shorter) in the direction of change in the duration (longer or shorter). In general, prosody cost is a quantification of the prosody continuity in the search unit when synthesizing speech. The lower the value, the better the connection, and the higher the value, the poor connection. It will be.

  For example, even if the sound connection is good (even if the connection cost is low), if a phoneme pair (speech segment data pair) that is out of the possible range of change is included, the prosody cost will increase. Become. Then, the speech segment search unit 9 excludes such phoneme pairs (speech segment data pairs) from the combination of speech segment data.

  Here, “phoneme pairs that are out of the possible range of change (speech segment data pair)” means that the basic frequency of the rear phoneme is higher than the basic frequency of the front phoneme, but the rear A phoneme pair whose phoneme fundamental frequency is decreasing (speech segment data pair), but the fundamental frequency of the back phoneme is higher than that of the front phoneme A phoneme pair (speech segment data pair), a phoneme pair (speech that the duration of the back phoneme is shorter than that of the front phoneme, but the duration of the back phoneme is short. A pair of phoneme data), a phoneme pair in which the duration of the back phoneme is longer than the duration of the back phoneme, but the duration of the back phoneme is longer than the duration of the front phoneme Pointing.

  Note that, in order to improve the accuracy of the synthesized speech, the speech unit search unit 9 is based on the prosody data output from the prosody generation unit 5 and the speech unit data stored in the speech unit database 11. Again, the direction of prosodic change (allowable range and transition probability) may be calculated.

  Further, in the speech segment search unit 9, when the allowable range is wide, that is, when the allowable range is wider than the preset setting range (excess allowable range), or there is a clear peak in the transition probability distribution. If not, it is not necessary to place importance on the direction of prosodic change, and the degree of influence that reduces the influence of the prosodic cost is calculated. That is, the speech segment search unit 9 obtains the ratio of the degree of influence between the prosody cost and the connection cost. The higher the degree of influence, the more the influence of the prosodic cost is diluted. As a result, the combination of speech segment data that minimizes the connection cost is preferentially adopted.

  The speech segment database 11 stores speech segment data in which phonemes and speech waveforms (speech waveform data) are associated with each other, and is configured by a general recording medium.

  The speech unit synthesizing unit 13 connects the speech unit data to each other according to the combination of the speech unit data searched by the speech search unit 9, and outputs the synthesized speech.

  Here, with reference to FIG. 3, the difference between the conventional method and the method according to the present invention will be described regarding the connection of the speech unit data existing in the speech unit database 11. FIG. FIG. 3 is a diagram schematically showing speech unit data existing in the speech unit data 11 and showing how to connect speech unit data based on the predicted prosody (conventional, the present invention). In FIG. 3A, the fundamental frequency (F0) in the speech element data of “a” (a), “i” (b), “u” (c) is shown. (A), “i” (I), and “u” (C) correspond to wavy lines (three are respectively shown) indicate the shape of the speech waveform.

  Further, in FIG. 3B, these speech segment data “a” (a), “i” (b), “u” (c) are represented by black circles (where conventional predictions are shown). x, y, z). Furthermore, in the place where the predicted prosody (the present invention) is shown, it is expressed as angles α and β that can take the allowable range of the next continuous speech segment data.

  As shown in FIG. 3A, when speech unit data “a”, “i”, “u” is to be connected, as shown in FIG. The speech unit data close to the absolute value (black circle in FIG. 3B) is selected even if the speech unit data is poorly connected. Therefore, in the predicted prosody according to the present invention, the speech unit data in which the direction of change in height matches and the connection is good (the connection cost and the prosodic cost are minimized) are matched according to the allowable range of the speech unit data to be connected next. Selected.

  Returning to FIG. According to the speech synthesizer 1, in the phoneme string obtained by decomposing the text data by the language analysis unit 3, the prosody generation unit 5 allows the fundamental frequency, amplitude, and duration of the phonemes located before and after an arbitrary phoneme. And the transition probabilities are generated, and the speech unit search unit 9 searches for speech unit data based on the prosodic information. Therefore, without connecting speech units having inferior speech waveform continuity. Therefore, it is possible to suppress deterioration of the sound quality of the synthesized speech.

  Further, according to the speech synthesizer 1, the language analysis unit 5 performs linguistic analysis on the input text data based on various elements, thereby improving the reliability of the generated prosodic information, and the synthesized speech Sound quality can be kept high.

  Furthermore, according to the speech synthesizer 1, the prosody data is narrowed down within a preset range for the prosody data, thereby reducing the amount of speech segment data to be searched and generating synthesized speech. The synthesis speed can be improved.

  Furthermore, according to the speech synthesizer 1, when the speech unit search unit 9 is wider than the set allowable range (excess allowable range), or when there is no peak in the transition probability, the excessive allowable range or the transition By calculating the degree of influence of the prosodic cost on the permissible range according to the probability distribution, it is possible to secure an appropriate connection between phonemes and to keep the quality of the synthesized speech in good quality.

<Operation of speech synthesizer>
Next, the operation of the speech synthesizer 1 will be described with reference to the flowchart shown in FIG. 4 (see FIG. 1 as appropriate).
First, the speech synthesizer 1 analyzes the input text data by the language analysis unit 3 and outputs the analyzed language analysis result to the prosody generation unit 5 (step S1).

  Subsequently, the speech synthesizer 1 uses the permissible range and the transition probability based on the language analysis result output from the language analysis unit 3 by the prosody generation unit 5 and the prosodic data stored in the prosody database 7. Is generated and output to the speech segment search unit 9 (step S2). Then, the speech synthesis device 1 uses the speech segment search unit 9 based on the prosodic information output from the prosody generation unit 5, among the speech unit data stored in the speech segment database 11, the connection cost. Then, a combination of speech unit data that minimizes the prosody cost is searched for and output to the speech unit synthesis unit 13 (step S3).

  Then, the speech synthesizer 1 synthesizes (connects) the combination of the speech unit data searched by the speech unit search unit 9 by the speech unit synthesizer 13 and outputs the synthesized speech (step S4).

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment. For example, although the present embodiment has been described as the speech synthesizer 1, it can also be configured as a speech synthesis program described using a general-purpose or special computer language so that the processing of each component of the device 1 can be realized. It is. In this case, the same effect as the speech synthesizer 1 can be obtained.

1 is a block diagram of a speech synthesizer according to an embodiment of the present invention. It is the figure which showed the prosody database and the prosody generation part typically. It is the figure which showed the speech segment database typically. 3 is a flowchart for explaining the operation of the speech synthesizer shown in FIG. 1.

Explanation of symbols

1 speech synthesizer 3 language analysis unit (language analysis means)
5 Prosody generation part (prosody information generation means)
7 Prosody database (Prosodic data storage means)
9 Speech segment search unit (speech segment data search means)
11 Speech segment database (speech segment data storage means)
13 Speech unit synthesis unit (speech unit data synthesis means)

Claims (5)

A speech synthesizer for synthesizing input text data,
Prosodic data storage means for storing prosodic data in which phonemes and prosodic information are associated;
Speech unit data storage means for storing speech unit data in which the phonemes and speech waveforms are associated;
Linguistic analysis of the text data, language analysis means for decomposing the text data into phoneme strings,
In the phoneme string decomposed by the language analysis means, the change direction of the fundamental frequency of a phoneme located before and after an arbitrary phoneme and the allowable range that is the changed range, and the allowable range from the phoneme are included in the allowable range. Prosody information generating means for generating prosody information consisting of transition probabilities that are the frequency of transition to any phoneme, using prosodic data stored in the prosodic data storage means,
Based on the prosodic information generated by the prosodic information generating means, the speech segment data stored in the speech segment data storing means is searched, and the speech segment data of the speech segment data that minimizes the connection cost and the prosodic cost is searched. Speech segment data search means for outputting a combination;
Speech unit data synthesis means for synthesizing and outputting a combination of speech unit data output by the speech unit data search means;
A speech synthesizer comprising:   The language analysis means, when language analysis of the text data, the type of phoneme or part of speech, the distance of dependency of syntax analysis, the presence or absence of punctuation marks, the presence or absence of an accent nucleus, the clause in the expiratory paragraph, the position of the word, The speech synthesizer according to claim 1, wherein language analysis is performed on at least one of the number of mora of the word.   3. The prosodic information generation means, when generating the prosodic information, uses the prosodic data having information related to the prosody within a preset range. Voice synthesizer.   2. The prosody information generation means, when generating the prosody information, calculates a ratio of the degree of influence between the prosody cost and the connection cost according to the distribution of the transition probabilities. The speech synthesizer according to claim 3. In order to synthesize input text data, prosodic data storage means for storing prosody data in which phonemes and information related to prosody are associated, and speech unit data in which the phonemes are associated with speech waveforms are stored. A computer comprising speech segment data storage means;
Language analysis means for analyzing the text data and decomposing the text data into phoneme strings;
In the phoneme string decomposed by the language analysis means, the change direction of the fundamental frequency of a phoneme located before and after an arbitrary phoneme and the allowable range that is the changed range, and the allowable range from the phoneme are included in the allowable range. Prosody information generating means for generating prosody information consisting of transition probabilities that are the frequency of transition to any phoneme using the prosodic data stored in the prosodic data storage means,
Based on the prosodic information generated by the prosodic information generating means, the speech segment data stored in the speech segment data storing means is searched, and the speech segment data of the speech segment data that minimizes the connection cost and the prosodic cost is searched. Speech segment data search means for outputting combinations;
Speech unit data synthesizing means for synthesizing and outputting a combination of speech unit data output by the speech unit data search means;
A speech synthesis program characterized by functioning as

Images