JP4034751B2 - Speech synthesis apparatus, speech synthesis method, and speech synthesis program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech synthesizer efficiently synthesizing a natural speech of high quality. <P>SOLUTION: The speech synthesizer is provided with: an acquiring means 110 of acquiring a meter series for a target speech to be synthesized for a plurality of segments respectively; merged speech element holding means 160 and 170 of holding merged speech elements obtained by merging a plurality of speech elements and merged speech element meter information showing meters of the merged speech elements while making them correspond to each other; a held speech distortion estimating means 130 of estimating the degree of distortion between segment meter information showing meters of segments obtained by the acquiring means 110 and the merged speech element meter information held in the merged speech element holding means 160 and 170; a merged speech element selecting means 140 of selecting a merged speech element on the basis of the degree of distortion estimated by the held speech distortion estimating means 130; and a speech synthesizing means 150 of generating a synthesized speech by connecting respective merged speech elements that the merged speech element selecting means 140 select for the respective segments. <P>COPYRIGHT: (C)2006,JPO&amp;NCIPI

Description

  The present invention relates to a speech synthesizer, a speech synthesis method, and a speech synthesis program that synthesize speech based on a prosodic sequence of speech.

  A text-to-speech synthesis technique that artificially creates a speech signal from an arbitrary sentence is known. Text-to-speech synthesis is generally performed in three stages: a language processing stage, a prosody processing stage, and a speech synthesis stage.

  In text-to-speech synthesis, morphological analysis, syntax analysis, and the like are first performed on text input at the language processing stage. Next, in the prosody processing stage, accent and intonation processing is performed. Then, phoneme series / prosodic information (basic frequency, phoneme duration, power, etc.) is output. Finally, in the speech signal synthesis stage, a speech signal is synthesized from the phoneme sequence / prosodic information.

  Thus, in text-to-speech synthesis, a speech signal is synthesized from an arbitrary prosodic symbol string. Therefore, the speech synthesis method used for the text-to-speech synthesis needs to be a method that can synthesize an arbitrary prosodic symbol string with an arbitrary prosody.

  Conventionally, as such a speech synthesis method, speech synthesis units are stored as small unit feature parameters (this is referred to as a representative speech segment) such as CV, CVC, and VCV (V represents a vowel and C represents a consonant). Is known to synthesize a voice by controlling the fundamental frequency and duration and connecting them after selectively reading (see, for example, Patent Document 1).

  Further, as a technique based on statistical learning, a voice synthesis technique based on HMM is disclosed (for example, see Non-Patent Document 1). In the speech synthesis method based on the HMM, the spectral envelope parameter and the fundamental frequency parameter are modeled simultaneously based on the hidden Markov model, and the spectral envelope parameter and the dynamic feature statistic are taken into consideration at the time of synthesis. Generate fundamental frequency parameters. The distribution corresponding to the unknown context is selected by following the decision tree held in each state of the HMM. This decision tree has a question in each node, follows the decision tree depending on whether the input attribute information corresponds to the question of each node, and selects the distribution in the leaf nodes.

Japanese Patent No. 2583074 Takakatsu Yoshimura, Keiichi Tokuda, Takashi Masuko, Takao Kobayashi, Tadashi Kitamura, “Simultaneous Modeling of Spectrum, Pitch and Duration in HMM-Based Speech Synthesis”, IEICE Transactions, November 2000, Vol. J83-D-II, No. 11, pp. 2099-21-8,

  In a speech synthesis method using a representative speech element, a representative speech element created in advance is used. However, in this method, usable speech segments are limited to representative speech segments created in advance. Therefore, there is a problem that it is difficult to cope with various variations of input prosodic and phonological environments.

  By increasing the number of representative speech segments to be created in advance, it is possible to cope with various variations of the input prosodic environment, but on the other hand, the number of representative speech segments to be created has been increased. In this case, the processing efficiency is lowered. Moreover, there is a limit to the computational resources allocated to speech synthesis, and the number of representative speech segments created in advance is also limited.

  In addition, in the unit selection type speech synthesis method, there is a problem that it is difficult to formulate a rule for selecting a speech unit sequence that can be heard naturally as a cost function. Furthermore, there is a problem that it is difficult to eliminate defective pieces.

  The present invention has been made in view of the above, and an object of the present invention is to provide a speech synthesizer that can efficiently synthesize natural and high-quality speech.

In order to solve the above-described problems and achieve the object, the present invention provides a plurality of speech units for the same speech unit, and a plurality of speech units having different prosody of the speech unit, and the speech Speech unit holding means for holding speech unit prosody information indicating the prosody of the unit in association with each other, teacher speech prosody information indicating the prosody of teacher speech set in advance, and the speech unit holding unit Speech unit selection means for selecting a plurality of speech units from the speech unit holding means based on the speech unit prosody information, and the plurality of speech units selected by the speech unit selection means. Based on the plurality of speech units included in the determined combination, a plurality of speech units based on the combination determination means for determining a combination of the plurality of speech units satisfying a predetermined condition from the segment Fusing Fusion speech unit creating means for creating a fused speech unit, and fusion indicating the prosody of the fused speech unit based on the prosodic information corresponding to each of the plurality of speech units included in the determined combination Fusion speech unit prosody information creating means for creating speech unit prosodic information, the fused speech unit created by the fused speech unit creating means, and the fused speech unit prosody information creating means created by the fused speech unit prosody information creating means Fusion speech unit holding means for holding the fusion speech unit prosodic information in association with each other, and acquisition means for acquiring a prosodic sequence for the target speech to be synthesized for each of a plurality of segments that are synthesis units of speech synthesis Segment prosody information indicating the prosody of the segment obtained by the acquisition unit and the fused speech unit prosody held in the fused speech unit holding unit Holding speech distortion estimation means for estimating the degree of distortion between the information and the fusion speech unit selection means for selecting the fusion speech unit based on the degree of distortion estimated by the holding speech distortion estimation means And speech synthesis means for generating synthesized speech by connecting the fused speech units selected by the fused speech unit selection means for each segment.

Also, the present invention provides a plurality of speech units for the same speech unit, and a plurality of speech units having different prosody of the speech unit and speech unit prosody information indicating the prosody of the speech unit. Based on the speech segment prosody information held in the speech unit holding means held in association with the speech unit prosody information indicating the prosody of the teacher speech set in advance, from the speech unit holding means A speech unit selection step for selecting a plurality of speech units, and a combination of the plurality of speech units satisfying a predetermined condition is determined from the plurality of speech units selected by the speech unit selection step. A combined speech unit creating step for creating a fused speech unit by fusing a plurality of speech units based on the plurality of speech units included in the determined combination. And fused speech unit prosody for creating fused speech unit prosody information indicating the prosody of the fused speech unit based on the prosodic information corresponding to each of the plurality of speech units included in the determined combination A fused speech in which an information creating step, the fused speech segment created by the fused speech segment creating step, and the fused speech segment prosodic information created by the fused speech segment prosodic information creating step are associated with each other; a storage step of storing the segment holding means, a prosody series with respect to the target speech to be speech synthesis, an acquisition step of acquiring the respective plurality of segments is a composite unit of speech synthesis, the fused speech unit holding means The fused speech segment prosody information held; segment prosody information indicating the prosody of the segment obtained in the acquisition step; Holding audio distortion estimating step of estimating the degree of distortion between, based on the degree of the distortion estimated in the holding audio distortion estimation step, a fused speech unit selection step of selecting the fused speech unit,
A speech synthesis step of generating synthesized speech by connecting the fused speech units selected for each segment in the fused speech unit selection step.

The present invention also provides a speech synthesis program for causing a computer to perform speech synthesis processing, a plurality of speech units for the same speech unit, and a plurality of speech units having different prosody of the speech unit. The speech unit prosody information held in the speech unit holding means that holds the speech unit prosody information indicating the prosody of the speech unit in association with the teacher and the teacher showing the prosody of the preset teacher speech Based on speech prosody information, a speech unit selection step for selecting a plurality of speech units from the speech unit holding means, and a plurality of speech units selected by the speech unit selection step are determined in advance. A combination determining step for determining a combination of a plurality of speech units that satisfy a given condition, and a plurality of speech units included in the determined combination, Based on the prosody information corresponding to each of the plurality of speech units included in the determined combination, and a fusion speech unit creation step of creating a fused speech unit by fusing the recorded speech units Fusion speech segment prosodic information creation step for creating fused speech segment prosodic information indicating the prosody of the segment, the fused speech segment created by the fused speech segment creation step, and the fused speech segment prosodic information A step of associating the fusion speech unit prosody information created in the creation step with the fusion speech unit holding means in association with each other, and a plurality of prosody sequences for the target speech to be synthesized as speech synthesis units an acquisition step of acquiring a segment for each, and the fused speech unit prosody information held in the fused speech unit holding means, said acquisition stearate Based on the degree of distortion estimated in the retained speech distortion estimation step, the retained speech distortion estimation step for estimating the degree of distortion between the segment prosody information indicating the prosody of the segment obtained in the step, A fusion speech unit selection step for selecting a fusion speech unit; and a speech synthesis step for generating a synthesized speech by connecting the fusion speech units selected for each segment in the fusion speech unit selection step. It is characterized by that.

  In the speech synthesizer according to the present invention, the fused speech unit holding unit holds the fused speech unit and the fused speech unit prosodic information of the fused speech unit in association with each other, and estimates by the held speech distortion estimation unit Since speech synthesis is performed using the fusion speech unit selected by the fusion speech unit selection means based on the degree of distortion, the processing efficiency is higher than when creating a fusion speech unit during speech synthesis. It is possible to achieve the effect of synthesizing a natural and high-quality voice.

  Hereinafter, embodiments of a speech synthesizer, a speech synthesis method, and a speech synthesis program according to the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

(Embodiment 1)
FIG. 1 is a block diagram showing the overall configuration of the text-to-speech synthesizer according to the first embodiment of the present invention. The text-to-speech synthesizer 10 includes a text acquisition unit 11, a language processing unit 12, a prosody processing unit 13, a speech synthesis unit 14, and a speech waveform output unit 15.

  The text acquisition unit 11 acquires text data that is a target of speech synthesis from the outside. The language processing unit 12 performs morphological analysis / syntax analysis of the text data acquired by the text acquisition unit 11. Then, the result is sent to the prosody processing unit 13.

  The prosodic processing unit 13 specifies the accent or intonation of the text data based on the language analysis result. That is, the characteristic about the prosody is specified. The prosody processing unit 13 generates a phoneme sequence (phoneme symbol string) and prosody information of a target speech that is a target of speech synthesis, based on the characteristics related to the specified prosody. Then, the prosodic sequence and prosodic information are sent to the speech synthesizer 14. Here, the prosodic information is information indicating a fundamental frequency, a phoneme duration, power, and the like.

  The speech synthesizer 14 generates a speech waveform from the phoneme sequence and prosodic information. The speech waveform generated in this way is output from the speech waveform output unit 15.

  FIG. 2 is a block diagram showing a detailed configuration of the speech synthesizer 14 of FIG. The speech synthesis unit 14 includes a phoneme sequence / prosodic information acquisition unit 110, a distortion estimation unit 130, a fusion speech unit selection unit 140, a fusion speech unit editing / connection unit 150, and a fusion speech unit creation unit 180. , A fusion speech unit storage unit 160 and a fusion speech unit phoneme environment storage unit 170 are provided.

  The phoneme sequence / prosodic information acquisition unit 110 acquires the phoneme sequence and prosody information of the target speech from the prosody processing unit 13. Hereinafter, the phoneme sequence and the prosody information acquired by the phoneme sequence / prosodic information acquisition unit 110 are referred to as an input phoneme sequence and input prosody information, respectively. The input phoneme sequence is a sequence of phoneme symbols, for example.

  On the other hand, the fusion speech unit storage unit 160 stores a plurality of fusion speech units that have already been created. Here, the fusion speech unit is a speech unit obtained by fusing a plurality of speech units for the same speech unit. Note that the speech unit in the present embodiment is a phoneme. Note that the speech unit is not limited to phonemes. The fusion speech unit storage unit 160 stores a plurality of speech units for the same phoneme, and a plurality of speech units having different phonemes.

  The fused speech unit storage unit 160 stores the fused speech units in units of speech (synthetic units) used when generating synthesized speech.

  Here, the synthesis unit is a phoneme or a combination of phonemes divided. For example, semi-phonemes, phonemes (C, V), diphones (CV, VC, VV), triphones (CVC, VCV), syllables (CV, V), etc. (V represents vowels and C represents consonants) . Or these may be mixed. In this case, the length may be variable.

  The fused speech unit phoneme environment storage unit 170 stores a fused speech unit phoneme environment for the fused speech unit stored in the fused speech unit storage unit 160.

  Here, the fused speech unit phoneme environment is information corresponding to a combination of factors that are environments for the fused speech unit. Factors include, for example, the phoneme name of the fusion speech unit, the preceding phoneme, the subsequent phoneme, the subsequent phoneme, the fundamental frequency, the phoneme duration, power, the presence or absence of stress, the position from the accent core, the time from breathing, There are speaking speed and emotion. Thus, the fusion speech unit phoneme environment is information including fusion speech unit prosody information indicating the prosody of the fusion speech unit.

  Note that the fusion speech unit stored in the fusion speech unit storage unit 160 and the fusion speech unit phoneme environment for the fusion speech unit stored in the fusion speech unit phoneme environment storage unit 170 are associated with each other. Yes. Specifically, for example, the fusion speech unit phoneme environment stored in the fusion speech unit phoneme environment storage unit 170 is stored in association with a fusion speech unit number for identifying the corresponding fusion speech unit. It may be.

  Here, the fusion speech unit storage unit 160 and the fusion speech unit phoneme environment storage unit 170 in the present embodiment constitute the fusion speech unit holding means described in the claims.

  The fusion speech unit creation unit 180 creates a fusion speech unit phoneme environment to be stored in the fusion speech unit storage unit 160 and a fusion speech unit phoneme environment storage unit 170 to store in the fusion speech unit phoneme environment storage unit 170. In the present embodiment, the fusion speech unit creation unit 180 creates a fusion speech unit and a fusion speech unit phoneme environment in advance and stores them in the fusion speech unit storage unit 160 and the fusion speech unit phoneme environment storage unit 170. is doing.

  Based on the fusion speech unit phoneme environment stored in the fusion speech unit phoneme environment storage unit 170 and the input prosodic information for the predetermined segment acquired from the distortion estimation unit 130, the distortion estimation unit 130 And the degree of distortion of the fused speech unit phoneme environment stored in the fused speech unit phoneme environment storage unit 170 is estimated.

  Here, the distortion estimation unit 130 according to the present embodiment constitutes the retained speech distortion estimation means and the created speech distortion estimation means of the present invention.

  The fusion speech unit selection unit 140 selects a fusion speech unit from the fusion speech unit storage unit 160 based on the degree of distortion estimated by the distortion estimation unit 130.

  Specifically, first, the distortion estimation unit 130 calculates the degree of distortion between the input prosodic information for a predetermined segment and each of the plurality of fusion speech unit phoneme environments stored in the fusion speech unit phoneme environment storage unit 170. presume. Then, the fused speech unit selection unit 140 specifies the minimum value of the degree of distortion obtained for each fused speech unit environment. Then, the fused speech unit corresponding to the fused speech unit environment showing the minimum value is selected from the fused speech unit storage unit 160. Thereby, it is possible to obtain a sequence of fused speech segments corresponding to a sequence of phoneme symbols of the input phoneme sequence. A method for estimating the degree of distortion will be described later.

  The fusion speech unit editing / connection unit 150 appropriately edits and connects the series of fusion speech units obtained for each segment. As a result, a speech waveform of synthesized speech is generated. The voice waveform thus generated is output to the outside via the voice waveform output unit 15.

  FIG. 3 is a block diagram showing a detailed functional configuration of the fused speech unit creation unit 180 described in FIG. The fusion speech unit creation unit 180 includes a speech unit storage unit 181, a fusion speech unit phoneme environment storage unit 182, a speech unit combination creation unit 183, a fusion speech unit creation unit 184, and a fusion speech unit A phoneme environment creation unit 185.

  The speech element storage unit 181 stores a large amount of speech elements. Further, the fusion speech unit phoneme environment storage unit 182 stores the speech unit phoneme environment for each speech unit stored in the speech unit storage unit 181. The synthesis unit of the speech unit stored in the speech unit storage unit 181 is the same as the synthesis unit of the fusion speech unit stored in the fusion speech unit storage unit 160.

  The speech unit stored in the speech unit storage unit 181 and the speech unit phoneme environment stored in the fused speech unit phoneme environment storage unit 182 are associated with each other. Specifically, for example, the speech unit phoneme environment stored in the fusion speech unit phoneme environment storage unit 182 may be stored in association with a speech unit number for identifying the corresponding speech unit. .

  The speech unit storage unit 181 and the fusion speech unit phoneme environment storage unit 182 in the present embodiment constitute speech unit holding means described in the claims.

  The speech unit combination creating unit 183 is based on the speech unit phoneme environment stored in the fusion speech unit phoneme environment storage unit 182, and the speech unit combination creation unit 183 includes a plurality of speech units stored in the speech unit storage unit 181. Then, a combination of a plurality of speech segments to be fused is determined.

  The fused speech unit creation unit 184 extracts speech units included in the combination determined by the speech unit combination creation unit 183 from the speech unit storage unit 181. Furthermore, a fused speech segment is created by fusing the extracted speech segments. The fused speech unit creation unit 184 stores the created fused speech unit in the fused speech unit storage unit 160.

  The fused speech unit phoneme environment creation unit 185 extracts the speech unit phoneme environment of the speech units included in the combination determined by the speech unit combination creation unit 183 from the fused speech unit phoneme environment storage unit 182. Further, based on the extracted speech unit phoneme environment, a fused speech unit phoneme environment is created. The fused speech unit phoneme environment creation unit 185 stores the created fused speech unit phoneme environment in the fused speech unit phoneme environment storage unit 170.

  Specifically, the fused speech unit phoneme environment creation unit 185 creates a fused speech unit phoneme environment using the centroid of the speech unit phoneme environment of each speech unit.

  As another example, a speech unit phoneme environment of each of a plurality of speech units included in the combination determined by the speech unit combination creation unit 183 may be created as a fusion speech unit phoneme environment.

  Here, the fused speech unit creating unit 184 according to the present embodiment constitutes a speech unit selecting unit and a fused speech unit creating unit described in the claims. Also, the fused speech unit phoneme environment creation unit 185 according to the present embodiment constitutes a speech unit selection unit and a fused speech unit prosody information creation unit described in the claims.

FIG. 4 is a block diagram showing a detailed functional configuration of the speech element combination creating unit 183 shown in FIG. The speech unit combination creation unit 183 includes a speech unit combination frequency information storage unit 1835, a phoneme sequence / prosodic information acquisition unit 1831, a speech unit selection unit 1832, a speech unit combination frequency information creation unit 1833, a speech A segment combination determining unit 1834.

  The phoneme sequence / prosodic information acquisition unit 1831 acquires phoneme and input prosody information for each of a plurality of segments obtained by dividing a phoneme sequence obtained by analyzing sentence data into synthesis units. The input prosodic information and the like are acquired from the prosodic processing unit 13 described with reference to FIG.

  The multiple speech unit selection unit 1832 estimates the degree of distortion between the input prosody information and the fused speech unit phoneme environment stored in the fused speech unit phoneme environment storage unit 182. Based on the degree of distortion, a plurality of speech units are selected from speech units stored in the speech unit storage unit 181. The selection method may be the same as the selection method in the fusion speech unit selection unit 140.

  The speech unit combination frequency information creation unit 1833 counts the usage frequency of the combination of the plurality of speech units selected by the multiple speech unit selection unit 1832. The counted usage frequency is stored in the speech unit combination frequency information storage unit 1835.

The speech unit combination determination unit 1834 determines a combination of a plurality of speech units based on the frequency information stored in the speech unit combination frequency information storage unit 1835. For example, the speech element combination determination unit 1834 may select a plurality of speech elements so that the frequency of use of the selected plurality of speech elements is equal to or higher than a predetermined threshold.

  As another example, a combination corresponding to a fusion speech unit that is frequently used may be selected from a plurality of combinations. For example, this is effective when the number of fused speech units to be stored in the fused speech unit storage unit 160 is limited.

  As described above, the method for selecting a combination of speech units to be selected for creating a fused speech unit is not limited to the present embodiment, and may be selected based on a predetermined condition.

  Hereinafter, each process of the speech synthesis unit 14 will be described in detail. Here, it is assumed that the speech unit of the synthesis unit is a phoneme.

  FIG. 5 schematically shows the data configuration of the fused speech unit storage unit 160. FIG. 6 schematically shows the data structure of the fused speech unit phoneme environment storage unit 170.

  As shown in FIG. 5, the fused speech unit storage unit 160 stores the speech signal of each phoneme as a pitch waveform. Furthermore, each audio signal is stored in association with a fusion speech unit number for identifying the phoneme.

  Here, the pitch waveform is a relatively short waveform that has a length up to several times the basic period of the voice and does not have a basic period, and its spectrum represents the spectrum envelope of the audio signal. Means.

  Further, as shown in FIG. 6, the fusion speech unit phoneme environment storage unit 170 sets the phoneme environment information of each fusion speech unit stored in the fusion speech unit storage unit 160 to the unit number of the phoneme. Stored in association. The fused speech segment phoneme environment storage unit 170 according to the present embodiment stores a phoneme symbol (phoneme name), a fundamental frequency, a phoneme duration, and a connection boundary cepstrum as a phoneme environment.

  In the present embodiment, the fusion speech unit is a phoneme unit, but other examples may be a semiphone, a diphone, a triphone, and a syllable. Moreover, these combinations may be sufficient.

  Next, the processing of the distortion estimation unit 130 described in FIG. 2 will be described in detail. The distortion estimation unit 130 estimates the degree of distortion based on the cost calculated by the cost function. Then, the fusion speech unit selection unit 140 selects a fusion speech unit based on the cost estimated by the distortion estimation unit 130.

  Here, the cost function is a function determined by the degree of distortion for all segments included in the text data.

  Hereinafter, the cost function will be described in detail. A sub-cost function is determined for each factor of distortion generated when a synthesized speech is generated by deforming and connecting fused speech segments. Here, the sub-cost function is used to estimate the degree of distortion of the synthesized speech with respect to the target speech that occurs when the synthesized speech is generated using the fused speech unit stored in the fused speech unit storage unit 160. This is a function for calculating the cost.

  The sub cost function is defined as Cn (ui, ui-1, ti) (n: 1,..., N, N is the number of sub cost functions). Here, ti is the speech unit of the portion corresponding to the i-th segment when the target speech (target speech) corresponding to the input phoneme sequence and the input prosodic information is t = (t1,..., TI). Ui represents a fusion speech unit having the same phoneme as ti among the fusion speech units stored in the fusion speech unit storage unit 160.

  Specifically, when calculating the cost, two types of sub-costs, a target cost and a connection cost, are used. Here, the target cost is a cost for estimating the degree of distortion of the synthesized speech with respect to the target speech generated by using the fusion speech unit. The connection cost is a cost for estimating the degree of distortion of the synthesized speech with respect to the target speech that occurs when the fusion speech unit is connected to another speech unit.

  Further, the basic frequency cost and the phoneme duration time cost are used as the target costs. Here, the fundamental frequency cost is a cost representing the difference (difference) between the fundamental frequency of the fused speech unit stored in the fused speech unit storage unit 160 and the target fundamental frequency. The phoneme duration time cost is a cost representing the difference (difference) between the phoneme duration time of the fusion speech unit and the target phoneme duration time. As the connection cost, a spectrum connection cost representing a spectrum difference (difference) at the connection boundary is used.

ここで、ｖｉは融合音声素片記憶部１６０に記憶されている音声素片ｕｉの音素環境を、ｆは音素環境ｖｉから基本周波数を取り出す関数を表す。また、音韻継続時間長コストは、次式によって定義される。

ここで、ｇは音素環境ｖｉから音韻継続時間長を取り出す関数を表す。スペクトル接続コストは、２つの音声素片間のケプストラム距離によって算出される。 Specifically, the fundamental frequency cost is defined by the following equation.

Here, vi represents the phoneme environment of the speech unit ui stored in the fusion speech unit storage unit 160, and f represents a function for extracting the fundamental frequency from the phoneme environment vi. Further, the phoneme duration time cost is defined by the following equation.

Here, g represents a function for extracting the phoneme duration from the phoneme environment vi. The spectrum connection cost is calculated from the cepstrum distance between two speech segments.

ここで、ｈは融合音声素片ｕｉの接続境界のケプストラム係数をベクトルとして取り出す関数を表す。 Note that the cepstrum distance between two speech segments is defined by the following equation.

Here, h represents a function that extracts a cepstrum coefficient of the connection boundary of the fusion speech unit ui as a vector.

ここで、ｗｎはサブコスト関数の重みを表す。本実施形態では、簡単のため、ｗｎはすべて「１」とする。上記式（４）は、ある合成単位に、ある融合音声素片を当てはめた場合の当該融合音声素片の合成単位コストである。 The weighted sum of these sub-cost functions is defined as a composite unit cost function. The composite unit cost function is defined by the following equation.

Here, wn represents the weight of the sub cost function. In this embodiment, for simplicity, wn is all “1”. The above equation (4) is the synthesis unit cost of the fusion speech unit when a fusion speech unit is applied to a synthesis unit.

融合音声素片選択部１４０は、（５）に示したコスト関数を使って１セグメントあたり（すなわち、１合成単位あたり）の融合音声素片を選択する。選択の際は、融合音声素片記憶部１６０に記憶されている融合音声素片群の中から、上記式（５）で算出されるコストの値が最小の融合音声素片の系列を求める。このコストが最小となる融合音声素片の組合せを最適素片系列と呼ぶこととする。すなわち、最適素片系列中の各融合音声素片は、入力音韻系列を合成単位で区切ることにより得られる複数のセグメントのそれぞれに対応し、最適素片系列中の各融合音声素片から算出された上記合成単位コストと式（５）より算出されたコストの値は、他のどの融合音声素片系列よりも小さい値である。 For each of a plurality of segments obtained by dividing the input phoneme sequence by synthesis unit, the result of calculating the synthesis unit cost from the above equation (4) is the sum of all segments is called the cost. A cost function for calculation is defined as shown in the following equation (5).

The fused speech element selection unit 140 selects the fused speech elements per segment (that is, per synthesized unit) using the cost function shown in (5). At the time of selection, from the fused speech unit group stored in the fused speech unit storage unit 160, a sequence of fused speech units having a minimum cost value calculated by the above equation (5) is obtained. A combination of fused speech units that minimizes the cost is called an optimal unit sequence. That is, each fusion speech unit in the optimum unit sequence corresponds to each of a plurality of segments obtained by dividing the input phoneme sequence by synthesis unit, and is calculated from each fusion speech unit in the optimum unit sequence. The value of the cost calculated from the synthesis unit cost and the equation (5) is smaller than any other fused speech element sequence.

  Note that dynamic programming (DP) may be used for searching for the optimum segment sequence. Thereby, the efficiency of the search process can be further increased.

  Next, the processing of the fusion speech unit editing / connecting unit 150 described in FIG. 2 will be described in detail with reference to FIG. The fusion speech unit editing / connection unit 150 transforms the fusion speech unit of the optimum unit sequence selected by the fusion speech unit selection unit 140 according to the input prosodic information. Then, the synthesized fused speech unit is connected to generate a speech waveform of synthesized speech.

  The fusion speech unit storage unit 160 stores the fusion speech units in the form of pitch waveforms. Therefore, the pitch waveform is superimposed so that the fundamental frequency and the phoneme duration length of the fusion speech unit become the fundamental frequency of the target speech and the phoneme duration length of the target speech indicated in the input prosodic information, respectively. Generate a speech waveform.

  Referring to FIG. 7, the fusion speech unit selected for each synthesis unit of phonemes “m”, “a”, “d”, and “o” is transformed and connected to generate a speech waveform “Mado”. The process in the case of producing | generating is demonstrated concretely.

  As shown in FIG. 7, according to the target fundamental frequency and the target phoneme duration duration indicated in the input prosodic information, for each segment (synthesis unit), each pitch waveform in the fused speech unit Change the basic frequency (change the pitch) and increase / decrease the number of pitch waveforms (expand / expand time). After that, synthesized speech is generated by connecting adjacent pitch waveforms within and between segments.

  In addition, although the distortion estimation part 130 concerning this Embodiment utilized the calculation result by a cost function as a degree of distortion, the value which evaluates the degree of distortion is not limited to this.

  Next, the process of the fusion speech unit creation unit 180 will be described. The speech unit storage unit 181 and the fused speech unit phoneme environment storage unit 182 store speech units obtained by analyzing a speech database and phoneme environment information thereof. A large amount of speech units are stored in the speech unit storage unit 181, and information on phoneme environments (phoneme environment information) of these speech units is stored in the fusion speech unit phoneme environment storage unit 182. . The speech unit storage unit 181 stores speech units of speech units (synthesis units) used when generating synthesized speech. The synthesis unit of the speech unit is the same as that of the fused speech unit, and the type of phoneme environment information is also the same as that of the fused speech unit.

  FIG. 8 schematically shows the data structure of the speech unit storage unit 181. The speech unit storage unit 181 stores a speech signal waveform of each phoneme and a unit number for identifying the phoneme in association with each other. FIG. 9 schematically shows the data structure of the fused speech unit phoneme environment storage unit 182. Similar to the fused speech unit phoneme environment storage unit 170, the fused speech unit phoneme environment storage unit 182 stores the phoneme environment information of each speech unit stored in the speech unit storage unit 181 and the unit of the phoneme. Numbers are stored in association with each other.

  Each speech unit stored in the speech unit storage unit 181 is labeled for each phoneme with respect to a large number of separately collected speech data, and a speech waveform cut out for each phoneme is used as a speech unit. Accumulated.

  For example, FIG. 10 shows the result of labeling the voice data 101 for each phoneme. FIG. 10 shows phoneme symbols for the speech data (speech waveform) of each phoneme divided by the labeling boundary 102. Note that phoneme environment information (eg, phoneme (in this case, phoneme name (phoneme symbol)), fundamental frequency, phoneme duration length, etc.) for each phoneme is also extracted from the speech data.

  The same unit number is assigned to each voice waveform obtained from the voice data 101 by the above processing and the information of the phoneme environment corresponding to the voice waveform. Then, as shown in FIG. 8 and FIG. 9, the speech unit storage unit 181 and the fusion speech unit phoneme environment storage unit 182 respectively store them. Here, the phoneme environment information includes the phoneme of the speech unit, its fundamental frequency, and the phoneme duration.

  In this example, the speech unit is extracted in units of phonemes. However, the same applies to the case where the speech unit is a semi-phoneme, a diphone, a triphone, a syllable, or a combination or variable length thereof. .

  The fused speech unit creation unit 184 acquires a plurality of speech units included in the combination created by the speech unit combination creation unit 183, which will be described later, from the speech unit storage unit 181. Then, a plurality of acquired speech units are merged to create a fused speech unit. Note that the fused speech segment creation unit 184 performs different processing depending on whether the target speech segment is a voiced sound or an unvoiced sound.

  First, the case of voiced sound will be described. In the case of voiced sound, a pitch waveform is extracted from the speech segment and fused at the level of the pitch waveform to create a new pitch waveform. The pitch waveform can be extracted by simply cutting out with the fundamental period synchronization window, by inverse discrete Fourier transform of the power spectrum envelope obtained by cepstrum analysis or PSE analysis, and by the impulse response of the filter obtained by linear prediction analysis. There are various methods such as a method for obtaining a waveform and a method for obtaining a pitch waveform that reduces distortion with respect to natural speech at the level of synthesized speech by a closed loop learning method.

  In the present embodiment, the pitch waveform is extracted using a method of cutting out with a basic period synchronization window. With reference to FIG. 11, a processing procedure in the case where one new speech unit is generated by fusing the M speech units determined by the speech unit combination creating unit 183 will be described.

  In step S111, marks (pitch marks) are given to the respective speech waveforms of the M speech units for each periodic interval. FIG. 12A shows a case where pitch marks 122 are attached to the speech waveform 121 of one speech unit among the M speech units at every periodic interval.

  In step S112, as shown in FIG. 12B, the pitch waveform is cut out with reference to the pitch mark to cut out the pitch waveform. A Hanning window 123 is used as the window, and the window length is twice the basic period. Then, as shown in FIG. 12C, the windowed waveform 124 is cut out as a pitch waveform.

  The process shown in FIGS. 12-1 to 12-3 (the process of step S112) is performed on each of the M speech units. As a result, a series of pitch waveforms consisting of a plurality of pitch waveforms is obtained for each of the M speech segments.

  Next, the process proceeds to step S113, and the pitches in the series of all M pitch waveforms are matched with the one having the largest number of pitch waveforms among the series of pitch waveforms of the M speech units of the segment. The pitch waveforms are duplicated so that the number of pitch waveforms is the same (for a series of pitch waveforms with a small number of pitch waveforms).

  FIG. 13 shows a series of pitch waveforms e1 to e3 cut out in step S112 from each of M speech segments d1 to d3 of the segment (for example, three here). The number of pitch waveforms in the pitch waveform series e1 is 7, the number of pitch waveforms in the pitch waveform series e2 is 5, and the number of pitch waveforms in the pitch waveform series e3 is 6. That is, among the pitch waveform series e1 to e3, the series having the largest number of pitch waveforms is the series e1.

  Accordingly, the number of pitch waveforms in the series e1 (for example, the number of pitch waveforms here is 7). For the other series e2, e3, each of the pitch waveforms in the series is copied to make the number of pitch waveforms seven. As a result, new pitch waveform series corresponding to the series e2 and e3 are e2 ′ and e3 ′, respectively.

  Next, the process proceeds to step S114. In this step, processing is performed for each pitch waveform. In step S114, the pitch waveforms corresponding to the M speech units of the segment are averaged for each position to generate a new pitch waveform sequence. The generated new pitch waveform sequence is used as a fused speech unit.

  FIG. 14 shows pitch waveform series e1, e2 ′, e3 ′ obtained in step S113 from each of M (for example, three in this case) speech elements d1 to d3 of the segment. Since there are seven pitch waveforms in each series, in step S114, the first to seventh pitch waveforms are averaged with three speech segments, and a new pitch consisting of seven new pitch waveforms is obtained. A waveform series f1 is generated. That is, for example, the centroid of the first pitch waveform of the series e1, the first pitch waveform of the series e2 ′, and the first pitch waveform of the series e3 ′ is obtained, and is obtained as a new pitch waveform series f1. The first pitch waveform. The same applies to the second to seventh pitch waveforms of the new pitch waveform series f1. A series f1 of pitch waveforms is the above “fusion speech unit”.

  On the other hand, in the process of the fusion speech unit creating unit 184, in the case of an unvoiced segment, in the segment selection step S111, among the M speech units of the segment, the M speech units are converted into speech units. Is selected, and the speech waveform of the selected speech segment is used as it is. That is, the speech waveform of the selected speech unit is stored in the fusion speech unit storage unit 160. For convenience, this is also called a fusion speech unit. When the combination is ranked, the speech segment is determined by selecting the first segment.

融合音声素片の継続時間長Ｔは、各音声素片の継続時間長をＴｍ（１≦m≦Ｍ）とすると、次式によって定義される。

融合音声素片の接続境界のケプストラムｃは、各音声素片の接続境界のケプストラムをｃｍ（１≦m≦Ｍ）とすると、次式によって定義される。

The fused speech element phoneme environment creation unit 185 creates a phoneme environment of the fused speech element based on the phoneme environment of the combination. The phoneme environment of the fusion speech unit is obtained as the centroid of each phoneme environment. In this case, the fundamental frequency f of the fusion speech unit is defined by the following equation, where fm (1 ≦ m ≦ M) is the fundamental frequency of each speech unit.

The duration time T of the fusion speech unit is defined by the following equation, where Tm (1 ≦ m ≦ M) is the duration time of each speech unit.

The cepstrum c at the connection boundary of the fusion speech unit is defined by the following equation, where the cepstrum at the connection boundary of each speech unit is cm (1 ≦ m ≦ M).

  Through these processes, a fusion speech unit and its phoneme environment are created. Then, it is stored in the fused speech unit storage unit 160 and the fused speech unit phoneme environment storage unit 170.

  Next, the processing of the speech element combination creation unit 183 will be described in detail. The speech unit combination creating unit 183 creates a combination of speech units to be merged in the fused speech unit creating unit 184. In the present embodiment, a plurality of speech units are selected based on the cost function described above in the process of the fused speech unit selection unit 140. Furthermore, a combination of a plurality of speech units to be merged is determined based on the usage frequency.

  The cost function described above calculates the cost based on the phoneme environment information of the fusion speech unit. Here, the cost function is calculated based on the phoneme environment information corresponding to the speech unit. First, text data for obtaining the use frequency of each speech element combination is prepared. Each text data is processed by the text acquisition unit 11, the language processing unit 12, and the prosody processing unit 13 in FIG. 1 to obtain a phoneme sequence and prosodic information. For each segment obtained by dividing the phoneme sequence by synthesis unit, per segment based on the cost between the prosodic information and the phoneme environment information included in the fused phoneme phoneme environment storage unit 182 ( That is, a plurality of speech segments are selected (per synthesis unit).

  FIG. 15 is a flowchart for explaining the processing at this time. First, in step S151, the optimal speech unit path is obtained using a cost function and dynamic programming, as in the optimal path calculation of the fused speech unit.

  Next, proceeding to step S152, a plurality of speech segments are selected per segment using the optimal segment sequence. Here, it is assumed that the number of segments is J and M speech units are selected per segment. Details of step S152 will be described.

  In steps S153 and S154, one of the J segments is set as a target segment. Steps S153 and S154 are repeated J times, and processing is performed so that J segments become the target segment once. First, in step S153, speech segments of the optimal segment series are fixed to segments other than the target segment. In this state, the speech units stored in the speech unit storage unit 181 are ranked with respect to the segment of interest according to the cost value of Expression (5), and the top M pieces are selected.

  For example, as shown in FIG. 16, it is assumed that the input phoneme sequence is “ts · i · i · s · a ·. In this case, the synthesis unit corresponds to each of phonemes “ts”, “i”, “i”, “s”, “a”,..., And each of these phonemes corresponds to one segment. FIG. 16 shows a case where a segment corresponding to the third phoneme “i” in the input phoneme sequence is set as a target segment, and a plurality of speech segments are obtained for this target segment. For segments other than the segment corresponding to the third phoneme “i”, the speech units 161a, 161b, 161d, 161e,... In the optimal unit sequence are fixed.

  In this state, among the speech units stored in the speech unit storage unit 181, for each speech unit having the same phoneme name (phoneme symbol) as the phoneme “i” of the segment of interest, Equation (5) is obtained. To calculate the cost. However, when the cost is calculated for each speech unit, the value changes for the target cost of the target segment, the connection cost between the target segment and the previous segment, the target segment and the next segment. Since these are the connection costs with the segments, only these costs need be considered.

  (Procedure 1) Among the speech elements stored in the speech element storage unit 181, one of the speech elements having the same phoneme name (phoneme symbol) as the phoneme “i” of the segment of interest is selected as the speech element. Let u3. From the fundamental frequency f (v3) of the speech element u3 and the target fundamental frequency f (t3), the fundamental frequency cost is calculated using Equation (1).

  (Procedure 2) The phoneme duration length cost is calculated from the phoneme duration length g (v3) of the speech unit u3 and the target phoneme duration length g (t3) using Equation (2).

  (Procedure 3) The first spectrum connection cost is calculated from the cepstrum coefficient h (u3) of the speech unit u3 and the cepstrum coefficient h (u2) of the speech unit 161b (u2) using Equation (3). To do. Further, the second spectrum connection cost is calculated from the cepstrum coefficient h (u3) of the speech element u3 and the cepstrum coefficient h (u4) of the speech element 161d (u4) using Equation (3).

  (Procedure 4) Calculate the weighted sum of the fundamental frequency cost, the phoneme duration time cost, and the first and second spectrum connection costs calculated by using each sub-cost function in (Procedure 1) to (Procedure 3). The cost of the speech unit u3 is calculated.

  (Procedure 5) Among the speech elements stored in the speech element storage unit 181, for each speech element having the same phoneme name (phoneme symbol) as the phoneme “i” of the segment of interest, the above (Procedure 1) to When the cost is calculated according to (Procedure 4), ranking is performed so that the speech unit having the smallest value has a higher rank (Step S153 in FIG. 15). Then, the top M speech segments are selected (step S154 in FIG. 15). For example, in FIG. 16, the speech unit 162a has the highest ranking, and the speech unit 162d has the lowest ranking.

  The above (Procedure 1) to (Procedure 5) are performed for each segment. As a result, M speech segments are obtained for each segment.

  For each segment of all input sentences, M speech units are selected by the above procedure, and the unit numbers of the selected M speech units are input to the speech unit combination frequency information creation unit 1833. hand over. The speech unit combination frequency information creation unit 1833 accumulates the frequency information of the unit number combinations in the multiple speech unit combination frequency information storage unit 1835.

  FIG. 17 shows an example of multiple speech unit combination frequency information stored in the multiple speech unit combination frequency information storage unit 1835. The multiple speech unit combination information includes the combination number, phoneme (phoneme name), and the number of appearances of speech units from the 1st to Mth speech units. When the unit number of the input M speech units is present in the multiple speech unit combination frequency information, 1 is added to the appearance frequency corresponding to the combination, and when there is not, the combination is added, The appearance frequency is 1. By performing this for all combinations of segments, appearance frequency information for the input sentence is created.

Next, the speech unit combination determination unit 1834 determines the combination of speech units to be actually merged. There are several ways to determine the combination, but the threshold for the frequency of occurrence is determined in advance, and the method of using the segment whose frequency of occurrence of the combination of multiple speech segments is greater than the threshold, the upper limit for the number of segments per phoneme And selecting the segments in order of appearance frequency, determining the size of the entire fused speech segment group, and selecting in order of appearance frequency within a range not exceeding the size.

  In the frequency information of FIG. 17, when the appearance frequency threshold is set to 30, the number 0 / a / and the number 2 / i / create a fusion speech unit, while the number 1 / a / creates Will not.

  Here, the difference between the speech synthesis method according to the first embodiment and the conventional speech synthesis method will be described. In the method based on COC and the method based on HMM, a fused phoneme parameter is held, and at the time of synthesis, synthesis is performed based on the fused phoneme parameter, but a decision tree is used for selection. For this reason, the selection method is different from the present embodiment in which selection is performed based on the degree of distortion of prosodic information.

  Compared to the clustering method in the form of a decision tree, the method of this embodiment has a high degree of freedom during synthesis and can easily select a fused speech unit from a large number of fused speech units. Therefore, a scalable synthesizer There is an advantage that a synthesized speech with high sound quality can be obtained as the size of the fusion speech unit storage unit 160 is increased.

  In the conventional unit selection type speech synthesis, synthesis is performed by selecting and connecting one speech unit per synthesis unit, but in this embodiment, the selected speech unit is not the speech waveform itself, It is a fused speech unit. By using the fused speech unit, a stable and high-quality speech unit is obtained, and a more natural and high-quality synthesized speech can be generated. Further, since the fusion speech unit per synthesis unit is created in advance, the processing amount at the time of synthesis is close to that of the unit selection type speech synthesis method, and speech synthesis can be performed at high speed.

  FIG. 18 is a diagram illustrating a hardware configuration of the text-to-speech synthesizer 10 according to the first embodiment. The text-to-speech synthesizer 10 has, as a hardware configuration, a ROM 52 that stores a speech synthesis program for executing speech synthesis processing in the text-to-speech synthesizer 10, and controls each unit of the text-to-speech synthesizer 10 according to the program in the ROM 52. Then, a CPU 51 that executes buffering time change processing, a RAM 53 that stores various data necessary for control of the text-to-speech synthesizer 10 and a communication I / O that communicates by connecting to a network. F57 and a bus 62 for connecting each part are provided.

  The speech synthesis program in the text-to-speech synthesizer 10 described above is a file in an installable or executable format and is a computer-readable recording medium such as a CD-ROM, floppy (R) disk (FD), or DVD. May be recorded and provided.

  In this case, the speech synthesis program is loaded onto the main storage device by being read from the recording medium and executed by the text speech synthesizer 10, and each unit described in the software configuration is generated on the main storage device. It is like that.

  Further, the speech synthesis program of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network.

(Embodiment 2)
Next, the text-to-speech synthesizer 10 according to the second embodiment will be described. FIG. 19 is a block diagram of a detailed functional configuration of the speech synthesizer 14 of the text-to-speech synthesizer 10 according to the second embodiment.

  The speech synthesis unit 14 according to the second exemplary embodiment does not include the fused speech unit phoneme environment storage unit 170. Further, the speech synthesis unit 14 according to the second exemplary embodiment includes a fused speech unit combination storage unit 200.

  The fusion speech unit combination storage unit 200 stores a combination of speech units included in the fusion speech unit stored in the fusion speech unit storage unit 160 in association with each fusion speech unit.

  FIG. 20 schematically shows the data configuration of the fused speech unit combination storage unit 200. The fusion speech unit combination storage unit 200 stores phoneme names, ranks and numbers of combinations of speech units, and fusion speech unit numbers in association with each other.

  In Embodiment 2, the phoneme sequence / prosodic information acquisition unit 110 sends the input prosody sequence and input prosodic information acquired from the prosody processing unit 13 to the fused speech unit creation unit 180. The fused speech segment creation unit 180 selects a combination of a plurality of speech segments based on the acquired input prosodic sequence and input prosodic information. Then, the distortion estimation unit 130 estimates the degree of distortion between the combination of the speech units selected by the fusion speech unit creation unit 180 and the input prosodic information acquired from the phoneme sequence / prosodic information acquisition unit 110. .

  The fused speech element selection unit 140 selects a combination that minimizes the degree of distortion estimated by the distortion estimation unit 130. Then, it is determined whether or not the selected combination is stored in the fused speech unit combination storage unit 200. When stored in the fusion speech unit combination storage unit 200, the fusion speech unit corresponding to the combination is extracted from the fusion speech unit storage unit 160. On the other hand, if the selected combination is not stored in the fused speech unit combination storage unit 200, the fused speech unit creating unit 180 is instructed to create a fused speech unit for the combination.

  FIG. 21 is a block diagram of a detailed functional configuration of the fused speech unit creation unit 180 according to the second embodiment. The fusion speech unit creation unit 180 according to Embodiment 2 does not have the fusion speech unit phoneme environment creation unit 185. Then, the fused speech unit creation unit 180 does not create a fused speech unit phoneme environment.

  The speech element combination creation unit 183 creates a speech element combination based on the input prosody information acquired from the phoneme sequence / prosodic information acquisition unit 110. The speech element combination creating unit 183 selects a plurality of speech elements by the process described with reference to FIG. 15 in the first embodiment. The speech unit combination creation unit 183 stores combination information indicating the created speech unit combination in the fusion speech unit combination storage unit 200. The fusion speech unit creation unit 184 creates a fusion speech unit from a plurality of instructed speech units according to an instruction from the fusion speech unit selection unit 140.

  FIG. 22 is a flowchart of a process in which the fused speech unit selection unit 140 according to the second embodiment selects a fused speech unit.

  First, in step S212, based on the degree of distortion estimated by the distortion estimation unit 130, a combination of speech units to be a fusion speech unit is determined. Next, it is determined whether or not the combination determined in step S212 is stored in the fused speech unit combination storage unit 200.

  In the present embodiment, when the speech unit numbers from the first place to the Mth place of the speech unit combination determined in step S212 match the fusion speech unit, the fusion speech unit is the fusion speech unit. It is determined that the data is held in the storage unit 160. On the other hand, if they do not match, it is determined that they are not held in the fused speech unit storage unit 160. When it is determined that the combined speech unit of the combination determined in step S212 is held in the fused speech unit storage unit 160, the process proceeds to step S213.

  In step S213, the fused speech unit combination storage unit 200 is referred to, and a fused speech unit number corresponding to the combination is acquired. And based on the acquired fusion speech unit number, the corresponding fusion speech unit is acquired from the fusion speech unit storage unit 160.

  If it is determined in step S212 that the fusion speech unit does not exist in the fusion speech unit storage unit 160, the plurality of speech units determined in step S211 are determined in step S214 with respect to the fusion speech unit creation unit 180. Send an instruction to create a fusion speech unit from the combination of pieces. In step 215, the corresponding fused speech segment is acquired from the fused speech segment creating unit 180 as a response to the instruction sent to the fused speech segment creating unit 180 in step S 214.

  As described above, the text-to-speech synthesizer 10 according to the present embodiment newly creates a fused speech unit when the fused speech unit is not held in the fused speech storage unit 160, and the fused speech unit is created. Since speech synthesis is performed using speech segments, synthesized speech with higher sound quality can be efficiently generated.

  FIG. 23 shows an example of a fused speech unit sequence. In FIG. 23, for each phoneme of “ts, i, i, s, a”, a fusion speech unit extracted from the fusion speech unit storage unit 160 and a fusion speech unit creation unit 180 are newly created. It shows which one of the fused speech segments is used.

  The fusion speech segments corresponding to ts, i, i, s, a are 221a, 221b, 221c, 221d, and 221e. Here, it is assumed that 221b and 221d do not exist in the fused speech unit storage unit 160, and 221a, 221c, and 221e exist in the fused speech unit storage unit 160.

  In this case, the three segments are created in advance. On the other hand, the remaining two pieces need to be fused at the time of synthesis. Therefore, the number of fusion processes can be reduced to 2/5 compared with the case where all the pieces are fused at the time of synthesis.

  Since the unit fusion process is a process with a large amount of calculation, the process at the time of synthesis is accelerated. In addition, when speech segments are stored in the hard disk drive, the seek time of each speech segment can be reduced.

  That is, in the case of merging at the time of synthesis, it takes M seek times that are the number of fused speech units, whereas in the case of merging in advance, the fusion is performed by fusing M segments in one seek time. A speech segment can be acquired.

  As described above, in the second embodiment, by synthesizing a part of the fusion speech unit used for synthesis in advance, a synthesized speech equivalent to the case where all of the speech units are fused at the time of synthesis can be obtained, and the speech can be transmitted at high speed. Can be synthesized.

  Other configurations and processes of the text-to-speech synthesizer 10 according to the second embodiment are the same as those of the text-to-speech synthesizer 10 according to the first embodiment.

  In the determination step S212 of the second embodiment, when all the combinations of M speech units corresponding to the segments input in S211 match the combinations stored in the fusion speech unit combination storage unit 181. In addition, the fusion speech unit corresponding to the combination is acquired from the fusion speech unit storage unit 160 to obtain a fusion speech unit. Otherwise, the M speech units selected are the speech unit storage unit 181. However, the present invention is not limited to this.

  For example, the lower limit value N of the number of combinations to be matched may be determined in advance. When M or more speech units corresponding to each segment match N or more speech units with the combination in the fused speech unit combination storage unit 181, the fused speech units corresponding to the combination are fused. Obtained from the speech segment storage unit 160. On the other hand, if there are fewer than N matching combinations, a new fused speech unit is created by fusing M speech units.

  As a result, the proportion of using the fusion speech unit acquired from the fusion speech unit storage unit 160 in the fusion speech unit sequence at the time of synthesis increases, and the speech synthesis process is further speeded up.

  As another example, the top N speech units of the combination of M speech units determined by the processing shown in FIG. 15 are the combinations stored in the fusion speech unit combination storage unit 181. It may be based on whether or not it matches the top N.

  When the top N speech units of the M combinations match the top N speech units stored in the fused speech unit combination storage unit 181, the fused speech unit corresponding to the combination is merged speech Obtained from the segment storage unit 160. On the other hand, if they do not match, a new fused speech unit is created by fusing the M speech units.

  Since the higher speech units match, the value of the cost function of the fused speech unit acquired from the fused speech unit storage unit 160 becomes the value of the cost function of the combination of the selected speech units. Approaching, high-quality synthesized speech can be obtained.

(Third embodiment)
Next, the text-to-speech synthesizer 10 according to the third embodiment will be described. FIG. 24 is a block diagram of a detailed functional configuration of the speech synthesizer 14 of the text-to-speech synthesizer 10 according to the third embodiment. In the speech synthesizer 14 according to the third embodiment, the fused speech unit selection unit 140 is based on the degree of distortion estimated by the distortion estimation unit 130 and is stored in the fused speech unit storage unit 160. It is determined whether or not a segment is selected.

  More specifically, when the degree of distortion acquired from the distortion estimation unit 130 is smaller than a predetermined distortion reference value, the corresponding fusion speech unit is extracted from the fusion speech unit storage unit 160. On the other hand, if the degree of distortion acquired from the distortion estimation unit 130 is equal to or greater than the distortion reference value, instead of extracting from the fusion speech unit storage unit 160, the fusion speech unit creation unit 180 is informed of the fusion speech unit. Instruct the creation of. The text-to-speech synthesizer 10 according to the third embodiment is different from the text-to-speech synthesizer 10 according to the other embodiments in this respect.

  FIG. 25 is a flowchart showing processing by the fused speech unit selection unit 140. First, in step S242, the degree of distortion for the prosodic information of each segment is acquired from the distortion estimation unit 130. It should be noted that the minimum value is obtained from the degree of distortion for the plurality of fused speech units from the distortion estimation unit 130.

  Next, the processing from step S243 is performed on each segment. In step S243, it is determined whether or not the degree of distortion acquired from the distortion estimation unit 130 is smaller than a predetermined distortion reference value. If it is greater than or equal to the distortion reference value, that is, if the distortion is large and unacceptable, in step S245, the fused speech unit creating unit 180 is instructed to create a new fused speech unit.

  In step 246, a new fused speech segment is acquired as a response to the instruction. In this case, the fused speech unit creation unit 180 acquires the corresponding input prosodic information and the like via the dividing unit 120. Based on the acquired input prosody information and the like, a plurality of speech segments are selected. The selected speech unit is fused to obtain a fused speech unit.

  On the other hand, if it is smaller than the distortion reference value, that is, if the distortion is small, the corresponding fused speech element is selected from the fused speech element storage unit 160 in step S244. Thus, the processing by the fused speech element selection unit 140 is completed.

  The other configuration and processing of the text-to-speech synthesizer 10 according to the third embodiment are the same as those of the text-to-speech synthesizer 10 according to the first embodiment.

  In this embodiment, a distortion reference value, that is, a threshold value is set in advance. Then, based on the threshold value, it is determined whether or not to create a new fused speech segment. In determining whether or not to create a new fused speech unit, the degree of distortion of prosodic information may be taken into consideration, and the determination is not limited to a determination based on a threshold value.

  FIG. 26 is a diagram for explaining another example of a method for determining whether or not to create a new fused speech unit. First, in step S251, the degree of distortion E1 of the input fusion speech unit for each segment is acquired. In step S252, a plurality of speech units are selected from the speech unit storage unit 181 by the process shown in FIG. In step S253, an average value E2 of the degree of distortion of the prosodic information selected in step 252 is calculated. In step S254, it is determined whether or not the difference between E1 and E2 is greater than a predetermined threshold. Then, based on the determination result, it is determined whether or not to create a new fused speech segment.

  Specifically, if the difference between E1 and E2 is predetermined and smaller than the threshold value, the fused speech unit is selected from the fused speech unit storage unit 160. On the other hand, if the difference between E1 and E2 is greater than or equal to the threshold, an instruction to create a new fused speech segment is issued.

  In the third embodiment, as in the second embodiment, when the degree of distortion is small, a fusion speech unit that is created in advance and held in the fusion speech unit storage unit 160 is used. Can do. Therefore, speech synthesis can be performed at high speed.

  Further, the newly created fused speech unit may be added to the fused speech unit storage unit 160 as appropriate. Thereby, compared with the text-to-speech synthesizer 10 according to the first embodiment in which combinations of a plurality of segments are limited in advance, there are more variations of combinations to be merged. Therefore, high-quality synthesized speech can be obtained.

(Embodiment 4)
Next, the text-to-speech synthesizer 10 according to the fourth embodiment will be described. The text-to-speech synthesizer 10 according to the fourth embodiment updates the contents of the fused speech unit storage unit 160 and the fused speech unit phoneme environment storage unit 170.

  FIG. 27 is a block diagram of a functional configuration of the speech synthesizer 14 according to the fourth embodiment. The speech synthesis unit 14 according to the fourth embodiment further includes an update unit 210 in addition to the configuration of the speech synthesis unit 14 according to the second embodiment. The updating unit 210 acquires a combination of segments from the fusion speech unit editing / connecting unit 150. Then, it is determined whether or not to add the combination to the fused speech unit storage unit 160.

  FIG. 28 is a flowchart showing an update process in the update unit 210. First, in step S <b> 271, a plurality of speech unit combination sequences used at the time of synthesis are acquired from the fusion speech unit editing / connecting unit 150. In step S272, it is determined whether or not the input combination of segments is to be added to the fusion speech unit storage unit 160. For example, the determination is made based on whether or not the combination acquired from the fused speech unit editing / connecting unit 150 is already stored in the fused speech unit storage unit 160.

  If it is determined to be added, the fused speech unit and its combination information are added to the fused speech unit storage unit 160 in step S273. On the other hand, if it is determined not to be added, the combination acquired from the fusion speech unit editing / connecting unit 150 is discarded. Thus, the update process is completed.

  The other configurations and processes of the text-to-speech synthesizer 10 according to the fourth embodiment are the same as the configurations and processes of the text-to-speech synthesizer 10 according to the second embodiment.

  As a first modification example according to the fourth embodiment, the updating unit 210 further includes a fusion stored in the fusion speech unit storage unit 160 in addition to the process of adding a combination to the fusion speech unit storage unit 160. You may perform the process which deletes an audio | voice element. For example, the updating unit 210 monitors the frequency of use of each fused speech unit stored in the fused speech unit storage unit 160. Then, it may be deleted when the use frequency is equal to or less than a predetermined value.

  As a second modification example according to the fourth embodiment, the updating unit 210 may determine whether or not to add to the fused speech unit storage unit 160 according to the use frequency of the combination. Thus, the determination criterion in step S272 is not limited to the present embodiment.

  The updating unit 210 holds the acquired number of times for each combination acquired from the fusion speech unit editing / connecting unit 150. Then, when the same combination is acquired more than a predetermined number of times, the combination may be stored in the fused speech unit storage unit 160. On the other hand, when it is not acquired more than a predetermined number of times, it is discarded.

  More specifically, the updating unit 210 includes a combination temporary holding unit (not shown) configured by, for example, a cache memory. The temporary holding unit holds the combination only for a predetermined period. Then, the number of times of the combination held in the temporary holding unit is counted and held. The updating unit 210 according to the present example constitutes an updating unit and a usage frequency counting unit according to the present invention.

  As a result, only the fusion speech unit for the frequently used combination can be added to the fusion speech unit storage unit 160. Therefore, the memory can be used effectively and the efficiency of the speech synthesis process can be improved.

  As a third modification, the similarity between speech units is defined in the second modification, but the similarity between fused speech units may be defined in the same manner. In other words, in the present embodiment, fused speech segment creating section 180 can create a fused speech segment based on the combination frequency and similarity.

  In this example, the similarity between two fused speech units is the reciprocal of the cost of the two fused speech units. The costs of the two fused speech segments can be defined in the same manner as in equations (16) to (19).

  FIG. 29 is a flowchart showing the fused speech segment creation processing according to the third modification. First, in step S291, a combination of a plurality of speech units is input in order of use frequency. This is created by the speech element combination creation unit 183.

  Next, the following processing is performed for each combination. That is, in step S292, the similarity between each fusion speech unit in the fusion speech unit storage unit 160 and the fusion speech unit created from the acquired combination is obtained. Here, if there is no fused phoneme unit corresponding to the phoneme in the fused phoneme storage unit 160, the similarity is set to zero. If this similarity is larger than a preset threshold, the process proceeds to step S293, and if it is smaller, the process proceeds to step S294.

  Step S293 corresponds to the case where it is determined that there is a similar fusion speech unit. In this case, together with the acquired combination, the unit number of the fusion speech unit having the maximum similarity is added to the fusion speech unit combination storage unit 200.

  Step S294 corresponds to the case where it is determined that there is no similar fused speech unit in the fused speech unit storage unit 160. In this case, a fusion speech unit corresponding to the input combination is added. In step S295, the corresponding combination is added to the fused speech unit combination storage unit 200. As a result, the fusion speech unit in the fusion speech unit storage unit 160 accumulates fusion speech units having a similarity lower than a predetermined threshold value, thereby reducing the memory usage.

  As a fourth modification, in the present embodiment, based on a predetermined condition, a fusion speech unit stored in the fusion speech unit storage unit 160 is used in advance, or a plurality of speech units are used. Although it has been determined whether to create a new fused speech unit from a piece, the condition may be determined in consideration of the amount of computation that can be used, the required specifications for speech synthesis, and the like.

  That is, by using the fusion speech unit stored in advance in the fusion speech unit storage unit 160, the processing efficiency can be improved, but the sound quality may be lowered.

  Specifically, for example, when a fusion speech unit that matches a part of a combination of speech units to the fusion speech unit is selected from the fusion speech unit storage unit 160, a fusion speech created in advance is selected. Since the segment is used, high-speed processing is possible. On the other hand, since the speech unit that does not match is included, the created fusion speech unit is different from the optimum one.

  Therefore, in this example, the frequency of using the fusion speech unit stored in the fusion speech unit storage unit 160 is controlled from the viewpoint of the amount of computation. Thereby, it is possible to control from both the viewpoint of the amount of calculation and the viewpoint of the quality of the synthesized speech.

  It should be noted that a condition determined from the viewpoint of calculation amount or the like may be set as the initial set value in the speech synthesizer 14, and as another example, the condition is appropriately changed from the viewpoint of calculation amount or the like after the initial setting. May be.

  Further, the fusion speech unit creation unit 180 may limit the fusion speech units to be stored in the fusion speech unit storage unit 160 by clustering speech unit groups.

  Specifically, first, the similarity between each speech unit held in the speech unit storage unit 181 is calculated. Then, speech segments are clustered based on the similarity. More specifically, speech units having a high degree of similarity are defined as the same speech unit group. Then, a fusion speech unit for each speech unit group obtained by clustering is created. Furthermore, a fusion speech unit speech environment for the fusion speech unit is created. Then, the updating unit 210 stores the newly created fused speech unit and the fused speech unit speech environment in the fused speech unit storage unit 160 in association with each other.

  For example, clustering of speech unit groups is performed based on the similarity between two speech units. Then, only the fused speech unit having the highest similarity may be held in the fused speech unit storage unit 160 by clustering.

  Specifically, first, the similarity between two segments is defined based on a cost function. Here, the similarity is the reciprocal of the cost between two segments, and clustering is performed so as to minimize the cost.

ここで、ｖｉは音声素片記憶部１８１に記憶されている音声素片ｕｉの音素環境を、ｆは音素環境ｖｉから平均基本周波数を取り出す関数、ｇは音素環境ｖｉから音韻継続時間長を取り出す関数ｈは音声素片ｕｉの平均的なケプストラム係数をベクトルとして取り出す関数を表す。 Based on the above-described cost function, the cost between the two segments is the fundamental frequency cost represented by Equation (16), the duration cost represented by Equation (17), and the average spectrum represented by Equation (18). A linear combination of costs is obtained (formula (19)).

Here, vi is the phoneme environment of the speech unit ui stored in the speech unit storage unit 181, f is a function that extracts the average fundamental frequency from the phoneme environment vi, and g is the phoneme duration length from the phoneme environment vi. The function h represents a function for extracting an average cepstrum coefficient of the speech unit ui as a vector.

これは、すべての素片ｕｉ（１＜ｉ＜Ｉ）に対してＭ個の素片の中でコスト最小の候補ｕｍ（１＜ｍ＜Ｉ）を求め、そのコストを加算したものである。このトータルコストを最小化するようにＭ個の素片を求め、すべての素片をM個の素片中コスト最小の素片に対応付けることによりクラスタリングを行う。 In this way, after obtaining the cost between the two element sides, M pieces are selected such that the total cost is minimized. The selected M pieces and the total cost (total cost) are expressed as in Expression (20).

This is obtained by obtaining a candidate um (1 <m <I) having the smallest cost among M pieces for all the pieces ui (1 <i <I) and adding the costs. Clustering is performed by obtaining M pieces so as to minimize the total cost and associating all the pieces with the piece having the smallest cost among the M pieces.

  A fused speech segment is created by fusing the segments of each cluster obtained by the above calculation. Further, the prosodic information of the fusion speech segment is obtained by obtaining the centroid of the prosodic information of each segment. And it is set as fusion speech unit phoneme environment information.

  As another example, a DTW (dynamic time warping) distance of a cepstrum parameter may be used instead of Expression (18). In this case, a cepstrum corresponding to each pitch waveform is obtained, and time axis expansion / contraction is performed based on the dynamic programming so that the cepstrum distance is minimized to obtain a minimum cepstrum distance.

  In this example, the similarity is defined based on the cost. However, the present invention is not limited to this. For example, the similarity is defined based on the cepstrum distance that is simply time-expanded or the square error of the waveform when the prosody is deformed. May be. The HMM may be learned in each cluster, and the likelihood may be defined as the similarity.

  This makes it possible to create a fusion speech unit to be stored in advance in the fusion speech unit storage unit on the basis of the minimum cost, to efficiently create a fusion speech unit group, and to reduce the amount of memory used. Can do.

  As another example, a threshold may be set for the similarity of fused speech units, and whether to store in the fused speech unit storage unit 160 based on the threshold may be determined. Specifically, the similarity between the fusion speech units is determined. When the similarity is equal to or higher than a predetermined threshold value, the fusion speech unit storage unit 160 stores the similarity. On the other hand, if the similarity is smaller than the threshold value, it is discarded without being stored in the fused speech unit storage unit 160.

  As described above, the present invention has been described using the embodiment, but various changes or improvements can be added to the above embodiment.

  As an example of such change, in the present embodiment, as described with reference to FIG. 11 and the like, the fused speech segment creating unit 184 creates a voiced fused speech segment by averaging pitch waveforms. However, the method of creating the fused speech segment is not limited to this. For example, closed loop learning may be used. By using closed loop learning, an optimum pitch waveform sequence can be created at the level of the synthesized sound without extracting the pitch waveform of each speech unit.

  Here, closed-loop learning is a method of generating a representative speech segment that reduces the distortion of natural speech at the level of synthesized speech that is actually synthesized by changing the fundamental frequency and prosodic duration. That is, in closed-loop learning, a segment whose distortion is reduced at the level of the synthesized speech is generated (see Japanese Patent No. 3281281).

  A case where voiced speech segments are fused using closed loop learning will be described. The speech segment obtained by the fusion is obtained as a series of pitch waveforms as in the first embodiment. A speech unit is represented by a vector u configured by connecting these pitch waveforms.

ｒｊのピッチマークとｕのピッチ波形とのマッピング、およびｒｊのピッチマーク位置より行列Ａｊは決定される。行列Ａｊの例を図３０に示す。次に、合成音声セグメントｓｊとｒｊの誤差を評価する。ｓｊとｒｊの誤差ｅｊを次式（１０）で定義する。

ただし、次式（１１）、（１２）に示すように、ｇｉは、２つの波形の平均的なパワーの差を補正して、波形の歪みのみを評価するためのゲインであり、ｅｊが最小となるような最適なゲインを用いている。

ベクトルｒｉ全てに対する誤差の総和を表す評価関数Ｅを次式（１３）で定義する。

Ｅを最小にする最適なベクトルｕは、Ｅをｕで偏微分して「０」とおくことで得られる次式（１４）、（１５）を解くことによって求められる。

式（１５）はｕについての連立方程式であり、これを解くことによって新たな音声素片ｕを一意に求めることができる。ベクトルｕが更新されることによって、最適ゲインｇｊが変化するため、上述したプロセスをＥの値が収束するまで繰り返し、収束した時点のベクトルｕを、融合によって生成された音声素片として用いる。 First, an initial value of a speech unit is prepared. As the initial value, a series of pitch waveforms obtained by the method described in the first embodiment may be used. Further, random data may be used. Also, let rj (j = 1, 2,..., M) be a vector representing the waveform of the combination speech unit created by the speech unit combination creation unit 183. Next, using u, the speech is synthesized with rj as the target. The generated synthesized speech segment is represented as sj. sj is represented by the product of a matrix Aj representing the superposition of pitch waveforms and u as shown in the following equation (9).

The matrix Aj is determined from the mapping between the pitch mark of rj and the pitch waveform of u and the pitch mark position of rj. An example of the matrix Aj is shown in FIG. Next, the error between the synthesized speech segments sj and rj is evaluated. An error ej between sj and rj is defined by the following equation (10).

However, as shown in the following equations (11) and (12), gi is a gain for correcting only the waveform distortion by correcting the average power difference between the two waveforms, and ej is the minimum. The optimal gain is used.

An evaluation function E representing the sum of errors for all vectors ri is defined by the following equation (13).

The optimal vector u that minimizes E is obtained by solving the following equations (14) and (15) obtained by partial differentiation of E by u and setting it to “0”.

Equation (15) is a simultaneous equation for u, and a new speech unit u can be uniquely obtained by solving this. Since the optimum gain gj changes as the vector u is updated, the above-described process is repeated until the value of E converges, and the vector u at the time of convergence is used as the speech segment generated by the fusion.

  Further, the pitch mark position of rj when obtaining the matrix Aj may be corrected based on the correlation between the waveform of rj and u.

Alternatively, the vector rj may be divided into bands, the closed loop learning described above is performed for each band to obtain u, and a united speech unit may be generated by adding u of all bands.
In this way, by using closed loop learning for unit fusion, it is possible to generate a speech unit in which the synthesized speech is less degraded by changing the pitch period.

  In addition, when a newly created fused speech unit is stored in the fused speech unit storage unit 160, the similarity to the already stored fused speech unit may be calculated. Specifically, when the speech unit creation unit 180 creates the fusion speech unit, the created fusion speech unit and the fusion speech unit already stored in the fusion speech unit storage unit 180 Calculate similarity. If the similarity is smaller than a predetermined value, the speech unit creation unit 180 newly stores the fusion speech unit in the fusion speech unit storage unit 180. As a result, it is possible to avoid storing relatively similar fused speech segments, so that the memory can be used effectively.

1 is a block diagram showing an overall configuration of a text-to-speech synthesizer according to a first embodiment of the present invention. It is a block diagram which shows the detailed structure of the speech synthesis part 14 of FIG. FIG. 3 is a block diagram illustrating a detailed functional configuration of a fusion speech unit creation unit 180 described in FIG. 2. It is a block diagram which shows the detailed functional structure of the speech unit combination preparation part 183 shown in FIG. It is a figure which shows typically the data structure of the fusion speech unit memory | storage part 160. FIG. It is a figure which shows typically the data structure of the fusion speech element phoneme environment storage part 170. FIG. It is a figure for demonstrating the process of the fusion speech unit edit and the connection part 150 demonstrated in FIG. 3 is a diagram schematically showing a data configuration of a speech unit storage unit 181. FIG. It is a figure which shows typically the data structure of the fusion speech unit phoneme environment storage part 182. It is a figure which shows the result of having performed the labeling for every phoneme with respect to the audio | voice data 101. FIG. It is a figure for demonstrating the process sequence in the case of uniting the M speech unit determined by the speech unit combination production | generation part 183, and producing | generating one new speech unit. It is a figure for demonstrating the process which provides a pitch mark with respect to the audio | voice waveform of a speech unit in step S111. It is a figure for demonstrating the process which provides a pitch mark with respect to the audio | voice waveform of a speech unit in step S111. It is a figure for demonstrating the process which provides a pitch mark with respect to the audio | voice waveform of a speech unit in step S111. It is a figure which shows the series e1-e3 of the pitch waveform cut out by step S112 from each of the speech segment d1-d3. It is a figure which shows series e1, e2 ', e3' of the pitch waveform calculated | required by step S113 from each of the speech segments d1-d3. It is a flowchart which shows the process which selects a speech segment. It is a figure which shows an input prosodic sequence. It is a figure which shows the example of the several speech unit combination frequency information stored in the several speech unit combination frequency information storage part 1835. FIG. 1 is a diagram illustrating a hardware configuration of a text-to-speech synthesizer 10 according to Embodiment 1. FIG. It is a block diagram which shows the detailed functional structure of the speech synthesizer 14 of the text speech synthesizer 10 concerning Embodiment 2. FIG. It is a figure which shows typically the data structure of the fusion speech unit combination memory | storage part 200. FIG. FIG. 6 is a block diagram showing a detailed functional configuration of a fusion speech unit creation unit 180 according to the second exemplary embodiment. It is a flowchart which shows the process which the fusion speech unit selection part 140 concerning Embodiment 2 selects a fusion speech unit. It is a figure which shows the example of a fusion speech unit series. It is a block diagram which shows the detailed function structure of the speech synthesizing part 14 of the text-to-speech synthesizer 10 concerning Embodiment 3. It is a flowchart which shows the process by the fusion speech unit selection part 140. FIG. It is a figure for demonstrating the other example of the determination method of whether to produce a new fusion speech unit. It is a block diagram which shows the function structure of the speech synthesizer 14 concerning Embodiment 4. 5 is a flowchart showing an update process in an update unit 210. It is a flowchart which shows the fusion speech unit creation process concerning the example 3 of a change. It is a figure which shows the example of matrix Aj.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Text speech synthesizer 11 Text acquisition part 12 Language processing part 13 Language processing part 14 Speech synthesis part 15 Speech waveform output part 110 Phonological sequence and prosodic information acquisition part 120 Dividing part 130 Distortion estimation part 140 Fusion speech unit selection part 150 Fusion Speech unit editing / connecting unit 160 Fusion speech unit storage unit 170 Fusion speech unit phoneme environment storage unit 180 Fusion speech unit creation unit 181 Speech unit storage unit 182 Fusion speech unit phoneme environment storage unit 183 Speech unit combination Creation unit 184 Fusion speech unit creation unit 185 Fusion speech unit phoneme environment creation unit 200 Fusion speech unit combination storage unit 210 Update unit 1831 Phoneme sequence / prosodic information acquisition unit 1832 Multiple speech unit selection unit 1833 Speech unit combination frequency Information creation unit 1834 Multiple speech unit combination determination unit 1835 Speech unit combination Allowed frequency information frequency information storage unit 51 CPU
52 ROM
53 RAM
57 Communication I / F
62 Bus

Claims (17)

A plurality of speech units corresponding to the same speech unit and having different prosody of the speech unit and speech unit prosody information indicating the prosody of the speech unit are stored in association with each other. Speech segment holding means;
Based on teacher speech prosody information indicating a preset prosody of a teacher speech and the speech unit prosody information held in the speech unit holding unit, a plurality of speech units are generated from the speech unit holding unit. Speech segment selection means for selecting
A combination determining unit that determines a combination of a plurality of speech units that satisfy a predetermined condition from the plurality of speech units selected by the speech unit selection unit;
Based on the plurality of speech units included in the determined combination, fused speech unit creating means for creating a fused speech unit by fusing a plurality of speech units;
Fusion speech segment prosodic information creation for creating fused speech segment prosodic information indicating the prosody of the fused speech segment based on the prosodic information corresponding to each of the plurality of speech segments included in the determined combination Means,
Fused speech unit holding for holding the fused speech unit created by the fused speech unit creating unit and the fused speech unit prosody information created by the fused speech unit prosody information associated with each other Means,
Acquisition means for acquiring a prosodic sequence for a target speech to be synthesized for each of a plurality of segments that are synthesis units of speech synthesis;
Retained speech distortion estimation for estimating the degree of distortion between segment prosodic information indicating the prosody of the segment obtained by the acquiring means and the fused speech segment prosodic information held in the fused speech segment holding means Means,
Based on the degree of distortion estimated by the retained speech distortion estimation means, fused speech segment selection means for selecting the fused speech segment;
A speech synthesizer comprising: speech synthesis means for generating synthesized speech by connecting the fused speech units selected by the fused speech unit selection means for each segment. The speech synthesis means selects the fused speech selected by the fused speech segment selection means when the degree of distortion estimated by the retained speech distortion estimation means is smaller than a predetermined retained speech distortion reference value. The speech synthesis apparatus according to claim 1 , wherein the speech synthesis is performed using a segment. It said speech synthesis means, when the degree of the distortion the holding audio distortion estimating means has estimated for each fused speech unit stored in the fused speech unit holding means is the holding audio distortion reference value or more 3. The speech synthesizer according to claim 2 , wherein speech synthesis is performed using the fused speech unit created by the fused speech unit creating means. The fused speech unit selection means any of claims 1 to 3, characterized by selecting the fused speech unit corresponding to the minimum value of the degree of the distortion estimated by the holding audio distortion estimator The speech synthesizer according to claim 1. Further comprising combination information holding means for holding combination information indicating a combination of the plurality of speech units included in the fused speech unit held in the fused speech unit holding means;
The holding audio distortion estimating means of claims 1 to 4, characterized in that for estimating the degree of coincidence between the combination of the combination information holding means and combinations in speech of the segment is held as the degree of the strain The speech synthesizer as described in any one of Claims. The fusion speech unit selection means is configured such that the retained speech distortion estimation means matches a combination of the speech units of the speech of the segment and a combination of the fusion speech units held in the fusion speech unit holding means. 6. The speech synthesizer according to claim 5 , wherein when it is determined, a fusion speech unit corresponding to the combination is selected. The fusion speech unit selection means is a combination of the fusion speech unit held by the fusion speech unit holding means and a part of the combination of the speech units of the speech of the segment by the held speech distortion estimation means. The speech synthesizer according to claim 6 , wherein the combination is determined to match when a part of the matches. The combination information holding unit holds a priority order for the combination in association with each combination,
In the fused speech unit selection means, the held speech distortion estimation means matches the combination of the speech units of the speech of the segment and the combination of the fused speech units held in the fused unit holding means. The speech synthesis apparatus according to claim 6 or 7 , wherein the fusion speech unit is selected when the priority of the fusion speech unit is equal to or higher than a predetermined priority reference value. The speech synthesizer performs the speech synthesis using a retained speech distortion reference value determined based on at least one of a computation amount of the speech synthesis process and a sound quality of the synthesized speech to be synthesized. The speech synthesizer according to claim 2 . A frequency information creating means for counting the frequency of use of the combination ;
The combination determining means determines the combination in which the use frequency is equal to or higher than a predetermined threshold;
The speech synthesizer according to any one of claims 1 to 9 . According to any one of claims 1 to 10, characterized in that the fused speech unit created by the fused speech unit producing means further comprising updating means for storing the fused speech unit holding means Voice synthesizer. The updating means further comprises a usage frequency counting means for counting the usage frequency of the fused speech unit created by the fused speech segment creating means,
The updating means stores the corresponding fused speech element in the fused speech element holding means when the usage frequency counting means counts a value equal to or greater than a predetermined usage frequency reference value. The speech synthesizer according to claim 11 . Similarity calculating means for calculating the similarity between the fused speech unit created by the fused speech unit creating means and the fused speech unit held in the fused speech unit holding means;
When the similarity calculated by the similarity calculation unit is smaller than a predetermined value, the fusion speech unit created by the fusion speech unit creation unit is stored in the fusion speech unit holding unit. speech synthesis apparatus according to any one of claims 1 to 10, characterized in, further comprising a updating means for. The similarity calculation means includes: a time-stretched spectral distance between two speech segments, a square error of a waveform when prosody is deformed, a pitch pattern distance corresponding to the speech segment, and a prosodic duration distance The speech synthesis apparatus according to claim 13 , wherein the similarity is calculated using at least one. 2. The speech synthesis according to claim 1 , wherein the fused speech segment prosodic information creation unit creates a centroid of the prosodic information for each of the plurality of speech segments as the fused speech segment prosodic information. apparatus. A plurality of speech units corresponding to the same speech unit and having a plurality of speech units having different prosody of the speech unit and speech unit prosody information indicating the prosody of the speech unit in association with each other Based on the speech unit prosody information held in the unit holding unit and the teacher speech prosody information indicating the preset prosody of the teacher speech, a plurality of speech units are obtained from the speech unit holding unit. A speech segment selection step to select;
A combination determining step of determining a combination of a plurality of the speech elements that satisfy a predetermined condition from the plurality of speech elements selected by the speech element selection step;
Based on the plurality of speech units included in the determined combination, a fused speech unit creating step for creating a fused speech unit obtained by fusing a plurality of the speech units;
Fusion speech segment prosodic information creation for creating fused speech segment prosodic information indicating the prosody of the fused speech segment based on the prosodic information corresponding to each of the plurality of speech segments included in the determined combination Steps,
In the fused speech unit holding means, the fused speech unit created by the fused speech unit creating step and the fused speech unit prosody information created by the fused speech unit prosody information creating step are associated with each other. A save step to save;
An acquisition step of acquiring a prosodic sequence for a target speech to be synthesized for each of a plurality of segments that are synthesis units of speech synthesis;
Said fused speech unit prosody information held in the fused speech unit holding means, holding audio distortion to estimate the degree of distortion between the resulting segment prosody information indicating the prosody of said segments in said obtaining step An estimation step;
A fusion speech unit selection step of selecting the fusion speech unit based on the degree of distortion estimated in the retained speech distortion estimation step;
A speech synthesis method comprising: a speech synthesis step of generating synthesized speech by connecting each fused speech unit selected for each segment in the fused speech unit selection step. A speech synthesis program for causing a computer to execute speech synthesis processing,
A plurality of speech units corresponding to the same speech unit and having a plurality of speech units having different prosody of the speech unit and speech unit prosody information indicating the prosody of the speech unit in association with each other Based on the speech unit prosody information held in the unit holding unit and the teacher speech prosody information indicating the preset prosody of the teacher speech, a plurality of speech units are obtained from the speech unit holding unit. A speech segment selection step to select;
A combination determining step of determining a combination of a plurality of the speech elements that satisfy a predetermined condition from the plurality of speech elements selected by the speech element selection step;
Based on the plurality of speech units included in the determined combination, a fused speech unit creating step for creating a fused speech unit obtained by fusing a plurality of the speech units;
Fusion speech segment prosodic information creation for creating fused speech segment prosodic information indicating the prosody of the fused speech segment based on the prosodic information corresponding to each of the plurality of speech segments included in the determined combination Steps,
In the fused speech unit holding means, the fused speech unit created by the fused speech unit creating step and the fused speech unit prosody information created by the fused speech unit prosody information creating step are associated with each other. A save step to save;
An acquisition step of acquiring a prosodic sequence for a target speech to be synthesized for each of a plurality of segments that are synthesis units of speech synthesis;
Said fused speech unit prosody information held in the fused speech unit holding means, holding audio distortion to estimate the degree of distortion between the resulting segment prosody information indicating the prosody of said segments in said obtaining step An estimation step;
A fusion speech unit selection step of selecting the fusion speech unit based on the degree of distortion estimated in the retained speech distortion estimation step;
A speech synthesis program comprising: a speech synthesis step of generating synthesized speech by connecting each fused speech unit selected for each segment in the fused speech unit selection step.

Images