JP2012141354A - Method, apparatus and program for voice synthesis - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method, an apparatus and a program for voice synthesis, capable of synthesizing a voice compatible with a target text and having the characteristic of a target speaker by using a similar speaker voice database obtained from a small amount of the voice data of the target speaker.SOLUTION: A method includes: steps of pre-storing a similar speaker voice database that comprises plural pieces of partial voice data having a high degree of speaker similarity with voice data of a target speaker and a voice fragment at least representing a similar speaker identifier indicating a speaker vocalizing partial voice data and phoneme information indicating a vocalized phoneme of the partial voice data; searching, on the basis of the phoneme information, the similar speaker voice database for a voice phoneme candidate(s) conforming to a phoneme context of a target text in synthesis unit; and calculating a degree of conformity, as a total cost, between the target text in synthesis unit and the voice phoneme candidate by using at least a degree of speaker similarity corresponding to the similar speaker identifier of each voice phoneme candidate.

Description

  The present invention relates to a speech synthesis method, a speech synthesizer, and a speech synthesis program that synthesize speech having characteristics of an arbitrary speaker corresponding to a target text.

  A speech synthesizer 10 described in Patent Document 1 is known as a prior art. The outline of the speech synthesizer 10 will be described with reference to FIG.

  When a text to be synthesized (hereinafter referred to as “target text”) is input, first, the text analysis unit 11 performs morphological analysis such as dependency and part-of-speech analysis, kanji-kana conversion, and accent processing. A symbol string indicating phoneme distinction is output to the segment selection connection unit 14, and the number of mora in the exhalation paragraph, accent shape, and utterance speed are output to the prosody generation unit 12.

Next, the prosody generation unit 12 generates a pitch pattern, a time length pattern and an amplitude pattern of each phoneme based on the received information, and outputs them to the segment selection connection unit 14.
Finally, the segment selection connection unit 14 selects an optimum waveform from the speech database 15 based on the symbol string indicating the phoneme distinction, the pitch pattern, the time length pattern, and the amplitude pattern, and synthesizes the speech by connecting them. ,Output.

  In the case of Patent Document 1, if there are a large number of segments of the same context in the speech database 15, variations in the pitch pattern, time length pattern, and amplitude pattern increase, and the quality of the synthesized speech is improved. However, in order to obtain synthesized speech with sufficient quality, a large amount of speech is required. Therefore, in many speech synthesizers, the number of speakers that can be synthesized is limited to about a few prepared in advance. If the user wants to generate or select the voice of his / her favorite speaker, he / she needs a large amount of voice (at least several hours of voice) of the speaker to be synthesized (hereinafter referred to as “target speaker”). It becomes.

A speech synthesizer 20 described in Non-Patent Document 1 is known as a prior art that solves such a problem. The speech synthesizer 20 will be described with reference to FIG.
The multi-speaker voice database 21 stores voice data of a large number of speakers in advance.
The model learning unit 22 receives voice data of a large number of speakers from the multi-speaker voice database 21, and learns an average voice model having average voice characteristics of a large number of speakers.

  The conversion rule learning unit 23 learns a conversion rule for converting the average voice model into an adaptive model from the average voice model and the target speaker's voice data, and outputs this to the adaptation unit 24. The adaptive model is a speech model similar to a speech model obtained from a large amount of speech data of the target speaker.

The adaptation unit 24 adapts the conversion rule to the average voice model and converts it into an adaptive model.
When the target text is input, the synthesizer 25 generates and outputs synthesized speech based on the adaptive model.

Japanese Patent No. 2761552

Masanori Tamura, Takashi Masuko, Keiichi Tokuda, Takao Kobayashi, “Speaker Adaptation of Pitch Spectrum in HMM-Based Speech Synthesis”, IEICE Transactions, April 2002, vol. J85-D-II, no. 4, pp. 545-553

  Compared with Patent Document 1, Non-Patent Document 1 can significantly reduce the amount of target speaker's speech data required to create a database and model necessary for speech synthesis. However, Non-Patent Document 1 also requires about a few minutes of target speaker's voice data in order to learn conversion rules. Therefore, the target speaker must be restrained for a long time when recording audio. For example, in order to record 5 minutes of audio data, a restraint time of about 30 minutes is required.

  Therefore, the present invention provides a speech synthesis method for synthesizing speech having characteristics of a target speaker corresponding to a target text, using a similar speaker speech database obtained from speech data of a smaller amount of the target speaker. An object is to provide a device and a speech synthesis program.

  In order to solve the above problems, according to the first aspect of the present invention, a synthesized speech corresponding to the target text and having the speech characteristics of the target speaker is generated. A synthesis unit suitable for assembling a plurality of speech data having a high speaker similarity with the target speaker's speech data as an index indicating whether or not the two speech data are similar Divided partial voice data, information given to the partial voice data, a similar speaker identifier indicating the speaker who has issued the partial voice data, and phoneme information indicating the utterance phoneme of the partial voice data; Is stored in advance as a similar speaker speech database consisting of speech segments indicating at least. Analyzes the target text and obtains reading information of the target text. The reading information is converted into a phoneme context that is a sequence of phonemes. Based on the phoneme information, a speech unit candidate that matches the phoneme context in synthesis units is searched from the similar speaker speech database. Using at least the speaker similarity corresponding to the similar speaker identifier of each speech unit candidate, the degree of matching between the synthesis target text and the speech unit candidate is calculated as the total cost, and this total cost is the best. Are selected as selected speech segments. Partial speech data corresponding to the selected speech segment is read from the similar speaker speech database, and this partial speech data is connected to obtain synthesized speech.

  In order to solve the above-mentioned problem, according to the second aspect of the present invention, a synthesized speech corresponding to the target text and having the speech characteristics of the target speaker is generated. The partial speech data of an appropriate synthesis unit for assembling the synthesized speech, the information given to the partial speech data, the similar speaker identifier indicating the speaker who has issued the partial speech data, and the partial speech data Speaker similarity that stores a similar speaker speech database that includes at least a speech segment that indicates phoneme information indicating a utterance phoneme, a similar speaker identifier, and a speaker similarity corresponding to the similar speaker identifier Based on the phoneme information, a storage unit, a text analysis unit that analyzes the target text and obtains reading information of the target text, a phoneme context conversion unit that converts the reading information into a phoneme context that is a sequence of phonemes, A speech unit candidate search unit that searches speech unit candidates that match the context by synthesis unit from a similar speaker speech database, and corresponds to a similar speaker identifier of each speech unit candidate Using at least the speaker similarity, the degree of matching between the target text of the synthesis unit and the speech unit candidate is calculated as the total cost, and the speech unit candidate when the total cost is the best is selected as the selected speech unit. And a segment connection unit that reads partial speech data corresponding to the selected speech segment from the similar speaker speech database and connects the partial speech data to obtain synthesized speech.

  The present invention performs speech synthesis using a similar speaker voice database composed of a plurality of voice data having a high speaker similarity and the speaker similarity, so that the voice data of a speaker more similar to the target speaker Can be easily used in speech synthesis, and the similarity of the synthesized speech to the target speaker can be improved.

The block diagram which shows the structure of the conventional speech synthesizer 10. FIG. The block diagram which shows the structure of the conventional speech synthesis apparatus 20. FIG. The block diagram which shows the function structural example of the speech synthesizer 100,200,300. The figure which shows the processing flow of the speech synthesizer 100,200,300. The figure which shows the data structure of a speech unit. The block diagram which shows the function structural example of the similar speaker audio | voice database construction part 110. FIG. The block diagram which shows the function structural example of the similar speaker selection part 111. FIG. The figure which shows the processing flow of the similar speaker selection part 111. FIG. The figure which shows the processing flow of the speaker integration part 115. FIG. FIG. 3 is a block diagram illustrating an example of a functional configuration of a speech synthesizer 150. The figure which shows the processing flow of the speech synthesizing part 150. The block diagram which shows the function structural example of the similar speaker audio | voice database construction part 210. FIG. The figure which shows the processing flow of the similar speaker audio | voice database construction part 210. The figure which shows the processing flow of the speaker conversion rule learning part 213 and the speaker unit converter 214. The block diagram which shows the function structural example of the similar speaker audio | voice database construction part 310. FIG. The figure which shows the processing flow of the similar speaker audio | voice database construction part 310.

Hereinafter, embodiments of the present invention will be described.
<Speech Synthesizer 100 according to First Embodiment>
The speech synthesizer 100 according to the first embodiment will be described with reference to FIGS. 3 and 4. The speech synthesizer 100 includes a multi-speaker speech database construction unit 101, a multi-speaker speech database 103, a similar speaker speech database construction unit 110, a similar speaker speech database 130, and a speaker similarity storage unit 140.

<Multi-speaker speech database construction unit 101 and multi-speaker speech database 103>
The multi-speaker voice database construction unit 101 records the voices of a large number of speakers (for N names) in advance, and constructs a multi-speaker speech database 103 used by the similar speaker speech database construction unit 110 (s101).

  It should be noted that since the voices of many speakers to be recorded are used by the similar speaker voice database construction unit 110 and the voice synthesis unit 150, it is desirable to satisfy the following requirements (1) and (2). (1) The amount of voice data per person recorded (the time of the voice section excluding the silent section) is longer than the time during which the model for voice synthesis can be learned. Note that the learning time varies depending on the speech synthesis system to be used. For example, in the unit selection speech synthesis, a speech data amount of about several hours is required. (2) It is desirable that the number N of speakers to be recorded is at least 100 for each gender, and 200 or more in total.

Furthermore, after the end of voice recording, a voice segment is added to the recorded voice data. Therefore, the multi-speaker speech database 103 holds speech data with speech segments for N names.
Here, the voice data is voice waveform data obtained by performing A / D conversion on a real voice signal that reads out a word or a sentence and converting it into digital data. This speech waveform data can be used as a material for waveform-connected speech synthesis.

FIG. 5 shows an example of a data structure (table) made up of speech segments. A speech unit indicates various pieces of information of speech data (hereinafter referred to as “partial speech data”) in an appropriate unit (hereinafter referred to as “synthesis unit”) for assembling synthesized speech. At least partial speech data is generated. And a phoneme information indicating the synthesized phonetic phoneme. Further, for example, position information (start time, end time) indicating the position of partial audio data with respect to all audio data, F0 pattern information of partial audio data, and the like may be included. Note that the speech unit may be assigned manually or automatically by a computer. For example, phoneme information and position information may be automatically performed by a computer using the technique described in Reference Document 1 below.
(Reference 1) Japanese Patent Application Laid-Open No. 2004-77901

  In this embodiment, the synthesis unit is a phoneme from the viewpoint of specific description. In addition, for example, a syllable or semi-syllable may be used, or a combination of phonemes, syllables, semi-syllables, or the like may be used as a synthesis unit, and can be arbitrarily determined.

<Similar Speaker Speech Database Construction Unit 110, Similar Speaker Speech Database 130, and Speaker Similarity Storage Unit 140>
The similar speaker speech database construction unit 110 constructs a similar speaker speech database composed of speech data similar to the speech data of the target speaker using a plurality of speech data (s110). As shown in FIG. 6, the similar speaker voice database construction unit 110 includes a similar speaker selection unit 111 and a speaker integration unit 115.

<Similar speaker selection unit 111>
The similar speaker selection unit 111 receives a plurality of voice data stored in the multi-speaker voice database 103 as input, and uses the voice data of each speaker and the voice data of the target speaker. A plurality of speech data having a high speaker similarity are selected (s111), and speech data with speech units of similar speakers is output to the speaker integration unit 115. The speaker similarity is an index indicating whether or not two voice data are similar.

For example, the speaker similarity is obtained based on the technique described in Reference 2 using a Gaussian Mixture Model (GMM) used in speaker identification / authentication.
(Reference 2) DA Reynolds, “Speaker identification and verification using Gaussian mixture speaker models”, Speech Communication, 1995, vol.17, pp.91-108
In this case, as shown in FIG. 7, the similar speaker selection unit 111 includes a mixed normal distribution learning unit 111a, a multi-speaker mixed normal distribution storage unit 111b, a speaker similarity calculation unit 111c, and a similar speaker extraction unit 111d. .

(Mixed normal distribution learning unit 111a and multi-speaker mixed normal distribution storage unit 111b)
The mixed normal distribution learning unit 111a receives voice data for N names, and performs the following processing (s111a-2 and s111a-3 in FIG. 8) on the voice data of all speakers (s111a-1). , S111a-4, s111a-5). Using the voice data of each speaker n, a spectrum parameter (cepstrum, mel cepstrum, etc.) is acquired from each voice data (s111a-2). Further, the mixture normal distribution λ _n is learned using the spectral parameters obtained from the respective speech data, and the mixture weights w _n (m), the average vector μ _n (m), and the variance vector ν _n (m ) Is estimated (s111a-3), and these values are output to the multi-speaker mixed normal distribution storage unit 111b. Here, m = 1, 2,..., M, and M is the number of mixtures in the mixed normal distribution.

Further, the mixed normal distribution learning unit 111a similarly learns a mixed normal distribution λ _U (Universal Background Model; UBM) for all audio data using spectral parameters obtained from all (N names) audio data. Then, the mixture weights w _U (m), the average vector μ _U (m), and the variance vector ν _U (m), which are model parameters, are estimated (s111a-6), and these values are stored in a multi-speaker mixed normal distribution storage unit. To 111b.

(Speaker similarity calculator 111c)
The speaker similarity calculation unit 111c receives the mixture weights w _n (m), w _U (m), the average vectors μ _n (m), μ _U (m), and the variance vector ν from the multi-speaker mixed normal distribution storage unit 111b. _n (m) and ν _U (m) are acquired, and these values and the voice data of the target speaker are input. First, the speaker similarity calculation unit 111c acquires the spectrum parameter series X from the speech data of the target speaker. Next, the speaker similarity L _n of each speaker n is calculated as the following log likelihood (s111c-2). The speaker similarity L _n of all the speakers _n is calculated (s111c-1, s111c-3, s111c-4) and output to the similar speaker extraction unit 111d.

The number of dimensions and the number of frames of the spectral parameter series X are R and T, respectively, x (t) is a vector of spectral parameters of the t-th frame, and χ _t (r) is the r-th dimension of the t-th frame. It is a spectral parameter. Μ _i (m, r) and σ _i (m, r) are parameters of the mixed normal distribution λ _i and represent the average and standard deviation of the r-th dimension of the m-th mixture distribution. From equations (1) to (4), the speaker similarity L _n is larger for speech data having speech features similar to those of the speech data of the target speaker.

(Similar speaker extraction unit 111d)
The similar speaker extraction unit 111d receives the speaker similarity L _n and extracts the upper S names having the highest speaker similarity (s111d). However, the speaker with the upper S name is called a similar speaker s, 2 ≦ S ≦ N, and s = 1, 2,. The similar speaker extraction unit 111 d outputs the extracted speaker similarity L _s of the upper S name to the speaker similarity storage unit 140, and the extracted voice data with the speech unit of the upper S name extracted by the speaker integration unit 115. Output to. For example, the speaker similarity storage unit 140 stores a speaker identifier of a similar speaker (hereinafter referred to as “similar speaker identifier”) and a speaker similarity corresponding to the similar speaker.

<Speaker integration unit 115>
The speaker integration unit 115 integrates a plurality of selected speech unit-attached speech data, and constructs a similar speaker speech database composed of partial speech data and speech segments of the partial speech data (s115).

  For example, as shown in FIG. 9, the audio data is integrated. First, all the partial speech data of the synthesis unit phoneme p included in the speech data of the similar speaker s and the corresponding speech segment are extracted (s115c). This is performed for all similar speakers (s115b, s115d, s115e), and the partial speech data corresponding to the extracted phoneme p of the synthesis unit is added to the similar speaker speech database 130. At that time, the speech unit corresponding to the partial speech data has the same configuration as that of the multi-speaker speech database (see FIG. 5), but the order of the speech unit numbers is changed and the start time and end time are similar. The position is changed to the position of each partial voice data on the speaker voice database 130. The above processing is performed on all phonemes (s115a, s115g, s115h), and the speech data of similar speakers are integrated.

  Normally, when a single voice database is created by integrating the voice data of a plurality of speakers, there is a possibility that abnormal noise or the like may occur at the time of waveform connection because the voice feature amounts between the speakers differ greatly. Yes, the quality of the synthesized speech is degraded. However, since a similar speaker is selected by the similar speaker selection unit 111, a difference in speech feature amount between the speakers is reduced. As a result, the quality of synthesized speech is unlikely to deteriorate. Furthermore, by integrating the speech data of a plurality of similar speakers, variations in speech data such as intonation and phoneme environment before and after increase in the similar speaker speech database 130. This improves the naturalness of the synthesized speech.

<Speech synthesizer 150>
The speech synthesizer 150 uses the speech data with speech units of similar speakers stored in the similar speaker speech database 130 and the speaker similarity stored in the speaker similarity storage unit 140 as target text. Corresponding synthesized speech is generated (s150 in FIG. 4).

  As shown in FIG. 10, the speech synthesis unit 150 includes a text analysis unit 151, prosody generation unit 152, prosody model storage unit 153, phoneme context conversion unit 154, speech unit candidate search unit 155, unit selection unit 156, and unit. And a single connection portion 157.

  The target text input to the speech synthesizer 150 may be input from an input unit (not shown), or may be stored in advance in a storage unit (not shown). In the present invention, the type of the target text is not particularly limited, and in this embodiment, Japanese text mixed with kanji and kana is used.

<Text analysis unit 151>
First, the text analysis unit 151 acquires the target text, performs morphological analysis on the target text, and outputs reading information corresponding to the target text to the phoneme context conversion unit 154 and prosodic information to the prosody generation unit 152 (s151). ).

  The outline of the morphological analysis will be described. The text analysis unit 151 converts the target text into kana by referring to a word model, a kanji conversion model, etc. (these are stored in a storage unit (not shown) as necessary). (Acquire reading information). In addition, when the target text is Japanese, if a plurality of words are gathered to form a phrase or the like, a phenomenon such as an accent moving or disappearing occurs, so these rules (accent combining rules) are stored in advance in the storage unit as data. The text analysis unit 151 stores the accent type of the target text according to the accent combination rule. Furthermore, when the target text is Japanese, there is a characteristic tendency that an accent is attached to each semantic or grammatical unit, so these rules (phrase rules) are stored in advance in the storage unit as data, In accordance with this phrase rule, the text analysis unit 151 determines a connection of several groups with one accent as an exhalation paragraph (acquisition of prosodic information). In addition, the pose position can be included in the prosodic information.

The outline of the morpheme analysis described here is an example of morpheme analysis and is not intended to exclude other morpheme analysis methods. In the speech synthesizer / method according to the present invention, various morphological analyzes can be used, and these are achieved by conventional methods (see, for example, References 3 and 4), and thus the details thereof are omitted.
(Reference 3) Japanese Patent No. 3379634 (Reference 4) Japanese Patent No. 3518340

<Prosody Generation Unit 152 and Prosody Model Storage Unit 153>
Next, the prosody generation unit 152 receives the prosody information output from the text analysis unit 151 as an input, refers to the prosody model storage unit 153, estimates the prosodic parameters that are information about the prosody, and outputs this (s152). .

  Examples of prosodic parameters include the F0 pattern (fundamental frequency pattern), the average value of the F0 pattern, the slope of the F0 pattern, the phoneme duration (phoneme utterance length), and the like. For example, the phoneme duration can be appropriately set in accordance with the phoneme position in the exhalation paragraph, the utterance speed, the phoneme environment before and after the phoneme, etc., which are regulated in advance. Also, the F0 pattern can be obtained by a so-called Fujisaki model. The prosody model such as the Fujisaki model is stored in the prosody model storage unit 153 in advance. Note that “estimation” means that information necessary for speech synthesis (F0 pattern, phoneme duration, etc.) is determined to be a specific one.

The outline of prosodic parameter acquisition described here is merely an example, and is not intended to exclude other methods. In the speech synthesizer / method according to the present invention, the prosodic parameters can be acquired by using a conventional prosodic parameter acquiring method, and the details thereof are omitted. Refer to References 5 and 6 for the acquisition of the F0 pattern, and refer to References 7 and 8 for the phoneme duration, for example.
(Reference 5) Japanese Patent No. 3420964 (Reference 6) Japanese Patent No. 3344487 (Reference 7) Yoshinobu Kaiki, Kazuya Takeda, Yoshinori Mozaka, “Control of vowel duration using language information”, electronic information Journal of Communications Society Vol. J75-A, No.3, pp.467-473, 1992.
(Reference 8) MDRiley, "Tree-based modeling for speech synthesis", In G. Bailly, C. Benoit, and TR Sawallis, editors, Talking Machines: Theories, Models, and Designs, pages 265-273. Elsevier, 1992 .

<Phoneme context conversion unit 154>
The phoneme context conversion unit 154 obtains the phoneme context by using the reading information output by the text analysis unit 151 as input, and outputs it (s154).

  The phoneme context is an arrangement of phonemes. For example, if the reading information is “Kyowa Halle”, the phoneme context is “/ k // y // O // W // A // H // A /”. / R // E / ". The phoneme context conversion unit 154 converts the reading information into a phoneme string (acquisition of phoneme context) with reference to a kana phoneme conversion model or the like (stored in the storage unit as necessary).

<Speech segment candidate search unit 155>
Next, the speech unit candidate search unit 155 uses the phoneme context as an input, and based on the phoneme information, similar speech unit candidates (hereinafter referred to as “speech unit candidates”) that match the phoneme context in synthesis units. The speaker voice database 130 is searched for and output (s155).

Various methods can be adopted as a method for searching for speech element candidates. For example, it can be performed by the method described in Reference 9.
(Reference 9) JP 2009-122381 A

  All speech units whose phoneme information matches a part of the phoneme context are searched from the similar speaker speech database 130 and set as speech unit candidates.

  In the case where the synthesis unit is a phoneme, for example, the phoneme context is “/ k // y // O // W // A // H // A // R // E /”. Phonemes "/ k /", "/ y /", "/ O /", "/ W /", "/ A /", "/ H /", "/ A /", "/ R /" , For each “/ E /”, search for all speech units having phoneme information matching the corresponding phoneme from the similar speaker speech database 130, and identify these speech units as speech unit candidates for each phoneme of the phoneme context. To do. That is, in this example, one or a plurality of speech segment candidates are determined for each phoneme in the phoneme context.

<Element selection unit 156>
The unit selection unit 156 calculates, as an overall cost, the degree of matching between the target text of the synthesis unit and the speech unit candidate using at least the speaker similarity L _s corresponding to the similar speaker identifier of each speech unit candidate. Then, the speech unit candidates when the total cost is the best are selected as the selected speech units, respectively (s156).

  For example, using speech unit candidates as input, calculate the sub-cost of each speech unit candidate using one or more sub-cost functions, calculate the total cost consisting of sub-costs, and connect the waveform using the total cost The selected speech segment to be used for is identified and output.

  For example, each sub-cost represents the degree of matching between the prosodic parameter obtained from the target text and the prosodic parameter of the speech segment candidate.

Although it is a sub-cost calculation method, various methods can be arbitrarily adopted. As an example, it can be calculated using a sub-cost function as shown in Reference 10.
(Reference 10) “Waveform Selection Method Considering Spectral Continuity in Waveform Editing Type Synthesis Method”, Proc. Of the Acoustical Society of Japan, 2-6-10, pp.239-240, 1990/9

The sub-cost function corresponding to the F0 pattern average value Vp of the prosody parameters of the speech unit candidate and the F0 pattern average value Vs of the speech unit candidate of the synthesis unit of the target text is
C ₁ (Vp, Vs) = (Vp−Vs) ² (11)
It is.

The sub cost function corresponding to the slope Fp of the F0 pattern of the prosodic parameter of the speech unit candidate and the slope Fs of the F0 pattern of the speech unit candidate of the synthesis unit of the target text is:
C ₂ (Fp, Fs) = (Fp−Fs) ² (12)
It is.

The sub-cost function corresponding to the phoneme duration Tp of the prosodic parameter of the speech unit candidate and the phoneme duration Ts of the speech unit candidate of the synthesis unit of the target text is:
C ₃ (Tp, Ts) = (Tp−Ts) ² (13)
It is.

When using speaker similarity as one of the sub-cost functions, the sub-cost function is
C ₄ (L ₁ , L _s ) = (L ₁ −L _s ) ² (14)
It is. Note that L ₁ is the highest speaker similarity among the S speaker similarity L _{s in} the similar speaker selection unit 111, and L _s is a similar speaker of a speech unit candidate to be subjected to sub cost calculation. This is the speaker similarity of s (s = 1, 2,..., S). When the speech unit that is the target of sub-cost calculation is the speaker with the highest speaker similarity, C ₄ (L ₁ , L _s ) is 0, and the speaker with the lower speaker similarity has C (L ₁ , L _s ) is a large value. Note that the speaker similarity is acquired from the speaker similarity storage unit 140 using the similar speaker identifier of the speech unit candidate as a key.

Next, the segment selection unit 156 calculates an overall cost including these sub costs. Various methods can be adopted for the total cost. As an example, the total cost Q is calculated by multiplying each sub-cost value by the weight w _k as follows.

However, K is the number of sub-costs (for example, K = 4). The total cost Q is obtained for one or a plurality of speech segment candidates for each synthesis unit of the target text. However, each of the weights w _k is a positive value and can be set arbitrarily. In the above example, each sub-cost C _k takes a value of 0 or more. Since the speech unit candidates that are superior to the phoneme context have values closer to 0, the total cost Q takes a value of 0 or more. As the cost Q is closer to 0, it can be determined as a better segment candidate.

  Then, the segment selection unit 156 specifies one speech segment candidate for each synthesis unit of the target text so that the total sum of the total costs Q is the best (in this example, the minimum), and the phoneme context of the target text A series of speech segments corresponding to is determined. The identified speech element candidate is the selected speech element.

<Unit Connection Unit 157>
Finally, the unit connection unit 157 reads the partial speech data corresponding to the selected speech unit from the similar speaker speech database 130, and connects the partial speech data according to the sequence of the speech units to generate the synthesized speech. Generate (s157).

The partial speech data corresponding to the selected speech segment may be simply connected in order of time, or the waveform may be connected by interpolating between different partial speech waveform data in terms of time or frequency (see Reference 11). ).
(Reference 11) Japanese Patent Laid-Open No. 7-072897

<Effect>
With such a configuration, the voice data of a speaker that is more similar to the target speaker can be easily used in speech synthesis, and the similarity of the synthesized speech to the target speaker can be improved.

  In addition, the selection of similar speakers performs speech synthesis because sufficient performance can be obtained with a small amount of speech data of several seconds to several tens of seconds compared to learning of conversion rules from the average voice model of Non-Patent Document 1. Therefore, it is possible to reduce the amount of voice data required for the target speaker. In addition, since a large amount of voice data of the target speaker is not necessary, the voice recording restraint time can be made extremely short.

  Furthermore, by integrating the voice data of a plurality of names, the number of segments of the same context existing in the integrated voice database can be increased, and the naturalness of the synthesized voice can be improved.

<Modification 1>
Only parts different from the first embodiment will be described.
In the unit selection unit 156 ′ in FIG. 10, the degree of matching between the target text in the synthesis unit and the speech unit candidate using at least the speaker similarity L _s corresponding to the similar speaker identifier of each speech unit candidate. Is calculated as an overall cost as follows, and speech unit candidates when the total cost is the best are selected as selected speech units (s156 ′ in FIG. 11).

The weight w _s is 1 for the speaker with the highest speaker similarity, and for other similar speakers, if the weight w _s becomes larger than 1 according to the speaker similarity, another calculation formula is used. You can ask for it. Further, this weight may be used for the total cost as a whole or as a weight for an individual sub-cost function (for example, it is used only for the sub-cost function C ₁ of the F0 average value).

In this case, the speaker similarity L _s may or may not be used as a sub-cost.
By adopting such a configuration, it is possible to obtain the same effect as that of the first embodiment, and it is possible to select a flexible segment.

<Other variations>
The speech synthesizer 100 does not necessarily include the multi-speaker speech database construction unit 101. In that case, the multi-speaker voice database 103 constituted by another device or the like may be obtained from a recording medium or downloaded via a communication line and stored. Furthermore, the speech synthesizer 100 may not include the multi-speaker speech database 103 and the similar speaker speech database construction unit 110. In that case, the similar speaker voice database 130 constituted by another device or the like and the speaker similarity of the similar speaker may be obtained from a recording medium or downloaded via a communication line and stored. The speech synthesizer 150 can perform speech synthesis similar to that of the first embodiment even using the similar speaker speech database 130 and speaker similarity obtained in this way.

  Voice data is not voice waveform data, but is obtained as a result of performing signal processing on voice waveform data, resulting in voice feature values (pitch parameters (basic frequency, etc.), spectral parameters (cepstrum, mel cepstrum, etc.)) But you can. In this case, it is possible to omit the process of acquiring the spectrum parameters (cepstrum, mel cepstrum, etc.) from the speech waveform data in the similar speaker selection unit 111. Also, the segment connecting unit 157 generates and outputs speech waveform data using the connected partial speech data (speech feature amount).

  The speech segment may include accent information (accent type, accent phrase length), part of speech information, and the like. The multi-speaker voice database 103 may include speaker information (gender, age, hometown), recording environment (microphone type, recording booth information), and the like. By adding such information to the speech data of the target speaker, it is possible to generate synthesized speech with higher accuracy.

  The method for obtaining the speaker similarity in the similar speaker selection unit 111 may be another method. For example, the similarity may be obtained directly from speech waveform data without using spectral parameters.

  It is not always necessary to store speech waveform data in the similar speaker speech database. It may be a database consisting only of speech segments. In this case, the speech waveform data is obtained from the multi-speaker speech database 103 using speech units in the similar speaker speech database as keys.

In this embodiment, the average value of the F0 pattern, the slope of the F0 pattern, the phoneme duration, and the speaker similarity are used as the sub-costs, but at least the speaker similarity may be used. The speech unit of the speaker determined to be closest to the target speaker is easily selected, and the similarity of the synthesized speech to the target speaker is improved. Further, the sub cost may be obtained from other information. For example, the segment selection unit 156 may calculate the sub-cost corresponding to the reading with the phoneme context as an input (indicated by a broken line in FIG. 10). The sub-cost function corresponding to the reading is
C (j) = 1 / e ^j (23)
It is. Here, j is the number of phonemes in which the phoneme context of the target text matches the phoneme context of the speech unit candidate of the synthesis unit.

<Second embodiment>
A speech synthesizer 200 according to the second embodiment will be described with reference to FIG. Only parts different from the first embodiment will be described. The speech synthesizer 200 includes a multi-speaker speech database construction unit 101, a multi-speaker speech database 103, a similar speaker speech database construction unit 210, a similar speaker speech database 130, and a speaker similarity storage unit 140. The configuration of the similar speaker voice database construction unit 210 is different from that of the first embodiment.

[point]
In the first embodiment, when the speaker integration unit 115 integrates the extracted S-named speakers, if there is a large difference in speech feature between the speakers, the quality of the synthesized speech may be degraded. . Therefore, in the second embodiment, the speaker integration unit 215 uses the target speaker's voice feature value to normalize the extracted speaker's voice feature value to the target speaker's feature value. This reduces the difference in the voice feature amount of the voice and prevents the quality degradation of the synthesized voice.

<Similar speaker voice database construction unit 210>
The similar speaker voice database construction unit 210 will be described with reference to FIGS. 12 and 13.
The similar speaker speech database construction unit 210 constructs a similar speaker speech database composed of speech data similar to the speech data of the target speaker using a plurality of speech data (s210). As shown in FIG. 12, the similar speaker voice database construction unit 210 includes a similar speaker selection unit 111, a speech unit assignment unit 212, a speaker conversion rule learning unit 213, a speaker unit conversion unit 214, and a speaker. A person integration unit 215.

<Voice segment giving unit 212>
The speech segment adding unit 212 adds a speech segment to the input target speaker's speech data (s212). The information given as the speech unit is the same as the speech unit given by the multi-speaker speech database construction unit 101. However, the phoneme number and the start time and end time of each phoneme are for the target speaker's voice data. This speech segment may be given manually or automatically from voice data and utterance text.

<Speaker conversion rule learning unit 213>
The speaker conversion rule learning unit 213 converts the voice data of each similar speaker into voice data having the voice feature amount of the target speaker, using the voice data of the target speaker and the voice data of a plurality of selected similar speakers. The speaker conversion rule to be learned is learned (s213). For example, the speaker conversion rule is learned by the method described in Reference 12.
(Reference 12) MJ F Gales and PC Woodland, “Mean and variance adaptation within the MLLR framework,” Computer Speech and Language, 1996, vol.10, pp.249-264

  First, a statistical model (for example, Hidden Markov Model (HMM) is obtained from speech feature amounts obtained from speech data of S similar speakers selected by the similar speaker selection unit 111 and speech units corresponding to the speech data. ) Or GMM) (s213b in FIG. 14). Note that a statistical model of all speakers (for N names) may be obtained in advance using voice data in the multi-speaker voice database 103 and stored in the multi-speaker voice database.

Next, the CMLLR transformation matrix W (which converts the spectral parameters of the similar speaker s into the spectral parameters of the target speaker using all the spectral parameters and speech segments of the target speaker and the statistical model of the similar speaker s). The speaker conversion rule φ _s ) is learned (s213c). The speaker conversion rule φ _s is output to the speaker unit converter 214. The transformation matrix W is obtained by solving the following equation.

Here, (·) ′ represents a transposed matrix of •. x _t is the spectral parameter of the target speaker at time t, and μ _g and U _g ⁻¹ are the mean and the inverse of the covariance matrix of the state g of the HMM, respectively. Further, γ _g (t) is a probability that x _t is output in the state g, and is obtained from x _t and μ _g , U _g ⁻¹ . The transformation matrix W can be obtained for each state of the HMM, but in the present embodiment, one transformation matrix is obtained by sharing all the states. Note that the speaker conversion rule φ _s is learned for each similar speaker s.

<Speaker unit converter 214>
The speaker unit converter 214 converts the voice data of each similar speaker according to the speaker conversion rule φ _s (s214). For example, the similarity obtained by converting the spectral parameter x _{s, t} at time t in the speech database of the similar speaker s using the conversion matrix W (speaker conversion rule φ _s ) into the target speaker characteristics. Obtain spectral parameters x ￣ _{s, t} of speaker s.

This process is performed on the voice data of all similar speakers at all times.
Hereinafter, as an example of conversion of the F0 parameter, a linear conversion method performed by the following processing will be described. The average μ _h and variance ν _h are obtained from all logarithm F0 values obtained from the target speaker's voice data. Further, the average μ _s and the variance ν _s are obtained from all logarithmic F0 values obtained from the speech data of the similar speaker s. And the logarithm F0 value after conversion of the similar speaker s is calculated | required with the following formula | equation.

Here, z _t is the t-th logarithmic F0 values of similar speakers s before conversion, the y _t is the t-th logarithmic F0 values of similar speakers s after conversion. T is the total number of frames of the logarithm F0 of the similar speaker s, and t = 1, 2,.

  The same processing (s213b to s214) is performed on the speech data of all similar speakers (s213a, s213d, and s213e), and the speech data subjected to the conversion processing for each speaker is output to the speaker integration unit 215. .

<Speaker integration unit 215>
The speaker integration unit 215 integrates not the voice data itself of the similar speaker s but the voice data converted using the speaker conversion rule φ _s to construct a similar speaker voice database (s215). The construction method is the same as in the first embodiment.

<Effect>
By adopting such a configuration, compared to the first embodiment, a difference in speech feature amount between speakers is reduced, and quality degradation of synthesized speech is prevented.

<Modification>
The speaker conversion rule learning unit 213 uses only the target speaker's voice data and a plurality of selected similar speaker's voice data, and converts each similar speaker's voice data into the target speaker's voice data. You may learn the rules. In that case, the speech element providing unit 212 may not be provided.

<Third embodiment>
A speech synthesis apparatus 300 according to the third embodiment will be described with reference to FIG. Only parts different from the second embodiment will be described. The speech synthesizer 300 includes a multi-speaker speech database construction unit 101, a multi-speaker speech database 103, a similar speaker speech database construction unit 310, a similar speaker speech database 130, and a speaker similarity storage unit 140. The configuration of the similar speaker voice database construction unit 310 is different from that of the first embodiment.

[point]
In the first embodiment and the second embodiment, the similar speaker voice database 130 is generated from a plurality of speakers similar to the target speaker, but in this embodiment, the voice data obtained by the speaker integration unit 215 is used. As a basis, it is possible to generate a model closer to the target speaker by converting speech data using a model adaptation technique for each speech synthesis unit.

<Similar speaker voice database construction unit 310>
The similar speaker voice database construction unit 310 will be described with reference to FIGS. 15 and 16.
The similar speaker speech database construction unit 310 constructs a similar speaker speech database composed of speech data similar to the speech data of the target speaker using a plurality of speech data (s310). As shown in FIG. 15, the similar speaker voice database construction unit 310 includes a similar speaker selection unit 111, a speech unit assignment unit 212, a speaker conversion rule learning unit 213, a speaker unit conversion unit 214, and speaker integration. A unit 215, a synthesis unit conversion rule learning unit 317, and a synthesis unit conversion unit 318.

<Composition Unit Conversion Rule Learning Unit 317>
The synthesis unit conversion rule learning unit 317 uses the target speaker's speech data and the partial speech data of the first similar speaker speech database 130 to obtain the partial speech data of each similar speaker for each of the same states. A synthesis unit conversion rule for converting into partial speech data having speech features is learned (s317). For example, the composition unit conversion rule is learned by the method described in Non-Patent Document 1. Note that the voice data in the first similar speaker voice database 130 is configured by the same method as the voice data in the similar speaker voice database 130 of the first embodiment and the second embodiment.

  First, the synthesis unit conversion rule learning unit 317 uses the first similar speaker voice database 130 to generate a statistical model (HMM) for each partial voice data having the same state, using the spectrum parameter and the voice element. To learn. This statistical model is an average statistical model of similar speakers.

  Next, the synthesis unit conversion rule learning unit 317 uses the first similar speaker voice database 130 and the statistical parameters learned from the spectral parameters obtained from the target speaker's partial voice data and the speech segments for the partial voice data. From the above, a conversion matrix for converting spectral parameters and speech units obtained from average speech data of all similar speakers into spectral parameters and speech units of the target speaker is learned. Note that a transformation matrix is learned for each identical state. The learning method is the same as in the second embodiment. This MLLR transformation matrix is acquired as a synthesis unit transformation rule.

  The synthesis unit conversion rule is learned for each same state. However, if the target speaker has a small amount of speech data, the synthesis unit conversion rule for speech data corresponding to all states cannot be learned. In the present embodiment, a binary tree in which each leaf node is each voice is created, and a synthesis unit conversion rule is learned at the lowest node where the data amount of the target speaker is a certain value or more. Thereby, even when the voice data of the target speaker is small, the synthesis unit conversion rule can be learned for all states.

<Composition Unit Conversion Unit 318>
The synthesis unit conversion unit 318 converts the speech data of each similar speaker according to the synthesis unit conversion rule for each synthesis unit (s318). The learned synthesis unit conversion rule is applied to the first similar speaker voice database 130 to obtain the second similar speaker voice database 330. Note that the method described in Non-Patent Document 1 can be used as a method for applying the composition unit conversion rule.
By adopting such a configuration, it is possible to generate synthesized speech that is closer to the target speaker than in the second embodiment.

<Program and storage medium>
The speech synthesizer described above can also be functioned by a computer. In this case, each process of a program for causing a computer to function as a target device (a device having the functional configuration shown in the drawings in various embodiments) or a processing procedure (shown in each embodiment) is processed by the computer. A program to be executed by the computer may be downloaded from a recording medium such as a CD-ROM, a magnetic disk, or a semiconductor storage device or via a communication line into the computer, and the program may be executed.

<Others>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

100, 200, 300 Speech synthesis apparatus 101 Multi-speaker speech database construction unit 103 Multi-speaker speech database 110, 210, 310 Similar speaker speech database construction unit 111 Similar speaker selection unit 111a Mixed normal distribution learning unit 111b Multi-speaker Mixed normal distribution storage unit 111c Speaker similarity calculation unit 111d Similar speaker extraction unit 115, 215 Speaker integration unit 130 Similar speaker voice database 130 First similar speaker voice database 140 Speaker similarity storage unit 150 Speech synthesis unit 151 Text Analysis Unit 152 Prosody Generation Unit 153 Prosody Model Storage Unit 154 Phoneme Context Conversion Unit 155 Speech Segment Candidate Search Unit 156 Segment Selection Unit 156 Segment Selection Unit 157 Segment Connection Unit 212 Speech Segment Assignment Unit 213 Speaker Conversion Rule learning unit 214 Speaker unit conversion unit 317 Synthesis unit conversion rule習部 318 Synthesis unit conversion unit 330 second similar-speaker speech database

Claims (9)

A speech synthesis method for generating synthesized speech corresponding to a target text and having speech characteristics of a target speaker,
Speaker similarity is used as an index indicating whether or not two voice data are similar, and using the voice data of a plurality of speakers, the speaker similarity between the voice data of each speaker and the voice data of the target speaker A similar speaker selection step for selecting a plurality of voice data having a high speaker similarity,
A combination of partial speech data suitable for combining synthesized speech data and assembling a synthesized speech, and information given to the partial speech data, which indicates the speaker who issued the partial speech data A speaker integration step of constructing a similar speaker speech database comprising at least a speech unit indicating a similar speaker identifier and phoneme information indicating a phoneme of the partial speech data;
A text analysis step of analyzing the target text and obtaining reading information of the target text;
A phoneme context conversion step for converting the reading information into a phoneme context that is a sequence of phonemes;
Based on the phoneme information, a speech unit candidate search step for searching the similar speaker speech database for speech unit candidates that match the phoneme context in a synthesis unit;
Using at least the speaker similarity corresponding to the similar speaker identifier of each of the speech unit candidates, the degree of matching between the target text of the synthesis unit and the speech unit candidate is calculated as a total cost, and this total cost is A unit selection step for selecting the speech unit candidate at the best time as a selected speech unit;
A segment connection step of reading partial speech data corresponding to the selected speech segment from the similar speaker speech database and connecting the partial speech data to obtain the synthesized speech;
Speech synthesis method. A speech synthesis method for generating synthesized speech corresponding to a target text and having speech characteristics of a target speaker,
A synthesis unit suitable for assembling a plurality of speech data having a high speaker similarity with the target speaker's speech data as an index indicating whether or not the two speech data are similar Divided partial voice data, information given to the partial voice data, a similar speaker identifier indicating the speaker who has issued the partial voice data, and phoneme information indicating the utterance phoneme of the partial voice data; A similar speaker speech database comprising at least speech segments indicating
A text analysis step of analyzing the target text and obtaining reading information of the target text;
A phoneme context conversion step for converting the reading information into a phoneme context that is a sequence of phonemes;
Based on the phoneme information, a speech unit candidate search step for searching the similar speaker speech database for speech unit candidates that match the phoneme context in a synthesis unit;
Using at least the speaker similarity corresponding to the similar speaker identifier of each of the speech unit candidates, the degree of matching between the target text of the synthesis unit and the speech unit candidate is calculated as a total cost, and this total cost is A unit selection step for selecting the speech unit candidate at the best time as a selected speech unit;
A segment connection step of reading partial speech data corresponding to the selected speech segment from the similar speaker speech database and connecting the partial speech data to obtain the synthesized speech;
Speech synthesis method. The speech synthesis method according to claim 1 or 2,
The speaker similarity of the speech data of each similar speaker s is L _s , the speaker similarity of the speaker with the highest speaker similarity is L ₁ ,
In the segment selection step,
The sum of K sub-costs C _k is _defined as the total cost Q, and at least one of the sub-costs,
C (L ₁ , L _s ) = (L ₁ −L _s ) ²
Use
Speech synthesis method. The speech synthesis method according to claim 1 or 2,
The speaker similarity of the speech data of each similar speaker s is L _s , the speaker similarity of the speaker with the highest speaker similarity is L ₁ ,
In the segment selection step,
The sum of K sub-costs C _k is the total cost Q, the weight for each sub-cost is w _k , and the total cost is

Asking,
Speech synthesis method. The speech synthesis method according to claim 1,
A speaker who learns speaker conversion rules for converting each similar speaker's voice data into voice data having the target speaker's voice characteristics using the target speaker's voice data and voice data of a plurality of selected similar speakers A conversion rule learning step;
A speaker unit conversion step of converting voice data of each similar speaker according to the speaker conversion rule, and
The speaker integration step integrates speech data obtained by converting speech data of a plurality of selected similar speakers, and includes a similar speaker speech database including the partial speech data and the speech segments of the partial speech data. Build,
Speech synthesis method. The speech synthesis method according to claim 1,
A speaker who learns speaker conversion rules for converting each similar speaker's voice data into voice data having the target speaker's voice characteristics using the target speaker's voice data and voice data of a plurality of selected similar speakers A conversion rule learning step;
A speaker unit conversion step of converting voice data of each similar speaker according to the speaker conversion rule, and
The speaker integration step integrates speech data obtained by converting speech data of a plurality of selected similar speakers, and includes a similar speaker speech database including the partial speech data and the speech segments of the partial speech data. Build
Synthetic unit conversion for converting the voice data of each similar speaker into voice data having the voice characteristics of the target speaker for each same state using the target speaker's voice data and the partial voice data of the similar speaker's voice database A synthesis unit conversion rule learning step for learning a rule;
A synthesis unit conversion step of converting the partial speech data of each similar speaker according to a synthesis unit conversion rule for each synthesis unit;
Speech synthesis method. A speech synthesizer that generates synthesized speech corresponding to a target text and having speech characteristics of a target speaker,
Speaker similarity is used as an index indicating whether or not two voice data are similar, and using the voice data of a plurality of speakers, the speaker similarity between the voice data of each speaker and the voice data of the target speaker A similar speaker selection unit that selects a plurality of voice data having high speaker similarity,
A combination of partial speech data suitable for combining synthesized speech data and assembling a synthesized speech, and information given to the partial speech data, which indicates the speaker who issued the partial speech data A speaker integration unit for constructing a similar speaker voice database including a speech element indicating at least a similar speaker identifier and phoneme information indicating a phoneme of the partial voice data;
A speaker similarity storage unit for storing the similar speaker identifier and the speaker similarity corresponding to the similar speaker identifier;
A text analysis unit that analyzes the target text and obtains reading information of the target text;
A phoneme context conversion unit that converts the reading information into a phoneme context that is a sequence of phonemes;
Based on the phoneme information, a speech unit candidate search unit that searches the similar speaker speech database for speech unit candidates that match the phoneme context in a synthesis unit;
Using at least the speaker similarity corresponding to the similar speaker identifier of each of the speech unit candidates, the degree of matching between the target text of the synthesis unit and the speech unit candidate is calculated as a total cost, and this total cost is A speech segment selection unit that selects the speech segment candidate at the best time as a selected speech segment;
A segment connection unit that reads partial speech data corresponding to the selected speech segment from the similar speaker speech database and connects the partial speech data to obtain the synthesized speech;
Speech synthesizer. A speech synthesizer that generates synthesized speech corresponding to a target text and having speech characteristics of a target speaker,
The partial speech data of an appropriate synthesis unit for assembling the synthesized speech, the information given to the partial speech data, the similar speaker identifier indicating the speaker who has issued the partial speech data, and the partial speech data A similar speaker speech database comprising at least speech segments indicating phoneme information indicating utterance phonemes;
A speaker similarity storage unit for storing the similar speaker identifier and the speaker similarity corresponding to the similar speaker identifier;
A text analysis unit that analyzes the target text and obtains reading information of the target text;
A phoneme context conversion unit that converts the reading information into a phoneme context that is a sequence of phonemes;
Based on the phoneme information, a speech unit candidate search unit that searches the similar speaker speech database for speech unit candidates that match the phoneme context in a synthesis unit;
Using at least the speaker similarity corresponding to the similar speaker identifier of each of the speech unit candidates, the degree of matching between the target text of the synthesis unit and the speech unit candidate is calculated as a total cost, and this total cost is A speech segment selection unit that selects the speech segment candidate at the best time as a selected speech segment;
A segment connection unit that reads partial speech data corresponding to the selected speech segment from the similar speaker speech database and connects the partial speech data to obtain the synthesized speech;
Speech synthesizer.   A speech synthesis program for causing a computer to execute the speech synthesis method according to claim 1.

Images