JPH1049193A - Natural speech voice waveform signal connecting voice synthesizer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the voice synthesizer in which a rhythm control rule is not used, no signal process is executed, an arbitrary phoneme string is converted into uttered voices and a voice quality close to natural is obtained. SOLUTION: A weighted coefficient learning section 11 computes the acoustic distance in second acoustic feature parameters between one target phoneme of same phoneme kinds and other phoneme candidates based on extracted first acoustic feature parameters and determines the weighted coefficient vectors which represent the degree of contribution in the second acoustic feature parameters by conducting a statistical analysis. A voice unit selecting section 12 retrieves the phoneme candidate columns which make the target cost, representing the approximate cost between a target phoneme and a phoneme candidate, and the cost including the connecting cost, representing the approximate cost between two phoneme candidates to be connected next to each other, become a minimum against the inputted phoneme columns and outputs the retrieved information. A voice synthesizing section 13 successively reads the voice segment of the voice waveform signals corresponding to the retrieved information, connects them and conducts a voice synthesis.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a spontaneously uttered speech waveform signal connection type speech synthesizer for synthesizing an arbitrary phoneme string by connecting speech segments of a naturally uttered speech waveform signal.

[0002]

2. Description of the Related Art FIG. 2 is a block diagram of a conventional speech synthesizer. As shown in FIG. 2, for example, LPC analysis is performed on the signal waveform data of the learning speaker to extract feature parameters including 16th-order cepstrum coefficients. The extracted feature parameters are stored in a feature parameter memory 62 which is a buffer memory, and then input from the memory 62 to the parameter time series generation unit 52. Next, the parameter time series generation unit 52 performs signal processing such as time normalization and generation processing of a parameter time series using the prosody control rules in the memory 63 based on the extracted feature parameters, A parameter time series such as a 16th-order cepstrum coefficient required for speech synthesis is generated and output to the speech synthesis unit 53.

The speech synthesizer 53 is a known speech synthesizer, and includes a pulse generator 53a for generating a voiced sound,
A noise generator 53b for generating an unvoiced sound, and a filter 53c capable of changing a filter coefficient, wherein a voiced sound generated by the pulse generator 53a and a noise generator 53b Switch to the unvoiced sound generated by the
Further, by changing a filter coefficient corresponding to a transfer function of the filter 53c, a voice-synthesized voice signal is generated, and the voice is output from the speaker 54.

[0004]

However, in the conventional speech synthesizer, signal processing using prosody control rules is required, and speech synthesis is performed based on the processed characteristic parameters. However, there was a problem that the voice quality was extremely poor.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above problems and to convert an arbitrary phoneme sequence into a uttered voice without using prosody control rules and without performing signal processing. To provide a speech synthesizer capable of obtaining a voice quality close to nature.

[0006]

According to the present invention, a spontaneously uttered speech waveform signal connection type speech synthesizing apparatus according to a first aspect of the present invention comprises: first storage means for storing a speech segment of a naturally uttered speech waveform signal; Based on the audio segment of the audio waveform signal stored by the first storage means and the phoneme string corresponding to the audio waveform signal, index information for each phoneme in the audio waveform signal and indicated by the index information Voice analysis means for extracting and outputting a first acoustic feature parameter for each phoneme and a prosodic feature parameter for each phoneme indicated by the index information; index information output from the voice analysis means; Second storage means for storing the first acoustic feature parameter and the prosodic feature parameter; and first acoustic feature parameter stored by the second storage means. The acoustic distance in the second acoustic feature parameter between one target phoneme of the same phoneme type and the other phoneme candidates is calculated based on the data and the prosodic feature parameter. By performing a predetermined statistical analysis on each phoneme candidate for each of the second acoustic feature parameters based on the distance, each target representing the degree of contribution of the second acoustic feature parameter to each phoneme candidate is obtained. Weight coefficient learning means for determining a weight coefficient vector for each phoneme; third storage means for storing a weight coefficient vector for each target phoneme in the second acoustic feature parameter determined by the weight coefficient learning means; The weight coefficient vector for each target phoneme stored by the third storage means, and the prosodic feature parameter stored by the second storage means. Based on the phoneme sequence of the input natural utterance sentence, a target cost representing an approximate cost between a target phoneme and a phoneme candidate and an approximate cost between two phoneme candidates to be connected adjacently are represented. A speech unit selection unit that searches for a combination of phoneme candidates that minimizes the cost including the connection cost, and outputs index information of the retrieved combination of phoneme candidates, and index information output from the speech unit selection unit. The audio segment of the audio waveform signal corresponding to the index information is
And a speech synthesizing means for synthesizing and outputting a speech corresponding to the input phoneme sequence by sequentially reading out from the storage means and outputting the combined speech.

According to a second aspect of the present invention, in the voice synthesizing apparatus according to the first aspect, the voice analyzing means generates a phoneme sequence corresponding to the voice waveform signal based on an input voice waveform signal. It is characterized by comprising a phoneme predicting means for predicting.

[0008] Furthermore, the voice synthesizing device according to claim 3 is
3. The speech synthesizer according to claim 1, wherein the weighting factor learning unit is configured to calculate the weighting factor based on the calculated acoustic distance.
After extracting the best plurality of N1 phoneme candidates, a linear regression analysis is performed on each of the second acoustic feature parameters, so that the degree of contribution of each of the phoneme candidates in the second acoustic feature parameter is determined. It is characterized in that a weight coefficient vector for each target phoneme to be represented is determined.

Further, the speech synthesizing apparatus according to claim 4 is
3. The speech synthesizer according to claim 1, wherein the weighting factor learning unit is configured to calculate the weighting factor based on the calculated acoustic distance.
After extracting the best top N1 phoneme candidates, a statistical analysis using a predetermined neural network is performed on each of the second acoustic feature parameters, thereby obtaining the second phoneme candidates for each phoneme candidate. And determining a weighting coefficient vector for each target phoneme, which represents the degree of contribution in the acoustic feature parameter.

According to a fifth aspect of the present invention, in the voice synthesizing apparatus according to any one of the first to fourth aspects, the voice unit selecting means includes the target cost and the connection cost. After extracting the top N2 phoneme candidates with the best cost, a combination of phoneme candidates with the lowest cost is searched.

Further, the speech synthesizer according to claim 6 is
The speech synthesizer according to any one of claims 1 to 5, wherein the first acoustic feature parameter includes a cepstrum coefficient, a delta cepstrum coefficient, and a phoneme label. Further, the speech synthesizer according to claim 7 is the speech synthesizer according to one of claims 1 to 5, wherein the first acoustic feature parameter is:
It is characterized by including a formant parameter and a vocal tract sound source parameter.

According to an eighth aspect of the present invention, there is provided the speech synthesizer according to any one of the first to seventh aspects, wherein the prosodic feature parameters include a phoneme time length and a speech fundamental frequency F _0. And power.

Further, the speech synthesizing device according to claim 9 is
The speech synthesizer according to any one of claims 1 to 8, wherein the second acoustic feature parameter includes a cepstrum distance.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a spontaneously uttered speech waveform signal connection type speech synthesis apparatus according to an embodiment of the present invention. For example, in the conventional speech synthesizer shown in FIG. 2, from the extraction of the text corresponding to the input uttered speech to the generation of the speech waveform signal is performed as a series of processes, in the present embodiment, the speech is largely classified. For example, it is classified into the following four processing units. (1) voice analysis of voice waveform signal data in the voice waveform signal database in the voice waveform signal database memory 21;
Specifically, the speech analysis unit 10 performs processing including generation of a phoneme symbol sequence, alignment of phonemes, and extraction of feature parameters. (2) A weighting factor learning unit 11 that determines while learning the optimal weighting factor. (3) A voice unit selection unit 12 that selects a voice unit based on an input phoneme sequence and outputs index information of voice waveform signal data corresponding to the input phoneme sequence. (4) Based on the index information output from the voice unit selection unit 12, the voice waveform signal database in the voice waveform signal database memory 21 is randomly accessed to reproduce the voice waveform signal of each optimal phoneme candidate. Speaker 14
A voice synthesizer 13 for outputting to

[0015] Specifically, the speech analysis unit 10 determines, for each phoneme in the speech waveform signal, the speech segment of the speech waveform signal of the natural utterance to be inputted and the phoneme sequence corresponding to the speech waveform signal. Index information, a first acoustic feature parameter for each phoneme indicated by the index information,
A first prosodic feature parameter for each phoneme indicated by the index information is extracted and output. The feature parameter memory 30 stores the index information output from the speech analysis unit 10, the first acoustic feature parameter, and the first prosodic feature parameter. Next, based on the first acoustic feature parameter and the prosodic feature parameter stored in the feature parameter memory 30, the weighting factor learning unit 11 generates one target phoneme of the same phoneme type and other phoneme candidates. Of the second acoustic feature parameter between the two, and performs a predetermined statistical analysis on each phoneme candidate for each of the second acoustic feature parameters based on the calculated acoustic distance. By doing so, a weight coefficient vector for each target phoneme, which indicates the degree of contribution of the second acoustic feature parameter to each phoneme candidate, is determined. The weight coefficient vector memory 31 stores a weight coefficient vector for each target phoneme in the second acoustic feature parameter determined by the weight coefficient learning unit 11 and a second given prosody characteristic for each phoneme candidate. A weight coefficient vector for each target phoneme, which represents the degree of contribution in the parameter, is stored. Further, the voice unit selection unit 12
Is the weight coefficient vector for each target phoneme stored in the weight coefficient vector memory 31 and the feature parameter memory 30
A target cost representing an approximate cost between a target phoneme and a phoneme candidate for a phoneme sequence of a natural utterance sentence based on the first prosodic feature parameter stored in
A combination of phoneme candidates that minimizes a cost including a connection cost representing an approximate cost between two phoneme candidates to be connected adjacently is searched, and index information of the searched combination of phoneme candidates is output. Then, the voice synthesizer 13
Is based on the index information output from the audio unit selecting unit 12, by sequentially reading out the audio segments of the audio waveform signal corresponding to the index information from the audio waveform signal database memory 21, connecting them, and outputting them to the speaker 14. The speech synthesizer synthesizes and outputs speech corresponding to the input phoneme string.

Here, the processing of the voice analysis unit 10 must be performed once for a new voice waveform signal database, and the processing of the weight coefficient learning unit 11 generally only needs to be performed once. The optimal weight coefficient obtained by the above can be reused even under different speech synthesis conditions. Further, the processing of the voice unit selection unit 12 and the voice synthesis unit 13 is executed each time the input phoneme sequence to be voice-synthesized changes.

The speech synthesizer according to the present embodiment predicts all necessary feature parameters based on the input of a given level, and obtains a sample closest to the desired speech feature (ie, a speech waveform signal of a phoneme candidate). ) Is selected from the audio waveform signal database in the memory 21. At a minimum, processing is possible if a sequence of phoneme labels is given, but if the basic speech frequency F ₀ and phoneme time length are given in advance, higher quality synthesized speech can be obtained. When only word information is given as an input, it is necessary to predict a phoneme sequence based on a dictionary or a rule such as a phoneme hidden Markov model (hereinafter, a hidden Markov model is referred to as an HMM). If no prosodic feature is given, a standard prosody is generated based on known features of phonemes in various environments in the speech waveform signal database.

In the present embodiment, if at least text data in which the recorded content in the voice waveform signal database memory 21 is written in the orthographic format exists as in the text database in the text database memory 22, any voice waveform signal can be used. Although the database can be used as speech waveform signal data for synthesis, the quality of output speech is greatly affected by the recording state, the balance of phonemes in the speech waveform signal database, and the like, and the speech waveform signal database in the memory 21 is rich. If the content is appropriate, more diverse voices can be synthesized. Conversely, if the voice waveform signal database is poor, the synthesized voice will have a strong sense of discontinuity and will be jerky.

Next, phoneme labeling for natural uttered speech will be described. The choice of speech unit depends on the method of labeling and searching phonemes in the speech waveform signal database. Here, in the preferred embodiment, the speech unit is a phoneme. First, the utterance content of the orthography given to the recorded voice is converted into a phoneme sequence, and further assigned to a voice waveform signal. Extraction of the prosodic feature parameters is performed based on this. The input of the voice analysis unit 10 is voice waveform signal data in the memory 21 accompanied by the phoneme notation in the memory 22, and the output is a feature vector or a feature parameter. This feature vector becomes a basic unit representing a voice sample in the voice waveform signal database, and is used for selecting an optimum voice unit.

In the first stage of the processing of the speech analysis unit 10, the orthographic text to the phoneme symbols for describing how the utterance contents written in the orthography are pronounced in the actual speech waveform signal data. Conversion. Then, in a second stage, to determine the start and end times of each phoneme to measure prosodic and acoustic features,
This is a process of associating each phoneme symbol with a speech waveform signal (hereinafter, this process is referred to as a phoneme alignment process). Further, the third step is to generate a feature vector or feature parameter of each phoneme. The feature vector stores, as essential items, a phoneme label, start time (start position) of the phoneme in each file in the speech waveform signal database in the memory 30, speech fundamental frequency F ₀ , phoneme time length, and power information. And
Information such as stress, accent type, position with respect to the prosodic boundary, and spectrum inclination are stored as options of the feature parameter. The above characteristic parameters are arranged as shown in Table 1 below, for example.

[0021]

[Table 1] 情報 Index information: Index number (for one file Start time (start position) of the phoneme in each file in the audio waveform signal database in the memory 30 ───────────────────────── ────────── First acoustic feature parameter: 12th-order mel-cepstral coefficient 12th-order メ ル -mel-cepstral coefficient Phoneme label Discrimination feature: vowel (vocalic) (+) / non-vocal (non-vocalic) ) (−) Consonant (+) / non-consonantal (−) interrupted (+) / continuity (continuant) (−) deterrent (checked) (+) / Unchecked (-) Rough (strident) (+) / Maturity (mellow) (-) Voiced (+) / Unvoiced (unvo iced) (-) Intensity (compact) (+) / Diffuse (-) Low tone (grave) (+) / High tone (acute) (-) Inflection (flat) (+) / Normal tone (plain) (-) sharp tone (sharp) (+) / normal tone (plain) (-) tension (tense) (+) / relaxation (lax) (-) nasal tone (nasal) ) (+) / Oral (oral) (-) ─────────────────────────────────── 1 Prosodic feature parameters of: phoneme time length voice fundamental frequency F ₀ power ───────────────────────────────────

Alternatively, the first acoustic feature parameters may preferably be formant parameters and vocal tract sound source parameters. The start time (start position), the first acoustic feature parameter, and the first prosodic feature parameter in the index information are stored in the feature parameter memory 30 for each phoneme. Here, the parameter value of (+) or (-) is given to each of the feature parameters of, for example, 12 discrimination features added to the phoneme label for each item. Further, Table 2 shows an example of the characteristic parameter which is the output result of the voice analysis unit 10. Here, in the voice waveform signal database memory 21, for example, an index number is assigned to each paragraph of a plurality of sentences or a file of one sentence, and a file to which one index number is assigned is assigned. By adding the start time of the phoneme measured from the start time in the file to indicate the position of an arbitrary phoneme in the file and the phoneme time length of the phoneme, the speech segment of the speech waveform signal of the phoneme is added. Can be specified.

[0023]

[Table 2] An example of a feature parameter which is an output result of the voice analysis unit 10 Index number X0005 {phoneme time length fundamental frequency power ...} ... ────────────────────── # 120 90 4.0 ... s 175 98 4.7 ... ei 95 102 6.5 ... dh 30 114 4.9 ... ih 75 143 6.9 ... s 150 140 5.7 ... p 87 137 5.1 ... l 34 107 4.9 ... ii 150 98 6. 3 ... z 140 87 5.8 ... # 253 87 4.0 ...…

In Table 2, # indicates a pause. When selecting a speech unit, it is necessary to check in advance how much each acoustic and prosodic feature parameter contributes to each phoneme. In the fourth step, the speech waveform A weighting factor for each feature parameter is determined using all speech samples in the signal database.

In the process of generating a phoneme symbol sequence in the speech analysis unit 10, as described above, in the present embodiment, if at least one of the recorded contents is described in orthography,
Any audio waveform signal database can be used as audio waveform signal data for synthesis. If only word information is given as input, it is necessary to predict phoneme sequences based on dictionaries and rules. Further, in the phoneme alignment processing in the speech analysis unit 10, in the case of a read-aloud speech, each word is often pronounced close to its standard pronunciation, and it is rare to hesitate or stutter. In the case of such speech waveform signal data, phoneme labeling is correctly performed by a simple dictionary search, and learning of a phoneme model of a phoneme HMM for phoneme alignment becomes possible.

In training a phoneme model for phoneme alignment, unlike perfect speech recognition, it is not necessary to completely separate speech waveform signal data for training and speech waveform signal data for testing. Learning can be performed using the signal data. First, a model for another speaker is used as an initial model. Viterbi learning is performed using all voice waveform signal data so that only standard pronunciation or limited pronunciation changes are allowed for all words, and appropriate segmentation is performed. The phonemes are aligned using an algorithm, and the feature parameters are re-estimated. Pauses between words are processed according to the rules for generating pauses between words.
If alignment fails due to a pause in the word, it must be corrected manually.

It is necessary to select what phoneme label is to be used as phoneme notation. If a phoneme set exists for which a well-learned HMM model can be used, it is advantageous to use it. Conversely, if the speech synthesizer has a complete dictionary, a method of completely collating the label of the speech waveform signal database with the dictionary is also effective.
We have a choice for learning the weighting factor, so if the most important criterion is whether the speech synthesizer can match what is predicted by the speech synthesizer later in the speech waveform signal database. good. Subtle differences in pronunciation are automatically identified by the prosodic environment of the pronunciation, so there is no need to manually label phonemes.

As the next stage of the preprocessing, prosodic feature parameters for describing the articulatory features of individual phonemes are extracted. In conventional phonetics, language sounds are classified based on features such as articulation position and articulation style. On the other hand, phonetics that take into account the prosody, such as the First School, place clear articulations and emphasis in order to capture small differences in sound quality arising from differences in prosody context. To distinguish where they are. Although there are various methods for describing these differences, the following two methods are used here. First, in the low-order level, in order to obtain a one-dimensional feature, power, stretch and voice fundamental frequency F ₀ of the phoneme duration, a value obtained by averaging for a phoneme. On the other hand, at a higher level, a method of marking prosody boundaries and emphasis points in consideration of the above differences in prosody features is used. Since these two types of features are closely related to each other, one can predict the other, but both have a strong influence on the characteristics of each phoneme.

Just as there is a degree of freedom in a method of describing a prosodic feature parameter as well as in a method of defining a phoneme set for describing a speech waveform signal database, the method of selecting these is determined by a speech synthesizer. Depends on predictive ability. If the speech waveform signal database is pre-labeled, the task of the speech synthesizer is to properly learn from the internal representation how to perform the actual speech in the speech waveform signal database. On the other hand, if the speech waveform signal database is not labeled with phonemes, it is necessary to consider from what feature parameters the speech synthesizer can predict the most appropriate speech unit. . This study and the learning of the determination of the optimal feature parameter weight are executed by the weight coefficient learning unit 11 which determines the weight while learning the weight coefficient for each feature parameter.

Next, the weight coefficient learning processing executed by the weight coefficient learning section 11 will be described. In order to select the optimal sample for the acoustic and prosodic environment of a given target speech from the speech waveform signal database, first determine which features and how much are contributed by the difference between phonemic and prosodic environments. There is a need. This is because the types of important feature parameters change depending on the characteristics of phonemes. For example, the fundamental voice frequency F ₀ is extremely effective for selecting voiced sounds, but has little effect on selecting unvoiced sounds. Further, the acoustic characteristics of the fricative sound vary depending on the types of the phonemes before and after. In order to select an optimal phoneme, how much weight is given to each feature is automatically determined by an optimal weight determination process, that is, a weight coefficient learning process.

In the process of determining the optimum weighting factor, which is performed by the weighting factor learning unit 11, what is performed first is used when selecting an optimum sample from all corresponding speech samples in the speech waveform signal database. It is to list features. Here, phonemic features such as articulation positions and articulation styles, and prosodic feature parameters such as the preceding phoneme, the phoneme fundamental frequency F ₀ , the phoneme time length, and the power of the succeeding phoneme are used. Specifically, a second prosodic feature parameter, which will be described in detail later, is used. Next, in the second stage, for each phoneme, one voice sample (or a voice waveform signal of a phoneme) is focused on, and another feature parameter is determined in order to determine which feature parameter is important in selecting an optimal candidate. The acoustic distance including the difference of the phoneme time length from all the phoneme samples is obtained, and the top N2 best similar speech samples, that is, the speech segment of the speech waveform signal of the N2 best phoneme candidate is selected.

Further, in the third stage, a linear regression analysis is performed, and a weight coefficient indicating the importance of each feature parameter in various acoustic and prosodic environments is obtained by using the similar speech samples. For example, the following feature parameter (hereinafter, referred to as a second prosodic feature parameter) is used as the prosodic feature parameter in the linear regression analysis processing. (1) a first prosodic feature parameter of a preceding phoneme (hereinafter, referred to as a preceding phoneme) that precedes the phoneme to be processed by one; (2) a succeeding phoneme that follows by only one from the phoneme to be processed ( (Hereinafter, referred to as a succeeding phoneme.) The first prosodic feature parameter of the phoneme label of (3) the phoneme time length of the phoneme; (4) the speech fundamental frequency F _{0 of the} phoneme; F ₀ ; and (6) the fundamental sound frequency F _{0 of the} succeeding phoneme. Here, the preceding phoneme is a phoneme that precedes the phoneme by one, but is not limited to this, and may include a phoneme that precedes by a plurality of phonemes. Further, the succeeding phoneme is a phoneme following only one phoneme from the phoneme, but is not limited thereto.
A phoneme following only a plurality of phonemes may be included. further,
The voice fundamental frequency F ₀ of the following phoneme may be excluded. In the above embodiment, the linear regression analysis is performed to determine the weighting coefficient. However, the present invention is not limited to this. For example, various statistical analyzes such as a statistical analysis using a predetermined neural network are performed. The weighting factor may be used to determine the weighting factor.

Next, a description will be given of the processing of the voice unit selection unit 12 for selecting a natural voice sample. In the conventional speech synthesizer, a phoneme sequence is determined for a target utterance, and a target value of F ₀ and a phoneme time length for prosody control is calculated. On the other hand, in the present embodiment, only the prosody is calculated in order to appropriately select the optimum speech sample, but the prosody is not directly controlled.

FIG. 3 shows that the input of the processing of the speech unit selection unit 12 in FIG. 1 is performed by inputting the phoneme sequence of the target utterance, the weight vector for each feature obtained for each phoneme, and all the samples in the speech waveform signal database. This is the feature vector to be represented.
On the other hand, the output is index information indicating the position of the phoneme sample in the speech waveform signal database, and each speech unit for connecting speech segments of the speech waveform signal (specifically, a phoneme, and in some cases, a plurality of phonemes). Are successively selected and may become one voice unit) and the voice unit time length.

The optimum speech unit is obtained as a path that minimizes the sum of the target cost representing the approximation cost of the difference from the target utterance and the connection cost representing the approximation cost of discontinuity between adjacent speech units. For the route search, a known Viterbi learning algorithm is used. The target sound t ₁ ⁿ = (t
₁ ,..., T _n ), by minimizing the sum of the target cost and the connection cost, each feature is close to the target speech, and the discontinuity between speech units is small in the speech waveform signal database. A combination of voice units u ₁ ⁿ = (u ₁ ,..., U _n ) can be selected. By indicating the positions of these voice units in the voice waveform signal database, voice synthesis of any uttered content is possible. become.

As shown in FIG. 3, the selection cost of each voice unit includes a target cost C ^t (u _i , t _i ) and a connection cost C ^c (u
_i-1 , u _i ), and the target cost C ^t (u _i , t _i ) is
Voice unit (phoneme candidate) u in voice waveform signal database
_i and the speech unit to be realized as synthesized speech (target phoneme)
is the predicted value of the difference between t _i and the consolidation cost C ^c (u _i−1 ,
u _i ) is a predicted value of the discontinuity that occurs in the connection between the connection unit (two connected phonemes) u _i−1 and u _i . For example, the conventional ATRν-Ta researched and put into practical use by the present applicant has
The lk speech synthesis system also took a similar idea in that the target cost and the connection cost were minimized, but using the prosodic feature parameters directly for unit selection is a new feature of the speech synthesis device of this embodiment. It is a feature.

Next, the calculation of the cost will be described. Target cost is the weighted sum of the difference between the elements of the feature vectors of the speech units of the feature vector and the candidate selected from among the speech waveform signal database of speech units to be realized, each target sub-cost C ^t _j ^(t _i, u Given the weighting factor w ^t _j of _i ), the target cost C ^t (t _i , u _i ) can be calculated by the following equation.

[0038]

(Equation 1)

Here, the difference between the elements of the feature vector is represented by p target sub-costs C ^t _j (t _i , u _i ) (where j is a natural number from 1 to p). Is between 20 and 30 in a preferred embodiment.
Variable within the range. In a more preferred embodiment, the number of dimensions p = 30 and the target sub-cost C ^t (t _i ,
The feature vector or feature parameter of the variable _j in u _i ) and the weight coefficient w ^t _j is the above-mentioned second prosodic feature parameter.

On the other hand, connection cost _{^{C c (u i-1,}} u i) likewise the q connecting subcost ^{_{C c j (u i-1}} , u i) ( although, j is a natural number from 1 to q ). The concatenated sub-cost is u
_i-1 and u _i can be determined from the acoustic features of. In a preferred embodiment, the consolidation sub-costs are:
(1) Cepstrum distance at phoneme connection point, (2) Absolute value of difference of logarithmic power, (3) Absolute value of difference of voice fundamental frequency F ₀ , that is, q = 3. These three types of acoustic feature parameters, the phoneme label of the preceding phoneme, and the phoneme label of the succeeding phoneme are referred to as third acoustic feature parameters. Each consolidation sub-cost C ^c _j (u
_i-1 , u _i ) weight w ^c _j is empirically (or experimentally)
In this case, the connection cost C ^c (u _i−1 , u _i ) can be calculated by the following equation.

[0041]

(Equation 2)

If the phoneme candidates u _i-1 and u _i are continuous speech units in the speech waveform signal database,
The connection is natural and the connection cost is zero. Here, in a preferred embodiment, the connection cost is determined based on the first acoustic feature parameter and the first prosodic feature parameter in the feature parameter memory 30, and is a continuous quantity of the third third feature parameter. Since it handles acoustic feature parameters, for example, while taking an arbitrary analog amount from 0 to 1,
Since the target cost deals with the above-mentioned 30 second acoustic feature parameters indicating whether or not the discriminating features of the preceding or succeeding phonemes match, for example, 0 (when the features match) or 1 (When the features do not match). Then, the connection cost of the N voice units is the sum of the target cost and the connection cost of each voice unit, and is expressed by the following equation.

[0043]

(Equation 3)

At this time, S represents a pause, and C ^c
(S, u ₁₎ and C ^c (u _n, S) represents the connection costs in connection to pause from the pause to the first speech unit and from the last speech unit. As is clear from this expression, in the present embodiment, the pose is handled in exactly the same way as other phonemes in the speech waveform signal database. Further, if the above equation is directly expressed by a sub-cost, the following equation is obtained.

[0045]

(Equation 4)

The voice unit selection processing is for determining the combination of voice units / u ₁ ⁿ that minimizes the overall cost determined by the above equation. Here, in the specification of the Japanese application, since the overline cannot be described,
Use / instead of the overline.

[0047]

[Number 5] ^{_{/ u 1 n = min C (}} t 1 n, u 1 n) u 1, u 2, ..., u n

In the above equation (5), the function min is a combination of phoneme candidates (ie, phoneme string candidates) u ₁ , u ₂ ,... Which minimizes the argument C (t ₁ ⁿ , u ₁ ⁿ ) of the function. , u _n
= / U ₁ ⁿ .

The learning process of the weight coefficient in the weight coefficient learning section 11 of FIG. 1 will be described below. The weight of the target sub-cost is determined using a linear regression analysis based on the acoustic distance. In the weight coefficient learning process, a different weight coefficient can be determined for every phoneme, or a weight coefficient can be determined for each phoneme category (for example, all nasal sounds). Although a common weighting factor can be determined for all phonemes, different weighting factors are used here for each phoneme. Each token (or each voice sample) in the database in the feature parameter memory 30
Is described as a set of a first acoustic feature parameter and a first prosodic feature parameter related to the acoustic feature of each token. The weighting factor is the strength of the relationship between each of the first acoustic feature parameter and the first prosodic feature parameter and the difference or acoustic distance in the second acoustic feature parameter of the phoneme in the token or context. Learned to determine the contribution (degree of contribution). The processing flow in the linear regression analysis is shown below.

<1> The following four processes (a) to (d) are repeatedly performed for all samples in the speech waveform signal database belonging to the phoneme type (or phoneme category) currently being learned. (A) The taken voice sample is regarded as the target utterance content. (B) Calculate the acoustic distance between all other samples belonging to the same phoneme type (category) in the audio waveform signal database and the audio sample. (C) Top N1 items close to the target phoneme (for example, N1 =
There are 20. Select the best phoneme candidate in ()). (D) The target phoneme itself t _i and the top N1 selected in (c) above
Target sub-cost C ^t _j (t _i , u _i ) for samples
Ask for. <2> For all target phonemes t _i and the top N1 optimal samples, acoustic distances and target sub-costs C ^t _j (t _i ,
u _i ). <3> By performing a linear regression analysis on the p target sub-costs, the contribution of each of the first acoustic feature parameter and the first prosodic feature parameter representing the target phoneme to each other is predicted. Then, a linear weighting coefficient of p target sub-costs for the phoneme type (category) is obtained. The cost is calculated using the weight coefficient. And
The processing from <1> to <3> is repeated for all phoneme types (categories).

If the acoustic distance of the target voice unit is directly obtained, a weight coefficient should be applied to each target sub-cost in order to select the closest voice sample. This is the purpose of the weight coefficient learning unit 11. The advantage of this embodiment is that the audio segments of the audio waveform signal in the audio waveform signal database can be used directly.

In the speech synthesizer of FIG. 1 configured as described above, the speech analysis unit 10 and the weight coefficient learning unit 11
The voice unit selecting unit 12 and the voice synthesizing unit 13 are configured by, for example, a digital computer or an arithmetic control unit such as a micro processing unit (MPU), while a text database memory 22, a phoneme HMM memory 23, The feature parameter memory 30 and the weight coefficient vector memory 31 are configured by a storage device such as a hard disk. Here, in the preferred embodiment, the audio waveform signal database memory 21 stores the CD-R
This is a storage device in the OM format. Hereinafter, processing in each of the processing units 10 to 13 of the speech synthesizer configured as described above and illustrated in FIG. 1 will be described.

FIG. 4 is a flowchart of the voice analysis processing executed by the voice analysis unit 10 of FIG. In FIG. 4, first, in step S11, a signal of a naturally uttered speech waveform signal is input from the speech waveform signal database memory 21 and A / D converted to be converted into digital speech waveform signal data. The text data in which the voice sentence is written is stored in the text database memory 22.
Input from the text database inside. Here, there is no need for text data, and in the case where there is no text data, text data may be obtained by performing voice recognition using a known voice recognition device from a voice waveform signal. The digital audio waveform signal data after the A / D conversion is divided into, for example, audio segments every 10 milliseconds. Then, step S12
Then, it is determined whether or not the phoneme sequence is predicted. If the phoneme sequence is not predicted, the process proceeds to step S14 after predicting and storing the phoneme sequence using, for example, a phoneme HMM in step S13. If the phoneme sequence is predicted or given in advance in step S12, or if a phoneme label is manually given, the process directly proceeds to step S14.

In step S14, the start position and the end position of a plurality of sentences or one sentence of the speech waveform signal for each phoneme segment are recorded, and an index number is assigned to the file. Next, step S15
Then, the first acoustic feature parameter for each phoneme segment is extracted using, for example, a known pitch extraction method. In step S16, phoneme labeling is performed on each phoneme segment, and the phoneme label and the first acoustic feature parameter corresponding to the phoneme label are recorded. Further, in step S17, a first acoustic feature parameter for each phoneme segment, a phoneme label, and the first prosodic feature parameter for the phoneme label are stored in a file index number, a start position in the file, and a time length. At the same time, it is stored in the feature parameter memory 30. Finally, in step S18, index information including a file index number, a start position in the file, and a time length is assigned to each phoneme segment, and the index information is stored in the feature parameter memory 30. The voice analysis processing ends.

FIGS. 5 and 6 show the weight coefficient learning unit 1 of FIG.
6 is a flowchart of a weight coefficient learning process executed by the first embodiment. In FIG. 5, first, at step S21, one phoneme type is selected from the feature parameter memory 30. Next, in step S22, a second acoustic feature parameter is extracted from the first acoustic feature parameter of the phoneme having the same phoneme type as the selected phoneme type, and is taken as the second acoustic feature parameter of the target phoneme. . Then, in step S23, the Euclidean cepstrum distance that is an acoustic distance between the remaining phonemes other than the target phoneme having the same phoneme type and the target phoneme in the second acoustic feature parameter, And logarithmic phoneme time length. In step S24, it is determined whether or not the processing in steps S22 and S23 has been performed for all the remaining phonemes. Repeat the process.

On the other hand, if the processing is completed in step S24, the top N1 best phoneme candidates are selected in step S26 based on the distance and time length obtained in step S23. Next, the top N1 best phoneme candidates selected in step S27 are ranked from the first to the N1th. Then, in step S28, a scale conversion value is calculated by subtracting an intermediate value from each distance with respect to the ranked N1 best phoneme candidates. Then, in step S29, it is determined whether or not the processing from steps S22 to S28 has been completed for all phoneme types and phonemes. If not, another phoneme type or phoneme is selected in step S30. , The processing from step S22 is repeated. On the other hand, step S29
If the process has been completed in step, the process proceeds to step S31 in FIG.

In FIG. 6, in step S31, one phoneme type is selected. Next, in step S32,
A second acoustic feature parameter of each phoneme is extracted for the selected phoneme type. Then, in step S33, by performing a linear regression analysis based on the scale conversion value for the selected phoneme type, the contribution to the scale conversion value in each second acoustic feature parameter is calculated, and the calculated contribution is calculated. Is stored in the weight coefficient vector memory 31 as a weight coefficient for each target phoneme. Step S34
Then, it is determined whether or not the processes in steps S32 and S33 have been completed for all phoneme types. If not, another phoneme type is selected in step S35, and the process from step S32 is repeated. . On the other hand, if the processing has been completed in step S34, the weight coefficient learning processing ends. The contribution in each second prosodic feature parameter is given in advance empirically (or experimentally), and the contribution is stored in the weight coefficient vector memory 31 as a weight coefficient vector for each target phoneme.

FIG. 7 is a flowchart of the voice unit selection process executed by the voice unit selection unit 12 of FIG. In FIG. 7, first, in step S41, the first phoneme from the beginning is selected from the input phoneme sequence. Next, in step S42, the weight coefficient vector of the phoneme having the same phoneme type as the selected phoneme is read from the weight coefficient vector memory 31, and the target sub-cost and necessary feature parameters are read from the feature parameter memory 30 and listed. Then, it is determined in step S43 whether or not all phonemes have been processed. If the process has not been completed, the next phoneme is selected in step S44, and the process in step S42 is repeated. On the other hand, step S43
If not completed, the process proceeds to step S45.

In step S45, the total cost of each phoneme candidate is calculated using Equation 4 for the input phoneme sequence. Next, in step S46, the top N2 best phoneme candidates are selected for each target phoneme based on the calculated cost. Then, step S4
In step 7, by using Viterbi search using equation 5, index information of a combination of phoneme candidates that minimizes the overall cost is searched together with the start time and time length of each phoneme. Then, the voice unit selection process ends.

Further, the speech synthesis unit 13 accesses the speech waveform signal database memory 21 based on the index information output from the speech unit selection unit 12 and the start time and time length of each phoneme. The digital audio waveform signal data of the unit-selected phoneme candidate is read out, and D /
The analog audio signal after the A conversion is converted and output via the speaker 14. As a result, the synthesized voice corresponding to the input phoneme sequence is output from the speaker 14.

As described above, in the speech synthesizer of the present embodiment, a method for minimizing the processing using a large-scale natural speech database in order to maximize the naturalness of the output speech has been described. . This embodiment includes four processing units 10 to 13. <Speech analysis unit 10> A processing unit which receives arbitrary speech waveform signal data accompanied by the transcribed text in the orthography and inputs a feature vector describing the properties of all phonemes in the speech waveform signal database. <Weighting coefficient learning unit 11> Using feature vectors of the audio waveform signal database and original waveforms of the audio waveform signal database, optimization of each characteristic parameter for selecting an audio unit so as to be most suitable for synthesizing a target audio. A processing unit that determines a weight coefficient as a weight vector. <Speech unit selection unit 12> A processing unit that creates index information in the speech waveform signal database memory 21 from the description of the feature vectors and weight vectors of all phonemes in the speech waveform signal database and the utterance content of the target speech. <Speech synthesizing section 13> In accordance with the created index information, the audio segment of the audio waveform signal data in the audio waveform signal database in the memory 21 is jumped and accessed, the audio segment of the target audio waveform signal is connected, and A processing unit that performs A conversion, outputs the result to the speaker 14, and synthesizes voice.

In the present embodiment, the compression of the audio waveform signal and the modification of the audio fundamental frequency F ₀ and the phoneme time length are not required. However, the audio samples are carefully labeled instead, and a large-scale audio waveform signal database is used. It is necessary to select the optimal one from among the above. The basic unit of the speech synthesis method of the present embodiment is a phoneme, which is generated by a dictionary or a text-phoneme conversion program. Even if the same phoneme is included in the speech waveform signal database, a sufficient variation of the phoneme is included. Is required. In the process of selecting a speech unit from the speech waveform signal database, a combination of phoneme samples that match the target prosodic environment and have the lowest discontinuity between adjacent speech units when connected is selected. For this purpose, an optimal weight coefficient of each feature parameter is determined for each phoneme.

The features of the speech synthesizer of the present embodiment are as follows. <Use of prosodic information as unit selection criterion> From the standpoint that spectral features are inseparable from prosodic features, prosodic features were introduced into speech unit selection criteria. <Automatic learning of weighting factors for acoustic and prosodic feature parameters> In the speech waveform signal database, it is determined how much various feature amounts such as phoneme environment, acoustic features, and prosodic features contribute to the selection of speech units. A speech synthesizer based on a corpus was automatically determined by using all the speech samples. <Direct Connection of Speech Waveform Signal> By the automatic learning described above, an optimum speech sample was selected from a large-scale speech waveform signal database, thereby constructing an arbitrary speech synthesis apparatus that does not use any signal processing. <External information conversion of speech waveform signal database> By completely treating the speech waveform signal database as external information, simply replacing the speech waveform signal data stored in a CD-ROM or the like enables use in any language and any speaker. We built a speech synthesizer that can do it.

[0064]

EXAMPLES Using the speech synthesizer of this embodiment, the inventor has evaluated various speech waveform signal databases including four languages so far. As is well known, conventionally, it was technically very difficult to synthesize a high-quality voice using the voice of a female speaker, but with the method of the present embodiment, there was no difference due to gender or age. At present, in Japanese, the highest quality synthesized speech is obtained when using a speech waveform signal database in which a young female speaker reads a short story. On the other hand, for German, a synthesized speech is output using CD-ROM data of a sentence to which a prosody label and a detailed phoneme label are added. This indicates that the speech synthesizer of the present embodiment can technically freely use various existing speech waveform signal data, instead of speech waveform signal data recorded specifically for speech synthesis. Also, in English, the best sound quality is obtained with a 45-minute audio waveform signal data of a radio announcer of the Boston University News Corpus. For Korean, a short story is read aloud.

[0065]

As described above in detail, according to the spontaneously uttered speech waveform signal connection type speech synthesizer according to the first aspect of the present invention, the first storage for storing the speech segment of the speech waveform signal of the naturally uttered speech. Means, index information for each phoneme in the audio waveform signal based on the audio segment of the audio waveform signal stored by the first storage means, and a phoneme sequence corresponding to the audio waveform signal; A first acoustic feature parameter for each phoneme indicated by, and a prosodic feature parameter for each phoneme indicated by the index information, and output. Second storage means for storing the index information, the first acoustic feature parameter, and the prosodic feature parameter; and the first acoustic feature stored by the second storage means. Calculating an acoustic distance in a second acoustic feature parameter between one target phoneme of the same phoneme type and another phoneme candidate based on the signature parameter and the prosodic feature parameter; By performing a predetermined statistical analysis for each of the second acoustic feature parameters for each phoneme candidate based on the target distance,
Weighting factor learning means for determining a weighting factor vector for each target phoneme representing the degree of contribution of the second acoustic feature parameter to each phoneme candidate; and the second acoustic feature determined by the weighting factor learning means A third storing a weighting coefficient vector for each target phoneme in the parameter;
A natural utterance sentence based on a weighting coefficient vector for each target phoneme stored by the third storage means and a prosodic feature parameter stored by the second storage means. A target cost representing an approximate cost between the target phoneme and the phoneme candidate for the phoneme sequence of
A speech that searches for a combination of phoneme candidates that minimizes a cost including a connection cost representing an approximate cost between two phoneme candidates to be connected adjacently and that outputs index information of the searched combination of phoneme candidates. Based on the unit selection means and the index information output from the audio unit selection means,
Speech synthesis means for synthesizing and outputting speech corresponding to the input phoneme sequence by sequentially reading out speech segments of a speech waveform signal corresponding to the index information from the first storage means, and connecting and outputting the speech segments. And Therefore,
Without using prosody control rules and without performing signal processing,
An arbitrary phoneme sequence can be converted into a uttered voice, and a voice quality that is more natural than that of the conventional example can be obtained.

In the voice synthesizing device according to the second aspect, the voice analyzing means preferably corresponds to the voice waveform signal based on an input voice waveform signal. Phoneme prediction means for predicting a phoneme sequence to be executed. Therefore, since it is not necessary to provide a phoneme sequence in advance, the manual operation can be simplified.

Further, in the speech synthesizing apparatus according to the third aspect, in the speech synthesizing apparatus according to the first or second aspect,
The weighting coefficient learning means preferably extracts the best plurality of N1 phoneme candidates based on the calculated acoustic distance, and then performs a linear regression analysis on the second acoustic feature parameter. A weight coefficient vector for each target phoneme, which represents the degree of contribution in the second acoustic feature parameter for each phoneme candidate, is determined. Therefore,
Without using prosody control rules and without performing signal processing,
An arbitrary phoneme sequence can be converted into a uttered voice, and a more natural voice quality can be obtained.

Further, in the voice synthesizing device according to the fourth aspect, in the voice synthesizing device according to the first or second aspect,
Preferably, the weighting coefficient learning unit extracts a plurality of N1 best phonemes based on the calculated acoustic distance, and then performs a predetermined neural processing on each of the second acoustic feature parameters. By executing a statistical analysis using a network, a weighting coefficient vector for each target phoneme, which indicates the degree of contribution of the second acoustic feature parameter for each phoneme candidate, is determined. Therefore, an arbitrary phoneme sequence can be converted into a uttered voice without using prosody control rules and without performing signal processing, and a more natural voice quality can be obtained.

According to a fifth aspect of the present invention, in the voice synthesizing apparatus according to any one of the first to fourth aspects, it is preferable that the voice unit selecting means includes the target cost and the target cost. After extracting the top N2 phoneme candidates having the best cost including the connection cost, a combination of phoneme candidates with the lowest cost is searched. Therefore, an arbitrary phoneme sequence can be converted into a uttered voice without using prosody control rules and without performing signal processing, and a more natural voice quality can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram of a spontaneously uttered speech waveform signal connection type speech synthesis apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram of a conventional speech synthesizer.

FIG. 3 is a model diagram showing a definition of a voice unit selection cost calculated by a voice unit selection unit in FIG. 1;

FIG. 4 is a flowchart of a voice analysis process executed by the voice analysis unit of FIG. 1;

FIG. 5 is a flowchart of a first part of a weight coefficient learning process executed by the weight coefficient learning unit in FIG. 1;

FIG. 6 is a flowchart of a second part of the weight coefficient learning process executed by the weight coefficient learning unit in FIG. 1;

FIG. 7 is a flowchart of a voice unit selection process executed by the voice unit selection unit of FIG. 1;

[Explanation of symbols]

 Reference Signs List 10: voice analysis unit, 11: weight coefficient learning unit, 12: voice unit selection unit, 13: voice synthesis unit, 14: speaker, 21: voice waveform signal database memory, 22: text database memory, 23: phoneme HMM memory, 30: Feature parameter memory 31: Weight coefficient vector

Claims (9)

[Claims] A first storage unit for storing a speech segment of a naturally uttered speech waveform signal; a speech segment of the speech waveform signal stored by the first storage unit; and a phoneme corresponding to the speech waveform signal. Based on the sequence, index information for each phoneme in the audio waveform signal, a first acoustic feature parameter for each phoneme indicated by the index information, and a prosodic feature for each phoneme indicated by the index information Voice analysis means for extracting and outputting the parameter; index information output from the voice analysis means;
Based on the first acoustic feature parameter and the prosodic feature parameter stored by the second storage means, and An acoustic distance in a second acoustic feature parameter between one target phoneme of the same phoneme type and another phoneme candidate is calculated, and the above-mentioned acoustic distance is calculated for each phoneme candidate based on the calculated acoustic distance. A weighting factor for determining a weighting factor vector for each target phoneme that represents the contribution of the second acoustic feature parameter to each phoneme candidate by performing a predetermined statistical analysis for each second acoustic feature parameter Learning means, and a third coefficient storing a weight coefficient vector for each target phoneme in the second acoustic feature parameter determined by the weight coefficient learning means. Storage means; a weighting coefficient vector for each target phoneme stored by the third storage means; and a prosodic feature parameter stored by the second storage means. For the phoneme sequence, the cost including the target cost representing the approximate cost between the target phoneme and the phoneme candidate and the connection cost representing the approximate cost between two phoneme candidates to be connected adjacently is minimized. A voice unit selecting means for searching for a combination of phoneme candidates and outputting index information of the searched combination of phoneme candidates; and a voice corresponding to the index information based on the index information output from the voice unit selecting means. By sequentially reading the audio segments of the waveform signal from the first storage means, connecting them, and outputting them, the speech corresponding to the input phoneme sequence is synthesized and output. Natural speech waveform signal, characterized in that a voice synthesizing means connected speech synthesis device. 2. The speech synthesis device according to claim 1, wherein said speech analysis means comprises phoneme prediction means for predicting a phoneme sequence corresponding to said speech waveform signal based on an inputted speech waveform signal. apparatus. 3. The weighting factor learning means extracts a plurality of N1 best phoneme candidates based on the calculated acoustic distance, and then linearly extracts each of the second acoustic feature parameters. 3. The speech synthesizer according to claim 1, wherein a weighting coefficient vector for each target phoneme, which represents a degree of contribution of the phoneme candidate in the second acoustic feature parameter, is determined by performing a regression analysis. 4. The weighting factor learning means extracts a plurality of N1 best phoneme candidates based on the calculated acoustic distance, and then determines a predetermined number of N1 phoneme candidates for each of the second acoustic feature parameters. A weighting coefficient vector for each target phoneme, which represents a degree of contribution in the second acoustic feature parameter for each phoneme candidate, is determined by performing a statistical analysis using the neural network. 3. The speech synthesizer according to 1 or 2. 5. The voice unit selecting means includes a plurality of top Ns having the best cost including the target cost and the connection cost.
2. The method according to claim 1, wherein after extracting two phoneme candidates, a combination of phoneme candidates having a minimum cost is searched.
5. The speech synthesizer according to any one of claims 1 to 4. 6. The apparatus according to claim 1, wherein the first acoustic feature parameter includes a cepstrum coefficient, a delta cepstrum coefficient, and a phoneme label. . 7. The apparatus according to claim 1, wherein the first acoustic feature parameter includes a formant parameter and a vocal tract sound source parameter. 8. The speech synthesizer according to claim 1, wherein the prosodic feature parameters include a phoneme time length, a speech fundamental frequency F ₀ , and power. 9. The method according to claim 1, wherein the second acoustic feature parameter includes a cepstrum distance.
A speech synthesizer according to one of the preceding claims.