JP3091426B2 - Speech synthesizer with spontaneous speech waveform signal connection - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice synthesizer capable of voice synthesizing in a more natural intonation. SOLUTION: A weight coefficient learning part 11 calculates an acoustic distance in a second acoustic characteristic parameter between a target phoneme of the same phoneme kind and a phoneme coordinate excepting it based on a first characteristic extracted acoustic characteristic parameter to be linear recurrence analyzed, and decides a weight coefficient vector showing a contribution degree in the second acoustic characteristic parameter. A voice unbolt selection part 12 retrieves a phoneme coordinate line that a cost containing a garter cost showing an approximate cost between the target phoneme and the phoneme coordinate and a connection cost showing the approximate cost between two phoneme coordinates to be connected adjacently becomes minimum to output its index information, and a voice synthetic part 13 reads out successively a voice segment of a voice waveform signal answering to the index information to connect and pie synthesize. At this time, the difference of the tilt of the voice basic frequency F0 pattern of the Phoneme coordinate is added to the target cost.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a spontaneous speech sound waveform signal connection type speech synthesizer for synthesizing an arbitrary phoneme sequence by connecting speech segments of a spontaneous speech waveform signal.

[0002]

2. Description of the Related Art FIG. 2 is a block diagram of a conventional speech synthesizer. As shown in FIG. 2, for example, LPC analysis is performed on the signal waveform data of the learning speaker to extract feature parameters including 16th-order cepstrum coefficients. The extracted feature parameters are stored in a feature parameter memory 62 which is a buffer memory, and then input from the memory 62 to the parameter time series generation unit 52. Next, the parameter time series generation unit 52 executes signal processing such as time normalization and generation processing of a parameter time series using the prosody control rules in the memory 63 based on the extracted feature parameters, A parameter time series such as a 16th-order cepstrum coefficient required for speech synthesis is generated and output to the speech synthesis unit 53.

The speech synthesizer 53 is a known speech synthesizer, and includes a pulse generator 53a for generating a voiced sound,
A noise generator 53b for generating an unvoiced sound; and a filter 53c capable of changing a filter coefficient. Based on an input parameter time series, a voiced sound generated by a pulse generator 53a and a noise generator 53b Switch to the unvoiced sound generated by the
Further, by changing a filter coefficient corresponding to a transfer function of the filter 53, a voice-synthesized voice signal is generated, and the voice is output from the speaker.

[0004]

However, in the conventional speech synthesizer, signal processing using prosody control rules is required, and speech synthesis is performed based on the processed characteristic parameters. However, there was a problem that the voice quality was extremely poor.

[0005] In order to solve this problem, the applicant of the present invention disclosed in Japanese Patent Application No. 8-120113 (hereinafter referred to as a comparative example) a method of connecting segments of a naturally uttered speech waveform signal. Has proposed a speech synthesizer that synthesizes speech by using. However, in the comparative example, there was a problem that it was difficult to synthesize speech with more natural intonation.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above problems and to convert an arbitrary phoneme sequence into a uttered voice without using prosody control rules and without performing signal processing. Another object of the present invention is to provide a speech synthesizer which can obtain a voice quality close to nature and can synthesize speech with more natural intonation as compared with the comparative example.

[0007]

According to the present invention, a spontaneously uttered speech waveform signal connection type speech synthesizing apparatus according to a first aspect of the present invention comprises: first storage means for storing a speech segment of a naturally uttered speech waveform signal; Based on the audio segment of the audio waveform signal stored by the first storage means and the phoneme string corresponding to the audio waveform signal, index information for each phoneme in the audio waveform signal and indicated by the index information Voice analysis means for extracting and outputting a first acoustic feature parameter for each phoneme and a first prosodic feature parameter for each phoneme indicated by the index information, and an index output from the voice analysis means Second storage means for storing information, the first acoustic feature parameter, and the first prosodic feature parameter; and first sound information stored by the second storage means. Based on the characteristic parameters, an acoustic distance in a second acoustic characteristic parameter between one target phoneme of the same phoneme type and another phoneme candidate is calculated, and based on the calculated acoustic distance, By performing a linear regression analysis on the two acoustic feature parameters,
Weighting factor learning means for determining a weighting factor vector for each target phoneme representing the degree of contribution of the second acoustic feature parameter for each phoneme candidate; and the second acoustic feature determined by the weighting factor learning means A third storage unit for storing a weight coefficient vector for each target phoneme in the parameters and a weight coefficient vector for each target phoneme that indicates a degree of contribution in a second prosody characteristic parameter for each phoneme candidate given in advance; A natural utterance sentence based on the weighting coefficient vector for each target phoneme stored by the third storage means and the first prosodic feature parameter stored by the second storage means. And a target cost representing the approximate cost between the target phoneme and the phoneme candidate,
Voice unit selection means for searching for a combination of phoneme candidates which minimizes a cost including a connection cost representing an approximate cost between two phoneme candidates and outputting index information of the searched combination of phoneme candidates; Based on the index information output from the selection means, the audio segments of the audio waveform signal corresponding to the index information are sequentially read out from the first storage means, connected and output, so that Voice synthesis means for synthesizing and outputting a corresponding voice, wherein the voice unit selection means adds a difference in inclination between the voice basic frequency F ₀ of the target phoneme and the voice basic frequency F ₀ of the phoneme candidate to the target cost. It is characterized by doing.

According to a second aspect of the present invention, a spontaneously uttered speech waveform signal connection type speech synthesizer includes a first storage unit for storing a speech segment of a naturally uttered speech waveform signal, and the first storage unit. Based on the stored speech segment of the speech waveform signal and the phoneme sequence corresponding to the speech waveform signal, index information for each phoneme in the speech waveform signal and a first phoneme for each phoneme indicated by the index information. Voice analysis means for extracting and outputting an acoustic feature parameter and a first prosodic feature parameter for each phoneme indicated by the index information; index information output from the voice analysis means; Storage means for storing the acoustic characteristic parameters of the first and second prosody characteristic parameters, and the first acoustic characteristic parameters stored by the second storage means An acoustic distance in a second acoustic feature parameter between one target phoneme of the same phoneme type and another phoneme candidate based on the calculated acoustic distance. Coefficient learning means for determining a weight coefficient vector for each target phoneme representing a contribution in the second acoustic feature parameter for each phoneme candidate by performing a linear regression analysis on the characteristic feature parameters, and the weight coefficient learning means Weighting factor vector for each target phoneme in the second acoustic feature parameter determined by the above, and weighting for each target phoneme representing the degree of contribution in the second prosodic feature parameter for each phoneme candidate given in advance Third storage means for storing a coefficient vector, and a weight coefficient vector for each target phoneme stored by the third storage means , Based on the first prosodic feature parameters stored by said second storage means, with respect to a sequence of phonemes natural speech statement being input,
A combination of phoneme candidates that minimizes a cost including a target cost representing an approximate cost between a target phoneme and a phoneme candidate and a connection cost representing an approximate cost between two phoneme candidates to be connected adjacently. A voice unit selecting means for outputting index information of a combination of searched phoneme candidates, and a voice segment of a voice waveform signal corresponding to the index information based on the index information output from the voice unit selecting means. A voice synthesizing means for sequentially synthesizing and outputting a voice corresponding to the input phoneme string by sequentially reading out from the first storage means, connecting and outputting, and the voice unit selecting means includes a target phoneme. Voice fundamental frequency F ₀
When the difference between the median value of the phoneme candidate and the median value of the voice fundamental frequency F ₀ of the phoneme candidate is equal to or greater than a predetermined threshold value, a predetermined penalty cost is added to the target cost.

According to a third aspect of the present invention, there is provided a spontaneously uttered speech waveform signal connection type speech synthesizing apparatus, comprising: a first storage unit for storing a speech segment of a naturally uttered speech waveform signal; and the first storage unit. Based on the stored speech segment of the speech waveform signal and the phoneme sequence corresponding to the speech waveform signal, index information for each phoneme in the speech waveform signal and a first phoneme for each phoneme indicated by the index information. Voice analysis means for extracting and outputting an acoustic feature parameter and a first prosodic feature parameter for each phoneme indicated by the index information; index information output from the voice analysis means; Storage means for storing the acoustic characteristic parameters of the first and second prosody characteristic parameters, and the first acoustic characteristic parameters stored by the second storage means An acoustic distance in a second acoustic feature parameter between one target phoneme of the same phoneme type and another phoneme candidate based on the calculated acoustic distance. Coefficient learning means for determining a weight coefficient vector for each target phoneme representing a contribution in the second acoustic feature parameter for each phoneme candidate by performing a linear regression analysis on the characteristic feature parameters, and the weight coefficient learning means Weighting factor vector for each target phoneme in the second acoustic feature parameter determined by the above, and weighting for each target phoneme representing the degree of contribution in the second prosodic feature parameter for each phoneme candidate given in advance Third storage means for storing a coefficient vector, and a weight coefficient vector for each target phoneme stored by the third storage means , Based on the first prosodic feature parameters stored by said second storage means, with respect to a sequence of phonemes natural speech statement being input,
A combination of phoneme candidates that minimizes a cost including a target cost representing an approximate cost between a target phoneme and a phoneme candidate and a connection cost representing an approximate cost between two phoneme candidates to be connected adjacent to each other. A voice unit selecting means for outputting index information of a combination of searched phoneme candidates, and a voice segment of a voice waveform signal corresponding to the index information based on the index information output from the voice unit selecting means. Voice synthesizing means for sequentially synthesizing and outputting the speech corresponding to the input phoneme string by sequentially reading out from the first storage means, outputting the combined speech, and the speech unit selecting means Adding the absolute value of the sum of the difference between the basic voice frequencies F ₀ of two target phonemes and the basic voice frequency F ₀ of two consecutive phoneme candidates to the connection cost. Features.

According to a fourth aspect of the present invention, in the voice synthesizing apparatus according to the first aspect, the voice unit selecting means includes a central value of a basic voice frequency F ₀ of a target phoneme and a basic voice of a phoneme candidate. When the difference between the median values of the frequencies F ₀ is equal to or greater than a predetermined threshold value, a predetermined penalty cost is further added to the target cost.

Further, the speech synthesizing device according to claim 5 is
In speech synthesis device according to claim 1 or 4, wherein said speech unit selection means comprises: a difference between the voice fundamental frequency F ₀ of the two target phonemes consecutive and voice fundamental frequency F ₀ of two consecutive phonemes candidate difference Is added to the connection cost.

Further, in the speech synthesizer according to the present invention, in the speech synthesizer according to any one of the first to fifth aspects, the voice unit selecting means determines the target cost and the connection cost. After extracting the top N2 phoneme candidates with the highest included cost, a combination of phoneme candidates with the lowest cost is searched.

According to a seventh aspect of the present invention, in the voice synthesizing apparatus according to any one of the first to sixth aspects, the voice analyzing means is configured to output the voice based on an input voice waveform signal. It is characterized by comprising a phoneme predicting means for predicting a phoneme sequence corresponding to the waveform signal.

Further, the speech synthesizing apparatus according to claim 8 is
8. The speech synthesizer according to claim 1, wherein the weighting factor learning unit extracts a plurality of N1 best phoneme candidates based on the calculated acoustic distance. By performing a linear regression analysis on the second acoustic feature parameter, the second
And determining a weighting coefficient vector for each target phoneme, which represents the degree of contribution in the acoustic feature parameter.

According to a ninth aspect of the present invention, in the speech synthesizer according to any one of the first to eighth aspects, the first acoustic feature parameter is a cepstrum coefficient and a delta cepstrum. It is characterized by including a coefficient and a phoneme label.

[0016] The speech synthesizing apparatus according to claim 10 is
In speech synthesis device according to one of claims 1 to 9, the first prosodic feature parameters of a phoneme duration, a voice fundamental frequency F _0, characterized in that it comprises a power.

Further, the speech synthesizer according to claim 11 is the speech synthesizer according to any one of claims 1 to 10, wherein the second acoustic feature parameter is:
(A) a phoneme label of a preceding phoneme preceding the phoneme to be processed, (b) a phoneme label of a succeeding phoneme following the phoneme, (c) a cepstrum distance at a connection point between the phonemes, and (d) a phoneme. The absolute value of the difference in log power between
(E) It is characterized by including the absolute value of the difference of the voice fundamental frequency F ₀ between phonemes.

Still further, a speech synthesizer according to claim 12 is the speech synthesizer according to any one of claims 1 to 11, wherein the second prosodic feature parameter is:
(A) The first phoneme preceding the phoneme to be processed
(B) a first prosodic feature parameter of a phoneme label of a succeeding phoneme following the phoneme, (c) a phoneme time length of the phoneme, and (d) a speech fundamental frequency of the phoneme. F ₀ and (e) a basic sound frequency F _{0 of the} preceding phoneme.

[0019]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a spontaneously uttered speech waveform signal connection type speech synthesis apparatus according to an embodiment of the present invention. For example, the conventional speech synthesizer shown in FIG. 2 performs a series of processes from extraction of a text corresponding to an input uttered speech to generation of a speech waveform signal, whereas the present embodiment largely classifies the speech. For example, it is classified into the following four processing units. (1) voice analysis of voice waveform signal data in the voice waveform signal database in the voice waveform signal database memory 21;
Specifically, the speech analysis unit 10 performs processing including generation of a phoneme symbol sequence, alignment of phonemes, and extraction of feature parameters. (2) A weighting factor learning unit 11 that determines while learning the optimal weighting factor. (3) A voice unit selection unit 12 that selects a voice unit based on an input phoneme sequence and outputs index information of voice waveform signal data corresponding to the input phoneme sequence. (4) Based on the index information output from the voice unit selection unit 12, the voice waveform signal database in the voice waveform signal database memory 21 is randomly accessed to reproduce the voice waveform signal of each optimal phoneme candidate. Speaker 14
A voice synthesizer 13 for outputting to

More specifically, the speech analysis unit 10 determines a phoneme for each phoneme in the speech waveform signal on the basis of the speech segment of the speech waveform signal of the natural utterance input and the phoneme sequence corresponding to the speech waveform signal. Index information, a first acoustic feature parameter for each phoneme indicated by the index information,
A first prosodic feature parameter for each phoneme indicated by the index information is extracted and output. The feature parameter memory 30 stores the index information output from the speech analysis unit 10, the first acoustic feature parameter, and the first prosodic feature parameter. Next, based on the first acoustic feature parameter stored in the feature parameter memory 30, the weighting factor learning unit 11 determines whether the second phoneme between one target phoneme of the same phoneme type and another phoneme candidate is different.
Calculating the acoustic distance in the acoustic feature parameters of
By performing a linear regression analysis on the second acoustic feature parameter based on the calculated acoustic distance, a weight coefficient vector for each target phoneme representing a contribution degree in the second acoustic feature parameter for each phoneme candidate is obtained. decide. The weight coefficient vector memory 31 stores a weight coefficient vector for each target phoneme in the second acoustic feature parameter determined by the weight coefficient learning unit 11 and a second given prosody characteristic for each phoneme candidate. A weight coefficient vector for each target phoneme, which represents the degree of contribution in the parameter, is stored. Further, the voice unit selection unit 12
On the basis of the weight coefficient vector for each target phoneme stored in the weight coefficient vector memory 31 and the first prosodic feature parameter stored in the feature parameter memory 30, a phoneme sequence of a natural utterance sentence is input. A phoneme candidate having a minimum cost including a target cost representing an approximate cost between a target phoneme and a phoneme candidate and a connection cost representing an approximate cost between two phoneme candidates to be connected adjacently. The combination is searched, and index information of the searched combination of phoneme candidates is output. Then, based on the index information output from the voice unit selection unit 12, the voice synthesis unit 13
The voice segments of the voice waveform signal corresponding to the index information are sequentially read out from the voice waveform signal database memory 21, connected and output to the speaker 14, thereby synthesizing and outputting the voice corresponding to the input phoneme string. .

Here, the processing of the voice analysis unit 10 must always be performed once for a new voice waveform signal database, and the processing of the weighting coefficient learning unit 11 generally only needs to be performed once. The optimal weight coefficient obtained by the above can be reused even under different speech synthesis conditions. Further, the processing of the voice unit selection unit 12 and the voice synthesis unit 13 is executed each time the input phoneme sequence to be voice-synthesized changes.

The speech synthesizer according to the present embodiment predicts all necessary feature parameters based on the input of a given level, and obtains a sample closest to a desired speech feature (ie, a speech waveform signal of a phoneme candidate). ) Is selected from the audio waveform signal database in the memory 21. At a minimum, processing is possible if a sequence of phoneme labels is given, but higher quality synthesized speech can be obtained if the speech fundamental frequency F ₀ and phoneme time length are given in advance. When only word information is given as an input, it is necessary to predict a phoneme sequence based on a dictionary or rule such as a phoneme hidden Markov model (hereinafter, a hidden Markov model is referred to as HMM). If no prosodic feature is given, a standard prosody is generated based on known features of phonemes in various environments in the speech waveform signal database.

In this embodiment, if at least text data describing the recorded content in the voice waveform signal database memory 21 in the orthographic format exists as in a text database in the text database memory 22, any voice waveform signal Although the database can be used as speech waveform signal data for synthesis, the quality of output speech is greatly affected by the recording state, the balance of phonemes in the speech waveform signal database, and the like, and the speech waveform signal database in the memory 21 is abundant. If the content is appropriate, more diverse voices can be synthesized. Conversely, if the voice waveform signal database is poor, the synthesized voice will have a strong sense of discontinuity and will be jerky.

Next, a description will be given of phoneme labeling for a natural uttered voice. The choice of speech unit depends on the method of labeling and searching phonemes in the speech waveform signal database. Here, in the preferred embodiment, the speech unit is a phoneme. First, the utterance content of the orthography given to the recorded voice is converted into a phoneme sequence, and further assigned to a voice waveform signal. Extraction of the prosodic feature parameters is performed based on this. The input of the voice analysis unit 10 is voice waveform signal data in the memory 21 accompanied by the phoneme notation in the memory 22, and the output is a feature vector or a feature parameter. This feature vector becomes a basic unit representing a voice sample in the voice waveform signal database, and is used for selecting an optimum voice unit.

In the first stage of the processing of the speech analysis unit 10, a speech from a orthographic text to a phoneme symbol for describing how an utterance content written in orthography is pronounced in actual speech waveform signal data. Conversion. Then, in a second stage, to determine the start and end times of each phoneme to measure prosodic and acoustic features,
This is a process of associating each phoneme symbol with a speech waveform signal (hereinafter, this process is referred to as a phoneme alignment process). Further, the third step is to generate a feature vector or feature parameter of each phoneme. The feature vector stores, as essential items, information on a phoneme label, a start time (start position) of the phoneme in each file in the speech waveform signal database in the memory 30, a speech fundamental frequency F ₀ , a phoneme time length, and power. And
Information such as stress, accent type, position with respect to the prosodic boundary, and spectrum inclination are stored as options of the feature parameter. The above characteristic parameters are arranged as shown in Table 1 below, for example.

[0026]

[Table 1] 情報 Index information: Index number (for one file Start time (start position) of the phoneme in each file in the audio waveform signal database in the memory 30 ───────────────────────── ────────── First acoustic feature parameter: 12th-order mel-cepstral coefficient 12th-order Δ-mel-cepstral coefficient Phoneme label Discrimination feature: Vowel (vocalic) (+) / non-vocal (non-vocalic) ) (-) Consonant (+) / non-consonant (-) interrupted (+) / continuity (continuant) (-) deterrent (checked) (+) / Unchecked (-) Rough (strident) (+) / Maturity (mellow) (-) Voiced (+) / Unvoiced (unvo iced) (-) Intensity (compact) (+) / Diffuse (-) Low tone (grave) (+) / High tone (acute) (-) Inflection tone (flat) (+) / Normal tone (plain) (-) sharp tone (sharp) (+) / normal tone (plain) (-) tension (tense) (+) / relaxation (lax) (-) nasal tone (nasal) ) (+) / Oral (oral) (-) ─────────────────────────────────── 1 Prosodic feature parameters of: phoneme time length voice fundamental frequency F ₀ power ───────────────────────────────────

Start time (start position) in the index information;
The first acoustic feature parameter and the first prosodic feature parameter are stored in the feature parameter memory 30 for each phoneme. Here, for example, 12
The characteristic parameter of the discrimination feature is given a parameter value of (+) or (-) for each item. Further, Table 2 shows an example of the characteristic parameter which is the output result of the voice analysis unit 10. Here, in the voice waveform signal database memory 21, for example, an index number is assigned to each paragraph of a plurality of sentences or a file of one sentence, and a file to which one index number is assigned By adding the start time of the phoneme measured from the start time in the file to indicate the position of an arbitrary phoneme in the file and the phoneme time length of the phoneme, the speech segment of the speech waveform signal of the phoneme is added. Can be specified.

[0028]

[Table 2] An example of a feature parameter that is an output result of the voice analysis unit 10 Index number X0005 {phoneme time length fundamental frequency power ...} ... ────────────────────── # 120 90 4.0 ... s 175 98 4.7 ... ei 95 102 6.5 ... dh 30 114 4.9... ih 75 143 6.9... s 150 140 5.7... p 87 137 5.1... 3 ... z 140 87 5.8 ... # 253 87 4.0 ...

In Table 2, # indicates a pause. When selecting a speech unit, it is necessary to check in advance how much each acoustic and prosodic feature parameter contributes to each phoneme. In the fourth stage, the speech waveform A weighting factor for each feature parameter is determined using all audio samples in the signal database.

In the process of generating a phoneme symbol sequence in the speech analysis unit 10, as described above, in this embodiment, if at least one of the recorded contents is described in orthography,
Any audio waveform signal database can be used as audio waveform signal data for synthesis. If only word information is given as input, it is necessary to predict phoneme sequences based on dictionaries and rules. Further, in the phoneme alignment processing in the speech analysis unit 10, in the case of a read-aloud speech, each word is often pronounced close to its standard pronunciation, and it is rare to hesitate or stutter. In the case of such speech waveform signal data, phoneme labeling is correctly performed by a simple dictionary search, and learning of a phoneme model of a phoneme HMM for phoneme alignment becomes possible.

In the training of the phoneme model for phoneme alignment, unlike the case of complete speech recognition, it is not necessary to completely separate the speech waveform signal data for training and the speech waveform signal data for test. Learning can be performed using the signal data. First, a model for another speaker is used as an initial model. Viterbi learning is performed using all voice waveform signal data so that only standard pronunciation or limited pronunciation changes are allowed for all words, and appropriate segmentation is performed. The phonemes are aligned using an algorithm, and the feature parameters are re-estimated. Pauses between words are processed according to the rules for generating pauses between words.
If alignment fails due to a pause in the word, it must be corrected manually.

It is necessary to select what phoneme label is to be used as phoneme notation. If a phoneme set exists for which a well-learned HMM model can be used, it is advantageous to use it. Conversely, if the speech synthesizer has a complete dictionary, a method of completely collating the label of the speech waveform signal database with the dictionary is also effective.
Since we have a choice for learning the weighting factor, if the most important criterion is to be able to match the equivalents predicted by the speech synthesizer later from the speech waveform signal database, good. Subtle differences in pronunciation are automatically identified by the prosodic environment of the pronunciation, so there is no need to manually label phonemes.

As the next stage of the preprocessing, prosodic feature parameters for describing the articulatory features of individual phonemes are extracted. In conventional phonetics, language sounds are classified based on features such as articulation position and articulation style. On the other hand, phonetics that take into account the prosody, such as the First School, place clear articulations and emphasis in order to capture small differences in sound quality arising from differences in prosody context. To distinguish where they are. Although there are various methods for describing these differences, the following two methods are used here. First, at a lower level, a value obtained by averaging the power, the extension of the phoneme time length, and the basic voice frequency F ₀ for a certain phoneme is used to obtain a one-dimensional feature. On the other hand, at a higher level, a method of marking prosody boundaries and emphasis points in consideration of the above differences in prosody features is used. Since these two types of features are closely related to each other, one can predict the other, but both have a strong influence on the characteristics of each phoneme.

Just as there is a degree of freedom in the method of describing the prosodic feature parameters as well as in the method of defining the phoneme set for describing the speech waveform signal database, the method of selecting these is determined by the speech synthesizer. Depends on predictive ability. If the speech waveform signal database is pre-labeled, the task of the speech synthesizer is to properly learn from the internal representation how to perform the actual speech in the speech waveform signal database. On the other hand, if the speech waveform signal database is not labeled with phonemes, it is necessary to consider from what feature parameters the speech synthesizer can predict the most appropriate speech unit. . This study and the learning of the determination of the optimal feature parameter weight are executed by the weight coefficient learning unit 11 which determines the weight while learning the weight coefficient for each feature parameter.

Next, the weight coefficient learning processing executed by the weight coefficient learning section 11 will be described. In order to select the best sample for the acoustic and prosodic environment of a given target speech from the speech waveform signal database, first determine which features and how much are contributed by the difference between phonemic and prosodic environments. There is a need. This is to change the type of critical characteristic parameters depending on the nature of the phonemes, for example, although voice fundamental frequency F ₀ is very effective for the selection of voiced, little effect on the selection of the unvoiced sound. Further, the acoustic characteristics of the fricative sound vary depending on the types of the phonemes before and after. In order to select an optimal phoneme, how much weight is given to each feature is automatically determined by an optimal weight determination process, that is, a weight coefficient learning process.

In the process of determining the optimum weighting factor executed by the weighting factor learning section 11, the first process used is to select the optimum sample from all the corresponding utterance samples in the speech waveform signal database. It is to list features. Here, phonemic features such as articulation positions and articulation styles, and prosodic feature parameters such as the sound fundamental frequency F ₀ , phoneme time length, and power of the preceding phoneme, the phoneme, and the succeeding phoneme are used. Specifically, a second prosody parameter, which will be described in detail later, is used. Next, in the second stage, for each phoneme, one speech sample (or a speech waveform signal of a phoneme) is focused on to determine which feature parameter is important and how important it is in selecting an optimal candidate. The acoustic distance including the difference in the phoneme time length from all phoneme samples is obtained, and the N2 best similar speech samples, that is, the speech segment of the speech waveform signal of the N2 best phoneme candidate is selected.

Further, in the third stage, a linear regression analysis is performed, and a weight coefficient indicating the importance of each feature parameter in various acoustic and prosodic environments is obtained by using the similar speech samples. For example, the following feature parameter (hereinafter, referred to as a second prosodic feature parameter) is used as the prosodic feature parameter in the linear regression analysis processing. (1) a first prosodic feature parameter of a preceding phoneme (hereinafter, referred to as a preceding phoneme) that precedes the phoneme to be processed by one; (2) a subsequent phoneme that follows by only one from the subject phoneme to be processed ( (Hereinafter referred to as a succeeding phoneme.) The first prosodic feature parameter of the phoneme label; (3) the phoneme time length of the phoneme; (4) the speech fundamental frequency F _{0 of the} phoneme; (5) the speech fundamental frequency of the preceding phoneme. F ₀ ; and (6) the fundamental sound frequency F _{0 of the} succeeding phoneme. Here, the preceding phoneme is a phoneme that precedes the phoneme by one, but is not limited to this, and may include a phoneme that precedes by a plurality of phonemes. Further, the succeeding phoneme is a phoneme following only one phoneme from the phoneme, but is not limited thereto.
A phoneme following only a plurality of phonemes may be included. further,
The voice fundamental frequency F ₀ of the following phoneme may be excluded.

Next, the processing of the voice unit selection unit 12 for selecting a natural voice sample will be described. In the conventional speech synthesizer, a phoneme sequence is determined for a target utterance, and a target value of F ₀ and a phoneme time length for prosody control is calculated. On the other hand, in the present embodiment, only the prosody is calculated in order to appropriately select the optimum speech sample, but the prosody is not directly controlled.

FIG. 3 shows that the input of the processing of the speech unit selection unit 12 of FIG. 1 is performed by inputting a phoneme sequence of the target utterance, a weight vector for each feature obtained for each phoneme, and all samples in the speech waveform signal database. This is the feature vector to be represented.
On the other hand, the output is index information indicating the position of a phoneme sample in the speech waveform signal database, and each speech unit for connecting speech segments of the speech waveform signal (specifically, a phoneme, and in some cases, a plurality of phonemes). Are successively selected and may become one voice unit) and the voice unit time length.

The optimum speech unit is obtained as a path that minimizes the sum of the target cost representing the approximation cost of the difference from the target utterance and the connection cost representing the approximation cost of discontinuity between adjacent speech units. For the route search, a known Viterbi learning algorithm is used. The target sound t ₁ ⁿ = (t
₁ ,..., T _n ), by minimizing the sum of the target cost and the connection cost, each feature is close to the target speech, and the discontinuity between speech units is small in the speech waveform signal database. A combination of voice units u ₁ ⁿ = (u ₁ ,..., U _n ) can be selected. By indicating the positions of these voice units in the voice waveform signal database, voice synthesis of arbitrary utterance contents is possible. become.

As shown in FIG. 3, the selection cost of each voice unit includes a target cost C ^t (u _i , t _i ) and a connection cost C ^c (u
_i-1 , u _i ), and the target cost C ^t (u _i , t _i ) is
Voice unit (phoneme candidate) u in voice waveform signal database
_i and the speech unit to be realized as synthesized speech (target phoneme)
is the predicted value of the difference between t _i and the consolidation cost C ^c (u _i−1 ,
u _i ) is a predicted value of the discontinuity that occurs in the connection between the connection unit (two connected phonemes) u _i−1 and u _i . For example, a conventional ATRν-Ta researched and put to practical use by the present applicant has been proposed.
The lk speech synthesis system also took a similar idea in that the target cost and the connection cost were minimized, but using the prosodic feature parameters directly for unit selection is a new feature of the speech synthesis device of this embodiment. It is a feature.

Next, the calculation of the cost will be described. The target cost is a weighted sum of the difference between each element of the feature vector of the speech unit to be realized and the feature vector of the candidate speech unit selected from the speech waveform signal database, and each target sub-cost C ^t _j (t _i , u Given the weighting factor w ^t _j of _i ), the target cost C ^t (t _i , u _i ) can be calculated by the following equation.

[0043]

(Equation 1)

Here, the difference between the elements of the feature vector is represented by p target sub-costs C ^t _j (t _i , u _i ) (where j is a natural number from 1 to p). Is between 20 and 30 in a preferred embodiment.
Variable within the range. In a more preferred embodiment, the number of dimensions p = 30 and the target sub-cost C ^t (t _i ,
The feature vector or feature parameter of the variable _j in u _i ) and the weight coefficient w ^t _j is the above-mentioned second prosodic feature parameter.

On the other hand, connection cost _{^{C c (u i-1,}} u i) likewise the q connecting subcost ^{_{C c j (u i-1}} , u i) ( although, j is a natural number from 1 to q ). The concatenated sub-cost is u
_i-1 and u _i can be determined from the acoustic features of. In a preferred embodiment, the consolidation sub-costs are:
(1) Cepstrum distance at phoneme connection point, (2) Absolute value of logarithmic power difference, (3) Absolute value of difference of voice fundamental frequency F ₀ , that is, q = 3. These three types of acoustic feature parameters, the phoneme label of the preceding phoneme, and the phoneme label of the succeeding phoneme are referred to as second acoustic feature parameters. Each consolidation sub-cost C ^c _j (u
_i-1 , u _i ) weight w ^c _j is empirically (or experimentally)
In this case, the connection cost C ^c (u _i−1 , u _i ) can be calculated by the following equation.

[0046]

(Equation 2)

If the phoneme candidates u _i-1 and u _i are continuous speech units in the speech waveform signal database,
The connection is natural and the connection cost is zero. Here, in a preferred embodiment, the connection cost is determined based on the first acoustic feature parameter and the first prosodic feature parameter in the feature parameter memory 30, and is a continuous quantity of the above-mentioned three second features. Since it handles acoustic feature parameters, for example, while taking an arbitrary analog amount from 0 to 1,
Since the target cost deals with the 30 second acoustic feature parameters indicating whether or not the discriminating features of the preceding or succeeding phonemes match, for example, 0 (when the features match) or 1 (When the features do not match). Then, the connection cost of the N voice units is the sum of the target cost and the connection cost of each voice unit, and is expressed by the following equation.

[0048]

(Equation 3)

At this time, S represents a pause, and C ^c
(S, u ₁₎ and C ^c (u _n, S) represents the connection costs in connection to pause from the pause to the first speech unit and from the last speech unit. As is clear from this expression, in the present embodiment, the pose is handled in exactly the same way as other phonemes in the speech waveform signal database. Further, if the above equation is directly expressed by a sub-cost, the following equation is obtained.

[0050]

(Equation 4)

The voice unit selection process is for determining the combination of voice units / u ₁ ⁿ that minimizes the overall cost determined by the above equation. Here, in the specification of the Japanese application, since the overline cannot be described,
Use / instead of the overline.

[0052]

[Number 5] ^{_{/ u 1 n = min C (}} t 1 n, u 1 n) u 1, u 2, ..., u n

In the above equation (5), the function min is a combination of phoneme candidates (ie, phoneme string candidates) u ₁ , u ₂ ,... Which minimizes C (t ₁ ⁿ , u ₁ ⁿ ) which is an argument of the function. , u _n
= / U ₁ ⁿ .

Incidentally, the unnaturalness of the intonation of the speech synthesized by the speech synthesizer of the comparative example is caused by a gap in the speech fundamental frequency F ₀ between phoneme units or a phoneme having an inappropriate fundamental frequency pattern in an accent nucleus. This is probably due to the choice of units. Gap voice fundamental frequency F ₀ is the shape of the voice fundamental frequency F ₀ pattern between adjacent phonemes, to produce the difference in size, it is necessary to select criteria consider these. In order to express an appropriate accent, the relative fundamental voice frequency F between phoneme units is used.
It is necessary to consider the size of ₀ .

Therefore, in the present embodiment, the gap of the fundamental voice frequency F ₀ pattern between phoneme units is reduced,
In order to select a phoneme unit that more faithfully reflects the shape of the estimated voice fundamental frequency F ₀ pattern, the following cost function related to the voice fundamental frequency F ₀ has been added.

(A) The slope of the basic sound frequency F ₀ (hereinafter referred to as the slope)
It is called slope cost. ): Considering the slope of the voice fundamental frequency F ₀ pattern for each phoneme in the voice database, the difference between the desired voice fundamental frequency F ₀ to be realized (hereinafter referred to as target speech fundamental frequency F ₀ ) and the slope is targeted. Add to cost. That is, adding the difference of inclination between the voice fundamental frequency F ₀ of the voice fundamental frequency F ₀ and the phoneme candidate target phoneme target cost. The inclination of the voice fundamental frequency F ₀ pattern is considered only for vowels that are sufficiently present in the speech database, and other voiced sounds are not considered. Further, in order to reduce the influence of an error in extracting the basic voice frequency F ₀ from the original voice waveform, the extracted voice basic frequency F ₀ was smoothed, and then the slope was calculated by regression analysis.

(B) Threshold value of voice basic frequency F ₀ (hereinafter referred to as threshold cost): if the difference between the median values of voice basic frequency F _{0 in} the target cost is not less than a predetermined threshold value. For example, a predetermined penalty cost of 20, for example, is added. That is, when the difference between the median value of the basic voice frequency F ₀ of the target phoneme and the median value of the basic voice frequency F ₀ of the phoneme candidate is equal to or greater than a predetermined threshold, a predetermined penalty cost is added to the target cost.

(C) The difference between the fundamental voice frequencies F ₀ (hereinafter referred to as the difference)
It is called differential cost. ): In order to make the difference between the sound fundamental frequencies F ₀ of two consecutive phonemes close to the difference between the target sound fundamental frequencies F ₀ ,

| U ′ _f0i −u _f0i | is added to the connection cost. here,

[Equation 7] u _'f0i = and _{u f0i-1 + t f0i -t} f0i-1. t _f0i-1 and t _f0i represent the target speech fundamental frequencies F ₀ of the i−1 and i-th phonemes, respectively, and u _f0i−1 and u _f0i represent the speech fundamental frequencies F of the i−1 and i-th phonemes, respectively.
Represents ₀ . U ′ _f0i is a new target voice fundamental frequency F ₀ of the i-th phoneme. That is, the following equation can be obtained from Equations 6 and 7.

| U ′ _f0i −u _f0i | = | u _f0i−1 −u _f0i + t
_{f0i− tf0i−1− uf0i} | Therefore, the sound fundamental frequency F ₀ of two consecutive target phonemes
And the speech fundamental frequency F of two consecutive phoneme candidates
The absolute value of the value added to the difference of ₀ is added to the connection cost.

The above three costs may be added alone or in any combination.

The learning process of the weight coefficient in the weight coefficient learning section 11 of FIG. 1 will be described below. The weight of the target sub-cost is determined using a linear regression analysis based on the acoustic distance. In the weight coefficient learning process, a different weight coefficient can be determined for every phoneme, or a weight coefficient can be determined for each phoneme category (for example, all nasal sounds). Although a common weighting factor can be determined for all phonemes, different weighting factors are used here for each phoneme. The processing flow in the linear regression analysis is shown below.

<1> The following four processes (a) to (d) are repeatedly performed on all samples in the speech waveform signal database belonging to the phoneme type (or phoneme category) currently being learned. (A) The taken voice sample is regarded as the target utterance content. (B) Calculate the acoustic distance between all other samples belonging to the same phoneme type (category) in the audio waveform signal database and the audio sample. (C) Top N1 items close to the target phoneme (for example, N1 =
There are 20. Select the best phoneme candidate in ()). (D) The target phoneme itself t _i and the top N1 selected in (c) above
Target sub-cost C ^t _j (t _i , u _i ) for samples
Ask for. <2> For all target phonemes t _i and the top N1 optimal samples, acoustic distances and target sub-costs C ^t _j (t _i ,
u _i ). <3> Perform linear regression analysis to obtain linear weighting coefficients of p target sub-costs for the phoneme type (category). The cost is calculated using the weight coefficient. And
The processing from <1> to <3> is repeated for all phoneme types (categories).

If the acoustic distance in the target phoneme unit is directly obtained, to select the closest voice sample, it is necessary to determine what weighting factor should be applied to each target sub-cost. This is the purpose of the weight coefficient learning unit 11. An advantage of this embodiment is that audio segments of the audio waveform signal in the audio waveform signal database can be used directly.

In the speech synthesizer of FIG. 1 configured as described above, the speech analysis unit 10 and the weight coefficient learning unit 11
The voice unit selecting unit 12 and the voice synthesizing unit 13 are configured by, for example, a digital computer or an arithmetic control unit such as a micro processing unit (MPU), while a text database memory 22, a phoneme HMM memory 23, The feature parameter memory 30 and the weight coefficient vector memory 31 are configured by a storage device such as a hard disk. Here, in the preferred embodiment, the audio waveform signal database memory 21 stores the CD-R
This is a storage device in the OM format. Hereinafter, processing in each of the processing units 10 to 13 of the speech synthesizer configured as described above and illustrated in FIG. 1 will be described.

FIG. 4 is a flowchart of the voice analysis process executed by the voice analysis unit 10 of FIG. In FIG. 4, first, in step S11, a signal of a naturally uttered speech waveform signal is input from the speech waveform signal database memory 21 and A / D converted to be converted into digital speech waveform signal data. The text data in which the voice sentence is written is stored in the text database memory 22.
Input from the text database inside. Here, there is no need for text data, and in the case where there is no text data, text data may be obtained by performing voice recognition using a known voice recognition device from a voice waveform signal. The digital audio waveform signal data after the A / D conversion is divided into, for example, audio segments every 10 milliseconds. Then, step S12
Then, it is determined whether or not the phoneme sequence is predicted. If the phoneme sequence is not predicted, the process proceeds to step S14 after predicting and storing the phoneme sequence using, for example, a phoneme HMM in step S13. If the phoneme sequence is predicted or given in advance in step S12, or if a phoneme label is manually given, the process directly proceeds to step S14.

In step S14, the start position and the end position of a plurality of sentences or one sentence of the speech waveform signal for each phoneme segment are recorded, and an index number is assigned to the file. Next, step S15
Then, the first acoustic feature parameter for each phoneme segment is extracted using, for example, a known pitch extraction method. In step S16, phoneme labeling is performed on each phoneme segment, and the phoneme label and the first acoustic feature parameter corresponding to the phoneme label are recorded. Further, in step S17, a first acoustic feature parameter for each phoneme segment, a phoneme label, and the first prosodic feature parameter for the phoneme label are stored in a file index number, a start position in the file, and a time length. At the same time, it is stored in the feature parameter memory 30. Finally, in step S18, index information including a file index number, a start position in the file, and a time length is assigned to each phoneme segment, and the index information is stored in the feature parameter memory 30. The voice analysis processing ends.

FIGS. 5 and 6 show the weight coefficient learning unit 1 of FIG.
6 is a flowchart of a weight coefficient learning process executed by the first embodiment. In FIG. 5, first, at step S21, one phoneme type is selected from the feature parameter memory 30. Next, in step S22, a second acoustic feature parameter is extracted from the first acoustic feature parameter of the phoneme having the same phoneme type as the selected phoneme type, and is taken as the second acoustic feature parameter of the target phoneme. . Then, in step S23, the Euclidean cepstrum distance that is an acoustic distance between the remaining phonemes other than the target phoneme having the same phoneme type and the target phoneme in the second acoustic feature parameter, And logarithmic phoneme time length. In step S24, it is determined whether or not the processing in steps S22 and S23 has been performed for all the remaining phonemes. If the processing has not been completed, another remaining phoneme is selected in step S25, and the process from step S23 is performed. Repeat the process.

On the other hand, if the processing is completed in step S24, the top N1 best phoneme candidates are selected in step S26 based on the distance and time length obtained in step S23. Next, the top N1 best phoneme candidates selected in step S27 are ranked from the first to the N1th. Then, in step S28, a scale conversion value is calculated by subtracting an intermediate value from each distance with respect to the ranked N1 best phoneme candidates. Then, in step S29, it is determined whether or not the processing from steps S22 to S28 has been completed for all phoneme types, and if not, step S30
After selecting another phoneme type, the process from step S22 is repeated. On the other hand, when the processing is completed in step S29, the process proceeds to step S31 in FIG.

In FIG. 6, in step S31, one phoneme type is selected. Next, in step S32,
A second acoustic feature parameter of each phoneme is extracted for the selected phoneme type. Then, in step S33, by performing a linear regression analysis based on the scale conversion value for the selected phoneme type, the contribution to the scale conversion value in each second acoustic feature parameter is calculated, and the calculated contribution is calculated. Is stored in the weight coefficient vector memory 31 as a weight coefficient for each target phoneme. Further, the contribution in each second prosodic feature parameter is given in advance empirically (or experimentally), and the contribution is stored in the weight coefficient vector memory 31 as a weight coefficient vector for each target phoneme. In step S34, it is determined whether or not the processing in steps S32 and S33 has been completed for all phoneme types.
After selecting another phoneme type in step 5, the process from step S32 is repeated. On the other hand, if the processing has been completed in step S34, the weight coefficient learning processing ends.

FIG. 7 is a flowchart of the voice unit selection process executed by the voice unit selection unit 12 of FIG. In FIG. 7, first, in step S41, the first phoneme from the beginning is selected from the input phoneme sequence. Next, in step S42, the weight coefficient vector of the phoneme having the same phoneme type as the selected phoneme is read from the weight coefficient vector memory 31, and the target sub-cost and necessary feature parameters are read from the feature parameter memory 30 and listed. Then, it is determined in step S43 whether or not all phonemes have been processed. If the processing has not been completed, the next phoneme is selected in step S44, and the process in step S42 is repeated. On the other hand, step S43
If not, the process proceeds to step S45.

In step S45, the total cost of each phoneme candidate is calculated using Equation 4 for the input phoneme sequence. Next, in step S46, the top N2 best phoneme candidates are selected for each target phoneme based on the calculated cost. Then, step S4
In step 7, by using Viterbi search using equation 5, index information of a combination of phoneme candidates that minimizes the overall cost is searched together with the start time and time length of each phoneme. Then, the voice unit selection process ends.

Further, the speech synthesis unit 13 accesses the speech waveform signal database memory 21 based on the index information output from the speech unit selection unit 12 and the start time and time length of each phoneme. The digital voice waveform signal data of the unit-selected phoneme candidate is read out, and the D /
The analog audio signal after the A conversion is converted and output via the speaker 14. As a result, the synthesized voice corresponding to the input phoneme sequence is output from the speaker 14.

As described above, in the speech synthesizing apparatus according to the present embodiment, a method for minimizing the processing using a large-scale natural speech database in order to maximize the naturalness of the output speech has been described. . This embodiment includes four processing units 10 to 13. <Speech analysis unit 10> A processing unit which receives arbitrary speech waveform signal data accompanied by the transcribed text in the orthography and inputs a feature vector describing the properties of all phonemes in the speech waveform signal database. <Weighting coefficient learning unit 11> Using feature vectors of the audio waveform signal database and original waveforms of the audio waveform signal database, optimization of each characteristic parameter for selecting an audio unit so as to be most suitable for synthesizing a target audio. A processing unit that determines a weight coefficient as a weight vector. <Speech unit selection unit 12> A processing unit that creates index information in the speech waveform signal database memory 21 from the description of the feature vectors and weight vectors of all phonemes in the speech waveform signal database and the utterance content of the target speech. <Speech synthesizing section 13> In accordance with the created index information, the audio segment of the audio waveform signal data in the audio waveform signal database in the memory 21 is jumped and accessed, the audio segment of the target audio waveform signal is connected, and D / D A processing unit that performs A conversion, outputs the result to the speaker 14, and synthesizes voice.

In the present embodiment, the compression of the audio waveform signal and the modification of the audio fundamental frequency F ₀ and the phoneme time length become unnecessary, but instead, the audio samples are carefully labeled, and a large-scale audio waveform signal database is used. It is necessary to select the optimal one from among the above. The basic unit of the speech synthesis method of the present embodiment is a phoneme, which is generated by a dictionary or a text-phoneme conversion program. Even if the same phoneme is included in the speech waveform signal database, a sufficient variation of the phoneme is included. Is required. In the process of selecting a speech unit from the speech waveform signal database, a combination of phoneme samples that match the target prosodic environment and have the lowest discontinuity between adjacent speech units when connected is selected. For this purpose, an optimal weight coefficient of each feature parameter is determined for each phoneme.

[0074]

EXAMPLE A listening experiment was conducted on the speech synthesizer configured as described above as follows. Synthesized speech was created from 50 randomly selected newspaper articles using the speech synthesizer of the comparative example and the present embodiment, and presented to the subject. Accenting was performed semi-automatically. For the naturalness of the synthesized speech, the subject evaluated (a) intonation and (b) continuity and clarity evaluations from 5 "very good" to "very bad".
It was evaluated on a scale. When evaluating continuity and clarity,
Subjects were instructed to ignore intonation.
The following five types of synthesized speech were used. (1) Speech synthesis was performed using the speech synthesizer of the comparative example. (2) Using the speech synthesis apparatus of the present embodiment, speech synthesis was performed by adding only the slope cost among the additional costs. (3) Using the speech synthesizer of the present embodiment, speech synthesis was performed by adding only the threshold cost among the additional costs. Here, the threshold was set to 20 Hz from preliminary examination. (4) Using the speech synthesizer of the present embodiment, speech synthesis is performed by adding only the difference cost among the additional costs. (5) Using the speech synthesizer of the present embodiment, speech synthesis was performed by adding all additional costs. The subject,
Six of them are unfamiliar with synthesized speech.

FIG. 8 shows the results of the intonation evaluation experiment.
Shown in As is clear from FIG. 8, by introducing each of the costs proposed this time individually, the evaluation of bad intonation and extremely bad is reduced by about 20%, and the evaluation of good / extremely is increased by about 10%. ing. Further, in a listening experiment in which these three additional costs were introduced at the same time, the evaluation of bad / extremely bad was about half that of the comparative example, confirming the usefulness of the proposed selection criterion.

Next, FIG. 9 shows the results of an experiment for evaluating continuity and clarity. When the threshold cost and the differential cost were individually introduced, and when all the additional costs were introduced, the evaluation was slightly lower than the comparative example. The reason for this is that by adding these additional costs, the effect of the cost of cepstrum and phoneme duration is relatively smaller than in the comparative example, and the sense of discontinuity or inappropriate This is probably because a phoneme unit having a long phoneme duration has been selected. On the other hand, when the inclination cost was introduced, the continuity and clarity were not much deteriorated as compared with the comparative example. From these facts, it is considered appropriate to introduce only the slope cost that contributes to the improvement of the naturalness of intonation, and most preferably has almost the same evaluation as that of the comparative example also in regard to continuity and clarity.

As described above, the natural speech waveform signal connection type speech synthesizer of the comparative example improves the naturalness of the intonation of the synthesized speech by improving the selection criteria of phoneme units to be considered when connecting the speech waveform signals. Was planned. The slope of the voice fundamental frequency F ₀ pattern, the threshold for the difference between the target speech fundamental frequency F _0, by introducing the consideration selection criteria the difference between the target speech fundamental frequency F ₀ of the successive phonemes, the synthesized speech Listening experiments confirmed that the naturalness of intonation was improved. Of these, it was confirmed that the quality of the synthesized speech was not significantly degraded when only the slope of the speech fundamental frequency F ₀ pattern was considered. Other methods improve the naturalness of intonation.

[0078]

As described above in detail, according to the spontaneously uttered speech waveform signal connection type speech synthesizing apparatus according to the first aspect of the present invention, the first storage for storing the speech segment of the spontaneously uttered speech waveform signal. Means, index information for each phoneme in the audio waveform signal based on the audio segment of the audio waveform signal stored by the first storage means, and a phoneme sequence corresponding to the audio waveform signal; Voice analysis means for extracting and outputting a first acoustic feature parameter for each phoneme indicated by, and a first prosodic feature parameter for each phoneme indicated by the index information; A second storage unit that stores the output index information, the first acoustic feature parameter, and the first prosody feature parameter, and a second storage unit that stores the index information; Based on one acoustic feature parameter, an acoustic distance in a second acoustic feature parameter between one target phoneme of the same phoneme type and another phoneme candidate is calculated, and the calculated acoustic distance is calculated. Weighting factor learning means for determining a weighting factor vector for each target phoneme representing a contribution in the second acoustic feature parameter for each phoneme candidate by performing a linear regression analysis on the second acoustic feature parameter based on the second regression analysis. And a weight coefficient vector for each target phoneme in the second acoustic feature parameter determined by the weight coefficient learning means;
Third storage means for storing a predetermined weighting coefficient vector for each target phoneme, which represents a degree of contribution in a second prosodic feature parameter for each phoneme candidate, the third storage means being provided in advance;
Based on the weighting coefficient vector for each target phoneme stored by the storage means and the first prosodic feature parameter stored by the second storage means. A phoneme candidate having a minimum cost including a target cost representing an approximate cost between a target phoneme and a phoneme candidate and a connection cost representing an approximate cost between two phoneme candidates to be connected adjacently. A voice unit selecting means for searching for a combination and outputting index information of the searched combination of phoneme candidates; and a voice of a voice waveform signal corresponding to the index information based on the index information output from the voice unit selecting means. By sequentially reading out the segments from the first storage means, connecting them and outputting them,
Voice synthesis means for synthesizing and outputting voices corresponding to the input phoneme sequence, wherein the voice unit selection means includes a gradient between a voice basic frequency F ₀ of a target phoneme and a voice basic frequency F ₀ of a phoneme candidate. Is added to the target cost. Therefore,
Without using prosody control rules and without performing signal processing,
An arbitrary phoneme sequence can be converted into a uttered voice, and a more natural voice quality can be obtained. In addition, speech synthesis can be performed with more natural intonation than in the comparative example.

Further, in the spontaneously uttered speech waveform signal connection type speech synthesizer according to claim 2 of the present invention, the first storage means for storing the speech segment of the spontaneously uttered speech waveform signal; Based on the audio segment of the audio waveform signal stored by the storage means and a phoneme string corresponding to the audio waveform signal, index information for each phoneme in the audio waveform signal and for each phoneme indicated by the index information Speech analysis means for extracting and outputting a first acoustic feature parameter and a first prosodic feature parameter for each phoneme indicated by the index information; index information output from the speech analysis means; Second storage means for storing the first acoustic feature parameter and the first prosodic feature parameter; and first acoustic feature stored by the second storage means. Calculating an acoustic distance in a second acoustic feature parameter between one target phoneme of the same phoneme type and another phoneme candidate based on the characteristic parameter, and calculating the acoustic distance based on the calculated acoustic distance. Weighting factor learning means for determining a weighting factor vector for each target phoneme representing a contribution in the second acoustic feature parameter for each phoneme candidate by performing a linear regression analysis on the two acoustic feature parameters; A weighting coefficient vector for each target phoneme in the second acoustic feature parameter determined by the coefficient learning means, and a target phoneme representing a contribution given in advance to the second prosodic feature parameter for each phoneme candidate. Storage means for storing a weight coefficient vector for each target phoneme, and a weight for each target phoneme stored by the third storage means. And the coefficient vector,
A target cost representing an approximate cost between a target phoneme and a phoneme candidate for a phoneme sequence of a natural utterance sentence based on the first prosodic feature parameter stored by the second storage means. And searching for a combination of phoneme candidates that minimizes the cost including the connection cost representing the approximate cost between two phoneme candidates to be connected adjacently,
A voice unit selecting means for outputting index information of the combination of the searched phoneme candidates; and, based on the index information output from the voice unit selecting means, a voice segment of a voice waveform signal corresponding to the index information, A voice synthesizing means for sequentially synthesizing and outputting a voice corresponding to the input phoneme string by sequentially reading out from the storage means and outputting the combined sound; and the voice unit selecting means includes a voice fundamental frequency of a target phoneme. when the median F _0, the difference between the median of the speech fundamental frequency F ₀ of the phoneme candidate is equal to or greater than a predetermined threshold value, it adds a predetermined penalty cost target cost. Therefore, an arbitrary phoneme sequence can be converted into a uttered voice without using prosody control rules and without performing signal processing, and a more natural voice quality can be obtained. In addition, speech synthesis can be performed with more natural intonation than in the comparative example.

Further, in the spontaneously uttered speech waveform signal connection type speech synthesizing apparatus according to claim 3 of the present invention, the first storage means for storing a speech segment of a spontaneously uttered speech waveform signal; Based on the audio segment of the audio waveform signal stored by the storage means and the phoneme sequence corresponding to the audio waveform signal, index information for each phoneme in the audio waveform signal and for each phoneme indicated by the index information Voice analysis means for extracting and outputting a first acoustic feature parameter and a first prosodic feature parameter for each phoneme indicated by the index information; and index information output from the voice analysis means; Second storage means for storing the first acoustic feature parameter and the first prosodic feature parameter, and first sound stored by the second storage means Based on the characteristic parameters, an acoustic distance in a second acoustic characteristic parameter between one target phoneme of the same phoneme type and another phoneme candidate is calculated, and based on the calculated acoustic distance, By performing a linear regression analysis on the two acoustic feature parameters,
Weighting factor learning means for determining a weighting factor vector for each target phoneme representing the degree of contribution of the second acoustic feature parameter for each phoneme candidate; and the second acoustic feature determined by the weighting factor learning means A third storage unit for storing a weight coefficient vector for each target phoneme in the parameters and a weight coefficient vector for each target phoneme that indicates a degree of contribution in a second prosody characteristic parameter for each phoneme candidate given in advance; A natural utterance sentence based on the weighting coefficient vector for each target phoneme stored by the third storage means and the first prosodic feature parameter stored by the second storage means. And a target cost representing the approximate cost between the target phoneme and the phoneme candidate,
Voice unit selecting means for searching for a combination of phoneme candidates which minimizes a cost including a connection cost representing an approximation cost between two phoneme candidates, and outputting index information of the searched combination of phoneme candidates; Based on the index information output from the selection means, the audio segments of the audio waveform signal corresponding to the index information are sequentially read from the first storage means, connected and output, and Voice synthesizing means for synthesizing and outputting a corresponding voice, wherein the voice unit selecting means comprises two consecutive voices.
The absolute value of the sum of the difference between the basic voice frequencies F ₀ of two target phonemes and the basic voice frequency F ₀ of two consecutive phoneme candidates is added to the connection cost. Therefore, an arbitrary phoneme sequence can be converted into a uttered voice without using prosody control rules and without performing signal processing, and a more natural voice quality can be obtained. In addition, speech synthesis can be performed with more natural intonation than in the comparative example.

According to a fourth aspect of the present invention, in the first aspect of the present invention, the voice unit selecting means includes a central value of a basic voice frequency F ₀ of a target phoneme and a voice of a phoneme candidate. When the difference between the median values of the fundamental frequencies F ₀ is equal to or greater than a predetermined threshold, a predetermined penalty cost is further added to the target cost. Therefore, an arbitrary phoneme sequence can be converted into a uttered voice without using prosody control rules and without performing signal processing, and a more natural voice quality can be obtained. In addition, speech synthesis can be performed with more natural intonation than in the comparative example.

Further, in the speech synthesizing apparatus according to the fifth aspect, in the speech synthesizing apparatus according to the first or fourth aspect,
The voice unit selecting means adds the absolute value of the sum of the difference between the basic voice frequencies F ₀ of two consecutive target phonemes and the basic voice frequency F ₀ of two consecutive phoneme candidates to the connection cost. . Therefore, an arbitrary phoneme sequence can be converted into a uttered voice without using prosody control rules and without performing signal processing, and a more natural voice quality can be obtained. In addition, speech synthesis can be performed with more natural intonation than in the comparative example.

[Brief description of the drawings]

FIG. 1 is a block diagram of a spontaneously uttered speech waveform signal connection type speech synthesis apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram of a conventional speech synthesizer.

FIG. 3 is a model diagram showing a definition of a voice unit selection cost calculated by a voice unit selection unit in FIG. 1;

FIG. 4 is a flowchart of a voice analysis process executed by the voice analysis unit of FIG. 1;

FIG. 5 is a flowchart of a first part of a weight coefficient learning process performed by the weight coefficient learning unit in FIG. 1;

FIG. 6 is a flowchart of a second part of the weight coefficient learning process executed by the weight coefficient learning unit in FIG. 1;

FIG. 7 is a flowchart of a voice unit selection process executed by the voice unit selection unit of FIG. 1;

FIG. 8 is a graph showing a listening experiment result of the speech synthesizer of FIG. 1, showing an intonation evaluation result.

FIG. 9 is a graph showing the results of a listening experiment of the speech synthesizer shown in FIG. 1 and showing evaluation results of continuity and clarity.

[Explanation of symbols]

 Reference Signs List 10: voice analysis unit, 11: weight coefficient learning unit, 12: voice unit selection unit, 13: voice synthesis unit, 14: speaker, 21: voice waveform signal database memory, 22: text database memory, 23: phoneme HMM memory, 30: Feature parameter memory 31: Weight coefficient vector

────────────────────────────────────────────────── ─── Continuing on the front page (72) Nick Campbell, Inventor 5 Shiraya, Seika-cho, Soraku-cho, Soraku-gun, Kyoto, Japan 5 AIT R Voice, Inc. Voice Translation and Communication Research Laboratories (72) Inventor Yoshio Higuchi Kyoto 5, Shiraka-cho, Seika-cho, Soraku-gun, Futaba, 5th, Sanraya, ATR Co., Ltd. ATR Corporation Voice Translation and Communication Research Laboratory (56) References JP-A-6-167989 (JP, A) JP-A-8-263095 (JP, A) JP-A-9-204192 (JP, A) JP-A-10-49193 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G10L 13/00-13/06

Claims (12)

(57) [Claims] A first storage unit for storing a speech segment of a naturally uttered speech waveform signal; a speech segment of the speech waveform signal stored by the first storage unit; and a phoneme corresponding to the speech waveform signal. On the basis of the sequence, index information for each phoneme in the audio waveform signal, a first acoustic feature parameter for each phoneme indicated by the index information, and a first acoustic feature parameter for each phoneme indicated by the index information Voice analysis means for extracting and outputting a prosodic feature parameter; index information output from the voice analysis means;
A second phoneme storing the first acoustic feature parameter and the first prosodic feature parameter, and the same phoneme based on the first acoustic feature parameter stored by the second memory. Calculating an acoustic distance in a second acoustic feature parameter between one target phoneme of the type and the other phoneme candidates, and performing a linear regression on the second acoustic feature parameter based on the calculated acoustic distance. A weighting factor learning unit that determines a weighting factor vector for each target phoneme that represents a contribution in the second acoustic feature parameter for each phoneme candidate, and Weight coefficient vector for each target phoneme in the second acoustic feature parameter, and a second predetermined prosodic feature parameter for each phoneme candidate. A third storage means for storing a weighting coefficient vector for each target phoneme representing a contribution in the above, a weighting coefficient vector for each target phoneme stored by the third storage means, and a second storage means And a target cost representing an approximate cost between the target phoneme and the phoneme candidate for the phoneme sequence of the input natural utterance sentence based on the first prosodic feature parameter stored by Voice unit selection means for searching for a combination of phoneme candidates which minimizes a cost including a connection cost representing an approximate cost between two phoneme candidates to be performed, and outputting index information of the searched combination of phoneme candidates; On the basis of the index information output from the audio unit selection means, the audio segments of the audio waveform signal corresponding to the index information are sequentially read out from the first storage means, and are successively read. By and outputs, and a speech synthesis means for outputting synthesized speech corresponding to the phoneme string that is the input, the speech unit selection means, the target phoneme of the voice fundamental frequency F
₀ phoneme candidate speech fundamental frequency F ₀ and the natural speech waveform signal connected speech synthesis apparatus characterized by adding to the target cost of the difference between the slope of the. 2. A first storage means for storing a voice segment of a naturally uttered voice waveform signal; a voice segment of the voice waveform signal stored by the first storage means; and a phoneme corresponding to the voice waveform signal. On the basis of the sequence, index information for each phoneme in the audio waveform signal, a first acoustic feature parameter for each phoneme indicated by the index information, and a first acoustic feature parameter for each phoneme indicated by the index information Voice analysis means for extracting and outputting a prosodic feature parameter; index information output from the voice analysis means;
A second phoneme storing the first acoustic feature parameter and the first prosodic feature parameter, and the same phoneme based on the first acoustic feature parameter stored by the second memory. Calculating an acoustic distance in a second acoustic feature parameter between one target phoneme of the type and the other phoneme candidates, and performing a linear regression on the second acoustic feature parameter based on the calculated acoustic distance. A weighting factor learning unit that determines a weighting factor vector for each target phoneme that represents a contribution in the second acoustic feature parameter for each phoneme candidate, and Weight coefficient vector for each target phoneme in the second acoustic feature parameter, and a second predetermined prosodic feature parameter for each phoneme candidate. A third storage means for storing a weighting coefficient vector for each target phoneme representing a contribution in the above, a weighting coefficient vector for each target phoneme stored by the third storage means, and a second storage means And a target cost representing an approximate cost between the target phoneme and the phoneme candidate for the phoneme sequence of the input natural utterance sentence based on the first prosodic feature parameter stored by Voice unit selection means for searching for a combination of phoneme candidates which minimizes a cost including a connection cost representing an approximate cost between two phoneme candidates to be performed, and outputting index information of the searched combination of phoneme candidates; On the basis of the index information output from the audio unit selection means, the audio segments of the audio waveform signal corresponding to the index information are sequentially read out from the first storage means, and are successively read. By and outputs, and a speech synthesis means for outputting synthesized speech corresponding to the phoneme string that is the input, the speech unit selection means, the target phoneme of the voice fundamental frequency F
₀ and median, when the difference between the center value of the audio fundamental frequency F ₀ of the phoneme candidate is equal to or greater than a predetermined threshold value, the natural speech waveform signal, characterized by adding a predetermined penalty cost target cost Connected speech synthesizer. 3. A first storage means for storing a speech segment of a naturally uttered speech waveform signal, a speech segment of the speech waveform signal stored by the first storage means, and a phoneme corresponding to the speech waveform signal. On the basis of the sequence, index information for each phoneme in the audio waveform signal, a first acoustic feature parameter for each phoneme indicated by the index information, and a first acoustic feature parameter for each phoneme indicated by the index information Voice analysis means for extracting and outputting a prosodic feature parameter; index information output from the voice analysis means;
A second phoneme storing the first acoustic feature parameter and the first prosodic feature parameter, and the same phoneme based on the first acoustic feature parameter stored by the second memory. Calculating an acoustic distance in a second acoustic feature parameter between one target phoneme of the type and the other phoneme candidates, and performing a linear regression on the second acoustic feature parameter based on the calculated acoustic distance. A weighting factor learning unit that determines a weighting factor vector for each target phoneme that represents a contribution in the second acoustic feature parameter for each phoneme candidate, and Weight coefficient vector for each target phoneme in the second acoustic feature parameter, and a second predetermined prosodic feature parameter for each phoneme candidate. A third storage means for storing a weighting coefficient vector for each target phoneme representing a contribution in the above, a weighting coefficient vector for each target phoneme stored by the third storage means, and a second storage means And a target cost representing an approximate cost between the target phoneme and the phoneme candidate for the phoneme sequence of the input natural utterance sentence based on the first prosodic feature parameter stored by Voice unit selection means for searching for a combination of phoneme candidates which minimizes a cost including a connection cost representing an approximate cost between two phoneme candidates to be performed, and outputting index information of the searched combination of phoneme candidates; On the basis of the index information output from the audio unit selection means, the audio segments of the audio waveform signal corresponding to the index information are sequentially read out from the first storage means, and are successively read. By and outputs, and a speech synthesis means for outputting synthesized speech corresponding to the phoneme string that is the input, the speech unit selection means, two goals phonemes successive speech fundamental frequency F ₀ difference and the difference between the natural speech waveform signal connected speech synthesis apparatus characterized by adding the absolute value to the connection cost of the sum of the voice fundamental frequency F ₀ of two consecutive phonemes candidates. 4. The speech unit selecting means, wherein a difference between a median value of the speech fundamental frequency F ₀ of the target phoneme and a median value of the speech fundamental frequency F ₀ of the phoneme candidate is equal to or more than a predetermined threshold value.
2. The speech synthesizer according to claim 1, wherein a predetermined penalty cost is further added to a target cost. 5. The speech unit selection means according to claim 1, wherein the difference between the speech fundamental frequencies F ₀ of the two consecutive target phonemes and the consecutive two
One difference between the sum of the absolute values speech synthesizing apparatus according to claim 1 or 4, wherein adding the coupling cost voice fundamental frequency F ₀ of the phoneme candidate. 6. The voice unit selecting means includes a plurality of top Ns having the best cost including the target cost and the connection cost.
2. The method according to claim 1, wherein after extracting two phoneme candidates, a combination of phoneme candidates having a minimum cost is searched.
6. The speech synthesizer according to any one of claims 1 to 5. 7. The speech analysis device according to claim 1, wherein said speech analysis means includes a phoneme prediction means for predicting a phoneme sequence corresponding to said speech waveform signal based on an inputted speech waveform signal. A speech synthesizer according to one of the above. 8. The weighting factor learning means extracts a best plurality of N1 phoneme candidates based on the calculated acoustic distance, and then performs a linear regression analysis on the second acoustic feature parameter. The speech synthesis according to any one of claims 1 to 7, wherein a weight coefficient vector for each target phoneme, which represents a degree of contribution in the second acoustic feature parameter for each phoneme candidate, is determined by the following. apparatus. 9. The apparatus according to claim 1, wherein the first acoustic feature parameter includes a cepstrum coefficient, a delta cepstrum coefficient, and a phoneme label. . 10. The first prosodic feature parameter is:
Phoneme time length, the voice fundamental frequency F _0, the speech synthesis device according to one of claims 1 to 9, characterized in that it comprises a power. 11. The second acoustic feature parameter is:
(A) a phoneme label of a preceding phoneme preceding the phoneme to be processed, (b) a phoneme label of a succeeding phoneme following the phoneme, (c) a cepstrum distance at a connection point between the phonemes, and (d) a phoneme. The absolute value of the difference in log power between
(E) speech synthesis device according to one of claims 1 to 10, characterized in that it comprises an absolute value of the difference between the voice fundamental frequency F ₀ between phonemes. 12. The second prosodic feature parameter is:
(A) The first phoneme preceding the phoneme to be processed
(B) a first prosodic feature parameter of a phoneme label of a succeeding phoneme following the phoneme, (c) a phoneme time length of the phoneme, and (d) a speech fundamental frequency of the phoneme. The speech synthesizer according to any one of claims 1 to 11, comprising F ₀ and (e) a speech fundamental frequency F ₀ of the preceding phoneme.