JP3346671B2 - Speech unit selection method and speech synthesis device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech synthesizer,
In particular, the present invention relates to a technique for selecting a synthesis unit (speech waveform unit).

[0002]

2. Description of the Related Art Conventionally, in a rule synthesizing apparatus using speech waveform segments, a large amount of speech waveforms are converted to PCM (Pulse Code Modulat).
Ion) digital sound waveform data (PCM
In general, the audio waveform data is described with positional information, pitch shape, duration length, prosody information such as power, and types of preceding and succeeding phonemes, etc., for the speech waveform data. It is. When performing speech synthesis, a prosody pattern is set according to rules for the input text, a speech unit is selected so as to be closest to the prosody pattern, and the prosody of the selected unit is further transformed. By making the prosody pattern close to the target prosody pattern, it is possible to perform high-quality regular speech synthesis as compared with the case where synthesized speech using LSP (Line Spectrum Pair) is performed. For a rule synthesizing apparatus using such speech waveform segments, see Hirokawa, "Rule synthesis method using waveform dictionary, IEICE Technical Report SP88-9 (198
8-5)) can be referred to.

[0003]

In the above-mentioned speech synthesis technique, since the speech waveform is connected on the PCM data, the spectrum shapes between the segments are greatly different, which may cause abnormal noise or noise at the segment connection. May be. In a rule synthesizing apparatus using speech waveform segments, the quality of synthesized speech is greatly influenced by the presence or absence of noise at such segment connection sections. However, according to the conventional method, since the stored information does not have parameters of natural speech such as a spectrum shape, only matching of the prosodic smoothness and the phoneme sequence can be used as a reference for waveform selection. Prediction of noise contamination in the section cannot be made. For this reason, there is a problem that it is impossible to judge whether the trade is good or bad without actually listening to the synthesized speech by a human or performing some kind of acoustic analysis.

In view of the above problems, an object of the present invention is to make it possible to select a unit in consideration of continuity between synthesis units when selecting a synthesis unit in a rule synthesizing apparatus using a speech waveform unit. It is still another object of the present invention to provide a technique capable of evaluating the presence / absence of noise mixing in a unit connection unit and the quality of the entire synthesized speech.

[0005]

The present invention first provides a speech unit selection method. According to this method, when selecting a synthesis unit to be used for speech synthesis from a segment dictionary in which a plurality of speech segments are stored, specific information of a cepstrum distance between segments, for example, a representative value of the distance is used for each of the synthesis units. Accumulating in a single dictionary, extracting a plurality of synthesis units similar to the prosodic pattern of the synthesis target voice in descending order of similarity, and connecting cost based on the inter-unit cepstrum distance from the extracted plurality of synthesis units. And selecting the minimum combination unit combination in this order. When the quality of the synthesized speech based on the selected set of synthesis units is larger than a predetermined reference value, the synthesis unit having a high connection cost is replaced with an alternative unit having a minimum cepstrum distance between adjacent units. Note that the quality of the synthesized speech in this case can be evaluated by, for example, an integrated value of the cepstrum distance between the segments, and the evaluation threshold can be experimentally determined.

The present invention also provides a speech synthesizer suitable for implementing the above method. The apparatus includes a unit dictionary storing a plurality of speech units together with specific information of an inter-unit cepstrum distance, a speech unit selection unit that selects a synthesis unit to be used for speech synthesis from the unit dictionary, Means for generating a synthesized speech based on a synthesis unit, wherein the speech unit selection unit extracts a plurality of synthesis units similar to the prosodic pattern of the synthesis target speech in descending order of similarity. It is characterized by having a primary selection unit and a secondary selection unit that selects a set of synthesis units that minimizes the connection cost based on the inter-unit cepstrum distance from a plurality of extracted synthesis units.

In the speech synthesizing apparatus having the above-described configuration, the speech unit selection unit further quantifies the quality of the synthesized speech based on the set of synthesis units selected by the secondary selection unit and obtains the quality. A synthesis quality determination unit that determines the quality of the quantitative value, and an alternative unit generation that generates an alternative unit that minimizes the cepstrum distance between adjacent units in the set of the synthesis unit when the quality value determination is negative. And replacing the synthesis unit which increases the connection cost with the alternative element. The replacement unit generating unit includes a first unit that selects a plurality of preceding sound replacement candidates and a succeeding sound replacement candidate whose prosody pattern is close to the target value, and a first sound replacement candidate that has the smallest connection cost of the adjacent unit. And a second means for specifying the set as a substitute element.

[0008]

According to the present invention, similar to the conventional method, a candidate having a high degree of similarity to a prosody pattern is extracted as a synthesis unit candidate.
The cepstrum distance between the element connection parts is taken as the connection cost,
For example, using a dynamic programming method, a combination of synthesis units that minimizes the connection cost is selected. As a result, a unit having the highest continuity of vocal tract characteristics can be selected from a given synthesis unit. In this case, by referring to the connection cost when the synthesis unit is selected, it is possible to estimate the state of noise mixing in the synthesized speech. In addition, since the connection cost is high and noise or noise is likely to be mixed in a portion that causes a reduction in the synthesis quality, if there is a synthesis unit having a high connection cost, another replacement unit is used instead. Replace with

[0009]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. FIG. 1 is a configuration diagram of a speech synthesizer according to one embodiment of the present invention. First, an overview of the overall processing of the speech synthesizer of the present embodiment will be described with reference to FIG. The speech synthesizer 1 sends a Japanese text mixed with kanji and kana input from an input terminal IN to a text analyzer 2. The text analysis unit 2 divides the input text into phrases using a dictionary (not shown), gives each phrase a reading in Roman alphabet, and further gives an accent type to each phrase. The reading and accent type of each phrase obtained here are
It is sent to the prosody generation unit 3.

The prosody generation unit 3 generates three types of prosody patterns, a pitch pattern, a phoneme duration time pattern, and a power pattern, based on the reading information and the accent type information in the unit of a phrase.
The generated prosody pattern and the romaji reading information in units of phrases are sent to the segment selection unit 4. In the unit selection unit 4,
A synthesis unit suitable for use in synthesis is selected from the segment dictionary 5 in consideration of the given text reading information and prosody pattern. This will be described later in detail. The synthesis unit selected by the unit selection unit 4 is sent to the unit transformation connection unit 6. The unit transformation connection unit 6 performs a transformation process on the prosody pattern obtained by combining the synthesis units selected by the unit selection unit 4 so as to be close to the prosody pattern generated by the prosody generation unit 3. The latter element is connected and sent to the output terminal OUT. Note that the speech synthesizer 1 of the present embodiment
It is assumed that a plurality of synthesis units are prepared for each VCV unit type in order to use a CV type speech waveform segment unit (synthesis unit) and improve synthesis quality.

Next, an example of the configuration of the segment dictionary 5 used in this embodiment will be described with reference to FIG. The speech synthesizer 1 converts a large amount of speech waveforms into PCM data, and accumulates the speech waveforms in the waveform dictionary 5 in correspondence with phonological information, prosodic information, and spectral information. Then, based on the prosody pattern of the input text estimated by the text analysis unit 2, an optimal synthesis unit is selected from the segment dictionary 5,
The synthesized speech is obtained by connecting the selected synthesis units smoothly.

In the same conventional speech synthesis system, since there is no spectrum information as dictionary information, only a match between prosodic smoothness and a phoneme sequence can be used as a reference for selecting a waveform. Therefore, in the present embodiment, cepstrum analysis is performed when the segment dictionary 5 is created, and a low-order term of the cepstrum appearing there is used as a feature vector, and clustering is performed using all vectors other than a silent section.
After the clustering process, an identification number is assigned to each cluster, and the cluster to which the cepstrum at the start / end position of each synthesis unit belongs is registered as dictionary information. As a result, it is possible to monitor the segment continuity at the spectrum envelope level and predict in advance the occurrence of abnormal noise and the inclusion of noise.

As shown in FIG. 2A, for example, the unit dictionary 5 includes, in a unit dictionary table 201, a unit number for uniquely treating a unit and a phoneme type indicating a phoneme environment of the unit. Is stored. Also, as the prosody information, a pitch frequency at a segment boundary, segment duration information, and power are stored. As the analysis position of the pitch frequency, the pitch frequency at the segment start position and the segment end position in VCV units is used. In addition, as the duration information, the segment start position,
Using the information on the consonant start position and the end position of the segment, and holding the information such as the cepstrum distance table 701, which will be described later, which allows the length of the entire segment itself and the boundary position between the preceding vowel and the succeeding syllable The stored representative value of the distance (representative value of the inter-element cepstrum distance) is also stored. As the power information, information of order 0 of the cepstrum is used. The low-order terms of the cepstrum are stored in a table format so as to correspond one-to-one with the category numbers when clustered.
This will be described later. If there is sufficient storage capacity, cepstrum information need not be clustered. Regarding the analysis position, the analysis is performed at the element start and end positions also in the cepstrum.

In the example of creating a composite unit from “Aka” / aka / in FIG. 2B, the composite unit “aka” 210
Is created, the pitch frequency at the corresponding segment start position 206 and segment end position 207 is analyzed and used.
However, when there is no preceding or succeeding vowel obtained from the beginning or the end such as “#a” 209 or “a #” 211, the pitch frequency is uniquely set to 0 Hz. Note that / # / in the combined unit notation represents a silent section such as a pause. Also, the start position and the end position are the same positions as the positions where the pitch frequency is analyzed, and the synthesis unit “aka”
The consonant start position 210 uses the information of the syllable boundary 203. However, even when there is no consonant in the vowel concatenation, the start position of the syllable is the consonant start position.

Next, the detailed configuration of the segment selection unit 4 will be described with reference to FIGS.
This will be described with reference to FIG. The unit selection unit 4 receives the Romanized notation and the prosodic pattern of the Japanese text input to the speech synthesizer 1. Here, first, the search section extraction unit 41 extracts all types of VCV units used for synthesis from Romanized notation. Based on the determined VCV unit type, the search section search table 40 of the content example shown in FIG.
1, a search section in the segment dictionary 5 is determined. Next, the processing is shifted to the primary selection unit 42.

The primary selection unit 42 selects a segment suitable for use in the synthesis based on the pitch, duration and power prosody information in the search section already obtained for each synthesis unit. When determining the synthesis unit suitable for synthesis using the prosody information, the prosody pattern determined by the prosody generation unit 3 is compared with the prosody pattern of the speech unit to calculate an evaluation value. As an evaluation criterion, an expression is used in which an evaluation value becomes smaller as the target prosody pattern becomes more similar. An example of an equation for obtaining the evaluation value Hev is shown below.

[0017]

Hev = P + D ² + E ² (1) where P = ｛(previous vowel pitch target value−unit preceding vowel pitch)
² + (previous vowel pitch target value-segment preceding vowel pitch) ² Ｄ D = (duration length target value / fragment duration time) E = (power target value / fragment power)

Each parameter of the evaluation formula Hev is obtained from the prosody generation unit 3 and the segment dictionary 5. Using this evaluation formula Hev, all the synthesis units in the search section are evaluated, and registered in the candidate table 501 having the contents shown in FIG. In the present embodiment, a unit close to the prosody pattern of the speech to be synthesized is extracted from the synthesis unit having the same type and many prosody variations using Expression (1). According to the evaluation value of equation (1),
It is possible to perform an order of closeness with respect to a synthesis unit in the dictionary, and a smaller evaluation value is closer to a target prosody pattern and is registered as a higher candidate. This process is performed for each synthesis unit used for synthesis.

The candidate table 501 shown in FIG. 5 shows an example in which "red" / aka / is synthesized.
In this case, the necessary synthesis units are / # a /, / aka /,
/ A # /. For each synthesis unit, the target prosodic pattern and the corresponding type of segment in the dictionary are evaluated using equation (1), and the one with the smallest evaluation value is defined as the first candidate. I will register as. When registering, it is described using a unit number that uniquely points to the unit. It is also possible to add a limit on the number of candidates, and it is not necessary to prepare the same number of candidates for each type of synthesis unit. Here, by setting an upper limit on the number of candidates, the processing in the secondary selection unit 43 in the subsequent stage can be reduced. Further, the number of candidates can be limited by setting an upper limit threshold value for the evaluation value Hev. The above processing is applied to all synthesis units used for synthesis to generate information to be stored in the search section table 401, and sends this to the secondary selection unit 43.

The secondary selection unit 43 selects a unit (synthesis unit) from the synthesis unit candidates obtained by the primary selection unit 42 so as to minimize the distortion of the unit connection. The processing concept will be described with reference to FIGS. Secondary selection unit 43
Then, from the combination unit candidates determined by the primary selection unit 42,
A synthesis unit to be finally used for synthesis is selected in consideration of the continuity of the spectrum. The selection of the synthesis unit uses the cepstrum distance between the segments as the connection cost, and selects a path so that the connection cost between the segments is minimized.

The selection of the minimum path can be efficiently realized by, for example, dynamic programming (DP). In the example of FIG. 6, the connection cost of the path indicated by the thick line is the minimum, and as a selection result, / # a / is selected as the first candidate, / aka / is selected as the second candidate, and / a # / is selected as the first candidate. ing. The path connection costs are CD (cepstrum distance, the same applies hereinafter) 1, CD2, CD
.., CDn, and these are used for determination of the combined quality described later. Further, the distance between the clusters is held in the form of a cepstrum distance table 701 shown in FIG. 7, and the calculation of the connection cost in the secondary selection unit 43 is simplified. The cost of the connection distortion minimum path obtained by the above processing is sent to the combined quality determination unit 44.

In the synthesis quality judging unit 44, the RMS value (root means s) of the connection cost in the subsequent unit deformation connection unit 6
quare) Rc is calculated, and if the value is smaller than a preset value, the process is transferred to the combining unit extracting unit 46. On the other hand, if Rc is larger than the threshold value, the processing is shifted to the alternative unit generation unit 45. When Rc is calculated by the combined quality determination unit 44, for example, Expression (2) can be used.

[0023]

In general, the connection cost is high in the portion that causes the deterioration of the synthesis quality, and abnormal noise or noise is likely to occur. Therefore, in such a case, it is necessary to replace a segment having a high connection cost with a segment created by combining other segments. The alternative unit generating unit 45 performs a process for this purpose, and inputs the type of unit requiring the alternative unit, the prosody pattern, and the cepstrum information of the preceding and following units. In the alternative unit generating unit 45, the units in which the value of Rc is increased are sequentially stored as alternative unit replacement candidates. The alternative unit generation information is sequentially generated based on the generated replacement candidate order, and the alternative unit generation processing is stopped when the value of Rc falls below the threshold value. Then, the substitute unit generation information at this time is transmitted to the synthesis unit extraction unit 46. In addition,
When a replaceable unit cannot be generated, the unit candidate obtained by the secondary selection unit 43 is directly used as the synthesis unit extraction unit 4
6 is output.

The details of the processing in the substitute unit generation unit 45 will be described with reference to FIGS. Referring to FIG. 8, when the obtained connection cost is larger than a preset threshold value, alternative processing determining section 451 determines whether consonant C of the replacement target VCV unit is voiced or unvoiced. It is determined whether or not there is, and if the sound is unvoiced, a substitute unit generation process is performed.

The alternative unit candidate extraction section 452 includes a preceding vowel V portion 901 and a succeeding syllable C in VCV units shown in the upper part of FIG.
The V portion 902 is handled as a minimum unit. Regarding the preceding vowel V, the vowel type and the succeeding consonant C match, and the primary vowel V whose prosody pattern is close to the target value is selected by the primary selection unit 4.
As in the case of 2, the candidate ranking is set using the equation (1). Also,
In the succeeding syllable CV, a candidate whose prosody pattern is close to the target value is selected in the same manner as the primary selection unit 42 regardless of the preceding vowel V, and the candidate order is set. The candidate for the preceding vowel V and the candidate for the succeeding syllable CV obtained here are respectively substituted for the alternative unit selection unit 453.
To send to. The alternative unit selection unit 453 adopts, for each candidate of the preceding vowel V and the following syllable CV, the candidate having the smallest connection cost with each adjacent unit as the replacement unit. The unit generation unit 454 combines the preceding vowel replacement unit and the succeeding syllable replacement unit obtained here to generate an alternative unit by combining them as shown in the latter part of FIG. The synthesis unit extraction unit 46 extracts the real data (PCM data) of the unit from the unit dictionary 5 and outputs it to the unit transformation connection unit 6 together with alternative unit generation information or unit candidates.

As described above, according to the present embodiment, it is possible to select a synthesis unit in consideration of both the prosody pattern and the continuity of the spectrum envelope in the unit combining unit. By using the integrated value of the cepstrum distance as an evaluation criterion, it is possible to predict the presence or absence of noise contamination that causes a reduction in synthesized sound quality.
Further, when the connection cost is high, other alternative segments in the segment dictionary 5 are replaced with segments causing quality deterioration, so that a synthesized speech of a certain quality or more is always generated. Becomes possible. Note that the segments (synthesis units) and parameter values used in the present embodiment are used for convenience in describing the present invention, and are not necessarily limited to such examples. .

[0028]

As is apparent from the above description, according to the present invention, it is possible to select a synthesized voice having the highest continuity of the vocal tract characteristics and to estimate a noise mixing state in the synthesized voice.
Conventional problems can be solved. In addition, since the synthesis unit causing the deterioration of the synthesis quality is replaced with another alternative unit that contributes to the quality improvement, the quality of the synthesized voice is always higher than a certain level. There is. The present invention can be applied to a speech synthesizer connected to a personal computer or the like or speech synthesis software, and can also be easily applied to various services such as ticket reservation using a telephone. .

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of a speech synthesizer according to one embodiment of the present invention.

FIG. 2A is an explanatory diagram of table contents in a segment dictionary used in the embodiment, and FIG. 2B is an explanatory diagram of a synthesis unit.

FIG. 3 is a block configuration diagram of a segment selection unit according to the embodiment.

FIG. 4 is an explanatory diagram showing an example of a catch table used in the segment selection unit of FIG. 3;

FIG. 5 is a conceptual explanatory diagram of a table used when determining a section in which to search a segment dictionary according to the embodiment;

FIG. 6 is a conceptual diagram in a case where a segment selection is performed by a secondary selection unit of the segment selection unit according to the embodiment.

FIG. 7 is a conceptual diagram of a cepstrum distance table used in a segment selection unit in FIG. 3;

FIG. 8 is a block diagram of a replacement unit generating unit according to the embodiment.

FIG. 9 is a conceptual explanatory diagram when generating a substitute unit according to the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Speech synthesizer 2 Text analysis part 3 Prosody generation part 4 Unit selection part 5 Unit dictionary 6 Unit transformation connection part

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-3-119394 (JP, A) JP-A-5-73092 (JP, A) JP-A 1-28893 (JP, A) (58) Field (Int.Cl. ⁷ , DB name) G10L 13/06

Claims (7)

(57) [Claims] 1. A speech unit selection method for selecting a synthesis unit to be used for speech synthesis from a unit dictionary in which a plurality of speech units are stored, wherein specific information of an inter-unit cepstrum distance is provided for each synthesis unit. And extracting a plurality of synthesis units similar to the prosodic pattern of the synthesis target voice in descending order of similarity. The connection cost based on the inter-unit cepstrum distance from the extracted synthesis units is minimized. Selecting a combination of synthesis units , and further selecting a combination of synthesis units that minimizes the connection cost.
In the step, the low-order terms of the cepstrum are characterized.
Using all vectors except silence
Distance between each cluster obtained by performing rastering
Is used to determine a set of synthesis units that minimizes the connection cost.
A method for selecting a speech unit, which comprises selecting. 2. The method according to claim 1, further comprising : determining a start position and an end position of each synthesis unit.
The cluster information to which the pstrams belong is
2. The method according to claim 1, further comprising the step of performing registration.
How to select speech unit. 3. When the quality of a synthesized speech based on the selected set of synthesis units is greater than a predetermined reference value, a synthesis unit having a high connection cost is replaced with a minimum cepstrum distance between adjacent segments. 2. The speech unit selection method according to claim 1, wherein the speech unit is replaced with a unit. 4. A unit dictionary in which a plurality of speech units are stored together with specific information of an inter-unit cepstrum distance, and a speech unit selection unit for selecting a synthesis unit used for speech synthesis from the unit dictionary. Means for generating a synthesized speech based on the synthesized unit, wherein the speech unit selection unit extracts a plurality of synthesis units similar to the prosodic pattern of the synthesis target voice in descending order of similarity. A primary selection unit that selects a set of synthesis units that minimizes the connection cost based on the inter-unit cepstrum distance from the extracted plurality of synthesis units, and further includes a connection selection unit.
In selecting the set of composite units that minimizes the cost,
The low-order terms of the
By performing clustering using all the vectors
The connection cost is minimized using the distance between each cluster
A voice synthesizing device, wherein a set of small synthesis units is selected . 5. The unit dictionary further includes:
To which cluster the cepstral at the start and end positions belongs
Is registered as dictionary information.
The speech synthesizer according to claim 4. 6. The speech unit selection unit further quantifies the quality of a synthesized speech based on the set of synthesis units selected by the secondary selection unit, and determines whether a quantitative value obtained thereby is good or bad. A synthesis quality determination unit for determining, and a replacement unit generation unit that generates a replacement unit that minimizes the cepstrum distance between adjacent units in the combination of the synthesis units when the determination of the quality value is negative. 5. The speech synthesizer according to claim 4 , wherein a synthesis unit having a high connection cost is replaced with said substitute unit. 7. The alternative unit generating unit includes: first means for selecting a plurality of preceding sound replacement candidates and subsequent sound replacement candidates whose prosody pattern is close to a target value; 7. The speech synthesizer according to claim 6 , further comprising: second means for specifying a set of sound replacement candidates as the replacement unit.