JP3278486B2 - Japanese speech synthesis system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for synthesizing moras, which is a unit of prosody corresponding to a single syllable in Japanese speech. In particular, it is an object of the present invention to adjust the time length between mora and to facilitate natural synthesis of Japanese speech.

[0002]

2. Description of the Related Art Conventionally, as a technique in such a field, CO2 is used.
Prototype of a text synthesis board using C (Context Oriented Clustering) method (Technical research report of IEICE:
(November 30, 1990).
The COC method described in this document is a method in which a phoneme including various variations generated by a statistical method from a database of natural speech is used as a synthesis unit.

[0003]

According to the conventional Japanese speech synthesis system, the following problems occur. FIG. 15 is a diagram for explaining conventional control of phoneme duration. As shown in this figure, a method conventionally considered as a phoneme duration control method is a control method that gives a time length for each phoneme.
It can be seen that the time of each phoneme is really variable. When controlling the duration of phonemes using this method, factors to be considered include the type of own sound,
The type of the preceding sound, the type of the preceding sound, the type of the preceding sound, the type of the preceding sound, the position in a word, the position in a sentence, the presence or absence of an accent, etc. Etc .: "Setting of phoneme duration in sentence speech" IEICE Technical Report, SP90
-2 (Heisei 02-04)). As described above, the length of time that appears differently depending on the type of phoneme and how it is connected is called the rhythm of words. When synthesizing natural speech, it is necessary to give each phoneme an accurate time length. In other words, since the rhythm is different for a phoneme that corresponds to a phoneme connected before and after the phoneme, various phoneme data need to be accumulated, and a necessary portion must be cut out from the data and used. However, the combination of these factors alone is an astronomical number and cannot be considered in all cases. We also know that these factors alone are not enough. For this reason, there has been a problem that it is necessary to accumulate a huge amount of data, and it is difficult to improve the quality of the synthesized speech sound in Japanese because complicated processing is required.

Accordingly, an object of the present invention is to provide a Japanese speech synthesis system capable of easily synthesizing natural speech in view of the above problems.

[0005]

SUMMARY OF THE INVENTION In order to solve the above problems, the present invention provides a Japanese speech synthesis system for synthesizing a mora which is a unit of prosody corresponding to a single syllable. Dvp, the time length of the consonant part between the two mora is dc, the first half time length of the succeeding vowel between the two mora is dvs, and the constant determined by the preceding vowel is Avp, Bvp, When the determined constants are represented by Ac and Bc, and the constants determined by the succeeding vowels are represented by Avs and Bvs, the time interval Dsegv between the energy centroids of the respective vowel parts in two adjacent mora is expressed as follows.

(Equation 3) By applying the following equation, the second half time length dv of the preceding vowel part
p, means for determining the duration dc of the consonant part and the first half duration dvs of the succeeding vowel part. Further, with respect to the time interval Dsegv,

(Equation 4) By applying the following equation, the second half time length dv of the preceding vowel part
p, means for determining the duration dc of the consonant part and the first half duration dvs of the succeeding vowel part. Further, the constant Avp determined by the preceding vowel and the constant Bvp determined by the succeeding vowel may divide the type of vowel into a plurality of groups, and each group may use a constant value. .

[0006]

According to the Japanese speech synthesis system of the present invention for forming vowel length / consonant length rules between vowel part energy centroids, the energy of the speech waveform of any two adjacent mora is determined, and When the time integration of the energy of the vowel part of the waveform is taken and the mora interval is determined by the time length between the vowel part energy centroid points between the centroid points of the time integration of the energy of the two moras, the preceding and following vowels change. Even though, the time length between the vowel part energy centroid positions hardly changes. Even if the preceding consonant changes, the time length between the vowel part energy centroid positions hardly changes. Further, the length of the vowel and the length of the consonant constituting the mora are determined using the time length between the vowel part energy centroid positions and the consonant length as parameters. Therefore, the mora interval, vowel length and consonant length constituting the mora can be obtained quantitatively and accurately. Specifically, the vowel length and consonant length constituting the mora are quantitatively determined as a linear function of the time length between the positions of the vowel part energy centroids, and the vowels constituting the mora are grouped into a plurality of groups. in the process of quantifying the mora interval is simplified by forming the number of primary functions. Therefore, by adjusting the phoneme duration of a sentence to be synthesized at the aforementioned mora interval, a natural Japanese synthesized sound can be easily obtained.

[0007]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration of a Japanese speech synthesis system according to an embodiment of the present invention. The Japanese speech synthesis system shown in this figure analyzes the Japanese sentence to be synthesized when a synthesized sentence is input, and generates pronunciation information such as accent information, pause, and vowel de-voice necessary for speech synthesis processing. Analysis unit 1 for converting to a phoneme symbol string to which is added
A phoneme duration generator 2 for controlling the phoneme duration of the phoneme symbol string generated by the text analyzer 1 according to a rhythm rule described later; and a phoneme duration generator 2.
Of the vowel energy given by the rhythm rule of Dcegv (The Center of Energy Gravity of Vo
The sound amplitude pattern generation unit 3 that determines the power pattern of the voice according to the power rule so as to observe the wels), and the number of mora that are the units of the prosody corresponding to a single syllable for each accent phrase from the prosody control rule A pitch pattern generation unit 4 that determines a point pitch pattern from an accent type and interpolates them to generate a continuous point pitch pattern, and a sound source generation unit that generates a sound source based on the power pattern and the continuous point pitch pattern 5, a phonological improvement rule, and a phonological combination rule, determine a type of phonology such as a vowel or a consonant, combine the spectra of each phonology to form a formant pattern, and generate a formant pattern. A speech synthesizer 7 for creating a synthesized speech from the formant pattern.

FIG. 2 is a block diagram showing a configuration of the phoneme duration generating unit 2 of FIG. The phoneme duration generation unit 2 shown in FIG. 2 includes a VCV partitioning unit 21 for partitioning a phoneme symbol string generated by the text analysis unit 1 into VCV (vowel / consonant / vowel) partition units; Consonant and speech rate SR (SpeechRa
te) is used as a parameter to calculate a vowel part energy center-of-gravity point time length Dcegv from a rhythm rule stored therein, and to set each section part for a vowel part energy center-of-gravity point time length setting unit 2
2 and the time length Dcegv between the vowel part energy centroids, the duration time of each vowel / consonant existing between Dcegvs is determined, and the result is used as the sound amplitude pattern generation unit 3, the pitch pattern generation unit 4, the sound source A vowel length / consonant length setting unit 23 in Dcegv to output to the generation unit 5 is provided.

Next, a rhythm rule for supplying the vowel part energy centroid point time length Dcegv to the phoneme duration time generation unit 2 which is a characteristic part of the present invention will be described in detail. The present inventors, in order to easily synthesize a high quality Japanese voice, <br/> the applied child as prosodic rules found rhythm laws in Japanese is considered essential. This is because, despite the efforts of many researchers, rhythm rules have not been found. The structure of Japanese language can be CV (consonant / vowel), VCV (vowel / consonant / vowel), CVC (consonant / vowel / consonant), and the like. Is given as the time length between VCV (vowel / consonant / vowel) vowel part energy centroids, and within a certain tempo range, it is almost determined by the type of consonant existing between vowels. It became clear.

First, the description will be made based on the following hypothesis of Japanese rhythm. That is, even a very small child can correctly form a sense of rhythm unique to Japanese. Also, no matter what the mora series, a person can pronounce with a natural rhythm. From these phenomena, Japanese rhythm rules should be very simple.
About this, Kaneda Kazu (Japanese Phonological Research (196
7) Tokyodo Shuppan) points out the following rules.

[Rule] Japanese rhythm is taken in mora units and is basically governed by a single concept of isochronism. In addition, when considering the case of uttering slowly, it is understood that the rhythm is taken in the vowel part in the mora. In addition, it is believed that people are listening to that energy. Therefore, the physical rhythm can be defined as follows.

[Definition] The rhythm of Japanese is given by the time length between the vowel energy centroids in the mora. The Japanese rhythm is basically isochronous, but the biggest factor that disturbs it is thought to be the additional physical constraints imposed by the structure of the vocal organs. This constraint is clearly greater when consonants are uttered than when vowels are uttered, and the time length between vowel energy centroids seems to be greatly dependent on differences in consonants (27 types).

FIG. 3 is a diagram showing time length control focusing on the time length between the vowel part energy centroids. This will be described in detail with reference to this drawing. A Japanese mora using a voice waveform in the drawing generally includes a consonant part and a vowel part. As described above, since the timing of the rhythm is taken in the vowel part in the mora, attention is paid particularly to the energy of the vowel part in the mora. That is, the energy of the vowel part is determined, and the point at which the center of gravity of the part is given is defined as the vowel part energy centroid point. One vowel energy centroid is determined for each syllable, so if the time length between the vowel energy centroids is t1, t2, t3, t4 as shown in the figure, the time length between mora can be given. .

In the time length control method focusing on the vowel part energy centroid, it is not necessary to consider the difference in the type of vowels because the energy of the waveform is taken into account. Consider a consonant part existing between the vowel part energy centroids. Consonants have less energy than vowels, but they have great physical limitations such as ease of pronunciation and difficulty. I mentioned earlier that the Japanese rhythm rule is based on isochronism in energy units. The time length between the vowel energy centroids should be equidistant in nature, but this isochronism is greatly disturbed by the ease and difficulty of generating consonants that exist between them. It can be thought that it appears as various time lengths.

Therefore, the following hypothesis is provided and verified through analysis of actual speech. [Hypothesis] Time length between vowel energy center of gravity (rhythm)
Is determined only by the type of consonant between target vowels below a certain tempo, and does not depend on the type of vowels before and after. Next, audio materials and analysis methods will be described. As audio material, 7
One sentence containing the meaningless word part of Mora, "It's" This is a OO Menkai "" was used. There are two speakers, one male and one female.

Assuming that Ci is the i-th consonant and Vi is the i-th vowel, the circles in the meaningless word are C2 V2,
It can be represented as C3 V3. In order to verify the previous hypothesis between the second and third mora, [Analysis 1] when the previous vowel V2 is different, [Analysis 2] when the rear vowel V3 is different, and [Analysis 3] the previous consonant C2 The analysis of the time length between the vowel part energy centroids was performed in four cases, that is, when the consonants C3 sandwiched between [Analysis 4] were different.
Ten utterances are made for each phoneme type, and the average time length and the standard deviation of the time length between the vowel part energy centroid positions are obtained. At this time, in order to absorb a subtle difference in tempo that occurs for each utterance, the duration of the meaningless word portion, that is, the utterance speed was normalized to 7 mora / sec by linear expansion and contraction.

Since the same tendency is generally observed in the analysis results of men and women, only the case of one person (female voice) will be described below. FIG. 4 shows a case where the previous vowel is different,
FIG. 5 is a diagram showing the influence of the time length between the vowel part energy centroids in the case of FIG. As shown in FIG. 3, consonants 1, vowels 1, consonants 2, and vowels 2 are arranged in this order, and consonants 1, 2, and vowels 2
Is fixed, and the vowel 1 is variously changed, and the time length between the vowel part energy centroids is measured. As shown in FIG. 5, ◆ is the average time length between the vowel energy centroids, the vertical line is the standard deviation, and the horizontal axis is the phoneme of the second and third mora eyes, for example, “ baba, biba, buba, beba,
boba, Nba ". FIG. 5 shows the time length between the vowel part energy centroids when the previous vowel V2 is different. The average time length is 139 to 145 ms.
It is distributed over a width of about ± 3.5 ms. From the fact that the discrimination limit of the phoneme duration is about 10 to 20 ms (see: Hashimoto et al., 7th International Conference on Acoustics, 129-132 (197)
1)), it can be said that there is almost no influence by the difference of the previous vowel V2.

FIG. 6 is a diagram showing the case where the rear vowel is different, and FIG. 7 is a diagram showing the effect of the time length between the gravities of the vowel parts in the case of FIG. Similarly to the above, as shown in FIG. 6, consonants 1, 2, and vowel 1 are fixed and vowel 2, for example, as shown in FIG. 7, "baba, babi, babu, babe, babo, baN
The time length between the vowel part energy centroids is measured by variously changing “de”. As shown in FIG. 7, even if the rear vowel V2 is different, the distribution of the average time length between the vowel energy centroids is 140 to 154 ms, ± about 7 ms, and the difference of the rear vowel V3 is the same as the previous vowel. It can be seen that there is almost no influence by the influence.

FIG. 8 is a diagram showing the case where the previous consonant is different, and FIG. 9 is a diagram showing the effect of the time length between the gravities of the vowel parts in FIG. Similarly to the above, the vowels 1 and 2 and the consonant 2 are fixed as shown in FIG. 7 and the consonant 1 is variously changed to, for example, "byaba, baba, gyaba,..., Naba" as shown in FIG. The time length between the local energy centroids is measured. As shown in FIG. 8, even when the consonant C2 is different, the average time between the vowel energy centroids is 121 to 153 m.
s, ± 16 ms, and the discrimination limit (10 to
Considering the region of 20 ms), it is understood that the influence is exerted though it is small.

FIG. 10 is a diagram showing the case where the consonants are different, and FIG. 11 is a diagram showing the influence of the time length between the vowel part energy centroids in the case of FIG. Similarly to the above, as shown in FIG. 10, the vowels 1, 2, and the consonant 1 are fixed, and the consonant 2 is variously changed to, for example, "bara, bada, bawa,..., Bahya". The length is measured. FIG.
As shown in FIG. 1, consonant 2 is sandwiched between vowels to be analyzed,
If they are different, the average time length distribution width between the vowel energy centroids is 113 to 195 ms, ± about 41 ms, which clearly exceeds the discrimination limit (10 to 20 ms).

From the above results, the average time length is concentrated around 140 ms in both the effect of the difference in the type of the preceding vowel (FIG. 5) and the effect of the difference in the type of the rear vowel (FIG. 7). Although there is a slight variation among vowel types, it is concluded that the effects of vowel types are negligible considering the discrimination limit, that is, the fact that a human cannot perceive a change in sound length of about 10 to 20 ms. it can. On the other hand, the effect of the difference in consonants between (FIG. 11) is about 110-110.
It was widely distributed at 190 ms, and the magnitude of the effect due to the difference in type was confirmed.

FIG. 12 is a diagram for explaining a method for deriving the time length between the vowel part energy centroids. In the i-th mora and (i + 1) -th mora sound waveforms each having a consonant part and a vowel part as shown in FIG. 6A, a sound energy E representing the sound waveform as shown in FIG. Then, the sum (integration) of the voice energy as shown in FIG.
1) Sum the total energy of Mora as Etotal (i), Eto
tal (i + 1), ＥEtotal (i), ＥEtotal (i + 1) of the i-th and (i + 1) -th mora
Is defined as the center of gravity of the vowel energy of the i-th mora and the (i + 1) -th mora, and the interval between them is defined as the time length between the vowel energy centroids.

The table below shows the time length between the vowel energy centroids for various consonants derived by the above method. In the above, the rhythm of Japanese speech is defined as the vowel part energy center-of-gravity point time length (Dcegv) and is regularized.
Rules for vowel length / consonant length distribution in cegv have not been determined, and conventionally, individual vowel lengths / consonant lengths could not be determined regularly. Based on the analysis results of the vowel length and consonant length in Dcegv below, these are stored in the rhythm rule of FIG. 2 as rules, and individual vowel lengths and consonant lengths are set in the Dcegv inner vowel length / consonant length setting unit 23. .

FIG. 13 is a diagram for explaining a method of setting the time length in Dcegv. As shown in the figure, the inside of Dcegv is divided into three parts: a preceding vowel second half dvp, a consonant part dc, and a subsequent vowel first half dvs. The values of dvp, dc, and dvs were obtained for various Dcegv values obtained by changing the utterance speed. As described above, the audio material contains a 7-mora nonsense word part, which is a sentence "It is"-/ bx /-/ kx / -Menkai "(X is an arbitrary vowel). The analysis was performed between the second and third mora in the meaningless word. The speaker is a single woman.

FIG. 14 shows, as an example, Dcegv and dvp, dc, dv, which are the analysis results of “this- / ba /-/ ka /-
It is a graph which shows s. In the figure, the horizontal axis represents the Dcegv value [ms] obtained by changing the utterance time, and the vertical axis represents dvp (、), dc (▲), d for the Dcegv value at that time.
vs (◇) value [ms]. From FIG. 14, dvp,
It can be seen that dc and dvs can be approximated by the following linear equation using Dcegv as a parameter.

[0026]

(Equation 1)

Here, Avp and Bvp are coefficients related to the preceding vowel, Ac and Bc are coefficients related to the consonant, and Avs and Bvs are coefficients related to the succeeding vowel. As a result of the same analysis for other vowel sequences, it was found that the linear function of Dcegv can be approximated within the discrimination limit. The value of each coefficient in the consonant / k / obtained for each combination of vowels is shown below. This equation can be modified as follows.

[0028]

(Equation 5)

[0029]

[Table 1]

By the way, for an arbitrary VCV (vowel / consonant / vowel), the combination of the preceding vowel and the succeeding vowel must have 25 types of coefficients, which not only increases the data to be stored but also complicates the processing. Becomes Therefore, a case of two groupings will be described below as an example of vowel grouping for simplifying the use of the coefficient.

Using these coefficients, Dcegv corresponding to speech speeds of 11 mora / sec, 7.7 mora / sec, and 6 mora / sec.
Per value (100ms, 160ms, 200ms)
vp, dc and dvs were determined. Especially Dcegv is 100m
The results for s, 160 ms are shown below.

[0032]

[Table 2]

"Difference" in the table means the difference between the maximum value and the minimum value of the time length of the common preceding vowel or the succeeding vowel for each row.
The “median value” is a value of (maximum value + minimum value of each row) / 2 regarding the time length of a different preceding vowel or succeeding vowel in each row. If the preceding vowel is / a /, dvp even if the type of the following vowel is different
There is almost no difference (less than 10-20 ms) in the values. Similarly, when the succeeding vowel is / a /, there is almost no difference in the dvs value even if the type of the preceding vowel is different. Thus, first, it can be said that the time length of a vowel part can be determined only by its own type, regardless of the type of an adjacent vowel.

Next, dvp, dc, dvs in Dcegv
The setting rule of is defined as follows. That is, although the time length differs depending on the type of the five vowels, / a // i // u // e // o / Those whose length is always longer-/ a // e /, and those whose time length is always shorter than the median-/ i // u // o /, can be divided into two groups. Therefore, the second
Considering the discrimination limit of the duration (10 to 20 ms), it is not necessary to analyze all combinations of the five vowels, and it is sufficient to analyze / a /, / i / and use the obtained result. I understand. That is, while Dcegv is a rule determined by the type of consonant, the inside of Dcegv is the second
Is a rule determined by two groups of vowels (long and short durations). Note that a similar tendency was observed from the analysis results performed for other consonants.

From the above considerations, that is, an arbitrary VCV
Dcegv [ms] of (vowel / consonant / vowel) can be determined by the following equation. For example, for the phonological sequence / xkx /, / aka /, / ika /, / aki /
Using the following coefficients, dvp, dc,
dvs is required.

[0036]

(Equation 3)

When the duration time according to the present setting rule was compared with the actually measured value, it was shown that the duration was within the discrimination limit and could be set with high accuracy. The vowel grouping has been described above.
It is determined by the balance between the degree of reduction, that is, to select multiple groups convenient depending on the accuracy with which person using seek.

[0038]

As described above, according to the present invention, 2
The vowel energy, which is between the centroids of the time integrals of the energies of two mora, is used to determine the mora interval from the time length between the centroid positions, and the mora interval is determined using the consonant and speech speed between the two mora as parameters, and the vowel energy is determined. The vowel length and consonant length that compose the mora are determined by using the time length between concentric points and the consonant length as parameters, and the phonological duration of the sentence to be synthesized is adjusted at the mora interval. It will be easy to get.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a Japanese speech synthesis system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a phoneme duration generation unit 2 of FIG. 1;

FIG. 3 is a diagram illustrating time length control focusing on a vowel energy barycenter time length.

FIG. 4 is a diagram showing a case where a previous vowel is different.

FIG. 5 is a diagram showing the influence of the time length between vowel energy center points in the case of FIG. 4;

FIG. 6 is a diagram illustrating a case where a back vowel is different.

FIG. 7 is a diagram showing the effect of the time length between vowel energy center points in the case of FIG. 6;

FIG. 8 is a diagram showing a case where a previous consonant is different.

9 is a diagram showing the influence of the time length between vowel energy center points in FIG. 8;

FIG. 10 is a diagram illustrating a case where consonants are different.

11 is a diagram showing the influence of the time length between vowel energy center points in the case of FIG. 10;

FIG. 12 is a diagram illustrating a method for deriving a time length between vowel energy center points.

FIG. 13 is a diagram illustrating a method for setting a time length in Dcegv.

FIG. 14 is a graph showing, as an example, Dcegv and dvp, dc, and dvs, which are analysis results of “this- / ba /-/ ka / -noodle”.

FIG. 15 is a diagram for explaining conventional control of phoneme duration.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Text analysis part 2 ... Phoneme duration length generation part 3 ... Sound source amplitude pattern generation part 4 ... Pitch pattern generation part 5 ... Sound source generation part 6 ... Spectrum pattern generation part 7 ... Voice synthesizer 21 ... VCV division division part 22 … Vowel part energy center-of-gravity time length setting part 23… Dcegv internal vowel length / consonant length setting part

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-6-222793 (JP, A) JP-A-6-266391 (JP, A) Masayo Kato, Shinichiro Hashimoto, Japanese focusing on vowel energy center of gravity Rhythm rule of IEICE Technical Report, IEICE, VOL92 No. 35 SP92 7-12, 33-40 Masayo Kato, Shinichiro Hashimoto, Japanese rhythm rules focusing on vowel energy center of gravity, Proceedings of the Acoustical Society of Japan Conference, Japan, The Acoustical Society of Japan, Vol. 1991 Fall , 249-250 Masayo Kato, Shinichiro Hashimoto, Extension of Japanese rhythm rule focusing on vowel energy center of gravity, Proceedings of the Annual Conference of the Acoustical Society of Japan, Japan, Acoustical Society of Japan, Vol. 1992 Spring, 239-240 (58) Field surveyed (Int. Cl. ⁷ , DB name) G10L 13/08 G10L 13/06

Claims (3)

(57) [Claims] 1. A Japanese speech synthesis system for synthesizing a mora, which is a unit of prosody corresponding to a single syllable, wherein the second half time length of a preceding vowel is dvp, the time length of a consonant part between the two mora is dc, The first half time length of the succeeding vowel between the two mora is dvs, the constants determined by the preceding vowel are Avp and Bvp, the constants determined by the consonants are Ac and Bc, and further determined by the succeeding vowel. When the constants are represented by Avs and Bvs, a time interval Dcegv between the energy centroids of the respective vowel parts in two adjacent moras is expressed as follows. By applying the following equation, the second half time length dv of the preceding vowel part
p, a means for determining the duration dc of the consonant part and the first half duration dvs of the succeeding vowel part. 2. The second half time length of the preceding vowel is dvp, the time length of the consonant part between the two mora is dc, the first half time length of the succeeding vowel between the two mora is dvs, and When the determined constants are represented by Avp and Bvp, and the constants determined by the succeeding vowels are represented by Avs and Bvs, the time interval Dsegv between the energy centroid points of each vowel part in two adjacent mora is expressed as follows. Equation 2 By applying the following equation, the second half time length dv of the preceding vowel part
p, a means for determining a time length dc of the consonant part and a first half time length dvs of the succeeding vowel part. 3. The Japanese speech synthesis system according to claim 1, wherein the constant Avp determined by the preceding vowel and the constant Bvp determined by the succeeding vowel include a plurality of vowel types. A Japanese speech synthesis system characterized by dividing and using a constant value in each group.