JP2941168B2 - Speech synthesis system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech synthesis system for controlling a fundamental frequency by detecting a prosodic phrase boundary in accordance with a stochastic context-free grammar in order to obtain a natural synthesized speech.

[0002]

2. Description of the Related Art In order to obtain a natural synthesized speech, it is important to estimate the boundaries of prosodic phrases, that is, the boundaries at which the fundamental frequency needs to be reestablished. .

FIG. 3 is a graph showing a temporal change with respect to the fundamental frequency when the fundamental frequency Fo is not reestablished in the input audio signal, while FIG.
Is a graph showing a temporal change with respect to the fundamental frequency when the fundamental frequency Fo is reestablished in the input audio signal. If the basic frequency Fo is not reestablished while a sentence composed of a plurality of word strings is being uttered, the basic frequency Fo decreases with the passage of time of the uttered voice, as shown in FIG. On the other hand, when the fundamental frequency Fo is reestablished at the boundary of the prosodic phrase in the above sentence, as shown in FIG. 4, the fundamental frequency Fo rises without declining with time of the uttered voice. The frequency Fo is reestablished.

For example, for the purpose of natural speech synthesis,
Study on prosody control including prosodic phrase boundary estimation
It is called a conventional example. ), Kazuo Hakoda et al., "Study of Tone Coupling Type Derivation Rule for Sentence Speech", IEICE Technical Report,
SP89-5, pp33-38, disclosed in May 1989. In this conventional study, a prosodic phrase boundary was estimated from information on dependency relations using heuristic prosodic control rules based on statistical analysis, that is, empirically created prosodic control rules based on human senses. I do.

[0005]

However, the dependency relationship reflects the syntactic structure and the semantic relationship between words, and it is difficult to formulate this accurately, and it must be given by hand. In order to obtain a more natural synthesized speech, the information necessary for speech synthesis becomes enormous, which causes a problem that manual processing becomes complicated.

An object of the present invention is to solve the above problems and to provide a speech synthesis system capable of reducing input information necessary for speech synthesis as compared with the conventional example and obtaining a natural synthesized speech. Is to do.

[0007]

According to a first aspect of the present invention, there is provided a speech synthesis system for controlling a fundamental frequency based on an input word string to synthesize and output the speech of the word string. In a speech synthesis system comprising means,
The speech synthesis system converts a predetermined stochastic context-free grammar into:
A stochastic context-free grammar trained to include information on the structure of prosodic phrases is generated by learning using a corpus bracketed by data of the reconstructed position of the fundamental frequency created in advance according to a predetermined algorithm. A first learning unit, wherein the speech synthesizing unit is a boundary at which the rebuilding of the fundamental frequency occurs according to a prosody control rule including a rule using a stochastic context-free grammar generated by the first learning unit. Control means for detecting a boundary between prosodic phrases in the input word string and controlling the fundamental frequency is provided. A speech synthesis system according to a second aspect is the speech synthesis system according to the first aspect, further comprising a stochastic context-free grammar having a predetermined initial value.
A second learning means for generating a learned stochastic context-free grammar by learning using a corpus bracketed by a pre-created dependency structure according to a predetermined algorithm; Is characterized by re-learning the stochastic context-free grammar generated by the second learning means. Further, a speech synthesis system according to a third aspect is the speech synthesis system according to the first or second aspect, wherein the algorithm is an inside / outside algorithm.

[0008]

In the speech synthesizing system configured as described above, the first learning means converts a predetermined stochastic context-free grammar into parentheses with data of a pre-created fundamental frequency rebuilding position according to a predetermined algorithm. By learning using the attached corpus, a probabilistic context-free grammar that is learned to include information on the structure of prosodic phrases is generated.
Next, the speech synthesizer controls a fundamental frequency based on the input word string to synthesize and output a speech of the word string. Here, the control means of the speech synthesizer includes:
According to a prosody control rule provided with a rule using a stochastic context-free grammar generated by the first learning means, a boundary of a prosodic phrase at which the fundamental frequency is reestablished is detected in the input word string. To control the fundamental frequency. This makes it possible to more accurately detect the boundaries of the prosodic phrases and synthesize and output a more natural voice. Also, by learning a stochastic context-free grammar of a predetermined initial value using a corpus bracketed by a pre-made dependency structure according to a predetermined algorithm,
The apparatus may further comprise a second learning means for generating a learned stochastic context-free grammar, wherein the first learning means re-learns the stochastic context-free grammar generated by the second learning means. You may. This makes it possible to more accurately detect the boundaries of prosodic phrases and synthesize and output a more natural voice. Further, the algorithm is preferably an inside / outside algorithm.

[0009]

Embodiments of the present invention will be described below with reference to the drawings. As described above, conventionally, a dependency structure, words before and after a boundary, and the like are used as factors for estimating a prosodic phrase boundary. Of these, the dependency structure, which is an important factor, reflects the syntactic structure and the semantic relationship between words, and it is difficult to formulate this accurately. On the other hand, in the present embodiment, learning of a stochastic context-free grammar (SCFG) is performed based on a dependency structure given by a human in advance and an actual fundamental frequency rebuilding characteristic. On the basis of this, a prosody phrase boundary is detected and estimated, and a fundamental frequency is controlled to execute speech synthesis. That is, here, the SCFG probability learning unit 30 first determines the predetermined initial value probability context-free grammar (SC
FG) 31 using the inside / outside algorithm to learn the structure of the prosodic phrase, and from the learned stochastic context-free grammar (SCFG) 32 to the prosodic phrase boundary estimation rule creating unit 34, for example, a neural network After creating a rule for estimating the boundary of the prosody phrase and including the rule in the prosody control rule 33, the speech synthesis control unit 1
0 performs speech synthesis processing by estimating a prosodic phrase boundary from parameters and words before and after the boundary based on the prosodic control rule 33.

The inside / outside algorithm (for example, K. Lari et al.) Proposed by K. Lari et al. In 1990 for use in the probability learning unit 30 of the SCFG of this embodiment. , “The
estimation of stochastic context-free grammars usi
ng the Inside-Outside Algorithm ", Computer Speechan
d Language, Vol.4, pp35-56, Academic Press limited,
See 1990. ) Will be described below. The inside-outside algorithm assumes that the input source can be modeled as a context-free hidden Markov process proposed by Baker in 1979. This algorithm allows the evaluated grammar to have any degree of ambiguous expression. Now, Om = O ₁ , O ₂ ,..., O _T
Is a probabilistic context-free grammar (S
CFG) Let the observation sequence be generated by G.

[0011]

## EQU1 ## i → jk and i → m

Here, i, j, and k are each a unique integer corresponding to each of the nonterminal symbols, and m is an integer corresponding to the terminal symbol. This stochastic context-free grammar (SCF
The matrices of parameters describing G) are A and B, where they can be expressed as:

[0013]

## EQU2 ## a [i, j, k] = P (i⇒jk / G)

[Equation 3] b [i, m] = P (i⇒m / G)

Therefore, a [i, j, k] is a nonterminal symbol i
Is the probability of generating a pair of non-terminal symbols j and k. Similarly, b [i, m] represents the probability when non-terminal symbol i generates one terminal symbol m. Since the arbitrary context-free grammar proposed by Chemksky in 1956 can be transformed into the standard form of Chemsky proposed by Chemkky in 1959, these parameters are arbitrary stochastic context-free. Enough to describe the language. In order to prevent inconsistency, the constraint expressed by the following equation (4) must be always satisfied. In this specification, for example, i = 1 to n
The form of the series sum ま で is expressed as Σ _{i = 1} ⁿ using superscripts and subscripts.

[0015]

Σ _j , _k a [i, j, k] + Σ _m b [i, m] =
1, for all i

This constraint simply means that every non-terminal must generate either a pair of non-terminals or one terminal. When applied to stochastic context-free grammar (SCFG), two specific problems need to be dealt with: recognition problems and learning problems. This recognition problem involves the calculation of the probability of a starting symbol S to generate an observation sequence O given a grammar G, as shown in Equation 5 below. In Equation 5, m given to O indicates a matrix, and so on.

[0017]

[Equation 5] P (S⇒ * Om / G)

Here, * indicates a derived sequence consisting of one or more steps. In addition, in the original paper, the symbol ⇒ * is marked with an asterisk (*) above the symbol ⇒, but in this specification, due to restrictions of online application,
⇒ Describe as *. The problem of learning is the learning sequence O
^A set of grammar rules G given ⁽¹⁾ , O ⁽²⁾ , ..., O ^(Q)
Is concerned with determining Similar to the forward (α) and backward (β) probabilities of the conventional Markov model algorithm, an inner probability (e) and an outer probability (f) are defined to facilitate analysis of a stochastic context-free Markov grammar. . The quantity e (s, t, i) is the observation sequence O (s),
.., O (t) is defined as the probability of the non-terminal symbol i that generates O (t) as follows:

[0019]

E (s, t, i) = P (i⇒ * O (s) ... O (t) / G)

When determining an iterative procedure for calculating the quantity e, there are two cases as follows.

(A) Case 1: (when s = t) Only one observation is omitted, and therefore the transition rule of the form i → m is expressed as:

[0022]

[Mathematical formula-see original document] e (s, s, i) = P (i => O (s) / G) = b [i, O (s)]

(B) Case 2: (when s ≠ i) In this case, since more than one observation is included,
Rules of the form i → jk need to be applied. Referring to FIG. 7, which shows the calculation of the inner probability, the quantity e (s, t, i)
Is clearly expressed by the following equation.

[0024]

[Equation 8] e (s, t, i) = Σ j, k Σ r = s t-1 a [i, j, k] e (s, r,
j) e (r + 1, t, k), for all i.

Thus, the quantity e is calculated by an iterative method by determining e for all sequence lengths 1, where all sequence lengths are 2, and so on. Next, the outer probabilities are defined as follows.

[0026]

F (s, t, i) = P (S⇒ * O (1) ... O (s-1), i, O
(t + 1)… O (T) / G)

Here, f (s, t, i) indicates that i is generated in the rewriting process, and a series of sentences which are not governed by i is O in the left direction.
(1)... O (s−1), and O (t) in the rightward direction.
+1)... O (T) (see FIG. 8). In this case, the non-terminal symbol i can be one of two possible settings: j → ik or j → ki, as shown in FIG. Here, it can be expressed as the following equation.

[0028]

F (s, t, i) = Σ _j , _k [Σ _{r = 1} ^s−1 f (r, t, j) a [j,
k, i] e (r, s-1, k) + Σ _{r = t + 1} ^T f (s, r, j) a [j, i, k] e (t + 1,
r, k)], and

F (1, T, i) = 1, if i = S; = 0, otherwise.

After the inside probabilities have been calculated from the bottom up, the outside probabilities are calculated from the top down. For the recognition process, the values e and f can be used to calculate the sentence probabilities as:

[0030]

[Number 12] P (S⇒ * Om / G) = Σ i e (s, t, i) f (s, t, i)

The above equation (12) holds for any s where s ≦ t. By placing s = 1 and t = T in Equation 12, the following equation is obtained.

[0032]

P (S⇒ * Om / G) = Σ _i e (1, T, i) f (1, T, i) = e (1, T, S)

Therefore, P (S⇒ * O) on the left side of Expression 13
/ G) can be calculated only from the inside probabilities. A similar equation can be obtained using the outer probability term by placing s = t in Equation 12 above.

[0034]

[Number 14] P (S⇒ * Om / G) = Σ i e (s, s, i) f (s, s, i) = Σ i b [i, O (s)] f (s, s, i)

The problem of learning a stochastic context-free grammar (SCFG) is more complicated. We begin our discussion by considering the product equation:

[0036]

E (s, t, i) f (s, t, i) = P (S⇒ * Om, i⇒ *
O (s) ... O (t) / G) = P (S⇒ * Om / G) · P (i⇒ * O (s) ... O (t) / S⇒ * O
m, G)

The last step in Equation 15 is the result of applying Bayes' theorem. here,

[0038]

[Mathematical formula-see original document] P = P (S = * Om / G)

Then, the following equation is obtained from the above equation (15).

[0040]

[Mathematical formula-see original document] P (i⇒ * O (s) ... O (t) / S⇒ * Om, G) = (1 / P) e (s, t, i) f (s, t, i)

Therefore, the following equation is obtained.

[0042]

Equation 18] P (when i is used in the ^{_{derivation) = Σ s = 1 T Σ}} t = s T (1 / P) e (s, t, i) f (s, t, i)

Here, consider a case where a rule of i → jk is applied to a certain derivation. Then, by substituting Equation 8 into Equation 17, the following Equation 19 can be obtained.

[0044]

[Equation 19] P (i⇒jk⇒ * O (s) ... O (t) / S⇒ * Om,
G) = (1 / P) Σ _{r = s} ^t-1 a [i, j, k] e [s, r, j] e (r + 1, t, k) f (s, t, i) All For j, k and t> s

Therefore, the following equation (20) is obtained from equations (18) and (19).
Get.

[0046]

[Mathematical formula-see original document] P (when i → jk, i is used) = { _{s = 1} ^T-1 _{} t = s +} ^1T (1 / P) _{} r =} ^st-1 a [i, j ,
k] e (s, r, j) e (r + 1, t, k) f (s,
t, i)

Next, the definition of the following equation 21 is used.

[0048]

A [i, j, k] = P (when i → jk / i is used) = {P (when i → jk, i is used)} / P (i is used (When it is done)

Therefore, the re-evaluation equation for a [i, j, k] is expressed by the following equation 22 from equations 18 and 20.

[0050]

Equation 22] ah [i, j, k] = {(1 / P) Σ s = 1 T-1 Σ t = s + 1 T Σ r = s t-1 a [i, j, k] e ( s, r, j) e
(r + 1, t, k ) f (s, t, i)} / {(1 / P) Σ s = 1 T Σ t = s T e (s, t, i) f
(s, t, i)}, for all i, j, k

Here, h of ah [i, j, k] is used as a substitute for the hat symbol given above the symbol a in the present specification, and h is similarly used hereinafter. Further, by making the same argument, the re-evaluation equation for b [i, m] can be expressed by the following equation (23).

[0052]

Equation 23] bh [i, m] = { (1 / P) Σ t ∈ O (t) = m e (t, t, i) f (t, t, i)} / {(1 / P) Σ
^{_{s = 1 T Σ t = s}} T e (s, t, i) f (s, t, i)}

In practice, a single observation is not enough to accurately evaluate the parameters of a stochastic context-free grammar (SCFG). Therefore, the above equations need to be extended to handle any number of observations. here,
Suppose we have a set of Q observations represented by:

[0054]

[Equation 24] O≡ [O ⁽¹⁾ , O ⁽²⁾ ,..., O ^(Q) ]

Further, the following equations 25 and 26 are set.

[0056]

## EQU25 ## w _q (s, t, i, j, k) = (1 / P _q ) Σr _{= s} ^t-1 a [i, j, k] e
_q (s, r, j) e _q (r + 1, t, k) f _q (s, t, i)

(26) v _q (s, t, i) = (1 / P _q ) e _q (s, t, i) f _q (s, t, i)

Assuming that the plurality of observations are independent, by adding the contribution to the numerator and denominator of Equations 22 and 23 from each of w _q and v _q , 28 can be obtained.

[0058]

[Number 27] ah [i, j, k] = {Σ q = 1 Q Σ s = 1 Tq-1 Σ t = s + 1 Tq w q
(s, t, i, j , k)} / {Σ q = 1 Q Σ s = 1 Tq Σ t = s Tq v q (s, t, i)}

Equation 28] bh [i, m] = { Σ q = 1 Q Σ t ∈ O (t) = m v q (t, t, i)}
/ {Σ _{q = 1} ^Q Σ _{s = 1} ^Tq Σ _{t = s} ^Tq v _q (s, t, i)}

The inside-outside algorithm performs the following processing when iterative processing is performed as follows.
3, Equation 27 and Equation 28 are used. (1) An appropriate initial value for the A matrix and the B matrix is selected assuming the constraint conditions defined by the above equation (4). (2) The following calculation is repeated until the change in P becomes smaller than a predetermined threshold value. A = ... {Equation 27}; B = ... {Equation 28}; P = ... {Equation 13}.

In the above description, the inside / outside algorithm has been described. However, the inside / outside algorithm is used for learning by the inside / outside algorithm when using bracketed text as a learning corpus (for example, Fernando's algorithm). Perera (Fernando)
Pereira) and “inside-Outside Reestimation From
Partially Bracketed Corpora ", The proceeing of A
See CL, 1992. This will be described below. The basic idea of the inside-outside algorithm is to use the current rule probabilities and the learning set W to evaluate the expected frequency of the determined type derivation step, Compute an estimate of the probability of the new rule, such as the appropriate ratio of the estimates of. These are the bits most freely referred to as inside probabilities and outside probabilities, most conveniently expressed as relative frequencies. More precisely, for each w such that w∈W, the inside probability I _p ^w (i, j) is A
While p is to assess the likelihood of when deriving the _i w _j, the probability O _p ^w (i, j) of the out side, the likelihood of format ₀ w _i A _pj w derivation statement from the symbol A ₁ of the start To evaluate.

When applying the above inside-outside algorithm to partially bracketed learning texts, bracketing takes into account the constraint that possible derived statements be included in possible phrases. There is a need.
Obviously, if a non-zero value for the inside of the probability I _p ^w (i, j) and the outside of the probability O _p ^w (i, j) and that if _i w _j may parentheses compatible w Alternatively, or equivalently, it should be possible only if (i, j) is valid for the bracketing of w. Thus, in the following, a series of parenthesized statements c =
In 1979, Baker assumed a corpus C of (w, B) and included that component when its span was compatible with the bracketing of a series of sentences.
r), Lari in 1990
And Young, and one more
We will modify the standard formula for re-evaluating inside and outside probabilities and rule probabilities, as revealed by Jelinek et al. In 990. For this purpose, each series of bracketed statements c =
An auxiliary function represented by the following equation 29 is defined for (w, B).

[0062]

(29) ch (i, j) = 1, if (i, j) is valid for b∈B; = 0, if not

The formula for reevaluation for the extended algorithm is shown below.

[0064]

Equation 30] ^{_{I p c (i-1,}} i) = U p, m, where is c = (w, B) and b _m = w _i.

Equation 31] ^{_{I p c (i, k)}} = ch (i, k) Σ q, r Σ i <j <k B
_p , _q , _r I _q ^c (i, j) I _r ^c (j, k)

Equation 32] ^{_{O p c (0, │c│)}} = 1, if p = 1 if; = 0, if not,

(33) _Op ^c (i, k) = ch (i, k) Σ _q , _r {Σ _{j = 0} ^i-1 O _q ^c (j,
^{_{k) I r c (j,}} i) B q, r, p + Σ j = k + 1 │ c │O q c (i, j) B q, p, r I r
^c (k, j)}

Equation 34] _{Bh p, q, r = {} Σ c ∈ C (1 / P c) Σ 0 ≦ i <j <k ≦ │ w │
_{B p, q, r I q} c (i, k) I r c (j, k) O p c (i, k)} / (Σ c ∈ C P p c /
P ^c )

(35) U h _p , _m = {Σ _c ∈ _C (1 / P ^c ) Σ ₁ ≤ _i ≤ │ _c │, _{c = (w} ,
_{B), wi = bm U p} , m O p c (i-1, i)} / (Σ c ∈ C P p c / P c)

P ^c = I ₁ ^c (0, │c│)

Equation 37] ^{_{P p c = Σ 0 ≦ i}} <j ≦ │ c │I p c (i, j) O p c (i, j)

For each bracketed sentence c in the learning corpus, the probability of the inside of the longer span of the sentence c is the probability of the inside for the shorter span having the reproduction formula given by Eqs. Is calculated from Equation 31 can be from the statement c = (w, B) there is bracketed B compatible with A _p, calculates the expected relative frequency of derivation of _i w _k. The multiplier ch (i,
k) can derive _i w _k when (i, k) is valid for B, ie, _Ap is as compatible as B.

Similarly, the outside probability with respect to the shorter span of the sentence c is calculated from the inside probability and the outside probability with respect to the longer span using the reproduction formula given by Expressions 32 and 33. be able to. If each statement inside of probability and outside of probability and is temporarily calculated for in the corpus, binary display rules Bh _p consisting of two components, _q, and probabilities revaluation of _r, 1 single The re-evaluated probabilities of the rule U h _p , _m consisting of components are, instead of a series of unbracketed statements,
197 except using a series of bracketed statements
Revealed by Baker in 9 and 19
Revealed by Lari and Young in 1990, and in 1990 by Jeline
k) Calculated by revaluation formulas (34 and 35) similar to the original formulas revealed by others.

[0067] The denominator of the ratio, represented by the number 34 and number 35, when derivation that is compatible with one bracketed by a series of statements in C, including at least, one of the expansion of non-terminal symbol A _p Probability can be evaluated. The numerator of Equation 34 above can be used to evaluate the probability that the compatible derivation of a series of bracketed statements in C contains the rule A _p → A _{q Ar} while the numerator of Equation 35 Is C
Derivation with a compatible set of statements can evaluate the probability of when rewriting the A _p to b _m in. Thus, Equation 34 above shows that rewriting A _p in a compatible derivation of a series of bracketed statements in C is A _p
→ A _q A _r becomes rule can evaluate the probability when using, the number 35, when the occurrence of A _p in deriving equation that is compatible with a series of statements in C are rewritten to b _m Probability can be evaluated. These are the best current estimate for the probability of a binary rule with two components and the probability of a rule with one component.

Next, the processing using the re-evaluated probabilities is performed until the increase in the evaluated probabilities of the learning text given to the model becomes negligible, or the probabilities become a total negligible amount. Or the next number 3
Iteratively, until the decrease in the cross-entropy estimate (log of negative probability) represented by 8 becomes negligible.

[0069]

Equation 38] Hh (C, G) = - (Σ c ∈ C logP c) / (Σ c ∈ C │c
│)

Here, for comparison with the original algorithm, the evaluation value Hh of the cross entropy of the unbracketed text W for the grammar G other than Equation 38 above
It is necessary to use (W, G).

In the present embodiment, the inside
To learn a prosodic phrase structure with a stochastic context-free grammar (SCFG) using an outside algorithm, learn a stochastic context-free grammar (SCFG) with text that is morphologically analyzed and bracketed by a dependency structure Using the obtained data as the initial grammar, learning was further performed using texts in parentheses with data on the repositioning position of the fundamental frequency in natural speech.

To learn a stochastic context-free grammar (SCFG) using the inside / outside algorithm, it is necessary to determine the number of terminal symbols and non-terminal symbols. Ideally, the terminal symbol of the stochastic context-free grammar (SCFG) is a word, but it is not realistic because a corpus containing all the words is difficult to obtain and the learning time is enormous. Therefore, in the present embodiment, terminal symbols whose total number is about the type of part of speech plus several are considered by subclassifying particles, and only 23 types of part of speech and their case particles are classified into seven classes (but,,, and). , In, and,
Other), the total of 29 types of terminal symbols are shown in Table 1 below.
Used as shown. The number of non-terminal symbols is 20, and numbers from 1 to 20 are used as non-terminal symbols.

[0073]

[Table 1] 終端 Terminal part of speech ───────────────────t1 adjective t4 ordinary Noun t5 sa-transitive noun t6 pronoun t7 number noun t8 adverb t9 adverb t10 conjunction t11 intimate verb t12 auxiliary verb t13 adjunct t14 connecting particle t16 final particle t17 suffix t18 pronoun t30 auxiliary verb t19 auxiliary verb t19 auxiliary verb Body particle t35 Parallel particle t36 Particle t50 Case particle "ga" t51 Case particle "no" t52 Case particle "ni" t53 Case particle "wo" t54 Case particle "de" t55 Case particle "to" t56 Case particle "others" ───────────────────

The following parameters which can be calculated from the stochastic context-free grammar (SCFG) in order to use the stochastic syntactic structure captured by the stochastic context-free grammar (SCFG) for detection and estimation of prosodic phrase boundaries are proposed. As shown in FIG. 5, for each word, the appearance probability of a sentence having a syntactic structure including a left branch structure with a dependency depth m (hereinafter, referred to as a left branch structure probability m) and the right of the dependency depth n are shown. The appearance probability of a sentence having a syntactic structure including a branching structure (hereinafter referred to as right branching structure probability n) is defined as a probability context-free grammar (SCF).
G), and use these probabilities as parameters for detecting and estimating the boundaries of prosodic phrases.

Here, the left branching structure probability m (the symbol: Pleftm) and the right branching structure probability n (the symbol: PR
The calculation method of (ightn) will be described in detail using a specific example. Now, 5 like t1, t4, t5, t6, t7
When a word string that can be represented by a number of terminal symbols is input, there are 14 possible tree structures, that is, syntactic structures, as shown in (a) to (n) of FIG.

In the example of FIG. 6A, the terminal symbols t1 and t1
4 has a left branching structure with a depth of 1 at the dependency, and a left branching structure at a depth of 1 with the dependency and a left branching structure at a depth of 2 at the terminating symbol t5; Further, the modification has a left branching structure with a depth of 2 and a terminating symbol t6 with a depth 3 of the modification,
Further, it has a left branched structure with a depth of 3 and a branching structure with a depth of 4 with a terminal symbol t7. In the example of FIG. 6B, the terminal symbol t4
Has a left branch structure with a dependency depth of 1 at t5 and
In addition, the modification has a left branching structure with a depth of 1 and a termination symbol t6 with a depth 2 of the modification,
Further, the left branch structure having a depth of 2 and the right end branch structure having a terminal symbol t7 have a depth of 3 and the right branch structure having a depth of 3 and the terminal symbol. At t1, it has a branching structure with a dependency depth of 4. Further, in the example of FIG. 6C, the terminal symbols t4 and t5 have a right branch structure with a dependency depth of 1, and the dependency symbol has a right branch structure with a depth of 1 and the terminal symbol t1. Has a left branching structure with a depth of 2 at the dependency, and further has a left branching structure with a depth of 2 at the depth of the dependency and a left branching structure at a depth of 3 at the terminating symbol t6; Further, the modification has a branch structure with a depth of 4 and a left branch structure with a depth of 3 and a terminal symbol t7. Hereinafter, FIGS. 6D to 6N have a branched structure as shown.

The appearance probabilities of all the tree structures shown in FIG. 6 are calculated based on the stochastic context-free grammar (SCFG), and the sum of them is defined as the total appearance probability Pall. Now, as an example, consider the left branching structure probability 1 (PLeft1) of the terminal symbol t5. In FIG. 6, the left branch structure including the terminal symbol immediately before the terminal symbol t5 is as follows.
(B), (e), and (j), where P is the sum of the appearance probabilities of the respective structures.

[0078]

[Expression 39] PLeft1 = P / Pall

In this embodiment, a stochastic context-free grammar (SCF
G) for learning the probability of 5)
A corpus (hereinafter referred to as a corpus with dependency information) in which three sentences are morphologically cut and bracketed based on a dependency structure (hereinafter referred to as a corpus with dependency information) and bracketed based on the repositioning position of the fundamental frequency in natural speech. Text (hereafter,
A corpus with rebuild information. ) Was prepared. Specific examples of the learning corpus of the stochastic context-free grammar (SCFG) will be described below.

(A) Corpus bracketed by dependency structure (A1) Example sentence: All reality was twisted towards myself. Corpus with dependency information: (((t9)
((T4) (t53)))
((T4) (((t4) (
t51) (t4) (t56)) (
(T32) (t32) (t12)
(T34) (t12))))) (A2) Example sentence: I visited New York just for a week. Corpus with dependency information: (((t7) (t17) (t
17) (t13)) (((t30) (t53)) ((t5) (
t19) (t12)))) (A3) Example sentence: Play a game on a video game or a personal computer. Corpus with dependency information: ((((t4) (t35)
) ((t4) (t54))) ((((t4) (t53))
((t32) (t14))) (t32))) (A4) Example sentence: It is necessary to determine the benefit level in consideration of price fluctuations. Corpus with dependency information: (((((((t4)
(t51)) ((t4) (t53))) ((t5) (t19
) (t14))) (((t4) (t4) (t53)) (t3
2))) ((t4) (t50))) (t32))

(B) Corpus bracketed by data on the position of the fundamental frequency Fo in the following corpus. In the following corpus, (↑) indicates the position where the fundamental frequency Fo is reestablished, and indicates the boundary of the prosodic phrase. (B1) Example sentence: (↑) Everything (↑) twisted all the way to myself. Corpus with rebuild information: (t9 (t4 t53))
t4 ((t4 t51 t4 t56) (t32 t32 t12t34
t12)) (B2) Example: Just a week (↑) I visited New York. Corpus with rebuild information: (t7 t17 t17 t13)
((t30 t53) (t5 t19 t12)) (B3) Example sentence: Play (↑) a game (テ レ ビ) on a video game or PC and play (↑). Corpus with rebuild information: ((t4 t35) (t4
t54)) ((t4 t53) (t32 t14)) t32 (B4) Example sentence: Considering changes in prices, it is necessary to determine the benefit level. Corpus with rebuild information: ((t4 t51) (t4
t53) (t5 t19 t14)) ((t4 t4 t53) t
32 (t4 t50) t32)

FIG. 2 is a flowchart of the SCFG probability learning process executed by the SCFG learning unit 30 of FIG. A stochastic context-free grammar (SCF) for obtaining the above parameters such as the probability of appearance for detecting and estimating the boundaries of prosodic phrases
In order to create G), learning was performed according to the following procedure.

As shown in FIG. 2, first, in step S1, the probability learning unit 30 of the SCFG calculates an initial value SCFG 31 to which an appearance probability is randomly given in advance according to the above-described inside / outside algorithm in detail. The learning is performed using the corpus bracketed by the dependency structure created as described above. More specifically, the process of step S1 estimates a new probability in accordance with Equations 34 and 35 with a corpus bracketed by the dependency structure created in advance as described above as an input. this,
This is repeated until the decreasing value of the value represented by Expression 38 becomes negligible.

Next, in step S2, the SCF
The G probability learning unit 30 puts the SCFG learned in step S1 in parentheses with the data of the rebuilding position of the fundamental frequency Fo created in advance according to the inside / outside algorithm described above in detail. Learning is performed using a corpus to obtain learned SCFG 32 data. Then, as shown in FIG.
The data of the CFG 32 is included in the prosody control rules 33. Specifically, the process of step S2 estimates a new probability in accordance with Equations 34 and 35, using a corpus bracketed with data on the repositioning position of the fundamental frequency Fo created in advance as described above as an input. . This is given by Equation 38
Repeat until the decrease in the value represented by is negligible.

The creation of the prosodic phrase boundary estimating rule in the prosodic phrase boundary estimating rule creating unit 34 is performed, for example, by using a known neural network or discriminant analysis method (for example, Yutaka Tanaka and Kazumasa Wakimoto, Statistical analysis of variates "(see Gendai Mathematics). This discriminant analysis method is a method of discriminating and predicting to which group an individual belongs from the values of these variables based on a sample of past data obtained for each group with respect to a plurality of variables. The application of the discriminant analysis method to the present embodiment is based on the above-described plurality of variables, the left branching structure probability m, the right branching structure probability n, and the type of part of speech. Selected to be divided.

Further, as described above, the speech synthesis control unit 10 generates a probabilistic context-free grammar (SCF
In accordance with the prosody control rules 33 provided with G), as described in detail below, the following data required for speech synthesis is calculated and output. (A) Pitch data corresponding to the fundamental frequency. (B) Data of voiced / unvoiced switching. (C) Amplitude data. (D) Filter coefficient data. Here, an example of the learned SCFG 32 data is shown in Table 2 below.

[0087]

[Table 2] 内容 Contents of Stochastic Context Free Grammar (SCFG) ────────────────── ─ 1 → 1 1 5.677289731860455 × 10 ^-5 1 → 1 2 0.003216677708041557 1 → 1 3 1.000394189802561 × 10 ^-15 :: 1 → 1 19 1.015469740215695 × 10 ^-15 1 → 1 20 1.148333794722905 × 10 ^-15 1 → 2 1 0.0001832178839199974 1 → 2 2 0.000521748447310258:: 1 → 20 19 1.374865835459389 × 10 ^-15 1 → 20 20 0.001676333237389523 2 → 1 1 1.542003529882383 × 10 ^-15 2 → 1 2 1.783061119126052 × 10 ^-15 :: 20 → 20 19 1.586308114265936 × 10 ^-10 20 → 20 20 2.291887552593505 × 10 ^-6 1 → t1 1.155866936964327 × 10 ^-15 1 → t4 1.004501712835847 × 10 ^-15 1 → t5 1.000076431187449 × 10 ^-15 1 → t6 1.00213816472346 × 10 ^-15 :: 1 → t55 1.000679862632174 × 10 ^-15 1 → t56 1.000019916972904 × 10 ^-15 :: 20 → t55 1.196873334230712 × 10 ^-15 20 → t56 1.128862755279862 × 10 ^-15注 (Note) 1 to 20: Non-terminal symbol t1 to t56: Terminal symbol Last number: Probability

In Table 2 above, for example, “1” in the first row
→ 1 1 5.677289731860455 × 10 ^-5 "is the non-terminal symbol 1
Indicates that the probability of occurrence of the rewriting rule branching to non-terminal symbol 1 and non-terminal symbol 1 is 5.677289731860455 × 10 ^-5 , and so on.

Next, referring to the block diagram of the voice synthesizing system according to the embodiment of the present invention shown in FIG. 1, after the uttered voice is input to the microphone 1, the synthesized voice is output from the speaker 25. The configuration and operation up to this point will be described.

The uttered voice of the speaker is input to the microphone 1, converted into a voice signal, and then input to the feature extraction unit 2. Next, the feature extraction unit 2 converts the input audio signal into A
After the / D conversion, for example, LPC analysis is performed to extract 34-dimensional feature parameters including logarithmic power, 16th-order cepstrum coefficient, Δlogarithmic power, and 16th-order Δcepstrum coefficient. The time series of the extracted feature parameters is input to the speech synthesis control unit 10 via the buffer memory 3.
Based on the input feature parameters, the speech synthesis control unit 10 detects and determines a boundary between prosodic phrases, that is, a prosodic phrase in a speech unit, according to a prosodic control rule including a rule using the learned SCFG 32, Based on the determined prosodic phrase, the speech unit is expanded and contracted and connected as is known, and further, based on the value of the obtained spectral feature parameter of the speech unit, the data is subjected to speech synthesis by a known method. , Voiced / unvoiced switching, amplitude, and filter coefficient data, and outputs the data to the pulse generator 21, switch SW, amplitude changing amplifier 23, and filter 24, respectively. Here, the speech synthesis control unit 1
When 0 is detected as a prosodic phrase boundary, the basic frequency Fo is controlled so as to reestablish the basic frequency Fo as shown in FIG.
Output to 1.

The voice synthesizing section 20 includes a pulse generating circuit 21, a noise generating circuit 22, a switch SW, and an amplitude changing type amplifier 2.
3 and a filter 24. Pulse generation circuit 2
Reference numeral 1 denotes a voiced excitation source which generates an impulse having a unit size at the start of each pitch cycle,
To the amplitude changing type amplifier 23 through. On the other hand, the noise generation circuit 22 is a non-voiced excitation source, generates random noise of uncorrelated and uniformly distributed standard deviation 1 and average value 0, and outputs the amplitude-change type amplifier 23 through the switch SW. Output to Accordingly, the switch SW is switched to the pulse generating circuit 21 when generating a voiced sound, and is switched to the noise generating circuit 22 when generating an unvoiced sound. Further, the amplitude changing amplifier 23 changes and amplifies the amplitude of the input signal based on the input amplitude information, and outputs the amplified signal to the filter 24. And the filter 24
Sets the filter coefficient corresponding to the transfer function to the input filter coefficient, filters the input signal with the set filter coefficient, and outputs the signal through the speaker 30.

In this embodiment, both the steps S1 and S2 are provided in the probability learning process of the SCFG of FIG. 2, but the present invention is not limited to this, and learns a stochastic context-free grammar (SCFG). In such a case, the learning processing of step S1 alone may be performed.

Further, the present inventor detected and estimated the boundaries of prosodic phrases using a known neural network in order to check the effectiveness of the parameters proposed in this embodiment. The following prosody phrase detection and estimation method can be applied to the prosody phrase boundary estimation rule creating unit 34 and the speech synthesis control unit 10. The structure of the neural network is a four-layered hierarchical structure, which includes an input layer, a first intermediate layer, a second intermediate layer, and an output layer. Here, the input layer is composed of 50 units and one threshold unit, the first intermediate layer is composed of 25 units, the second intermediate layer is composed of 25 units, and the output layer is composed of 25 units. Is two units. This output data is a prosodic phrase boundary for learning (0,
On the other hand, the input data consisted of a set of a total of 50 input parameters as shown below, which was used for the input data. At this time, the teacher data was determined using the state of the boundary of the one speaker used, that is, information on whether or not the boundary was a prosodic phrase. A neural network was learned from a set of the above teacher data and 50 input parameters, and the learned neural network was used as a prosodic phrase boundary estimation rule.

(A) Left branching structure probability m and right branching structure probability n in the following words, where m, n =
A total of 10 parameters of 1, 2, 3, 4, and 5 or more (a-1) Independent word immediately before the word immediately before the boundary of the prosodic phrase (a-2) Word immediately before the boundary of the prosodic phrase (a-3 ) Word immediately after the boundary of the prosodic phrase (a-4) Independent word immediately after the word immediately after the boundary of the prosodic phrase Therefore, 10 parameters x 4 words = 40 parameters in total. (B) Five parameters of types of terminal symbols of five words immediately before the boundary of the prosodic phrase. (C) Five parameters of the type of terminal symbol of five words immediately after the prosodic phrase boundary.

The learned neural network has 50
The input parameters were input, and the prosodic phrase boundaries were detected and estimated. In detecting and estimating the boundaries of the prosodic phrases, the larger of the two output data was compared, and the larger one was determined as the detection result, that is, the detection of whether or not the boundary was the prosodic phrase was performed. There are degrees of freedom in the boundaries of the prosodic phrases, and in addition to the boundaries of the prosodic phrases in which all the speakers reset the fundamental frequency Fo, the boundaries of the prosodic phrases in which nobody resets the fundamental frequency Fo, There is a boundary where the speaker resets the fundamental frequency Fo. Therefore, the result of detection and estimation was evaluated for a boundary where all speakers are in agreement. The results are shown in Table 3 below. As shown in Table 3, it was confirmed that the boundaries of the prosodic phrases could be detected and estimated with high accuracy, and the detection and estimation of the boundaries of the prosodic phrases using the stochastic context-free grammar (SCFG) were possible. In addition, a stochastic context-free grammar (SC
Since the structure of the prosodic phrase is used in the learning of FG), for example, the data of the predetermined prosodic phrase is used for the data of the SCFG 32 learned by using the neural network as described above, for example. By performing estimation and learning, it is possible to perform inductive learning on the boundaries of prosodic phrases.

[0096]

[Table 3] Detection and estimation results of prosodic phrase boundaries ─────────────────────────────────── Calculation data boundaries Estimated error rate [%] (number of errors / total number) 全員 Everyone is fundamental frequency The fundamental frequency has been restored. The restoration has not been performed. Prosodic phrase boundary Prosodic phrase boundary 句デ ー タ Data after learning 0.6 (4/680) 7.6 (435/5715) ─────────────────────────し て Not learned 7.1 (48/680) 16.9 (964/5715) Data デ ー タ────────────────

As described above, in this embodiment, parameters derived from stochastic context-free grammar (SCFG) are used as input parameters for detecting and estimating the boundaries of prosodic phrases. Therefore, it has been found that the use of stochastic context-free grammar (SCFG) is effective for detecting and estimating the boundaries of prosodic phrases.

In the above embodiment, a neural network is used as means for detecting and estimating the boundaries of prosodic phrases from input parameters. However, the present invention is not limited to this. It is also possible to use a method of predicting the attribute of an event related to those factors based on the continuous value.

Also, in the above embodiment, as the terminal symbols of the stochastic context-free grammar (SCFG), only 23 classes of part-of-speech and their case particles are classified into seven classes (but,,,,,
And 20) were used as the number of terminal symbols and non-terminal symbols in a total of 29 types, but the present invention is not limited to this.
Any number of terminal symbols and non-terminal symbols in Stochastic Context Free Grammar (SCFG) can be used.

As described in detail above, according to the present invention, in a speech synthesis system provided with speech synthesis means for performing speech synthesis by controlling a fundamental frequency to obtain a natural synthesized speech, Stochastic context-free grammar (SCF)
A control means for controlling the fundamental frequency by detecting the boundary of the prosodic phrase, that is, the boundary at which the fundamental frequency is reestablished, using G). Further, the control means performs control using a stochastic context-free grammar (SCFG) created by learning a prosodic phrase structure by a learning method using linguistic information and prosodic information. Therefore, the embodiment according to the present invention has the following specific effects. (1) Compared to a method of controlling the boundaries of prosodic phrases using a dependency structure or the like as in the conventional example, it is not necessary to add the information of the dependency structure to the input text, so that the input information is reduced. Can be. (2) Since the fundamental frequency is controlled by detecting the boundaries of the prosodic phrases based on the prosodic control rules including the rules using SCFG, more natural synthesized speech can be obtained.

[0101]

As described above in detail, according to the speech synthesizing system according to the first aspect of the present invention, the fundamental frequency is controlled based on the input word string to synthesize the speech of the word string. In the speech synthesis system provided with speech synthesis means for outputting, the speech synthesis system converts a predetermined stochastic context-free grammar into a corpus bracketed by data of a pre-created fundamental frequency rebuilding position according to a predetermined algorithm. Learning means for generating a probabilistic context-free grammar learned to include information on the structure of a prosodic phrase, wherein the speech synthesis means is generated by the first learning means. In accordance with the prosody control rules having rules using the stochastic context-free grammar, the boundaries of the prosodic phrases, which are the boundaries at which the fundamental frequency is reestablished, are added to the input word string. There detected and it comprises control means for controlling the fundamental frequency. Therefore, it is possible to more accurately detect the boundaries of prosodic phrases and synthesize and output a more natural voice. Also, compared to a method of controlling the boundaries of prosodic phrases using a dependency structure or the like as in the conventional example, it is not necessary to add information on the dependency structure to the input text, so that the input information can be reduced. it can.

Further, in the speech synthesis system according to the second aspect, in the speech synthesis system according to the first aspect,
Further, by learning a stochastic context-free grammar of a predetermined initial value using a corpus bracketed by a pre-made dependency structure according to a predetermined algorithm, a learned stochastic context-free grammar is generated. The first learning means re-learns the stochastic context-free grammar generated by the second learning means. Therefore, it is possible to more accurately detect the boundaries of prosodic phrases and synthesize and output a more natural voice.

[Brief description of the drawings]

FIG. 1 is a block diagram of a speech synthesis system according to an embodiment of the present invention.

FIG. 2 is a flowchart of an SCFG probability learning process executed by an SCFG learning unit 30 of FIG. 1;

FIG. 3 shows a fundamental frequency Fo of an input audio signal.
6 is a graph showing a temporal change with respect to a fundamental frequency when the rebuilding is not performed.

FIG. 4 shows a fundamental frequency Fo of an input audio signal.
6 is a graph showing a temporal change with respect to a fundamental frequency when the rebuilding is performed.

5 is a diagram showing an example of a sentence composed of a plurality of word strings to be processed in the speech synthesis system of FIG. 1, wherein a left branching structure with a dependency depth m and a right branching with a dependency depth m are shown. It is a figure showing a structure.

6 is a diagram showing a tree structure in an example of a sentence composed of five word strings to be processed in the speech synthesis system of FIG. 1, and showing a method of calculating a left branching structure probability and a right branching structure probability. is there.

FIG. 7 is a diagram showing a calculation method of an inside probability executed in an inside / outside algorithm used in the speech synthesis system of FIG. 1;

8 is a diagram showing a definition of an outside probability used in an inside / outside algorithm used in the speech synthesis system of FIG. 1;

9 is a diagram illustrating a method of calculating an outside probability executed in an inside / outside algorithm used in the speech synthesis system in FIG. 1;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Microphone, 2 ... Feature extraction part, 3 ... Buffer memory, 10 ... Voice synthesis control part, 20 ... Voice synthesis part, 21 ... Pulse generation circuit, 22 ... Noise generation circuit, 23 ... Variable gain type amplifier, 24 ... Filter Reference numeral 25: speaker, 30: SCFG probability learning unit, 31: initial value SCFG, 32: learned SCFG, 33: prosody control rule, 34: prosody phrase boundary estimation rule creation unit, SW: switch.

Continuation of the front page (56) References JP-A-5-134692 (JP, A) JP-A-3-119395 (JP, A) JP-A-60-195596 (JP, A) LARI, S.M. J. YOUNG, "Application of stoichiometric context-free grammar usinng the Inside-Outside algorithm", Computer Speech and Language. 5, No. 3 (1991), pp. 237-257. Pereira, Y .; Shaves, "INSIDE-OUTSIDE RESTMATION FROM P ARTIALLY BRACTED CORPARA", Proc. 30th Annual Meeting of the Association for Computational Linguistics (1992), pp128-135 (58) Fields investigated (Int.Cl. ⁶ , DB name) G10L 3/00-9/20 JICST file (JOIS)

Claims (3)

(57) [Claims] 1. A speech synthesis system comprising speech synthesis means for controlling a fundamental frequency based on an input word string to synthesize and output speech of the word string, wherein the speech synthesis system has a predetermined probability. By learning a context-free grammar according to a predetermined algorithm using a corpus bracketed by data of a pre-established fundamental frequency reconstruction position, a probability context learned to include information on the structure of a prosodic phrase A first learning unit for generating a free grammar, wherein the speech synthesis unit reestablishes the fundamental frequency according to a prosody control rule including a rule using a stochastic context free grammar generated by the first learning unit. Control means for controlling the fundamental frequency by detecting a boundary of a prosodic phrase, which is a boundary at which the occurrence occurs, in the input word string. Speech synthesis system. 2. The speech synthesizing means further learns a stochastic context-free grammar of a predetermined initial value according to a predetermined algorithm using a corpus bracketed by a previously created dependency structure. And a second learning means for generating the generated stochastic context-free grammar, wherein the first learning means re-learns the stochastic context-free grammar generated by the second learning means. 2. The speech synthesis system according to 1. 3. The speech synthesis system according to claim 1, wherein said algorithm is an inside / outside algorithm.