JP3303835B2 - Apparatus and method for generating pitch pattern for rule synthesis of speech - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device which generates a pitch pattern that is smooth and suitable to express naturalness and individuality for speech synthesis by rule. SOLUTION: The device is provided with an input section 11 which inputs a class of control points and the shape of the segment made by the points, a control point inspection section 12 which inspects the appropriateness concerning the class of the control points inputted, a pitch pattern generating section 14 which provides a restriction to the range of the values of a derivative for every interval of a spline function and then generates a spline function that passes through a desired point where the function values and the first order differential coefficients at the connecting points of the adjacent interval are equal on both sides, and a display section 20 which displays a decision control point, an auxiliary control point and a generated pitch pattern on a screen.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech synthesizer, and more particularly, to a pitch pattern generator and a method used in a speech rule synthesizer.

[0002]

2. Description of the Related Art Pitch patterns (fundamental frequency patterns) are very important information for expressing the naturalness and personality of speech in a speech rule synthesizer. Conventionally, several methods for efficiently generating a pitch pattern have been proposed.

As a first method, for example, in Publication 1 (Japanese Patent Laid-Open No. 63-85797), a phrase command and an accent command are generated from an input text, and the impulse response and the step response of the critical braking second-order linear system are generated. Discloses a method of expressing a pitch pattern using a model.

As a second method, a method of designating a plurality of pitch frequencies during utterance as point pitches and performing linear interpolation between them is well known. This is described, for example, in Publication 2 (Iwata and Midome, “Modeling of Prosodic Structure Independent of Utterance Tempo”, IEICE Spring Meeting 1994, SA-5
-2, March 1994).

[0005] As a third method, a method of performing approximation using a piecewise polynomial (spline function) instead of linear interpolation is also used. For example, Publication 3 (JP-A-4-36299)
8) discloses a method using a cubic or higher-order spline function or B-spline.

[0006]

However, the above-mentioned conventional method has the following problems.

In the first method, the pitch pattern is represented as a sum of a breath paragraph component and an accent phrase component represented by each response. According to this method, since there are few parameters to be controlled, there is a characteristic that control is relatively easy and a stable pattern can be obtained. On the other hand,
Since the components of each impulse response or step response can only change exponentially, the approximation accuracy cannot be so high. For this reason, there is a problem that even when a free pitch pattern is manually designed, a large restriction is imposed on a shape that can be created.

In the second method, for example, by using the frequency at the center of time of a syllable or a mora as a point to be specified, automatic generation from actual voice data can be relatively easily performed. Easy to design. On the other hand, in order to increase the approximation accuracy, it is necessary to increase the number of designated points, and there is a problem that the number of parameters increases. Publication 2 also discloses a technique for reducing the number of designated points. However, even with this, the pitch pattern is still a polygonal line, and therefore, when speech is synthesized, the pitch pattern is different at the bending point of the pitch pattern. The problem that sound is generated remains.

In the third method, the first,
The accuracy of the approximation can be higher than in the second method,
A smooth pitch pattern can be obtained. Further, the flexibility of the shape that can be taken is higher than that of the conventional technology 2. On the other hand, in general, when a pitch pattern is approximated by a spline function, a pattern which is impossible in an actual voice pitch pattern, such as pitch pattern irregularity (undulation phenomenon), may be generated. This poses a problem during automatic generation from actual audio data or manual design. This causes adverse effects such as a decrease in naturalness of speech and a decrease in intelligibility when the generated pitch pattern is used for speech synthesis.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made in view of the above-mentioned problems, and has as its object to provide a pitch pattern generating apparatus and method capable of obtaining a synthesized speech with high smoothness in a speech rule synthesizing apparatus. Is to provide.

It is another object of the present invention to provide a pitch pattern generating apparatus and method capable of efficiently calculating a pitch pattern satisfying the constraint. Other objects, features, and the like of the present invention will be easily clarified in the following description.

[0012]

According to the present invention, there is provided a pitch pattern generating apparatus for outputting a spline function representing a pitch pattern used in rule synthesis of speech, wherein a derivative of a derivative is provided for each section of the spline function. Means for generating a spline function passing through a desired point in which a function value and a first-order differential coefficient at a connection point of an adjacent section are equal on both sides of the section while providing a constraint on a range of values. .

[0013]

Embodiments of the present invention will be described. According to the present invention, when a pitch pattern is approximated by a spline function, a constraint is provided for each section of the spline function, thereby avoiding generation of a pattern that is impossible as an actual voice pitch pattern.

[0014] More specifically, the pitch pattern of the voice is roughly determined such that the pitch pattern rises or falls depending on the boundary of the accent phrase or the accent position.

Then, for example, the value of the first derivative (first derivative of the time derivative of the spline function) in a certain interval is
By always limiting to positive or 0, the section can be monotonically increased.

Furthermore, if the value is restricted to the value of the second derivative, it can be defined as a convex upward and monotonically increasing. Thereby, a pitch pattern having a shape suitable for the pitch pattern and passing through a desired point is generated.

In a preferred embodiment of the present invention, in the preferred embodiment, a set of a fixed control point relating to the pitch pattern and an auxiliary control point in the section, and constraint information regarding the shape of the section defined by the control point are provided. Input means for inputting a set of control points, checking means (12) for verifying whether or not the input control point set is information capable of generating a pitch pattern, and inputting and verifying the input and validity. Spline function that passes through the determined control point and the auxiliary control point and satisfies the constraint on the shape of the section, that is, a connection point between adjacent sections after providing a constraint on the range of derivative values for each section of the spline function Pitch pattern generation means (14) for generating a pitch pattern by generating a spline function passing through a desired point having the same function value and first-order derivative on both sides; Comprising serial each determined control point and the auxiliary control points, and a display means for displaying the generated pitch pattern on the screen (20), the.

In the embodiment of the present invention, the constraint information input from the input means (11) is checked for constraints given to each section, and a section in which monotonic increase or monotonic decrease continues is detected, and a monotonic increase or A configuration may be provided that includes a continuous estimation section detection unit (18) that combines sections in which monotonous decrease continues as one estimated continuous section.

Also, in the embodiment of the present invention, information on monotonous increase / monotone decrease for each section is included as information for restricting the number of fixed control points, the type of sentence such as declarative sentence or question sentence, and the shape of the section. The input correspondence table (17) is stored in the storage means, and the input means (11) specifies the number of input definite control points and information specifying the input sentence type instead of the constraint information for each section. From, search the input correspondence table,
It may be configured to determine the constraint information of each section.

[0020]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of the present invention;

FIG. 1 is a diagram showing the configuration of the first embodiment of the present invention. Referring to FIG. 1, an input unit 11, a control point inspection unit 12, a pitch pattern generation unit 14, a display unit 20
And

The input unit 11 includes a set of control points,
The constraint of the section created by the control point is input.

As shown in FIG. 2, a set of input control points is composed of a time and a frequency of each of a fixed control point (と) and an auxiliary control point (×). In FIG. 2, the horizontal axis represents time, and the vertical axis represents frequency. The fixed control point and auxiliary control point are
Although it does not directly correspond to the parameters of the spline function, the x-coordinate of the determined control point, that is, the time, is set as the connection point of the section of the spline function.

In FIG. 2, for the sake of convenience of explanation, each fixed control point and auxiliary control point are numbered. Also, two adjacent fixed control points form one section. In this embodiment, it is assumed that the restriction of the section takes either a monotonically increasing value or a monotonically decreasing value.

The control point checking unit 12 checks the validity of the set of control points input from the input unit 11. In this embodiment, the following five points are confirmed.

(1) It is confirmed that the total number of fixed control points is two or more.

(2) Confirm that the times of the respective fixed control points do not overlap.

(3) Confirm that the frequency of each fixed control point is positive.

(4) It is confirmed that the time of the auxiliary control point is always one in one section and does not overlap with the time of the fixed control point forming the section.

(5) It is confirmed that the vertical relationship between the frequencies of the fixed control points forming each section satisfies the restrictions of the section. For example, when the constraint of the section i is monotonically increasing, it is confirmed that the constraint that the value (frequency) of the fixed control point i + 1 must be equal to or larger than the value (frequency) of the fixed control point i is satisfied.

If the control point inspection unit 12 determines that the pattern is not valid, an error is reported without generating a pitch pattern.

The pitch pattern generator 14 generates a spline function passing through the given fixed control point and the given auxiliary control point. Although not all parameters of the spline function can be determined only by the given determined control points and auxiliary control points, by changing the remaining parameters within a possible range and sequentially checking whether or not the desired condition is satisfied, it is preferable to It is possible to obtain a spline function that satisfies the condition. Of course, the number of control points may be increased from the beginning to determine the spline function.

By the way, although a pitch pattern that passes through a fixed control point can be generated, the pitch pattern may not pass through an auxiliary control point and may not satisfy a desired constraint. In such a case, a pitch pattern is not generated, an error is output, and the processing is interrupted.

The pitch pattern generation unit 14 generates the shape of the pitch pattern itself, which is the output of the pitch pattern generation apparatus. However, the present invention is not limited to such a configuration, but instead defines a spline function. May be output.

The display unit 20 displays each fixed control point, auxiliary control point, and generated pitch pattern on a screen. Also, at this time, by displaying the corresponding times and units together, a display that is easy for the user to see can be obtained.

FIG. 4 is a diagram showing an example of the pitch pattern displayed on the display unit 20, wherein the horizontal axis represents time and the vertical axis represents frequency. As shown in FIG. 4, the fixed control points, the auxiliary control points, and the pitch patterns satisfying the restrictions are displayed for each section.

In one embodiment of the present invention, each of the sections has a value of either monotonically increasing or monotonous phenomenon as a constraint. In addition to this, it is monotonically increasing when convex upward, monotonically increasing when convex downward, and convex when upward. May be finely divided, such as monotonically decreasing, and downwardly convex, monotonically decreasing. In this case, it can be determined by the sign of the first derivative of the spline function and the sign of the second derivative in the interval.

The pitch pattern determined in the embodiment of the present invention is assumed to be a series of continuous patterns corresponding to one accent phrase, but the entire exhalation paragraph separated by a pause is represented by one pitch pattern. In the same manner, it is also possible to extend one sentence by expressing one sentence with one pitch pattern.

In the embodiment of the present invention, the restriction of the section is given from the input unit 11 from the outside. However, the restriction may be fixed so that no input is made. In this case, it is effective when the range to which the spline function is applied is one accent phrase and its shape is determined to be undulating.

Next, a second embodiment of the present invention will be described. FIG. 5 is a diagram showing the configuration of the second embodiment of the present invention. Referring to FIG. 5, a second embodiment of the present invention is shown in FIG.
In addition to the configuration of the above-described embodiment, a continuous estimation section detection unit 18 is further provided.

The continuous estimation section detecting section 18 examines restrictions given to each section, and detects a section in which the monotone increase or the monotone decrease continues. For example, when it is specified that the section 1 to the section a-1 is monotonically increasing and the section a to the section b is designated as monotonically decreasing, the continuous estimation section detecting unit 18 converts the section a to the section b into one section. It is detected as an estimated continuous section.

Next, the spline function generation processing in the pitch pattern generation section 14 will be described.

First, whether or not a pitch pattern passing through the auxiliary control point i can be generated in the section i can be determined as follows.

Step 1: The coordinates of the auxiliary control points at both ends of section i are (xi, Yi0) and (xi + 1, Yi1), respectively. Here, the x coordinate is time, and the y coordinate is frequency. The sections i are arranged without overlapping in the ascending order of time.

Step 2: A function representing the pitch pattern in the section i is defined as Si (t). At this time, (xi, Yi0), (xi +
1, Yi1), the relationship of the following equation (1) holds.

Yi0 = Si (xi),... (1-1) Yi1 = Si (xi + 1).

Step 3: The inverse of the sign of the differential coefficient at xi and xi + 1 of the function Si (x) is converted to a parameter Di0,
It is defined as Di1. Here, the reason why the sign is inverted is that, of the sections handled in the present embodiment, there are more sections with monotonous decrease. Since the case of monotonous increase can be handled in the same way, the monotonically decreasing section will be described below.

The parameters Di0 and Di1 are given by the following equations (2). Note that dSi (x) / dx | x = xi is obtained when x = xi
Represents the first-order derivative of Si (x) with respect to x.

Di0 = −dSi (x) / dx | x = xi (2-1) Di1 = −dSi (x) / dx | x = xi + 1 (2-2)

Step 4: By performing the following variable conversion,
The function Si (t) can be expressed as follows. That is, Si (t) is obtained by normalizing the variable x of Si (x) by t = (x−xi) / hi. Where hi is the section width of the x-axis (= (xi +
1−xi)).

Si (x) | t = (x−xi) / hi−> Si (t) (3-1)

Where t = (x−xi) / hi (3-2) hi = (xi + 1−xi) (3-3)

Gi = Yi0−Yi1 (3-4) where Si (t) is expressed by the following equation (3-5).

[0054]

Step 5: Differentiating Si (t) with respect to x, the following equation (4) is established.

DSi (x) / dx = −Di0 + 2 · (2 · Di0 + Di1−3 · (1 / hi) · Gi) · t−3 · ((Di0 + Di1) −2 · (1 / hi ) ・ Gi) ・ t ² … (4)

Step 6: In order for the function to be monotonically decreasing within the interval, the derivative of step 5 needs to have the same sign (negative) within the interval. The area satisfying this condition is classified for each section. The symbol in each parenthesis corresponds to the corresponding area in FIG. Further, the unhatched area in FIG. 3 does not satisfy the monotonic condition. In addition, each area [1], [1-1], [1-2], and [1-
3], [2], [3], [3-1], [3-1], [3-2], [3-
3] is determined under the following conditions.

[1] When the second order coefficient is positive: Di0 + Di1 <(2 / hi) · Gi (5-1)

[0059]

[0060]

[0061]

[2] When the second order coefficient is 0: Di0 + Di1 = (2 / hi) · Gi (5-5)

[3] When the second order coefficient is negative: Di0 + Di1> (2 / hi) · Gi (5-6)

[0064]

[0065]

[0066] [3-3] 2 · Di0 + Di1 > (3 / hi) · Gi, (3 / hi) · Gi <Di0 + 2 · Di1, Di0 2 + Di0 · Di1 + Di1 2 - 2 · Di0 · (3 / hi) ・ Gi − 2 ・ Di1 ・ (3 / hi) ・ Gi + ((3 / hi) ・ Gi) ² ≤ 0… (5-9)

Step 7: Assume that Si (t) is represented by a cubic (fourth-order) B-spline. That is, the weight coefficient of each support function of the B-spline, the value at each node, that is, the coordinates of the definite control point, and the derivative thereof are expressed by the following equation (6).
Holds. Here, ai, 0 and ai, 1 are weighting factors for section i.

Yi0 = ai, 0 · hi / (hi−1 + hi) + ai, 1 · hi−1 / (hi−1 + hi) (6-1) Di0 = (3 / hi) · (Yi0 − Ai, 1)… (6-2) ai, 0 = Yi0 + (hi / 3) · Di0 · hi−1 / hi… (6-3) ai, 1 = Yi0 − (hi / 3) · Di0… (6-4)

In the sections at both ends, h−1 = 0 or hm = 0 may be set.

Step 8: From this, the condition of monotonous decrease is satisfied by [1-2], [1-3], [2], [3-
1], the relationship between the coefficient of the spline function to be generated, the function value at each node, and the possible range of the differential coefficient can be understood.

Step 9: Here, for the section i = a,
Since the section i-1 is monotonically increasing and the section i is monotonically decreasing, the differential coefficient at this connection point xi is Di0 = 0. Therefore, in FIG. 3, whether or not a spline function satisfying the range of step 8 can be generated is determined by whether or not a straight line passing through the point of Di0 = 0 intersects with the range of step 8.

Step 10: If the range of Step 9 is satisfied, a spline function passing through the auxiliary control point is generated in that range. Here, if it is calculated in step 7 that a spline function passing through the auxiliary control point cannot be generated due to the given constraint, an error is returned.

Step 11: When the parameters of the section i are determined, the value of the connection point Di1 on the opposite side is determined. By repeating this, the parameters of the spline function in each section can be obtained in order as indicated by points a, b, c, d, and e in FIG.

Step 12: If the end of the estimated continuous section is connected to the next estimated continuous section, that is, if the transition from monotonous decrease to monotone increase is performed in this example, Di1 = 0 (i =
b). If this condition cannot be satisfied, an error is output.

In step 8, it is possible to further narrow down possible values by utilizing not only the condition of the differential coefficient but also the difference in the shape of Si (t) depending on each condition of step 6.

Instead of returning an error in step 10, the spline function may be generated using the end point of the possible range that is closest to the condition passing the auxiliary control point. In this case, a spline function is generated so as to pass as close as possible to the desired auxiliary control point.

Next, a third embodiment of the present invention will be described. FIG. 7 is a diagram showing the configuration of the third exemplary embodiment of the present invention. In the third embodiment of the present invention, an input correspondence table 17 is added to the input unit 11.

When inputting the shape of each section in the input unit 11, instead of inputting a constraint for each section, a sentence type such as "declaration sentence" or "question sentence" is input. Input unit 1
1 searches the input correspondence table 17 based on the number of input fixed control points and the sentence type, and determines the constraint of each section. FIG.
Is a diagram showing an example of the contents of the input correspondence table 17 in the third embodiment of the present invention, and shows a sentence type input conversion table.

Referring to FIG. 8, definition information of the number of fixed control points, sentence type ("declarative sentence", "question sentence"), whether pitch patterns can be generated, and monotone increase / decrease for each section as the section shape Consisting of

In the third embodiment of the present invention, the input correspondence table 17 is appropriately replaced with another one, so that it is possible to cope with input restrictions and differences in patterns. For example, the number of confirmed control points is “5”, and only a pattern of monotonically increasing in section 1 and monotonically decreasing in sections 2, 3, and 4 can be permitted.

In this example, the accent phrase is represented by four sections: a rising section, a flat section, a falling section, and a terminating section. In the end section, a pattern that is monotonically decreasing can be realized.

[0082]

As described above, according to the present invention, the following effects can be obtained.

The first effect of the present invention is that a pitch pattern which is smooth and has no frequency unevenness can be obtained, so that a synthesized speech having high smoothness and naturalness can be obtained by using this.
That's what it means.

A second effect of the present invention is that, in addition to the first effect, a pitch pattern satisfying the constraint can be calculated efficiently.

The third effect of the present invention is that, when a pitch pattern is manually designed, the work efficiency can be improved.
This means that the present invention can also be applied to a case where a pitch pattern is automatically designed using pitch frequency information of existing audio data.

A fourth effect of the present invention is that when a pitch pattern is automatically designed, the effect can be obtained by a simple designation, and the operability is improved.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is a diagram for explaining the first embodiment of the present invention, and is a diagram for explaining examples of input information and an output pitch pattern.

FIG. 3 is a diagram for explaining the first embodiment of the present invention, and is a diagram for explaining a method of determining whether a pitch pattern passing an auxiliary control point can be generated.

FIG. 4 is a diagram for explaining the first embodiment of the present invention, and is a diagram showing a display example of a display unit.

FIG. 5 is a diagram showing a configuration of a second exemplary embodiment of the present invention.

FIG. 6 is a diagram for explaining a second embodiment of the present invention, and is a diagram for explaining a method of sequentially generating a pitch pattern satisfying a constraint.

FIG. 7 is a diagram showing a configuration of a fourth embodiment of the present invention.

FIG. 8 is a diagram for explaining a fourth embodiment of the present invention, and is a diagram for explaining an example of the contents of an input correspondence table.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Input part 12 Control point inspection part 14 Pitch pattern generation part 17 Input correspondence table 18 Continuous estimation area detection part 20 Display part

Continuation of front page (56) References JP-A-4-362998 (JP, A) JP-A-8-76781 (JP, A) JP-A-7-261778 (JP, A) JP-A-6-259542 (JP) JP-A-10-74268 (JP, A) JP-A-5-90839 (JP, A) JP-A-6-23197 (JP, A) JP-A-5-19780 (JP, A) 6-58603 (JP, B2) (58) Field surveyed (Int. Cl. ⁷ , DB name) G10L 13/08

Claims (14)

(57) [Claims] 1. A pitch pattern generation device for outputting a spline function representing a pitch pattern used in rule synthesis of speech, wherein a range of values of a derivative is set for each section of the spline function and adjacent to each other. A pitch pattern generation device comprising means for generating a spline function passing through a desired point, wherein a function value and a first-order derivative at a connection point of the section are equal on both sides of the section. 2. A pitch pattern generating device in a speech rule synthesizing device, wherein a set of a fixed control point related to a pitch pattern and an auxiliary control point in a section and constraint information on a shape of a section defined by the control point are input. An input unit, an inspection unit for verifying whether or not the input control point set is information capable of generating a pitch pattern, and passing the fixed control point and the auxiliary control point whose validity has been verified. Pitch pattern generating means for generating a pitch pattern by generating a spline function that satisfies the constraint on the section shape. 3. Checking the constraint given to each section with respect to the constraint information input from the input means, detecting a section in which monotonic increase or monotonic decrease continues, and identifying a section in which monotonic increase or monotone decrease continues in one section. 3. The pitch pattern generation device according to claim 2, further comprising a continuous estimation section detection unit that combines the continuous sections. 4. An input correspondence table including monotonically increasing / monotonically decreasing information for each section as information for restricting the number of fixed control points, the type of a sentence such as a declarative sentence or a question sentence, and the shape of the section. The input means searches the input correspondence table from information specifying the input sentence type instead of the number of input fixed control points and the constraint information for each section, and The pitch pattern generation device according to claim 2, wherein the constraint information is determined. 5. The pitch pattern generation means determines a derivative at a specific connection point of an adjacent section, determines a parameter of a spline function of the section from the derivative and a desired control point, and The method according to any one of claims 1 to 4, wherein a process of determining a differential coefficient of the connection point on the opposite side is repeated to sequentially determine parameters of a spline coefficient of each section. Pitch pattern generator. 6. When the pitch pattern generation means cannot generate a spline function passing through a desired control point, the pitch pattern generation means generates a spline function passing as close as possible to the desired control point. The pitch pattern generation device according to claim 1, wherein 7. The entire range to which the pitch pattern is applied is represented by four sections: an ascending section, a flat section, a descending section, and a terminating section. In the ascending section, the value range of the derivative is positive or zero. 7. The pitch pattern generation device according to claim 1, wherein the interval, the descending interval, and the end interval are set to be negative or zero. 8. The entire phrase range to which the pitch pattern is applied is expressed by accent phrases, ascending sections, flat sections, descending sections, end sections,
It is expressed by five sections of the terminal rising section, and the derivative value range is positive or zero in the rising section and the terminal rising section, and is negative or zero in the flat section, the falling section, and the terminal section. The pitch pattern generation device according to any one of claims 1 to 6, wherein 9. The pitch pattern generation device according to claim 1, wherein said pitch pattern generation means outputs a parameter defining the generated spline function. 10. A pitch pattern generation method for outputting a spline function representing a pitch pattern used in rule synthesis of speech, wherein a range of a derivative value is set for each section of a spline function, and adjacent to each other. A pitch pattern generation method, characterized by generating a spline function passing through a desired point, in which a function value and a first-order derivative at a connection point of the section are equal on both sides of the section. (A) determining a differential coefficient at a specific connection point in an adjacent section; and (b) determining a parameter of a spline function of the section from the differential coefficient and a desired point. 11. The method according to claim 10, further comprising: (c) determining a differential coefficient of a connection point on the opposite side from the parameter, thereby sequentially determining parameters of a spline coefficient of each section. Pitch pattern generation method. 12. A spline function which passes as close as possible to a desired point when a spline function passing through the desired point cannot be generated. A pitch pattern generation method according to one of the preceding claims. 13. The entire range to which the pitch pattern is applied is represented by four sections: an ascending section, a flat section, a descending section, and a terminating section. In the ascending section, the derivative value range is positive or zero. 13. The pitch pattern generation method according to claim 10, wherein the interval, the descending interval, and the end interval are set to be negative or zero. 14. The entire range to which the pitch pattern is applied is represented by five sections of an ascending section, a flat section, a descending section, a terminal section, and a terminal ascending section.
13. The pitch pattern generation method according to claim 10, wherein a value range of the derivative is positive or zero, and is negative or zero in a flat section, a descending section, and a terminal section. .