JP2000075880A - Pitch pattern deformation method and its recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To conduct an appropriate pitch deformation keeping autonomy employing a small number of pitch patterns during a rule voice synthesis. SOLUTION: Pitch patterns are categorized by rhythm conditions, each pitch pattern is made into a vector, the pitch pattern vector is averaged for every rhythm category and the difference vector between the average of various different rhythm conditions and the pitch pattern vector is beforehand obtained as a deformation rule. Then, a deformation rule is selected from the rhythm category of an input source and the rhythm category of an application object (S502), the deformation rule is added to the vectored input pattern (S501), and a deformation is executed (S503). Then, corresponding to the deformation of the deformed vector, the input pitch power is deformed to obtain a deformed pitch pattern (S504).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pitch pattern control method in rule speech synthesis and a program recording medium.

[0002]

2. Description of the Related Art At present, the use of synthesized speech is widely used for creating voice messages and narrations in multimedia contents. As a method of controlling the prosody of a text-synthesized speech, there is an approach using a prosody database. In this method, a large amount of natural speech pitch patterns are stored in a prosody database, and prosody control is performed by selecting a pitch pattern suitable for a text to be synthesized from the database. Since prosody information suitable for the text to be synthesized can be obtained from the database to use the prosody information of the real voice, prosody control with high naturalness becomes possible. However, the same prosodic conditions (accent type,
If there is no pitch pattern data having the number of moras, etc., there is a problem that it is not possible to provide an appropriate pitch.

In addition, at present, the prosody automatically generated by the text-to-speech synthesis system does not always have high naturalness, including the above-mentioned problem. It is often necessary to tune the voice using a voice message creation tool. Prosody information of the synthesized speech tuned by the user can be stored for reuse. However, considering that the stored speech is reused, if the prosody conditions of the input pitch pattern and the text of the application destination are different, the prosody will be unnatural as it is. Therefore, there is a problem that the user has to manually perform the tuning operation again in order to obtain a synthesized speech having a natural prosody.

[0004]

SUMMARY OF THE INVENTION The present invention relates to a method of generating a new synthesized speech using a natural speech database or a registered synthesized speech parameter. It is an object of the present invention to provide a pitch pattern deformation method and a program recording medium for performing the same.

[0005]

Means for Solving the Problems To solve the above problems, the method of the present invention uses an accent type, a mora number, a mora number, and a pitch pattern information in advance in a prosody database.
Leading accent type, trailing accent type, leading boundary condition,
The prosodic category (condition) represented by a set of subsequent boundary conditions, accent phrase positions, etc., represents the average pitch pattern shape of the same accent phrase as a vector, and the pitch pattern between accent phrases having different prosodic categories (conditions) is expressed. The shape difference is represented by a difference vector, and this difference vector is used as a deformation rule.

To determine the pitch height of the modified pitch pattern, the pitch pattern information in the prosody database is used to determine the accent type, the number of mora, the preceding accent type, the subsequent accent type, the preceding boundary condition, and the subsequent boundary condition. The pitch average of accent phrases having the same prosodic category (condition) represented by boundary conditions, accent phrase positions, and the like, and the average of the difference in pitch between each accent phrase and the preceding or / and subsequent accent phrase are determined. When the height is determined, the average pitch of the application destination (synthesized text) is determined from the pitch average and the pitch difference average according to the prosodic category (condition) of the application destination (synthesized text).

Further, when the input pitch pattern is deformed, the input pitch pattern is expressed as a vector, and the input pitch pattern is represented by an accent type, a mora number, a preceding accent type, a subsequent accent type, a preceding boundary condition, a subsequent boundary condition, an accent phrase position, and the like. By applying a transformation rule corresponding to the difference between the represented prosody category (condition) and the prosody category (condition) to which the pitch pattern is applied to the input pitch pattern vector, the input pitch pattern is desired (applied) First)
Is transformed into a shape that matches the prosody category (condition).

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below. Here, the input and modification of the pitch pattern are performed using the accent phrase as a basic unit. Accent phrases have prosodic attributes such as accent type, mora number, leading accent type, trailing accent type, leading boundary condition, trailing boundary condition, and accent phrase position. , A prosodic category.

FIG. 1 shows a basic configuration of a pitch pattern deformation process according to the present invention. In the procedure shown in this figure, an input pitch pattern is expressed as a vector in step 101, and a transformation rule for applying to the input pitch pattern based on the prosody category of the input pitch pattern and the prosody category to which the modified pitch pattern is applied in step 102. Is selected, and this deformation rule is applied to the input pitch pattern vector to perform a deformation process. As the transformation rules, transformation rules between prosodic categories are generated in advance. Then, the transformed pitch pattern is restored based on the parameters of the vector transformed in step 103.

Referring to FIG. 2, the flow of processing for generating a transformation rule required for transformation will be described. Here, the accent phrases in the pitch pattern database are classified into prosody categories, the pitch patterns in each prosody category are averaged, and the pitch pattern transformation rules between the prosody categories are obtained. The deformation rule is generated based on pitch patterns accumulated in a large amount of pitch pattern database. Data in the pitch pattern database is stored using accent phrases as basic units. It is assumed that the pitch pattern database stores data such as pitch pattern data of each accent phrase, accent type, number of mora, position of the accent phrase in the sentence, boundary conditions before and after the accent phrase. The pitch pattern has a pitch value for each frame. In this embodiment, one frame is set to 2.5 msec.

In step 201, the accent phrases in the pitch pattern database 21 are categorized (classified) based on the prosodic categories of the accent phrases. Here, categorization conditions include accent type, number of mora, leading accent type, trailing accent type,
The preceding boundary condition, the succeeding boundary condition, and the accent phrase position are used. The condition of the accent type is 0 to 6 types, and the condition of the number of moras is 2 to 8 moras. The leading accent type indicates whether or not the accent type of the accent phrase preceding the accent phrase is the zero type. The succeeding accent type indicates whether the accent type of the accent phrase following the accent phrase is type 1. The preceding boundary condition indicates that the relationship between the accent phrase and the preceding accent phrase is one of strong connection, weak connection, a pause between the accent phrase and the preceding accent phrase, and the accent phrase at the beginning of the sentence. . The subsequent boundary condition indicates that the relationship between the accent phrase and the subsequent accent phrase is a strong connection, a weak connection, a pause exists between the accent phrase and the subsequent accent phrase, or the accent phrase is at the end of the sentence. . The accent phrase position indicates the position of the accent phrase from the beginning of the sentence or the pause portion. The condition of the accent phrase position is from 0 (immediately after the beginning of a sentence or immediately after a pause) to 5. The prosodic category is a combination of these conditions,
A search is made for all combinations of conditions, that is, whether there is an accent phrase matching the prosodic category in the database. For example, one of the prosodic categories is $ (accent phrase) 1 type, (number of mora) 4, (preceding accent type) 0
Type, (succeeding accent type) other than type 1, (preceding boundary condition)
Strong coupling, (subsequent boundary condition) pose, (accent phrase position) 4}. The categorization result indicating which accent phrase is included in the prosodic category is stored in the storage device 22. In this way, the accent phrases in the pitch pattern database 21 are categorized according to the prosodic conditions.

Steps 202 to 203 show a procedure for expressing the pitch pattern as a vector. This process is performed for each prosodic category categorized in step 201. In step 202, parameters representing the characteristics of the pitch pattern shape of each pitch pattern are used as elements of the pitch pattern vector. An example of vectorization in this embodiment will be described with reference to FIG. Here, a method based on a point pitch model will be described as the shape parameter of the pitch pattern. In the figure, V represents a vowel, and C represents a consonant. The pitch at the center of the vowel duration and the pitch at the beginning and end of the accent phrase are used as the shape parameters of the pitch pattern. Each element of the pitch pattern vector is a difference between the pitch of the mora and the pitch of the mora preceding the mora. The mora number of a certain accent phrase a is m, i-th
When the pitch difference between the mora and the (i + 1) th mora is xi, and the maximum number of mora in all prosodic categories is M, the vector expression of the pitch pattern of a certain accent phrase a is Xa = (x0,
x1, x2, xm-1, 0,..., 0, xm). Mm elements of value 0 are inserted between the elements xm-1 and xm. This processing is performed to obtain a difference in pitch pattern vector between prosodic categories having different numbers of moras.

This vectorization is performed on the pitch patterns of all accent phrases in the prosodic category. In step 203, the pitch pattern vectors obtained in step 202 are averaged, and this average vector is used as a pitch pattern vector expressing the characteristics of the pitch pattern of each prosodic category. n is the total number of accent phrases belonging to category k,
Assuming that Xj is a pitch pattern vector of an accent phrase belonging to category k, the average of pitch pattern vectors in prosody category k is obtained as Xk = (1 / n) Σn ^-1 _{j = 0} Xj.

The pitch pattern vector of the prosody category is obtained in steps 202 to 203 as described above.
The obtained pitch pattern vector 23 is stored in a storage device. In step 204, a difference vector as a transformation rule between the prosody categories is obtained from the pitch pattern vector 23 of the prosody category. A difference vector D _KL from the prosody category K to the prosody category L is obtained as D _KL = X _L −X _K. Where K = 0,..., N−1, L = 0,
.., N−1, K ≠ L. N is the total number of all prosodic categories.

With the above procedure, the transformation rule 24 between the prosodic categories is generated. This transformation rule 24 is also stored in the storage device. FIG. 4 shows a procedure for generating information necessary for determining the height of the pitch pattern when the deformation rule is applied in this embodiment. In step 401, the average pitch value of each accent phrase in the pitch pattern database is determined.

In step 402, a coefficient indicating the relative height of the accent phrase and the accent phrase before and after the accent phrase in the pitch pattern database and the average pitch value between the accent phrase before and after the accent phrase is calculated. Ask. This height coefficient r is given by the formula (hp-h) / (hp-h
s). Where hp, hs, h are
The average value of the preceding accent phrase, the subsequent accent phrase, and the pitch of the accent phrase is shown.

In step 403, an average value for each prosody category is calculated for the height coefficient obtained in step 402. The average value 42 of the height coefficient is stored in the storage device. Using the height coefficient r of the prosodic category obtained here,
The height of the accent phrase can be estimated based on the prosodic category information of the accent phrase and the pitch height of the preceding and following accent phrases.

Next, the procedure of applying the deformation rule in the deformation method for the input pitch pattern of this embodiment will be described with reference to FIG. In the pitch pattern deformation method, a pitch pattern deformation rule is applied to an input pitch pattern to deform the input pitch pattern. The input of this pitch pattern transformation device includes an input pitch pattern, a prosody category of the input pitch pattern, a prosody category to which the pitch pattern is applied (transformed), and an average height of each pitch of preceding and succeeding accent phrases to which the pitch pattern is applied. Given.

In step 501, the process proceeds to step 202.
Input pitch pattern in the same way as
Express. In step 502, the rhyme of the input pitch pattern
Deformation to use from the rhythm category and the prosody category to which it applies
Select a rule. As an example, the prosody of the input pitch pattern
When the category is K and the prosodic category of the application is L,
Form rule (difference vector) D _KLFrom the transformation rule data 24
select.

In step 503, a transformation rule is applied. The difference vector selected in step 502 is added to the input pitch pattern vector obtained in step 501 to obtain a deformed vector. That is, assuming that the vector of the input pitch pattern is X _in , the vector X ′ _in of the modified pitch pattern is obtained from X ′ _in = X _in + D _KL .

[0021] At step 504, the pitch pattern vector of the input pitch contour is, to be equal to the pitch contour of the deformed vector X _'in, transforming the shape of the input pitch contour. This shape deformation is performed by increasing or decreasing the pitch value for each frame. FIG. 6 shows a specific example. FIG. 6A is a diagram showing a pitch change amount of each frame of the input pitch pattern. t1, t2, ...
Indicates the vowel center point of the input pitch pattern and the positions of the frames at the beginning and end of the accent phrase. u1, u2, ...
Indicates a pitch change amount at t1, t2,.
This can be obtained from each element of the difference vector _KKL . In this embodiment, the pitch variation between the center of the mora and between the center of the mora and the start and end of the accent phrase are obtained by linear interpolation.

FIG. 6B shows how the pitch change for each frame shown in FIG. 6A is applied to the input pitch pattern to deform the pitch pattern. As a result, the pitch pattern vector after the deformation becomes equal to the vector X ′ _in obtained in step 503. That is, a pitch pattern obtained by modifying the input pitch pattern according to the application prosody category is obtained.

In step 505, the time length of the pitch pattern deformed in step 504 is matched with the phoneme duration of the pitch pattern application destination. The duration is given in phoneme units. Time length correction is performed by linearly expanding and contracting the pitch pattern in the time direction in units of phonemes. Step 50
In step 6, the pitch height suitable for the deformed pitch pattern is determined from the pitch heights of the accent phrases before and after the pitch pattern application destination. From the height coefficient data 42, the height coefficient r of the prosody category to which the pitch pattern is applied is obtained. The pitch height hp of the preceding accent phrase to which the pitch pattern is applied,
Similarly, assuming the pitch height hs of the subsequent accent phrase,
The pitch height h suitable for the deformed pitch pattern is given by h =
hp-r (hp-hs).

In step 507, the average of the pitch heights of the pitch patterns obtained in step 505 is matched with the height h obtained in step 506. Through the above processing, the input pitch pattern can be modified to match the desired (application destination) prosody category. Note that the height determination in step 506 may use only the height of one of the pitches of the preceding or following accent phrase of the application-target accent phrase. If the accent phrase is the beginning or end of a sentence, the average of the difference between the beginning or end and the height of the adjacent accent phrase is determined in advance for each prosodic category. May be obtained from the application prosody category to determine the height.

[0025]

As described above, according to the present invention, a pitch pattern can be transformed into a form suitable for a desired prosody category condition. Thus, for example, in a speech synthesizer that performs prosody control based on a pitch pattern of natural speech selected from a prosody database in the process of synthesizing speech corresponding to input text by rules, the same prosody as the text to be synthesized in the prosody database is used. Even when pitch pattern data of a condition (category) does not exist, speech synthesis with high naturalness can be performed by appropriately modifying pitch patterns of other prosody conditions (categories). Also,
In a speech synthesizer that synthesizes a new synthesized speech by rules using existing rule-synthesized speech parameters, a pitch pattern matches a desired prosody category when the original synthesized speech and the synthesized text belong to different prosody categories. By transforming into a shape, the naturalness of the synthesized speech can be maintained, and the work of correcting the pitch pattern by the user can be reduced.

Next, the effects of the present invention will be shown by experiments.
A transformation rule was generated from the speech database using 2862 phrases. About 5 sentences in the database (a),
(B), (c) Three types of synthesized speech were created. (A) Speech synthesized using speech and prosody information obtained from the ATR database (original speech) itself. (B) Speech synthesized by replacing the pitch pattern of an accent phrase that uttered another sentence with the pitch pattern of one accent phrase in (a) (speech before deformation). (C) The pitch pattern replaced by (b) is transformed by the embodiment shown in FIG. 5 and speech is synthesized using the modified speech (voice after transformation).

These synthesized voices are presented to 12 subjects,
The naturalness of intonation was evaluated on a five-point scale. All phoneme durations given to the synthesizer were those of the original speech. In addition, the subject instructed to evaluate only the intonation of the portion where the pitch pattern was deformed (deformed). FIG. 7 shows the results of this experiment. From this figure, it was found that as a result of the deformation processing according to the present invention, a quality close to the original voice can be obtained.

[Brief description of the drawings]

FIG. 1 is a diagram showing a base structure of a pitch pattern deformed portion of the present invention.

FIG. 2 is a flowchart for generating a deformation rule from a pitch pattern database in the present invention.

FIG. 3 is a diagram showing an example of a vector expression in vectorization of the pitch pattern in FIG. 2;

FIG. 4 is a flowchart for calculating an accent phrase pitch average value and a pitch height coefficient to be used for determining a height at the time of pitch pattern deformation in one embodiment of the present invention.

FIG. 5 is a flowchart showing an algorithm for deforming a pitch pattern in the embodiment of the present invention.

FIG. 6 is a view for explaining a method of transforming a pitch pattern from a vector element in one embodiment of the present invention.

FIG. 7 is a view showing experimental results showing the effects of the present invention.

Claims (6)

[Claims] 1. A pitch pattern of a synthesized speech is input, and a prosody composed of a set of an accent type, a mora number, a preceding accent type, a subsequent accent type, a preceding boundary condition, a subsequent boundary condition, an accent phrase position, and the like of the input pitch pattern. In a processing method of applying a prosodic category different from the category to the input pitch pattern to deform the pitch pattern, the pitch pattern information in the prosody database is used to represent the average pitch pattern shape of the accent phrase having the same prosodic category in advance. The difference between the average pitch pattern shape and the accent phrase having different prosody categories is represented by a difference vector, and this difference vector is stored as a deformation rule, and the input pitch pattern and the prosodic category of the pitch pattern application destination are stored. Apply the deformation rule corresponding to the difference to the above input pitch pattern And vector modification, a pitch pattern deformation method characterized by obtaining a modified pitch pattern corresponding from its deformed vector with the desired prosody category. 2. The pitch pattern deformation method according to claim 1, wherein a pitch average value of accent phrases having the same prosody category and an accent phrase set having the same prosody category are obtained from pitch pattern information in a prosody database in advance. In,
The difference between the pitch average value of at least one of the accent phrase preceding and succeeding each accent phrase is determined, and the modified pitch is calculated using the preceding or / and subsequent pitch average value in the applicable prosodic category. A pitch pattern deformation method characterized by determining a height for a pattern. 3. A process of expressing an input pitch pattern as a vector, a process of selecting a deformation rule corresponding to a difference between a prosody category of the input pitch pattern and a prosody category to which the input pitch pattern is applied, from the database, A program for executing, by a computer, a process of performing a vector transformation on the input pitch pattern represented by the vector and a process of transforming the input pitch pattern based on the vector-deformed input pitch pattern with a rule difference vector. recoding media. 4. A process of selecting height coefficient data corresponding to the application prosody category, and a process of obtaining a height of a pitch pattern preceding or / and succeeding the pitch pattern of the application prosody category. Using the obtained pitch pattern height and the selected height coefficient, correcting the height of the deformed input pitch pattern, and the program having the computer execute the program. Recording medium. 5. The recording medium according to claim 4, wherein the program has a process of matching the deformed input pitch pattern with a time length of a pitch pattern of an application destination, which is performed prior to the height correction process. A recording medium characterized by the above-mentioned. 6. A prosody category including a set of accent type, mora number, preceding accent type, subsequent accent type, preceding boundary condition, subsequent boundary condition, accent phrase position, etc., based on pitch pattern information in the prosody database. The average pitch pattern shape of a phrase is expressed as a vector, and the difference in the average pitch pattern shape with accent phrases having different prosodic categories is expressed as a difference vector, and this difference vector is recorded as a deformation rule. Recording medium.