JPH10254471A - Voice synthesizer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To optimize the evaluation function itself which is used to evaluate for the optimization of rhythm parameters. SOLUTION: Employing a prescribed evaluation function, an evaluation section 251 successively compares and evaluates the rhythm parameter time series, which are successively generated while switching the rhythm parameter values that are applicable and generated using rhythm rules by a parameter generating section 23 from various learning text information and the rhythm parameter time series which are generated from corresponding natural voice data by a parameter analysis section 24. Based on the evaluation values, an optimum rhythm parameter value, which is used for the rhythm rules by a parameter generating rule learning section 26, is determined. Moreover, a learning section 26 outputs the corresponding synthesized voices based on the rhythm parameter time series, which are successively generated by the section 23, to an operator trial hearing evaluation. The evaluation function is corrected to the direction in which no inconsistency exists between the output and the evaluation result of the section 251 employing the evaluation function corresponding to the trial hearing evaluation result.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention generates information on phonemes and prosody of synthesized speech from input text information, and determines prosody parameter values and phoneme symbols of each phoneme of the synthesized speech based on the information. The present invention relates to a speech synthesizer that generates speech parameters necessary for speech synthesis based on the prosody parameter values and phoneme symbols and outputs a synthesized speech based on the speech parameters.
The present invention relates to a speech synthesis device suitable for optimizing prosody rules.

[0002]

2. Description of the Related Art Speech synthesis from text information is generally realized by controlling two features of a phonological surface and a prosodic surface. Of these, the naturalness of the phonological aspect has been dramatically improved by recent advances in hardware technology.

Originally, a prosody parameter in a prosody control model is generated from text information according to a prosody rule (prosody parameter generation rule), and a prosody parameter time series is generated by applying the parameter to the prosody control model to generate a speech waveform. In the flow of a series of speech synthesis processes performed, the weight of the process (general process) from generation of text information to generation of a prosody parameter is large with respect to prosody control. On the other hand, regarding the control of the phonetic surface, the weight of the processing from the prosodic parameters to the generation of the speech waveform (the latter half of the processing) is large. For this reason, the latter half of the processing when the data size to be handled is large has been greatly restricted by hardware.

However, at present, it is possible to easily realize a storage capacity and a processing speed sufficient to maintain and resynthesize the natural sound of a phoneme, and to obtain a synthetic sound having a natural sound sound in general. I have. For this reason, quality improvement of the prosody control is a major issue so that the prosody control does not fail and unnaturalness is not noticeable.

[0005] The prosody control is based on a sequence of linguistic and phonological attributes obtained by analyzing input text information, and based on a prepared prosody rule (prosody parameter generation rule), a pitch, a phoneme time length, a pause, and the like. This is a process of converting into model parameters, such as length and power, in a prosody control model, that is, prosody parameters. In order to improve the quality of the prosody control, it is necessary to optimize (tune) the prosody rule so as to generate a natural synthesized speech.

Conventionally, in order to improve the quality of prosody control, attention has been paid to the characteristics of linguistic and phonological attributes obtained by text analysis, and a prosody obtained by analyzing a small number of natural speech data for the arrangement. The approach was to treat it as a matter of defining the correspondence that maps to parameters. However, this correspondence is only defined on the assumption that the sequence of the attributes of interest is the main factor that reveals its prosodic features, and is an example of analysis of natural speech data on which it depends. There was a problem that the validity was diminished as the number was small.

[0007] For this reason, recently, approaches using a relatively large-scale natural speech and a speech corpus which collects analysis data thereof and treating the statistical tendency as a rule have been increasing. This method evaluates what is the main factor that reveals prosodic features based on the results of statistical analysis of a large corpus, so the rules obtained are relatively valid. It can be said that. However, in order to statistically handle a large amount of data, an evaluation value is calculated (ie, scored) based on an objective scale defined in advance.
However, there is a problem in that it is not clear how the scale will affect subjective hearing. The performance of the rules created by this method depends on how much data can be added to the corpus.However, creating a corpus with large models, a wide range of variations, and uniform quality is very important. There is also a problem that the realization is difficult because of the great effort required.

[0008] From such awareness of the problem, a project for preparing a large-scale corpus is operated and provided for research purposes, but the topics and styles to be handled are relatively “high-quality” sentences, and are transmitted through a computer network. Considering the style of various texts that are increasing in circulation, it can be said that the balance is somewhat lacking.

[0009] From the above, the problem of improving the quality of prosodic control is that a large number of similar sentences are collected and considered at the same time.
Flexibility to incorporate a variety of styles and new examples, how to reflect on subjective hearing, and selection of the main factors that make prosodic features manifest It is necessary to solve the problem by taking into account the judgment from a statistical point of view, so that is not arbitrary.

[0010]

However, in the prior art, since a prosodic parameter value uniquely determined by analyzing a small number of natural speech data is used, an unsuitable sentence example is inappropriate. There is a problem that many parameters are determined. In addition, since it is uniquely determined, there is a problem that it is not known whether the parameters are optimal for every sentence.

Therefore, there is an example of the prior art in which the prosodic parameters are automatically optimized. However, in this type of conventional technology, a prosody parameter (model parameter) estimated from natural speech data (consisting of an accent command value, a phrase command value, and the like) and text information corresponding to the natural speech data are used in a speech synthesizer. However, there is a problem that the time change of the prosody parameter is not taken into account because the comparison between the prosody parameter and the prosody parameter determined for speech synthesis is used for the evaluation. In addition, there is a problem that it is not easy to convert natural voice data to model parameters, such as a fundamental frequency control parameter, and it is not easy to collect a large amount of data.

Also, there are a few examples of the prior art in which a comparison between time series of a prosody parameter time series that can be relatively easily analyzed from natural speech data and a prosody parameter time series determined in a synthesizer is used for evaluation. . But,
In such prior art, taking optimization of the fundamental frequency parameter as an example, the sum of squared errors is simply used as an evaluation function, and data loss in unvoiced sections of natural voice data, consonants, etc. There has been a problem that the system is susceptible to local fluctuations and errors in parts having little perceptual influence.

In the prior art, there is another problem that it is not easy to find and improve inappropriate prosody rules. Further, in the related art, there is a problem that the prosody rule in which natural voice data is insufficient cannot be optimized.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a speech synthesizer capable of optimizing an evaluation function itself used for evaluation for optimizing prosodic parameters.

Another object of the present invention is to provide a speech synthesizer capable of extracting and improving inappropriate prosody rules. Still another object of the present invention is to perform clustering using elements of prosodic parameter determinants that are not taken into account in the prosody rules, and to divide the prosody rules when it is better to divide the prosody rules. It is an object of the present invention to provide a speech synthesizer that can perform the speech synthesis.

Still another object of the present invention is to provide a speech synthesizing apparatus capable of optimizing a prosody rule by utilizing an audition evaluation by an operator with respect to a prosody rule lacking natural speech data. is there.

[0017]

A speech synthesizing apparatus according to a first aspect of the present invention sequentially analyzes various types of learning text information to generate information representing phonemes and prosody of synthesized speech. A prosody rule storage unit in which various prosody rules for generating a prosody parameter are registered in advance, the information generated by the text analysis unit, and a corresponding prosody rule registered in the prosody rule storage unit. A parameter time series generating means for sequentially generating a first prosody parameter time series for speech synthesis while sequentially selecting a prosody parameter value to be applied in the prosody rule from a plurality of predetermined candidates, Parameter analysis means for analyzing natural speech data corresponding to the learning text information to generate a second prosody parameter time series, and the parameter analysis means Evaluation means for comparing and evaluating, using a predetermined evaluation function, a second prosody parameter time series generated and a first prosody parameter time series generated in sequence while switching prosody parameter values by the parameter time series generation means. A prosody parameter value learning means for optimizing a prosody parameter value applied by a corresponding prosody rule based on the evaluation result of the evaluation means, and a first prosody parameter time series generated by the parameter time series generation means. A trial evaluation requesting means for outputting a synthesized speech corresponding to the original for trial listening evaluation by the operator, an operator input means for inputting the trial listening result by the operator, and a trial listening evaluation by the operator inputted from the operator input means Correct the above evaluation function so that the result and the corresponding evaluation result of the above evaluation means are consistent. Characterized by comprising an evaluation function learning means that.

Here, to optimize the prosodic parameter values,
For example, for each prosody parameter value candidate, an average value of the evaluation function values (only the sum may be obtained) may be obtained, and a prosody parameter having the best value may be obtained.

In such a configuration, a prosody parameter time series obtained from natural speech data is compared with a prosody parameter time series obtained for textural synthesis from text information corresponding to the natural speech data, and the prosody parameter time series is obtained. Evaluation is performed and this evaluation (objective evaluation)
As a result of considering the correlation between the result of (1) and the result of the evaluation (subjective evaluation) by the operator's audition, an evaluation function with less inconsistency between the two results is obtained, and the evaluation function itself can be optimized. Here, in the optimization of the evaluation function, a weight (the initial value is, for example, 1) is given to the reference evaluation function, and the objective evaluation result is divided into a group having a good subjective evaluation result and a group having a bad subjective evaluation result (a predetermined threshold value). If a weight is selected such that the statistical difference between the distributions of the objective scores of the two groups becomes larger than a predetermined threshold value, and a weighted evaluation function having the selected weight is adopted, Good.

A speech synthesizer according to a second aspect of the present invention performs statistical processing on the evaluation result of the evaluation means in place of the sample evaluation requesting means in the speech synthesis apparatus according to the first aspect, and performs an inappropriate process. A first prosody parameter time series for speech synthesis to which the modified rule is applied, and the corresponding synthesized speech is generated by the parameter time series generation means. A trial evaluation request means for outputting for trial evaluation by the operator is provided, and a trial evaluation result by the operator input from the operator input means instead of the evaluation function learning means in the speech synthesizer according to the first aspect. A prosodic rule learning means for correcting and optimizing the inappropriate prosodic rule based on the above.

Here, in order to detect an inappropriate prosody rule, an optimum parameter value variance for each piece of text information (sentence example) to which the rule is applied is determined, and whether the variance is greater than or equal to a predetermined threshold value It is only necessary to determine whether or not.

In such a configuration, inappropriate prosody rules can be extracted and improved. A speech synthesizer according to a third aspect of the present invention corresponds to the speech synthesizer according to the second aspect, in which the evaluation result of the evaluation means and the audition evaluation result by the operator input from the operator input means correspond. Evaluation for storing, for each prosody rule, elements of prosody parameter determinants considered by the rule and elements of other prosody parameter determinants not considered by the rule determined by the corresponding text information. In addition to providing a result storage unit, instead of the prosody rule learning unit in the speech synthesizer according to the second aspect, the evaluation result of the evaluation unit stored in the evaluation result storage unit is statistically processed to be inappropriate. Prosody rules are detected, and clustering is performed using elements of the prosodic parameter determinants that are not considered in the rules. Characterized in that a prosodic rule learning means for dividing the rules.

Here, the prosody rule learning means statistically processes the evaluation result of the evaluation means stored in the evaluation result storage means to detect an inappropriate prosody rule, and determines a prosody parameter which is not considered by the rule. A clustering unit that performs clustering using the factor element; and when the distribution of the clustering result of the clustering unit is divided into a plurality, the text information closest to the center of gravity of the distribution is selected for each distribution, and the corresponding Prosody rule division means for dividing a prosody rule to generate a plurality of new prosody rules, wherein the first prosody parameter time series corresponding to each text information selected by the prosody rule division means is
In accordance with each prosody rule generated by the prosody rule division means, the parameter time series generation means generates the corresponding synthesized voice for the trial listening evaluation by the operator. It is preferable to determine whether or not to employ the plurality of generated prosody rules based on the result of the trial listening evaluation performed by the operator on each piece of text information.

In such a configuration, clustering is performed using elements of prosody parameter determinants not considered in the prosody rules (prosody parameter determinants that may be better considered) to divide the prosody rules. If the evaluation value is higher, the prosodic rule is divided,
The prosody rule can be optimized.

Further, when the number of natural speech data corresponding to the learning text information is equal to or less than a predetermined threshold value, a request is sent to one of the trial listening evaluation request means of the speech synthesis apparatus according to the first to fourth aspects. In addition to having a function of outputting a synthesized voice corresponding to the learning text information for trial listening evaluation by the operator, the prosodic parameter value learning means is based on the trial listening evaluation result by the operator and the evaluation result of the evaluation means. It is characterized by having a function of optimizing a prosody parameter value applied by a corresponding prosody rule.

In such a configuration, optimization of the prosody rule can be performed by using the trial listening evaluation by the operator with respect to the prosody rule lacking the natural voice data.

[0027]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a speech synthesizer according to one embodiment of the present invention.

The speech synthesizing apparatus shown in FIG. 1 comprises a text input unit 1, a parameter generation / evaluation unit 2, a synthesizer 3, a speech synthesis unit dictionary 4, and a speech output unit 5. It outputs synthesized speech.
Output of synthesized speech from text information in this speech synthesizer is performed as follows.

First, the text input unit 1 manages input of arbitrary text information (hereinafter, simply referred to as text) to be subjected to speech synthesis. The text input unit 1 includes a keyboard capable of inputting a text (for example, a sentence mixed with Chinese characters and kana) by an operator (user) operation, a hard disk device capable of storing text and reading the stored text, a CD-ROM. This is realized using an external storage device such as a device, or a communication interface capable of receiving and inputting text transferred via a network or the like.

When an arbitrary text is input by the text input unit 1, the input text is used to generate a parameter.
It is provided to a text analysis unit 231 in a parameter generation unit 23 provided in the evaluation unit 2. Text analysis unit 231
Performs morphological analysis, syntactic analysis, and the like on the input text to generate information on phonemes and prosody of the synthesized speech. Information on the phonemes and prosody is given to the parameter time series generation unit 232.

The parameter time series generation unit 232 has a function of generating a prosody parameter and a function of generating a phoneme symbol, and stores the information on the phoneme and the prosody given from the text analysis unit 231 and the prosody rule storage unit 233. A time series of prosody parameters such as a fundamental frequency (pitch frequency), duration, power, and pause of each phoneme, and a phoneme symbol string are generated based on the prosody rules (prosody parameter generation rules). I do. The prosody parameter time series and the phoneme symbol string are provided to the synthesizer 3.

The synthesizer 3 has a voice parameter generating function and a voice synthesizing function.
32, a speech synthesis unit required for the synthesis is selected from the speech synthesis unit dictionary 4 in accordance with the speech symbol string given from 32, and the selected speech synthesis unit is connected according to the prosodic parameter time series given from the parameter time series generation unit 232. do it,
Generate a time series of voice parameters. Here, the speech synthesis unit stored (registered) in the speech synthesis unit dictionary 4 is
For example, after obtaining a feature parameter of a predetermined voice obtained by analyzing a voice uttered by an announcer or the like, in a predetermined synthesis unit such as a Japanese syllable unit, all syllables included in the Japanese voice are extracted from the feature parameter. This is a speech unit created by cutting out. The speech synthesis unit dictionary 4 is realized using a hard disk device, a ROM, and the like.

The synthesizer 3 synthesizes speech based on the time series of the generated speech parameters. The voice synthesized by the synthesizer 3 is supplied to a voice output unit 5 and subjected to D / A (digital / analog conversion), and then output as a synthesized voice from a speaker or the like.

The operation of the speech synthesizer shown in FIG. 1 for generating a synthesized speech from an input text has been described. Next, the prosody rule evaluation / learning function, which is a feature of the speech synthesizer of FIG. 1, will be described.

First, the parameter generation / evaluation unit 2 includes a text data storage unit 21 and a voice data storage unit 22. The text data storage unit 21 stores text data used for learning, and is stored in the voice data storage unit 22.
Stores voice data (natural voice data) with phoneme label information corresponding to the text data stored in the text data storage unit 21.

In this embodiment, a mode (normal mode) for generating a synthesized speech from the input text and a learning mode for learning the prosody rules stored in the prosody rule storage unit 233 can be selected and designated. Has become. Here, the normal mode is automatically set when there is no special designation from the operator (for example, when the system is started). Switching from the normal mode to the learning mode and switching from the learning mode to the normal mode require selection and designation from the operator input unit 28. In the learning mode, a rule specialization automatic mode for automatically specializing prosody rules and a rule specialization operator intervention for requesting trial evaluation by an operator and specializing the prosody rules in consideration of the evaluation result are provided. The mode can be selected and specified. The rule specialization automatic mode and the rule specialization operator intervention mode will be described later.

In the learning mode, the text analysis unit 231 in the parameter generation unit 23 reads the learning text data stored in the text data storage unit 21 under the control of the parameter generation rule learning unit 26, and makes it known. Morphological analysis, parsing, etc. of text data (sentence)
For each accent unit in the middle, information on phonemes and prosody, such as reading information, grammatical attributes (part of speech, etc.), accent type, mora number, etc. is given. When the text analysis unit 231 obtains information on phonemes and prosody of the learning text data, the text analysis unit 231 converts the information into the parameter time series generation unit 23.
Give to 2.

The parameter time series generation unit 232, when given information on phonemes and prosody from the text analysis unit 231, accesses the prosody rule storage unit 233 implemented using an external storage device such as a hard disk device. The prosody rule storage unit 233 stores a set of prosody rules (prosody parameter generation rules) related to prosody such as fundamental frequency, phoneme duration, pause length, and power. Part 23
1, the corresponding prosody rule is selected from the prosody rule storage unit 233 based on the information on the phoneme and the prosody given from 1 and the prosody rule is applied to the information to obtain the fundamental frequency,
A time series of prosodic parameters such as phoneme duration, pause length, and power is generated. Here, with respect to the fundamental frequency, a prosody control model such as a superposition model that assumes that the fundamental frequency pattern is the sum of a speech component and an accent component,
For example, Fujisaki model (Transactions of IEICE Transactions A Vol.J7
2-A No.1 pp32-40 1989-01), the phrase command value (for example, the size of the phrase command, the time of the phrase command), the accent command value (the size of the accent command, Start point and end point)
The model parameters (prosodic parameters) in the model are determined, and then the prosody parameters determined by the prosody rules are applied to the model in the parameter time series generation unit 232, thereby generating a prosody parameter time series.

The parameter generation / evaluation unit 2 also includes a parameter analysis unit 24, a parameter generation rule evaluation unit 25, and a parameter generation rule learning unit 26. The parameter analysis unit 24 is activated by the parameter generation rule learning unit 26 in the learning mode, is analyzed by the text analysis unit 231, and is converted into a prosodic parameter time series by the parameter time series generation unit 232. By reading and analyzing the audio data with label information, a corresponding analysis parameter time series (prosodic parameter time series) is generated.

In the learning mode, the prosody parameter time series generated by the parameter time series generation unit 232 is
This is given to the evaluation unit 251 in the parameter generation rule evaluation unit 25. The evaluation unit 251 includes the parameter analysis unit 24
Is also provided (corresponding to the prosody parameter time series from the parameter time series generation unit 232).

The evaluation unit 251 compares the prosody parameter time series from the parameter time series generation unit 232 and the prosody parameter time series (analysis parameter time series) from the parameter analysis unit 24 with the influence of the prosody parameters of the prosody rule of interest. An evaluation score (evaluation value) is calculated by making a comparison within the range. Here, the range affected by the prosodic parameters of the prosodic rule of interest is such that when the prosodic parameters are changed in several ways, the resulting synthesized speech has a change larger than a human can recognize. Time range. For example, in the case of the fundamental frequency, several Hz or more,
Alternatively, it can be defined as a range in which a change of several percent or more occurs. The evaluation score is given by an evaluation function prepared for each prosodic parameter type (such as duration and power). The evaluation score calculated by the evaluation unit 251 is stored in the evaluation result storage unit 252 in the parameter generation rule evaluation unit 25 in association with the applied prosody rule, the target text data, and the applied prosody parameter value.

The above operation is performed for each prosody rule of interest.
The prosody parameter value applied by the parameter time series generation unit 232 is repeatedly switched from one of a plurality of candidates prepared in advance under the control of the parameter generation rule learning unit 26.

The parameter generation rule evaluation section 25 is provided with a statistical rule evaluation section 253. The statistical rule evaluation unit 253 statistically analyzes the evaluation score stored in the evaluation result storage unit 252 for each prosody rule.

The parameter generation rule learning unit 26 uses the corresponding prosody rule (prosody parameter generation rule) stored in the prosody rule storage unit 233 or the rule according to the evaluation result of the statistical rule evaluation unit 253 or the like. Modify the prosodic parameter values. When the parameter generation rule learning unit 26 detects an inappropriate prosody rule (the definition thereof will be described later), the parameter generation rule learning unit 26 transmits the rule to an information presentation unit 27 configured using a CRT display, a liquid crystal display, or the like. By displaying, the rule is presented to the operator, and in addition, the operator is urged to perform a trial evaluation.

At this time, the parameter generation rule learning section 26
The control operation of causing the text analysis unit 231 to sequentially analyze text data to which an inappropriate (problematic) prosody rule is applied, and generating the corresponding prosody parameter time series by the parameter time series generation unit 232, The prosody parameter value applied in the prosody rule is repeated while selectively switching from a plurality of candidates. Here, the prosody parameter value applied in another prosody rule, that is, the prosody parameter of the prosody rule that is not focused on, which affects the sound in the listening range, includes the prosody parameter value that is currently optimal. Used.

Natural sound data may be used to generate a synthesized sound for trial evaluation. That is, the prosody of the natural voice data is changed in accordance with the prosody parameter time series generated by the parameter generation unit 23 with respect to the range affected by the target prosody parameter, and the synthesized sound generated thereby is previewed by the operator. No problem.

For example, when the prosody parameter of interest is the magnitude of the accent command value determined by a certain prosody rule, first, natural voice data is analyzed to extract a time series of fundamental frequencies, while the parameter generation unit 23 Generates an accent component time series of an accent phrase including the corresponding accent command (a phrase including one accent). At this time,
The start and end points of the accent command correspond to the natural voice data by referring to the phoneme labels assigned to the natural voice data. Next, in the time series of the fundamental frequency analyzed from the natural speech data, a linear approximation of the fundamental frequency corresponding to the time point of the accent component 0 in the accent component time series generated by the parameter generation unit 23 will be described. It is regarded as a tonal component, and after removing portions other than the tonal component from the portion corresponding to the accent phrase, the accent component time series generated by the parameter generating unit 23 is added instead. At this time, when a time series of the same fundamental frequency is generated by switching the corresponding accent command value, an error from the time series of the fundamental frequency of the original natural voice data at any parameter value is equal to or less than a threshold. It is better to confirm that there is, that is, that the assumed speech component is not inappropriate. Change the waveform of the audio data according to the time series of the fundamental frequency generated in this way (TD-PSOLA method
Speech Commun. 9,453-467, 1990) allows the operator to listen to the synthesized sound generated by the Speech Commun.

The parameter time series generation section 232 generates a phoneme symbol string together with the prosody parameter time series. The prosodic parameter time series is provided to the parameter generation rule evaluation unit 25 (the evaluation unit 251 therein), and also to the synthesizer 3 together with the corresponding phoneme symbol string as in the normal mode. Then, the synthesizer 3 generates a time series of speech parameters necessary for speech synthesis based on the prosody parameter time series and the phoneme symbol string, synthesizes the speech, and outputs the synthesized speech by the speech output unit 5. Thus, the operator sequentially listens to and evaluates each synthesized speech when the prosody control is performed while applying the problematic prosody rule and sequentially switching prosody parameter value candidates (prosody parameter values of interest). be able to.

The result (subjective score) of the trial listening evaluation by the operator is input via the operator input unit 28 and stored in the evaluation result storage unit 252. Further, the result of the trial listening evaluation is passed to the parameter generation rule learning unit 26, and is used for correcting the prosody rule in the learning unit 26.

FIG. 2 is a block diagram showing the configuration of the evaluation unit 251. The evaluation unit 251 includes, as shown in FIG.
Analysis parameter evaluation data generation unit 251a, prosody parameter evaluation data generation unit 251b, and comparison unit 251
c.

Analysis Parameter Evaluation Data Generation Unit 251
In a, analysis parameter time series (prosodic parameter time series) which is the result of voice data analysis by the parameter analysis unit 24
Is input from the parameter analysis unit 24 together with the phoneme label information attached to the corresponding audio data. The analysis parameter evaluation data generation unit 251a also receives information on the prosody parameter of interest from the parameter time series generation unit 232. Accordingly, the analysis parameter evaluation data generation unit 251a generates an analysis parameter time series (a prosody parameter time series) in a range obtained by adding several words before and after (a predetermined number) to the range affected by the prosody parameter of interest. It is extracted as evaluation data based on the phoneme label information.

The information on the prosody parameter of interest is stored in the prosody parameter evaluation data generation unit 25.
1b. The prosody parameter time series generated by the parameter time series generation unit 232 is also input to the prosody parameter evaluation data generation unit 251b. As a result, the prosody parameter evaluation data generation unit 251a extracts, as evaluation data, a prosody parameter time series in a range obtained by adding several words before and after (a predetermined number) to the range affected by the prosody parameter of interest. .

The comparison unit 251c represents the evaluation data (prosody parameter time series for evaluation) extracted by the analysis parameter evaluation data generation unit 251a and the prosody parameter evaluation data generation unit 251b, as represented by the sum of squared errors. Then, an evaluation score (evaluation value) indicating the similarity is calculated. The evaluation score calculated by the comparison unit 251c is stored in an area (table area) in the evaluation result storage unit 252 secured according to the applied prosody rule.

FIG. 3 shows the storage format of the evaluation score in the evaluation result storage unit 252. More specifically, the storage of the evaluation score in the area in the evaluation result storage unit 252 secured in accordance with the applied prosody rule. Here is an example of the format.

Here, the ID (identifier) of the data to which the corresponding prosody rule is applied, the element of the determinant of the prosody parameter, and various prosody parameter values (6 in the figure, A0 to A5).
The score (evaluation score) when the seed is given is stored as a set. The prosodic parameter determinants include those that are considered (described) in advance in the corresponding prosody rule and those that are not considered (not described). It is used at the time of conversion. The score (evaluation score) for each prosodic parameter value includes, in addition to the comparison / evaluation result (objective score) in the evaluation unit 251 (in which the comparison unit 251c is included), the subjective score for the data on which the operator performed the audition evaluation There is. Here, a four-point evaluation value is used as the subjective score, and a double circle is the best, and in the following, a circle (○), a triangle (△), and a cross (×) are arranged in this order. The objective score is assumed to be a score based on an error function, and the smaller the value, the better the evaluation.

Next, details of the evaluation processing under the control of the parameter generation rule learning section 26 will be described with reference to the flowcharts of FIGS. First, the parameter generation rule learning unit 26 determines and selects the rule i of interest from the prosody rule storage unit 233 (steps 401 and 40).
2). Next, the parameter generation rule learning unit 26 selects one of the prosody parameter value candidates Aj prepared in advance in the prosody rule storage unit 233 corresponding to the selected rule i,
It is set as the prosodic parameter value of the rule i (steps 403 and 404).

Next, the parameter generation rule learning section 26 determines and selects learning text data k (here, for example, in units of one sentence) to which the rule i of interest is applied (step 40).
5, 406), and activates the parameter generation unit 23 and the parameter analysis unit 24.

Thus, in the parameter generation unit 23,
The text data k in the text data storage unit 21 is analyzed by the text analysis unit 231 and the prosody parameter time series is generated by the parameter time series generation unit 232 based on the analysis result. In the prosody parameter time series generation processing in the parameter time series generation section 232, the prosody parameter value Aj set by the parameter generation rule learning section 26 is used for a portion to which the rule i can be applied, and another rule is applied. For the possible parts, the prosodic parameter values that are currently considered optimal for the rule are used. The prosody parameter time series generated by the parameter time series generation unit 232 is evaluated by the evaluation unit 251 in the parameter generation rule evaluation unit 25.
Given to.

On the other hand, the parameter analysis unit 24 analyzes the learning speech data with phoneme label information in the speech data storage unit 22 corresponding to the text data k, and calculates a corresponding analysis parameter time series (prosodic parameter time series). Generation processing is performed. The prosody parameter time series generated by the parameter analysis unit 24 is also provided to the evaluation unit 251.

The evaluator 251 controls the parameter time series generator 232 under the control of the parameter generation rule learning unit 26.
It is checked whether or not there is a part to which rule i is applied in the prosodic parameter time series from (step 407). If there is a part to which the rule i is applied, the part is compared and evaluated with the analysis parameter time series from the parameter analysis unit 24, and the evaluation result (evaluation score) is stored in the evaluation result for the rule i. The corresponding data portion (the position of the data portion in the data k) and the parameter value Aj are stored in the area in the section 252 in association with each other (step 408).

When the process proceeds to the prosody parameter time-series portion corresponding to the final portion of the data k in this way, the parameter generation rule learning section 26 checks whether or not there is unprocessed text data (step 501). Then, select one unprocessed text data (step 50).
2, 406), and the same processing (steps 407, 408) is performed on the data.

When the evaluation using the rule i for all the text data is completed in this way, the parameter generation rule learning unit 26 determines whether or not the rule i has been evaluated for all the prosody parameter value candidates. Is checked (step 503). If there is a candidate for the unevaluated prosodic parameter value, one candidate for the unevaluated prosodic parameter value is selected and set as a parameter of rule i (steps 504 and 404). The process proceeds to the processing after step 405.

When the rule i is evaluated for all the prosody parameter value candidates, the parameter generation rule learning unit 26 returns to the evaluation result storage unit 2 for the rule i.
On the basis of the evaluation score of the area inside 52, an average value (sum may be obtained) of the evaluation scores is obtained for each candidate (A0 to A5 in the example of FIG. 3), and the average value (sum) is optimized ( The candidate Ax to be minimized) is determined as the optimal parameter of the rule i (step 506). In the example of FIG. 3, A2 is the above A
It corresponds to x.

Next, the parameter generation rule learning unit 26 checks whether or not all the rules have been evaluated (step 506). Select one (Step 50
7, 402), and proceeds to the processing after step 403. .

After all the rules have been evaluated, the parameter generation rule learning section 26 checks whether or not any of the rules has changed the optimum parameter (step 508). If the parameters have been changed, it is determined that an interaction between the parameters has been found, and the process returns to the first step 401, and the processing from step 401 onward is repeated until the optimal parameters of all rules are not changed.

Next, the parameter generation rule specializing process, which is one of the parameter generation rule learning processes by the parameter generation rule learning section 26, will be described with reference to the flowcharts of FIGS.

First, the parameter generation rule learning unit 26 determines and selects the rule i of interest (steps 601 and 60).
2). Then, the statistical rule evaluation unit 253 stores the optimum parameter value for each sentence example (text data portion) to which the rule i selected by the parameter generation rule learning unit 26 is applied in the area in the evaluation result storage unit 252 for the rule i. To calculate the variance (step 60).
3).

The parameter generation rule learning unit 26 checks whether the variance calculated by the statistical rule evaluation unit 253 is equal to or larger than a predetermined threshold (step 60).
4). If the calculated variance falls below the threshold, the parameter generation rule learning unit 26
Is determined to be appropriate, and the same processing (step 60) is performed for the next rule (steps 705, 706, 602).
3).

On the other hand, if the calculated variance is equal to or larger than the threshold, the parameter generation rule learning unit 26 determines that the corresponding rule i is inappropriate, and the text analysis unit 231 for the corresponding text data ( Clustering is performed using the elements (accent type 0, accent type 1, accent type 2, etc.) of the parameter deciding factors (accent type, etc.) not taken into account in the rule i determined by the analysis result (step 605). From the clustering result, it is checked whether or not the sum of the evaluation scores is improved by specializing the rule i (step 606).

If it is determined that the condition is improved, if the rule specializing automatic mode is set (step 70).
1) The parameter generation rule learning unit 26 specializes, for example, divides the corresponding rule i to generate two rules, and stores them in the prosody rule storage unit 233 (step 702).

On the other hand, if the rule specialization operator intervention mode is set, the parameter generation rule learning unit 26
In the case where the rule i is specialized by moving the parameter generation unit 23 and the synthesizer 3, the synthesized speech when the specialized rule is applied to the text data selected as described below is In addition to the output from the audio output unit 5, the guidance information requesting the trial evaluation is displayed on the information presentation unit 27, and the trial evaluation by the operator is performed (step 703).

In this case, the operator inputs the result of the trial evaluation from the operator input unit 28 according to the guidance displayed on the information presentation unit 27. The parameter generation rule learning unit 26 checks whether or not the operator's preview evaluation result input from the operator input unit 28 can be improved by specializing the rule i (step 704). i is specialized, for example, divided to generate two rules (step 702). Step 70 above
As an implementation example of 3,704, assuming that the rule i is divided, the operator performs an audition evaluation on several typical text data, and as a result, the difference between the scores of the two is equal to or greater than a threshold value. If it occurs, a method of determining the division is applicable.

For example, when the rule R is divided into the rules R1 and R2, as shown in FIG. 8A, as the typical data of the data to which the rules R1 and R2 are applied, as shown in FIG. For each, the space represented by the parameter determinant not considered in the rule (for example, the accent type as one coordinate axis, the accent types 0, 1 and 2 as its elements as values on the coordinate axis, and the number of mora Is defined as another coordinate axis, the value of the number of mora that can be taken is the value on that coordinate axis, such that each parameter determinant not considered in the rule is defined as a coordinate axis. Data X1 and X2 closest to the center of gravity of the distribution of data (sentence example) to be applied
Choose

Then, a synthesized speech in the case where the rules R1 and R2 are applied to the selected data X1 and X2 is output.
The operator performs a trial evaluation, and evaluates how much the distribution (subjective score) of the trial evaluation score for each prosodic parameter value differs between the data X1 and X2 as shown in FIG. If it is determined that the values differ by a certain threshold or more, division may be performed.

Next, a generalization method of the parameter generation rule, which is one of the parameter generation rule learning, will be described.
As shown in FIG. 9, the number of data to which the rule p is applied and the rule q
Is small (the data is represented by black circles), and when the rule p or q is applied to the data, the space represented by the parameter determinant considered in the rule (parameter determinant) Center of gravity Gp of distribution of data (sentence example) group to which the rule is applied in (space)
, Gq is shorter than a certain threshold value and if the optimum parameter values given to the rules p and q are the same or very close, a new rule r including both rules p and q is created. Is possible.

Next, the evaluation unit 251 (the comparison unit 251 of the
The evaluation unit 2 using the objective score calculated by the comparison and evaluation in c) and the subjective score given by the operator through the operator input unit 28 by the operator audition.
The method of optimizing the evaluation function at 51 will be described with reference to FIGS.

FIG. 10 shows an example in which, when a certain rule is applied to a sentence i and a sentence j, the objective score and the subjective score are particularly different for the sentence j. In this case, with the optimal prosody parameter value A3 determined from the objective score, a prosody that is very inaudible at the time of sentence j is generated. This can occur when a square error or the like is simply used as the evaluation function. Therefore, it is necessary to determine an evaluation function with less inconsistency between the objective score and the subjective score.

In the present embodiment, for example, fi represented by the following equation fi = Σαj * dj (1) is applied as an evaluation function of the fundamental frequency control parameter. Here, j is the category number when the phonemes are classified into a plurality of types, dj is the square error of the phonemes of category j, αi is the weight for category j, and Δαj * dj is for all categories j of the values of αj * dj. Is the sum of As a method of classifying phoneme types, for example, a method of dividing into consonants and vowels can be applied.

In this embodiment, the optimum evaluation function fi is determined by changing the weight αj with respect to dj on the square error of category j for each category j. The graph of FIG. 11A shows the distribution of the value of the objective score with respect to the subjective score when fi is used as the evaluation function for data having both the objective score and the subjective score.
Here, the objective score is data of good evaluation (double circle and ○)
And bad evaluation data (△ and ×), and weight aj such that the statistical difference between the two objective scores is as large as possible (greater than a predetermined threshold).
Is selected for each category j, an evaluation function fi in which inconsistency between the subjective score and the objective score does not easily occur can be obtained.

As an example of the statistical difference index of the distribution,
As shown in FIG. 11B, distribution of objective scores (distribution of frequency of occurrence) for data of poor evaluation with subjective scores 1
When performing a t-test on the difference between 11a and the distribution of the objective score (distribution of occurrence frequency) 111b for the data with a good subjective score, the significance level of the difference can be used. Differences are considered large.

For rules with insufficient natural voice data, for example, rules where the number of applicable natural voice data is equal to or less than a certain threshold value, the operator makes a trial listening using only the text data stored in the text data storage unit 21. Evaluation is performed, and the score of the preview evaluation (subjective score)
Can be used as an objective score together with the original objective score to select an optimal prosodic parameter.

The above-described speech synthesizer having the configuration shown in FIG. 1 includes a computer, for example, a personal computer 120 having a built-in speaker 121 as shown in FIG. 12 and a text input unit 1, a parameter generation / evaluation unit 2, and a synthesizer. 3. Using a recording medium on which a program for functioning as the voice synthesis unit dictionary 4 and the voice output unit 5 is recorded, for example, a floppy disk 122, mounting the floppy disk 122 on the personal computer 120, and recording the floppy disk 122 The program is read and executed by the personal computer 120 using a personal computer. Here, the speaker 121
Is used as a part of the audio output unit 5. As a recording medium on which the program is recorded, a CD-ROM, a memory card, or the like can be used in addition to the floppy disk 122.

[0083]

As described in detail above, according to the present invention, a prosody parameter time series obtained from natural speech data and a prosody parameter time series obtained for speech synthesis from text information corresponding to the natural speech data The evaluation of prosodic parameters (objective evaluation) is performed by comparing the results of the evaluation with the objective evaluation (comparative evaluation) and the subjective evaluation (evaluation based on the operator's audition). The evaluation function itself used for the evaluation can be optimized.

According to the present invention, an inappropriate prosody rule is detected by statistically processing a result of comparison and evaluation between natural speech data and the corresponding synthesized speech data, and the rule is corrected.
By allowing the operator to perform a trial evaluation on the synthesized speech generated by applying the modified rule, it is possible to correct and optimize inappropriate prosodic rules based on the trial evaluation result. .

Further, according to the present invention, by performing clustering using elements of prosodic parameter determinants not taken into account by the prosody rules, it is determined whether or not the prosody rules should be divided, and the evaluation value is determined. Can be divided in the direction in which

According to the present invention, the prosody rule can be optimized by using the trial listening evaluation by the operator with respect to the prosody rule lacking the natural voice data.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a speech synthesizer according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of an evaluation unit 251 in FIG.

FIG. 3 is a diagram showing an example of a storage format of evaluation scores in an evaluation result storage unit 252 in FIG. 1;

FIG. 4 is an exemplary view showing a part of a flowchart for explaining an evaluation process in the embodiment;

FIG. 5 is a view showing the rest of the flowchart for explaining the evaluation processing in the embodiment.

FIG. 6 is a view showing a part of a flowchart for explaining parameter generation rule specialization processing by a parameter generation rule learning unit 26 in the embodiment.

FIG. 7 is a view showing the rest of the flowchart for describing parameter generation rule specialization processing by the parameter generation rule learning unit 26 according to the embodiment.

FIG. 8 is an exemplary view for explaining division conditions of a parameter generation rule in the embodiment.

FIG. 9 is an exemplary view for explaining generalization of a parameter generation rule according to the embodiment;

FIG. 10 is a diagram showing an example in which, when a certain parameter generation rule is applied to a sentence i and a sentence j, the objective score and the subjective score are different.

FIG. 11 is a view for explaining optimization of the evaluation function in the embodiment.

FIG. 12 is an exemplary external view of a personal computer that implements the speech synthesizer of FIG. 1;

[Explanation of symbols]

REFERENCE SIGNS LIST 1 text input unit 2 parameter generation / evaluation unit 3 synthesizer 4 voice synthesis unit dictionary 5 voice output unit 21 text data storage unit 22 voice data storage unit 23 parameter generation unit 24 parameter analysis unit 25 ... Parameter generation rule evaluation unit 26 ... Parameter generation rule learning unit (prosody parameter value learning unit, trial listening evaluation request unit, evaluation function learning unit, prosody rule learning unit, clustering unit, prosody rule division unit) 27 ... Information presentation unit 28 ... Operator input unit 231 text analysis unit 232 parameter time series generation unit 233 prosody rule storage unit 251 evaluation unit 252 evaluation result storage unit 253 statistical rule evaluation unit.

Claims (7)

[Claims] An information on a phoneme and a prosody of a synthesized speech is generated from input text information, and a prosody parameter value and a phoneme symbol of each phoneme of the synthesized speech are determined based on the information. And a speech synthesizer for generating speech parameters necessary for speech synthesis based on phonetic symbols and outputting synthesized speech based on the speech parameters. Text analysis means for generating information representing prosody, prosody rule storage means in which various prosody rules for generating prosody parameters are registered in advance, the information generated by the text analysis means, and the prosody rule storage Based on the corresponding prosody rule registered in the means, a prosody parameter value to be applied in the prosody rule is determined among a plurality of predetermined candidates. Parameter time series generating means for sequentially generating a first prosody parameter time series for speech synthesis while sequentially selecting the second prosody parameter time series by analyzing natural speech data corresponding to the learning text information. A parameter analysis means for generating, the second prosody parameter time series generated by the parameter analysis means, and the first prosody parameter time series generated in order while switching the prosody parameter value by the parameter time series generation means Evaluating means for comparing and evaluating using a predetermined evaluation function, prosody parameter value learning means for optimizing a prosodic parameter value to be applied in a corresponding prosodic rule based on the evaluation result of the evaluating means, A synthesized speech corresponding to the first prosody parameter time series generated by the series generation means is operated. Means for requesting a trial evaluation to be output for the trial evaluation by the operator, operator input means to which the result of the trial evaluation by the operator is input, and the evaluation corresponding to the result of the trial evaluation by the operator input from the operator input means. An evaluation function learning unit for correcting the evaluation function in a direction in which the evaluation result of the unit does not contradict the evaluation function learning unit. 2. Generating information on phonemes and prosody of synthesized speech from input text information, determining prosody parameter values and phoneme symbols of each phoneme of synthesized speech based on the information, And a speech synthesizer for generating speech parameters necessary for speech synthesis based on phonetic symbols and outputting synthesized speech based on the speech parameters. Text analysis means for generating information representing prosody, prosody rule storage means in which various prosody rules for generating prosody parameters are registered in advance, the information generated by the text analysis means, and the prosody rule storage Based on the corresponding prosody rule registered in the means, a prosody parameter value to be applied in the prosody rule is determined among a plurality of predetermined candidates. Parameter time series generating means for sequentially generating a first prosody parameter time series for speech synthesis while sequentially selecting the second prosody parameter time series by analyzing natural speech data corresponding to the learning text information. A parameter analysis means for generating, the second prosody parameter time series generated by the parameter analysis means, and the first prosody parameter time series generated in order while switching the prosody parameter value by the parameter time series generation means Evaluation means for comparing and evaluating the evaluation result; prosody parameter value learning means for optimizing a prosody parameter value to be applied with a corresponding prosody rule based on the evaluation result of the evaluation means; and statistically processing the evaluation result of the evaluation means. The first prosody rule for speech synthesis, in which an inappropriate prosody rule is detected, the rule is modified, and the modified rule is applied. A trial evaluation requesting means for generating a prosodic parameter time series by the parameter time series generating means and outputting a corresponding synthesized voice for a trial listening evaluation by an operator; and an operator input means for receiving a trial listening evaluation result by the operator. And a prosody rule learning means for correcting and optimizing the inappropriate prosody rule based on a trial evaluation result by the operator input from the operator input means. 3. Generating information on phonemes and prosody of a synthesized speech from input text information, determining prosody parameter values and phonemic symbols of each phoneme of the synthesized speech based on the information, And a speech synthesizer for generating speech parameters necessary for speech synthesis based on phonetic symbols and outputting synthesized speech based on the speech parameters. Text analysis means for generating information representing prosody, prosody rule storage means in which various prosody rules for generating prosody parameters are registered in advance, the information generated by the text analysis means, and the prosody rule storage Based on the corresponding prosody rule registered in the means, a prosody parameter value to be applied in the prosody rule is determined among a plurality of predetermined candidates. Parameter time series generating means for sequentially generating a first prosody parameter time series for speech synthesis while sequentially selecting the second prosody parameter time series by analyzing natural speech data corresponding to the learning text information. A parameter analysis means for generating, the second prosody parameter time series generated by the parameter analysis means, and the first prosody parameter time series generated in order while switching the prosody parameter value by the parameter time series generation means Evaluation means for comparing and evaluating, and an inappropriate prosody rule is detected by statistically processing the evaluation result of the evaluation means, and the rule is modified, and the modified rule is applied, and the modified rule is applied. One prosodic parameter time series is generated by the parameter time series generating means, and the corresponding synthesized speech is evaluated by the operator for trial listening evaluation. Means for requesting a trial evaluation to be output to the user, operator input means to which the result of the trial evaluation by the operator is inputted, and the result of the evaluation by the operator and the result of the trial evaluation by the operator inputted from the operator input means. For each prosody rule, a set of prosody parameter determinant factors considered in the rule and other prosody parameter determinant factors not considered in the rule determined by the corresponding text information is stored. Evaluation result storage means; prosody parameter value learning means for optimizing a prosody parameter value to be applied with a corresponding prosody rule based on the evaluation result of the evaluation means stored in the evaluation result storage means; Statistical processing is performed on the evaluation result of the evaluation means stored in the storage means to detect an inappropriate prosody rule. By performing clustering using the prosodic parameters determinant factors of consideration, speech synthesis apparatus characterized by comprising a prosodic rule learning means for dividing the rules on the basis of the clustering result. 4. The prosody rule learning means statistically processes an evaluation result of the evaluation means stored in the evaluation result storage means to detect an inappropriate prosody rule, and detects the prosody rule which is not considered by the rule. Clustering means for performing clustering using the element of the parameter determining factor, and when the distribution of the clustering result of the clustering means is divided into a plurality, and for each distribution, select text information closest to the center of gravity of the distribution, Prosody rule dividing means for generating a plurality of new prosody rules by dividing the corresponding prosody rules, wherein the audition evaluation requesting means includes a means for dividing the text information selected by the prosody rule dividing means. The corresponding first prosodic parameter time series is generated according to each of the prosodic rules generated by the prosodic rule dividing means. The prosody rule dividing means generates the corresponding synthesized speech for the trial listening evaluation by the operator, and the prosody rule dividing means generates the trial listening result by the operator for the text information selected by itself. 4. The speech synthesizer according to claim 3, wherein it is determined whether or not to use the plurality of generated prosody rules based on the rule. 5. The test evaluation evaluation means, wherein when the number of natural voice data corresponding to the learning text information is equal to or less than a predetermined threshold value, an operator evaluates a synthetic voice corresponding to the learning text information by an operator. Wherein the prosodic parameter value learning means optimizes a prosodic parameter value to be applied in a corresponding prosodic rule based on a trial listening evaluation result by the operator and an evaluation result of the evaluating means. The speech synthesizer according to claim 1. 6. Generating information on phonemes and prosody of a synthesized speech from input text information, determining prosody parameter values and phoneme symbols of each phoneme of the synthesized speech based on the information, and generating the prosody parameter values. An evaluation function optimization method applied to a speech synthesizer that generates speech parameters required for speech synthesis based on a phonetic symbol and outputs a synthesized speech based on the speech parameters, comprising: The information is sequentially analyzed to generate information representing the phoneme and the prosody of the synthesized speech, and based on the generated information and various corresponding prosody rules prepared in advance, the prosody rule is used. While sequentially generating a first prosody parameter time series for speech synthesis while sequentially selecting a prosody parameter value to be applied from among a plurality of predetermined candidates, the learning text information And generating a second prosody parameter time series by analyzing the natural speech data corresponding to the first and second prosody parameter time series. Is compared and evaluated using a predetermined evaluation function, and based on the evaluation result, the prosody parameter value applied by the corresponding prosody rule is optimized, while the first prosody parameter value is sequentially generated while switching the prosody parameter value. Output the synthesized speech corresponding to the prosodic parameter time series of
An evaluation function, comprising: inputting a preview evaluation result by the operator; and correcting the evaluation function in a direction in which the input preview evaluation result by the operator and the evaluation result using the corresponding evaluation function are consistent. Optimization method. 7. Generating information on phonemes and prosody of a synthesized speech from input text information, determining prosody parameter values and phoneme symbols of each phoneme of the synthesized speech based on the information, An evaluation function optimization method applied to a speech synthesizer that generates speech parameters required for speech synthesis based on a phonetic symbol and outputs a synthesized speech based on the speech parameters, comprising: The information is sequentially analyzed to generate information representing the phoneme and the prosody of the synthesized speech, and based on the generated information and various corresponding prosody rules prepared in advance, the prosody rule is used. While sequentially generating a first prosody parameter time series for speech synthesis while sequentially selecting a prosody parameter value to be applied from among a plurality of predetermined candidates, the learning text information And generating a second prosody parameter time series by analyzing the natural speech data corresponding to the first and second prosody parameter time series. Is compared and evaluated using a predetermined evaluation function, and based on the evaluation result, a prosody parameter value to be applied in a corresponding prosody rule is optimized. Detecting, modifying the rule, generating a first prosodic parameter time series for the speech synthesis to which the modified rule is applied, and outputting the corresponding synthesized speech for a trial listening evaluation by the operator; Inputting a result of the audition evaluation by the operator, and correcting and optimizing the inappropriate prosody rule based on the result of the audition evaluation by the input operator. Law optimization method.