KR100346790B1 - Postprocessing method for automatic phoneme segmentation - Google Patents

Abstract

The present invention relates to an automatic synthesis unit generator technology of a text / voice converter, and to a post-division processing method for reducing a phoneme boundary error in a phoneme partition using an automatic phoneme divider. In order to reduce, the boundary error of the phoneme by the statistical method is corrected, and then the phoneme boundary search interval is set by the statistical method, and the post-processing using the neural network is performed by applying robust feature parameters to the phoneme environment. It can be used for constructing large-scale synthesis rhyme database and automatic generation of synthesis unit.In the post-processing part, correction using statistical information and application of feature parameter feature set according to phonological environment and acoustic characteristics using neural network Regularization can be used to improve phone segmentation and performance. It is possible to improve performance than automatic phoneme splitter by correcting synthesized database spoken by lip word with postprocessor, and to improve absolute error, and postprocessed according to phonological environment for synthetic database spoken by sentence unit. When applied, it is possible to reduce the range of automatic segmentation error by postprocessing using neural network, which shows superior performance compared to applying postprocessing alone, and improves the frame segmentation rate and improves the performance in terms of absolute error. It has an effect that can be used to build a large amount of synthesis rhyme database and automatic generation of synthesis units.

Description

Postprocessing method for automatic phoneme segmentation}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic synthesis unit generator technology of a text / voice converter, and more particularly, to a phoneme post-processing method for reducing phoneme boundary errors in phoneme division using an automatic phoneme divider.

In general, automatically divided phoneme boundaries always have errors, and these errors become inaccurate data when using phoneme information, making them difficult to use for rhyme modeling or synthesis units.

Recently, a synthesis system using such a synthesis method has been developed at home and abroad as a speech synthesis method using a large-scale rhyme database generates synthesized sound having better performance than the conventional synthesis method.

However, since it takes considerable time and cost to produce a large database, automatic phoneme splitting technology becomes very important as the automation of database creation is urgently needed.

ETRI's voice recognition system, which is currently used as an automatic phoneme-division technology, is about 80% or more of the phonemes of segmentation in FM radio news sentences, dialogue sentences, and reading texts. The error due to the difference is segmented within 30msec, and shows about 60% performance for the isolated word.

However, since the phoneme is divided in the phoneme splitting performance to generate a synthesized sound that is somewhat degraded when used as a speech synthesis unit, it is necessary to correct the phoneme boundary using a postprocessor.

The conventional post-processor has a structure for training a phoneme splitter (neural network, HMM) by extracting spectral feature parameters, which is almost the same as that of an automatic phone splitter.

Therefore, the performance by the post-processor is difficult to improve more than the performance of the automatic phone splitter.

As described above, the conventionally divided phoneme boundary always has errors, and these errors become inaccurate data when using phoneme information, which makes it difficult to use them for rhyme modeling or synthesis units. The performance of the divider is about 20% absolute sentence error more than 30msec in sentence unit speech, the performance of the isolated word was very poor.

In order to solve the above problems, an object of the present invention is to provide a method for improving the performance of the automatic phoneme segmentation used in the automatic synthesis unit generator of the text / speech converter.

In order to achieve the above object, the present invention is characterized by post-treatment with a trained neural network after training a manually corrected phoneme boundary database (DB).

According to the present invention, the automatic phone splitter divides the phoneme partition type so that the phoneme boundary type is constant according to the phonetic environment. And the like.

The type of phoneme boundary error is constant according to phoneme environment, which means that the degree of phoneme boundary error and the direction of error is almost constant according to phoneme environment compared to manual division by voice expert.

Therefore, it is possible to know the error characteristics of the automatic phoneme splitter by comparing the automatic segmentation result with the manually corrected segmentation result.

In order to quantify the error characteristic, the error (= | manually corrected phoneme location-automatically divided phoneme location |) is calculated for each phoneme environment and the standard deviation is obtained.

The phonological environment mean value is applied to the remaining automatic segmentation results excluded from the training data, and the phoneme boundary error can be corrected first using the statistics (mean, standard deviation) of phoneme boundary errors by phoneme environment. It is used to set the phoneme boundary search section during post-processing.

The characteristic of dividing the phone to include almost stable sections of phonemes means that large errors beyond the left and right phoneme boundaries rarely occur.

In particular, in the field of speech synthesis, the phoneme to be divided is known in advance, so using this characteristic, different feature parameters can be used depending on the phonetic environment.

For example, a phoneme boundary correction in a "voiced + unvoiced" environment can provide time-domain feature parameters such as zero crossing rate, energy, and a "non-vowel + vowel" environment a high-band / low-band energy ratio of a frequency. The environment may use frequency domain spectrum information.

Therefore, it is possible to use robust feature parameters according to phonological environment than the conventional method using the same feature parameters for all phonemes, thereby improving phoneme splitting performance.

If you already know the information about the divided phonemes, using a knowledge-based expert system phoneme splitting method that uses the information to regulate the acoustic changes between phonemes by the experts and results in near manual division. We can obtain a large amount of time, but we need to take considerable time to analyze and normalize the acoustic characteristics between the various phoneme boundaries, so we order these acoustic changes through a training algorithm (multilayer neural network).

The training process is the process of extracting the rules from the manual segmented results. The manual segmented results are input to the neural network, and the rules are prepared by neural network training.

In the present invention, the performance of the automatic phoneme divider and the post processor is analyzed and analyzed by phonological environment, and the phoneme divider is divided into a phonological environment with a higher phoneme performance and a phonological environment with a higher postprocessor performance.

This is because the performance of the post-processor is not always improved, but may be lower than the automatic divider performance.

The performance of the automatic phoneme divider by phoneme environment showed better performance in the postprocessor at all "phoneme + cost" and "vowel + noise" boundaries.

In the other phonological environment, the performance of the post-processor is better. Among them, the robust feature parameters are re-selected for each phonological environment.

In order to select robust feature parameters for each phonological environment, the present invention sets the ninth order such as zero-crossing ratio extracted from the time domain and energy information for each band, which is spectrum information appearing in the energy and frequency domain, as Feature Parameter I, and sets the hearing characteristics of a person. After the speech features were extracted using the 13th-order Perceptual Linear Prediction (PLP) coefficients as the feature parameter Feature Set II, the neural network was trained, and the performance of post-segmentation was investigated.

The feature set I is "[voice, unvoiced consonant] + vowel", "all phonemes + unvoiced consonant", whereas in the feature set II, "[noise, voiced, unvoiced consonant] + vowel", "non-voice + naive", In the case of "noise + voice" and "non-voice + unvoiced", the performance was high.

Therefore, the performance of the automatic splitter showed a performance difference according to the phoneme or phoneme environment, and it was found that the segmentation performance of Feature Set I and II was different for a particular phoneme environment. Strong in classification of unvoiced sound and Feature Set II showed good performance in vowel, voiced and non-voiced parts.

In the final post-processing process, a new phoneme boundary is determined through the invention, that is, correction using statistical information, robust feature parameters for each phonological environment, and neural network training.

1 is a neural network training flow chart of the present invention,

2 is a phoneme post-processing flow chart to which the present invention is applied.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a neural network training flow diagram of the present invention, and relates to a training arrangement for creating an acoustic change rule at the phoneme boundary.

In the training process, the mean value and standard deviation of the phoneme boundary error, which are statistical information, are extracted from the manual phoneme segmentation database (S1) (S2), and the feature parameters for training the neural network are extracted (S3). Train neural networks according to I and II (S4).

At this time, the training is completed when the neural network output value reaches the threshold, otherwise the training is repeated.

That is, if it is determined whether to continue the training (S5), if the training is not continued, the neural network training is performed again according to the feature parameter, and if the training is continued after the determination, the weight (weight) I, which is the training parameter of the neural network, is continued. After extracting the value II, save it to the hard disk (S6) and finish the training (S7).

Figure 2 is a phoneme post-processing flow chart to which the present invention is applied, and relates to a configuration for post-processing using a trained neural network.

The post-processing process primarily corrects the error using the statistical information extracted during the training process (corrected phoneme position = auto-segmented phoneme position + error average) (S8) and performs automatic phoneme division database (DB). In order to train the neural network (S9), two types of parameters are extracted from manually segmented voice data (S10).

The phonological environment is selected after the parameter extraction (S11). In the present invention, the ninth order such as the zero crossing rate extracted in the time domain and the energy information for each band, which is spectral information appearing in the energy and frequency domain, is set as the feature parameter I (S11a). For example, the 13th order of the perceptual linear prediction (PLP) coefficient modeling the human auditory characteristics is used as the feature parameter Feature Set II (S11b).

In addition, the characteristic parameters and weight values are applied differently for each phonological environment (S11, S12), and the phonological environment is better when the phonological environment is Ⅲ, as shown in Table 1 below. Finish the post-treatment without going through the process (S13).

Type I Ⅱ Ⅲ Environment [Voice, unvoiced consonant] + vowel all phonemes except non-phone + unvoiced [Non-sound, voiced, unvoiced consonant] + vowel sound + non-unified sound + non-sounded sound + unvoiced consonant All phonemes except nasal + nasal vowel + sound

After the above process, the phoneme boundary search section (S15) is determined according to the feature parameter. In step 1, the automatic segment boundary is determined as the neural network output value search section from the left and right two frames (± 25 msec). Determine the maximum two of the neural network outputs above the threshold and see the postprocessed boundary as the location closer to the auto segment.

In step 2, when there is no neural network output value above the threshold in the search section, the search section is extended from a table in which statistical data according to the phonological environment is registered to a 90% confidence interval, and in the extended search section. The process of 1 is repeated, in which the extended range of the search section follows Equation 1 below.

mean of if error <0 [mean-standard deviation * 1.645, 25]

Else [-25, Mean + Standard Deviation * 1.645]

When the search section is determined, if a neural network output value of more than a threshold value exists within the section, the position having the maximum value among the threshold values is selected as the post-processed phoneme boundary (S16).

If there is no neural network output value above the threshold in the search section (S17), if the output value does not exist, the automatic phoneme division result is used as it is (S15). In the processing portion, correction using statistical information is performed (S18), and the post-processing is completed (S13).

As described above, the present invention can postprocess the result of automatic analysis using a multilayer neural network, thereby reducing the range of boundary errors, and ultimately, can be used for constructing a large-scale synthesis rate database and automatically generating the synthesis unit. In the part, it can be confirmed that the phoneme segmentation and performance can be improved by using statistical information, applying feature parameter feature sets according to phonological environment, and using acoustic characteristic regularization using neural networks.

In addition, the analysis results corrected by the post-processor in the synthetic database of isolated words showed about 25% improvement over the automatic phoneme splitter, and the absolute error improved by about 39%.

In addition, when the postprocessing was applied separately according to the phonological environment, the synthesized database spoken by sentence unit showed better performance than the postprocessing alone, and the frame segmentation rate improved about 17.7% for the automatic segmenter. In addition, in terms of absolute error, the performance of 28.6% shows that the post-processing using neural networks can reduce the range of automatic segmentation errors, and can be used to construct a large-scale synthesis rhyme database and automatically generate synthesis units. Has

Claims (7)

In the phoneme segmentation method for reducing phoneme boundary errors in phoneme segmentation using an automatic phoneme divider, A first step of correcting boundary errors of phonemes by a statistical method; A second step of setting a phoneme boundary search section in a statistical manner after performing the first step; Applying a robust feature parameter to the phonological environment after performing the second step; And a fourth step of performing post-processing using a neural network after performing the third step. The method of claim 1 wherein the first step is A first substep of extracting an average value and a standard deviation of a phoneme boundary error, which is statistical information, from a manual phoneme segmentation database; A second substep of extracting feature parameters for training a neural network after performing the first substep; And a third substep of training the neural networks according to the feature parameters I and II extracted after performing the second substep. The method of claim 2, wherein the third substep is Completion of training when the neural network output value reaches a threshold value, and repeats the training if it is not. The method of claim 1, wherein the second step A first sub-step of performing neural network training again according to the feature parameter if the training is not continued after determining whether to continue the training; And a second sub-step of finishing the training after extracting the weighting values I and II, which are training parameters of the neural network, in the hard disk when continuing the training after the determination. The method of claim 1, wherein the third step A first sub-step of correcting errors primarily by using statistical information extracted in a post-processing training process; A second substep of receiving an autonomous phoneme division database after performing the first substep and extracting a plurality of types of parameters from manually segmented voice data for training a neural network; Perceptual linear prediction (PLP) modeling the auditory characteristics of a person using feature set Ⅰ, the 9th order, such as the zero-crossing ratio extracted from the time domain after the second sub-step, and the band-specific energy information, which is the spectrum information appearing in the energy and frequency domain. A third sub-step of selecting a phonological environment by extracting a speech feature using the coefficient 13th as a feature parameter Feature Set II; And a fourth sub-step of performing post-processing by applying the feature parameter weighting value to each phonological environment after performing the third sub-step. The method of claim 5, wherein the first substep The phoneme post-processing method of claim 1, wherein the corrected phoneme position is determined by using the automatically segmented phoneme position and an average value of errors. The method of claim 1, wherein the fourth step A first sub-step of selecting a position having a maximum value among the threshold values as a post-processed phoneme boundary if a neural network output value of a threshold value or more exists in the interval after the search interval determination; A second sub-step of determining whether a neural network output value greater than or equal to a threshold value exists in the search section and using the automatic phoneme division result if it does not exist; And a third sub-step for finishing the post-processing after performing correction using statistical information in the post-processing part when the neural network output value above the threshold exists in the search section.