KR100620898B1 - Method of speaking rate conversion of text-to-speech system - Google Patents

Abstract

The present invention relates to a method of converting a speech rate of a text-to-speech system, and extracts a speech list from a synthesis DB to generate speech for each speech style consisting of fast speech, normal speech, and slow speech for each synthesis unit. Constructing a duration probability distribution, searching for an optimal synthesis unit candidate sequence through Viterbi search in response to the requested synthesis, generating a duration target parameter of the synthesis unit, and performing the optimal synthesis unit candidate By proceeding to the step of generating the synthesized sound by obtaining the optimum sequence unit candidate string again through the duration parameter of the column, signal processing is performed for the synthesized sound as in the conventional SOLA method because the synthesized sound is generated by 2-pass searching using the new duration. Not only does it have the advantage of not needing, but also the rate of ignition in the equation itself to obtain the new duration. It includes the consideration of the bovine context sensitive context and utterance speed, it is advantageous for the identification of such a context there is no need to create a separate training or prediction models.

Text-to-Speech System, Speaking Rate Conversion, SOLA, Break Indexing

Description

Speech rate conversion method of speech synthesis system {METHOD OF SPEAKING RATE CONVERSION OF TEXT-TO-SPEECH SYSTEM}

1 is a flowchart illustrating a synthesized sound generating procedure of a general synthesizer;

2 is a diagram illustrating a process of constructing a DB for learning duration of a synthesis unit for each speech rate;

3 is a view illustrating a duration model training process dependent on the firing rate for each synthesis unit;

4 is a diagram illustrating an example of a distribution of durations of a synthetic unit having a duration characteristic dependent on a change in ignition rate;

5 is a view showing an example of the distribution of the duration of the synthetic unit having a duration characteristic independent of the change in the ignition rate,

6 is a view showing a conversion rate conversion process,

7 is a diagram illustrating a process of extracting a target duration of a synthesis unit from a one-pass optimal path;

8 is a view illustrating a process of obtaining a modified synthesis unit duration using a 1-pass result and a firing rate dependent duration model of the synthesis unit;

FIG. 9 is a diagram illustrating a search for an optimal synthesis unit candidate using the modified synthesis unit duration as target information and a result thereof.

<Description of the symbols for the main parts of the drawings>

10: preprocessor 20: language processor

30: rhyme processor 40: candidate searcher

41: synthesis unit DB 50: synthesis sound generator

The present invention relates to a method of converting a speech rate of a text-to-speech system, and more particularly, to a method of converting a speech rate of a speech synthesis system using a duration model for each speech rate and a two-step unit selection process. It is about.

The speech rate conversion method of the conventional speech synthesis systems includes a speech rate conversion using a frame-by-frame overlapping method, or partly, by a frame-based OLA (OverLap & Add) technique (particularly, a Synchronous OverLap & Add) method. There are ways to give effect of change of ignition rate conversion by different cutting rate by ignition rate. Here, the SOLA (Synchronous OverLap & Add) method adjusts the frame rate between frames when the voice is decomposed and decomposed into frame units of 20 to 30 msec (when the voice is controlled slowly, the frame period is increased and the voice is increased. When adjusting the speed quickly, reduce the frame period) and overlap & add the decomposed frame to regenerate the speech rate controlled voice. In the overlap & add section, the delay sample position is the maximum correlation between the previous frame and the current frame. To find and apply overlap & add at that point to adjust speed.

However, the first method for converting the speech rate, using the OLA technique is to perform uniform speech rate conversion over the entire sound synthesis without using the upper level knowledge for the speech rate conversion.

The intrinsic length of the phonemes constituting the voice has a context-dependent context and a context-dependent context. Existing OLA technique does not use this higher level knowledge, but converts the speech rate of frame unit for all the synthesized sounds, and changes the duration of context independent of speech rate conversion. A lot of cognitive effort is required when trying to recognize the content.

For example, in the case of Korean ruptured sounds, it is well known in many phonetic experiment papers that the length of the closure of the rupture sound is heard as a rupture sound when the length of the rupture sound is long. (In other words, a normal 'kimchi' can be heard as 'simply'). In another example, the friction sound 'ㅅ' is heard as 'ㅅ' when the length of the friction component is long, and as 'ㅅ' when the length of the friction component is short.

Therefore, the application of the frame-based OLA technique that does not use such higher level information may change the length of the friction component of 'ㅅ' having a length of 60 ms to 40 ms, which requires the user to make more efforts to recognize. (In other words, cognitive effort means that different phonemes in context are intercepted in speech-converted synthesized sounds, and the contents can be understood in the whole sentence, but attention is diverted by other phoneme effects. In other words, when the contents of the entire document are heard, the degree of memory decreases.

The second method for converting the speech rate is based on the fact that people's readings (that is, groups of words that speak together) change according to the speech rate conversion, so that different readings are made according to the speech rate (that is, in fast speech This is an example of changing the speech rate by forming a large group of words and cutting them off.

However, this method can have the effect that the sentence is tediously frequently cut off or cut off too long by simply making different readings, and more technically, the phoneme length does not change according to the conversion of the speech rate. There is a limit to the application of the conversion rate.

Accordingly, an object of the present invention is to overcome the above-described problems, and an object of the present invention is to improve the speech rate conversion so as to compensate for the disadvantage of not using the speech rate conversion information of a higher level of signal processing, which is a problem of the OLA technique. Phenomenon that is independent of speech rate conversion during synthesis, is automatically reflected to other sounds because the phoneme context independent of dependent phoneme context and speech rate conversion can be automatically learned from training data. The present invention provides a method of converting a speech rate of a speech synthesis system that can reduce the noise.

Another object of the present invention is to solve the disadvantage that can not only convert the speech rate by the length of the phoneme, which is a disadvantage of the speech rate conversion technology modified by the cut-off rule, so that only the limited speech level can be converted. The present invention provides a method of converting a speech rate of a speech synthesis system using a synthesis by generating a model that can be learned from training data so that a duration length dependent on a speech rate can be controlled in subword units.

Hereinafter, a method for converting a speech rate of a speech synthesis system using a duration model for each speech rate and a 2-step unit selection process will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a synthesis sound generation procedure of a general synthesizer. As shown in FIG. 1, a speech synthesis system including a preprocessor 10, a language processor 20, a rhyme processor 30, a candidate searcher 40, a synthesis unit DB 50, and a synthesized speech generator 60 is illustrated. By using the processing of the input sentence sequentially using the synthesized sound. Here, as described above, the speech rate is converted by applying the OLA technique on a frame-by-frame basis to the generated synthesized sound.

However, in the present invention, through the process of building a model in the duration of the synthesis unit dependent on the ignition rate represented by Fig. 2 and 3, the change in the ignition rate according to the ignition rate change as shown in Fig. 4 Continuous probability distribution for the duration of the synthetic unit independent of the change in the ignition rate does not change according to the change in the ignition rate as shown in FIG. The information thus obtained can be assumed to be a synthesis unit independent of the ignition rate of a synthesis unit whose movement range of the central value of the continuous probability distribution is less than x, and a synthesis unit independent of ignition rate. Here, x is a value indicating the degree of movement of the center value, and may be arbitrarily set, and the threshold value may be introduced to determine whether or not the synthesis unit is dependent on the firing rate. Using this information, it is possible to convert the firing rate only for the units that are dependent on the firing rate of the synthetic units. Here, the process of obtaining the continuous probability distribution of the durations of the synthetic units according to the firing rate is defined as the training process.

6 is a diagram illustrating a process of converting an ignition rate.

Referring to FIG. 6, as shown in FIG. 2, an optimal training list for generating a duration training model for each synthesis unit is extracted from the synthesis DB (S1). The extracted training list is recorded as normal speech, fast speech and slow speech (S2). Subsequently, as shown in FIG. 3, a continuous probability distribution of durations of firing rate dependent synthesis units is obtained from each of the normal fired training DB, the fast fired training DB, and the slow fired training DB (S3).

Subsequently, when there is a synthesis request by the user, an optimum synthesis unit duration is calculated through the Viterbi search (S4) in response to the user request (S5). This process is defined as a first unit selection process in the present invention. This is illustrated in Figure 7 the process of extracting the target duration of the synthesis unit from the one-pass optimal path.

Next, for the model of the duration of the composite units in the selected slow or fast utterance, the final duration affected by the rate of utterance is finally obtained using the target duration and the continuous probability distribution of the candidates of the composite unit in the normal utterance. A new target duration parameter of the synthesis unit is calculated (S6). The procedure for obtaining the new synthesis unit duration is shown in detail in FIG. 8.

Subsequently, the calculated new target duration parameter is again obtained through the Viterbi search (S7) to obtain the optimal synthesis unit candidate sequence dependent on the duration, and the synthesized sound is generated using the optimal synthesis unit candidate sequence dependent on the obtained duration. It is generated (S8). The optimal synthesis unit candidate search using the modified synthesis unit duration as target information and the results are well illustrated in FIG. 9.

Hereinafter, the ignition rate conversion process of the present invention will be described using specific equations.

T in [Equation 1] means a synthesis unit used in the synthesizer. In practical implementations, T means something like a semitone, or a phoneme, or a context-dependent phone (such as biphone, triphone, etc.). In general, the synthesizer defines and uses N synthetic units larger than one. In the case of Korean, if the phoneme unit is used as a synthesis unit, the number of consonants and vowels is N.

Here, S means a synthesis DB used in the synthesizer. In general, in the case of a large-scale corpus-based speech synthesis system, a synthesized DB composed of M sentences, words, or phrases is processed to produce a synthesized voice, and a synthesizer is implemented using the synthesized voice. In general, for large-capacity corpus, a synthesizer is developed using hundreds to thousands of sentences, so M can be assumed to be hundreds to thousands.

In the present invention, in order to obtain the continuous probability density of the duration of each synthesis unit with respect to the duration of the speech rate, fast speech and slow speech are required for the same contents, so that all the M sentences of the synthetic DB are fast and slow. Recording with is inappropriate. Therefore, we need to extract k sentences that are sufficient for duration modeling but much smaller than M. There are various ways to extract these k sentences. In the present invention, a method of extracting k sentences is not separately defined.

라 하고, 다음과 같이 정의할 수 있다. This list of voices

It can be defined as follows.

를 빠른 발화와 느린 발화로 발성시켜 각각을 훈련DB로 생성한다. 이렇게 생성된

에 대한 느린 발화의 훈련데이터를

라 정의하고 빠른 발화의 훈련데이터를

라 정의하기로 한다. Voice List extracted from Equation 3

Ignite with fast and slow utterances to generate each as a training DB. So generated

Slow ignition training data

Define training data for rapid ignition

Let's define.

을 다음과 같이 정의할 수 있다. This will give you a training set

Can be defined as:

은 별도로 녹음하는 것이 아니라 원본 합성DB에서 추출된 것을 의미하며, 원본 합성DB는 정상적인 발화(보통의 발화속도)속도로 녹음된 것임을 가정한다. here

Means that it is extracted from the original synthesis DB, not recorded separately. It is assumed that the original synthesis DB was recorded at a normal utterance (normal ignition rate) speed.

을 이용하여 각각의 발화속도에 대한 합성단위들의 지속시간에 대한 연속확률분포를 정의한다. 여기서는 연속확률분포는 가우시안(Gaussian) 분포를 가정한다. Training set defined in [Equation 4]

We define the continuous probability distribution of the durations of the composite units for each firing rate using. The continuous probability distribution assumes a Gaussian distribution.

가 있다면, 훈련DB구축을 위해 동일한 훈련셋을 이용하여 구축하였으므로 각 발화속도별 훈련셋

에 동일한 수 만큼 출현한다. 여기서, 각 셋에 들어있는

의 수를 L개라 가정하고, 각 훈련셋에 들어있는

의 지속시간 분포를 가우시안으로 모델링하기 위해서는 각 훈련셋에서의

에 대한 평균과 분산을 추정하면 된다. 여기에 ML기법을 적용하면 각 발성스타일에서의

에 지속시간 분포에 평균과 분산의 추정은, 주어진 훈련셋

각각에서의 샘플평균과 샘플분산으로 대치할 수 있다. For example, some synthetic unit

If there is, training set for each ignition rate because it was constructed using the same training set for training DB construction.

Appears at the same number as. Where each of the three

Assume the number of L's

To model Gaussian's duration distribution as Gaussian,

Estimate the mean and variance for. If you apply ML technique to each

Estimation of mean and variance in the distribution of durations in the given training set

It can be replaced by the sample mean and sample variance in each.

에 대한 가우시안연속확률분포에 관한 샘플평균과 샘플분산을 계산하여 얻을 수 있으며, 발화속도별 합성단위의 연속확률분포는 다음과 같이 표현할 수 있다. Therefore, after the basic statistical calculation process, the synthesis unit in the training DB constructed for each ignition rate

The sample mean and sample variance of the Gaussian continuous probability distribution for can be obtained by calculating, and the continuous probability distribution of the synthesis unit for each ignition rate can be expressed as

합성단위에 대한

를 구하게 되면

에 대한 발화속도의 변화에 따른 지속시간의 모델링 과정이 끝나게 된다. 이 과정을 L개의 합성단위에 대해 수행하여 훈련과정을 마치게 된다. After the above process finally

For compound units

If you find

The process of modeling the duration according to the change of the firing rate for is finished. This process is completed for L synthesis units to complete the training process.

The following describes how to perform the firing rate conversion using the probability distribution for each unit of composition for the duration of each firing rate.

라 하자. 그러면

는 다음과 같이 합성단위의 연속으로 정의할 수 있다. Synthesis sentences (or any unit, such as syllables, phonemes, words, phrases, words, paragraphs, documents, etc.) that are requested to be synthesized in a system

Let's do it. then

Can be defined as a sequence of synthetic units as follows:

는 n개의 합성단위열의 연결로 표현할 수 있다. That is, the input sentence

Can be expressed as a concatenation of n compound units.

에 대한 최적의 합성단위

를 구하는 과정은 대용량 코퍼스 기반 방식에서는 통상적인 방법이므로 상술하지 않는다. 이 과정을 제 1 단위 선택 (unit selection) 과정이라 정의한다. 상기 제 1 단위 선택 과정을 거쳐 최종적으로 선택된 n개의 최적합성단위 후보열을

라고 할 때, 다음과 같이 정의할 수 있다. We generate n composite unit sequences and use them to perform Viterbi search in consideration of the link cost between the target cost function and the composite unit candidates.

Optimal Synthetic Unit for

Since the process of obtaining is a conventional method in a large-scale corpus-based method, it is not described in detail. This process is defined as a first unit selection process. The n best synthesis unit candidate sequences finally selected through the first unit selection process are selected.

In this case, it can be defined as follows.

는 합성DB에서 선택하여온 것으로 그 지속시간은 미리 알 수 있다. 그러므로,

내의 각각의 최적합성단위후보

들의 지속시간을

라 정의할 수 있다. 이 과정을 도식화 한 것이 도 7이다. Each candidate compound unit candidate obtained in this way

Is selected from the synthesis DB, and its duration can be known in advance. therefore,

Candidates for each optimal synthesis unit in

Their duration

Can be defined. Fig. 7 illustrates this process.

이라고 할 때,

이면 발화속도를 빠르게 변환하는 것으로, 미리 훈련된

분포정보를 이용해

값을

값으로 변환하는 과정이다. The next step of the present invention, the conversion rate conversion rate required by the user

When we say,

Is to quickly change the rate of fire,

Using distribution information

Value

This is the process of converting it to a value.

This process is expressed as an expression as follows.

를 의미한다. 이 과정을 거쳐 모든

에 대한

를 구하게 되면 발화속도 변환을 위한 계산과정이 완료된다. Where t is

Means. After this process all

For

When is calculated, the calculation process for converting the firing rate is completed.

정보를

의

에 대한 지속시간 파라미터로 활용하여 단위 선택 과정을 거친다는 점이다. Next, the second unit selection process is again a process of searching for the best synthesis unit candidate using the Viterbi search of the conventional synthesizer. However, the difference from the first unit selection process is obtained by Equation (8).

Information

of

It is a unit selection process using as a duration parameter for.

As described above, in the speech rate conversion method of the speech synthesis system according to the present invention, using a small amount of training DB recorded for each speech rate, a continuous probability distribution function for the duration of the synthesis unit is obtained and compared with the normal speech. Considering the shift and variance of the central values, we can see how sensitive the synthesis unit of a context changes to what firing rate.

That is, in the duration distribution of a compounding unit, as shown in FIG. 4, when the distance between the center values for each firing rate and the magnitude of the dispersion degree are different, the compounding unit of the context in which the duration varies depending on the firing rate. 5, if the difference between the center value for each ignition rate and the difference in dispersion degree is small, it can be determined that this synthesis unit is a synthetic unit commonly used in a context that is less sensitive to the ignition rate. Alternatively, if the movement of the center value is greater than x or the variation of the variance is greater than y by introducing a threshold, it is defined as a context dependent on the ignition rate, and the duration change using the existing SOLA method only for the synthesis unit of this context is defined. If the movement of the center value is less than x, or if the change in the degree of dispersion is less than y, it is judged as a composite unit of the context that is not affected by the ignition rate and the existing SOLA method is not applied to this part. It is possible to solve the problem that the conversion of the speech rate cannot be performed in consideration of the context independent of the speech rate, which is a problem of the SOLA method.

를 이용하여 2-패스 검색하여 합성음을 생성하므로 기존의 SOLA방식처럼 합성음에 대해 신호처리를 하지 않아도 된다는 장점이 있으며, 이는 발화속도 변환의 실시간성을 높일 수 있다. The method also provides a new duration target modified during the unit selection process of the synthesizer.

Since two-pass search is used to generate synthesized sound, there is an advantage that signal processing is not required for synthesized sound as in the conventional SOLA method.

Since it performs a two-pass search (second unit selection process), it may raise the problem that it takes more time to search the same search space twice, but this part is optimal for one-pass search. If only N-best of each synthesis unit candidate is set as the search space, the search space can be reduced, so as not to impair the real-time.

를 구하는 계산식 자체에 발화속도에 민감한 문맥과 발화속도에 둔감한 문맥에 대한 고려가 포함되어 있어, 이러한 문맥의 식별을 위해 별도의 훈련이나 예측모델을 생성할 필요가 없다는 점이다. In addition, the biggest feature of the present invention

The calculation itself requires the consideration of contexts sensitive to speech rate and context insensitive to speech rate, so there is no need to create a separate training or prediction model to identify these contexts.

Claims (6)

A first step of extracting a speech list from the synthesis DB to generate a duration probability distribution for each synthesis unit by uttering each utterance style including fast utterance, normal utterance, and slow utterance Retrieving an optimal synthesis unit candidate sequence through a Viterbi search in response to the requested synthesis, and generating a duration target parameter of the synthesis unit; And A third step of reconstructing the optimum synthesis unit candidate sequence through the duration parameter of the optimal synthesis unit candidate sequence and generating a synthesized sound; Speech rate conversion method of the speech synthesis system comprising a. The method of claim 1, wherein the first step, Extracting an optimal training list for generating a duration training model for each synthesis unit from the synthesis DB; Recording the extracted training list with fast speech and slow speech; And Obtaining the continuous probability distribution of the duration of each firing rate dependent synthesis unit from each of the fast ignition training database and the slow ignition training database Speech rate conversion method of the speech synthesis system, characterized in that consisting of. The method according to claim 1 or 2, wherein in the first step, Synthesis unit for each utterance style (

Continuous probability distribution over the duration of

) Is a method of converting the speech rate of the speech synthesis system, characterized in that represented by the following [Equation 5]. [Equation 5]

4. The method according to claim 3, wherein the normal speech is obtained from an original synthesis DB. The method of claim 1, wherein the third step, For the model of the duration of the composite units in the selected slow or fast utterance, the target duration and the duration continuous probability distributions of the candidates for the composite unit in the normal utterance are used to determine the final composite units affected by the rate of utterance. Calculating a new target duration parameter; Re-establishing the optimal synthesis unit candidate sequence dependent on the duration by searching for the calculated new target duration parameter through Viterbi; And Generating a synthesized sound by using the optimum synthesized unit candidate sequence dependent on the re-established duration; Speech rate conversion method of the speech synthesis system, characterized in that consisting of. The method of claim 5, wherein The new target duration parameter

The conversion process to the speech rate conversion method of the speech synthesis system, characterized in that expressed by Equation 8 below. [Equation 8]

(here,

Is the fire rate conversion rate required by the user, and t is

Means each)

Images