JPH08263520A - System and method for speech file constitution - Google Patents

Abstract

PURPOSE: To constitute a speech file which is highly comprehensive for both context and rhythm in the speech file constitution system which is applied to waveform synthesis for obtaining a synthesized sound by connecting phoneme waveforms. CONSTITUTION: First, waveform data having the same phoneme labels are segmented as an initial cluster 110 form speech waveform data stored in a speech data base. Next, clustering based on context is performed in a specific characteristic parameter space for the initial cluster 110. Then, clustering in a specific rhythm pattern space is performed for respective clusters 130, 140, and 150 obtained by the context clustering. Lastly, waveform data which are closest to the centroids of fine clusters 131-133, 141-143, and 151-153 obtained by the rhythm clustering are extracted from the fine clusters 131-133, 141-143, and 151-153 and registered in the speech file 160 for synthesis.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention produces a synthetic voice by selecting and connecting an appropriate speech waveform segment from a set of speech waveform segments which are cut out from a natural utterance and stored in a memory, an external storage device or the like. The present invention relates to a technique for constructing an audio file applied to obtain waveform synthesis.

[0002]

2. Description of the Related Art Conventionally, research on a waveform synthesizing technique capable of obtaining a natural synthesized voice has been widely conducted, but a method of constructing a voice file for waveform synthesizing has not been established, and is mainly tested. The reality is that it is done manually based on the subjectivity of the person.

[0003] Of these, the one known as a conventional automatic audio file construction method is Context Oriented Clusterin.
g; hereinafter referred to as COC) (Ito, Nakajima, Hirokawa: Scholarly Technique S
P93-121, (1994-01)). This is the waveform data that is closest to the centroid of each cluster (cluster centroid) obtained by clustering speech waveforms based on context (phoneme sequence before and after the phoneme of interest) based on acoustic characteristics. Is a method of automatically configuring a sound file.

By the way, when performing waveform synthesis, in order to obtain a good synthesized voice, based on the context and prosody (characteristics such as the pitch shape of the voice, the temporal length, and the size) of the phoneme, It is necessary to select and use an appropriate waveform element. For this reason, sufficient comprehensiveness is assured for various audio events so that a situation in which no appropriate waveform element is found does not occur in the audio file that is referred to when selecting the waveform element. There must be.

Looking at the above-mentioned COC from this point of view,
COC is originally positioned as a voice file construction method in LSP (line spectrum pair) voice synthesis, and stores individual voice waveform segments in the form of LSP parameters. Here, since the LSP parameter has a feature of good interpolating property and high degree of freedom of deformation, in the LSP voice synthesis, an average spectral feature in each context is stored in the voice file and then the waveform synthesis is performed. Has
A method is used in which the prosodic features of the speech waveform segment are modified and used to suit the desired speech. Therefore, COC
In, the emphasis is placed on the classification of spectral features according to context, and it is not necessary to consider prosodic features in the audio file structure.

[0006]

On the other hand, in the case of a voice file in which the voice waveform itself is stored as a voice unit,
Since excessive waveform transformation leads to deterioration in the naturalness of the synthesized voice quality, the audio file must have a structure that covers not only spectral features but also prosodic features in order to minimize waveform transformation.

However, in the above COC, although the context is sufficiently exhaustive, the prosody is not taken into consideration. Therefore, when the dictionary (speech file) is constructed only by classifying the spectral features according to the context, the speech synthesis is performed. Will be more dependent on the waveform deformation. Therefore, in the case of prosody control, there is a case where the waveform must be excessively deformed, resulting in a problem that the naturalness of the waveform is impaired and the synthesized sound quality is deteriorated.

As described above, in the conventional audio file construction method, it is not possible to construct an audio file having high comprehensiveness in both context and prosody.

[0009] Therefore, an object of the present invention is to provide a voice file structuring system capable of structuring a voice file having high coverage in both context and prosody.

[0010]

SUMMARY OF THE INVENTION The present invention is a method of constructing a voice file used for waveform synthesis to obtain a synthesized voice by combining voice waveform segments cut out from a natural utterance. Context-based clustering in a given feature parameter space for a speech database accumulating speech waveform segments of the above and a set of speech waveform segments having the same phonemes in this speech database (hereinafter referred to as context clustering) And a prosodic clustering means for performing clustering in a predetermined prosodic parameter space (hereinafter referred to as prosodic clustering) on each cluster obtained by context clustering, and an individual cluster obtained by prosodic clustering. Seeking Ntoroido, characterized in that it comprises a speech element registering means for registering the audio file by extracting the voice waveform segments in the vicinity of the centroid.

The present invention also provides a method for constructing an audio file performed by the above method.

[0012]

According to the present invention, a set of speech waveform units of the same phoneme is extracted from a speech database prepared in advance, and first, by performing context clustering on the same phoneme waveform set, the context is based. Multiple clusters are obtained. Next, by applying prosodic clustering to each cluster obtained by this context clustering, each cluster is divided into a plurality of small clusters having different prosodic environments. Next, the waveform segment near the centroid is extracted from each of these small clusters and registered in the audio file. As a result, an audio file having a high degree of comprehensiveness in both context and prosody is constructed.

By using this audio file, a waveform segment having a prosody close to a desired prosody can be obtained for a desired phoneme and context at the time of waveform synthesis. It is possible to prevent the deterioration of the quality of the synthetic speech as a result.

In the preferred embodiment, the context clustering means and the prosody clustering means are each based on the gain of cluster partitioning in clustering,
It has an end determination means for determining the end of clustering. According to this end determination means, it is possible to end the clustering at an appropriate stage.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below in detail with reference to the drawings. It should be noted that the following embodiments can be realized by using a well-known general-purpose computer, and those skilled in the art can easily implement the present invention without understanding the hardware configuration thereof and only understanding the processing steps thereof. Therefore, only the processing steps of the embodiment will be described below.

[1] Voice Database FIG. 1 shows the configuration of a voice database prepared as a material for creating a voice file in one embodiment of the present invention.

The voice database 100 shown in FIG.
This is obtained by performing phoneme labeling on a naturally uttered voice and associating which section of the speech waveform corresponds to which phoneme. This voice database 10
Reference numeral 0 indicates a waveform data file 101 (in FIG. 1, the waveforms of individual phoneme sections are shown separately) in which voice waveform data is accumulated, and phonetic symbols (hereinafter referred to as “phonetic symbols”) corresponding to the respective phoneme sections of the waveform data. , Phoneme label) and phonetic symbols of its context (for example, one phoneme before and after each phoneme) and prosodic parameters (for example, pitch [Hz ], Pitch inclination [Hz / ms], power [d
B] and a prosody data file 103 in which the time length [ms]) is described.

[2] Overall Processing FIG. 2 schematically shows an overall processing procedure according to an embodiment of the present invention.

In FIG. 2, first, waveform data of a phoneme section having the same phoneme label is cut out from the original speech database (see FIG. 1) 100 (in the illustrated example, the phoneme label / a / is included). Waveform data has been extracted). Then, the waveform data set thus obtained is set as the initial cluster 110.

Next, for the initial cluster 110, context-based clustering (context classing) is performed in a space of feature parameters (hereinafter referred to as feature parameter space). Here, the characteristic parameter is a parameter representing the characteristic of the spectrum of the phoneme section, such as a cepstrum or an LSP.

By this context clustering,
The initial cluster 110 is divided into several clusters with different contexts. For example, in the illustrated example, the subsequent phoneme is divided into a cluster 120 that is a voiced sound and a cluster 130 that is an unvoiced sound. Further, the cluster 120 includes a cluster 140 of a phoneme at the beginning of a word and a cluster 150 of a phoneme in a word.
It is divided into and. Each cluster 130, 140, 15
0 is composed of waveform segment sets 139, 149, and 159 having similar contexts of feature parameters.

Next, the clusters 130, 140,
Further clustering (prosodic clustering) is performed on a space of prosodic parameters (for example, pitch, pitch gradient, power, and time length) (hereinafter referred to as prosodic parameter space) for each of 150. For example, if the phoneme waveform data (hereinafter referred to as cluster elements) belonging to each cluster 130, 140, 150 is plotted on the prosodic parameter space P1, P2, P3 of each cluster as shown in the figure, the prosody is By clustering,
Broken lines 134 to 136, 144 to 146, and 15 in the figure
Boundaries such as 4 to 156 are drawn, and finer clusters 131 to 133, 141 to 143, and 151 to 15 are created.
Divided into three.

Next, these subdivided clusters 131 ...
A centroid is obtained from each of 133, 141 to 143, and 151 to 153, waveform data in the vicinity of the centroid is extracted, and registered in the synthesis voice file 160. At the same time, the context and prosody parameters of the extracted waveform data are also stored in the synthesis voice file 160.

A voice file for one phoneme / a / is constructed by going through the above-mentioned processing steps. For other phonemes, a voice file is constructed through similar processing steps.

As described above, in the present embodiment, by further performing prosodic clustering on the clusters obtained by context clustering, a plurality of waveform data are extracted in consideration of the distribution of elements in the prosodic parameter space. , It is possible to improve not only the articulatory fluctuations, that is, the variation of acoustic features of the phoneme due to the influence of contexts before and after, but also the comprehensiveness of speech events including the difference of prosodic features.

[3] Context Clustering FIG. 3 is a flowchart showing the process of context clustering in this embodiment.

In FIG. 3, first, the voice database 1
From the phoneme-labeled waveform data in 00, all the waveform data to which the same phoneme label is given are taken out and set as the initial cluster 110 (step 201). Next, the individual waveform data (elements) in this initial cluster 110 are subjected to feature analysis (step 202). In this feature analysis, the order of feature parameters such as LPC (linear predictive coding) cepstrum is n, and the frame period of the analysis window function is variable, and the analysis is performed so that the number of frames is m frames. To n for each element
Obtain a × m-dimensional feature parameter matrix.

Next, using the result of this feature analysis, the cluster distortion of the initial cluster 110 is obtained (step 20).
3). This is done as follows.

First, the characteristic parameter of each element is converted from the n × m matrix format to the (n × m) × 1 dimensional vector format. To show this in a simplified example,

[Equation 1] As described above, for example, when the original format is a 4 × 3 dimensional matrix, the 12 components are arranged in one dimension and converted into a 12 dimensional vector format.

Next, in the vector space of this feature parameter, the sum of squares of the distances between all the elements of the initial cluster 110 and the previously obtained centroid is calculated,
This is defined as the cluster distortion of the initial cluster 110.

When the cluster distortion of the initial cluster 110 is calculated in this way, it is calculated as the context cluster table 20.
Register at 8. This context cluster table 20
In FIG. 8, as shown, for each cluster, the context belonging to it, its centroid, its cluster distortion, and the set of element waveforms included in it are registered. At the stage of obtaining the cluster distortion of the initial cluster 110, only the initial cluster 110 is registered in the context cluster table 208.

Next, the context cluster table 20
A cluster having the largest cluster distortion is obtained from the eight (step 204), the obtained cluster is taken out from the context cluster table 208, and further divided into two clusters according to the context (step 20).
5). At the initial stage, since only the initial cluster 110 is registered in the context cluster table 208, the initial cluster 110 is divided into clusters. The method of this cluster division is as follows.

For example, if the cluster to be divided is | A, B,
If five contexts C, D, E | are included, possible division methods are: | A, B, C | and | D, E |, | A, C, D, E | There are many methods such as | B | division, | A, B, E | and | C, D | division, and so on. Among them, the sum of the cluster distortions of the two clusters caused by the division is the smallest. Becomes 1
Choose one split method. That is, for all possible division methods, the gain Q of cluster division is calculated by the following equation (1).
Is calculated.

Q = VAR- (Va + Vb) (1) where VAR is the cluster distortion of the cluster to be divided, and V is
a and Vb are cluster distortions of the two clusters after the division. The calculation of the cluster distortion of the cluster after the division is the same as the method described above. The gain Q means a reduction amount of cluster distortion obtained by cluster division.

In this way, when the gain Q is obtained for all the division methods, the division method that maximizes the gain Q, in other words, the division method that minimizes the sum (Va + Vb) of the cluster distortion after division is selected, and this division is selected. The cluster to be divided is divided into two clusters by the method.

After the initial cluster 110 is divided in this way, the initial cluster 110 is deleted and divided into two in the context cluster table 208.
Two clusters are newly registered (step 206).

Next, returning again to step 203, the cluster having the largest cluster distortion is selected from the context cluster table 208, and the selected cluster is divided into two by the same method as described above to obtain the context cluster. The table 208 is rewritten (step 2)
04-206).

By repeating the above processing (steps 203 to 206), the initial cluster 110 is gradually subdivided into smaller clusters. Then, the termination determination of the context clustering is performed for each of the repeated loops (step 207). In this termination determination, the termination is determined when the number of generated clusters reaches a predetermined upper limit or when the condition that the gain Q falls below a predetermined threshold is satisfied.

Here, supplementary description will be given of the termination condition regarding the gain Q with reference to FIG. 4. Although the total sum of cluster distortion gradually decreases as the clustering progresses, the decrease amount of the total sum gradually increases. Because it becomes smaller
The gain Q of cluster division also gradually decreases. Therefore, when this gain Q falls below a threshold value that is empirically determined in advance, it is determined that sufficient cluster division has been performed to such an extent that the total sum of cluster distortion cannot be further reduced, and clustering is terminated. be able to.

By using the termination condition based on the gain Q, the clustering can be terminated at an appropriate stage without depending on the initial cluster distortion value.

When it is determined in step 207 that the process is finished, the process of context clustering is finished, and the process proceeds to the process of prosody clustering shown in FIG.

The context clustering described above is, in principle, executed for all phoneme labels in the speech database 100.

[4] Prosody Clustering FIG. 5 is a flowchart of prosody clustering.

This prosodic clustering is sequentially performed on each cluster in the context cluster table 208 obtained by the above context clustering.

In FIG. 5, first, a cluster targeted for prosodic clustering is selected by referring to the context cluster table 208, and prosodic parameters (pitch, pitch slope, time length,
Power, etc.) is read from the voice database 100 (step 301).

Next, the prosody parameters of all the elements read in step 301 are scanned to find the maximum value and the maximum value in the cluster for each individual parameter (that is, pitch, pitch slope, time length, power, etc.). The minimum value is obtained (step 302). When the process of step 302 ends, clustering on the prosody parameter space by the LBG algorithm is performed in the procedure described below.

Here, the LBG algorithm will be described with reference to FIGS. The black dots in the figure indicate individual elements.

First, as shown in FIG. 6, two centroids 401 and 402 are randomly defined in a predetermined parameter space (illustrated in a two-dimensional space for convenience), and the two centroids are temporarily divided into two. Let it be the centroid of clusters A and B. Then, the individual elements 403 of the cluster C
, The distance to each of the two centroids 401, 402 is measured based on a predetermined distance measure, and the element 403 belongs to the cluster A or B of the centroid having the shorter distance. By performing this for all the elements, the original cluster C is divided into the cluster A and the cluster B by the boundary line 404 as shown in FIG. 7 (first division).

Next, in each of the clusters A and B generated by the division, as shown in FIG. 8, new centroids 405 and 406 are obtained (centroid update).
Then, based on the new centroids 405, 406,
It is re-determined whether each element 403 belongs to the cluster A or B, and the boundary line of the clusters A and B is changed from the boundary line 404 of the first division to the boundary line 407 shown in FIG. 9 (second time). Split).

Such cluster division and centroid update are repeated until the centroid converges. This is the LBG algorithm.

It is the prosodic clustering of this embodiment that clustering by the LBG algorithm is performed on a prosodic parameter space consisting of pitch, pitch gradient, power, time length and the like.

In this prosodic clustering, the distance D between the two elements W1 and W2 is obtained by the following equation (2).

[0053]

[Equation 2] Where F1, V1, A1 and T1 are the pitch slope, fundamental frequency, power and time length of one element W1,
2, V2, A2, and T2 are pitch tilts of the other element W2,
Basic frequency, power, and time length. Also, the symbol ‖
... ‖ means normalization by the difference between the maximum value and the minimum value obtained for each parameter in step 302.

This equation (2) is, in short, after taking the difference between the same kind of parameters in the two elements W1 and W2, normalizing the difference by the difference between the maximum value and the minimum value of each parameter. After that, it means to take the sum of squares of them. The distance D (strictly speaking, the squared distance) thus obtained indicates the degree of similarity in the prosody of the two elements W1 and W2.

Alternatively, the distance D is expressed by the following equation (3):
The calculation may be performed by adding weighting according to the importance of each parameter.

[0056]

(Equation 3) Where ωf, ωv, ωa, and ωt are pitch tilts,
The weighting factors for the fundamental frequency, power, and time length are shown.

Referring again to FIG. 5, in prosodic clustering, first, two elements having different coordinates are set as initial centroids in the cluster to be clustered on the prosodic parameter space (step 30).
3). As a method of setting the initial centroid, for example, there are a method of randomly selecting two elements, a method of previously obtaining the centroid of the target cluster, and selecting two elements in the vicinity thereof.

The initial centroid thus selected is provisionally registered in the prosody cluster table 309. In the prosody cluster table 309, as shown in the figure, the context included in each target cluster and the cluster distortion and centroid of the subdivided cluster obtained by dividing the context are registered. Where the cluster distortion is
It is defined as the sum of the distances (distance D obtained by the above-described method) between all the elements of the target cluster on the prosody parameter space and the centroid. For example, as shown in FIG. 10, it is the sum of squares of the distance 501 between all the elements in each cluster and the centroid.

The initial centroid is provisionally registered in the centroid column of the corresponding target cluster in the prosody cluster table 309. At the stage when this initial centroid is provisionally registered, the target cluster is still a subdivided cluster. It is not divided.

After temporarily registering the initial centroid in this way, the cluster having the largest cluster distortion is extracted from the cluster distortion column of the target cluster in the prosody cluster table 309 (step 304). However, when the initial centroid is provisionally registered, the target cluster is not yet divided, and the cluster distortion of the target cluster itself is registered in the prosody cluster table 309. Therefore, this target cluster has the maximum cluster distortion. It will be extracted as a thing.

Next, the target cluster extracted in step 304 is subjected to clustering by the LBG algorithm already described (step 305), and the target cluster is divided into two. When the two divisions are completed, the cluster distortions and centroids of the two newly created subdivision clusters are registered in the prosody cluster table 309 in the column corresponding to the original target cluster (step 306).

Thereafter, the process returns to step 304 again, the subdivided cluster having the largest cluster distortion is extracted, and the subdivided cluster is subjected to clustering by the LBG algorithm (step 305) and the prosody cluster table 309 is rewritten. (Step 306).

By repeating the above steps 304 to 306, the original target cluster is gradually subdivided. 11 to 13 show this process. The original target cluster is first divided into two subdivision clusters 500 and 502 as shown in FIG. 11, and then the subdivision cluster with the larger cluster distortion. 502 is divided into two subdivision clusters 503, 504, as shown in FIG. Further, among the three subdivided clusters in FIG. 12, the cluster 503 having the largest cluster distortion is divided into two subdivided clusters 505 and 506 as shown in FIG.

In this way, clustering is repeated for the cluster with the largest cluster distortion.

In each iterative loop of this clustering, the end judgment of the prosody clustering is performed (step 307). In this termination determination, the termination is determined when the condition that the number of generated subdivision clusters reaches a predetermined upper limit or the following expression (4) is satisfied is satisfied.

(SD−SD ′) / SD <ε (4) Here, SD represents the total sum of clustering distortion before clustering, and SD ′ represents the total sum of cluster distortion after clustering. ε indicates a threshold value for end determination.

Satisfying the above equation (4) means that the difference between the sum of cluster distortion before clustering and the sum of cluster distortion after clustering is
That is, the value obtained by dividing by the total sum of the cluster distortion before clustering (that is, the reduction rate of the cluster distortion, that is, a kind of normalized gain) is below the termination determination threshold value ε. This reduction rate of cluster distortion has the property of gradually decreasing as clustering progresses and the number of clusters increases. If this is below a predetermined threshold ε, cluster distortion is further reduced. It can be judged that clustering has been performed to a sufficient degree that cannot be done.

Therefore, by using the termination condition of equation (4), clustering can be terminated at an appropriate stage without depending on the initial cluster distortion.

When it is determined that the processing is completed in step 307, next, as shown in FIG. 14, for each of the subdivision clusters 500 to 506 obtained by the prosodic clustering,
The element (waveform data) closest to each centroid registered in the prosody cluster table 309 is extracted from the speech database 100, and this is the synthesis speech file 16
0 is registered (step 308).

The above prosodic clustering processing is executed for all clusters obtained by context clustering. As a result, the audio file for synthesis 16
0 is completed. As can be understood from the above description, the completed synthesis voice file 160 has voice waveform data that is highly exhaustive in terms of both context and prosody.

It should be noted that the above-mentioned contents are only related to one embodiment of the present invention, and it is needless to say that the present invention is not limited to the above contents. For example, although the phoneme is used as a unit of phoneme in the above embodiment, this is not necessarily the case, and the present invention can be applied to the case of handling various types of phoneme units such as CV syllables.

[0072]

As described above, according to the present invention,
It is possible to construct an audio file that is highly comprehensive in terms of both context and prosody.

[Brief description of drawings]

FIG. 1 is a diagram showing a voice database used in an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an overall synthesis voice file configuration process according to the embodiment.

FIG. 3 is a flowchart showing a process of context clustering in the embodiment.

FIG. 4 is a diagram showing a relationship between progress of context clustering and a change in the total sum of cluster distortions.

FIG. 5 is a flowchart showing processing of prosody clustering in the embodiment.

FIG. 6 is an explanatory diagram of prosody clustering by the LBG algorithm.

FIG. 7 is an explanatory diagram of prosody clustering by the LBG algorithm.

FIG. 8 is an explanatory diagram of prosody clustering by the LBG algorithm.

FIG. 9 is an explanatory diagram of prosody clustering by the LBG algorithm.

FIG. 10 is an explanatory diagram of cluster distortion used in prosody clustering.

FIG. 11 is an explanatory diagram of prosody clustering.

FIG. 12 is an explanatory diagram of prosody clustering.

FIG. 13 is an explanatory diagram of prosody clustering.

FIG. 14 is an explanatory diagram of prosody clustering.

[Explanation of symbols]

 100 voice database 110 initial cluster 160 voice file for synthesis 208 context cluster table 309 prosodic cluster table

Claims (3)

[Claims] 1. A method of constructing a voice file used for waveform synthesis for obtaining a synthesized sound by combining speech waveform segments cut out from natural speech, wherein a large number of speech waveform segments cut out from said spontaneous speech are accumulated. A speech database, and context clustering means for performing context-based context clustering in a predetermined feature parameter space for a set of speech waveform segments having the same phonemes in the speech database, and obtained by the context clustering. Prosodic clustering means for performing prosodic clustering in a predetermined prosodic parameter space for each cluster, and centroids of the individual clusters obtained by the prosodic clustering are obtained, and speech waveform segments in the vicinity of this centroid are extracted. And before Audio file configuration system, characterized in that it comprises a speech element registering means for registering the audio file, a. 2. The method according to claim 1, wherein each of the context clustering means and the prosody clustering means has an end determination means for determining the end of clustering based on the gain of cluster division in clustering. Audio file configuration method. 3. A method of constructing a voice file used for waveform synthesis for obtaining a synthetic sound by combining speech waveform segments cut out from natural speech, wherein a large number of speech waveform segments cut out from said spontaneous speech are accumulated. A step of preparing the speech database, a step of performing context clustering in a predetermined feature parameter space on the basis of context for a set of speech waveform segments having the same phoneme in the speech database, and obtained by the context clustering. A process of performing prosodic clustering in a predetermined prosodic parameter space on each of the obtained clusters, and obtaining the centroid of each cluster obtained by the prosodic clustering, and extracting a speech waveform segment near the centroid. And registering in the audio file Audio files configuring, characterized in that it comprises a.