JP5498439B2 - Phoneme Labeling Data Phoneme Duration Length Conversion Method, Apparatus and Program - Google Patents

Description

  The present invention relates to a phoneme labeling data phoneme duration conversion method for converting phoneme labeling data of a phoneme labeling database into, for example, phoneme labeling data for different speech synthesis systems, and an apparatus and program thereof.

  Phoneme labeling refers to assigning a label representing the type of phoneme uttered in the speech data and the boundary between phonemes. In order to use a speech database that has been phoneme-labeled with an existing phoneme system, for speech synthesis or speech recognition based on a different phoneme system, basically, New labeling with a new phoneme system is required.

  New labeling methods using a new phoneme system include manual labeling and automatic labeling. The automatic labeling method is disclosed in Non-Patent Document 1, for example.

Takashi Nakamura, Noboru Miyazaki, Hideyuki Mizuno, “Examination of multi-step automatic labeling method corresponding to pronunciation variation”, Proc. Of the Acoustical Society of Japan, p265-268, September 2009

  Although manual labeling of the conventional method can be performed with high accuracy, it requires a lot of time and expense due to manual labor, and is costly. On the other hand, the automatic labeling has a problem that the cost is kept low but the labeling accuracy is low and cannot be used as it is depending on the application. Therefore, a method of converting an existing phoneme database into a phoneme database corresponding to the target system seems to be the most realistic. However, since it is difficult to convert phoneme boundaries mechanically, there has been no apparatus or method for converting such a phoneme database.

  In other words, a new phoneme system may correspond to a single phoneme and multiple phonemes in the existing phoneme system, or vice versa. Yes, it was difficult to realize.

  The present invention has been made in view of the above problems, and is a phoneme labeling data phoneme duration conversion method for accurately converting an existing phoneme database to a different phoneme system without cost. The purpose is to provide.

  The phoneme labeling data phoneme duration conversion method of the present invention comprises a phoneme duration distribution estimation process, a phoneme conversion process, and a phoneme duration split process. The phoneme duration distribution estimation process consists of the reference labeling data, which is the phoneme labeling data in the new phoneme system of the conversion target speaker, and the average and variance values of phoneme durations of phoneme types in the phoneme system with multiple speakers. The multi-speaker labeling data is input as multi-speaker labeling data including a number of phoneme types that can be obtained as statistically reliable values, and the reference labeling data is linearly regressed with the multi-speaker labeling data. The average phoneme duration of the conversion target speaker corresponding to the phoneme type and the phoneme duration distribution that is the variance are obtained. The phoneme conversion process converts existing phoneme labeling data, which is phoneme labeling data before conversion of a conversion target speaker, into phoneme labeling data of a new phoneme system. In the phoneme duration division process, the phoneme duration distribution and the phoneme labeling data of the new phoneme system are input, and the phoneme labeling data having one phoneme duration in one phoneme information is passed as new phoneme labeling data. The phoneme labeling data having a plurality of phoneme information for one phoneme duration is divided into phoneme durations for each phoneme information and output as new phoneme labeling data.

  The phoneme labeling data phoneme duration conversion method of the present invention linearly regresses a small amount of phoneme labeling data of a new phoneme system with a large amount of multi-speaker labeling data, and supports all phoneme types of multi-speaker labeling data. The phoneme duration distribution of the conversion target speaker is obtained. A phoneme of a new phoneme system in which phoneme labeling data including a plurality of phonemes in one phoneme label is divided into durations for each phoneme based on an average value and a variance value obtained from a large amount of multi-speaker labeling data. Output as labeling data. Therefore, a large amount of phoneme labeling data of the new phoneme system can be accurately obtained from a small amount of phoneme labeling data.

The figure which shows the function structural example of the phoneme labeling data phoneme duration length conversion apparatus 100 of this invention. The figure which shows the operation | movement flow of the phoneme labeling data phoneme duration length conversion apparatus. The figure which shows the function structural example of the phoneme duration duration distribution estimation part. The figure which shows the operation | movement flow of the phoneme duration distribution distribution estimation part. The figure which shows the operation | movement flow of the phoneme conversion part. The figure which shows the operation | movement flow of the phoneme duration length division | segmentation part 60. FIG. Diagram illustrating an average value of the phoneme duration of the conversion target speaker and mu _x ', the correlation between the average value mu _x phoneme duration of a plurality of speakers. Diagram illustrating a phoneme duration of variance sigma _x ^'2 conversion target speaker, the correlation between variance sigma _x ² phoneme duration of a plurality of speakers.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 1 shows an example of the functional configuration of a phoneme labeling data phoneme duration conversion device 100 of the present invention. The operation flow is shown in FIG. The phoneme labeling data phoneme duration length converter 100 includes a phoneme duration distribution estimation unit 40, a phoneme converter 50, and a phoneme duration division unit 60. The function of each part of the phoneme labeling data phoneme duration conversion device 100 is realized by reading a predetermined program into a computer composed of, for example, ROM, RAM, CPU, etc., and executing the program by the CPU. It is.

  The phoneme duration distribution estimation unit 40 receives the reference labeling data 10 and the multi-speaker labeling data 20 as input, performs linear regression on the reference labeling data 10 with the multi-speaker labeling data 20, and performs all of the multi-speaker labeling data 20. The average phoneme duration length of the conversion target speaker corresponding to the phoneme type and the phoneme duration distribution that is the variance value are obtained (step S40). The phoneme duration distribution may be stored in a ROM, RAM, or hard disk as indicated by a broken line in FIG.

  The reference labeling data 10 is a small number of phoneme labeling data in the new phoneme system of the conversion target speaker. This is a number that can be obtained as a statistically reliable value of the average and variance of phoneme durations of less than half of the phonemes. When the multi-speaker labeling data 20 is, for example, 6022 sentences and 61 types of phonemes The reference labeling data 10 is a number of about 14 types of 5 sentences, for example. However, one sentence includes at least a subject and a predicate. Experiments confirmed that statistically reliable average and variance values were obtained with 5 sentences or more.

  The multi-speaker labeling data 20 includes all the phoneme types that can obtain the average value / variance value of the phoneme durations of the phoneme types in the new phoneme system of the multi-speakers as a statistically reliable value. Phoneme labeling data including.

  The phoneme conversion unit 50 converts the existing phoneme labeling data 30 into pre-division phoneme labeling data that is phoneme labeling data of a new phoneme system (step S50). Table 1 illustrates a part of the existing phoneme labeling data 30. The pre-division phoneme labeling data may be stored in a ROM, RAM, or hard disk in the same manner as the phoneme duration distribution.

Conversion of existing phoneme labeling data to a new phoneme system is performed by the following two-stage processing.
The existing phoneme labeling data 30 is data in which the phoneme label and the phoneme duration are in one-to-one correspondence. The existing phoneme labeling data 30 is existing data of a conversion target speaker of another phoneme system different from the new phoneme system to be converted, and is converted into pre-division phoneme labeling data of the new phoneme system by the phoneme conversion unit 50.
Table 2 illustrates a part of the pre-division phoneme labeling data. Table 2 is an example of the first stage processing in which the phonemes in Table 1 are converted based on the new phoneme system.

  “T” in the existing phoneme labeling data 30 corresponds to “cT, T” in the pre-division phoneme labeling data. In this way, “t” corresponds to two phonemes in the new phoneme system, but here, the duration length of each phoneme is not calculated, and the sum of the phoneme information of a plurality of phonemes and the phoneme duration length is 2 It corresponds to two phonemes.

  The phoneme duration division unit 60 receives the phoneme duration distribution output from the phoneme duration distribution estimation unit 40 and the pre-division phoneme labeling data output from the phoneme conversion unit 50 as one input per phoneme information. Phoneme labeling data having a phoneme duration of 1 is passed as new phoneme labeling data, and phoneme labeling data having multiple phoneme information for one phoneme duration is phoneme duration length for each phoneme information. Are output as new phoneme labeling data (step S60).

  Table 3 illustrates a part of the new phoneme labeling data. Table 3 is an example in which the phonemes in Table 2 are converted into new phoneme labeling data.

  The pre-division phoneme labeling data “cT, T” is divided into two phoneme information of “cT: 70 ms” and “T: 20 ms”. The pre-division phoneme labeling data “A” in which one phoneme duration corresponds to one phoneme information is output as new phoneme labeling data as it is.

  The division of the phoneme duration is performed based on the phoneme duration distribution obtained by linear regression of a small amount of reference labeling data 10 and a large amount of multi-speaker labeling data 20. Therefore, a large amount of phoneme labeling data of a new phoneme system can be generated with high accuracy from a small amount of phoneme labeling data of a part of new phoneme system.

Hereinafter, the operation of each unit of the phoneme labeling data phoneme duration conversion device 100 will be described in more detail.
(Phoneme duration distribution estimation unit)
FIG. 3 shows a functional configuration example of the phoneme duration distribution estimation unit 40. The operation flow is shown in FIG.
The phoneme duration distribution estimation unit 40 includes an average value / dispersion value calculation unit 41, a linear regression equation estimation unit 42, and a phoneme duration distribution distribution estimation unit 43. The average value / dispersion value calculation means 41 uses all the phoneme sets in the multi-speaker labeling data 20 as X, and the average value μ _x of the phoneme duration lengths of all phonemes x with x∈X and the variance value σ _x. ² is obtained (steps S41a to S41c).

Then, for the reference labeling data 10 also, all of the phoneme set X '(where, X'⊆X) as, X∈X' and number of occurrences n _x 'of all phonemes x is, the phoneme duration An average value μ _x ′ and a variance σ _x ′ ² are obtained, and a set M _{1 of} average values and a set Σ ₁ of variances are obtained (step S41e).

Linear regression estimator 42, the minimum and the average value mu _x 'and variance sigma _x' of the reference labeling data ^2, a linear regression expression between the average value mu _x multiple speakers labeling data 20 and variance sigma _x ² Obtained by the square method (step S42a).

A set Y of all phonemes for which n _x ′ ≧ n, where n is a specified value of the number of occurrences of a phoneme from which a statistically sufficiently reliable average value and variance value can be obtained (n is preferably at least 5 or more) For = {x | x∈X ′, n _x ′ ≧ n}, parameters a ₁ , a ₂ , b ₁ , and b ₂ are obtained by the least square method as shown in the following equation.

Here, N (Y) is the number of elements of the phoneme set Y.
A set of phonemes that have appeared in the multi-speaker labeling data 20 but did not appear in the reference labeling data 10 and phonemes that have appeared statistically unreliable times (phonemes x with nx ′ <n) Z = Let XY = {x | x∈X, n _x ′ ≧ n}.
The phoneme duration distribution estimation means 43 obtains the average value μ _x ′ and the variance value σ _x ′ ² using the formula (3) for all phonemes x (x∈Z) of Z, and sets the average value set M ₂ and a set of variance values Σ ₂ are obtained (step S43a).

  Then, the following average value set M and variance value set Σ are obtained as the phoneme duration distribution of all phonemes in the phoneme set X (step S43b).

(Phoneme conversion part)
The operation will be described with reference to the operation flow of the phoneme conversion unit 50 shown in FIG. The phoneme conversion unit 50 converts the existing phoneme labeling data 30 into new phoneme labeling data in a new phoneme system.

First, the phoneme sequence x = (x ₁ , x ₂ , ..., x _n ) in the existing phoneme system corresponds to the phoneme sequence y = (y ₁ , y ₂ , ..., y _m ) in the new phoneme system. In this case, F is the total conversion rule from the existing phoneme system to the new phoneme system, and F (x) = y. Also, let X be the set of all phoneme sequences x in which F (x) is defined (the domain of F) (step 500). Table 4 shows examples of conversion rules.

  Next, the phoneme number L of the phoneme sequence having the maximum number of phonemes in the set X of phoneme sequences is obtained (step S501). And the following operation is performed with respect to all the existing phoneme labeling data 30, and the phoneme labeling data before a division | segmentation by a new phoneme system is output.

  The number representing the label data number of the existing phoneme labeling data 30 is i, and i ← 1 is set (step S502). For example, in Table 1 above, the i-th label data is “t, 70” when i = 1, and “a, 60” when i = 2. The number k of phonemes constituting the phoneme string is set to k ← L. The process of step S502 is repeated when the number of phoneme labels of the existing phoneme labeling data is N, and N <i, that is, until all the label data are converted (step S509).

The i + k−1th phoneme information x _{i + k−1} is acquired from the i th phoneme information x _i of the existing phoneme labeling data 30, and the phoneme string x = (x _i , x _{i + 1} ,..., X _{i + k-1} ) is created (step S504).
In the case of X∈X (when defined in the pre-conversion phoneme string in the conversion rule), the following operation is performed (Yes in step S505).
Using the i-th phoneme duration information of the existing phoneme labeling data 30 as t _i , the phoneme information y = F (x) of the new phoneme system and the total value t ′ of the phoneme duration are obtained (step S507).

  Phoneme information y and phoneme duration information t ′ are output as pre-division phoneme labeling data (step S508). In order to advance the label data number by the number of converted phonemes, i ← i + k. The process returns to the process of setting the number k of phonemes constituting the phoneme string to k ← L (step S503).

  When x∈X is not satisfied (when not defined in the pre-conversion phoneme string in the conversion rule), k ← k−1 is set to reduce the number of phonemes constituting the phoneme string (step S506). The process returns to the next phoneme label process (step S504).

[Phoneme duration length division]
The operation will be described with reference to the operation flow of the phoneme duration division unit 60 shown in FIG. The phoneme duration length dividing unit 60 divides the phoneme duration length for the phoneme information converted from the pre-division labeling data obtained by the phoneme conversion unit 50 into a plurality of phonemes.

The following operation is performed on all pre-division labeling data.
The number representing the label data number of the pre-division phoneme labeling data is i, and i ← 1 (step S600). For example, in Table 2 above, “cT, T, 70” when i = 1, and “A, 60” when i = 2. If the number of phoneme labels in the pre-division phoneme labeling data is N, and N <i (that is, when all the label data have been divided), the process ends (step S609).

Phoneme information x _i = (x _{i, 1} , x _{i, 2} ,..., x _{i, n} ) and phoneme duration information t _i of i-th pre-division phoneme labeling data are acquired (step S601). In Table 2 above, when i = 1, x _i = (cT, T), t _i = 70 and x _{i, 1} = cT, x _{i, 2} = T.

The number of phonemes n constituting the phoneme sequence x _i is obtained (step S602). For example, n = 2 when x _i = (cT, T).
For all phonemes x _{i, k} (k = 1,2, ..., n) that make up phoneme sequence x _i , average value of phoneme duration μ _xik and variance σ _xik ² are obtained from phoneme duration distribution Then, the total values M and S of the respective values are obtained (step S603).

Next, the following operations are performed to output each phoneme information and phoneme duration.
The phoneme number in the phonemes constituting the phoneme sequence x _i is k, and k ← 1 (step S604). When n <k (that is, when all the phonemes in the phoneme string xi have been output), i ← i + 1 is returned to step S601, and the next label data is processed.

The output phoneme information is x and x ← x _{i, k} (eg, when x _i = (cT, T), when k = 1, when x = x _{i, k} = cT, k = 2 x = x _{i, k} = T) (step S605).
The average value μ _x and the variance value σ _x ² of the phoneme duration of the phoneme information x are acquired from the phoneme duration distribution, and the phoneme duration information t ′ divided by the equation (13) is obtained (step S606). .

The obtained phoneme information x and phoneme duration information t ′ are output as new phoneme labeling data (step S607). Then, the process returns to the next phoneme number processing (step S605) as k ← k + 1.
[About linear regression]
The effectiveness of the present invention will be described by showing the correlation coefficient of the linear regression equation obtained by the linear regression equation estimation means 42 of the phoneme duration distribution estimation unit 40. The correlation obtained based on the specific data is shown in FIGS. FIG. 7 shows the correlation of the average value of phoneme durations. FIG. 8 shows the correlation of the variance values.

  The correlation coefficient was obtained from the following data. The number of label data of multi-speaker labeling data is 33,763, 117 phonemes, the number of label data of speaker 1 to be converted is 6022, the number of phonemes is 89, and the number of label data of speaker 2 to be converted is 94 Data with 56 phonemes were used.

The correlation coefficient between the average value μ _x ′ and the average value μ _x is as follows: the correlation coefficient γ = 0.99 for speaker 1 (■ in FIG. 7), and the correlation coefficient γ = 0.96 for speaker 2 (♦ in FIG. 7). A very high correlation was found. In addition, the correlation between the variance value σ _x ′ ² and the variance value σ _x ² is γ = 0.81 for speaker 1 (■ in FIG. 8) and γ = 0.73 for speaker 2 (♦ in FIG. 8). showed that.

  From this correlation coefficient, by using the phoneme labeling data phoneme duration length conversion device 100 of the present invention, it is possible to accurately obtain a large amount of phoneme labeling data of a new phoneme system from a small amount of phoneme labeling data. I understand.

  When the processing means in the above apparatus is realized by a computer, the processing contents of the functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (5)

To obtain statistically reliable values of the reference labeling data, which is phoneme labeling data in the new phoneme system of the speaker to be converted, and the phoneme type average / dispersion value of the phoneme type in a phoneme system with multiple speakers The multi-speaker labeling data including the number of phonemes that can be input is input, the reference labeling data is linearly regressed with the multi-speaker labeling data, and the conversion corresponding to all the phoneme types of the multi-speaker labeling data is performed. The phoneme duration distribution estimation process for obtaining the average phoneme duration length of the target speaker and the phoneme duration length distribution which is a variance value;
A phoneme conversion process of converting existing phoneme labeling data, which is phoneme labeling data before conversion of a conversion target speaker, into phoneme labeling data of a new phoneme system;
Using the phoneme duration distribution and the phoneme labeling data of the new phoneme system as input, phoneme labeling data having one phoneme duration in one phoneme information is passed as new phoneme labeling data as it is. The phoneme labeling data having a plurality of phoneme information with respect to the duration length is divided into phoneme duration lengths for each phoneme information and output as new phoneme labeling data,
Phoneme labeling data phoneme duration conversion method. In the phoneme labeling data phoneme duration conversion method according to claim 1,
The above phoneme duration distribution estimation process is
An average value / variance value calculating step for calculating an average value / variance value of each of the reference labeling data and the multi-speaker labeling data;
A linear regression equation estimation step for estimating a linear relational equation for linearly regressing the average value / variance value of the reference labeling data with the multi-speaker labeling data;
A phoneme duration distribution estimation step for estimating an average value and a variance of phoneme durations of all phoneme types of the conversion target speaker corresponding to the multi-speaker labeling data by the linear relational expression;
A phoneme labeling data phoneme duration conversion method. In the phoneme labeling data phoneme duration conversion method according to claim 1 or 2,
The phoneme duration division process is
The phoneme number obtaining step of obtaining a phoneme number n composing the phoneme sequence x _i,
For all phonemes x _{i, k} (k = 1, 2,..., N) constituting the phoneme sequence x _i , the average phoneme duration length μ _{x, I, k} and variance σ _{x, I, k} ² are Obtaining from the above phoneme duration distribution, obtaining a total value M of average values and a total value S of variances, a total value obtaining step for obtaining respective total values;
The phoneme duration information t ′ of the above new phoneme labeling data,

Where t _i is the phoneme duration before division,
Obtaining phoneme duration information obtained in step 1,
A phoneme labeling data phoneme duration conversion method. Reference labeling data, which is phoneme labeling data in the new phoneme system of the conversion target speaker,
Multiple episodes that are phoneme labeling data including all phoneme types that can be obtained as statistically reliable values of the average and variance of phoneme durations of phoneme types in a phoneme system with multiple speakers Labeling data,
Existing phoneme labeling data, which is phoneme labeling data before conversion of the conversion target speaker,
The conversion target speaker corresponding to all phoneme types of the multi-speaker labeling data by linearly regressing the reference labeling data with the multi-speaker labeling data by using the reference labeling data and the multi-speaker labeling data as inputs. A phoneme duration distribution estimation unit for obtaining a phoneme duration distribution which is a mean value and a variance of phoneme durations of
A phoneme conversion unit that converts the existing phoneme labeling data into pre-division phoneme labeling data that is phoneme labeling data of a new phoneme system;
Using the above phoneme duration distribution and the phoneme labeling data of the new phoneme system as input, phoneme labeling data having one phoneme duration in one phoneme information is passed as it is as new phoneme labeling data. The phoneme labeling data having a plurality of phoneme information for the duration length is divided into phoneme duration lengths for each phoneme information and output as new phoneme labeling data; and
Phoneme labeling data phoneme duration conversion device.   A program for causing a computer to function as the phoneme labeling data phoneme duration conversion device according to claim 4.

Images