JP4809913B2 - Phoneme division apparatus, method, and program - Google Patents

Description

  The present invention relates to a technique for automatically determining a phoneme boundary time from speech.

  In conventional automatic phoneme segmentation technology, a statistical average spectral pattern of phonemes is prepared as a distribution, similarity (likelihood) with a given speech spectral pattern is calculated, and a likely phoneme is assigned to each frame. Thus, the phoneme boundary time is obtained. That is, a frame in which corresponding phonemes change before and after is set as a phoneme boundary time (see, for example, Non-Patent Document 1).

Tsuyoshi Kawai and Satoshi Toda, "Evaluation of automatic phoneme segmentation for waveform-connected speech synthesis", IEICE Technical Report, SP2002-170, pp.5-10, January 2003

  However, the statistical average spectral pattern of phonemes is smoothed during the averaging process, and detailed features of the spectral pattern are often lost. Therefore, when the spectrum pattern changes continuously and smoothly, there is no large difference in the spectrum pattern before and after the phoneme boundary time, so there is no likelihood difference in the statistical average spectrum pattern, and the estimated phoneme boundary There was a problem that the time was significantly different from the correct answer.

  In order to solve the above-described problem, the speech feature amount of each frame of the input speech is extracted. When the most probable phoneme is assigned to each frame using a statistic about the speech feature quantity of a plurality of phonemes, and the phonemes assigned in two consecutive frames are different, the time range over those two frames The phoneme boundary time is estimated by using any of the times included in the phoneme as the phoneme boundary time. Determine whether phoneme boundary time is reliable. By assigning to each phoneme that constitutes a phoneme boundary that the phoneme boundary time is determined to be unreliable, the duration of each phoneme is longer as the average value is larger, and as the variance is larger, the phoneme boundary is expanded. A phoneme boundary time of a phoneme boundary that is determined to be unreliable is estimated.

  By re-estimating the phoneme boundary time that cannot be relied on using the average value and variance of the phoneme duration, the phoneme boundary time can be estimated more accurately than in the past.

The functional block diagram of the example of a phoneme division | segmentation apparatus. The functional block diagram of Example 1 of a phoneme boundary time estimation result reliability determination part. The functional block diagram of Example 2 of a phoneme boundary time estimation result reliability determination part. The functional block diagram of the example of a 2nd phoneme boundary time estimation part. The flowchart of the example of the phoneme division | segmentation method. The flowchart of the example 2 of a process of the phoneme boundary time estimation result reliability determination part. The flowchart of the example of a process of a 2nd phoneme boundary time estimation part. The figure for demonstrating phoneme boundary time. The figure which showed the conditions of the propriety of reliability. The figure which shows the specific example of estimation of phoneme boundary time.

  Hereinafter, embodiments of the present invention will be described in detail.

  FIG. 1 is a functional block diagram of an example of a phoneme dividing device according to the present invention. FIG. 5 is a flowchart of an example of a phoneme division method according to the present invention.

  The phoneme dividing device includes a speech feature amount extraction unit 1, a first phoneme boundary time estimation unit 2, a phoneme boundary time estimation result reliability determination unit 3, a second phoneme boundary time estimation unit 4, a speech feature amount storage unit 6, and a continuation length. For example, the distribution storage unit 7 is included.

<Step S1>
The input voice is input to the voice feature amount extraction unit 1. The voice feature amount extraction unit 1 divides the input voice into frames having a predetermined time length, and calculates a voice feature amount for each frame (step S1). The voice feature amount of each frame is sent to the first phoneme boundary time estimation unit 2.

  As the speech feature amount, any speech feature amount may be used as long as it can assign a phoneme to a frame using the speech feature amount. For example, MFCC, cepstrum, mel cepstrum, filter bank, mel filter bank, etc. that are often used in voice recognition or the like can be used as the voice feature amount.

<Step S2>
Assume that the speech feature quantity storage unit 6 stores statistics about speech feature quantities of a plurality of phonemes. For example, it is assumed that a statistical spectrum pattern of phonemes is stored in the form of HMM (Hidden Markov Model), vector quantization, neural network, etc. that are often used in speech recognition and the like.

  For example, a statistical average spectrum is stored for each unit of monophone and triphone. A monophone is a phoneme model that does not consider what phonemes are present before and after the phoneme. A triphone is a phoneme model that takes into account the phonemes before and after the phoneme. / O /, / e / behind / i /) are considered different.

  The first phoneme boundary time estimation unit 2 assigns the most likely phoneme to each frame using the speech feature value and the statistic read from the speech feature value storage unit 6, and the phoneme assigned in two consecutive frames Are different from each other, the phoneme boundary time is estimated by setting one of the times included in the time range over the two frames as the phoneme boundary time (step S2). The phoneme boundary time estimation result including the phoneme boundary time is sent to the phoneme boundary time estimation result reliability determination unit 3.

  The first phoneme boundary time estimation unit 2 uses the speech feature value and the statistic read from the speech feature amount storage unit 6 to calculate the likelihood that is the likelihood when each phoneme is assigned to each frame, The phoneme with the highest likelihood is assigned to each frame. As necessary, the likelihood of the phonemes assigned to each frame is sent to the phoneme boundary time estimation result reliability determination unit 3.

  The time range over two frames will be described with reference to FIG. When the phoneme assigned to the frame k (in this example, / a /) is different from the phoneme assigned to the frame k + 1 (in this example, / i /), as illustrated in FIG. 8, the start time of the frame k Any time included in the time range from the end time of frame k + 1 to the end time of frame k + 1 is defined as a phoneme boundary time. For example, the center time of frame k + 1 is set as the phoneme boundary time.

  The first phoneme boundary time estimation unit 2 may estimate the phoneme boundary time by the method described in Non-Patent Document 1.

<Step S3>
The phoneme boundary time estimation result reliability determination unit 3 determines whether the phoneme boundary time estimated by the first phoneme boundary time estimation unit 2 is reliable (step S3). A reliability determination result, which is a determination result of whether or not the phoneme boundary time is reliable, is sent to the second phoneme boundary time estimation unit 4.

  Hereinafter, two examples of determination processing by the phoneme boundary time estimation result reliability determination unit 3 will be introduced.

<< Example 1 >>
FIG. 2 is a functional block diagram of the phoneme boundary time estimation result reliability determination unit 3 according to Example 1.

  The reliability determination condition storage unit 32 stores conditions for phonemes assigned to two consecutive frames corresponding to phoneme boundary times that cannot be trusted. FIG. 9 shows an example of this condition. In the example of FIG. 9, (1) when the phoneme boundary is between voiced sound and unvoiced sound, the time of the phoneme boundary is trusted. (2) When one of the phonemes constituting the phoneme boundary is a friction sound There are three conditions: the time of the phoneme boundary is trusted, and the time of the phoneme boundary is not trusted when the phoneme boundary is between voiced sounds instead of (3) and (2).

  The conditional reliability determination unit 31 receives phonemes assigned to two consecutive frames corresponding to the phoneme boundary time. In the example of FIG. 8, phonemes / a / and phonemes / i / are input. Whether the phoneme boundary time can be trusted by determining whether or not the input phoneme satisfies the condition stored in the reliability determination condition storage unit 32. Determine whether or not.

  The conditional reliability determination unit 31 will be described by taking as an example a case where a voice “real reality” is input. As shown in FIG. 10, the phonemes of / g /, / e /, / n /, / j /, / i /, / ts /, / u /, / o / Are sequentially associated.

  The phoneme boundary / i / − / ts / and the phoneme boundary / ts / − / u / are unvoiced sounds and the phonemes / i / and the phonemes / u / are voiced sounds. It is determined that the phoneme boundary times of these phoneme boundaries are reliable.

  The phoneme boundary / n / − / j / and the phoneme boundary / j / − / i / are phonetic / j / is a frictional sound and phoneme / n / and phoneme / i / are not a frictional sound. It is determined that the phoneme boundary times of these phoneme boundaries are reliable.

  Phoneme boundary / g / − / e /, phoneme boundary / e / − / n /, phoneme boundary / u / − / o / do not satisfy condition (2), and phoneme / g /, phoneme / e / And phoneme / n / are voiced sounds, it is determined by the condition (3) that the phoneme boundary times of these phoneme boundaries cannot be trusted.

<< Example 2 >>
FIG. 3 is a functional block diagram of the phoneme boundary time estimation result reliability determination unit 3 according to Example 2. FIG. 6 shows a flowchart of processing according to Example 2.

  In this example, the likelihood of the phoneme assigned to each frame calculated by the first phoneme boundary time estimation unit 2 and the phoneme boundary time estimation result are input to the phoneme boundary time estimation result reliability determination unit 3. .

  The likelihood change calculation unit 33 of the phoneme boundary time estimation result reliability determination unit 3 is an index indicating the magnitude of change in the likelihood of phonemes assigned to a predetermined number of frames including the frame of the phoneme boundary time. A likelihood change degree is obtained (step S31). The calculated likelihood change degree is sent to the likelihood reliability determination unit 34.

As the likelihood change degree, D defined by the following equation can be used. In the following equation, t is the frame number of the phoneme boundary time, K is a predetermined positive integer, L _i is the likelihood for the phoneme assigned to frame i, and w _k is the likelihood difference | L _t −L _{t -k | + | L t -L t} + k | of desirable monotonically decreases with increasing real number and k of a predetermined non-negative is the weight. K and w _k are appropriately set based on experiments and the like according to the required specifications and performance. The frame at the phoneme boundary time is one of two frames constituting the phoneme boundary time. In the example of FIG. 8, this is frame k or frame k + 1.

  The likelihood reliability determination unit 34 compares the likelihood change degree with a predetermined threshold value θ, and determines that the phoneme boundary time is not reliable if the likelihood change degree is equal to or less than the predetermined threshold value (step S32). . The predetermined threshold is appropriately set based on experiments or the like according to the required specifications and performance.

  For example, when D−θ is calculated, it is determined that the phoneme boundary time can be trusted if D−θ> 0, and the phoneme boundary time cannot be trusted if D−θ ≦ 0. Of course, the likelihood change D and the predetermined threshold value θ may be directly compared, and the reliability may be determined based on the magnitude relationship.

<Step S4>
The duration distribution storage unit 7 stores an average value and variance of durations of a plurality of phonemes.

  The second phoneme boundary time estimation unit 4 expands / contracts to the phonemes constituting the phoneme boundary for which the phoneme boundary time is determined to be unreliable as the average value of the duration of each phoneme increases. By assigning the determined time, the phoneme boundary time of the phoneme boundary at which it is determined that the phoneme boundary time is unreliable is estimated (step S4). For the average value and variance of the duration of each phoneme, values read from the duration distribution storage unit 7 are used.

  FIG. 4 shows a functional block diagram of an example of the second phoneme boundary time estimation unit 4. In FIG. 7, the flowchart of the example of a process of the 2nd phoneme boundary time estimation part 4 is shown.

For example, the phoneme duration maximum likelihood estimator 41 of the second phoneme boundary time estimator 4 calculates the phoneme continuation length m _i ^* of the i-th phoneme determined that the phoneme boundary time is unreliable based on the following equation: (Step S41). T is a time length formed by consecutive I (I is a positive integer) phonemes whose phoneme boundary times are determined to be unreliable, and m _i ^* is an estimated phoneme of the i-th phoneme in the time length T a duration, m _i is the mean value of the duration of the read i-th phoneme from duration distribution storage unit 7, sigma _i ² is duration of the read i-th phoneme from duration distribution storage unit 7 Is the dispersion of.

In the example of FIG. 10, it is determined that the phoneme boundary times of the phoneme boundary / g / − / e /, the phoneme boundary / e / − / n /, and the phoneme boundary / u / − / o / are not reliable. The time length of phoneme / g /, which is the first phoneme of time length T formed by three consecutive phonemes / g /, / e /, / n /, is calculated as follows using the above formula. Can do. In this example, T = 180 ms, average value of duration of phoneme / g / m ₁ = 20 ms, variance of duration of phoneme / g / σ ₁ ² = 0.003, average value of duration of phoneme / e / m ₂ = 95 ms, variance of duration of phoneme / e / σ ₂ ² = 0.012, average duration of phoneme / n / m ₃ = 45 ms, variance of duration of phoneme / n / σ ₃ ² = Suppose that it is 0.005.

^{_{m 1 * = m 1 + (}} σ 1 2 / Σ i = 1 I σ i 2) · (T-Σ i = 1 I m i)
= 20 + (0.003 / (0.003 + 0.012 + 0.005)). (180- (20 + 95 + 45))
= 20 + (0.003 / 0.020) .20
= 23

  The phoneme boundary time determination unit 42 of the second phoneme boundary time estimation unit 4 uses the time length assigned to the phoneme and the reliable phoneme boundary time in the reliability determination result to determine the final phoneme. The boundary time is determined and the result is output (step S4).

  In this way, the phoneme boundary time is temporarily estimated, and the phoneme boundary time that is estimated to be low in accuracy is estimated again by using the average value and variance of the phoneme duration, thereby estimating the phoneme boundary time. It can be performed with high accuracy.

  The detailed phoneme boundary time estimation unit 5 corrects the phoneme boundary time determined by the second phoneme boundary time estimation unit 4 by using the method described in Reference Document 1 to obtain a more accurate phoneme boundary time. You may ask for it. In the technique described in Reference 1, a phoneme boundary time with higher accuracy is set using a Markov model in which a search window is set before and after a phoneme boundary time determined in advance and a spectrum pattern near the phoneme boundary time is learned. Ask for. By narrowing the search window when using the technique described in Reference 1, it is possible to prevent the same context as the correct answer from appearing.

  [Reference 1] Lijuan Wang, Yong Zhao, Min Chu, Frank K. Soong, Jian-Lai Zhou and Zhigang Cao, “Context-Dependent Boundary Model for Refining Boundaries Segmentation of TTS Units,” IEICE Transactions 89-D (3) , pp.1082-1091, 2006

  The phoneme dividing device can be realized by a computer. In this case, the processing contents of each function that the apparatus should have are described by a program. Then, by executing this program on a computer, each processing function in this apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. In this embodiment, these apparatuses are configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

  The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Speech feature-value extraction part 2 1st phoneme boundary time estimation part 3 Phoneme boundary time estimation result reliability determination part 31 Condition reliability determination part 32 Reliability determination condition memory | storage part 33 Likelihood change calculation part 34 Likelihood reliability Determination unit 4 Second phoneme boundary time estimation unit 41 Phoneme duration maximum likelihood estimation unit 42 Phoneme boundary time determination unit 5 Detailed phoneme boundary time estimation unit 6 Speech feature storage unit 7 Duration distribution storage unit

Claims (6)

A voice feature amount extraction unit that extracts a voice feature amount of each frame of the input voice;
A speech feature memory unit statistics for audio feature amounts of a plurality of phonemes are stored,
When the most likely phoneme is assigned to each of the frames using the speech feature amount and the statistic read from the speech feature amount storage unit, and the phonemes assigned in two consecutive frames are different, those two A first phoneme boundary time estimation unit that estimates a phoneme boundary time by setting any of the times included in a time range over two frames as a phoneme boundary time;
A phoneme boundary time estimation result reliability determination unit that determines whether the phoneme boundary time is reliable;
A duration distribution storage unit in which average values and variances of durations of a plurality of phonemes are stored;
For each phoneme constituting the phoneme boundary determined to be unreliable, the duration read from the duration distribution storage unit of each phoneme is longer as the average value is larger, and is greatly expanded as the variance is larger. A second phoneme boundary time estimation unit that estimates a phoneme boundary time of a phoneme boundary that is determined to be unreliable by assigning time;
Phoneme splitting device. The phoneme division device according to claim 1,
The phoneme boundary time estimation result reliability determination unit includes a reliability determination condition storage unit the condition is stored for allocated to two consecutive frames corresponding to the phoneme boundary time can not be trusted phonemes, the A condition reliability determination unit that determines that the phoneme boundary is unreliable when the phonemes assigned to two consecutive frames corresponding to the phoneme boundary time satisfy the conditions read from the reliability determination condition storage unit; including,
A phoneme segmentation device characterized by the above. The phoneme division device according to claim 1,
The first phoneme boundary time estimation unit calculates a likelihood that is a likelihood when each phoneme is assigned to each of the frames, and assigns a phoneme having the highest likelihood to each of the frames.
A phoneme segmentation device characterized by the above. The phoneme dividing device according to claim 3, wherein
The phoneme boundary time estimation result reliability determination unit obtains a likelihood change degree that is an index indicating a magnitude of the change in the likelihood of the phonemes assigned to a predetermined number of frames including the frame of the phoneme boundary time. A likelihood change calculation unit, and a likelihood reliability determination unit that determines that the phoneme boundary time is unreliable if the likelihood change is equal to or less than a predetermined threshold,
A phoneme segmentation device characterized by the above. The audio feature amount storage unit, statistics about the audio feature amounts of a plurality of phonemes are stored,
The duration distribution storage unit, the average value of the duration of a plurality of phonemes and dispersion is stored,
A speech feature amount extraction unit that extracts a speech feature amount of each frame of the input speech;
The first phoneme boundary time estimator assigns the most likely phoneme to each of the frames using the speech feature and the statistics read from the speech feature storage, and is assigned in two consecutive frames. A first phoneme boundary time estimation step for estimating a phoneme boundary time by using any of the times included in the time range over the two frames as a phoneme boundary time when the phonemes are different;
Phoneme boundary time estimation result reliability determination, phoneme boundary time estimation result reliability determination step for determining whether the phoneme boundary time is reliable,
The larger the average value of the continuation length read from the continuation length distribution storage unit of each phoneme for each phoneme constituting the phoneme boundary determined by the second phoneme boundary time estimation unit to be unreliable. A second phoneme boundary time estimation step for estimating a phoneme boundary time of a phoneme boundary that is determined to be unreliable by assigning a longer and longer time to expand and contract as the variance increases;
Phoneme segmentation method.   A phoneme division program for causing a computer to function as each unit of the phoneme division device according to claim 1.

Images