JP2014232245A - Sound clarifying device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound clarifying technology for generating a clear sound even in a noise environment while preventing the natural degradation of the sound.SOLUTION: A determination unit 3 determines, using phoneme information about an inputted sound s(t), the type of phoneme corresponding to each frame constituting the sound s(t). A filter generation unit 11 generates an emphasis filter for emphasizing a sound spectrum S(i, f) of each frame corresponding to the type of phoneme corresponding to the determined each frame. A filter unit 7 generates an emphasis spectrum S'(i, f) by emphasizing the sound spectrum S(i, f) of each frame by using the emphasis filter corresponding to the type of phoneme corresponding to the determined each frame. A spectrum generation unit 8 generates an emphasized sound s'(t) which is a time-domain signal by performing inverse Fourier transform on the generated emphasis spectrum.

Description

  The present invention relates to a technique for clarifying speech such as synthesized speech and natural speech in a noisy environment.

  In recent years, due to the development and popularization of speech synthesis technology, opportunities to listen to synthesized speech messages in various places have increased. Synthetic voices are often heard not only in quiet places, but also in noisy environments such as airports, station platforms, and shopping streets where there is noisy surroundings. In such an environment with noise, there is a problem that it is difficult to hear the synthesized speech. At this time, if the volume is increased, the ease of listening to the voice is improved. However, there is an upper limit for the increase in volume, and if it is too strong, the sound may be distorted, making it difficult to hear the sound.

  Until now, there has been a clarification technology that makes it easy to hear speech in the presence of noise, regardless of whether it is a natural speech or a synthesized speech. In general, there are a plurality of peaks in the frequency spectrum of a vowel sound, which is called a formant. By emphasizing this formant part, it is known to clarify the sound without excessively increasing the volume, and the ease of hearing is improved by using an equalizer that emphasizes the power of the formant of the sound ( For example, refer nonpatent literature 1.).

Fumio Amano, “Hearing Support Function for Easy Phone”, Acoustical Society of Japan 2012 Spring Research Presentation, 1-2-2, pp.1563-1564

  However, the conventional speech clarification method emphasizes formants regardless of the type of phoneme in both synthesized speech and natural speech. The peak portion is emphasized, and the naturalness of the voice may be deteriorated. In addition, even for vowels and voiced consonants, the power and position of formants are important features for the naturalness of speech, so a filter design that emphasizes formants and provides the power of natural speech formants. Fine tuning was necessary.

  An object of the present invention is to provide a speech clarification device, method, and program for generating clear speech even under noise while preventing deterioration of speech naturalness without requiring fine tuning.

  A speech clarification device according to an aspect of the present invention includes a speech spectrum analysis unit that generates a speech spectrum S (i, f) of each frame by performing Fourier transform on the speech s (t) for each frame having a certain time length. And using the phoneme information of the input speech s (t), a determination unit that determines the type of phoneme corresponding to each frame constituting the speech s (t), and the phoneme corresponding to each determined frame Using a filter generation unit that generates an enhancement filter for enhancing the speech spectrum S (i, f) of each frame corresponding to the type, and an enhancement filter corresponding to the type of phoneme corresponding to each determined frame A filter unit that generates the enhanced spectrum S ′ (i, f) by enhancing the speech spectrum S (i, f) of each frame, and a time domain signal by performing an inverse Fourier transform on the generated enhanced spectrum. Generate emphasized speech s' (t) And a spectrum synthesizer.

  By clarifying the voice for each phoneme type, and by emphasizing the formant while maintaining the power relationship of the formant part of the natural voice, the naturalness of the voice is degraded without the need for fine tuning. The voice can be clarified without doing so.

The block diagram which shows the example of the audio | voice clarification apparatus of 1st embodiment. The flowchart which shows the example of the speech clarification method. The figure which shows the example of the audio | voice s (t). The figure which shows the example of phoneme information. The flowchart which shows the example of a process of an audio | voice spectrum analysis part. The flowchart which shows the example of a process of a judgment part. The block diagram which shows the example of a vowel filter production | generation part. The figure for demonstrating the example of a formant. The figure which shows the example of the correlation of the zone | band between formants. Shows an example of a vowel enhancement filter E ₁ (i, f). The block diagram which shows the example of a voiced consonant filter production | generation part. The figure which shows the example of the correlation between bands. Shows an example of a voiced consonant enhancement filter E ₂ (i, f). The figure which shows the example of the correlation between bands. The flowchart which shows the example of a process of a spectrum synthetic | combination part. The block diagram which shows the example of the speech clarification apparatus of 2nd embodiment.

  Hereinafter, embodiments of a speech clarification apparatus and method will be described with reference to the drawings.

[First embodiment]
As shown in FIG. 1, the speech clarification device according to the first embodiment includes a speech synthesis unit 1, a speech spectrum analysis unit 2, a determination unit 3, a filter generation unit 11, a filter unit 7, a spectrum synthesis unit 8, and a noise spectrum analysis. For example, the unit 9 is provided. The filter generation unit 11 includes, for example, a vowel filter generation unit 4, a voiced consonant filter generation unit 5, and an unvoiced consonant filter generation unit 6. The voice clarification method is realized by the voice clarification apparatus performing, for example, each step in FIG.

<Step S1>
The speech synthesizer 1 receives a text to be synthesized and outputs a speech waveform s (t). The speech synthesizer 1 also uses phoneme information (ph (k), Ts (k), Te (k)), which is intermediate information when generating a speech waveform, as an output (step S1). That is, the speech synthesizer 1 generates phoneme information based on the input text, generates a synthesized speech corresponding to the input text using the phoneme information, and sets it as a speech s (t). The phoneme information is information about phonemes constituting the speech s (t).

The voice s (t) and the emphasized voice s ′ (t) to be described later have a voice sampling time t (t = 0, 1,..., T−1) when the voice sampling frequency is f _s [Hz]. Is the amplitude at. An example of the voice s (t) is shown in FIG. The voice s (t) illustrated in FIG. 3 is a voice with f _s = 16000 and T = 20000, and T / f _s = 1.25 seconds.

  As described above, the speech synthesizer 1 receives text as input and synthesizes speech s (t) corresponding to the input text. The voice synthesis at that time can be realized by a known technique such as Reference 1.

  [Reference 1] Takashi Masuko, Keiichi Tokuda, Takao Kobayashi, Kiyoshi Imai, “HMM-based speech synthesis using dynamic features”, IEICE Transactions, D-II, vol. J79-D-II, No.12, pp.2184-2190, 1996

  When synthesizing speech, the speech synthesizer 1 generates information on the type of sound (phoneme) ph (k), phoneme start time Ts (k), and end time Te (k). ph (k) is phoneme information, and the type of the kth phoneme is output as predetermined alphabet information. The phoneme start time and end time are represented by the number of samples, for example. FIG. 4 shows an example of this phoneme information. The speech clarification device changes a speech clarification algorithm depending on the type of phoneme in order to achieve effective clarification. For this purpose, the speech synthesizer 1 also outputs this phoneme information ph (k), Ts (k), Te (k).

  The speech s (t) generated by the speech synthesizer 1 is provided to the speech spectrum analyzer 2. The phoneme information (ph (k), Ts (k), Te (k)) is provided to the determination unit 3.

<Step S2>
The speech spectrum analyzer 2 receives the speech s (t) and outputs a speech spectrum S (i, f). The speech spectrum analysis unit 2 analyzes the speech s (t) at p sample intervals, and extracts the speech spectrum S (i, f) [dB]. p is a frame shift width for performing speech analysis, and can be set to, for example, p = 0.032 * f _s (analysis at intervals of 32 ms).

i (i = 0,1, ..., [(T-1) / p]) is an analysis number (frame number) when analyzed at p-sample intervals, and t = ip + m (m = 0,1 , ..., p-1). Also, f (f = 0,1, ..., D-1) is (f / N) x (f _s / 2) [Hz] or more, ((f + 1) / N) x (f _s / 2 ) A number (band number) representing a frequency band below [Hz]. D is the FFT length can be, for example, D = 2 ^10. The speech spectrum S (i, f) is a complex number indicating the short-time spectrum of speech, and | S (i, f) | ² is the power of the frequency number f in the i-th frame, arg {S (i, f) } Indicates a phase. A flowchart of an example of processing performed by the speech spectrum analysis unit 2 is shown in FIG. The voice spectrum analysis unit 2 performs, for example, the following calculation.

  (1) Frame number i = 0.

  (2) When calculation is completed for all frames, that is, when i> [(T-1) / p], the process ends.

  (3) Set frequency number f = 0.

  (4) When calculation is completed for all frequency numbers f, that is, when f> D−1, proceed to (7).

(5) The voice s (t) is cut out using the window function w (p, f), and is set as the cut out voice s ₀ (f).

s ₀ (f) = w (p, f) s (ip + f)
w (p, f) is a window function used in frequency spectrum analysis, and is a function for smoothly cutting out speech. Various window functions have been proposed. For example, the window function can be controlled using a Hamming window represented by the following equation.

  (6) Return to (4) as f ← f + 1

(7) A discrete Fourier transform of length D is performed on the extracted speech sample s ₀ (f) (f = 0, 1,..., D−1) to obtain a speech spectrum S (i, f).

  (8) Return to (2) as i ← i + 1.

  In this way, the speech spectrum analysis unit 2 generates the speech spectrum S (i, f) of each frame by performing Fourier transform on the speech s (t) for each frame having a certain time length (step S2). The generated speech spectrum S (i, f) is provided to the filter generation unit 11 and the filter unit 7.

  The sound has different acoustic feature quantities that correlate with ease of hearing depending on the type of phoneme. For example, Reference Document 2 shows that feature quantities that are correlated with ease of hearing are different for vowels, voiced consonants, and unvoiced consonants. The present technology improves easiness of hearing while preventing deterioration of naturalness by switching the clarification process for each phoneme type of vowels, voiced consonants, and unvoiced consonants determined by the determination unit 3, for example.

  [Reference 2] Ayami Kamiyama, Yusuke Ijima, Mitsuaki Isogai, Hideyuki Mizuno, “Analysis of Relationship between Ease of Listening of Noise Superimposed Speech and Acoustic Features”, IEICE Technical Report, vol.112, no.81, SP2012-46, pp.69-74, 2012

<Step S3>
The noise spectrum analyzer 9 receives the noise n (t) as an input and outputs a speech spectrum N (i, f). In other words, the noise spectrum analysis unit 9 generates a noise spectrum N (i, f) of each frame by Fourier transforming the noise n (t) for each frame having a certain time length (step S3). The noise n (t) is, for example, noise in a field where the emphasized speech s ′ (t) is output.

  The processing of the noise spectrum analysis unit 9 is the same as the processing of the voice spectrum analysis unit 2 except that the input is noise n (t) instead of the voice s (t). To do.

  The generated noise spectrum N (i, f) is provided to the filter generation unit 11.

<Step S4>
The determination unit 3 determines the type of phoneme corresponding to each frame constituting the speech s (t) using the phoneme information of the input written s (t) (step S4). For example, the determination unit 3 receives the phoneme information (ph (k), Ts (k), Te (k)) and outputs vowel section information H _vo , voiced consonant section information H _vc and unvoiced consonant section information H _uc . To do. In other words, the determination unit 3 generates, for example, vowel segment information H _vo , voiced consonant segment information H _vc, and unvoiced consonant segment information H _uc for switching the clarification processing based on phoneme information. A flowchart of an example of processing performed by the determination unit 3 is shown in FIG.

Specifically, the determination unit 3 performs the following processing. First, as preprocessing, k = 0, vowel interval H _vo = φ, voiced consonant interval H _vc = φ, and unvoiced consonant interval H _uc = φ.

  (1) Exit when k> K.

  (2) i = [(Ts (k) -1) / p].

  (3) If i> [(Te (k) -1) / p], return to (2) as k ← k + 1.

  (4) The clarification process is switched by performing the following processes (i) to (iii) depending on the type of ph (k).

(i) When ph (k) is a vowel, _let H _vo ← H _vo ∪ {i}.

(ii) When ph (k) is a voiced consonant, H _vc ← H _vc ∪ {i}.

(iii) When ph (k) is an unvoiced consonant, _let H _uo ← H _uo ∪ {i}.

  (5) Return to (3) as i ← i + 1.

In this way, the determination unit 3 uses the phoneme information to determine, for example, whether the phoneme corresponding to each frame constituting the speech s (t) is a vowel, voiced consonant, or unvoiced consonant (step S4). ). The vowel section information H _vo , the voiced consonant section information H _vc, and the unvoiced consonant section information H _uc , which are information about whether the phoneme corresponding to each frame is a vowel, a voiced consonant, or an unvoiced consonant, respectively, 4. Presented to voiced consonant filter generation unit 5 and unvoiced consonant filter generation unit 6.

<Step S5 to Step S7>
The filter generation unit 11 generates an enhancement filter for enhancing the speech spectrum S (i, f) of each frame corresponding to the type of phoneme corresponding to each frame determined by the determination unit 3 (step S5). To step S7). The generated enhancement filter is provided to the filter unit 7.

  As will be described later, the emphasis filter corresponding to the type of phoneme generated by the filter generation unit 11 is an emphasis while maintaining the relative power relationship of these bands based on the normal distribution between the powers.

  In this example, the determination unit 3 determines whether the phoneme is a vowel, a voiced consonant, or an unvoiced consonant. Therefore, in this example, the filter generation unit 11 includes a vowel filter generation unit 4, a voiced consonant filter generation unit 5, and an unvoiced consonant filter generation unit 6. If it is determined that the phoneme is a vowel, the vowel filter generation is performed. When the unit 4 performs the process of step S5 described below and it is determined that the phoneme is a voiced consonant, the voiced consonant filter generation unit 5 performs the process of step S6 described below, and the phoneme is an unvoiced consonant. If it is determined, the unvoiced consonant filter generation unit 6 performs the process of step S7 described below.

<Step S5>
The vowel filter generation unit 4 receives the speech s (t), the speech spectrum S (i, f), the noise spectrum N (i, f), and the vowel section information H _vo as inputs, and the vowel enhancement filter E ₁ (i, f) Is output.

The vowel filter generation unit 4 generates a filter E ₁ (i, f) for clarifying vowels with respect to the frequency spectrum S (i, f) analyzed by the speech spectrum analysis unit 2. There are various methods for clarifying the vowels. For example, a method for clarifying the vowels by the functional configuration shown in FIG.

  In this example, the vowel filter generation unit 4 includes a formant extraction unit 41 and an enhancement filter generation unit 42.

  The formant extraction unit 41 receives the speech s (t) and outputs formant information F (i, j). The formant extraction unit 41 is realized by a known method such as Reference 3.

  [Reference 3] Takahiro Otsuka, “Robust ARX Speech Analysis Method Considering Source Pulse Train”, Journal of the Acoustical Society of Japan, Vol.58, No.7, pp.386-397, 2002.7

  A formant is a peak portion of a voice spectrum as shown in FIG. 8, and is distinguished by a number such as a first formant and a second formant from a low frequency. The position of this formant on the frequency axis characterizes the phoneme and speaker nature of speech. The formant extraction unit 41 extracts the formant frequency F (i, j) [Hz] from the speech s (t) at p sample intervals. i (i = 0,1,..., [(T-1) / p]) is the same as the speech spectrum analysis unit 2 and is an analysis number (frame number). Further, j (j = 1, 2,..., J) is a formant number, and F (i, j) is the position of the j-th formant. J is the number of formants to be extracted, and is a value of about 4 or 5. F (i, j) is F (i, j) = 0 for each j (j = 1, 2,..., J) when the i-th frame is a section where there is no formant such as a silent section and a silent section. It becomes.

The enhancement filter generation unit 42 receives the noise spectrum N (i, f), the speech spectrum S (i, f), and the vowel section information _Hvo, and makes the filter E ₁ (i, f) easy to hear vowels under noise. Is output.

  It is known that the phonological properties of speech vowels are characterized by formants, and it is important that the speech spectrum at the position of this formant frequency can be heard without masking even under noise. On the other hand, since the position and power of this formant characterize a natural sound, it is necessary to design an emphasis filter that emphasizes the formant so that the sound becomes natural.

FIG. 9 shows a band between formants of average power density of 20 speakers, and shows a correlation coefficient between 20 persons of average power per band number of a band including the j-th formant. The power per band number of the band including the jth formant indicates the relative power of the formant. The average power density is defined by the following formula P _d (i, f), and the relative power of the portion including the jth formant is B (j) below.

  B (1) and B (3), B (4), and B (3) and B (2) have a relatively strong correlation. It is necessary to emphasize formants while maintaining correlation. In addition, it is desirable that the sound power is not increased too much by emphasizing the formants.

Therefore, assuming that B (j) is a probability of normal distribution, the average μ _{B of} B = (B (1), B (2), ..., B (J-1)) ^T and the covariance matrix Σ _B To obtain an average power distribution. That is, P (B) = N (B; μ _B , Σ _B ). In addition, as shown in Reference Document 2, since the voice and noise power ratio of the third formant have a correlation with ease of hearing, for example, j = 3 based on Reference Document 2 A filter E ₁ (i, f) that emphasizes the formant is generated as follows.

(1) The voice / noise power ratio R = ₁₀ log ₁₀ (Ps (j) / Pn (j)) of the jth formant part is obtained using the following formulas Ps and Pn.

(2) For all i∈H _vo , f = 0,1, ..., D-1 when R is greater than R ′ than a predetermined power ratio, that is, R> R ′, E ₁ (i, f ) = 1.

(3) B = (B (1), B (2), ..., B (J-1)) ^T is obtained using the equation (* 1).

(4) The power Ps = (Ps (1), Ps (2),..., Ps (J-1)) ^T is obtained using the expression (* 2).

(5) Total power M ′ = Σi _{= 1} ^J Ps (i) −Ps (j) other than the j-th is obtained.

  (6) The target average power B ′ is obtained by the following (i)-(v), for example.

(i) A = Ps. / B = (Ps (1) / B (1), Ps (2) / B (2),…, Ps (J-1) / B (J-1)) ^T .

  (ii) The target average power B ′ (j) = 10 ^ ((R−R ′) / 20) B (j) for the j-th formant.

(iii) Conditional normal distribution P (B | B '(j) when B' (j) is obtained using _B mean μ _B = {μ _n } and covariance matrix Σ _B = {σ _nm } )) = N (B | B ′ (j); μ _B , Σ _B ) = N (B ′; μ ′ _B , Σ ′ _B ) average μ ′ _B and covariance matrix Σ ′ _B are obtained. In order to derive μ ′ _B and Σ ′ _B , an average vector μ _B1 obtained by removing the j-th element from the average μ _B of B below, and a matrix Σ obtained by removing the j-th row and j-th column from the variance matrix Σ _{B of B 11.} A vector Σ ₁₂ other than the j-th row in the j-th column, a vector Σ ₂₁ other than the j-th column in the j-th row, and an element Σ ₂₂ in the j-th row and j-th column are used.

Using the above, μ ′ _B and Σ ′ _B are obtained as follows.

  (iv) A vector A ′ obtained by removing the j-th element from A is obtained.

  (v) The remaining target average power B 'is obtained by the following equation.

B ′ = B ′ (1),..., B ′ (J−1)).

  (7) Obtain a gain width G = B ′ ./ B for each formant band.

(8) Generate a vowel enhancement filter E ₁ (i, f) that multiplies the power of each formant band by G. Various filters can be constructed. For example, as shown in FIG. 10, a vowel emphasis filter E ₁ (i, f) is formed with g (j) as the formant part and 1 as the midpoint between each formant. A filter satisfying the following relationship can be configured.

  The total target power is

It can be seen that the total of the newly calculated target power is the same as before the constant control. Also, since the new power only finds B ′ that maximizes the conditional probability distribution P (B | B ′ (j)) of B, the new average powers B ′ and Ps ( j) G (j) can be obtained. In this example, based on Reference Document 2, an example in which the third formant is emphasized with j = 3 is shown, but the second and fourth formants may be controlled in the same manner.

In this way, when the phoneme corresponding to a certain frame i is a vowel, the vowel filter generation unit 4 enhances the vowel enhancement filter E for enhancing the vowel with respect to the speech spectrum S (i, f) of the frame i. ₁ (i, f) is generated (step S5).

The vowel enhancement filter E ₁ (i, f) generated in this way assumes that the relative power of the bands including each formant follows a normal distribution, while maintaining the relationship of the relative powers of these bands. It can be said that it is a filter that emphasizes vowels. The generated vowel enhancement filter E ₁ (i, f) is provided to the filter unit 7.

<Step S6>
The voiced consonant filter generation unit 5 receives the noise spectrum N (i, f), the voice spectrum S (i, f), and the voiced consonant interval information H _vc and outputs a voiced consonant enhancement filter E ₂ (i, f). .

The voiced consonant filter generation unit 5 generates a filter E ₂ (i, f) for clarifying the voiced consonant with respect to the frequency spectrum S (i, f) analyzed by the voice spectrum analysis unit 2. Although voiced consonants also have formants, there are sounds with fewer formants than vowels, such as repellent sounds. Therefore, for voiced consonants, not a formant band, but a specific frequency band, and the other bands are controlled based on the correlation in the same way as the vowel filter generation unit, thereby improving the ease of listening to the voice. Can be considered. This specific band may be controlled as described in (1) to (8) below using M (k) and average power C. This process is performed by the voiced consonant enhancement unit 51 provided in the voiced consonant filter generation unit 5 as illustrated in FIG.

  FIG. 12 shows an example of the correlation of the power of voiced consonants based on the voice data of 20 speakers. Here, the band M to be divided is L = 5, M (0) = 0, M (1) = D * (2000 / fs), M (2) = D * (4000 / fs), M (3) = D * (6000 / fs), M (4) = D * (8000 / fs), M (5) = D * (16000 / fs), (boundary 0kHz, 1kHz, 2kHz, 4kHz, 6kH, 8kHz respectively) Band). The relative power of the band k is C (k) defined in the description of (3) below.

In the example of FIG. 12, it can be seen that C (1) and C (2), C (1) and C (3), and C (2) and C (3) have a relatively strong correlation. In order to become natural speech, it is necessary to emphasize while maintaining the relative power correlation between these bands. (1) Using the following formulas Ps and Pn, the voice-to-noise power ratio R = 10log ₁₀ (Ps (m) / Pn (m) in the m-th band (M (m-1) to M (m)) )).

(2) For all i∈H _vo , f = 0,1, ..., D-1 when R is greater than R ′ than the predetermined power ratio, that is, R> R ′, E ₂ (i, f ) = 1.

(3) C = (C (1), C (2), ..., C (L-1)) ^T is obtained using the following equation. L is the number of band division.

(4) The power Ps = (Ps (1), Ps (2),..., Ps (L-1)) ^T is obtained using the equation (* 3).

(5) Total power M ′ = Σ _{i = 1} ^M Ps (i) −Ps (m) other than the m-th is obtained.

  (6) The target average power C ′ is obtained by the following (i)-(v), for example.

(i) D = Ps. / C = (Ps (1) / C (1), Ps (2) / C (2),…, Ps (L-1) / C (L-1)) ^T .

  (ii) Target average power C ′ (j) = 10 ^ ((R−R ′) / 20) C (m).

(iii) Using C mean μ _C = {μ _n } and covariance matrix Σ _C = {σ _nm }, C ′ (m) is a conditional normal distribution P (C | C ′ (m )) = N (C | C ′ (m); μ _C , Σ _C ) = N (C ′; μ ′ _C , Σ ′ _C ) average μ ′ _C and covariance matrix Σ ′ _C are obtained. For the derivation of μ ′ _C and Σ ′ _C , the mean vector μ _c1 obtained by removing the m-th element from the mean _C of C below, and the matrix Σ obtained by removing the m-th row and m-th column from the _C dispersion matrix Σ _{C 11.} A vector Σ ₁₂ other than the m-th row in the m-th column, a vector Σ ₂₁ other than the m-th column in the m-th row, and an element Σ ₂₂ in the m-th row and the m-th column are used.

By obtaining the above, μ ′ _C and Σ ′ _C are obtained as follows.

  (iv) A vector D ′ obtained by removing the m-th element from D is obtained.

  (v) The remaining target average power C ′ is obtained by the following equation.

C ′ = (B ′ (1),..., B ′ (L−1)).

  (7) Obtain the gain width G = C ′ ./ C for each band.

(8) A filter E ₂ (i, f) that multiplies the power for each band by G is generated. Various filters can be constructed. For example, as shown in FIG. 13, a filter E ₂ (i, f) having g (j) at the center of the band and 1 at the boundary point of the band is formed and Σ _{f = M (m-1)} ^{M (m)} | E ₂ (i, f) S (i, f) | ² = Ps (j) G (j) can be configured.

  For example, the power ratio R ′ = 0 [dB], the band M to be divided is L = 5, M (0) = 0, M (1) = D * (2000 / fs), M (2) = D * (4000 / fs), M (3) = D * (6000 / fs), M (4) = D * (8000 / fs), M (5) = D * (16000 / fs), (0kHz, 1kHz, 2kHz respectively) , 4 kHz, 6 kHz, 8 kHz), and the band for calculating the SN ratio is preferably m = 2 based on Reference 2.

As described above, when the phoneme corresponding to a certain frame i is a voiced consonant, the voiced consonant filter generation unit 5 emphasizes the voiced consonant for the voice spectrum S (i, f) of the frame i. A consonant enhancement filter E ₂ (i, f) is generated (step S6).

The voiced consonant enhancement filter E ₂ (i, f) generated in this way assumes that the relative power of each band follows a normal distribution, while maintaining the relationship between the relative powers of these bands. It can be said that it is a filter that emphasizes. The generated voiced consonant enhancement filter E ₂ (i, f) is provided to the filter unit 7.

<Step S7>
The unvoiced consonant filter generation unit 6 receives the noise spectrum N (i, f), the voice spectrum S (i, f), and the unvoiced consonant section information _Huc and outputs an unvoiced consonant enhancement filter E ₃ (i, f). .

The unvoiced consonant filter generation unit 6 emphasizes a specific band in the same manner as the voiced consonant filter generation unit 5. The functional configuration of the unvoiced consonant filter generation unit 6 is the same as that of the voiced consonant filter generation unit 5, and the section to be clarified is i∈H _uc . Unvoiced consonants have many high frequency components, and the ease of hearing is related to the power of high frequency speech. Therefore, R '= 0 [dB], the band M to be divided is L = 5, M (0) = 0, M (1) = D (2000 / fs), M (2) = D (4000 / fs) , M (3) = D (6000 / fs), M (4) = D (8000 / fs), M (5) = D (16000 / fs), (0kHz, 1kHz, 2kHz, 4kHz, 6kH, 8kHz respectively) And a band m = 5 for calculating the SN ratio.

  FIG. 14 shows an example of the correlation of the power of unvoiced consonants based on the voice data of 20 speakers. The band M to be divided is as in the above example of division with L = 5. The relative power of the band k is C (k) defined in the description of the voiced consonant enhancement unit 51.

  In the example of FIG. 14, it can be seen that C (2) and C (3) have a relatively strong correlation. In order to become natural speech, it is necessary to emphasize while maintaining the relative power correlation between these bands.

Thus, when the phoneme corresponding to a certain frame i is an unvoiced consonant, the unvoiced consonant filter generation unit 6 is unvoiced for enhancing the unvoiced consonant with respect to the speech spectrum S (i, f) of that frame i. A consonant enhancement filter E ₃ (i, f) is generated (step S7).

The unvoiced consonant enhancement filter E ₃ (i, f) generated in this way assumes that the relative power of each band follows a normal distribution, while maintaining the relationship between the relative powers of these bands. It can be said that it is a filter that emphasizes. The generated unvoiced consonant enhancement filter E ₃ (i, f) is provided to the filter unit 7.

<Step S8>
The filter unit 7 receives the speech spectrum S (i, f), the vowel enhancement filter E ₁ (i, f), the voiced consonant enhancement filter E ₂ (i, f), and the unvoiced consonant enhancement filter E ₃ (i, f). As a result, the enhanced spectrum S ′ (i, f) is output.

  The filter unit 7 generates a convolution enhancement spectrum by convolving the filters generated by the vowel filter generation unit 4, the voiced consonant filter generation unit 5, and the unvoiced consonant filter generation unit 6 with the speech spectrum. The filter can be easily integrated by the following equation, and the enhanced spectrum can be calculated using the following equation E as S ^ '(i, f) = E (i, f) S (i, f).

  In order to reduce the discomfort of the discontinuous portions of the filter of each phoneme, smoothing may be performed in the i direction (that is, the time direction) after integrating the filters by the above formula.

As described above, when the phoneme corresponding to a certain frame i is a vowel, the filter unit 7 uses the vowel enhancement filter E ₁ (i, f) of the frame i and the speech spectrum S (i, f) is emphasized to generate an enhancement spectrum S ′ (i, f) corresponding to the frame i. If the phoneme corresponding to a frame i is a voiced consonant, the voiced consonant enhancement filter of the frame i is used. The speech spectrum S (i, f) of the frame i is emphasized to generate an enhanced spectrum S ′ (i, f) corresponding to the frame i, and the phoneme corresponding to the frame i is an unvoiced consonant. Emphasizes the speech spectrum S (i, f) of the frame i using the unvoiced consonant enhancement filter E ₃ (i, f) of the frame i, and the enhanced spectrum S ′ (i, f) corresponding to the frame i Is generated (step S8). The generated enhanced spectrum S ′ (i, f) is provided to the spectrum synthesis unit 8.

<Step S9>
The spectrum synthesizing unit 8 has a reverse input / output relationship with the speech spectrum analyzing unit 2 and outputs the enhanced speech s ′ (t) with the enhanced spectrum S ′ (i, f) as an input. Specifically, for example, the emphasized speech s ′ (t) is obtained by the following. A flowchart of an example of processing performed by the spectrum synthesizer 8 is shown in FIG.

  (1) Frame number i = 0.

  (2) When calculation is completed for all frames, that is, when i> [(T-1) / p], the process ends.

(3) The formant-enhanced spectrum S ′ (i, f) is subjected to inverse Fourier transform of length D to be converted into speech samples s ₁ (f) (f = 0, 1,..., D−1).

  (3) Set frequency number f = 0.

  (4) When calculation is completed for all frequency numbers f, that is, when f> D−1, proceed to (7).

(5) The obtained speech sample s ₁ (f) is added to the enhanced speech s ′ (t) by the following equation.

  (6) Return to (4) as f ← f + 1.

  (7) Return to (2) as i ← i + 1.

  As described above, the spectrum synthesizer 8 generates the enhanced speech s ′ (t), which is a time domain signal, by performing inverse Fourier transform on the generated enhanced spectrum (step S9).

  In this way, by performing speech clarification for each phoneme type of synthesized speech, and by emphasizing the formant while maintaining the power relationship of the formant part of natural speech, it is possible to perform speech without requiring fine tuning The voice can be clarified without degrading the naturalness of the sound.

[Second Embodiment]
The voice clarification device of the second embodiment includes, for example, a voice recognition unit 10 as illustrated in FIG. 16 instead of the voice synthesis unit 1 of the first embodiment. The speech recognition unit 10 receives speech s (t) and outputs phoneme information (ph (k), Ts (k), Te (k)).

  In the first embodiment, the speech synthesizer 1 uses the phoneme information (ph (k), Ts (k), Te (k)) generated during speech synthesis for clarification. In the second embodiment, this phoneme information is extracted by using the speech recognition unit 10 (step S1), thereby clarifying for each phoneme type. The voice recognition unit 10 can be realized by using an existing voice recognition technique described in Reference Document 4, for example.

  [Reference 4] P.C.Woodland, J.J.Odell, V.Valtchev 63 S.J.Young, “Large vocabulary continuous speech recognition using HTK”, ICASSP, vol.2, pp.II / 125-II128, 1994

  Since the other processes of the second embodiment are the same as those of the first embodiment, a duplicate description is omitted.

[Modification]
In the above example, the enhancement filter is generated and applied in all of the vowel interval, voiced consonant interval, and unvoiced consonant interval. However, the enhancement filter is generated and applied in at least one phoneme type interval. May be performed. For example, the enhancement filter may be generated and applied only for the vowel section. Further, for the unvoiced consonant section, it is not necessary to generate the enhancement filter and apply it.

  In the above example, the phonemes are classified into vowels, voiced consonants, and unvoiced consonants. This is an example of phoneme classification. Phonemes may be divided into other types. For example, filter generation for emphasizing some phonemes constituting a vowel and processing of applying the same are performed, and some other phonemes constituting other vowels are emphasized. Filter generation and filter application processing may be performed.

  When generating a voiced consonant enhancement filter, the calculation for generating the voiced consonant enhancement filter described above may be performed only for a band having a relatively strong correlation. For example, in the example shown in FIG. 12, C (1) and C (2), C (1) and C (3), and C (2) and B (3) have a relatively strong correlation. The calculation for generating the above-described voiced consonant enhancement filter may be performed only for the band from k = 0 to k = 3, that is, the band from 0 kHz to 4 kHz.

  Similarly, when generating an unvoiced consonant enhancement filter, the calculation for generating the unvoiced consonant enhancement filter described above may be performed only for a band having a relatively strong correlation.

  The processes described in the above apparatus and method are not only executed in time series according to the description order, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the process.

  Further, when each process in the speech clarification device is realized by a computer, the processing contents of the functions that the device should have are described by a program. Then, by executing this program on a computer, each process is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  Each processing means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

  Needless to say, other modifications are possible without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Speech synthesis part 2 Speech spectrum analysis part 3 Judgment part 4 Vowel filter production | generation part 41 Formant extraction part 42 Emphasis filter production | generation part 5 Voiced consonant filter production | generation part 51 Voiced consonant enhancement part 6 Unvoiced consonant filter production | generation part 7 Filter part 8 Spectrum synthesis part 9 Noise Spectrum Analysis Unit 10 Speech Recognition Unit 11 Filter Generation Unit

Claims (7)

A voice spectrum analyzer that generates a voice spectrum S (i, f) of each frame by performing Fourier transform on the voice s (t) for each frame of a certain time length;
Using the phoneme information of the input speech s (t), a determination unit that determines the type of phoneme corresponding to each frame constituting the speech s (t);
A filter generation unit that generates an enhancement filter for enhancing the speech spectrum S (i, f) of each frame corresponding to the type of phoneme corresponding to each of the determined frames;
Using the enhancement filter corresponding to the type of phoneme corresponding to each frame determined above, the speech spectrum S (i, f) of each frame is enhanced to generate the enhancement spectrum S ′ (i, f) A filter section to perform,
A spectrum synthesizer that generates an enhanced speech s ′ (t) that is a time domain signal by performing inverse Fourier transform on the generated enhanced spectrum, and
A speech clarification device including: The speech clarification device according to claim 1,
It further includes a speech synthesizer that generates phoneme information based on the input text and uses it as the phoneme information, generates a synthesized speech corresponding to the input text using the phoneme information, and uses it as the speech s (t) ,
Voice clarification device. The speech clarification device according to claim 1,
A speech recognition unit that generates the phoneme information by recognizing the speech s (t);
Voice clarification device. The speech clarification device according to any one of claims 1 to 3,
The enhancement filter corresponding to the phoneme type is a filter that emphasizes while maintaining the relative power relationship of these bands based on the normal distribution between the powers.
Voice clarification device. The speech clarification device according to any one of claims 1 to 4,
The determination unit uses the phoneme information of the input speech s (t) to determine whether the phoneme corresponding to each frame constituting the speech s (t) is a vowel, voiced consonant, or unvoiced consonant Judging
When the phoneme corresponding to each frame is a vowel, the filter unit vowel enhancement filter E ₁ (i, f) for enhancing the vowels with respect to the speech spectrum S (i, f) of each frame And a vowel consonant enhancement filter for enhancing the voiced consonant with respect to the speech spectrum S (i, f) of each frame when the phoneme corresponding to each frame is a voiced consonant. A voiced consonant filter generator that generates E ₂ (i, f), and when the phoneme corresponding to each frame is an unvoiced consonant, an unvoiced consonant for the speech spectrum S (i, f) of each frame is An unvoiced consonant filter generation unit that generates an unvoiced consonant enhancement filter E ₃ (i, f) for emphasis, and
When the phoneme corresponding to each frame is a vowel, the filter unit emphasizes the speech spectrum S (i, f) of each frame using the vowel enhancement filter E ₁ (i, f). An enhancement spectrum S ′ (i, f) corresponding to each frame is generated, and when the phoneme corresponding to each frame is a voiced consonant, each of the above-mentioned each using the voiced consonant enhancement filter E ₂ (i, f) The speech spectrum S (i, f) of the frame is emphasized to generate an enhanced spectrum S ′ (i, f) corresponding to each of the frames. When the phoneme corresponding to each frame is an unvoiced consonant, the unvoiced Using the consonant enhancement filter E ₃ (i, f), the speech spectrum S (i, f) of each frame is emphasized to generate the enhanced spectrum S ′ (i, f) corresponding to each frame.
Voice clarification device. The voice spectrum analysis unit generates a voice spectrum S (i, f) of each frame by Fourier transforming the voice s (t) for each frame of a certain time length, and
The determination unit determines a phoneme type corresponding to each frame constituting the speech s (t) using the input phoneme information of the speech s (t);
A filter generating step for generating an enhancement filter for enhancing the speech spectrum S (i, f) of each frame corresponding to the type of phoneme corresponding to each of the determined frames;
The filter unit emphasizes the speech spectrum S (i, f) of each frame by using the enhancement filter corresponding to the type of phoneme corresponding to each of the determined frames, thereby enhancing the enhanced spectrum S ′ (i, a filter step that generates f);
A spectrum synthesis step in which a spectrum synthesis unit generates an enhanced speech s ′ (t) that is a time domain signal by performing an inverse Fourier transform on the generated enhanced spectrum;
A speech clarification method including:   The program for functioning a computer as each part of the speech clarification apparatus described in any one of Claim 1 to 5.

Images