JP6400526B2 - Speech synthesis apparatus, method thereof, and program - Google Patents

Description

  The present invention relates to speech synthesis technology, and more particularly to technology for improving the quality of synthesized speech.

  With the recent development of statistical speech synthesis technology, high-quality synthesized speech can be generated. For example, with the development of HMM (Hidden Markov Model) speech synthesis technology (for example, see Non-Patent Document 1 etc.), by learning speech data of an arbitrary speaker, the synthesized speech with the voice quality and tone of the speaker can be learned. It can be generated.

  In addition, various methods (for example, see Non-Patent Document 2) have been proposed as techniques for improving the quality of synthesized speech. However, statistical speech synthesis cannot capture various phenomena of actual speech, and there is a limit to improving quality. Therefore, a method for improving the quality of synthesized speech by utilizing the quality of the original speech has been proposed. For example, in Non-Patent Document 3, a unit connection type speech synthesis method (for example, see Patent Document 1) that utilizes the quality of the original speech is combined with a HMM speech synthesis method, and the spectrum of the unit connection type speech synthesis method is applied to the high frequency part. There has been proposed a method for improving the quality of synthesized speech by using the spectrum of the HMM speech synthesis method in the low frequency part.

Japanese Patent No. 2761552

H. Zen, K. Tokuda, T. Masuko, T. Kobayashi and T. Kitamura, “A Hidden semi-Markov model-based speech synthesis system,” IEICE Trans. Inf. And Syst., Vol. E90-D, no .5, pp.825-834, 2007. T. Toda., And K. Tokuda. “A speech parameter generation algorithm considering global variance for HMM-based speech synthesis.” IEICE Trans. Inf. And Syst vol.E90-D, no.5, pp.816-824, 2007. T. Inoue, S. Hara, M. Abe, “A Hybrid Text-to-Speech Based on Sub-Band Approach,” in Proc of APSIPA DOI: 10.1109 / APSIPA. 2014.7041575, 2014.

  In the method of Non-Patent Document 3, the boundary frequency serving as the connection boundary between the high frequency part and the low frequency part greatly affects the quality of the synthesized speech. That is, as the boundary frequency is lowered, the spectrum by the unit connection type speech synthesis method in the synthesized speech increases and the spectrum by the HMM speech synthesis method decreases. For this reason, the lower the boundary frequency, the better the natural feeling (speech feeling) of the synthesized speech, but the higher the frequency of abnormal noise generation based on the distortion at the connection of adjacent phonemes (segments). On the other hand, as the boundary frequency is increased, the spectrum of the unit connection type speech synthesis method occupied in the synthesized speech decreases and the spectrum of the HMM speech synthesis method increases. For this reason, the higher the boundary frequency, the lower the frequency of abnormal noise generation based on the distortion at the phoneme connection, but the natural feeling of the synthesized speech decreases. Therefore, the quality of synthesized speech cannot be improved unless both the generation of abnormal sounds based on distortion at the phoneme connection portion and the improvement of natural feeling due to the introduction of the unit connection type speech synthesis method are taken into consideration.

  An appropriate boundary frequency considering these depends on the phoneme connected to the unit. However, in the method of Non-Patent Document 3, since a uniform boundary frequency is set, a sufficient quality improvement effect may not be obtained. Conventionally, it has been necessary to manually set the boundary frequency.

  An object of the present invention is to provide a technique for automatically determining an appropriate boundary frequency and improving the quality of synthesized speech.

  The boundary frequency is determined based on the distance between the spectral feature quantities before and after the boundary of the phoneme unit connected in accordance with the text, and the boundary frequency of the spectrum of the first synthesized speech obtained by the segment connection in accordance with the text is determined. A mixed spectrum is obtained by mixing the corresponding high-frequency component and the low-frequency component corresponding to the boundary frequency of the spectrum of the second synthesized speech obtained by applying an acoustic model for speech synthesis to the text.

  Thereby, an appropriate boundary frequency is automatically determined, and the quality of the synthesized speech can be improved.

FIG. 1 is a block diagram illustrating a speech synthesis apparatus according to an embodiment. FIG. 2A is a block diagram illustrating a boundary frequency determination unit according to the embodiment. FIG. 2B is a block diagram illustrating a waveform generation processing unit of the embodiment. FIG. 3 is a diagram for illustrating the speech synthesis method according to the embodiment. FIG. 4A is a flowchart for illustrating the boundary frequency determination method according to the embodiment. FIG. 4B is a conceptual diagram for illustrating the boundary frequency determination method according to the embodiment. FIG. 5 is a conceptual diagram for illustrating the waveform generation processing of the embodiment. FIG. 6 is a flowchart for illustrating the boundary frequency determination method of the embodiment. 7A and 7B are conceptual diagrams for illustrating the boundary frequency determination method according to the embodiment.

Embodiments of the present invention will be described below.
[Overview]
An overview of the embodiment will be described. In the embodiment, the boundary frequency is determined based on the distance between the spectral features before and after the segment boundary of the phoneme unit connected in accordance with the input “text”, and obtained by the segment connection in accordance with “text”. According to the boundary frequency of the spectrum of the second synthesized speech obtained by applying the “acoustic model” for speech synthesis to the “text”. A mixed spectrum obtained by mixing the low-frequency components is obtained. An example of the “acoustic model” is a probabilistic model such as an HMM. The higher frequency band than the boundary frequency may be “the high frequency side corresponding to the boundary frequency”, and the other band may be “the low frequency side corresponding to the boundary frequency”, or the high frequency above the boundary frequency May be “the high frequency side corresponding to the boundary frequency”, and the other band may be “the low frequency side corresponding to the boundary frequency”. The higher frequency band than the boundary frequency may be “a high frequency side corresponding to the boundary frequency”, and the lower frequency band than the boundary frequency may be “a low frequency side corresponding to the boundary frequency”. “A high frequency side corresponding to the boundary frequency” and “a low frequency side corresponding to the boundary frequency” may be determined with a frequency obtained by adding or subtracting a constant or a variable to or from the boundary frequency as a boundary. The band on the “high band side corresponding to the boundary frequency” and the band on the “low band side corresponding to the boundary frequency” may partially overlap. Here, since the boundary frequency is determined based on the distance between the spectral feature quantities before and after the segment boundary, setting of the boundary frequency can be automated, and the element occupied in the synthesized speech according to the magnitude of distortion at the phoneme connection portion. It is possible to adjust the band including the spectrum by the single connection type speech synthesis method. As a result, it is possible to improve the quality of the synthesized speech in consideration of both the generation of abnormal noise based on distortion at the phoneme connection portion and the improvement of natural feeling by introducing the unit connection type speech synthesis method.

  (A) a first method for determining a boundary frequency within a “predetermined frequency section” based on a distance of a spectrum feature amount according to the type of phoneme connected according to “text”; and (b) “predetermined” And select either the second method in which the frequency equal to or higher than the upper limit value of “frequency interval” is the boundary frequency, or (c) the third method in which the frequency equal to or lower than the lower limit value of “predetermined frequency interval” is the boundary frequency. Also good. As a result, an appropriate boundary frequency can be selected according to the type of phoneme connected, and the quality of the synthesized speech can be improved.

  For example, the distortion between the “present phoneme” that is each of the phonemes connected according to the “text” and the “preceding phoneme” immediately before the “present phoneme” has a great influence on the sound quality, and “ If the distortion between the “current phoneme” and the “subsequent phoneme” immediately after the “current phoneme” has little effect on the sound quality, the “first method” is selected to determine the boundary frequency for the “current phoneme”. . For example, when “present phoneme” represents a vowel or voiced consonant, “preceding phoneme” immediately before “present phoneme” represents a vowel or voiced consonant, and “succeeding phoneme” immediately after “present phoneme” represents an unvoiced consonant. , “First method” is selected for “current phoneme”. In the “first method” in these examples, the boundary frequency for the “current phoneme” is determined based on the distance of the spectral feature amount between the “current phoneme” and the “preceding phoneme” immediately before the “current phoneme”. As a result, both the generation of abnormal sounds based on the distortion between the “present phoneme” and the “preceding phoneme” and the improvement of the natural feeling through the introduction of the unit-connected speech synthesis method are supported. The quality of synthesized speech can be improved.

  For example, when the distortion between the “current phoneme” and the “succeeding phoneme” has a great influence on the sound quality, the “second method” is selected to determine the boundary frequency for the “current phoneme”. For example, when “present phoneme” represents a vowel or voiced consonant and “succeeding phoneme” immediately after “present phoneme” represents a vowel or voiced consonant, “second method” is selected for “present phoneme”. The As a result, the boundary frequency is increased, the spectrum of the unit-connected speech synthesis method that occupies the synthesized speech corresponding to the “present phoneme” is reduced, and the abnormal sound based on the distortion between the “present phoneme” and the “subsequent phoneme” is reduced. Suppresses the occurrence.

  For example, the distortion between “present phoneme” and “preceding phoneme” immediately before “present phoneme” has little effect on the sound quality, and “following phoneme” immediately after “present phoneme” and “present phoneme” When the effect of the distortion during the period on the sound quality is small, the “third method” is selected for the “current phoneme”. For example, when “present phoneme” represents a vowel or voiced consonant, “preceding phoneme” immediately before “present phoneme” and “succeeding phoneme” immediately following represent unvoiced consonant, and / or present phoneme represents unvoiced consonant. In this case, the “third method” is selected for the “current phoneme”. When the influence of distortion on the sound quality is small in this way, the boundary frequency is lowered, and the spectrum by the unit connection type speech synthesis method that occupies the synthesized speech corresponding to the “current phoneme” is increased. This improves the natural feeling of the synthesized speech while suppressing the generation of abnormal noise based on distortion.

  In the “first method”, the above-described “spectral feature amount distance” in the “first determination band” which is a partial band of the predetermined frequency section is obtained, and the “spectral feature amount” in the “first determination band” If the “distance” is less than the “allowable limit value (threshold)”, the frequency corresponding to the “first determination band” is set as the boundary frequency, and the “spectral feature distance” in the “first determination band” is “allowable”. If it is not less than the “limit value”, the same processing is executed again with the frequency band higher than the “first determination band” as the “second determination band” and the “second determination band” as the “first determination band”. May be. That is, the “spectral feature amount distance” is calculated sequentially from the “first determination band” on the low frequency side, and if the “spectral feature amount distance” is less than the “allowable limit value”, the “first determination” at that time A frequency corresponding to “band” may be used as the boundary frequency. Examples of the “frequency according to the first determination band” are the lower limit frequency of the “first determination band”, the upper limit frequency of the “first determination band”, or the center frequency of the “first determination band”. Examples of the “second determination band” include a band having the upper limit frequency of the “first determination band” or a frequency adjacent to the upper limit frequency as the lower limit frequency, or other frequencies higher than the upper limit frequency of the “first determination band”. Is a band having a lower limit frequency. An example of the initial value of the “first determination band” is a band whose lower limit is the lower limit frequency of a predetermined frequency section. From the viewpoint of human auditory characteristics, the higher the frequency, the smaller the influence that an abnormal sound based on the distortion at the connection part of adjacent phonemes has on the quality of the synthesized speech. Therefore, if the “spectral feature distance” in the “first determination band” is less than the “allowable limit value” and the distortion between phonemes is small, even in a band having a higher frequency than the “first determination band”. In many cases, the influence of abnormal sounds based on distortion between phonemes on the quality of synthesized speech is small. For this reason, the frequency corresponding to the “first determination band” in which the “spectral feature amount distance” is less than the “allowable limit value” is set as the boundary frequency, thereby suppressing the influence of abnormal noise based on the distortion and the synthesized speech. Can improve the natural feeling. Further, by repeating the above processing while increasing the frequency of the “first determination band” until the “spectral feature distance” becomes less than the “allowable limit value”, it is possible to suppress the influence of abnormal noise based on distortion. The lowest possible frequency can be used as the boundary frequency. As a result, it is possible to increase the spectrum of the unit connection type speech synthesis method that occupies the synthesized speech as much as possible and to improve the natural feeling of the synthesized speech while suppressing the influence of abnormal sounds based on distortion.

  The “allowable limit value” may be determined for each “first determination band” (for each frequency according to the first determination band), or may be uniform for all “first determination bands”. . When the “allowable limit value” is determined for each “first determination band”, the “allowable limit value” may be monotonously increased in a broad sense with respect to the frequency corresponding to the “first determination band”. For example, the “allowable limit value” may be larger as the frequency corresponding to the “first determination band” is higher. In this case, as the “allowable limit value” is larger, the frequency at which the “spectral feature distance” in the “first determination band” is less than the “allowable limit value” increases, and the frequency at which a lower frequency is selected as the boundary frequency. Becomes higher. On the other hand, the higher the frequency, the smaller the influence of abnormal sound on the quality of the synthesized speech due to distortion at the connection part of adjacent phonemes. Therefore, when the “allowable limit value” is in a monotonically increasing relationship with respect to the frequency corresponding to the “first determination band”, the unit-connected speech occupying the synthesized speech while suppressing the influence of abnormal noise based on distortion. The spectrum by the synthesis method can be increased as much as possible to improve the natural feeling of the synthesized speech.

  The “second method” and the “third method” are fundamental frequencies “obtained by applying the above-mentioned“ acoustic model ”to“ text ”with respect to the fundamental frequencies of phonemes connected in accordance with“ text ”. A method of determining the boundary frequency based on the “degree of change” may be used. The “boundary frequency” selected by the “second method” and the “third method” may be monotonically increasing in a broad sense with respect to the “change degree”. For example, in the “second method”, when the “degree of change” is within a predetermined range, the upper limit value of the “predetermined frequency section” is set as the boundary frequency, and otherwise, the upper limit value of the “predetermined frequency section” is set. It is good also considering the frequency (for example, Nyquist frequency) beyond as a boundary frequency. In the “third method”, when the “change degree” is within a predetermined range, a frequency less than the lower limit value of the “predetermined frequency section” (for example, 0 Hz) is set as the boundary frequency, and otherwise, the “predetermined frequency” The lower limit value of “section” may be the boundary frequency. In general, the greater the “change degree”, the lower the quality of synthesized speech obtained by the unit connection type speech synthesis method. Therefore, by determining the boundary frequency based on the “degree of change”, the quality of the synthesized speech can be improved.

[First Embodiment]
A first embodiment will be described.
<Configuration>
As illustrated in FIG. 1, the speech synthesizer 1 of this embodiment includes an input unit 11, a speech corpus storage unit 212, a speech database (DB) storage unit 122, an acoustic model storage unit 123, and a speech database (DB) construction unit 131. , An acoustic model generation unit 132, a unit connection type speech synthesis unit 133, an HMM speech synthesis unit 134, a boundary frequency determination unit 135, a spectrum mixing processing unit 136, and a waveform generation processing unit 137. As illustrated in FIG. 2A, the boundary frequency determination unit 135 of this embodiment includes a determination unit 1352. As illustrated in FIG. 2B, the spectrum mixing processing unit 136 of this embodiment includes a high-pass filter 1361, a low-pass filter 1362, and a mixing unit 1363. The speech synthesizer according to the embodiment includes, for example, a general purpose or dedicated processor including a processor (hardware processor) such as a central processing unit (CPU) and a memory such as random-access memory (RAM) and read-only memory (ROM). The computer is configured by executing a predetermined program. The computer may include a single processor and memory, or may include a plurality of processors and memory. This program may be installed in a computer, or may be recorded in a ROM or the like in advance. In addition, some or all of the processing units are configured using an electronic circuit that realizes a processing function without using a program, instead of an electronic circuit (circuitry) that realizes a functional configuration by reading a program like a CPU. May be. In addition, an electronic circuit constituting one device may include a plurality of CPUs.

<Pretreatment>
The speech corpus is stored in the speech corpus storage unit 121 as preprocessing. The speech DB constructing unit 131 uses the speech corpus stored in the speech corpus storage unit 121 to generate a speech DB that can be used for speech synthesis by the unit connection speech synthesis method. For example, the voice DB constructing unit 131 can generate a voice DB by using a known method described in Japanese Patent No. 2761552, Japanese Patent No. 4430960, and the like. The generated voice DB is stored in the voice DB storage unit 122. The acoustic model generation unit 132 generates an acoustic model for speech synthesis using the speech corpus stored in the speech corpus storage unit 121 as learning data. An example of the acoustic model is an acoustic model for HMM speech synthesis, and can be generated using a known method described in Non-Patent Document 1, for example. The acoustic model of the present embodiment models an HMM spectrum (frequency spectrum) and a fundamental frequency. The generated acoustic model is stored in the acoustic model storage unit 123.

<Speech synthesis processing>
Next, the speech synthesis process of this embodiment will be described with reference to FIG. The input unit 11 receives text representing a sentence that is a target of speech synthesis. The text is input to the unit connection type speech synthesis unit 133 and the HMM speech synthesis unit 134. The HMM speech synthesizer 134 applies text to the acoustic model stored in the acoustic model storage unit 123 (for example, applies a text analysis result of the text) to obtain and output an HMM spectrum, a fundamental frequency, and the like. The HMM spectrum, the fundamental frequency, and the like are sent to the spectrum mixing processing unit 136, and the fundamental frequency is further sent to the unit connection type speech synthesis unit 133 (step S134: HMM speech synthesis processing). The unit connection type speech synthesis unit 133 receives the text and the fundamental frequency as input, and performs unit connection (unit connection type speech synthesis) according to the text using the speech DB stored in the speech DB storage unit 122. , And obtain and output a segment spectrum (a spectrum of the first synthesized speech obtained by segment connection corresponding to the text) and a corresponding phoneme sequence with time information. The “phoneme sequence with time information” is a sequence of phonemes connected in segments, to which time information of each phoneme is given. The phoneme time information is, for example, information representing a position (for example, a phoneme start time or an end time) on the time axis of the phoneme with a predetermined time (for example, the start time of the phoneme sequence) as a reference. The unit spectrum and the corresponding phoneme sequence with time information are sent to the boundary frequency determination unit 135, and the unit spectrum is further sent to the spectrum mixing processing unit 136 (step S133: unit connection type speech synthesis process).

  The boundary frequency determination unit 135 receives a segment spectrum and a phoneme sequence with time information as input, and based on the distance between the spectral feature quantities before and after the segment boundary of the phonemes connected according to the text as described above The frequency B is determined and output (step S135: boundary frequency determination process).

<< Example of details of step S135 >>
The details of step S135 are illustrated using FIGS. 4A and 4B. In this embodiment, the boundary frequency is determined for each phoneme connected to the segment. In this embodiment, the calculation interval length T before and after the phoneme boundary T _i between the current phoneme (i-th phoneme) whose boundary frequency is to be determined and the preceding phoneme (i−1th phoneme) immediately before the current phoneme The “predetermined frequency section” in the time section is divided into N (for example, 10) bands (frequency bands) b ₁ ,..., B _N. However, T is a positive value, for example, 20 msec. N is a positive integer equal to or greater than 2, for example, T = 10. The upper limit value B _N and the lower limit value B ₀ of the “predetermined frequency section” are determined in advance. An example of the upper limit value B _N is the upper limit frequency (for example, 3 kHz) of the second formant of the vowel, and an example of the lower limit value B ₀ is the lower limit frequency (for example, 200 Hz) of the first formant of the vowel. The upper limit value of each band b _n is expressed as B _n . High frequency the larger the band _{b n} is n is in this embodiment, the upper limit _{B n'-1} band _{b n'-1} matches the lower limit of the band b _{n '.} The bands b ₁ ,..., B _N are preferably obtained by dividing the “predetermined frequency section” at equal intervals on the mel scale (mel frequency), but at equal intervals on the linear scale (linear frequency). It may be classified according to other criteria such as classified.

なお、スペクトル特徴量の例は、パワースペクトル、メルケプストラム係数、ケプストラム係数などである（ステップＳ１３５２ｂ）。 The determination unit 1352 (FIG. 2A) of the boundary frequency determination unit 135 initializes n: = 1. However, “n: = 1” means that n is 1 (1 is substituted for n) (step S1352a). The determination unit 1352 uses the average value S _n of the spectral feature amounts in the band b _n (the first determination band that is a part of a predetermined frequency section) in the time interval from T _i −T to T _i of the preceding phoneme. _{, I−1} and the average value S _{n, i} of the spectral feature values in the band b _n in the time interval from T _i to T _i + T of the current phoneme. For example, _let s _{n, i−1, t} be the spectral feature quantity at the discrete time t of the i−1 th phoneme (preceding phoneme) in the band b _n , and the i th phoneme (current phoneme) in the band b _n. Assuming that the spectral feature quantity at discrete time t is s _{n, i, t} , the following relationship holds.

Note that examples of the spectrum feature amount include a power spectrum, a mel cepstrum coefficient, a cepstrum coefficient, and the like (step S1352b).

Next, the determination unit 1352 calculates the distance D _{n, i} (the distance of the spectral feature amount in the first determination band b _n ) between S _{n, i−1} and S _{n, i} . When S _{n, i-1} and S _{n, i} are scalars, the distance D _{n, i} is the difference (absolute value) between S _{n, i -1} and S _{n, i} and S _{n, i-1} If S _{n, i} is a vector, the distance D _{n, i} is the norm of S _{n, i-1} and S _{n, i} (step S1352c). The determination unit 1352 compares the allowable limit value L _n and the distance D _{n, i} in the band b _n and determines whether L _n > D _{n, i} . However, the allowable limit value L _n is a value determined in advance by a listening experiment or the like. The allowable limit value L _n may be determined for each band b _n or may be uniform for all bands b ₁ ,..., B _N. When the permissible limit value L _n is determined for each band b _n , the permissible limit value L _n is in a monotonically increasing relationship with respect to a frequency (for example, B _n or B _n−1 ) corresponding to the band b _n. It is also possible (step S1352d). Here, if the distance D _{n, i} is less than the allowable limit value L _n (L _n > D _{n, i} ), the determination unit 1352 sets the frequency corresponding to the band b _n as the boundary frequency B. For example, B = _Bn may be set, B = _Bn-1 may be set, or B = ( _Bn-1 + _Bn ) / 2 may be set (step S1352g). On the other hand, if the distance D _{n, i} is not less than the allowable limit value L _n , the determination unit 1352 determines whether n = N (step S1352e). Here, if n = N is not satisfied, the determination unit 1352 sets n: = n + 1 (step S1352f), and executes the processing after step S1352b. In other words, the determination unit 1352 determines that the band b _{n + 1} ((b) having a frequency higher than that of the first determination band if the distance D _{n, i} of the spectral feature amount in the first determination band (band b _n ) is not less than the allowable limit value L _n. In this example, a process is performed in which one higher band is set as the second determination band, and the second determination band is set as the first determination band. On the other hand, if n = N, step S1352g is executed, and the frequency (for example, the upper limit value B _N ) corresponding to the band b _N is set as the boundary frequency B (step S1352g). The processes in steps S1352a to S1352g described above are executed with each phoneme connected as a current phoneme. By such processing, the boundary frequency B corresponding to each phoneme can be determined based on a quantitative criterion.

  The boundary frequency B output from the boundary frequency determination unit 135 is sent to the spectrum mixing processing unit 136. For each phoneme, the spectrum mixing processing unit 136 includes, for each phoneme, a high frequency component corresponding to the boundary frequency B of the input segment spectrum (the spectrum of the first synthesized speech obtained by segment connection corresponding to the text), the HMM A mixed spectrum is obtained by mixing low-frequency components corresponding to the boundary frequency B of the spectrum (the spectrum of the second synthesized speech obtained by applying an acoustic model for speech synthesis to text). In the example of FIG. 5, the segment spectrum and the boundary frequency B are input to the high pass filter 1361. The high-pass filter 1361 cuts a low-frequency part below the boundary frequency B of the segment spectrum by high-pass filter processing, and obtains and outputs a component on the high-frequency side of the segment spectrum. Further, the HMM spectrum and the boundary frequency B are input to the low pass filter 1362. The low-pass filter 1362 obtains and outputs a component on the low frequency side of the HMM spectrum by cutting the high frequency region above the boundary frequency B of the HMM spectrum by low-pass filter processing. The mixing unit 1363 receives the high-frequency component of the segment spectrum and the low-frequency component of the HMM spectrum as inputs, and mixes (synthesizes) these to obtain a mixed spectrum and output it. All of these processes are performed for each phoneme. The mixed spectrum is sent to the waveform generation processing unit 137 together with the fundamental frequency (step S136: spectrum mixing process).

  The waveform generation processing unit 137 generates and outputs a time-domain waveform (synthesized speech) corresponding to the mixed spectrum using the input mixed spectrum and fundamental frequency. For example, Reference 1 (H. Kawahara, “STRAIGHT, exploitation of the other aspect of VOCODER: Perceptually isomorphic decomposition of speech sounds”, Acoustic Science and Technology, Vol. 27, No. 6, pp.349 -353, 2006) can be used (step S137: waveform generation processing).

[Second Embodiment]
This embodiment is a modification of the first embodiment. In this embodiment, according to the type of phoneme connected according to the text, (a) a first method for determining a boundary frequency within a predetermined frequency interval based on the distance of the spectral feature amount, (b) the frequency interval Is selected from the second method in which the frequency equal to or higher than the upper limit value is set as the boundary frequency, or (c) the third method in which the frequency equal to or lower than the lower limit value of the frequency section is set as the boundary frequency. In the following, differences from the items already described will be mainly described, and for the items already described, the reference numbers used so far will be used and description thereof will be omitted.

<Configuration>
As illustrated in FIG. 1, the speech synthesizer 2 of this embodiment includes an input unit 11, a speech corpus storage unit 212, a speech DB storage unit 122, an acoustic model storage unit 123, a speech DB construction unit 131, and an acoustic model generation unit 132. , A unit connection type speech synthesis unit 133, an HMM speech synthesis unit 134, a boundary frequency determination unit 235, a spectrum mixing processing unit 136, and a waveform generation processing unit 137. As illustrated in FIG. 2A, the boundary frequency determination unit 235 according to the present embodiment includes a determination method selection unit 2351 and a determination unit 2352.

<Pretreatment>
The same as in the first embodiment.

<Speech synthesis processing>
The difference from the first embodiment is that the boundary frequency determination unit 235 performs the boundary frequency determination process in step S235 instead of the boundary frequency determination unit 135 performing the boundary frequency determination process in step S135. Others are as described in the first embodiment. Hereinafter, only the boundary frequency determination processing in step S235 will be described.

<< Step S235 >>
The boundary frequency determination unit 235 receives the segment spectrum and the phoneme sequence with time information as input, and based on the distance between the spectrum feature quantities before and after the segment boundary of the phonemes connected according to the text, as described above. The frequency B is determined and output (step S235: boundary frequency determination process). The process of step S235 of the present embodiment will be described using FIG.

  First, the determination method selection unit 2351 (FIG. 2A) of the boundary frequency determination unit 235 determines the type of phoneme connected by the phoneme sequence with time information (the combination of the type of the phoneme itself and the type of the adjacent phoneme). ), A method a (first method), a method b (second method), or a method c (third method) is selected for each phoneme based on the distance of the spectral feature amount (FIG. 7B). reference).

The method a is a method for determining the boundary frequency B in the “predetermined frequency section (section from the lower limit value B ₀ to the upper limit value B _N )”, and is the method described in the first embodiment. As described above, in the method a, based on the distance between the spectral feature amounts of “present phoneme” that is each of phonemes connected in units according to text and “preceding phoneme” immediately before “current phoneme”. The boundary frequency B for “current phoneme” is determined (for example, FIG. 4A). That is, the method a is not suitable for the unit-connected speech synthesis method when the distortion between the “present phoneme” and the “preceding phoneme” is large, but when the distortion between the “present phoneme” and the “preceding phoneme” is small. It is suitable for “present phonemes” to which the unit-connected speech synthesis method can be applied. The determination method selection unit 2351 may select the method a for such “current phoneme”.

Method b is a method for the boundary frequency B the upper limit B _N frequencies above the "predetermined frequency interval." The upper limit value of the boundary frequency B selected by the method b is, for example, the Nyquist frequency. In this case, the boundary frequency B between the upper limit value B _N and the Nyquist frequency is selected. The boundary frequency B selected by the method b is equal to or higher than the boundary frequency selected by the method a. That is, the method b is suitable for “present phonemes” that are basically unsuitable for the unit-connected speech synthesis method and “present phonemes” having such a relationship with adjacent phonemes. The determination method selection unit 2351 may select the method b for such “current phoneme”.

The system c is a system in which the boundary frequency B is a frequency equal to or lower than the lower limit B ₀ of the “predetermined frequency section”. The lower limit value of the boundary frequency B selected by the method c is, for example, 0 Hz. In this case, the boundary frequency B between ₀ Hz and the lower limit value B ₀ is selected. The boundary frequency B selected by the method c is equal to or lower than the boundary frequency selected by the method a. That is, the method c is suitable for the “present phoneme” that is basically suitable for the unit connection type speech synthesis method and the “present phoneme” having such a relationship with the adjacent phonemes. The determination method selection unit 2351 may select the method c for such “current phoneme”.

<Example 1 of method selection method>
A method for selecting a method is illustrated.
(A) The determination method selection unit 2351 has a large influence on the sound quality due to distortion between the “present phoneme” and the “preceding phoneme” immediately before the “present phoneme”, and the “present phoneme” and the “present phoneme”. If the distortion between the “subsequent phoneme” immediately after “has a little influence on the sound quality”, the method a is selected for the “current phoneme”.
(B) When the distortion between the “current phoneme” and the “subsequent phoneme” immediately after the “current phoneme” has a great influence on the sound quality, the determination method selection unit 2351 determines the method b for the “current phoneme”. Select.
(C) The determination method selection unit 2351 has less influence on the sound quality due to distortion between the “current phoneme” and the “preceding phoneme” immediately before the “current phoneme”, and the “current phoneme” and the “current phoneme”. The method c is selected for the “current phoneme” when the distortion between the “succeeding phoneme” immediately after “has a little influence on the sound quality”.

<Example 2 of method selection method>
A more specific method of selecting a method is illustrated (FIG. 7A).
(A) The determination method selection unit 2351 indicates that “present phoneme” represents a vowel or voiced consonant, “preceding phoneme” immediately before “current phoneme” represents vowel or voiced consonant, and “following” immediately after “present phoneme”. If “phoneme” represents an unvoiced consonant, method a is selected for “present phoneme”.
(B) When the “current phoneme” represents a vowel or a voiced consonant and the “succeeding phoneme” immediately after the “current phoneme” represents a vowel or a voiced consonant, Select scheme b.
(C) The determination method selection unit 2351 may be configured such that “present phoneme” represents a vowel or voiced consonant, “preceding phoneme” immediately before “current phoneme” and “succeeding phoneme” immediately after represent a voiceless consonant, and / or , “C” is selected for “present phoneme” when “present phoneme” represents an unvoiced consonant.

  By switching the boundary frequency determination method, it is possible to determine the optimum boundary frequency based on the phoneme type. Information for identifying the selected method, segment spectrum, and phoneme sequence with time information are sent to the determination unit 2352 (step S2351).

When information specifying the method a is sent to the determination unit 2352, the determination unit 2352 determines and outputs the boundary frequency B corresponding to the “current phoneme” by the process described in step S135 of the first embodiment (step S135). S2352a). When information specifying the method b is sent to the determination unit 2352, the determination unit 2352 determines and outputs a frequency (for example, a Nyquist frequency) equal to or higher than the upper limit value B _N as the boundary frequency B corresponding to the “current phoneme”. (Step S2532b). When the information specifying the method c is sent to the determination unit 2352, the determination unit 2352 determines and outputs a frequency (for example, 0 Hz) equal to or lower than the lower limit value B ₀ as the boundary frequency B corresponding to the “current phoneme”. (Step S2532c). The subsequent processing is the same as in the first embodiment.

[Third Embodiment]
This embodiment is a modification of the second embodiment. In this embodiment, the method b (second method) and the method c (third method) are based on the degree of change of the fundamental frequency of the “first synthesized speech” with respect to the fundamental frequency of phonemes connected in units according to text. To determine the boundary frequency B.

<Configuration>
As illustrated in FIG. 1, the speech synthesizer 3 of this embodiment includes an input unit 11, a speech corpus storage unit 212, a speech DB storage unit 122, an acoustic model storage unit 123, a speech DB construction unit 131, and an acoustic model generation unit 132. , A unit connection type speech synthesis unit 133, an HMM speech synthesis unit 134, a boundary frequency determination unit 335, a spectrum mixing processing unit 136, and a waveform generation processing unit 137. As illustrated in FIG. 2A, the boundary frequency determination unit 335 of this embodiment includes a determination method selection unit 2351 and a determination unit 3352.

<Pretreatment>
The same as in the first embodiment.

<Speech synthesis processing>
The difference from the second embodiment is that the boundary frequency determination unit 235 performs the boundary frequency determination process in step S335 instead of the boundary frequency determination unit 235 performing the boundary frequency determination process in step S235. The difference between step S335 and step S235 is that, instead of the determination unit 2352, the determination unit 3352 performs the following processing. Others are as described in the first and second embodiments. Only the processing of the determination unit 3352 will be described below.

  In this embodiment, information for specifying the method selected by the determination method selection unit 2351, a segment spectrum, and a phoneme sequence with time information are sent to the determination unit 3352. Further, the fundamental frequency output from the HMM speech synthesizer 134 in step S134 is also sent to the determination unit 3352. When information specifying the method a is sent to the determination unit 3352, the determination unit 3352 determines and outputs the boundary frequency B corresponding to the “current phoneme” by the process described in step S135 of the first embodiment (step S35). S2352a).

When information specifying the method b is sent to the determination unit 3352, the determination unit 3352 uses the average value F _0org of the fundamental frequency of the phoneme in the phoneme section of “current phoneme” and the HMM speech synthesis unit 134 in the phoneme section. It is determined _whether the ratio F _0org / F _0syn (the degree of change of the fundamental frequency) with the average value F _0syn of the fundamental frequencies of the synthesized sound exceeds the range of a predetermined value (for example, 0.5 or more and 2.0 or less) . The average value F _0syn of the fundamental frequency is determined based on the fundamental frequency output from the HMM speech synthesizer 134 (step S3352ba). When the ratio _{F 0org / F 0syn} is within a predetermined range of values, determining unit 3352 outputs as a boundary frequency B = _{B N} (step S3352bb). On the other hand, when the ratio F _0org / F 0 _syn is not within the predetermined value range, the determination unit 3352 determines and outputs the boundary frequency B in the above-described step S2352b (step S2352b).

Even when information specifying the method c is sent to the determination unit 3352, the determination unit 3352 determines _{whether the} ratio F _0org / F _0syn exceeds the predetermined value range (step S3352ca). When the ratio F _0org / F _0syn is within a predetermined value range, the determination unit 3352 determines and outputs the boundary frequency B in step S2352c described above (step S2352c). On the other hand, when the ratio F _0org / F _0syn is not within the range of the predetermined value, the determination unit 3352 outputs the boundary frequency B = B ₀ (step S2352cb). The subsequent processing is the same as in the first embodiment.

[Modification of Third Embodiment]
In the third embodiment, method b is selected and the ratio _{F 0org / F 0syn} is a B = _{B N} in the case of the range of a predetermined value, method b is selected and the ratio _{F 0org} / _{F The} Nyquist frequency was set to B when _0syn was not within a predetermined value range. However, when the method b is selected, the frequency from B _{N to the} Nyquist frequency may be set to B according to the ratio F _0org / F _0syn according to other criteria. For example, in the range from B _{N to the} Nyquist frequency, the frequency closer to B _N may be set to B as the ratio F _0org / F 0 _syn is _closer to a predetermined value (eg, 1).

In the third embodiment, when the method c is selected and the ratio F _0org / F 0 _syn is within a predetermined value range, B = 0 is set, the method c is selected, and the ratio F _0org / F _{0 syn} is selected. There was B = B ₀ if not within the predetermined range of values. However, when the method c is selected, a frequency of ₀ or more and B ₀ or less may be set to B according to the ratio F _0org / F _0syn according to other criteria. For example, in the range of ₀ or more and B ₀ or less, the frequency closer to 0 may be set to B as the ratio F _0org / F _0syn is _closer to a predetermined value (for example, 1).

In the third embodiment, the ratio F _0org / F 0 _syn is used as the “degree of change of the fundamental frequency”, but other indicators may be used instead. For example, the ratio F _0syn / F _0org , the difference | F _0org −F _0syn |, the square error {(F _0org ) ² − (F _0syn ) ² }, and the like may be used as the “basic frequency change degree”.

In the third embodiment, when the ratio F _0org / F _0syn exceeds the predetermined value range in step S3352ca, the boundary frequency B is set to the lower limit value B ₀ , but in this case, the boundary frequency B may be set as the Nyquist frequency. Good.

[Feature]
As described above, in each embodiment, the boundary frequency can be appropriately set according to a quantitative scale based on continuity between adjacent phonemes, the type of phoneme, the degree of change of the fundamental frequency, and the like. This makes it possible to generate synthesized speech that introduces the real voice feeling of the phoneme spectrum while suppressing the occurrence of abnormal sounds at the phoneme connection portion, taking advantage of the HMM speech synthesis method and the unit connection type speech synthesis method. .

  In each embodiment, the boundary frequency is quantitatively determined based on a preset allowable limit value, a phoneme type, a change degree of the fundamental frequency, and the like. Therefore, it is not necessary to manually determine the boundary frequency every time the speaker or the text to be synthesized is changed, and the boundary frequency can be automatically determined.

[Other variations]
The various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Needless to say, other modifications are possible without departing from the spirit of the present invention.

  When the above configuration is realized by a computer, the processing contents of the functions that each device should have are described by a program. By executing this program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such a recording medium are a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, and the like.

  This program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, this computer reads a program stored in its own recording device and executes a process according to the read program. As another execution form of the program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and each time the program is transferred from the server computer to the computer. The processing according to the received program may be executed sequentially. The above-described processing may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. Good.

  In the above embodiment, the processing functions of the apparatus are realized by executing a predetermined program on a computer. However, at least a part of these processing functions may be realized by hardware.

  1,2,3 Speech synthesizer

Claims (8)

A boundary frequency determination unit that determines a boundary frequency based on a distance between spectral feature amounts before and after a boundary between phoneme segments connected according to text;
A high frequency component corresponding to the boundary frequency of the spectrum of the first synthesized speech obtained by segment connection according to the text, and a second synthesis obtained by applying an acoustic model for speech synthesis to the text A spectrum mixing processing unit for obtaining a mixed spectrum obtained by mixing a low-frequency component corresponding to the boundary frequency of the spectrum of speech;
A speech synthesizer. The speech synthesizer of claim 1,
The boundary frequency determination unit
(A) a first method for determining the boundary frequency within a predetermined frequency section based on the distance of the spectral feature amount, according to the type of phoneme connected in units according to the text; and (b) the frequency section. A speech synthesizer that selects either the second method in which a frequency equal to or higher than the upper limit value is used as the boundary frequency, or (c) the third method in which a frequency equal to or lower than the lower limit value of the frequency section is used as the boundary frequency. The speech synthesizer according to claim 2,
In the second method and the third method, the boundary frequency is based on a change degree of a fundamental frequency obtained by applying the acoustic model to the text with respect to a fundamental frequency of phonemes connected in units according to the text. A speech synthesizer. The speech synthesizer according to claim 2 or 3,
In the first method, the boundary frequency for the current phoneme is based on a distance of the spectral feature amount between a current phoneme that is a phoneme connected in units according to the text and a preceding phoneme immediately before the current phoneme. Decide
The boundary frequency determination unit
The distortion between the current phoneme and the preceding phoneme immediately before the current phoneme has a large effect on the sound quality, and the distortion between the current phoneme and the subsequent phoneme immediately after the current phoneme has an effect on the sound quality. If so, select the first method for the current phoneme;
When the distortion between the current phoneme and the subsequent phoneme immediately after the current phoneme has a great influence on the sound quality, the second method is selected for the current phoneme,
The effect of distortion between the current phoneme and the preceding phoneme immediately before the current phoneme on the sound quality is small, and the distortion between the current phoneme and the subsequent phoneme immediately after the current phoneme has an effect on the sound quality. A speech synthesizer that, when smaller, selects the third method for the current phoneme. The speech synthesizer according to any one of claims 1 to 4,
The boundary frequency determination unit
Obtain the distance of spectral features of the first determination zone, which is part of a band of a predetermined frequency interval, the distance of the spectral feature amount in the first determination band is less than the allowable limit, the If the frequency corresponding to the first determination band is the boundary frequency, and the distance of the spectral feature amount in the first determination band is not less than the allowable limit value, a band having a frequency higher than the first determination band is A speech synthesizer that performs processing with the second determination band as the first determination band. The speech synthesizer according to claim 5,
The permissible limit value is a speech synthesizer that is monotonically increasing in a broad sense with respect to a frequency corresponding to the first determination band. A boundary frequency determination step for determining a boundary frequency based on a distance between spectral feature amounts before and after the boundary between phoneme segments connected according to text;
A high frequency component corresponding to the boundary frequency of the spectrum of the first synthesized speech obtained by segment connection according to the text, and a second synthesis obtained by applying an acoustic model for speech synthesis to the text A spectral mixing processing step for obtaining a mixed spectrum obtained by mixing a low-frequency side component corresponding to the boundary frequency of the voice spectrum;
A speech synthesis method comprising:   A program for causing a computer to function as the speech synthesizer according to claim 1.

Images