JP2007052456A - Method and system for generating dictionary for speech synthesis - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce the amount of calculations and memory capacity necessary for processing to reduce the "fuzziness" of the spectra of the speech by a window function for obtaining a fine phonemic piece and to realize speech synthesis of high sound quality with fewer computer resources. <P>SOLUTION: In the method for generating the dictionary to be used for speech synthesis processing, an alternative filter is first generated by approximating a correction filter for spectral correction to be obtained based on the speech waveform data by realizing the same in the amount of data or the amount of calculation smaller than that of the correction filter. Corrected waveform data is generated by acting a filter compensating the difference between the correction filter and the alternative filter on the speech waveform data, and the alternative filter and the corrected waveform data are stored as a part of the dictionary. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

The present invention relates to a method and apparatus for generating a speech synthesis dictionary for a speech synthesizer that synthesizes speech.

  Conventionally, as a speech synthesis method for obtaining a desired synthesized speech, a speech segment recorded and stored in advance is divided into a plurality of fine segments, and the fine segments obtained as a result of the division are rearranged. There is a method for obtaining a desired synthesized speech. In the rearrangement of these fine segments, synthetic speech having a desired time length and fundamental frequency is obtained by performing processing such as interval change, repetition, and thinning on the fine segments.

  FIG. 8 is a diagram schematically showing a method of dividing a speech waveform into fine segments. The speech waveform shown in FIG. 8 is divided into fine segments by a cutout window function (hereinafter referred to as a window function). At this time, a window function synchronized with the pitch interval of the original speech is used in the voiced portion (second half of the speech waveform). On the other hand, a window function with an appropriate interval is used in the unvoiced sound part.

  Then, as shown in FIG. 8, the duration of the voice can be shortened by thinning and using these fine pieces. On the other hand, if these fine segments are used repeatedly, the voice duration can be extended. Furthermore, as shown in FIG. 8, in the voiced sound part, it is possible to increase the fundamental frequency of the synthesized speech by narrowing the interval between the fine segments. On the other hand, it is possible to lower the fundamental frequency of the synthesized speech by increasing the interval between the fine segments.

  The desired synthesized speech can be obtained by superimposing the re-arranged fine segments again after repeating, thinning, and changing the interval as described above. Note that a unit such as a phoneme, CV / VC, or VCV is used as a unit for recording / accumulating the speech element. CV · VC is a unit in which a segment boundary is placed in a phoneme, and VCV is a unit in which a segment boundary is placed in a vowel.

  However, in the above-described conventional method, so-called “blurring” occurs in the spectrum of the voice by applying a window function to obtain fine segments from the voice waveform. That is, a phenomenon such as the formant of the voice spreading or the peaks and valleys of the spectrum envelope becoming ambiguous occurs, and the sound quality of the synthesized voice is deteriorated.

The present invention has been made in view of the above-described problems, and is designed to reduce speech “blurring” due to a window function applied to obtain a fine segment and to achieve high-quality speech synthesis. The purpose is to be able to provide a dictionary .

It is a further object of the present invention to provide a speech synthesis dictionary for reducing “blurring” of a speech spectrum so that high-quality speech synthesis can be realized with few hardware resources.

In order to achieve the above object, a dictionary generation method for speech synthesis according to the present invention includes:
  A method for generating a dictionary used for speech synthesis processing,
  A first generation step of generating a substitute filter by approximating a correction filter for spectrum correction obtained based on speech waveform data so as to be realized with a smaller data amount or calculation amount than the correction filter;
  A second generation step of generating corrected waveform data by causing a filter that compensates for a difference between the correction filter and the alternative filter to act on the speech waveform data;
  A storage step of storing the substitute filter generated in the first generation step and the modified waveform data generated in the second generation step as a part of the dictionary.

In order to achieve the above object, a dictionary generating apparatus for speech synthesis according to the present invention includes:
  An apparatus for generating a dictionary used for speech synthesis processing,
  First generating means for generating an alternative filter by approximating a correction filter for spectrum correction obtained based on speech waveform data so as to realize a data amount or calculation amount smaller than the correction filter;
  Second generation means for generating corrected waveform data by causing a filter that compensates for the difference between the correction filter and the alternative filter to act on the speech waveform data;
  An alternative filter generated by the first generation means, and storage means for storing the modified waveform data generated by the second generation means as a part of the dictionary.

According to the present invention, there is also provided a control program for causing a computer to execute the speech synthesis dictionary generation method.

According to the present invention described above , it is possible to reduce the amount of computation and storage capacity necessary for processing for reducing the “blurring” of the spectrum of speech due to the window function applied to obtain the fine segment, and to reduce the number of computers A speech synthesis dictionary for realizing speech synthesis with high sound quality using resources can be created .

  Hereinafter, some preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<First Embodiment>
In the Japanese Patent Application No. 2002-164624, the present applicant applies the spectrum correction filter to the fine element shown in FIG. 8 to correct the spectrum of the fine element, thereby correcting the above-described “blurring” of the audio spectrum. An improved speech synthesizer and method were proposed. This alleviates the decrease in the spread of speech formants and the ambiguity of the peaks and valleys in the spectral envelope caused by the application of a window function to obtain fine segments from the speech waveform, and the sound quality of the synthesized speech. It prevents the decline.

  FIG. 9 is a diagram schematically illustrating a method of applying the spectrum correction filter. By applying the corresponding spectrum correction filter 907 to each of the fine segments 903 cut out from the speech waveform 901 by the window function 902, the spectrum-corrected fine segments 904 (for example, fine segments whose formants have been corrected). Get. Then, a synthesized speech 906 is generated using the spectrally corrected fine segment 904.

  Here, the spectrum correction filter is obtained by acoustic analysis, and specific examples of the spectrum correction filter 907 applicable to the above processing include the following three filters.

  (1) First, when p-order linear prediction analysis is used for acoustic analysis, a filter having the characteristic expressed by the following [Equation 1] can be used as the spectrum correction filter 907.

  (2) In addition, when p-order cepstrum analysis is used for acoustic analysis, a filter having the characteristic expressed by the following [Equation 2] can be used as a spectrum correction filter.

  (3) Alternatively, it is also possible to use an FIR filter represented by the following [Equation 3], which is configured by cutting off the impulse response of the filter with an appropriate order.

  In the above equations, p is the analysis order, μ and γ are appropriate coefficients, α is a linear prediction coefficient, and c is a cepstrum coefficient. Β is an FIR filter coefficient obtained from the impulse response of the filter expressed by [Equation 1] and [Equation 2].

  Now, the calculation of the spectrum correction filter requires a product-sum operation at least about 10 to several tens of times per waveform sample. This is very large with respect to the calculation amount of the basic process of speech synthesis (the process shown in FIG. 8). Further, since the coefficient of the correction filter is usually obtained when the speech synthesis dictionary is created, a storage area for holding the correction filter coefficient is also required. That is, the size of the speech synthesis dictionary is enlarged.

  Of course, if the filter order p and the FIR filter order p ′ are reduced, the calculation amount and the storage capacity can be reduced. Alternatively, the storage capacity necessary to hold the spectrum correction filter coefficients can be reduced by clustering the spectrum correction filter coefficients. However, in this case, the effect of spectrum correction is diminished and the sound quality is degraded. Therefore, in the embodiment described below, the amount of calculation / storage capacity necessary for spectrum correction filtering is reduced, and the increase in the amount of calculation / storage capacity is suppressed, while the “blurring” of the voice spectrum is reduced, and high sound quality is achieved. Realize voice synthesis.

  In the first embodiment, an approximate filter with a reduced filter order is used to reduce the amount of calculation and storage capacity, and the waveform data in the speech synthesis dictionary is modified so as to be suitable for the approximate filter. To maintain the quality.

  FIG. 1 is a block diagram showing a hardware configuration in the first embodiment. In FIG. 1, reference numeral 11 denotes a central processing unit which performs processing such as numerical calculation and control. In particular, the central processing unit 11 performs speech synthesis processing according to the procedure described below. An output device 12 presents various information to the user under the control of the central processing unit 11. An input device 13 includes a touch panel or a keyboard, and is used by a user to give an operation instruction to the device and to input various information. Reference numeral 14 denotes an audio output device that outputs audio, and outputs the synthesized content.

  Reference numeral 15 denotes a storage device such as a disk device or a nonvolatile memory, which holds a speech synthesis dictionary 501 and the like. The speech synthesis dictionary 501 stores modified waveform data obtained by modifying a speech waveform by a method described later, and a spectrum correction filter approximated by a method described later. Reference numeral 16 denotes a read-only storage device, which stores the speech synthesis processing procedure of the present embodiment and necessary fixed data. Reference numeral 17 denotes a storage device that holds temporary information such as a RAM, which holds temporary data, various flags, and the like. Each of the above components (11 to 17) is connected by a bus 18. In the present embodiment, a control program for speech synthesis processing is stored in the ROM 16, and the central processing unit 11 executes this. However, such a control program is stored in the external storage device 15, It may be configured to be loaded into the RAM 17 upon execution.

  The operation of the audio output apparatus according to this embodiment having the above-described configuration will be described below with reference to FIGS. 2 and 3 are flowcharts for explaining audio output processing according to the first embodiment. FIG. 4 is a diagram showing the state of speech synthesis processing according to the first embodiment.

  In this embodiment, the spectrum correction filter is configured prior to speech synthesis, and configuration information (filter coefficients) for configuring the filter is held in a predetermined storage area (speech synthesis dictionary). Yes. That is, there are two processes: data creation processing (FIG. 2) and speech synthesis processing (FIG. 3) for creating a speech synthesis dictionary. Here, in the data creation process, the approximation of the spectrum correction filter is adopted to reduce the amount of configuration information, and the speech waveform of the speech synthesis dictionary is modified to prevent deterioration of the synthesized speech due to the approximation of the spectrum correction filter. To do.

  First, in step S1, waveform data (speech waveform 301 in FIG. 4) that is a source of synthesized speech is acquired. In step S2, acoustic analysis such as linear prediction (LPC) analysis, cepstrum analysis, and generalized cepstrum analysis is performed on the waveform data acquired in step S1, and parameters necessary for configuring the spectrum correction filter 310 are calculated. . The analysis of the waveform data may be performed at a predetermined time interval, or pitch synchronization analysis may be performed.

  Next, in the spectrum correction filter configuration step S3, the spectrum correction filter 310 is configured using the parameters calculated in step S2. For example, when p-order linear prediction analysis is used for acoustic analysis, a filter having the characteristic expressed by the above [Equation 1] is used as the spectrum correction filter 310. When p-th order cepstrum analysis is used, a filter having the characteristic expressed by [Equation 2] is used as the spectrum correction filter 310. Alternatively, the FIR filter represented by [Equation 3] configured by cutting off the impulse response of the filter with an appropriate order may be used as the spectrum correction filter 310. Actually, it is necessary to consider the system gain in each of the above equations.

  Next, in step S4, the spectrum correction filter 310 configured in step S3 is simplified by approximation to configure an approximate spectrum correction filter 306 that can be realized with a smaller amount of calculation and storage. As a simple example of the approximate spectrum correction filter 306, a filter in which the order of truncation of the FIR filter expressed by the above [Equation 3] is limited to a low order can be considered. Alternatively, the approximate correction filter can be configured by defining a difference in frequency characteristics with the spectrum correction filter as a distance in the spectrum region and obtaining a filter coefficient that minimizes the difference by the Newton method or the like.

  Next, in step S5, the approximate spectrum correction filter 306 configured in step S4 is recorded in the speech synthesis dictionary 501 (actually, the coefficients of the approximate spectrum correction filter are stored).

  In the next steps S6 to S8, the speech waveform data is corrected in order to reduce the sound quality degradation when the approximate spectrum correction filter configured in steps S4 and S5 and recorded in the speech synthesis dictionary 501 is applied to the speech waveform. To the speech waveform dictionary 501.

  First, in step S <b> 6, an inverse filter of the spectrum correction filter 310 and the approximate spectrum correction filter 306 is synthesized to configure the approximate correction filter 302. For example, when the filter represented by [Equation 1] is used as the spectrum correction filter and the low-order FIR filter represented by [Equation 3] is used as the approximate spectrum correction filter, the approximate correction filter is expressed by the following [Equation 4]. become that way.

  Next, in step S7, the approximate correction filter 302 is applied to the speech waveform data obtained in step S1, thereby creating a modified speech waveform 303. In step S8, the modified speech waveform obtained in step S7 is recorded in the speech synthesis dictionary 501.

  The above is the data creation process. Next, the speech synthesis process will be described with reference to the flowchart of FIG. In the speech synthesis process, the approximate spectrum correction filter 306 and the modified speech waveform 303 registered in the speech synthesis dictionary 501 by the data creation process are used.

  First, in the prosodic target value acquisition step S9, the target prosodic value of the synthesized speech is acquired. The target prosodic value of the synthesized speech may be given directly from the upper module as in the case of singing voice synthesis, or may be estimated using some means. For example, if it is speech synthesis from text, it is estimated from the language analysis result of the text.

  Next, in step S10, the modified speech waveform recorded in the speech synthesis dictionary 501 is acquired based on the target prosodic value acquired in step S9. In step S11, the approximate spectrum correction filter recorded in the speech synthesis dictionary 501 in step S5 is read. Note that the approximate spectrum correction filter that is read is an approximate spectrum correction filter corresponding to the modified speech waveform acquired in step S10.

  Next, in step S12, the window function 304 is applied to the modified speech waveform acquired in step S10, and the fine segment 305 is cut out. A Hanning window or the like is used as the window function. Next, in step S13, the approximate spectrum correction filter 306 read in step S11 is applied to each of the fine pieces 305 cut out in step S12 to correct the spectrum of the fine pieces 305. Thus, the spectrally corrected fine segment 307 is obtained.

  Next, in step S14, the fine segment 307 whose spectrum has been corrected in step S13 is rearranged (308) by thinning, repeating, and changing the interval so as to match the prosodic target value acquired in step S9. Change prosody. In step S15, the fine segments rearranged in step S14 are superimposed to obtain a synthesized speech 309 (speech segment). Thereafter, in step S16, the synthesized speech 309 (speech unit) obtained in step S15 is connected to obtain synthesized speech and output the speech.

  As for the rearrangement processing of the fine pieces, “decimation” may be executed before the approximate spectrum correction filter 306 is operated as shown in FIG. This is because it is possible to omit a useless process of performing filter processing on unnecessary fine pieces.

Second Embodiment
In the first embodiment, the example in which the order of the filter coefficient is reduced by approximation to reduce the calculation amount and the storage capacity has been described. In the second embodiment, a case where the storage capacity is reduced by clustering the spectrum correction filter will be described. The process of the second embodiment includes three processes: clustering processing (FIG. 5), data creation processing (FIG. 6), and speech synthesis processing (FIG. 7). The apparatus configuration for realizing this processing is the same as that of the first embodiment (FIG. 1).

  In the flowchart of FIG. 5, steps S1, S2, and S3 are processes constituting a spectrum correction filter, and are the same as those in the first embodiment (FIG. 2). These processes are performed on all waveform data included in the speech synthesis dictionary 501 (step S100).

  When the spectrum correction filters are configured for all waveform data, the process proceeds to step S101, and the spectrum correction filters obtained in step S3 are clustered. As the clustering, for example, a method called LBG algorithm can be applied. In step S102, the clustering result (clustering information) in step S101 is recorded in the external storage device 15. Specifically, a correspondence table of representative vectors (filter coefficients) and cluster numbers of each cluster is created and recorded. The representative vector constitutes a spectrum correction filter (representative filter) of the cluster. In this embodiment, a spectrum correction filter is configured for each waveform data registered in the speech synthesis dictionary 501 in step S3, and the coefficient of the spectrum correction filter corresponding to each waveform data is used as the cluster number for the speech synthesis dictionary 501. Hold in. That is, as will be described later with reference to FIG. 6, the speech synthesis dictionary 501 of the second embodiment includes waveform data of each speech waveform (correctly speech waveform data (described later with reference to FIG. 6)) and a cluster of spectrum correction filters. The number, each cluster number, and the representative vector (representative value of each coefficient) are registered.

  Next, the dictionary creation process (FIG. 6) will be described. In the dictionary creation process, the spectrum filter configuration process in steps S1 to S3 is the same as in the first embodiment. The difference from the first embodiment is that, instead of configuring the approximate spectral correction filter, the filter coefficient of the spectral correction filter is vector-quantized and registered by the cluster number. That is, first, in step S103, a vector closest to the spectrum correction filter obtained in step S3 is selected from the representative vector of the clustering information recorded in step S102. Next, in step S104, the number (cluster number) corresponding to the representative vector selected in step S103 is recorded in the speech synthesis dictionary 501.

  Furthermore, a modified speech waveform is generated and registered in the speech synthesis dictionary in order to reduce deterioration of the synthesized speech caused by quantizing the filter coefficient of the spectrum correction filter. That is, in step S105, a quantization error correction filter for correcting the quantization error is configured. The quantization error correction filter is configured by synthesizing an inverse filter of a filter configured using the representative vector and a spectrum correction filter of the speech waveform. For example, when the filter represented by [Equation 1] is used as the spectrum correction filter, the quantization error correction filter is represented by [Equation 5].

  In Equation 5, α ′ is a vector-quantized linear prediction coefficient. A quantization error correction filter can be configured similarly when other types of filters are used. The waveform data is corrected using the quantization error correction filter configured in this way to create a corrected speech waveform (step S7), and the obtained modified speech waveform is registered in the speech synthesis dictionary 501 (step S8). Since the spectrum correction filter is registered by the cluster number and the correspondence table (cluster information), the storage capacity required for the speech synthesis dictionary can be reduced.

  At the time of speech synthesis, as shown in the flowchart of FIG. 7, step S11 (step of reading the approximate spectrum correction filter) in the processing of the first embodiment is not necessary, and instead, step S106 (spectrum correction filter number (cluster number) ) And step S107 (processing for obtaining a spectrum correction filter from the read cluster number) are added.

  Similar to the first embodiment, the prosodic target value is acquired (step S9), and the modified speech waveform data registered in step S8 of FIG. 6 is acquired (step S10). In step S106, the spectrum correction filter number recorded in step S104 is read. Next, in step S107, a spectrum correction filter corresponding to the spectrum correction filter number is acquired based on the correspondence table recorded in step S102. Thereafter, the synthesized speech is output in steps S12 to S16 as in the first embodiment. That is, a fine unit is cut out by applying a window function to the modified speech waveform (step S12), and the spectrally corrected fine unit is obtained by applying the spectrum correction filter obtained in step S107 to the cut out fine unit. (Step S13), the fine segments whose spectrum has been corrected according to the prosodic target value are rearranged (Step S14), and the rearranged fine segments are superimposed to obtain a synthesized speech 309 (speech segment) (Step S15). .

  As described above, even if the spectrum correction filter is quantized by the clustering, it becomes possible to correct the quantization error by using the modified speech waveform corrected by the filter as shown in [Equation 5]. It is possible to reduce the storage capacity without impairing the storage capacity.

<Other embodiments>
In each of the above embodiments, when the waveform sampling frequency is high, band division may be performed by a band division filter, and spectrum correction filtering may be performed on each band-limited waveform. In this case, a filter is provided for each band, the target speech waveform itself is also divided into bands, and processing is performed for each waveform. The order of the spectrum correction filter is suppressed by the band division, and the calculation amount is reduced. The same effect can be obtained by expansion and contraction of the frequency axis such as a mel cepstrum.
An embodiment in which the first and second embodiments are combined is also possible. In this case, after the spectral correction filter before approximation is vector quantized, the filter based on the representative vector may be approximated, or the coefficient of the approximate spectral correction filter may be vector quantized.
In the second embodiment, the result of acoustic analysis may be converted once, and the converted vector may be vector quantized. For example, when a linear prediction coefficient is used for acoustic analysis, the linear prediction coefficient is not directly vector quantized, but is converted into an LSP coefficient, and the LSP coefficient is quantized. When configuring the spectrum correction filter, the quantized LSP coefficient can be inversely converted into a linear prediction coefficient and used. In general, since the LSP coefficient has better quantization characteristics than the linear prediction coefficient, more appropriate vector quantization is possible.

  Another object of the present invention is to supply a storage medium storing software program codes for implementing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in the.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a block diagram which shows the hardware constitutions in 1st Embodiment. It is a flowchart explaining the approximate spectrum correction filter registration process in the audio | voice output process by 1st Embodiment. It is a flowchart explaining the speech synthesis process in the audio | voice output process by 1st Embodiment. It is a figure showing the mode of the speech synthesis process of 1st Embodiment. It is a flowchart explaining the clustering process in the audio | voice output process by 2nd Embodiment. It is a flowchart explaining the spectrum correction filter registration process in the audio | voice output process by 2nd Embodiment. It is a flowchart explaining the speech synthesis process in the speech output process by 2nd Embodiment. It is the figure which showed typically the speech synthesis method by the division | segmentation, rearrangement, and the synthesis | combination of the audio | voice waveform into the fine fragment. It is the figure which showed typically the method of using a spectrum correction in the speech synthesis method by the division | segmentation, rearrangement, and the synthesis | combination of an audio | voice waveform into a fine segment.

Claims (7)

A method for generating a dictionary used for speech synthesis processing,
A first generation step of generating a substitute filter by approximating a correction filter for spectrum correction obtained based on speech waveform data so as to be realized with a smaller data amount or calculation amount than the correction filter;
A second generation step of generating corrected waveform data by causing a filter that compensates for a difference between the correction filter and the alternative filter to act on the speech waveform data;
A dictionary generation method for speech synthesis, comprising: a storage step of storing the substitute filter generated in the first generation step and the modified waveform data generated in the second generation step as a part of the dictionary.   The method of generating a dictionary for speech synthesis according to claim 1, wherein the alternative filter is a filter obtained by generating an FIR filter from an impulse response of the correction filter and cutting the FIR filter at a low order. .   The method for generating a dictionary for speech synthesis according to claim 1, wherein the substitution filter has a lower order than the correction filter.   The speech synthesis dictionary generation method according to claim 1, wherein the alternative filter is a filter obtained by vector quantization of filter coefficients of the correction filter. An apparatus for generating a dictionary used for speech synthesis processing,
First generating means for generating an alternative filter by approximating a correction filter for spectrum correction obtained based on speech waveform data so as to realize a data amount or calculation amount smaller than the correction filter;
Second generation means for generating corrected waveform data by causing a filter that compensates for the difference between the correction filter and the alternative filter to act on the speech waveform data;
A speech synthesis dictionary generation apparatus comprising: a substitute filter generated by the first generation means; and storage means for storing the modified waveform data generated by the second generation means as a part of the dictionary.   A control program for causing a computer to execute the speech synthesis dictionary generation method according to claim 1.   A storage medium storing the control program according to claim 6.

Images