JP6410890B2 - Speech synthesis apparatus, speech synthesis method, and speech synthesis program - Google Patents

Description

  The present invention relates to a speech synthesizer that synthesizes speech waveforms based on input time-series sound source control information and spectral characteristic information.

  Speech synthesis technology is a general term for a series of technologies for synthesizing speech waveforms from text. First, we explain the process of synthesizing speech waveforms from the spectrum information and sound source information of the speech that you want to synthesize. To do. In this process, the spectrum information and sound source information of the voice to be synthesized are obtained in advance from the corresponding natural voice or the like.

  A typical speech synthesis waveform synthesis method is a speech synthesis method based on a source filter model. This method is a method of first synthesizing a speech waveform having a desired characteristic by generating an appropriate sound source (source) waveform and passing it through a filter having an appropriate characteristic. For example, if it is considered that the sound source corresponds to the glottal volume flow accompanying vocal cord vibration and the filter corresponds to the vocal tract transfer characteristics, it can be said that the model corresponds to the human voice generation process.

  However, what can be observed from the speech waveform is the physical quantity for the final speech waveform, such as the spectral characteristics of the speech waveform and the fundamental frequency observed in the periodic speech waveform. Matching is difficult. Therefore, in practice, a speech waveform is often synthesized by directly giving a spectral characteristic of speech as a synthesis target by a filter to a spectrally white sound source waveform such as an impulse train or white noise.

  When the speech waveform has periodicity, the spectrum information to be observed includes a fundamental frequency component and its harmonic component derived from the periodicity. Normally, this periodicity is expressed on the sound source side by an impulse train or the like.

  Hereinafter, the spectrum information refers to smoothed spectrum information excluding the influence of the fundamental frequency and its harmonic components. The smoothing method includes a method of connecting only the peak points of the harmonic components on the frequency axis. In addition, although the sound waveform can be regarded as almost steady in a short time, it is time-varying over a long time. Therefore, in general, a characteristic at every certain interval (for example, about 1 to 20 milliseconds) is considered. The stationarity is assumed at each time. Here, the spectrum information of each sample is expressed by, for example, a multi-order mel cepstrum coefficient or a linear prediction coefficient.

  In general, voice with vocal cord vibration is called voiced sound, and voice without voice is called unvoiced sound. In voiced sound, waveform periodicity is usually observed. In speech waveform synthesis based on a source filter, a method is often used in which only an impulse train is used as a voiced sound source and only white noise is used as an unvoiced sound source. Although this method often has no problem in terms of linguistic intelligibility of synthesized speech, the actual voiced sound also includes a noisy component, resulting in a problem that its naturalness is lowered.

  Therefore, a method has been developed that improves the naturalness of synthesized speech by simultaneously generating an impulse train and white noise and using the combined waveform as a sound source waveform. However, usually, the optimum impulse to noise power ratio is not constant in each frequency band, and it differs depending on the type of speech to be synthesized. Therefore, it is necessary to change the amplitude characteristics of the impulse and white noise for each frequency band (subband) using a filter bank or the like.

  At this time, considering the correspondence with the conventional source / filter model, a method of controlling so that the result of adding the sound sources to white is often used. Hereinafter, such a sound source is referred to as a multiband mixed excitation source. Although it is considered that a certain degree of naturalness can be obtained without changing the mixing ratio for each subband, it is possible to synthesize speech with higher naturalness by changing the time as in the spectral information.

  Therefore, for speech synthesis, speech spectral information, voiced / unvoiced information, fundamental frequency information about voiced, and multiband mixed excitation sources are used for each interval on the time axis, and the characteristics are dynamically changed. Information on the mixing ratio of each subband in the case of changing is required. In the form of speech synthesis described below, for convenience of explanation, it is assumed that the power of the sound source is always constant and the power of the synthesized speech is controlled by being included in the spectrum characteristics.

Sei Imai, Kazuo Sumita, Chieko Furuichi, "Mel Log Spectrum Approximation (MLSA) Filter for Speech Synthesis", IEICE Transactions (A), J66-A, 2, Feb.1983, pp.122-129

  In the prior art as described above, a filter having a relatively large calculation amount such as an MLSA (Mel logarithmic spectrum approximation) filter is used as a filter of the source filter model (see Non-Patent Document 1). The MLSA filter employs a technique of constructing a circuit that approximately realizes a target characteristic by directly approximating an exponential function in a z-transform region by a rational approximation on the z-transform region by Padé approximation. Although there is an advantage that the mel cepstrum coefficient can be used as it is as a filter coefficient, the number of product-sum operations per sample of the waveform is approximately the product of the order of the filter and the order of the Padé approximation, and the amount of calculation is relatively large.

  For example, in terms of synthesized speech quality, it is necessary to use a 30-40th order mel cepstrum at the time of 16 kHz sampling. In that case, in order to approximate the exponential function with the required accuracy, a fourth order or fifth order Padé approximation is required. That is, a product-sum operation is required about 150 to 200 times per sample.

  Further, when performing multi-band mixing excitation, it is necessary to filter the impulse train and the white noise so that the specified mixing ratio is obtained, so that the amount of calculation further increases for each filtering process. For this reason, it is difficult to perform speech synthesis processing using a higher-order filter or mixed excitation in an environment where the calculation processing performance of a mobile terminal or the like is limited.

  In order to solve this problem, a method of performing subband encoding including sample rate reduction on a sound source waveform of an impulse train or a white noise train based on a quasi-orthogonal mirror image filter bank or the like can be considered. In this method, after adjusting the amplitude of each band element in the subband coding region, decoding processing is performed to synthesize a speech waveform.

  In the above method, the processing amount per sample can be made logarithmic order with respect to the number of subbands by using the filter bang processing using high-speed cosine transform or the like. In the conventional filter-based method, the processing amount per sample is in a linear order with respect to the filter order, so that the processing amount can be reduced as compared with the conventional method depending on setting conditions.

  Further, when all signal processing is linear processing, a method of combining white noise or impulse train pre-encoded in the subband code region is conceivable. When this method is used, the processing amount can be further reduced because the subband encoding processing at the time of speech synthesis becomes unnecessary.

  However, in this method, since the resolution in the frequency axis direction is determined by the number of subbands in the subband coding in generating the speech spectrum feature, in order to synthesize speech with reduced error from the desired spectrum feature, The number of subbands must be set large. However, increasing the number of subbands increases the amount of processing. Although the increase in the processing amount can be suppressed by increasing the frame period, on the other hand, if the frame period is increased, the resolution of the spectrum feature change in the time axis direction is insufficient and the quality is impaired.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a speech synthesizer, a speech synthesis method, and a speech synthesis program that can achieve high resolution while suppressing an increase in processing amount.

  (1) In order to achieve the above object, a speech synthesizer according to the present invention is a speech synthesizer that synthesizes speech waveforms based on input time-series sound source control information and spectrum characteristic information. A sub-band divided sound source generating unit for generating a sub-band divided sound source waveform vector corresponding to the input sound source control information based on the sub-band divided sound source waveform vectors accumulated by dividing the frequency band into a plurality of frequency bands; The subband power adjustment unit that adjusts the amplitude according to the input spectral characteristic information with respect to the subband divided sound source waveform vector, and the amplitude so as to simulate the spectral characteristic of the output target speech in a specific subband A sine wave synthesizer for synthesizing the adjusted sine wave; and the subband divided sound source waveform vector having the amplitude adjusted in the specific subband. And a subband division waveform vector generation unit that generates one subband division waveform vector by combining the sine wave synthesis component obtained by synthesizing the sine wave and the subband division waveform vector generated in the specific subband. A sub-band synthesizing unit that synthesizes a sub-band divided sound source waveform vector having the amplitude adjusted in a sub-band other than the specific sub-band into a single speech waveform. It is characterized by.

  As described above, the speech synthesizer according to the present invention combines periodic components of speech waveforms with sine waves whose amplitude characteristics are controlled so as to obtain desired spectral characteristics in some subbands. The harmonic component of the sex waveform element is directly synthesized. Thereby, even when the number of subbands in subband coding is small, it is possible to synthesize speech in which the input spectral characteristic information is reflected with higher accuracy in the frequency axis direction. As a result, sound can be reproduced with high resolution while suppressing an increase in processing amount.

  (2) In the speech synthesizer according to the present invention, the subband division sound source generation unit downsamples the sine wave synthesis so that the original waveform can be completely or approximately restored as the subband division sound source waveform vector. The unit generates the sine wave synthesis component in the specific subband at a sampling rate equal to the sampling rate in the downsampling.

  In this way, by outputting a sine wave synthesis result that has been sub-band encoded in advance without performing sub-band encoding processing at the time of speech synthesis, the synthesized speech waveform corresponding to the input is reduced while reducing the processing amount. Can be generated.

  (3) Further, the speech synthesizer according to the present invention is characterized in that the sine wave synthesis unit generates the sine wave synthesis component by using a subband of a part of the lower band as the specific subband. . Accordingly, it is possible to synthesize speech in which the input spectral characteristic information is reflected with high accuracy in the frequency axis direction in the subbands of a part of the lower band.

  (4) Further, the speech synthesizer according to the present invention is characterized in that the sine wave synthesis unit generates the sine wave synthesis component corresponding to the impulse sound source in the specific subband. As a result, it is possible to synthesize speech in which the input spectral characteristic information is reflected with high accuracy in the frequency axis direction for an impulse sound source of a specific subband.

  (5) The speech synthesis method of the present invention is a speech synthesis method for synthesizing speech waveforms based on input time-series sound source control information and spectrum characteristic information, and the sound source waveforms are divided into a plurality of frequency bands. Generating a sub-band divided sound source waveform vector corresponding to the input sound source control information based on the sub-band divided sound source waveform vector divided and accumulated; and for the generated sub-band divided sound source waveform vector Performing an amplitude adjustment for each subband according to the input spectral characteristic information; synthesizing an amplitude-adjusted sine wave so as to simulate the spectral characteristic of the output target speech in a specific subband; In the specific subband, the amplitude-adjusted subband divided sound source waveform vector and the sine wave synthesized from the sine wave Combining a component to generate one sub-band divided waveform vector; and synthesizing the generated sub-band divided waveform vector in the specific sub-band into a single speech waveform to generate sub-bands other than the specific sub-band. Synthesizing the sub-band divided sound source waveform vector whose amplitude has been adjusted in a band into a single speech waveform. As a result, it is possible to reproduce sound with high resolution while suppressing an increase in processing amount.

  (6) A speech synthesis program of the present invention is a speech synthesis program for synthesizing speech waveforms based on input time-series sound source control information and spectrum characteristic information, and the sound source waveforms are divided into a plurality of frequency bands. A process of generating a subband divided sound source waveform vector corresponding to the input sound source control information based on the divided subband divided sound source waveform vector, and the generated subband divided sound source waveform vector A process of performing amplitude adjustment according to the input spectral characteristic information, a process of synthesizing a sine wave whose amplitude has been adjusted so as to simulate the spectral characteristic of the output target speech in a specific subband, and the specific subband Combines the amplitude-adjusted sub-band divided sound source waveform vector and the sine wave synthesis component synthesized from the sine wave in a band. And processing for generating one subband division waveform vector, and synthesizing the generated subband division waveform vector in the specific subband into a single speech waveform, in subbands other than the specific subband A process for synthesizing the amplitude-adjusted subband-divided sound source waveform vector into a single speech waveform is executed by a computer. As a result, it is possible to reproduce sound with high resolution while suppressing an increase in processing amount.

  According to the present invention, in a specific subband, the processing in the subband divided sound source generation unit is not required during speech synthesis. In addition, by synthesizing spectral characteristics by directly controlling the amplitude characteristics of each harmonic component by sinusoidal synthesis for frequency bands that are affected by errors, the periodic waveform without increasing the number of filter bank bands. The reproduction accuracy of the spectral characteristics of the elements can be increased. As a result, speech synthesis processing with high spectral reproduction accuracy can be performed even in an environment where the calculation processing performance of a portable terminal or the like is limited.

It is a block diagram which shows the basic composition of the speech synthesizer which concerns on embodiment which becomes foundations. It is a block diagram which shows the specific structure of the speech synthesizer which concerns on embodiment which becomes fundamental. It is a block diagram which shows the actual circuit structure of a subband division part. It is a block diagram which shows the theoretical structure of a subband division part. It is a block diagram which shows the actual circuit structure of a subband synthetic | combination part. It is a block diagram which shows the theoretical structure of a subband synthetic | combination part. It is a graph which shows the amplitude characteristic with respect to a frequency about a band division | segmentation filter bank. It is a block diagram which shows the basic composition of the speech synthesizer which concerns on 1st Embodiment. It is a block diagram which shows the specific structure of the speech synthesizer which concerns on 1st Embodiment. It is a flowchart which shows an example of operation | movement of the speech synthesizer which concerns on 1st Embodiment. It is a flowchart which shows an example of operation | movement of the speech synthesizer which concerns on 1st Embodiment. It is a flowchart which shows an example of operation | movement of the speech synthesizer which concerns on 1st Embodiment.

  Next, embodiments of the present invention will be described with reference to the drawings. In order to facilitate understanding of the description, the same reference numerals are given to the same components in the respective drawings, and duplicate descriptions are omitted.

[Basic embodiment]
(Configuration of speech synthesizer)
FIG. 1 is a block diagram showing a basic configuration of the speech synthesizer 100, and FIG. 2 is a block diagram showing a specific configuration of the speech synthesizer 100. The speech synthesizer 100 divides and accumulates the sound source waveform by subband division by the subband division unit 110 and adjusts the amplitude for each subband according to the input information. Then, the sub-band synthesis unit 140 uses the sub-band divided sound source waveform vector whose amplitude has been adjusted to synthesize, and synthesizes a speech waveform having an approximate target spectral characteristic.

  The speech synthesizer 100 synthesizes a speech waveform based on the input time-series sound source control information and spectrum characteristic information. In the present embodiment, the sound source control information is a fundamental frequency of a voice waveform to be synthesized and a voiced weighting coefficient of each subband. As shown in FIG. 1, the speech synthesizer 100 includes a subband dividing unit 110, a subband divided sound source generating unit 120, a subband power adjusting unit 130, and a subband synthesizing unit 140.

  The subband dividing unit 110 divides the sound source waveform into a plurality of frequency bands, and generates a vector series by dividing the sound source waveform. It is preferable that the subband division unit 110 generates a subband division sound source waveform vector for thinning out and accumulating vectors from vector sequences within equal time intervals.

The subband splitting unit 110 includes, for example, analysis filter banks E ₀ (z) to E _M-1 (z) and a downsampler D ↓. The analysis filter banks E ₀ (z) to E _M-1 (z) are configured by filter banks that are equally divided into M frequency bands. The downsampler D ↓ is a vector obtained by thinning out the vector of (D-1) from the vector sequence of D samples (provided that D ≦ M) at equal time intervals with respect to the M-dimensional vector sequence after subband division. Process to leave only. Such thinning-out processing can reduce the pre-accumulation size and the processing amount of the synthesis filter bank.

  The subband division sound source generation unit 120 generates a subband division sound source waveform vector corresponding to the input sound source control information based on the subband division sound source waveform vector accumulated by dividing the sound source waveform into a plurality of frequency bands. To do. At that time, a plurality of subband divided sound source waveform vectors among the accumulated subband divided sound source waveform vectors are combined to generate a subband divided sound source waveform vector corresponding to the input sound source control information.

  The subband division sound source generation unit 120 further includes an accumulation unit 121 and a selection unit 122. The storage unit 121 stores a vector generated as a result of dividing a sub-band of a relatively short-time sound source waveform (sound source waveform segment) generated in advance. This vector is a vector having the same number of dimensions as the number of subband divisions, and this is called a subband division waveform vector.

  The selection unit 122 selects and adds the pre-stored subband division waveform vectors based on the input fundamental frequency information. In this manner, the subband division sound source generation unit 120 uses the selected subband division waveform vector or configures a plurality of types of subband division waveform vectors as subband division waveform vectors, and performs subband division. The generated sound source waveform vector is output. Note that the above accumulation is performed in advance as pre-processing, and the subsequent processing is performed when there is input information.

The subband power adjustment unit 130 performs amplitude adjustment for each subband according to the input spectral characteristic information on the generated subband divided sound source waveform vector. The subband power adjustment unit 130 is provided with a multiplier circuit for controlling the power of each subband. The subband power adjustment unit 130 adjusts the coefficients A _{0 to} A _M−1 for each subband based on the input spectral feature information. As a result, the spectral characteristics of the target speech are reproduced. The spectrum information to be input may be configured directly with the power information of each subband. For example, the mel cepstrum coefficient is input, the power information of each subband is calculated internally, and the result is It may be used.

The subband synthesizing unit 140 synthesizes the subband divided sound source waveform vector whose amplitude has been adjusted into a single speech waveform. That is, the sub-band division waveform is synthesized to generate a final synthesized speech waveform. The subband synthesizing unit 140 includes, for example, an upsampler D ↑ and synthesis filter banks R ₀ (z) to R _M-1 (z). The upsampler D ↑ inserts a zero value sample between the band-divided signals with respect to the sub-band divided sound source waveform vector whose amplitude has been adjusted, and performs D-times upsampling. The synthesis filter banks R ₀ (z) to R _M-1 (z) synthesize the sub-band divided sound source waveform vector divided into M frequency bands into a single speech waveform.

(Configuration of filter bank)
Multiplying the coefficients of a filter that constitutes a filter bank by the discrete Fourier transform (DFT), discrete cosine transform (DCT), or inverse transform coefficient series, shifts the characteristics of the underlying filter on the frequency axis. A filter characteristic of the shape is obtained. By configuring a filter bank with such a filter, a high-speed technique such as FFT (Fast Fourier Transform) can be used for calculations necessary for the filter bank processing. Thereby, the processing of subband division / subband synthesis can be speeded up.

  3A and 3B are block diagrams showing the actual circuit configuration and theoretical configuration of the subband splitting unit 110, respectively. 4A and 4B are block diagrams showing an actual circuit configuration and a theoretical configuration of the subband synthesis unit 140, respectively. Both examples show configuration examples using discrete cosine transform.

In each of the subband dividing unit 110 and the subband synthesizing unit 140, the actual circuit configuration is provided with a delay element z− ¹ and a discrete cosine transform element DCT or an inverse discrete cosine transform element IDCT. On the other hand, in a configuration that is theoretically equivalent to the subband dividing unit 110 and the subband synthesizing unit 140, a form that does not include the above elements is equivalent. In the configuration that is theoretically equivalent to the subband splitting unit 110, since the downsampling is performed after the filter processing, the processing sample rate is large and the processing amount is large. However, in the actual configuration, the downsampling is performed first. Since this is done, the amount of processing is reduced. The same applies to the subband synthesis unit 140.

  FIG. 5 is a graph showing amplitude characteristics with respect to frequency for the equal band division filter bank. When only DFT or DCT is used, a bandpass filter having an impulse response of a rectangular window function can be generally regarded as a filter bank composed of bandpass filters that are shifted on the frequency axis. Hereinafter, the base filter before the shift is referred to as a basic filter. Note that the basic filter is a band-pass filter having a frequency characteristic with a larger attenuation in the cut-off region, which is generally considered to be more preferable (if the frequency 0 is the center, it becomes a low-pass filter). It is also possible. However, it is necessary to design a filter so that the original speech waveform can be restored when subband synthesis is performed on the subband division processing result. That condition is called the perfect reconstruction condition. Further, depending on the filter configuration, exact restoration may not be possible. In such a case, the filter is designed so as to be restored approximately. When a DFT of length M is used, a filter bank is configured by M filters obtained by shifting the base filter by 2πk / M (k ≦ 0 <M) by a normalized angular frequency, and the DCT is When used, depending on the definition, in the case of the DCT transformation defined in the following example, a characteristic shifted by π (k + 1/2) / M at the normalized angular frequency, and π (−k + 1/2) A filter bank is composed of M filters having the frequency characteristics of the sum of the features shifted by / M.

In the following example, a pair of DCT transform and inverse DCT transform is used. The input / output of the DFT is defined as a complex number, whereas the input / output of the DCT is a real number, and the processing can be performed more easily. For example, the analysis filter bank can also be configured by using the expression (2) (0 ≦ k <M) as a filter using the M-th order DCT coefficient of the expression (1) as a coefficient.

  Due to the characteristics of the DCT coefficient, this is an M-divided equal-band division filter bank. Further, since this filter bank can be configured to satisfy the complete reconstruction condition, the input waveform can be restored from the band-divided waveform.

  In the above configuration, the number of subbands is a number that can simulate the spectrum described by the spectrum feature information with a predetermined accuracy. For example, when the spectral information of one sample is a k-th order mel cepstrum (the number of parameters including the zeroth order coefficient is k + 1), and k here is the number of dimensions necessary to express the spectral feature, In order to generally simulate such a spectrum, at least (k + 1) subbands are required.

  In addition, the subband power adjustment unit 130 operates to adjust the gain of each subband with respect to a white sound source and generate an audio waveform corresponding to the input spectral feature information. When performing multiband mixed excitation, normalization is performed so that the impulse sound source and the white noise source have equal power in advance, and control is performed so that the sum of the power weights of each subband is 1. A sound source can be obtained.

  As described above, the power value of each subband may be obtained by converting from the mel cepstrum coefficient or the like instead of directly inputting the power value of each subband as spectrum information. By controlling the spectral intensity at the center of the subband as a power value of the subband, the target spectral feature can be obtained approximately. The center of the subband is 0, 2π / M, 4π / M,... On the normalized angular frequency axis when configuring a filter bank based on DFT.

  On the other hand, when a filter bank based on the above-described DCT is configured, ± π / 2M, ± 3π / 2M,... However, in the case of using a basic filter whose input is a real number sequence and whose impulse response is symmetric, the frequency characteristics are all symmetric with respect to the frequency 0. Therefore, for example, only the range of 0 to π in the normalized angular frequency may be considered. . Since the spectral characteristic for each subband can be obtained from the filter bank coefficient, an error from the target spectral feature may be evaluated at a finer interval than the number of subbands on the frequency axis. For example, more accurate control can be realized by obtaining a subband power coefficient set that minimizes the mean square error by iterative approximate estimation or the like. The above example is merely an example, and the pair of DCT transform / inverse DCT transform can be replaced with another reversible transform pair.

(Sound source control method)
Next, a sound source control method will be described. First, it is assumed that the linearity of processing is guaranteed before and after subband division and subband synthesis. The filter bank based on the DFT or DCT described above satisfies this condition because its processing is configured only by a combination of linear operations.

  At this time, for the impulse train, for example, when 32 bands are divided from the past 32 samples and the analysis filter / synthesis filter of each band can be expressed by the FIR filter, the result of subband division is obtained as follows. be able to. That is, when the impulse sound source waveform in which the impulse is raised in the first and 20th samples of the input frame is divided into bands, the result of the subband division of the sound source waveform in which the impulse is raised only in the first sample Then, the sound source waveform in which the impulse is generated only by the 20th sample can be obtained by adding each element as a result of dividing the subband.

  That is, in the case of M-band division, it is only necessary for the impulse sound source to have prior accumulation of changes in M types of sound source waveforms. Actually, since the fundamental frequency used in speech synthesis is relatively low, there may be few cases where two or more impulses are included in the M samples of the sound source waveform. In this case, the processing amount of the addition process can be almost ignored.

  Note that the amount of processing for generating waveforms to be created and stored in advance is not a problem because it is not processing at the time of speech synthesis. Therefore, for example, it is also easy to control the position of the impulse with a time accuracy that is virtually equal to or higher than the sampling period, such as an impulse standing at the 1.5th sample. For such a sound source waveform, for example, a corresponding waveform using twice the sampling frequency is first created, and an antialiasing filter that is a high-frequency cutoff filter is applied to remove components higher than the Nyquist frequency at the original sampling frequency. And then thinning out the sample by 2: 1 downsampling.

  Such a method is particularly effective in the case where the sampling frequency is low and the error of the fundamental frequency of the synthesized speech becomes large if the impulse position is rounded to the sampling point. Conversely, when the sampling rate is high, a method of reducing the position accuracy of the impulse and reducing the number of subband division waveforms to be accumulated is also conceivable.

  On the other hand, for white noise sources, white noise may be synthesized by adding impulses, but an appropriate number and frame length of white noise sequences are pre-divided and accumulated, and then randomized for each frame. It may be configured approximately by selecting In this case, although it is necessary to accumulate the converted waveform, it is not necessary to calculate the weighted sum, and the amount of processing can be reduced. In order to generate white noise from only a relatively small number of accumulations, a method of constructing a band-divided sound source waveform by adding a plurality of accumulated band-divided sound source waveforms is also conceivable.

(Configuration using non-maximum decimation filter bank)
The decimation rate M in the filter bank is a value from 1 (not decimation at all) to M, and at least when power adjustment is not performed in each subband before recombination, the subband synthesis result is the value before subband division. It is theoretically known that a filter bank that matches the input signal can be constructed. For example, when a filter bank is constituted only by DFT or DCT and thinning-out rate L is thinned out, it is clear that if the calculation error is ignored, the input waveform can be completely restored by their inverse transformation.

  However, especially in the case of D = M (the thinning rate is the maximum and is called the maximum thinning), using DCT, in the normalized angular frequency (however, only the range from 0 to π is considered here because of its symmetry) 0) to π / M, π / M to 2π / M,. . . , (M-2) π / M to (M-1) π / M band components are all folded back and stored in the respective subbands regardless of whether they are passbands or stopbands. The At the time of synthesis, the aliasing noise components of the subbands cancel each other, thereby restoring the input waveform.

  When the filter of each subband is viewed as a band pass, the width of the pass band is also π / M, but in reality, an ideal filter that always has a gain of 1 in the pass band and always 0 in the cut-off band is It cannot be theoretically realized with a finite filter. Actually, there is a certain amount of passage even in the cut-off area, and in the case of maximum thinning, a large aliasing noise is included in each subband. For this reason, if the power of each subband is changed independently for each subband, the structure of aliasing noise canceling out between the subbands is destroyed, and the aliasing noise becomes a problem.

  On the other hand, if a value smaller than M is set in D, the folding width of the sample by thinning out becomes wider than the band width of the bandpass filter in the filter bank, so that the folding noise of each subband is reduced. Even when the power is adjusted independently every time, the influence of aliasing noise can be reduced. Such a setting is called non-maximum decimation. In general, the smaller the thinning rate D, the smaller the influence of aliasing noise, but the amount of information becomes redundant and the amount of data to be stored and processed increases. For this reason, it is preferable to set D as large as possible within a range necessary for suppressing the influence of aliasing noise.

  The aforementioned non-maximum decimation is equivalent to performing an overlap analysis of the frame shift D when viewed from the waveform series before band division and after band synthesis. In addition, for each D sample process in the time domain, one sample process in the subband division domain is performed. Here, for simplicity, it is assumed that D is a divisor of M. It is assumed that a filter bank that satisfies the complete reconstruction condition is used.

  First, the impulse sound source is controlled by the same method as the sound source control method described above, even if it is non-maximum decimation. However, for example, in a frame of length M, if the Nth sample (where M> N ≧ D) stands from the beginning, the NDth sample from the beginning in the next frame due to the frame shift of D samples. Impulse stands on the sample. At this time, the impulse sound source outputs a corresponding pre-stored subband division waveform vector at each timing.

  On the other hand, white noise can be generated by repeating the same waveform at an M × N sample period, for example, as the simplest method. In that case, a method of pre-accumulating M × N × (M / D) length M waveforms corresponding to the frame shift and sequentially outputting them in accordance with the frame shift is conceivable. Here, N may be any value as long as the noise period is such that there is no problem with hearing. For example, the noise period M × N may be longer than the period corresponding to the lower limit of the audible frequency (for example, 20 Hz).

  Alternatively, there is a method in which several white noise waveform segments having a length M are prepared in advance and are connected at random. Here, for one white noise waveform segment of length M, all sample values are treated as 0 outside the segment range on the time axis. The appearance pattern in one frame in the time domain of the white noise waveform segment alone is determined by the difference in the waveform start point in the frame, and −M + D, −M + 2D,..., −D, 0, D There are a total of (2 (M / D) -1) patterns of M-D.

  The noise waveform for one frame can be expressed by one type (when the starting point is 0) or a combination of two types of noise waveforms of length M. Therefore, when there are N types of white noise waveform segments of a sample of length M to be created in advance, one or two subband division waveforms are obtained from a total of N (2 (M / D) -1) pre-accumulations. A white noise source can be realized by acquiring and adding the subband division regions.

  In addition, by reducing the length of the white noise waveform segment to M / 2, M / 4,..., The number of appearance patterns in one frame is reduced, and the addition processing necessary for the sound source is reduced. It will increase. Conversely, the length of the white noise waveform segment can be increased. In this case, since the number of appearance patterns increases, the number of necessary accumulations increases, but the number of cases where a summing process is required with a sound source is reduced.

[First Embodiment]
(Apparatus configuration using sine wave synthesis)
In the above embodiment, a multiplication circuit is provided for each sound source to simultaneously reproduce the spectral envelope characteristics and adjust the mixing ratio of the sound source. For example, under the condition that the power of each band of the mixed excitation source is equal. It is also possible to first create a sound source waveform divided into subbands, and to perform power control for each subband.

  FIG. 6 is a block diagram showing a basic configuration of a speech synthesizer 200 using a sine wave synthesis component as a part of a mixed excitation source, and FIG. 7 is a block diagram showing a specific configuration of the speech synthesizer 200. . The basic configuration of the speech synthesizer 200 is the same as that of the speech synthesizer 100. The sound source waveform is divided into subbands by the subband dividing unit 210 and stored, and the amplitude is adjusted for each subband according to the input information. Then, the sub-band synthesis unit 140 uses the sub-band divided sound source waveform vector whose amplitude has been adjusted to synthesize, and synthesizes a speech waveform having an approximate target spectral characteristic.

  The voice synthesizer 200 synthesizes a voice waveform based on the input time-series sound source control information and spectrum characteristic information. The sound source control information is basic frequency and mixing weight information. The speech synthesizer 200 synthesizes a periodic waveform that is a constituent element of voiced sound by switching between an impulse train and a sine wave synthesis in units of subbands. For the subband synthesized from the impulse train, each element of the subband-encoded vector sequence is multiplied by an appropriate amplitude multiplier so as to obtain the target spectral characteristics.

  On the other hand, for subband components to be synthesized by sine wave synthesis, each harmonic component (component that is an integral multiple of the fundamental frequency of the driving sound source) included in the subband to be synthesized constitutes the target spectral characteristics. Each harmonic component is multiplied by an appropriate amplitude multiplier. At this time, the phase of the sine wave is controlled so as to have an equal delay, for example, when synthesized from the impulse train described above. For example, the addition of cosine functions whose frequency is an integer multiple of the fundamental frequency such that the phase becomes 0 at the time when the delay time of the band division filter is further added from the time at which the impulse in the impulse train is arranged. . Such addition of cosine functions can simulate the case of synthesis from a conventional impulse train, and satisfies the above conditions. Alternatively, different phase control methods may be used so as to be closer to the characteristics of natural speech. Note that one harmonic component is usually divided as a plurality of subband components due to the overlap characteristics of the band division filter. For this reason, it is necessary to control the amplitude and phase of the sine wave with a plurality of subbands for each harmonic component so that the synthesis result of the subband synthesizing unit becomes the target spectral characteristic.

  These two types of speech synthesis methods are combined in units of vector elements in subband coding to generate a subband-encoded synthesized speech sequence and decode it to generate a final speech waveform.

  As shown in FIG. 6, the speech synthesizer 200 includes a subband division unit 210, a subband division sound source generation unit 220, a subband power adjustment unit 130, a sine wave synthesis unit 232, a subband division waveform vector generation unit 236, and a subband division unit 210. A band synthesis unit 140 is provided.

  The subband division sound source generation unit 220 includes subband division sound source waveform vectors corresponding to subband impulse sound sources other than the subband list that synthesizes periodic components by sine wave synthesis, and subband division sound source waveforms corresponding to white noise sound sources. A subband division sound source waveform vector generated by weighted sum with the vector is generated. The subband splitting unit 210 includes an impulse side splitting unit 211a and a white noise side splitting unit 211b. The impulse side dividing unit 211a divides the impulse sound source into subbands, and the white noise side dividing unit 211b divides the white noise source into subbands.

  The subband division sound source generation unit 220 includes an impulse side accumulation unit 221a, an impulse side selection unit 222a, an impulse side weighting multiplication unit 223a, a white noise side accumulation unit 221b, a white noise side selection unit 222b, a white noise side weighting multiplication unit 223b, and An adder 224 is provided.

The impulse side accumulating unit 221a accumulates subband division sound source waveform vectors based on the impulse sound source. The impulse side selection unit 222a selects a sub-band division waveform vector based on the impulse sound source accumulated in advance based on the inputted fundamental frequency information. Impulse side weighting multiplier unit 223a multiplies _A pm _{to A p} a _(M-1) each of the weighting factors _A p0 _{to A p} for each element of the sub-band division waveform vector selected _(M-1).

On the other hand, the white noise side accumulating unit 221b accumulates the subband division sound source waveform vector based on the white noise source. The white noise side selection unit 222b selects the subband division waveform vector based on the white noise source accumulated in advance, for example, based on the method described in the above “sound source control method”. The white noise side weighting multiplication unit 223b multiplies each element of the selected subband division waveform vector by a weighting coefficient _{Aa0 to} Aa _(M-1) . Each coefficient is determined so that A _px + A _ax = 1.

  Adder 224 adds subband division waveform vectors weighted and multiplied on the impulse side and white noise side for subbands other than the subband list for synthesizing periodic components by sine wave synthesis. Thus, a plurality of types of subband division waveform vectors are generated as one subband division waveform vector based on the sound source information. When the mixed excitation source is used as a sound source, the mixing ratio adjustment between the impulse train and the noise source is simultaneously performed based on the sound source information.

  Note that the impulse side accumulation unit 221a, the impulse side selection unit 222a, and the impulse side weighting multiplication unit 223a constitute an impulse side subband division sound source generation unit 220a. The white noise side accumulation unit 221b, the white noise side selection unit 222b, and the white noise side weighting multiplication unit 223b constitute a white noise side subband division sound source generation unit 220b.

The subband power adjustment unit 130 performs amplitude adjustment for each subband according to the input spectral characteristic information on the generated subband divided sound source waveform vector. The subband power adjustment unit 130 includes a multiplication circuit for controlling the power of each subband, and adjusts the coefficients A _{0 to} A _M−1 for each subband based on the input spectral feature information.

The sine wave synthesizer 232 is a sine wave whose amplitude is adjusted so as to simulate the spectral characteristics of the output target speech for each sub-band for a sub-band list (specific sub-band) for synthesizing periodic components by sine wave synthesis. Is synthesized. Thereby, even when the number of subbands in subband coding is small, it is possible to synthesize speech in which the input spectral characteristic information is reflected with higher accuracy in the frequency axis direction. As a result, sound can be reproduced with high resolution while suppressing an increase in processing amount. Note that the sine wave composite component whose amplitude is adjusted for each subband is multiplied by a mixing weight coefficient A _{p0 to} Ap _(m−1) of the impulse component, and this is used as a component by the subband sine wave synthesis. save.

  The sine wave synthesis unit 232 preferably generates a sine wave synthesis component corresponding to the impulse sound source in a specific subband. Moreover, it is preferable that the sine wave synthesis unit 232 generates a sine wave synthesis component using a subband of a part of the lower band as a specific subband. Accordingly, it is possible to synthesize speech in which the input spectral characteristic information is reflected with high accuracy in the frequency axis direction in the subbands of a part of the lower band. The auditory characteristics can be improved especially by handling the lower frequency. Details of the operation of the sine wave synthesis unit 232 will be described later.

  The subband division waveform vector generation unit 236 generates one subband division waveform vector by combining a subband division excitation waveform vector whose amplitude is adjusted in a specific subband and a sine wave synthesis component obtained by synthesizing a sine wave. Note that the functions of the subband power adjustment unit 130 and the subband synthesis unit 140 are the same as those of the speech synthesis apparatus 100.

The subband synthesizing unit 140 synthesizes the subband divided sound source waveform vector whose amplitude has been adjusted into a single speech waveform. The subband synthesizing unit 140 includes, for example, an upsampler D ↑ and synthesis filter banks R ₀ (z) to R _M-1 (z).

  As described above, the speech synthesizer 200 divides each subband according to the type of the sound source waveform, and generates a subband division waveform vector to be a sound source using the band subband division waveform vector pre-stored at the time of speech synthesis. .

(Processing by sine wave synthesis unit)
Details of the operation of the sine wave synthesis unit 232 will be described below. The processing in the sine wave synthesizing unit 232 is, for example, generating a sine wave of a specific frequency at the same sampling rate as the final output waveform by the sine wave synthesizing unit 232, combining them to construct a periodic waveform, and subband coding It can be realized by doing.

  In the case where the subband coding process is configured only by a linear process, the sine wave synthesis unit 232 performs a sine wave sequence at the same sampling rate as the downsampled sampling rate for more efficient processing. Generate the subband code first, adjust the amplitude of the sine wave by multiplying the subband code vector by a constant, and then add the vectors to generate the output sine wave composite component. Also good.

  As described above, the sine wave synthesis unit 232 performs downsampling so that the original waveform can be completely or approximately restored as the subband divided sound source waveform vector, and at a sampling rate equal to the sampling rate in the downsampling, in a specific subband. It is preferable to generate a sine wave synthesis component. In this way, by outputting a sine wave synthesis result that has been sub-band encoded in advance without performing sub-band encoding processing at the time of speech synthesis, the synthesized speech waveform corresponding to the input is reduced while reducing the processing amount. Can be generated.

  Further, in the generation process of a subband-encoded sine wave, it is possible to directly obtain a subband-encoded vector sequence from its amplitude and phase characteristics without performing a normal subband encoding process. . This is possible because the frequency of the periodic waveform to be synthesized is given as sound source information, and the amplitude and phase characteristics can be obtained from the frequency-amplitude characteristics and frequency-phase characteristics of each filter used for subband coding. Become.

  The value of each sample in each subband is determined by the product of the amplitude of the sine wave in the phase obtained by using the amplitude characteristic determined from the spectral characteristic information and the amplitude characteristic of the band cutoff filter. Note that the phase reflects the phase characteristics of the band cut filter. The trigonometric function can be obtained efficiently, for example, by preparing a table of values for several hundred phases in advance and obtaining values defined by the table by linear interpolation. Similarly, the amplitude characteristic and the phase characteristic of the band cut-off filter can be efficiently obtained by combining the table and the primary interpolation.

  In the above calculation, the amplitude characteristic is not completely zero except for some points in the cutoff band of the filter in the subband coding. Therefore, if strict processing is performed, the subband encoding result for one sine wave sequence is obtained by calculating for all subband components, that is, all dimension elements of the subband division waveform vector. Will have to. The sine wave synthesizing unit 232 needs to calculate and calculate all the subband components to be synthesized.

  However, when the cutoff amount of the filter cutoff region is considered to be sufficiently large, the multiplication processing necessary for amplitude control of the sine wave can be reduced by regarding the value of the element of the dimension corresponding to the cutoff region as 0. For example, in MPEG audio in which 32-band division is performed, the edge frequency of the band-limiting filter is overlapped to the center frequency of adjacent subbands in the pseudo orthogonal mirror image filter bank used in encoding, and basically always at a certain frequency. Two subbands overlap each other, and the cut-off band of the band-limiting filter can be considered to be practically sufficient.

  For example, at the time of 32 kHz sampling, the pass band of the band limiting filter of the subband encoding unit is 0 Hz to 0.75 kHz, 0.25 kHz to 1.25 kHz, 0.75 kHz to 1.75 kHz, 1.25 kHz to 2.25 kHz, ... In this case, the 1 kHz sine wave is encoded mainly as the second and third subband components. Therefore, by the above approximation, the values of the remaining 30 subbands are fixed to 0, and the addition and multiplication processes necessary for generating the subband division waveform vector in the sine wave synthesis unit 232 can be omitted.

(Operation example of speech synthesizer)
Next, an operation example of the speech synthesizer 200 will be described. FIG. 8 and FIG. 9 are flowcharts showing an example of the operation of the speech synthesizer 200. In addition, A, B, and C in the figure indicate points connecting the flows of FIGS. 8 and 9. In this operation example, it is assumed that the frame shift is D samples, the length of one segment of the sound source waveform is M samples, and the number of divided bands is M.

  First, the in-frame start point s1 of the randomly selected noise element n1 is set to 0, and the in-frame start point s2 of the randomly selected noise element n2 is set to M (step T1). Next, the presence / absence of input data is determined (step T2). If there is no input data, the process ends. If there is input data, the fundamental frequency, mixing weight, and spectrum feature information are acquired as input data (step T3).

  The position of the impulse is determined from the input fundamental frequency (step T4). A subband division sound source waveform vector corresponding to each impulse is acquired from the accumulated subband division sound source waveform vector (step T5). The number of acquisitions is the same as the number of impulses. Then, the sum of the sub-band divided sound source waveform vectors acquired on the impulse side is calculated (step T6). Further, for the subbands other than the subband list for synthesizing the periodic component by sine wave synthesis, the elements of the subband divided sound source waveform vector are respectively multiplied by the mixing weight coefficient on the impulse side (step T7).

Let s be the first element of the subband list for synthesizing periodic components by sinusoidal synthesis (step T8). Let f _k be the minimum value included in the passband of the analysis filter (Es (z)) of the subband s among values of an integral multiple of the fundamental frequency, and x = 0 (step T9).

A cosine function centered on the impulse position with a frequency of f _k is expressed as P (w) by a frequency-amplitude characteristic function P (w) determined from spectral characteristics (where w = 2πf _k / f _s and f _s are sampling frequencies). Double. Then, when the filter E _s (z) is applied to the function, the value of the result at the target time t is calculated and added to x (step T10). That is, x is updated by x + | E _s (w) | P (w) cos (2πf _k (t−t ₀ ) + argE _s (w)). Note that E _s (w) is a function in which z of E _s (z) is replaced with exp (jw) (j is an imaginary unit), t is the target time, and t ₀ is the time of the impulse position.

f _k is increased by the fundamental frequency, and the synthesis target is updated to the next higher harmonic (step T11). It is determined whether f _k is included in the pass band of E _s (z) (step T12). If included, the process returns to step T10. Otherwise, the process proceeds to step T13. x is multiplied by the mixing weight coefficient of the impulse component of the subband s, and the result is stored as a component by the sine wave synthesis of the subband s (step T13).

  It is determined whether or not the subband s is the last element in the subband list for synthesizing the periodic component by sinusoidal synthesis (step T14). If it is not the last element, s is updated to the element next to s in the subband list for synthesizing the periodic component by sinusoidal synthesis (step T15), and the process returns to step T9. In the case of the last element, it is acquired from the subband divided sound source waveform vectors accumulated based on the information of (n1, s1) and (n2, s2), which are the two white noise subband divided sound source waveform vectors. (Step T16).

  Then, the sum of the acquired subband division sound source waveform vectors is calculated (step T17). Each element of the sub-band divided sound source waveform vector of the white noise source waveform is multiplied by (1−mixing weight coefficient) (step T18).

  Next, the intraframe start point s1 of the noise element n1 is set to s1-D, and the intraframe start point s2 of the noise element n2 is set to s2-D (step T19). It is determined whether or not the in-frame start point s1 is larger than −M (step T20). If larger, the process proceeds to step T22.

  When the intraframe start point s1 is −M or less, the noise element n1 is set to be the same as the noise element n2, and the intraframe start point s1 is set to zero. Also, the noise element n2 is newly selected at random, and the intraframe start point s2 of the noise element n2 is set to M (step T21).

  Next, for the subbands other than the subband list for synthesizing the periodic component by sine wave synthesis, the weighted subband divided sound source waveform vectors of the impulse side and the white noise side are used as the subband divided sound source waveform vectors of the mixed excitation source The sum is calculated (step T22). Then, each element of the sub-band divided sound source waveform vector of the sound source waveform is multiplied by a value based on the spectrum feature (step T23).

  For the subbands included in the subband list, the sum of the weighted subband divided sound source waveform vectors on the sine wave synthesis side and the white noise side is calculated (step T24). Sub-band synthesis processing is performed (step T25), D samples are output (step T26), and the process returns to step T2. Such processing reduces the amount of processing and enables sufficient speech synthesis processing and mixed excitation. The above operation is performed by executing a program by a computer in the apparatus.

  In the above embodiment, the amplitude is adjusted for each subband in accordance with the input information. However, the sound source waveform is decomposed into components of a plurality of frequency bands, and the decomposed components are respectively subband encoded, The amplitude may be adjusted for the band-coded decomposition component.

DESCRIPTION OF SYMBOLS 100 Speech synthesizer 110 Subband division part 120 Subband division sound source generation part 121 Storage part 122 Selection part 130 Subband power adjustment part 140 Subband synthesis part 200 Speech synthesizer 210 Subband division part 211a Impulse side division part 211b White noise Side division unit 220 Subband division sound source generation unit 220a Impulse side subband division sound source generation unit 220b White noise side subband division sound source generation unit 221a Impulse side accumulation unit 221b White noise side accumulation unit 222a Impulse side selection unit 222b White noise side selection Unit 223a impulse side multiplication unit 223b white noise side multiplication unit 224 addition unit 232 sine wave synthesis unit 236 subband division waveform vector generation unit

Claims (6)

A speech synthesizer that synthesizes a speech waveform in a plurality of divided frequency bands based on input time-series sound source control information and spectrum characteristic information,
A sine wave synthesizing unit that outputs a sine wave synthesis component obtained by adding and synthesizing a plurality of amplitude-adjusted sine waves so as to simulate the spectral characteristics of the output target speech in one or more subbands;
A subband synthesis unit that synthesizes the output sine wave synthesis component in the one or more subbands into a single speech waveform;
The sine wave synthesis unit has a sampling rate equal to or higher than the sampling rate when the output sine wave synthesis component is down-sampled so that the sub-wave synthesis unit can completely or approximately restore the original waveform. A speech synthesizer for generating the sine wave synthesis component that has been previously sub-band encoded in the sub-band.   2. The speech synthesizer according to claim 1, wherein the sine wave synthesis unit generates the sine wave synthesis component using a subband of a part of a lower band as the one or more subbands.   3. The sine wave synthesis unit generates the sine wave synthesis component in correspondence with an impulse position determined from a fundamental frequency input in the one or more subbands. The speech synthesizer described.   The sine wave synthesis unit outputs the sine wave synthesis component so that the phase becomes 0 at a time obtained by adding a delay time from the time at which the impulse determined from the fundamental frequency input in the one or more subbands is arranged. The speech synthesizer according to claim 3, wherein the speech synthesizer is generated. A speech synthesis method for synthesizing speech waveforms in a plurality of divided frequency bands based on input time-series sound source control information and spectrum characteristic information,
Outputting a combined sine wave component by combining a plurality of amplitude-adjusted sine waves so as to simulate the spectral characteristics of the output target speech in one or more subbands;
Synthesizing the output sine wave synthesis component in the one or more subbands into a single speech waveform;
The step of outputting the sine wave synthesis component is a sampling rate when the output sine wave synthesis component is downsampled so that the original waveform can be completely or approximately restored in the step of synthesizing the single sine wave synthesis component. A speech synthesis method characterized by generating the sine wave synthesis component previously sub-band encoded in the one or more sub-bands at a sampling rate equal to. A speech synthesis program for synthesizing speech waveforms in a plurality of divided frequency bands based on input time-series sound source control information and spectrum characteristic information,
Processing to output a sine wave synthesis component obtained by adding and synthesizing a plurality of amplitude-adjusted sine waves so as to simulate the spectral characteristics of the output target speech in one or more subbands;
Causing the computer to execute a process of synthesizing the output sine wave synthesis component in the one or more subbands into a single speech waveform;
The processing of outputting the sine wave synthesis component is performed at a sampling rate when the output sine wave synthesis component is down-sampled so that the original waveform can be completely or approximately restored in the synthesis of the single speech waveform. A speech synthesis program that generates the sine wave synthesis component that is pre-subband encoded in the one or more subbands at a sampling rate equal to.

Images