JP3321933B2 - Pitch detection method - Google Patents

Abstract

PURPOSE:To synthesize into natural voices even from special voices, such as husky voices and hoarse voices without changing entire band into UV. CONSTITUTION:The powder deviation and peak on the low region side of signal vectors are detected (S21, S22) when the result of judgment (S13) of V(vocal sound)/UV(unvocal sound) of respective bands obtd. by using the pitch obtd. by a pitch search (S11) and amplitude evaluation (S12) on the time base, such as self-correlation, is the entire band UV (S14) in pitch extraction with the voice analyzing synthesis system for executing a frequency analysis. The max. value near this peak is detected (S24) and the pitch P is calculated (S25) in accordance with the respective positions in the peak and max. frequency axes. The V/UV is then judged again (S27).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pitch detection method for dividing an input speech signal into blocks and detecting a pitch from the input speech signal when speech analysis processing is performed in units of the divided blocks.

[0002]

2. Description of the Related Art There are known various encoding methods for compressing an audio signal (including a voice signal and an acoustic signal) by utilizing a statistical property in a time domain and a frequency domain and a characteristic of human perception. ing. As this encoding method,
Coding in the time domain, coding in the frequency domain,
Analysis synthesis coding.

[0003] As an example of high-efficiency encoding of a speech signal or the like, M
BE (Multiband Excitation) coding, SBE (Singleband Excitation) coding, Harmonic coding, SBC
(Sub-band Coding: band division coding), LPC (Linear
Predictive Coding: Linear predictive coding) or DC
T (Discrete Cosine Transform), MDCT (Modified DC
Conventionally, when quantizing various information data such as spectrum amplitude and its parameters (LSP parameter, α parameter, k parameter, etc.) in T), FFT (Fast Fourier Transform), scalar quantization is conventionally performed. There are many.

In the speech analysis / synthesis system such as the PARCOR method, the timing of switching the excitation source is for each block (frame) on the time axis, so that voiced and unvoiced sounds cannot be mixed in the same frame. As a result, high-quality voice was not obtained.

[0005] On the other hand, in the MBE coding, each band (band) in which each harmonic (harmonic) of the frequency spectrum and 2-3 harmonics are grouped together for speech in one block (frame). For each band or for each band divided by a fixed bandwidth (for example, 300 to 400 Hz), voiced / unvoiced sound discrimination (V / UV discrimination) is performed based on the spectral shape in the band, so that sound quality is Improvement is recognized. V / U for each band
The V discrimination is mainly performed by observing how strongly the spectrum in the band has a harmonic structure.

The waveform of the voiced sound generally has a periodic structure. The basic period is called a pitch period, and the reciprocal thereof is called a pitch frequency. Detecting or extracting this pitch from an input speech signal is important in general speech coding processing, particularly in speech analysis and synthesis for analyzing and synthesizing speech. A so-called pitch lag in which a pitch cycle is represented by the number of samples is often used as pitch data in voice analysis synthesis.

[0007]

By the way, when the pitch of a speech signal is generally extracted, the peak of the auto-correlation or the auto-correlation of an LPC (linear predictive coding) residual signal is observed, so that it is very accurate. Can be estimated, but in special voices such as so-called faint and wrinkled voices,
The harmonic structure is disturbed, and if the pitch obtained on the time axis is used for discrimination between voiced sound and unvoiced sound (V / UV), it may be judged as unvoiced sound (UV) in all bands (bands). In particular, in the MBE coding described above, the advantage that voiced / unvoiced sounds (V / UV) coexist at the same time may not be reproduced.

The present invention has been made in view of the above circumstances, and prevents a special voice such as a faint voice or a wrinkled voice from being unvoiced (UV) in all bands, and has a natural voice. It is an object of the present invention to provide a pitch detection method that can be synthesized.

[0009]

According to a pitch detection method of the present invention, an input audio signal is divided into blocks on a time axis, and a pitch corresponding to a basic period of a sound is assigned to each of the divided blocks. In the pitch detection method for detecting the step of detecting the power unevenness and peak on the low frequency side of the spectrum of the signal in the divided block, and the step of detecting a local maximum near the detected peak, The above-described problem is solved by including a step of detecting a pitch based on the position of the detected peak and the position of the maximum value.

[0010] Here, such pitch detection is performed only when the pitch cannot be captured due to processing on the time axis such as autocorrelation, that is, the above-mentioned divided block is divided into a plurality of bands, and the divided blocks are divided. A determination may be made for each band as to whether it is a voiced sound or an unvoiced sound, and only when all the bands are determined to be unvoiced.

[0011] A high-efficiency voice coding method to which the pitch detection method according to the present invention is applied includes a voice analysis / synthesis method using multi-band excitation (MBE). In this multi-band excitation coding, the block is divided into a plurality of bands for each block, V (voiced sound) / UV (unvoiced sound) is determined for each of the divided bands, and V determined for each of these bands is determined. According to / UV, the portion determined as V synthesizes a voiced sound by synthesizing a sine wave or the like, and U
The portion determined to be V synthesizes an unvoiced sound by transforming the frequency component of the noise signal.

[0012]

Since the pitch is detected using the low-frequency side information of the voice spectrum, the pitch can be detected with high accuracy even if the pitch cannot be acquired by the processing on the time axis such as the autocorrelation.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First, pitch detection in the case of a special voice such as a faint voice or a wrinkle voice will be described.

When a voice spectrum is calculated by frequency analysis, a peak appears in a voiced part (Voiced Part) at an integer multiple of the fundamental frequency in synchronization with a pitch, as shown in FIG. Represents a structure having formants (F ₁ , F ₂ , F ₃ ).
On the other hand, in an unvoiced part, no pitch component appears and the formant structure is unclear, as shown in FIG.

However, as shown in FIG. 3, there are some voices in which the pitch, that is, the fundamental frequency does not appear even though the formant structure is clear.

The problem here is that, in order to perform the pitch extraction more accurately on the time axis, the formant component is excluded and the analysis is performed by, for example, LPC analysis. In such a case, when the pitch is extracted, it looks the same as in the case of the unvoiced part in FIG. In such a case, an accurate pitch cannot be obtained, and all bands (bands) are often evaluated as UV (unvoiced sound) in the subsequent V / UV (voiced / unvoiced sound) determination.

However, in the case of FIG. 3 described above, a portion which is locally V (voiced sound) (for example, A in the figure)
) Is included, it is necessary to detect a difference between the state of FIG. 2 and the state of FIG. 3 by some means and re-extract a new pitch.

First, in order to distinguish between the state shown in FIG. 2 and the state shown in FIG. 3, the uneven distribution of the power spectrum Sm, particularly, the uneven distribution on the low frequency side is obtained by the following equation.

[0019]

(Equation 1)

Here, peak (n, x _i ) represents a peak value in i = 0 to n of the data sequence x _i . Np is the number of points at the time of frequency analysis, and in this specific example, Np = 128
/ 4 kHz. Si represents, for example, the i-th (indexed i) spectrum data obtained by orthogonally transforming 256 samples of 8 kHz sampling data by FFT or the like. At the time of 8 kHz sampling, since the spectrum becomes symmetrical with respect to 4 kHz, 128 points up to 4 kHz are used among the 256 points of spectrum data obtained by orthogonally transforming 256 samples.

K obtained by this equation represents the ratio of the peak to the average value in the lower 1/3 range of the entire band to be analyzed, and this k is smaller than a predetermined threshold value Th _k Is larger, for example, assuming that Th _k = 8.0, k> 8.0.
Is satisfied, it is considered that a voiced part exists.

At this time, the index of Si giving the peak is set to j. That is, Sj = peak (Np / 3, Si) (2) Since Si has a voiced structure near i = j, that is, it can be expected that a pitch appears. Then, a new pitch is extracted from this.

[0023] As a specific method, the spectral data Si, locate the maximum point in the range of ± M points around the i = j, as shown in FIG. 4, the index of the + side j ^+, - side the j ^- to.

Here, a new pitch P is obtained according to the state of the maximum point as follows. (1) j ^+, j ^- if are both present. ^{P = N / min (j +} -j, j-j -) ··· (3) In this equation, min (a, b) represents a person not large a, the b. N
Is the number of samples in one block. In this embodiment, the sampling frequency fs is set to 8 kHz, and the above Np = 1
Since it is 28/4 kHz, N = 256. (2) j ^+, j ^- if the only one of present.

[0025] j ⁺ only ^{when: P = N / (j +} -j) ··· (4) j - only ^{when: P = N / (j-} j -) ··· (5) (3) If neither exists. No pitch correction by this method is performed.

Note that, as the pitch P, 20 ≦ P <148
Therefore, as the value of M, M ≦ L / 2
M = 12 which is an integer satisfying 0 is used.

By the way, an MBE (Multiband Ex) described later is used.
citation: multi-band excitation) When the above-mentioned pitch is used in coding or the like, it is necessary to perform amplitude evaluation and V / UV judgment again, but the pitch is effective near the peak of Si. Therefore, there is a high possibility that the entire band UV will result again. Therefore, it is conceivable to use the above value as the pitch, and forcibly use the voiced part extension, for example, for the determination (or correction) of V / UV again. As a specific example of this voiced part expansion,
When the band on the low frequency side is set to V (voiced sound), there is a method of forcibly setting V on the high frequency side.

Here, the pitch detection method according to the present invention
An example of a specific operation when applied to a speech analysis / synthesis apparatus (so-called vocoder) using BE (Multiband Excitation) will be described.

In this MBE vocoder, a signal for each block is converted on the frequency axis, divided into a plurality of bands, and it is determined whether each band is V (voiced sound) or UV (unvoiced sound).
That is, for example, the input audio signal is divided into blocks every fixed number of samples (for example, 256 samples), and
T is converted into spectrum data on the frequency axis by orthogonal transformation such as T, and the pitch of the voice in the block is extracted, and the spectrum on the frequency axis is band-divided at intervals corresponding to this pitch. About (band) V
(Voiced sound) / UV (unvoiced sound) is determined. This V / U
The V information is encoded and transmitted together with the spectrum amplitude data.

FIG. 5 is a flow chart showing a specific example of the operation of the main part when the embodiment of the pitch detecting method according to the present invention is applied to such an MBE vocoder. This figure 5
In the specific example shown in the above, when extracting the pitch corresponding to the basic period of the voice at the time of voice analysis, if the pitch on the time axis such as autocorrelation could not be caught, using the frequency spectrum information Improving accuracy. That is, when the determination of V (voiced sound) / UV (unvoiced sound) is determined to be UV (unvoiced sound) in all bands, it is determined whether or not this is appropriate, and if inappropriate, the pitch value and The V / UV judgment is recalculated and corrected.
Spectral information is used to confirm the validity of the judgment of all bands UV and to re-extract the pitch.

In the first step S11 in FIG. 5, a pitch search is performed by processing on the time axis such as autocorrelation.
Move to the next step S12. In this step S12, a high-precision (fine) pitch search by an analysis method based on synthesis as described later is performed while evaluating the amplitude. In the next step S13, V / UV (voiced sound / unvoiced sound) determination is performed using the fine pitch and amplitude data obtained in step S12, and the process proceeds to step S14, where all bands (bands) are determined to be UV. Is determined. If NO (at least one band is V), the flow advances to step S15 to shift to the next step of the normal operation of the MBE vocoder.

YES at step S14 (all bands UV)
When it is judged from, proceeds to step S21 above (1) Calculation of k of equation is performed, the threshold value Th _k (e.g. 8.0) This k is given in the next step S22 is greater than or (k> 8 .0) is determined. If YES, the process proceeds to step S23, in which a local maximum value near the position of the peak Sj (index i = j) is searched, and step S23 is performed.
24 + side, - each index j ⁺ side, j ^- of whether at least one is present the discrimination performed. When the answer to this step S24 is also YES, the process proceeds to step S25, where the pitch P is calculated from the above equations (3) to (5).
In step 26, the same amplitude evaluation as in step S12 is performed. In step S27, V / UV determination is performed again, and the process proceeds to step S15.

Here, the recalculation (or correction) of V / UV in step S27 is performed by a method of expanding the V (voiced sound) band, for example, when the V (UV) is lower than 500 to 700 Hz.
When a voiced sound is determined, the result of the determination is extended to the high frequency side, and V (voiced sound) is forcibly set to 3300 Hz. When it is determined NO in steps S22 and S24, both steps S1 and S24 are performed.
5 to V / V in normal MBE vocoder operation.
The process proceeds to the next step of the UV judgment.

Next, an MBE as a specific example of a speech efficient coding apparatus to which the pitch detection method according to the present invention is applied.
The vocoder will be described with reference to the drawings.

The MBE vocoder described below uses a DW G
riffin and JS Lim, "MultibandExcitation Vocode
r, "IEEE Trans. Acoustics, Speech, and Signal Process
ing, vol. 36, No. 8, pp. 1223-1235, Aug. 1988, and a conventional PARCOR (PARtial au).
To-CORrelation: a vocoder or the like switches between a voiced section and an unvoiced section for each block or frame during speech modeling, whereas the MBE vocoder uses the same time (the same block or frame). Modeling is based on the assumption that a voiced section (Voiced) section and an unvoiced section (Unvoiced) section exist in the frequency axis region of (in).

When assuming a speech analysis / synthesis system such as the MBE vocoder, the sampling frequency fs for an inputted speech signal on the time axis is usually 8 kHz, and the total bandwidth is 3.4 kHz (however, the effective band is 200 kHz). ~ 3400H
z), and the pitch lag (the number of samples corresponding to the pitch period) from the higher female voice to the lower male voice is 20 to
It is about 147. Therefore, the pitch frequency is 8000/147
It will vary from $ 54 (Hz) to about 8000/20 = 400 (Hz). Therefore, the above 3 ..
About 8 to 63 pitch pulses (harmonics) stand up to 4 kHz.

FIG. 6 is a block diagram showing a schematic configuration of the entire MBE vocoder. In FIG. 6, an audio signal is supplied to the input terminal 11.
This input audio signal is sent to a filter 12 such as an HPF (high-pass filter), and at least a low-frequency component (200 Hz or less) for removing a so-called DC (direct current) offset or band limitation (for example, limited to 200 to 3400 Hz). ) Is performed. The signal obtained through the filter 12 is sent to the pitch extraction unit 13 and the windowing processing unit 14, respectively. In the pitch extraction unit (pitch detection unit) 13,
The input audio signal data has a predetermined number of samples N (for example, N = 2
56) The block is divided into units (or cutout is performed by a rectangular window), and pitch extraction is performed on the audio signal in this block. Such a cut-out block (256 samples) is moved in the time axis direction at a frame interval of L samples (for example, L = 160) as shown in FIG. 7A, and the overlap between the blocks is N-. There are L samples (for example, 96 samples). The windowing processing unit 14 applies a predetermined window function, for example, a Hamming window, to one block N samples, and sequentially moves the windowed blocks in the time axis direction at intervals of one frame L samples.

When such a windowing process is expressed by a mathematical formula, _xw (k, q) = x (q) w (kL-q) (6) In the equation (6), k is a block number,
q represents a time index (sample number) of the data. The q-th data x (q) of the input signal before processing is windowed by a window function w (kL-q) of the k-th block. This shows that data x _w (k, q) can be obtained. The window function w _r (r) in the case of a square window as shown in FIG. 7A in the pitch extraction unit 13 is expressed as: w _r (r) = 1 0 ≦ r <N (7) = 0 r < 0, N ≦ r Further, the window function w _h (r) in the case of the Hamming window as shown in FIG. 7B in the windowing processing unit 14 is represented by w _h (r) = 0.54−0.46 cos (2πr / (N-1)) 0 ≦ r <N (8) = 0 r <0, N ≦ r When such a window function w _r (r) or w _h (r) is used, the window function w (r) (= w (kL−
zero data interval q)) is deformed this 0 ≦ kL-q <N, kL-N < window function in the case of q ≦ kL Thus for example the rectangular window _{w r (kL-q) =} 1
Is when kL−N <q ≦ kL, as shown in FIG. Further, the above equations (6) to (8) indicate that the length N
(= 256) indicates that the sample window advances by L (= 160) samples. Hereinafter, the above (7)
Each N point (0 ≦ r) extracted by each window function of the equation (8)
<N) are represented by x _wr (k, r) and x _wh , respectively.
(k, r).

As shown in FIG. 9, the windowing processing unit 14 _applies 179 to the sample sequence x _wh (k, r) of 256 samples per block to which the Hamming window of the above equation (8) has been applied.
Two samples of 0 data are added (so-called zero padding) to make 2048 samples, and the orthogonal transform unit 15 performs orthogonal transform such as FFT (fast Fourier transform) on the time axis data sequence of 2048 samples. Processing is performed. Or FF with 256 points without zero padding
There is also a method of applying T to reduce the processing amount.

The pitch extracting section (pitch detecting section) 13
Sample sequence of x _wr (k, r) above (1 block N samples)
Is performed based on the pitch. As the pitch extraction method, those using a periodicity of a time waveform, a periodic frequency structure of a spectrum, an autocorrelation function, and the like are known.
In the present embodiment, the autocorrelation method of the center clip waveform is adopted. As for the center clip level in the block at this time, one clip level may be set for each block, but the peak level of the signal of each part (each sub-block) obtained by subdividing the block is detected, and these are detected. When the difference between the peak levels of each sub-block is large, the clip level in the block is changed stepwise or continuously. The pitch period is determined based on the peak position of the autocorrelation data of the center clip waveform. At this time, a plurality of peaks are obtained from the autocorrelation data belonging to the current frame (the autocorrelation is obtained from data of one block N samples), and the maximum peak among the plurality of peaks is equal to or larger than a predetermined threshold. When the maximum peak position is the pitch period,
In other cases, a peak within a pitch range that satisfies a predetermined relationship with respect to a pitch obtained in a frame other than the current frame, for example, the preceding and following frames, for example, within a range of ± 20% around the pitch of the previous frame, Then, the pitch of the current frame is determined based on the peak position. In the pitch extraction unit 13, a relatively rough pitch search is performed by an open loop, and the extracted pitch data is sent to a high-precision (fine) pitch search unit 16, and a high-precision pitch search (pitch) by a closed loop is performed. Fine search) is performed.
Note that the input waveform, not the center clip waveform, is
A method of obtaining a pitch from the autocorrelation of the analyzed residual waveform may be used.

The high-precision (fine) pitch search unit 16 stores the coarse (rough) pitch data of the integer value extracted by the pitch extraction unit 13 on the frequency axis subjected to, for example, FFT by the orthogonal transformation unit 15. Data is supplied. In this high-precision pitch search section 16, an optimum decimal point (floating) is obtained by swinging ± several samples at intervals of 0.2 to 0.5 around the coarse pitch data value.
To the value of fine pitch data. As a method of fine search at this time, analysis by so-called synthesis
Using the (Analysis by Synthesis) method, the pitch is selected so that the synthesized power spectrum is closest to the power spectrum of the original sound.

The fine search of the pitch will be described. First, in the MBE vocoder, the FF
S (j) as spectrum data on the frequency axis orthogonally transformed by T or the like is expressed as S (j) = H (j) | E (j) | 0 <j <J (9) Model is assumed. Here, J corresponds to _{ω s / 4π = f s /} 2, the sampling frequency f
_s = when ω _s / 2π is, for example, 8kHz corresponding to 4kHz. In the above equation (9), when the spectrum data S (j) on the frequency axis has a waveform as shown in FIG.
H (j) indicates a spectrum envelope (envelope) of the original spectrum data S (j) as shown in FIG. 10B, and E (j) indicates an equivalent level as shown in FIG. 3 shows a spectrum of a periodic excitation signal (excitation). That is, the FFT spectrum S (j) is modeled as a product of the spectrum envelope H (j) and the power spectrum | E (j) | of the excitation signal.

The power spectrum of the above excitation signal | E (j)
| Takes into account the periodicity (pitch structure) of the waveform on the frequency axis determined according to the pitch, and converts the spectrum waveform corresponding to the waveform of one band (band) for each band on the frequency axis. It is formed by arranging it repeatedly. The waveform for one band was obtained by performing a FFT on a waveform obtained by adding (filling to zero) 0 data for 1792 samples to a Hamming window function of 256 samples as shown in FIG. 9 as a time axis signal. It can be formed by cutting out an impulse waveform having a certain bandwidth on the frequency axis according to the pitch.

Next, for each band divided according to the pitch, a value (a kind of amplitude) | A representative of the above H (j) (to minimize the error for each band) _m
| Here, for example, when the lower and upper points of the m-th band (band of the m-th harmonic) are a _m and b _m , respectively, the error ε _m of the m-th band is

[0045]

(Equation 2) Can be represented by | A _m | that minimizes this error ε _m
Is

[0046]

(Equation 3) When | A _m | in equation (11), the error ε _m is minimized.

The amplitude | A _m | is obtained for each band, and the error ε _m for each band defined by the above equation (10) is obtained using the obtained amplitude | A _m |. Then, the sum value of all such bands error epsilon _m of band-Σε
_{Find m} . Further, the error sum Shigumaipushiron _m of all such bands, calculated for different pitches to some small, obtaining the pitch as an error sum Shigumaipushiron _m is minimized.

That is, several types are prepared vertically, for example, in increments of 0.25 with the rough pitch obtained by the pitch extracting unit 13 as the center. Each respective pitches of different pitches to these plurality of types of fine finding the error sum Σε _m. In this case, when the pitch is determined, the bandwidth is determined. From the above equation (11), the power spectrum | S (j) | of the data on the frequency axis and the excitation signal spectrum | E
(j) | seek error epsilon _m of equation (10) using a, can be obtained sum Shigumaipushiron _m of all the bands. Obtains the error sum Shigumaipushiron _m for each pitch, is not to determine the pitch corresponding to error sum total value which is the smallest as the optimal pitch. As described above, the optimum fine (for example, in increments of 0.25) pitch is obtained by the high-precision pitch search unit, and the amplitude | A _m | corresponding to the optimum pitch is determined. The calculation of the amplitude value at this time is performed in the voiced sound amplitude evaluation unit 18V.

In the above description of the pitch fine search, in order to simplify the description, all bands are voiced (Vo
iced), but the MBE
In a vocoder, an unvoiced sound (Un
Since the model in which a voiced) region exists is used, it is necessary to discriminate voiced / unvoiced sounds for each band.

The optimum pitch from the high-precision pitch search section 16 and the amplitude | A from the amplitude evaluation section (voiced sound) 18V | A
_m | is sent to the voiced / unvoiced sound discriminating unit 17,
The voiced sound / unvoiced sound is determined for each band. NSR (noise-to-signal ratio) is used for this determination. That is, NSR which is the NSR of the m-th band
_m is

[0051]

(Equation 4) This NSR _m is equal to a predetermined threshold Th ₁ (for example, Th
₁ = 0.2) (that is, when the error is large), | A _m || E (j) |
It can be determined that the approximation of S (j) | is not good (the excitation signal | E (j) | is inappropriate as a basis),
V (Unvoiced). In other cases, it can be determined that the approximation has been performed to some extent, and the band is determined to be V (Voiced, voiced sound).

As described above, the number of bands (the number of harmonics) divided by the basic pitch frequency is as follows.
The number of V / UV flags for each band also fluctuates in the same manner because it fluctuates in a range of about 8 to 63 depending on the pitch of the voice (the magnitude of the pitch).

Therefore, in this embodiment, the V / UV discrimination results are grouped (or degenerated) for each of a fixed number of bands divided by a fixed frequency band. Specifically, a predetermined band including an audio band (for example, 0 to 0)
4000 Hz) is divided into N _B (for example, 12) bands, and a weighted average value is discriminated by a predetermined threshold value Th ₂ (for example, Th ₂ = 0.2) in accordance with the NSR value in each band, The V / UV of the band is determined.

This voiced / unvoiced (V / UV) discriminating section 1
In 7, when all bands are determined to be UV (unvoiced sound), as described above, the validity evaluation of the V / UV determination and the re-extraction of the pitch when it is not appropriate are performed using the frequency spectrum information. I have.

Specifically, the power eccentricity / peak detection unit 2
4, when the V / UV discriminating unit 17 discriminates that all bands are UV, using the spectral data from the orthogonal transform (FFT) unit 15, the power unevenness in the lower 1/3 region is determined by the above (1). The detection is performed by calculating k in the equation, and the peak is also detected at this time. The pitch calculation unit 25 calculates the (3) based on the peak position (the position on the frequency axis or the index j of the spectrum data) and the positions of the local maxima before and after the peak (the above j ⁺ , j ⁻ ). )
A new pitch P is obtained by the equation, the equation (4) or the equation (5). The new pitch P is stored in the high-precision pitch search unit 1 described above.
6 to perform fine search of the pitch as described above, and based on the obtained fine pitch, perform amplitude evaluation, V / UV discrimination, and the like again.

At the time of V / UV discrimination using the new pitch, the voiced portion is forcibly expanded. As a specific example of the voiced portion expansion, a description will be given of a process of expanding to the high frequency side when the low frequency side is V (voiced sound).

In this specific example, when the V / UV discrimination result of a predetermined number of bands below the first frequency on the low frequency side is V (voiced sound), a predetermined condition, for example, when the input signal level is a predetermined threshold Th greater than _S, zero cross rate of the input signal is under a predetermined threshold value Th _Z less condition, and a to a predetermined band up to a second frequency of the high frequency side performs expansion as the V. This extension is based on the observation that the structure of the low-frequency part (the strength of the pitch structure) in the speech spectral structure tends to represent the overall structure.

It is conceivable that the first frequency on the low frequency side is, for example, 500 to 700 Hz, and the second frequency on the high frequency side is, for example, 3300 Hz. This is the normal voice band 200-340
When a band including 0 Hz, for example, a band up to 4000 Hz is divided by a predetermined number of bands, for example, 12 bands, a V / UV discrimination result of, for example, two bands on the low band side, which is a band below the first frequency on the low band side. Is V (voiced sound), a predetermined band up to the second frequency on the high frequency side is extended under a predetermined condition so that V up to a band excluding two bands from the high frequency side. Is equivalent to performing

Next, the unvoiced sound amplitude evaluation unit 18U includes the on-frequency data from the orthogonal transformation unit 15, the fine pitch data from the pitch search unit 16, and the voiced sound amplitude evaluation unit 1
Data of amplitude | A _m | from 8V and V / UV (voiced / unvoiced) discrimination data from the voiced / unvoiced discrimination unit 17 are supplied. This amplitude evaluation unit (unvoiced sound) 1
In 8U, the amplitude of the band determined to be unvoiced (UV) by the voiced / unvoiced sound discriminating unit 17 is obtained again (amplitude re-evaluation is performed). The amplitude | A _m | _UV for this UV band is

[0060]

(Equation 5) Is required.

The data from the amplitude evaluation unit (unvoiced sound) 18U is sent to a data number conversion (a kind of sampling rate conversion) unit 19. The number-of-data converter 19 has a different number of divided bands on the frequency axis depending on the pitch.
This is to make the number constant considering the difference in the number of data (particularly the number of amplitude data). That is, for example, if the effective band is up to 3400 kHz, this effective band is divided into 8 to 63 bands according to the pitch, and the amplitude | A _m | (UV The number m _MX +1 of bands (including the amplitude | A _m | _UV ) also changes from 8 to 63.
For this reason, the data number converter 19 calculates the variable number m _MX +
One amplitude data is converted into a fixed number M (for example, 44) data.

In this embodiment, for example, for the amplitude data of one effective band on the frequency axis,
After enlarging the first data number by adding dummy data as to interpolate a value from the data in the block from the last data in a block in the N _F, O _S times the band-limited (e.g., 8 times) obtain an amplitude data of O _S times the number by performing oversampling, more N _M pieces of amplitude data is linearly interpolated in the O _S times the number _{((m MX +1) × O} S pieces) ( The number is expanded to 2048, for example, and the N _M pieces of data are decimated and the fixed number M (for example, 4
4 data).

The data (the fixed number M of amplitude data) from the data number conversion unit 19 is
, And are grouped into a vector for each of a predetermined number of data, and subjected to vector quantization. The (quantized part of) the quantized output data from the vector quantizing unit 20 includes the high-precision (fine) pitch data from the high-precision pitch search unit 16 and the voiced / unvoiced sound from the voiced / unvoiced sound discriminating unit 17. The data is sent to the encoding unit 21 together with the (V / UV) discrimination data and encoded.

Each of these data is obtained by processing the data in the block of N samples (for example, 256 samples), and the block is represented on the time axis by the frame of L samples. , The data to be transmitted is obtained in the frame unit. That is, the pitch data, V / UV discrimination data, and amplitude data are updated in the frame cycle. As described above, the V / UV discrimination data from the voiced / unvoiced discrimination unit 17 may be reduced (reduced) to about 12 bands as needed, and one or less bands in all the bands may be used. Voiced (V) region and unvoiced sound (U
V) Data indicating the position of division from the area. Further, when the above-mentioned pitch is detected again or when a predetermined condition is satisfied, V (voiced sound) on the low frequency side is extended to the high frequency side.
/ UV discrimination data pattern.

In the encoding unit 21, for example, CR
C addition and rate 1/2 convolutional code addition processing are performed. That is, important data among the pitch data, the voiced / unvoiced sound (V / UV) discrimination data, and the quantized output data are subjected to CRC error correction coding and then to convolutional coding. You. The encoded output data from the encoding unit 21 is transmitted to the frame interleave unit 2
2 and are interleaved together with a part (for example, low importance) data from the vector quantization unit 20, taken out from the output terminal 23, and transmitted (or recorded / reproduced) to the combining side (decoding side). .

Next, a schematic configuration of a synthesizing side (decoding side) for synthesizing an audio signal based on the above-described respective data transmitted (or recorded and reproduced) is shown in FIG.
1 will be described.

In FIG. 11, the input terminal 31 has
A data signal substantially equal to the data signal extracted from the output terminal 23 on the encoder side shown in FIG. 6 (ignoring signal deterioration due to transmission and recording / reproduction) is supplied. The data from the input terminal 31 is sent to the frame deinterleave unit 32, where the data is subjected to deinterleave processing which is the reverse of the interleave processing of FIG. 6 described above, and the main part (CRC and convolutional coded on the encoder side) In part,
Generally, a data portion having a high degree of importance is decoded by the decoding section 33 and sent to the bad frame mask processing section 34, and the remaining part (a part having a low degree of importance not subjected to encoding processing) is sent to the bad frame mask processing section 34 as it is. , Each sent. In the decoding unit 33, for example, a so-called Viterbi decoding process or an error detection process using a CRC check code is performed. Bad frame mask processing unit 34
Performs a process of obtaining the parameters of a frame with many errors by interpolation, and performs the above-described pitch data, voiced sound /
Unvoiced sound (V / UV) data and vector-quantized amplitude data are separated and extracted.

The vector-quantized amplitude data from the bad frame mask processing unit 34 is sent to an inverse vector quantization unit 35 and inversely quantized, and the data number inverse transform unit 3
6 to be inversely transformed. This data number inverse conversion unit 36
Then, the inverse conversion is performed in contrast to the data number conversion unit 19 in FIG. 1 described above, and the obtained amplitude data is converted to the voiced sound synthesis unit 37.
And sent to the unvoiced sound synthesizer 38. The pitch data from the mask processing unit 34 is sent to the voiced sound synthesis unit 37 and the unvoiced sound synthesis unit 38. The V / UV discrimination data from the mask processing unit 34 is also sent to the voiced sound synthesis unit 37 and the unvoiced sound synthesis unit 38.

In the voiced sound synthesizer 37, for example, a cosine
A voiced sound waveform on the time axis is synthesized by wave synthesis, and the unvoiced sound synthesizer 38 synthesizes an unvoiced sound waveform on the time axis by filtering, for example, white noise with a band-pass filter. The adder 41 adds and synthesizes the waveform and the waveform, and extracts the resultant from the output terminal 42. In this case, the amplitude data, the pitch data, and the V / UV discrimination data are updated and provided every frame (L samples, for example, 160 samples) at the time of the analysis, but the continuity between the frames is improved (smoothness). ), Each value of the amplitude data and the pitch data is set to 1
For example, each data value at the center position in the frame,
Each data value up to the center position of the next frame (one frame at the time of synthesis) is obtained by interpolation. That is, in one frame (for example, from the center of the above-described analysis frame to the center of the next analysis frame) at the time of synthesis, each data value at the leading sample point and each data value at the end (front of the next synthesized frame) sample point are obtained. And each data value between these sample points is obtained by interpolation.

Further, according to the V / UV discrimination data, all the bands are divided into a voiced (V) area and an unvoiced sound (U
V) area, and according to this division,
V / UV discrimination data for each band can be obtained.
As described above, regarding this division position, the V
May be extended to the high frequency side. If the analysis side (encoder side) has reduced (reduced) to a certain number (for example, about 12) bands on the analysis side (encoder side), this is resolved (restored) and changed at intervals according to the original pitch. Needless to say, the number of bands is set.

Hereinafter, the synthesizing process in the voiced sound synthesizing section 37 will be described in detail. The m-th discriminated as V (voiced sound)
1 on the time axis in the band (band of the m-th harmonic)
Synthetic frame (L samples, e.g., 160 samples) when the amount of the voiced and V _m (n), using a time index (sample number) n in the synthetic _{frame, V m (n) = A} m (n ) cos (θ _m (n)) 0 ≦ n <L (14) The final voiced sound V (n) is synthesized by adding (ΣV _m (n)) the voiced sounds of all the bands determined as V (voiced sound) in all the bands.

A _m (n) in the equation (14) is the amplitude of the m-th harmonic interpolated from the top to the end of the synthesized frame. In the simplest case, the value of the m-th harmonic of the amplitude data updated for each frame may be linearly interpolated.
That is, the m-th position at the end (n = 0) of the composite frame
The amplitude value of the harmonic is A _0m , and the end of the synthesized frame (n =
L: When the m-th harmonic amplitude value at the tip) of the next synthesized frame is A _Lm, the formula _{A m (n) = (Ln} ) A 0m / L + nA Lm / L ··· (15) A _m (n) may be calculated.

Next, the (14) the phase theta _m (n) in the expression by _{θ m (n) = mω O1} n + n 2 m (ω L1 -ω 01) / 2L + φ 0m + Δωn ··· (16) You can ask. In the equation (16), φ _0m indicates the phase of the m-th harmonic (frame initial phase) at the front end (n = 0) of the composite frame, and ω ₀₁ indicates the phase at the front end (n = 0) of the composite frame. The fundamental angular frequency ω _L1 indicates the fundamental angular frequency at the end of the combined frame (n = L: the leading end of the next combined frame). Δω in the above equation (16) is
The minimum Δω is set so that the phase φ _{Lm at} n = L becomes equal to θ _m (L).

The method of obtaining the amplitude A _m (n) and the phase θ _m (n) according to the V / UV discrimination result when n = 0 and n = L in an arbitrary m-th band will be described below. I do.

When the m-th band is V (voiced sound) for both n = 0 and n = L, the amplitude A _m (n) is calculated by the above equation (15). The amplitude A _m (n) may be calculated by linearly interpolating A _0m and A _Lm . Phase θ _m (n)
Is a _{n = 0 θ m (0)} = φ from _0m at n = L θ _m (L) is phi
Δω is set to be _Lm .

Next, when n = 0, V (voiced sound) and n =
If L (unvoiced sound) at L, the amplitude A _m (n)
Is linearly interpolated so that 0 A _m (L) from the transmission amplitude value A _{0 m} of A _m (0). The transmission amplitude value A _Lm at n = L is the amplitude value of the unvoiced sound, and is used in unvoiced sound synthesis described later. Phase θ _m (n) is set to _{θ m (0) = φ 0m} , and [Delta] [omega =
Set to 0.

Further, when n = 0, UV (unvoiced sound)
If V = voiced sound when n = L, the amplitude _Am
(n) sets the amplitude A _m (0) at n = 0 to 0 and performs linear interpolation so that the transmitted amplitude value A _Lm at n = L. Phase θ
The _m (n), the phase theta _m as (0) at n = 0, by using the phase value phi _Lm at the frame _{end, θ m (0) = φ} Lm -m (ω O1 + ω L1) L / 2 (17) and Δω = 0.

When both n = 0 and n = L are set to V (voiced sound), Δω is set so that θ _m (L) becomes φ _Lm.
The method for setting is described. In the above equation (16), n
= L, θ _m (L) = mω _O1 L + L ² m (ω _L1 −ω ₀₁ ) / 2L + φ
_{0m + ΔωL = m (ω O1} + ω L1) L / 2 + φ 0m + ΔωL = φ Lm becomes, and rearranging this, [Delta] [omega is, Δω = (mod2π ((φ Lm -φ 0m) - mL (ω O1 + ω L1) / 2 ) / L (18) In this equation (18), mod2π (x) means that the main value of x is −
This function returns a value between π and + π. For example, x = 1.3
mod2π (x) = -0.7π when π, mod2 when x = 2.3π
When π (x) = 0.3π, x = -1.3π, mod2π (x) = 0.7
π, and so on.

Hereinafter, the unvoiced sound synthesizing process in the unvoiced sound synthesizing section 38 will be described. The white noise signal waveform on the time axis from the white noise generation unit 43 is sent to the windowing processing unit 44, and windowing is performed with a predetermined length (for example, 256 samples) and an appropriate window function (for example, a Hamming window). STF
The power spectrum on the frequency axis of white noise is obtained by performing STFT (Short Term Fourier Transform) processing by the T processing unit 45. This STFT processing unit 4
5 is sent to the band amplitude processing unit 46, and the above-mentioned UV (unvoiced sound) band is multiplied by the above-mentioned amplitude | A _m | _UV, and the amplitude of the other V (voiced sound) band is set to 0. To The band amplitude processing section 46 is supplied with the amplitude data, the pitch data, and the V / UV discrimination data.

The output from the band amplitude processing section 46 is IS
The phase is sent to the TFT processing unit 47, and the phase is converted to a signal on the time axis by performing inverse STFT processing using the phase of the original white noise. The output from the ISTFT processing unit 47 is sent to an overlap addition unit 48, which repeats overlap and addition while weighting appropriately (to restore the original continuous noise waveform) on the time axis.
Synthesize a continuous time axis waveform. The output signal from the overlap adder 48 is sent to the adder 41.

As described above, the respective signals of the voiced sound portion and the unvoiced sound portion that have been synthesized in the synthesis portions 37 and 38 and returned on the time axis are added by the addition portion 41 at an appropriate fixed mixing ratio. The reproduced audio signal is extracted from the output terminal 42.

The present invention is not limited to the above embodiment. For example, the pitch detection using the above-mentioned spectrum information may be performed when the pitch cannot be completely captured by processing on the time axis such as autocorrelation. Although the determination is made when the determination of UV (unvoiced sound) is not appropriate for all bands, pitch detection using spectral information may be performed from the beginning. Also, the voice analysis side (encode side) in FIG.
Although the respective components are described in terms of hardware with respect to the configuration of FIG. 11 and the configuration of the speech synthesis side (decoding side) in FIG. 11, they can be realized by a software program using a so-called DSP (digital signal processor) or the like. is there. In addition, to combine (degenerate) the bands for each of the harmonics (harmonics) into a fixed number of bands,
This may be performed as needed, and the number of degenerate bands is not limited to 12 bands. Further, the process of dividing the entire band into the low-frequency side V region and the high-frequency side UV region at one or less division positions may be performed as necessary, and may not be performed. Furthermore, the present invention is not limited to the above-described multi-band excitation speech analysis / synthesis method, but can be easily applied to various speech analysis / synthesis methods using sine wave synthesis. The present invention can be applied not only to transmission and recording / reproduction, but also to various uses such as pitch conversion, speed conversion, and noise suppression.

[0083]

As is apparent from the above description, according to the pitch detection method of the present invention, the input audio signal is divided into blocks on the time axis, and the signal of each block is divided into the basic period of the audio. In a pitch detection method of detecting a corresponding pitch, by detecting the power unevenness and peak on the low band side of the spectrum of the signal in this divided block,
Since the maximum value near the detected peak is detected, and the pitch is detected based on the position of the peak and the position of the maximum value, the pitch cannot be captured by processing on the time axis such as autocorrelation. Pitch detection can be performed with high accuracy.

When the pitch cannot be detected due to the processing on the time axis such as the autocorrelation, for example, the voice detection is performed for each band obtained by dividing the divided block into a plurality of bands. Or unvoiced sound, so that it is performed only when the entire band is determined to be unvoiced. In particular, special voices such as faint or wrinkled voices also become UV (unvoiced) in all bands. No, natural speech can be synthesized.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an example of a spectral structure of a voiced sound signal.

FIG. 2 is a diagram illustrating an example of a spectral structure of an unvoiced sound signal.

FIG. 3 is a diagram showing an example of a signal spectrum structure in which a pitch (fundamental frequency) does not appear even if the formant structure is clear.

FIG. 4 is a diagram illustrating an example of a spectrum structure near a peak.

FIG. 5 is a flowchart for explaining the operation of the pitch detection method as one embodiment of the present invention.

FIG. 6 is a functional block diagram showing a schematic configuration of an analyzing side (encoding side) of an audio signal analyzing / synthesizing encoding apparatus as a specific example of an apparatus to which the pitch detecting method according to the present invention is applied.

FIG. 7 is a diagram for explaining windowing processing.

FIG. 8 is a diagram for explaining a relationship between a windowing process and a window function.

FIG. 9 is a diagram showing time axis data as an object of orthogonal transform (FFT) processing.

FIG. 10 is a diagram showing spectrum data on a frequency axis, a spectrum envelope (envelope), and a power spectrum of an excitation signal.

FIG. 11 is a functional block diagram showing a schematic configuration of a synthesis side (decode side) of an audio signal analysis / synthesis encoding apparatus as a specific example of an apparatus to which the high-efficiency encoding method according to the present invention is applied.

[Explanation of symbols]

 13 Pitch extraction unit 14 Window processing unit 15 Orthogonal transformation (FFT) unit 16 High-precision (fine) pitch search unit 17 Voiced / unvoiced (V / UV) discriminating unit 18V... Voiced sound amplitude evaluating unit 18U... Unvoiced sound amplitude evaluating unit 24. ... Pitch calculation unit 37 ... Voiced sound synthesis unit 38 ... Unvoiced sound synthesis unit

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G10L 11/04

Claims (2)

(57) [Claims] 1. A pitch detection method for dividing an input audio signal into blocks on a time axis and detecting a pitch corresponding to a basic period of audio for each signal of each of the divided blocks. Detecting the power unevenness and peak on the low frequency side of the spectrum of the signal within, detecting the local maximum value in the vicinity of the detected peak, and detecting the position of the detected peak and the position of the local maximum value. Detecting a pitch based on the pitch. 2. The method according to claim 1, wherein the divided block is divided into a plurality of bands, and whether each of the divided bands is voiced or unvoiced is determined. 2. The pitch detection method according to claim 1, further comprising: detecting a maximum value near the peak, and detecting a pitch based on the peak position and the position of the maximum value.