JP2006171008A - Device, method and program for extracting fundamental frequency, and recording medium with the program stored thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable extraction with an enhanced stability under the presence of noise and other disturbances of voice or speech. <P>SOLUTION: The frequency characteristics of the input signals, such as voice or speech signal and musical signal, is deformed to the frequency characteristics suitable for extracting the fundamental frequency by using preprocessing. A power S(ωc) is calculated for each frequency on the deformed input signal. An envelope component eliminating power is obtained by removing the envelope component from the power. The mean value of the envelope component eliminating power is subtracted from the envelope component eliminating power obtained for each frequency. Each frequency of plural frequencies in the assumed frequency range where the fundamental frequency exists is set as a candidate for the fundamental frequency. The center frequency which is near the frequency of an integral multiple of each candidate of the fundamental frequency is sought. Then, the sum of the subtracted value on each center frequency close to the integral multiple of the frequency of obtained for each frequency is sought, and this sum is set as a harmonic structure power. The frequency, giving the maximum value of the obtained harmonic structure power, is set as the fundamental frequency. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

The present invention relates to a fundamental frequency extraction method and apparatus, a fundamental frequency extraction program, and a recording medium for extracting the fundamental frequency by dividing it into a narrow frequency band from a plurality of sound signals such as sound signals under noise.
Fundamental frequency extraction is used as preprocessing for signal processing such as speech synthesis, speech recognition, speech coding, and the like. Therefore, high-accuracy fundamental frequency extraction under noise contributes to improving the performance of a signal processing device implemented as post-processing. Such a signal processing device includes the following.
1. A sound source that separates each component sound from the mixed sound of multiple sound sources based on fundamental frequency information
Separation device 2. Speech encoding / decoding device that encodes speech based on fundamental frequency information 3. Extracts melodies from fundamental frequencies of sounds sung by humans in noisy environments
3. Music search device for searching for music Music performance is obtained by receiving sound signals and extracting music information equivalent to music scores
4. Automatic music transcription device Machine control interface that passes commands to the machine at the fundamental frequency of human voice
Interface device and machine interaction device

Conventional example 1 of the fundamental frequency extracting apparatus will be described with reference to FIG.
This conventional example 1 utilizes the fact that periodic peaks appear on the logarithmic power spectrum at a frequency that is an integral multiple of the fundamental frequency. The input signal from the signal input unit 11 is Fourier-transformed for a short time by the logarithmic power spectrum extraction unit 12, and a logarithmic power spectrum is calculated by taking the logarithm of the square of the absolute value of each spectrum. Then, the short-time Fourier inverse transform is performed by the periodicity extracting unit 13, and the level corresponding to each period, that is, the periodicity is extracted. The maximum value extraction unit 14 extracts a time difference that maximizes the periodicity. The extracted time difference, that is, the reciprocal of the period is the fundamental frequency.

As shown in Non-Patent Document 1, the other conventional example 2 uses the instantaneous frequency to further emphasize the same logarithmic power spectrum peak as the conventional example 1 and to extract a high-accuracy fundamental frequency. It is. The instantaneous frequency component of the input signal is extracted, and the instantaneous frequency φ ′ (ω) (ω is the center frequency for each frequency band) for each frequency band and the spectrum S (ω) extracted by the logarithmic power spectrum extraction unit. From this, the instantaneous frequency spectrum G (λ ₀ ) with the peak enhanced is obtained using the following equation.

この瞬時周波数スペクトルＧ（λ₀ ）のピークの周期性を抽出することで、基
本周波数を抽出する。
阿部敏彦、小林隆夫、今井聖、「瞬時周波数に基づく雑音環境下でのピッチ推定」信学論、vol.J79-D-II, No.11, pp.1771-1781, Nov., 1996

The fundamental frequency is extracted by extracting the periodicity of the peak of the instantaneous frequency spectrum G (λ ₀ ).
Toshihiko Abe, Takao Kobayashi, Sei Imai, "Pitch Estimation under Instantaneous Frequency Based Noise Environment", Science, vol.J79-D-II, No.11, pp.1771-1781, Nov., 1996

In the conventional example 1 of the fundamental frequency extraction device described above, when a plurality of sounds other than the target sound and noise are included in the input signal, features other than the target sound are superimposed on the logarithmic power spectrum. For this reason, when the power of sound other than the target sound is increased, there has been a problem that an error in extraction of the fundamental frequency is increased.
In Conventional Example 2, since the instantaneous frequency spectrum emphasizes the frequency peak using the slope of the minute section of the instantaneous frequency, the unstable behavior of the instantaneous frequency appears in the instantaneous frequency spectrum as it is under noise. . For this reason, it is inappropriate as a feature quantity for stably extracting a fundamental frequency under noise.

According to the apparatus of the present invention, a power extraction unit that extracts the power of an input audio signal or an acoustic signal such as a music signal, an average value calculation unit that calculates an average value of the power of each frequency, For a plurality of frequencies, a harmonic structure power extraction unit for obtaining the sum of values obtained by subtracting the average value from each power related to this and an integer multiple of the frequency, and extracting the maximum value of the sum of the subtraction values, And a maximum value extraction unit that outputs the corresponding frequency as a fundamental frequency.
Further, according to the method of the present invention, a power extraction process for extracting the power of an input audio signal or an acoustic signal such as a music signal, an average value calculation process for obtaining an average value of power at each frequency, and the power On the other hand, for a plurality of frequencies, the harmonic structure power extraction process for extracting the harmonic structure power for obtaining the sum of the values obtained by subtracting the average value from the respective powers related to the integral multiple frequency, and the subtraction value A maximum value extracting process of extracting a maximum value of the sum and outputting a frequency corresponding thereto as a fundamental frequency.
First, occupancy extraction and fundamental frequency extraction using the occupancy associated with the present invention will be described.
In the invention related to the present invention, an occupancy representing the degree to which each frequency component of the input sound is not affected by noise is defined, a method and apparatus for extracting the occupancy, and a fundamental frequency using the occupancy An extraction method and apparatus are provided. For this reason, the following property regarding the instantaneous frequency is used.
The instantaneous frequency φ ′ is, for example, the time differentiation of the phase φ of each output wave when each frequency bin of the short-time Fourier transform is regarded as a group of narrow band pass filters output at equal intervals. It is. It is known that if there is an occupying frequency component with strong power in a certain band at a certain time, the instantaneous frequency becomes a substantially constant value in the bin near that frequency in the short-time Fourier transform (hereinafter referred to as STFT). It has been. Therefore, when the instantaneous frequency of a sound having a harmonic structure in an input signal with little noise is plotted on the vertical axis and the frequency bin of the STFT is plotted on the horizontal axis, a staircase pattern as shown by a thin solid line in FIG. 15A is obtained. . A point (φ ′ = ω _c , hereinafter referred to as a fixed point) where the horizontal portion of the staircase coincides with the center frequency ω _c of each frequency bin can be regarded as the frequency of each harmonic component. On the other hand, in an input signal with strong noise, the instantaneous frequency does not have a clear staircase shape, but becomes a gentle upward line as shown by a portion of 600 Hz or more of a thin solid line in FIG. 15B.

Ｂ（ω_c ）² は、中心周波数ω_c を持つ周波数ｂｉｎの近傍（ω∈Ω_c ）のｂ
ｉｎにおいて、各瞬時周波数（位相の微分値）φ′（ω）とω_c の差分をパワー
スペクトルＳ（ω）² で重み付き和をとったものである。占有的な周波数成分に
対応する不動点近傍では、φ′（ω）とω_c はほぼ同じ値をとるため、Ｂ（ω_c
）² は極小値をとると期待される。Ｂ（ω_c ）² の逆数（の対数）をとって、同
じ点で極大値を取るようにしたものがＤ₀ （ω_c ）である。なおＳ（ω）² によ
る重み付けは必ずしもしなくてもよいが、重み付けをした方が強いパワーを持つ
周波数の特徴がより強調される。また式（２）の分母はパワーによる定規化であ
る。
占有度Ｄ₀ （ω_c ）は、それ自身で調波構造を強調したスペクトル（占有度ス
ペクトルと呼ぶ）として見ることができるため、ケプストラム法のように対数パ
ワースペクトルに基づく基本周波数抽出法を、占有度スペクトルにそのまま適用
して基本周波数を抽出することができる。また、対数パワースペクトルを占有度
で重み付けした以下のスペクトルＤ_p も占有度スペクトルとして利用できる（式
中のａ，ｂは重み付け係数）。
Ｄ_p （ω_c ）＝log（Ｓ（ω_c ）^2a）＋ｂＤ₀ （ω_c ） （３）
＝log（Ｓ（ω_c ）^2a／Ｂ（ω_c ）^2b） （４）
Ｄ₀ （ω_c ），Ｄ_p （ω_c ）のどちらのスペクトルも、調波構造の強調効果に
より正確な基本周波数抽出が期待される。さらに、ＳＮＲの悪い状態でも、雑音
の影響の少ない周波数成分はそのまま強調され、雑音に埋もれた成分は抑制され
る。この結果、雑音下でも頑健な基本周波数抽出が実現出来る。 In order to evaluate how much the harmonic structure occupies the output of the frequency bin using the properties of the instantaneous frequency described above, the degree of dominance D ₀ (ω _c )
Is defined below.

B (ω _c ) ² is b in the vicinity (ω∈Ω _c ) of the frequency bin having the center frequency ω _c.
In, the difference between each instantaneous frequency (phase differential value) φ ′ (ω) and ω _{c is} a weighted sum of the power spectrum S (ω) ² . In the vicinity of the fixed point corresponding to the occupied frequency component, φ ′ (ω) and ω _c take almost the same value, so B (ω _c
² is expected to take a local minimum. D ₀ (ω _c ) is obtained by taking the reciprocal of B (ω _c ) ² and taking the maximum value at the same point. Although weighting with S (ω) ² is not necessarily performed, the characteristics of the frequency having stronger power are more emphasized when weighting is performed. The denominator of equation (2) is regularization by power.
Since the occupancy D ₀ (ω _c ) can be viewed as a spectrum that emphasizes the harmonic structure by itself (referred to as an occupancy spectrum), a fundamental frequency extraction method based on a logarithmic power spectrum, such as a cepstrum method, The fundamental frequency can be extracted by applying it directly to the occupancy spectrum. Further, the spectrum D _p of less weighted the logarithmic power spectrum in occupancy can be used as occupancy spectrum (a in the formula, b is the weighting factor).
D _p (ω _c ) = log (S (ω _c ) ^2a ) + bD ₀ (ω _c ) (3)
= Log (S (ω _c ) ^2a / B (ω _c ) ^2b ) (4)
In both spectra of D ₀ (ω _c ) and D _p (ω _c ), accurate fundamental frequency extraction is expected due to the enhancement effect of the harmonic structure. Furthermore, even in a state with a poor SNR, frequency components that are less affected by noise are emphasized as they are, and components buried in noise are suppressed. As a result, robust fundamental frequency extraction can be realized even under noise.

In order to obtain a refined fundamental frequency, the fixed point defined below is used. S
The center frequency of the frequency bin adjacent to the TFT is ω _c1 , ω _c2 (ω _c1 <ω _c2 ),
When the instantaneous frequency of each frequency bin is φ ′ (ω _c1 ), φ ′ (ω _c2 ), and the following equation is satisfied, the frequency ω at which φ ′ (ω) = ω is obtained between ω _c1 and ω _c2. It exists and is called a fixed point.
φ '(ω _c1 )> ω _c1 andφ' (ω _c2 ) <ω _c2
The frequency of the fixed point is considered to correspond to the frequency of each frequency component of the sound having the fundamental frequency. In particular, a fixed point with a large occupancy value is expected to correspond to a frequency component that is sufficiently stronger than the background noise, so the frequency of this fixed point is expected to give an accurate frequency component frequency. Is done. Further, a fundamental frequency candidate can be obtained by dividing the frequency of the frequency component by a certain integer. By calculating an average with a larger weight on a value having a large occupancy for a fundamental frequency candidate derived from this fixed point, a highly accurate fundamental frequency extraction method is configured even under noise.
Similarly, a method of using a signal power or a power from which an envelope component is removed instead of the occupancy can similarly form a refined method of extracting a fundamental frequency. In general, at a fixed point corresponding to a frequency component with strong power, the influence of the frequency component is stronger than that of background noise, so it is expected that the frequency of the fixed point will give a good approximation of the frequency component frequency. Therefore, in the present invention, a basic frequency extraction method with high accuracy is configured even under noisy conditions by calculating an average of basic frequency candidate values that place a greater weight on a high-power frequency.

Furthermore, a more accurate fundamental frequency extraction method is configured by combining with a sound source separation device. It is known that when a sound source separation device is used, a signal emitted from a sound source at a specific position can be emphasized or suppressed among two or more input signals measured at spatially different positions. However, since this separation signal also includes a certain degree of distortion in the separation result, the fundamental frequency extraction methods such as the conventional examples 1 and 2 may deteriorate the fundamental frequency extraction performance due to the distortion. It was. On the other hand, in the fundamental frequency extraction using the degree of occupancy, the fundamental frequency is extracted using only the occupying frequency component, so that it is not easily affected by distortion. For this reason, more accurate fundamental frequency extraction can be realized while avoiding the influence of noise suppressed by the sound source separation device.

The power of a signal not subjected to logarithmic conversion has a property that a difference between a noise component and a frequency component is large. In the present invention, focusing on this property of signal power, a fundamental frequency extraction method is configured for a signal that has not undergone frequency characteristic deformation. Also,
A fundamental frequency extraction method is configured in combination with a compensation method for returning a signal that has undergone frequency characteristic deformation to a state before undergoing the frequency characteristic deformation. This makes it possible to perform robust fundamental frequency extraction under background noise.

Prior to the description of the embodiments of the present invention, a related fundamental frequency extraction apparatus and method will be described.
Occupancy extraction (device)
An example of the occupancy extraction device is shown in FIG. An acoustic signal is converted into an input signal and input from the input unit 11, and this input signal is input by the instantaneous frequency extraction unit 21 for each frequency band instantaneous frequency φ ′ (ω ₁ ) to φ ′ (ω _n ) are extracted.
ω _{1 to} ω _n are the center frequencies of the respective bands. This frequency band is, for example, 50-100.
For example, 30-50 ms in the short-time Fourier transform unit 22.
Each input signal is Fourier-transformed for a short time, that is, converted into the frequency domain, and the converted spectrum is divided into n frequency bands by the band division phase detectors 23 ₁ to 23 _n. Phase of complex spectrum φ (ω ₁ ) to φ
(Ω _n ) is detected. Other methods such as wavelet transform and cosine transform may be used for transforming the input signal in the frequency domain. Alternatively, the input signal may be divided for each band by a band-pass filter (filter bank) having an interval of 50 to 100 Hz, and each output may be regarded as a sine wave to obtain the phase. In this apparatus, digital processing is generally performed.
In this way, the phases φ (ω ₁ ) to φ (ω _n ) for each band are differentiated from the differential units 24 _{1 to} 24.
Differentiated by _n , the instantaneous frequencies φ ′ (ω ₁ ) to φ ′ (ω _n ) are obtained.
These instantaneous frequencies φ ′ (ω ₁ ) to φ ′ (ω _n ) are input to the frequency difference extraction unit 25 and centered on the center frequency ω _c (c = 1, 2,..., N) for each frequency band. The difference between each instantaneous frequency and the center frequency ω _c is obtained for each of the bands ω _c −Δω to ω _c + Δω including the given front and rear bands. That is, φ ′ (ω ₁ −Δω
) −ω _{1 to} φ ′ (ω ₁ + Δω) −ω ₁ , φ ′ (ω ₂ −Δω) −ω _{2 to} φ ′ (ω ₂
+ Δω) −ω ₂ ,..., Φ ′ (ω _n −Δω) −ω _{n to} φ ′ (ω _n + Δω) −ω _n are obtained.
The integration range may be an appropriate fixed value corresponding to 50 to 100% of the assumed fundamental frequency, or may be adaptively changed as described later.

On the other hand, the input signal is input to the signal power extraction unit 26, and the input signal power S (ω _c ) ² of the center frequency ω _{c in} each frequency band is extracted. For example, the spectrum S (of the corresponding center frequency ω _c of the coefficient transformed into the frequency domain such as the short-time Fourier transform unit 22
ω _c ) is taken out and squared.
Each frequency difference φ ′ (ω _c −Δω) from the frequency difference extractor 25 and the center frequency power S (ω) ² from the signal power extractor 26 are input to the occupancy calculator 27 to calculate the occupancy. The The occupancy is D ₀ (ω _c ) defined by equation (1) or equation (3)
Alternatively, it is obtained by calculating D _p (ω _c ) defined by (4).
To determine the occupancy D ₀ (ω _c ), for example, as shown in FIG. 2A, the frequency difference φ ′ (ω
_c− Δω) −ω _{c to} φ ′ (ω _c + Δω) −ω _c is subjected to weighted addition of the power spectrum S (ω _c ) ^{2 by} the weighted addition unit 271. That is, each frequency difference φ ′ (p
) −ω _c (p = ω _c −Δω,..., Ω _c ,..., Ω _c + Δω) is squared by the square unit 272, and this square value (φ ′ (p) −ω _c ) ^{2 is} multiplied. The unit 273 is multiplied by S (ω _c ) ² , added by the adding unit 274, and the weighted addition result Σ (φ ′ (p) −ω _c ) ²
S (ω _c ) ² (Σ is p = ω _c −Δω to p = ω _c + Δω) is obtained.
On the other hand, the power spectrum of each frequency of the input signal of the frequency difference band ω _c -Δω~ω _c + Δω of ^{_{S (ω c -Δω) 2 ~S}} (ω c + Δω) 2 are input to the adder 275, these The addition value is added, and the addition value from the weighted addition unit 271 is divided by the division unit 276 to obtain B (ω _c ) ² . Further reciprocal-logarithmic arithmetic unit 278 B (ω _c) of the ^second inverse logarithm _{log (1 / B (ω c} ) 2) = D 0 (ω c)
Is calculated and output.

In order to obtain the occupancy D _p (ω _c ) according to the equation (3), for example, as shown in FIG.
The power S (ω _c ) ² of the center frequency of each band is multiplied by a power by the power unit 279, and as a result, the logarithm calculation unit 281 performs logarithm calculation on S (ω _c ) ² a. On the other hand, FIG.
D ₀ (ω _c ) obtained in step ( _b ) is multiplied by b by the multiplier 282, and the result bD ₀ (ω _c ) and the output log (S (ω _c ) ^2a ) of the logarithmic calculator 281 are added by the adder 283. And
It is output as D _p (ω _c ).
To obtain the occupancy D _p (ω _c ) according to the equation (4), for example, as shown in FIG.
S (ω _c ) ² is a-powered by the power unit 279, while the output B (ω _c ) ² of the dividing unit 276 in FIG. 2A is b-powered by the power unit 284, and the power result is obtained by the dividing unit 285. Division is performed to calculate S (ω _c ) ^2a / B (ω _c ) ^{2b, and a} logarithm is calculated by the logarithmic operation unit 285 and output as D _p (ω _c ).
It is good also as a = b in FIG. 2B and FIG. 2C. In this case, the power multiplier 279 and the multiplier 282 are omitted in FIG. 2B, and the power multipliers 279 and 284 are omitted in FIG. 2C. It should be noted that a and b need only be larger than 0, and S (ω _c ) ² and D ₀ (ω _c
) Or B (ω _c ) ² and a or b depending on the importance
To decide. This is determined by the noise mixing state of the input signal.

In the occupancy calculator 27, the frequency difference is added with the weight of the center frequency power S (ω _c ) ² , but this weighting is omitted, that is, the multiplier 273 is omitted in FIG. 2A and the frequency difference is added. Good. That is, the harmonic structure is emphasized more than the logarithmic power spectrum even by simply adding the frequency difference. In some cases, normalization by power may be omitted. That is, the adding unit 275 and the dividing unit 276 may be omitted in FIG. 2A.
Integration range in the equation (2), i.e. ω _c -Δω~ω _c + Δω is may be fixed, it is desirable to adaptively modify the estimate of the fundamental frequency of the input signal. That is, as shown by a broken line in FIG. 1, an integration range determination unit 28 is provided, and Δω determined by the integration range determination unit 28 is input to the frequency difference extraction unit 25, and the frequency range ω _c − of the frequency difference to be calculated. Δω˜ω _c + Δω is determined.
In other words, since the optimum value of the integration range varies depending on the fundamental frequency of the input speech, it is desirable to select a more appropriate integration range in order to obtain the fundamental frequency with better accuracy. For example, assuming that it is known in advance whether the speaker that is the sound source of the input signal is male or female, the optimal fixed integration range for each, for example, Δ
ω is set to about 80 Hz, and in the case of a woman, Δω is set to about 140 Hz, and this is set in the integration range determination unit 28. In another method, the fundamental frequency is extracted by the integration range determination unit 28 using another fundamental frequency extraction method such as the fundamental frequency extraction method described in the section of the prior art or other methods before applying the expression (2). The initial estimated value F ₀ is obtained, and, for example, 2 · Δω is about 50 to 100% with respect to the fundamental frequency in accordance with the initial estimated fundamental frequency, preferably 2 · Δω≈F ₀ × 0.75. The Δω may be determined and supplied to the frequency difference extraction unit 25.

Occupancy extraction (method)
Next, a processing procedure in the above-described occupation degree extraction apparatus, that is, a method for extracting the occupation degree will be described below.
FIG. 3 shows an example of a basic procedure. The instantaneous frequency for each frequency band of the input signal is extracted in the instantaneous frequency extraction process (S1). This instantaneous frequency extraction is performed, for example, by converting the input signal into a frequency domain signal by a short-time Fourier transform (Sa), and dividing the frequency domain signal into a narrow frequency band signal (Sb). ,
The phase φ (ω _c ) of each band signal is extracted (Sc), and each phase φ (
ω _c ) is differentiated to obtain an instantaneous frequency φ ′ (ω _c ) (Sd).
For these instantaneous frequencies φ ′ (ω _c ), subtract the center frequency ω _c from each value in the range of ω _c −Δω to ω _c + Δω including the bands before and after the center frequency ω _c as the center to obtain the frequency difference. Extract (S2).
Calculates the sum of the components of each ω _c -Δω~ω _c + Δω of the frequency difference, it calculates the occupancy of the omega _c using the sum (S3).
An example of obtaining the occupancy D ₀ (ω _c ) in the occupancy calculation of step S3 will be described with reference to FIG. First, weighted addition of the power spectrum of the frequency difference is performed for each band (S1). That each frequency difference in the band of ω _c -Δω~ω _c + Δω for each omega _c 2 squared (S1a), the power spectrum S to the square value (
ω _c ) ² is multiplied (S1b), and this power spectrum is multiplied for this band ω _c −Δω to ω _c + Δω (Sc).
On the other hand, for each center frequency omega _c calculates the sum of the power spectrum of the same band _{ω c -Δω~ω c + Δω (S2} ), the sum of the power spectrum, by dividing the weighted sum of the same band regular To obtain B (ω _c ) ² (S3). B (ω _c
) Take the reciprocal of ² , and perform logarithmic operation on the reciprocal to obtain D ₀ (ω _c ) (S4).
In FIG. 4A, steps S1 and S2 may be reversed in order.

Next, the order in which the occupancy D _p (ω _c ) is obtained by Expression (3) will be described with reference to FIG. 5A. The occupancy D ₀ (ω _c ) obtained in FIG. 4A is multiplied by a weight constant b to obtain bD ₀ (ω _c ).
(S1), and the power constant of ω _c is raised to the power of the weighting constant a.
(Ω _c ) ^2a is obtained (S2), and its logarithm log (S (ω _c ) ^2a ) is calculated (S3).
These and bD ₀ (ω _c ) are added to obtain an occupancy D _p (ω _c ) (S4). The order of steps S1 to S3 may be arbitrary.
Further, a procedure for obtaining the occupancy D _p (ω _c ) according to the equation (4) will be described with reference to FIG. 5B. The power of weight constant b is calculated for B (ω _c ) ² obtained in step S3 in FIG. 4A (S1), and the power of weight constant a is calculated for the power spectrum of ω _c (S2). The power multiplication result ratio S (ω _c ) ^2a / B (ω _c ) ^2b is obtained (S
3) The logarithm of this ratio is taken as the occupation degree D _p (ω _c ) (S4). Here, either step S1 or S2 may be performed first.

The occupancy extraction method described with reference to FIGS. 3 to 5 can be similarly modified in the occupancy extraction apparatus described above, and various conditions are also the same.
For example, the adaptive determination of the integration range Δω can be applied to this method as well. The instantaneous frequency extraction unit 21 in FIG. 1 and the instantaneous frequency extraction method in the instantaneous frequency extraction step S1 in FIG. 3 are not limited to the methods shown in these drawings. For example, “L. Cohen”, “Time-frequency analysis”. (Translated by Akira Yoshikawa and Shunsuke Sato), Chapter 2, Asakura Shoten (1998)
) "Or other methods may be used.

Basic frequency extraction (device)
Next, an example of a fundamental frequency extraction device using the above-described occupancy degree extraction device will be described.
As shown in FIG. 6, the input signal from the input unit 11 is input to the above-described occupancy degree extraction device (hereinafter referred to as an occupancy degree extraction unit) 31 and the occupancy levels D ₀ (ω ₁ ) to.
D ₀ (ω _n ) or D _p (ω ₁ ) to D _p (ω _n ) is extracted. These occupancy levels are input to the periodicity calculation unit 32, and the periodicity of the occupancy levels on the frequency axis is calculated. For example, the occupancy spectrum D ₀ (ω ₁ ) obtained at each time, for example, every 30 to 50 milliseconds.
˜D ₀ (ω _n ) or D _p (ω ₁ ) to D _p (ω _n ) is subjected to a short-time inverse Fourier transform, and the spectral peak periodicities P ₀ (T ₁ ) to P ₀ (T _n ) are Extracted. This periodicity is, for example, shown in FIG. 16 with time (period) T on the horizontal axis and level on the vertical axis.
These periodicity _{P 0 (T 1) ~P 0} (T n) are input to the maximum value extraction unit 33, the period T ₀ is extracted to provide the maximum value, the inverse of the period T ₀ is the inverse calculation unit 34 Calculated and output as a fundamental frequency F ₀ = 1 / T ₀ .

Next, another example of the fundamental frequency extraction device will be described with reference to FIG.
As in the case shown in FIG. 6, the occupancy (spectrum) is extracted by the occupancy extraction unit 31 from the input signal from the input unit 11. In this example, these occupancy spectra are input to the harmonic structure occupancy calculator 35, and the sum of the occupancy D _t0 (ω ₀ ) (or D _tp (ω ₀ )) relating to the harmonic structure defined below is obtained. The fundamental frequency is obtained by obtaining ω ₀ that is maximized.
_{D t0 (ω 0) = Σ} q D 0 (r (q · ω 0)) (5)
_{D tp (ω 0) = Σ} q D p (r (q · ω 0)) (6)
Here, ω ₀ is an arbitrary frequency, q is a harmonic order, r (·) is a frequency obtained by q · ω ₀ , and the frequency closest to the band center frequency ω _c in the band division used for occupancy extraction. Is a function that converts to The value after q may be any value, but the amount of calculation is simply increased. From this point, q · ω ₀ is 1500Hz
It is sufficient even if it is about to about 3000 Hz.

D _t0 (ω ₁ ) to D _t0 (ω _n ) or D _tp calculated by the harmonic structure occupancy calculating unit 35
(Ω ₁ ) to D _tp (ω _n ) are input to the maximum value extraction unit 36, the maximum value among these is extracted, and D _t0 (ω _c ) or D _tp (ω _c ) giving the maximum value The corresponding ω ₀ is output as the fundamental frequency F ₀ .
For example, as shown in FIG. 8, the harmonic structure occupancy calculating unit 35 sequentially supplies the multiplication unit 351 with ω _0.
And q · ω ₀ is calculated for each ω ₀ . If the average pitch period of men is 125 Hz, the frequency increased from about 90 Hz to about 100 Hz by 1 to several Hz may be sequentially set as ω ₀ . Multiplication result q · omega ₀ multiplier 351 is input to the corresponding center frequency detecting unit 352, closest to q · omega ₀ in ω ₁ ~ω _n ω
_c is obtained as omega _cq, occupancy D for each omega _cq in occupancy extraction unit 353
₀ (ω _cq ) or D _p (ω _cq ) is extracted, and the occupancy of each q extracted for each ω ₀ is added and output as D _t0 (ω ₀ ) or D _tp (ω ₀ ).
When the occupancy D ₀ (ω _c ) is used, by obtaining ω ₀ that maximizes the following equation, a fundamental frequency extraction device that is more resistant to noise than when using Equation (5) can be obtained.
D _t0 (ω ₀ ) = Σ _q (D ₀ (r (q · ω ₀ )) − D _0AV ) (7)
Here, D _0AV is an average value of occupancy degrees D ₀ (ω ₁ ) to D ₀ (ω _n ).
In this case, as indicated by a broken line in FIG. 8, the average value calculation unit 355 uses D ₀ (ω ₁
) To D ₀ (ω _n ), an average value D _0AV is calculated, and the adder 356 calculates Σ _q (D ₀ (ω _cq
) −D _0AV ) is calculated and output as D _t0 (ω ₀ ).
When the occupancy D _p (ω _c ) is used, D _p (ω ₁ ) to D _p (ω _n ) are regarded as a time series, high-pass filtering is performed, and the filtered D _p (ω ₁ )
By using ~ D _p (ω _n ) in the equation (6), it is possible to obtain a fundamental frequency extraction device with higher accuracy. That is, as shown by the broken line in FIG.
D _p (ω ₁ ) to D _p (ω _n ) are regarded as time series and are subjected to high-pass filtering, and fine change components D ′ _p (ω ₁ ) to D ′ _p (ω
_n ) is extracted, D ′ _p (ω _cq ) corresponding to each detected ω _cq is extracted by the occupancy extraction unit 358, and these are added by the addition unit 359, and D _tp (ω ₀ ) = Σ _q D
It is output as ′ _p (ω _cq ).

The fundamental frequency extraction device shown in FIG. 6 is resistant to noise, and the fundamental frequency extraction device shown in FIG. 7 is highly accurate. From this point, as shown in FIG. 6, the periodicity of the occupancy spectrum is calculated, the period of the maximum value is extracted, the fundamental frequency F ₀ is obtained from the reciprocal thereof, and is shown by a broken line in FIG. As described above, the fundamental frequency F ₀ is supplied to the harmonic structure occupation degree utilization fundamental frequency extraction unit 38, and in this extraction unit 38, each of the vicinity of the inputted fundamental frequency F ₀ , for example, 10% of F ₀ ± F _0. The harmonic structure occupancy calculation described above with reference to FIGS. 7 and 8 is performed with the frequency as ω ₀ , and Equation (5) or (6) or (7) or Σ _q D ′ _p (r ( q · ω ₀ )) to maximize ω ₀
And ω ₀ is output as the correct fundamental frequency F ₀ . In this way, a fundamental frequency extraction device that is resistant to noise and has high accuracy is configured.

Basic frequency extraction (method)
Next, the processing procedure of the fundamental frequency extraction apparatus described above, that is, an example of the fundamental frequency extraction method will be described.
FIG. 9 corresponds to the apparatus shown in FIG. 6. First, the occupancy (spectrum) D ₀ (ω _c ) from the input signal is obtained by the occupancy extraction method according to the present invention shown in FIGS. ) Or D _p (ω _c ) is extracted (S1), and the periodicity of the occupancy on the frequency axis of this occupancy spectrum is calculated. For example, the occupancy spectrum at each time is Fourier-transformed for a short time. Periodicity is obtained (S2). The period (time) T ₀ giving the maximum value of the periodicity of the occupancy is extracted (S3), and the reciprocal 1 / T ₀ of the period T ₀ is extracted.
= F ₀ is obtained to obtain the fundamental frequency F ₀ (S4).

Next, an example of a fundamental frequency extraction method corresponding to the apparatus shown in FIG. 7 will be described with reference to FIG. As in the previous case, the occupancy (spectrum) D ₀ (ω _c ) or D _p (ω _c ) is determined from the input signal by the occupancy extraction method according to the present invention shown in FIGS.
Is extracted (S1). Next, in this embodiment, with respect to the occupancy, the sum of the occupancy for a plurality of frequencies ω ₀ is obtained for each of the frequencies, and the harmonic structure occupancy D _t0 (ω ₀ ) or D _tp ( (ω ₀ ) is obtained (S2).
In this step S2, for example, each ω ₀ is multiplied by q (q = 1, 2,...) (S2a), and ω _c closest to each q · ω ₀ , that is, when the occupancy is extracted, the input signal is narrowed. The one closest to q · ω ₀ in the center frequencies ω ₁ ,..., Ω _n of each band when divided into bands is calculated, and ω _c is written as ω _cq (S2b). Occupancy degree D ₀ of each obtained ω _cq
(Ω _cq ) or D _p (ω _cq ) is obtained (S 2 c), and for each ω ₀ , the sum of the obtained D ₀ (ω _cq ) or D _p (ω _cq ) Σ _q D ₀ (ω _cq ) or Σ _q D _p (ω _cq ) is obtained, that is, the harmonic structure occupancy D _t0 (ω ₀ ) or D _tp (ω ₀ ) is obtained (S2d).
In this way, the harmonic structure occupancy for each omega ₀ obtained D _t0 (ω ₀₎ or D _tp (
The largest one of ω ₀ ) is extracted, and the extracted maximum D _t0 (ω ₀ ) or D _tp (ω
The ω ₀ of ₀₎ and the fundamental frequency F ₀ (S3).

In the method shown in FIG. 10, a modification similar to that described with reference to FIG. 8 can be considered. That is, as indicated by a broken line in FIG.
The sum of the difference between the occupancy D _{0 (ω 1)} ~D ₀ calculates the average value D _0AV of (ω _n) (S4), for each omega ₀ the determined D ₀ and (omega _cq) and the average value D _0AV Σ _q (D ₀ (ω _cq ) −
D _0AV) a D _t0 (ω ₀₎ as determined (S5), Turning now to step S3, to obtain a F ₀ seeking omega ₀ giving the maximum value in these D _t0 (ω _0).
Alternatively, after the step S2b or in advance, a high-pass filter process is performed with the occupancy D _p (ω ₁ ) to D _p (ω _n ) as a time series, and D consisting only of fine change components excluding slowly changing components. _'p (ω ₁₎ ~D' seek _{p (ω n) (S6)} , determined for each q a D _'p (ω _cq) in place of step S2c the D _p (ω _cq), step S2d the D _tp = Σ _q D 'by calculating _{p (ω cq)} proceeds to step S3.
As shown in FIG. 6, the periodicity of the occupancy is obtained and the period T ₀ giving the maximum value is obtained.
Look, Searching for the inverse F ₀ = 1 / T ₀ as the fundamental frequency, as omega ₀ a frequency near the F ₀ by further harmonic structure occupancy utilizing fundamental frequency extraction unit 38 as shown by a broken line in FIG. 6 In addition, the fundamental frequency with higher accuracy can be obtained. Also in the fundamental frequency extraction method, as shown by a broken line in FIG. 9, the frequencies near the fundamental frequency F ₀ obtained in step S4 after step S4, for example, frequencies in a band of F ₀ ± F ₀ × 0.1 are obtained. performing step S2 after the processing shown in FIG. 10 as omega _0,
A fundamental frequency with higher accuracy may be obtained (S5). This step S
5, various modifications indicated by broken lines in FIG. 10 can be applied.

Modification FIG. 11 shows a modification of the fundamental frequency extraction apparatus of the present invention. The difference from the apparatus shown in FIGS. 6 and 7 is that the occupancy periodicity P ₀ (T ₁ ) ˜
P ₀ (T _n ) or the occupancy sum D _t0 (ω ₁ ) to D from the harmonic structure occupancy calculator 35
_t0 (ω _n ) or D _tp (ω ₁ ) to D _tp (ω _n ) is smoothed so as to be temporally continuous by the fundamental period or fundamental frequency smoothing unit 37, and the smoothed occupancy periodicity Alternatively, the occupation degree sum may be supplied to the maximum value extraction unit 35 or 36 to prevent erroneous extraction based on the abnormal value.
That is, the extraction accuracy of the fundamental frequency obtained at each time is further improved by using temporal continuity. This corresponds to the periodicity of the fundamental frequency extraction method shown in FIG. 9 or the time series of the sum of occupancy related to the harmonic structure of the fundamental frequency extraction method shown in FIG. Next, as shown by the broken line,
Further, as shown by a broken line after step S2d in FIG.
7, the peak position with a small frequency gap is tracked along the time axis.

For this peak tracking, for example, a known algorithm such as dynamic programming (hereinafter referred to as DP) can be applied. In addition, since the fundamental frequency extraction is assumed as a pre-process for various audio processes, it may be desirable to perform a sequential process instead of a batch process such as DP. In this case, a sequential DP in which the DP algorithm is improved can be applied. In the sequential DP, at each time, a normal DP is executed on a time series of the sum of periodicity or occupancy before the current time that has already been obtained to obtain the current fundamental frequency. With this method, it is possible to estimate the fundamental frequency at the current time in consideration of frequency continuity from the past to the present. And originally D
Since P is a sequential algorithm that updates the optimum path up to the current time during execution, even if it is a sequential DP, no extra calculation occurs compared to a normal DP.

Next, FIG. 12 shows an embodiment of a fundamental frequency extraction device for sound source signals separated by a sound source separation device. Two or more channels of acoustic signals are input by the signal input unit 41, and the input signals of these plural channels are emphasized by the target sound source signal from the positional relationship between the sound source and the signal input unit by the sound source separation device 42, or sound other than the target sound source signal. The target sound source signal is separated by suppressing the signal, and the fundamental frequency of the separated target sound source signal is extracted by the fundamental frequency extraction device 43 shown in any of FIGS.

FIG. 13 shows a configuration example of a sound source separation device 42 using a dummy head microphone. For example, short-time Fourier transform is performed in the frequency analysis units 421 _R and 421 _{L on} each of the two-channel input signals input from the left and right ear signal input units 41 _L and 41 _R, and the converted spectrum is obtained. Thus, the intensity and phase of the signal are obtained for each of the left and right frequencies, and the intensity difference and phase difference between the left and right inputs are obtained by the intensity difference extraction unit 422 and the phase difference extraction unit 423 for each frequency. Using the characteristics of the dummy head related to the intensity difference and phase difference of the sound coming from the direction of the target sound source,
For each frequency, a range of difference in sound intensity and time difference from the target direction is obtained. Using this property, the target direction frequency band selection units 424 and 425 check whether or not the input sound falls within this range at each frequency, and the target direction frequency band signal passing unit 42
In the case of a sound other than the target direction at 6, the input signal of that frequency is replaced with 0. By applying a short time inverse Fourier transform to the left and right signals obtained as a result, only the sound coming from the target direction can be separated. This sound source separation device is, for example, J. Acoust. Soc
Jpn (E) 20, 2 (1999) pp. 147-149.

The sound signal thus separated is a sound signal having a large distortion because sounds in several frequency bands are replaced with zero. However, when the target sound signal has an occupying frequency component that is stronger than noise, the component remains in the separated sound signal. Therefore, the fundamental frequency extraction method using the occupancy according to the present invention can be applied as it is, and a fundamental frequency extraction method that is not easily affected by separation distortion in addition to the noise suppression effect of the sound source separation device can be configured.
Many sound source separation methods using a plurality of microphones are known, such as an independent component analysis method, a null beam former method, a delay sum method, and a mint method. Whichever method is used, the fundamental frequency is extracted from the separated sound signal by the method using the occupancy according to the present invention. Can be configured.

ここでｑは高調波の次数、ｒ（・）はｑ・ω₀を最も近い周波数ｂｉｎの中心
周波数ω_cに変換する関数、Ｅ（Ｄ₀（ω_c））はＤ₀（ω_c）の全周波数にわたる
平均値である。同基本周波数抽出部はこうして求められた調波構造占有度に関し
て、以下の式に従って、最大値を与える基本周波数の初期設定値を抽出する（Ｓ
１）。

Adaptive Integration Range Determination Method FIG. 19 shows a processing procedure for adaptively determining the integration range and extracting the fundamental frequency when the approximate fundamental frequency of the input signal is not obtained.
First, an input signal input from the input unit is received by a fundamental frequency extraction unit based on the occupancy level, and the occupancy levels obtained by equations (1) and (2) are extracted. At this time, for the integration range required in Expression (2), an integration range (about 260 Hz width in the case of an utterance of an adult speaker) that can be commonly used for the fundamental frequency of the sound included in the input sound is used. Next, the fundamental frequency extraction unit obtains the harmonic structure occupation degree with respect to the occupation degree thus obtained. This is calculated, for example, using the following equation in connection with the method described in FIG.

Here, q is the harmonic order, r (·) is a function for converting q · ω ₀ to the center frequency ω _c of the nearest frequency bin, and E (D ₀ (ω _c )) is D ₀ (ω _c ). Average value over all frequencies. The fundamental frequency extraction unit extracts an initial set value of a fundamental frequency that gives a maximum value according to the following formula with respect to the harmonic structure occupancy thus obtained (S
1).

Next, the integration range determination unit 28 determines an optimal integration range for the initial fundamental frequency thus determined (S2). The optimum integration range is STFT frequency bin
Is about 60% to 100% of the initial estimated value of the fundamental frequency.
Using the integration range thus obtained, for the same input signal, the fundamental frequency extraction unit based on the occupancy degree calculates the occupancy degree, harmonic structure occupancy degree, and maximum value in the same manner as the initial setting of the fundamental frequency. Extraction is performed to extract a more accurate fundamental frequency (S3).
The extraction of the degree of occupancy is performed the second time by saving the intermediate calculation result when the integration is partially performed in the process of calculating Equation (2) to obtain the initial setting value of the fundamental frequency. The first halfway result can be used without calculating the formula (2). Thereby, calculation cost can be shortened.

[Embodiment of the Invention]
Basic Frequency Extraction Method Using Power Spectrum Instead of Occupancy FIGS. 20 and 21 show a basic frequency extraction apparatus and processing procedure using the power of an input signal from which an envelope component has been removed.
First, the frequency characteristics of the input signal are transformed into those suitable for basic frequency extraction using preprocessing. To this end, for example, a high-pass filter is applied to the time-series input signal to suppress the low-frequency and emphasize the high-frequency, or conversely, the low-pass filter is applied. For example, processing that suppresses high frequencies. In the case of an input signal that has not been subjected to frequency characteristic deformation or an input signal that does not need to be corrected, this processing can be omitted. (The above is the process of S1.)
Next, the power extraction unit 51 calculates the power S (ω _c ) ² for each frequency ω _c (ω _{c1 to} ω _cn ) of the input signal. This can be calculated, for example, by taking the square of the output of each frequency bin of the STFT.

Next, the envelope component removing unit 52 removes the envelope component of the power. For example, the following method can be used. First, a discrete Fourier transform is further applied to the power (S (ω _c ) ² ) of each frequency arranged along the frequency axis (referred to as frequency characteristics). Next, the signal corresponding to the low frequency of the discrete Fourier transform is replaced with 0, and then the discrete inverse Fourier transform is performed to return the signal corresponding to the frequency characteristic. At this time, since the obtained signal is generally a complex number, the extracted real part of this signal is the power from which the envelope component is removed.

ここで、ｑは高調波の次数、ｒ（・）はｑ・ω₀を最も近い周波数ｂｉｎの
中心周波数ω_cに変換する関数、Ｅ（Ｓ（ω_c））はＳ（ω_c）の全周波数にわた
る平均値（平均値抽出部５４）である。
こうして求められた調波構造パワーの最大値を最大値抽出部５５が抽出し、以
下の式に従って、最大値を与える基本周波数を抽出する。（以上がＳ２の処理で
ある。）

なお、図２２に示したように、包絡成分抽出部を省略すれば計算精度はやや落ち
るが、その見返りとして計算コストを削減することができる。 Next, the harmonic structure power extraction unit 53 extracts the harmonic structure power S _t0 (ω ₀ ) ² based on the following equation for the power obtained by removing the envelope thus obtained.

Here, q is the harmonic order, r (·) is a function that converts q · ω ₀ to the center frequency ω _c of the nearest frequency bin, and E (S (ω _c )) is the total of S (ω _c ). The average value over the frequency (average value extraction unit 54).
The maximum value extraction unit 55 extracts the maximum value of the harmonic structure power thus obtained, and extracts the fundamental frequency that gives the maximum value according to the following equation. (The above is the process of S2.)

As shown in FIG. 22, if the envelope component extraction unit is omitted, the calculation accuracy is slightly reduced, but in return, the calculation cost can be reduced.

（ ここで、φ₁’＞ω_c1 、 φ₂’＜ω_c2 ）
ここで、ω_c1、ω_c2は、となりあった周波数ｂｉｎの中心周波数（ω_c1＜ω_c2）
、φ₁’，φ₂’はそれぞれの瞬時周波数である。また、式（５）を計算する代わ
りにφ’＝ω_c1、または、φ’＝ω_c2とすることで、計算精度はやや落ちるが計
算コストを少なくすることができる。
上記の計算と並行して、占有度抽出部６３が各周波数ｂｉｎの占有度を抽出す
る。概算基本周波数抽出部６４において、概算基本周波数を抽出する際に占有度
がすでに計算されている場合には、この処理は必要ない。 Refined Fundamental Frequency Extraction Method FIG. 23 shows a functional configuration for obtaining a more refined fundamental frequency F ′ ₀ that is roughly calculated.
When receiving the input signal, the instantaneous frequency extraction unit 61 extracts the instantaneous frequency for each frequency. From the obtained instantaneous frequency, the fixed point extraction unit 62 extracts a fixed point that satisfies the following expression and its frequency φ ′.

(Where φ ₁ '> ω _c1 , φ ₂ '<ω _c2 )
Here, ω _c1 and ω _c2 are the center frequencies (ω _c1 <ω _c2 ) of the _existing frequency bin.
, Φ ₁ ′, φ ₂ ′ are respective instantaneous frequencies. In addition, by calculating φ ′ = ω _c1 or φ ′ = ω _c2 instead of calculating equation (5), the calculation cost can be reduced, but the calculation cost can be reduced.
In parallel with the above calculation, the occupancy degree extraction unit 63 extracts the occupancy degree of each frequency bin. If the occupancy is already calculated when the approximate fundamental frequency extraction unit 64 extracts the approximate fundamental frequency, this processing is not necessary.

ここで、ｃはすべての不動点の占有度を正の値にするためのバイアスで、εは任
意の小さい正の値でよい。
この占有度を用いた基本周波数の精緻化法は、占有度の代わりに、パワ−抽出
部５１で抽出したパワーもしくは包絡成分除去部６８において包絡成分を取り除
いたパワーを用いることで、全く同様に構成することができる。図２４にその機
能構成を示す。 Finally, the refined fundamental frequency extraction unit 65 is an integer multiple of the approximate fundamental frequency F ′ ₀ (= i
) Fixed point φ′∈Φ ′ (i · F ′ ₀ ) (Φ
'(F) represents a set of fixed points in the vicinity of the frequency F. )
A value obtained by dividing the instantaneous frequency φ ′ of the fixed point by an integer (= i) is a basic frequency candidate value,
The refined fundamental frequency is obtained by calculating the average value by weighting each occupancy D ₀ (r (φ ′)). This is calculated according to the following formula.

Here, c is a bias for setting the occupancy of all the fixed points to a positive value, and ε may be an arbitrarily small positive value.
The refinement method of the fundamental frequency using the occupancy is exactly the same by using the power extracted by the power extraction unit 51 or the power from which the envelope component is removed by the envelope component removal unit 68 instead of the occupancy. Can be configured. FIG. 24 shows the functional configuration.

Each of the above-described occupancy extraction device and fundamental frequency extraction device can also function by causing a computer to execute a program. In this case, an occupancy degree extraction program for causing a computer to execute one of the occupancy degree extraction methods described in the embodiments, or a fundamental frequency extraction program for causing a computer to execute the fundamental frequency extraction method is a CD-ROM, flexible May be installed in a computer via a recording medium such as a magnetic disk or a communication line.

Experimental Example FIGS. 15A and 15B show the occupancy D ₀ (ω _c ) at each frequency bin with a solid solid line for the case where there is no noise and the case where white noise of 0 dB is added. According to the occupancy of this thick solid line, it can be seen from FIG. 15A that sharp peaks are obtained even at frequencies near the center of each harmonic component. From FIG. 15B, it can be seen that there is a sharp peak up to the third harmonic, but a peak higher than the fourth harmonic is suppressed, and the influence of white noise is large. This is in good agreement with the result of visual evaluation of the logarithmic power spectrum indicated by the broken line, indicating that the occupancy is an appropriate measure for evaluating the influence of noise.

FIG. 17A shows the fundamental frequency extraction accuracy rate of the target speech under white noise and interfering speech (the rate at which the extracted fundamental frequency is within ± 5% from the correct answer value) under white noise. Thirty sentences (total of 120 sentences) spoken by 2 males and females (total 4 persons) are used for the target speech, and the background noise is white noise alone (noise-1), and white noise is one additional person. A speech (noisy-2) including disturbing speech (total of 60 sentences for each male and female) was used. In noise-2, the powers of the two noises are the same, and the power ratio between the target speech and one of the noises is described as SNR. Method of adaptively determining the integration range (occupancy method 1), using prior information (whether the input signal is male or female) (occupancy method 2)
The cepstrum method (conventional method) is indicated by a broken line, a thick solid line, and a dotted line with □, respectively. Note that the correct fundamental frequency of the target voice is the EGG (
electro glottal graph). Also, D _p (ω _c ) was used as the degree of occupation. From both figures, it can be seen that the occupancy method 2 can extract the fundamental frequency most stably under any background noise. Also, the occupancy method 1 has little performance degradation in response to an increase in noise intensity, and the accuracy rate is high next to the occupancy method 2 near 0 dB. From this, it can be said that the fundamental frequency extraction resistant to noise can be performed by using the degree of occupation.

FIG. 18 shows a time series of basic frequencies extracted by the method of the occupancy method 1 using the cepstrum method (conventional method) and D _p (ω _c ) as the occupancy under white noise of 0 dB. 18A shows the correct answer, FIG. 18B shows the conventional method, and FIG. 18C shows the occupancy method 1. Compared with the correct answer value, the cepstrum method has a very large error, whereas the occupancy method 1 stably extracts a value close to the correct answer.
FIG. 25 shows the F ₀ correct answer rate of the target speech under background noise (the ratio at which the estimated F ₀ is within ± 5% of the correct answer value). Thirty sentences (total of 120 sentences) of 2 males and females (total of 4 persons) were used for the target speech, and white noise and multi-talker noise were used for the background noise. Multi talker noise simulates a cocktail party environment, and was created by duplicating 10 utterances randomly selected from the above 120 sentences. A basic frequency combining method that uses an occupancy to adaptively determine the integration range (using equation (1) to maximize the harmonic structure occupancy) and a refinement method using the occupancy Frequency extraction method (
The proposed cepstrum method was compared with the proposed method. The correct answer F ₀ is extracted from each EGG (electro glottal graph) signal collected at the time of voice collection using each F ₀ extraction method, and compared with F ₀ extracted from the target voice under noise. From the figure, F ₀ extraction can be performed more robustly under each SNR in the method using occupancy than the conventional method.

FIG. 26 shows a result of using the fundamental frequency extraction method using the power from which the envelope component is removed according to the embodiment of the present invention. A method that does not perform high-pass filter processing on the input signal to correct frequency characteristics before extraction processing (PowerSpec-1), a method that performs it (PowerSpec-2), and a high-pass filter only when the correct answer F ₀ is obtained The processing method (PowerSpec-3) was compared. The result is PowerSpec-3 is the best. This shows that in the method using the power of the signal from which the envelope component is removed, the preprocessing may be changed for extracting the correct answer F ₀ and the target voice F ₀ . When pre-processing is selected, it is robust against background noise.

The figure which shows the function structure of the example of an occupation degree extraction apparatus. The figure which shows the function structure of the specific example of the occupation calculation part in FIG. The flowchart which shows the procedure of the example of an occupation degree extraction method. The flowchart which shows the example of the specific procedure of the occupancy degree process in step S3 in FIG. The flowchart which shows the other example of the specific procedure of the occupation calculation process in step S3 in FIG. The figure which shows the function structure of the example of a fundamental frequency extraction apparatus. The figure which shows the function structure of the other example of a fundamental frequency extraction apparatus. The figure which shows the function structure of each specific example of the harmonic structure occupation degree calculating part 35 in FIG. The flowchart which shows the procedure of the example of a fundamental frequency extraction method. The flowchart which shows the procedure of the other example of a fundamental frequency extraction method. The figure which shows the function structure of the partial deformation | transformation of the example of a fundamental frequency extraction apparatus. The figure which shows the example of a fundamental frequency extraction apparatus provided with a sound source separation apparatus. The figure which shows the function structure of the specific example of the sound source separation apparatus 42 in FIG. The figure which shows the function structure of the conventional fundamental frequency extraction apparatus. The figure which shows the example of the instantaneous frequency of a voiced sound, a logarithmic power spectrum, and occupancy. The figure which shows the example of the periodicity of an occupancy spectrum. The figure which shows the experimental result of the correct answer rate of a prior art and each fundamental frequency extraction method. The figure which shows the experimental result of fundamental frequency extraction. The flowchart which shows the procedure of the adaptive integration range determination method, and the fundamental frequency extraction method using it. The figure which shows the function structural example of the fundamental frequency extraction apparatus of this invention which uses the power of the input signal which removed the envelope component. The flowchart which shows the procedure of the fundamental frequency extraction method using the power of the input signal, or the power which removed the envelope component, and the fundamental frequency extraction method which combined frequency characteristic correction. The figure which shows the function structure of the fundamental frequency extraction apparatus using the power of an input signal. The figure which shows the function structure of the more fundamental frequency extraction apparatus which used the degree of occupation. The figure which shows the function structure of the more detailed fundamental frequency extraction apparatus using the power of which the power of the input signal or the envelope component was removed. Experimental results of the accuracy rate comparing the fundamental frequency extraction method combining the fundamental frequency extraction method adaptively determining the integration range using occupancy and the refinement method using occupancy with the conventional cepstrum method are shown. Figure. A method that does not perform high-pass filter processing (PowerSpec-1), a method that applies power (PowerSpec-2), and a method that applies a high-pass filter only when finding a correct answer (PowerSpec) The figure which shows the experimental result of the correct answer rate in -3).

Claims (10)

A power extraction unit that calculates the power of an acoustic signal (hereinafter referred to as an input signal) such as an input audio signal or music signal for each center frequency of each frequency band;
An average value calculation unit for calculating an average value of powers of all center frequencies;
A plurality of frequencies within a frequency range in which a fundamental frequency is assumed to be present are used as candidates for the fundamental frequency, a center frequency that is close to an integer multiple of each candidate of the fundamental frequency is obtained, and the above obtained for each frequency. For each center frequency close to an integer multiple, a harmonic structure power extraction unit that calculates the sum of the power of the center frequency minus the average value;
A maximum value extraction unit that extracts the maximum value of the sum for each candidate for the fundamental frequency and outputs the frequency corresponding thereto as a fundamental frequency;
A fundamental frequency extraction device comprising: From the power of the extracted input signal, an envelope of the frequency characteristic is extracted, and an envelope component removing unit that removes the envelope from the power is provided.
2. The fundamental frequency extracting apparatus according to claim 1, wherein a fundamental frequency is extracted from the power from which the envelope component is removed.   3. The fundamental frequency extracting apparatus according to claim 1, further comprising a frequency characteristic correcting unit that corrects a frequency characteristic of an input signal as preprocessing for all processes. A power extractor for calculating the power of the input signal for each center frequency of each frequency band;
An instantaneous frequency extraction unit that extracts an instantaneous frequency of the center frequency of each frequency band from the input signal;
A fixed point extraction unit that extracts a fixed point that is a frequency at which the center frequency and the instantaneous frequency of each frequency band coincide with each other;
An approximate fundamental frequency extractor for calculating an approximate value of the fundamental frequency;
A fundamental frequency refinement unit that further refines the approximate fundamental frequency,
The fundamental frequency refinement unit selects a fixed point existing in the vicinity of an integer multiple of the approximate fundamental frequency, and calculates the power obtained by the power extraction unit for the fundamental frequency candidate obtained by dividing the frequency by an integer. A fundamental frequency extracting apparatus that extracts a more refined fundamental frequency by taking an average with a weight as a weight. A power extraction process for calculating the power of an acoustic signal (hereinafter referred to as an input signal) such as an input audio signal or music signal for each center frequency of each frequency band;
An average value calculation process for obtaining an average value of powers of all center frequencies,
A plurality of frequencies within a frequency range in which a fundamental frequency is assumed to be present are used as candidates for the fundamental frequency, a center frequency that is close to an integer multiple of each candidate of the fundamental frequency is obtained, and the above obtained for each frequency. For each center frequency close to an integer multiple, a harmonic structure power extraction process for calculating the sum of the power at the center frequency minus the average value,
Extracting the maximum value of the sum for each candidate for the fundamental frequency, and outputting the corresponding frequency as the fundamental frequency;
A fundamental frequency extraction method characterized by comprising: From the extracted input signal power, the envelope of the frequency characteristic is extracted, and the envelope component removal process of removing this from the power,
6. The fundamental frequency extraction method according to claim 5, wherein the fundamental frequency is extracted from the power from which the envelope component is removed.   7. The fundamental frequency extraction method according to claim 5, further comprising a frequency characteristic correction process for correcting the frequency characteristic of the input signal as preprocessing for all processes. A power extraction process for calculating the power of the input signal for each center frequency of each frequency band,
An instantaneous frequency extraction process for extracting an instantaneous frequency for each frequency band from the input signal;
A fixed point extraction process for extracting a fixed point that is a frequency at which the center frequency and the instantaneous frequency of each frequency band coincide with each other;
An approximate fundamental frequency extraction process for calculating an approximate value of the fundamental frequency;
With a fundamental frequency refinement process to further refine the approximate fundamental frequency,
In the fundamental frequency refinement process, the power obtained in the power extraction process is selected for a fundamental frequency candidate obtained by selecting a fixed point existing in the vicinity of an integer multiple of the approximate fundamental frequency and dividing the frequency by an integer. By taking the average as
A fundamental frequency extraction method characterized by extracting a more refined fundamental frequency.   A fundamental frequency extraction program for causing a computer to execute each step of the fundamental frequency extraction method according to claim 5.   A computer-readable recording medium on which the fundamental frequency extraction program according to claim 9 is recorded.

Images