KR20120072099A - Pitch estimation system in an integrated time and frequency domain by applying interpolation - Google Patents

Abstract

PURPOSE: A pitch estimating system using an interpolation method in a time-frequency mixing domain is provided to use the interpolation method when a time domain is mixed with a frequency domain. CONSTITUTION: A second autocorrelator(520) outputs an autocorrelated value of a frequency domain. A first interpolator interpolates an autocorrelated value of a time domain. A second interpolator interpolates the autocorrelated value of the frequency domain. A pitch estimator(540) applies a weighted value to the interpolated autocorrelated value of the time domain and the interpolated autocorrelated value of the frequency domain. The pitch estimator estimates a final pitch.

Description

Pitch Estimation System in an Integrated Time and Frequency Domain by Applying Interpolation

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pitch estimation system, and more particularly, to a pitch estimation system of a time-frequency mixed region using interpolation that can reduce the pitch double / half error occurrence rate.

In general, voiced sound is generated by vibration of the vocal cords, and the vibration of the vocal cords appears as a periodic characteristic of the voiced sound signal.

This periodic characteristic is called pitch, and it corresponds to the fundamental frequency in the short-time spectrum analysis, and the pitch characteristic corresponds to the pitch of the sound.

Currently, pitch is used for efficient speech compression of speech coders, and is being applied to speaker recognition and speech recognition methods using pitch information, and recently, it has also been used in music retrieval systems by user hum. Therefore, the study of the pitch estimation method can be seen that the ripple effect is very large in the speech signal processing system.

In the conventional time domain pitch estimation method, when the phoneme spans a transition period, the level change in the frame is severe, the pitch period is fluctuated, and the pitch estimation is difficult. Especially in the noise-mixed voice, the decision logic for pitch estimation is complicated. There is a disadvantage that the detection error increases.

Conventional frequency domain pitch estimation method is less affected by phoneme variation, fluctuations or background noise, but it is complicated to calculate because it needs to be converted to frequency domain, and processing time is long to increase the precision of fundamental frequency. There is this.

Moreover, when autocorrelation is applied to the two methods described above, the time-correlated autocorrelation frequently causes a doubling error for low frequencies, and the autocorrelation in the frequency domain has a pitch halving error for high frequencies. There is a common problem.

In order to improve this, a pitch estimation method of a time-frequency hybrid region has also been proposed, but this method also has a large computational problem.

An object of the present invention is to provide a pitch estimation system of a time-frequency mixed region using an interpolation method that can use an interpolation method when mixing a time domain and a frequency domain.

According to an aspect of the present invention, there is provided a pitch estimating system comprising: a first autocorrelator configured to autocorrelate a voiced sound signal having a predetermined frequency lower than a predetermined reference frequency in a time domain to output an autocorrelation value in a time domain; A second autocorrelator which auto-correlates the voiced sound signal having the reference frequency or more in the frequency domain and outputs the autocorrelation of the frequency domain; A first interpolation processor for interpolating the autocorrelation of the time domain; A second interpolation processor for interpolating the autocorrelation of the frequency domain; And a pitch estimator applying a weight to the interpolated autocorrelation value and the frequency domain autocorrelation value to estimate a final pitch.

According to another aspect of the present invention, there is provided a pitch estimation system, including: a first autocorrelator configured to autocorrelate a voiced sound signal to calculate a plurality of candidate pitches in a time domain; A second autocorrelator performing autocorrelation only on adjacent indices of the plurality of candidate pitches in the frequency domain; An interpolation processor configured to interpolate a calculation result of the second autocorrelator in a frequency domain; And a pitch estimator for determining a final pitch from the interpolated calculation result.

According to the present invention, when the autocorrelation of the time domain and the frequency domain is mixed, the problem that the low frequency region or the high frequency region is unambiguously expressed due to the resolution mismatch between the time and the frequency domain using interpolation can be improved, and the pitch double / The half-half error can be reduced.

In addition, the present invention can provide a high performance even by reducing the FFT coefficient, can reduce the calculation amount of the FFT operation, can significantly reduce the calculation amount for autocorrelation, can reduce the system load, and increase the processing speed have.

1A is a graph showing a voiced sound signal in a time domain.
1B is a graph showing the autocorrelation of the time domain in the time domain.
1C is a graph showing the autocorrelation of the frequency domain in the time domain.
1D is a graph showing the autocorrelation of the time-frequency mixing region in the time domain.
2A is a graph showing autocorrelation in the time domain.
2B is a graph showing autocorrelation in the frequency domain.
2C is a graph showing the autocorrelation of the frequency domain in the time domain.
2D is a graph showing the autocorrelation of the frequency domain to which the interpolation method is applied in the time domain.
3 is a graph illustrating a process of combining time-frequency domains using linear interpolation.
4 is a block diagram illustrating a pitch estimation system using IACF 1 according to an embodiment of the present invention.
5 is a block diagram illustrating a pitch estimation system using IACF 2 and IACF 3 according to an embodiment of the present invention.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms, and only the present embodiments are intended to complete the disclosure of the present invention, and the general knowledge in the art to which the present invention pertains. It is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used herein, the terms " comprises, " and / or "comprising" refer to the presence or absence of one or more other components, steps, operations, and / Or additions.

The present invention can improve the pitch doubling / half error that can occur when using only the time- or frequency-domain autocorrelation method by estimating the pitch using the auto-correlation method of the time-frequency mixed domain.

Hereinafter, the pitch estimation method using the time domain and frequency domain autocorrelation method or the time-frequency mixed domain autocorrelation method will be described with reference to graphs and equations before explaining the configuration of the present invention.

First, the time domain autocorrelation function (FACF) calculates the autocorrelation value Rt (τ) of the time domain as shown in Equation 1 below. Where x [n] is the nth voiced sound signal, τ is the delay corresponding to the pitch, and N _t is the frame length of the voiced sound signal.

Then, the pitch estimation system using the time domain autocorrelation method determines the time index corresponding to the maximum peak of the autocorrelation value of the time domain as the final pitch.

Next, the frequency domain autocorrelation function (FACF) calculates the autocorrelation value R _f (ω _τ ) of the frequency domain as shown in Equation 1 below. Where X [ω _n ] is the log magnitude of the ω _nth voiced signal, ω _τ is the delay corresponding to the fundamental frequency, and N _f is the FFT coefficient. Here, since X [ω _n ] is unnecessary information in which harmonics are not well expressed except for a large portion of energy, it may be applied by clipping to an average value.

The pitch extraction system using the frequency domain autocorrelation method determines the frequency index corresponding to the maximum peak of the autocorrelation value of the frequency domain calculated by Equation 2 as the final fundamental frequency. At this time, the fundamental frequency is an inverse of the pitch, and is an interval between harmonics caused by a window.

Here, since the fundamental frequency is inversely related to the pitch, the autocorrelation method of the time domain and the frequency domain has a limited resolution that can be expressed by sampling and FFT coefficients.

In particular, the resolution of the autocorrelation in the frequency domain depends on the frame length and the length and type of the window function. Therefore, when using a long frame length and window function, it is easy to estimate the pitch in the frequency domain. However, pitch estimation in the frequency domain is difficult to detect the pitch when the pitch is changed within the analysis frame section, and there is a disadvantage in that a large amount of calculation is required.

Therefore, in the following specification, a case where a hamming window is used and a frame having a size of 32.5 ms is used is described as an example.

On the other hand, Time-Frequency Domain Autocorrelation (TFAC) of time-frequency mixed domains combines autocorrelation values according to Equation 3 below. At this time, Rt (τ) and R _f (ω _τ ) are normalized to a maximum of 1 in consideration of the magnitude of the signal in order to combine the autocorrelation of time and frequency domain.

In Equation 3, β is a weight of the autocorrelation value in the time domain and the frequency domain, and is a parameter capable of adjusting the doubling of the pitch and the half-half error. Hereinafter, the case where β is set to 0.5 will be described as an example.

는 하기의 수학식 4와 같이 Round 함수(

)에 의해 산출되는 시간영역의 τ와 가장 인접한 ω_τ이다.Of Equation 3 above

Is a Round function (

Ω _τ closest to τ in the time domain calculated by

Hereinafter, the pitch doubling error can be improved by using the autocorrelation method of the time-frequency mixing region with reference to the graphs of FIGS. 1A to 1D.

FIG. 1A is a graph showing a voiced sound signal in a time domain, FIG. 1B is a graph showing a time domain autocorrelation in a time domain, and FIG. 1C is a graph showing a frequency domain autocorrelation in a time domain, and FIG. 1D. Is a graph showing the autocorrelation of the time-frequency mixing region in the time domain. 1A-1D, * is the maximum peak of autocorrelation.

As shown in FIG. 1B, when the time-domain autocorrelation method of Equation 1 is used, an integer multiple of the actual pitch τ is extracted as a pitch, and a pitch double error corresponding to twice the actual pitch occurs.

However, as shown in FIG. 1D, when the time domain autocorrelation method of FIG. 1B is combined with the autocorrelation method of the frequency domain of FIG. 1C, the pitch doubling error can be corrected.

On the other hand, the above-described autocorrelation of the time-frequency mixed domain has been described in the case of using ω _τ closest to the τ of the time domain by the round function when combining the time domain and the frequency domain. Can be. The autocorrelation method of the time-frequency mixed region using the interpolation method will be described below.

When combining the time domain and the frequency domain, the following two advantages can be obtained by using the interpolation method.

First, the interpolation method can improve the resolution mismatch between time and frequency domain.

As described above, the pitch and the fundamental frequency have an inverse relationship. In the time domain autocorrelation, the resolution is determined by sampling, and in the frequency domain autocorrelation, the resolution is determined by the FFT coefficient N _f . Thus, when combining the two regions, a resolution mismatch occurs.

For example, the time domain and frequency domain have different resolutions, 8 kHz and 1028-FFT. In addition, the samples in the time domain are inversely proportional intervals of N _f / n in the frequency domain, but the samples in the frequency domain have a difference expressed in direct proportional intervals in the time domain. Therefore, when using the resolutions of 8kHz and 1028-FFT, the autocorrelation of the time domain below 250Hz based on 250Hz is well represented, and the autocorrelation of the frequency domain above 250Hz is well represented.

The inconsistency between the two regions can be solved by the interpolation method. Hereinafter, a process of solving the inconsistency of the mixed region by applying the interpolation method will be described with reference to FIGS. 2A to 2D.

FIG. 2A is a graph illustrating autocorrelation in the time domain, FIG. 2B is a graph illustrating autocorrelation in the frequency domain, FIG. 2C is a graph illustrating the autocorrelation in the frequency domain, and FIG. 2D is an interpolation method. It is a graph showing the autocorrelation of the applied frequency domain in the time domain. 2A to 2D,?,?, *,?,? Are the peak indices of autocorrelation values in the time domain.

FIG. 2A shows autocorrelation values in the time domain, and in FIG. 2A, peaks of autocorrelation values are expressed at direct intervals by periodic characteristics of the voiced sound signal. Here, the case of using the sampling frequency of 8kHz as an example of the time domain autocorrelation.

In FIG. 2B, it can be seen that the peak indices of the time domain are mapped in the inverse relationship in the frequency domain.

In FIG. 2C, when the autocorrelation of the frequency domain is expressed in the time domain, it can be seen that the front part of the peak index of the time domain is indistinctly expressed.

However, as shown in FIG. 2D, when the interpolation method is additionally applied to the autocorrelation value of the frequency domain transformed into the time domain, the high frequency region corresponding to the front part of the peak index of the time domain is also well represented.

Second, using interpolation can reduce the amount of computation.

이므로, 신호의 길이가 커짐에 따라 증가한다. 뿐만 아니라, N_f의 증가분만큼 자기상관의 계산량도 늘어난다.Since the resolution of the frequency domain is determined by the FFT coefficient N _f , N _f must be increased to express the autocorrelation of the frequency domain in detail. By increasing Nf, the amount of calculation of the FFT and autocorrelation also increases. For example, if the length of the signal is N _f , the amount of computation of the FFT is approximately

Therefore, it increases as the length of the signal increases. In addition, the amount of autocorrelation increases by an increase of N _f .

By the way, when the interpolation method is applied, even if the length of N _f / 8 is used, the performance is hardly deteriorated compared with the case of using the autocorrelation value of the frequency domain using N _f .

For example, if N _f is 2048, the FFT calculation of time-frequency mixed domain autocorrelation requires 22528 multiplications, but using interpolation requires only 2048 multiplications. I can reduce it by a factor of two. In addition, the calculation amount of autocorrelation of Equation 2 may be further reduced.

The time-frequency mixed domain autocorrelation method using the interpolation method according to the present invention has the following three methods.

The first method (hereinafter referred to as Interpolated Autocorrelation Function 1) is to apply interpolation method considering both time and frequency domain. In this case, the applied interpolation method may be spline interpolation with high accuracy.

For example, a system using IACF 1 whose sampling frequency in the time domain is 8 kHz and the FFT coefficient in the frequency domain is 1024 points FFT is interpolated to the autocorrelation of the frequency domain below 250 Hz and above the time domain. Interpolate the autocorrelation values and then combine them by applying weights.

The second method (hereinafter referred to as IACF 2) finds candidate pitches (ie, peak indices) from the time-domain autocorrelation, and applies spline interpolation by performing autocorrelation of the frequency domain only on the candidate pitches. Then, in the frequency domain, it is not necessary to perform autocorrelation on all pitches, so that the calculation amount can be greatly reduced.

The third method (hereinafter referred to as IACF 3) is similar to the second method, but instead of the spline interpolation method, which is complicated to calculate, the linear interpolation method with a small amount of calculation represented by Equation 5 below is applied.

는 시간영역의 자기상관치의 피크 인덱스이며,

는 시간영역의 자기상관치이다.

는 주파수영역의 자기상관치의 피크 인덱스이며,

는 주파수영역의 자기상관치이다. 또한,

,

는

의 좌우 인접 인덱스이며,

,

는 인접 인덱스의 자기상관치이다.here,

Is the peak index of autocorrelation in the time domain,

Is the autocorrelation of the time domain.

Is the peak index of autocorrelation in the frequency domain,

Is the autocorrelation of the frequency domain. Also,

,

Is

Is the left and right adjacent index of

,

Is the autocorrelation of the neighbor index.

는 주파수영역의 자기상관치를 보간 처리한 결과로부터 산출된다.As shown in FIG. 3,

Is calculated from the result of interpolating the autocorrelation of the frequency domain.

On the other hand, since IACF 1, IACF 2 and IACF 3 have little performance difference in pitch estimation, the best method among the three methods is IACF 3, which has a small amount of calculation.

Hereinafter, the specific structure of the system which estimates a pitch using IACF1, IACF2, and IACF3 mentioned above with reference to FIGS. 4-5 is demonstrated.

4 is a diagram illustrating a pitch estimation system using IACF 1 according to an embodiment of the present invention.

As shown in FIG. 4, the pitch estimation system 40 using IACF 1 according to an embodiment of the present invention includes a first autocorrelator 410, a second autocorrelator 430, and a first interpolation processor 420. And a second interpolation processor 440 and a pitch estimator 450.

The first autocorrelator 410 autocorrelates a voiced sound signal below a predetermined reference frequency in the time domain and outputs the autocorrelation value in the time domain.

Here, the reference frequency may be a frequency that can further reduce the doubling error and half-half error according to the sampling frequency of the autocorrelation operation in the time domain and the FFT coefficient of the autocorrelation operation in the frequency domain.

The second autocorrelator 430 autocorrelates a voiced sound signal having a reference frequency or more in the frequency domain and outputs the autocorrelation of the frequency domain.

The first interpolation processor 420 interpolates the autocorrelation of the time domain using, for example, a spline interpolation method.

The second interpolation processor 440 interpolates the autocorrelation of the frequency domain using, for example, a spline interpolation method.

The pitch estimator 450 normalizes the autocorrelation values of the interpolated time domain and the frequency domain to, for example, a maximum of 1, and then detects the pitches, and applies a weight index to the detected pitches to determine a time index corresponding to the maximum peak. Determine the final pitch.

를 결정한다.In this case, the pitch estimator 450 uses Equation 3 described above, using an interpolation method.

.

Hereinafter, a pitch estimation system using IACF 2 and IACF 3 will be described with reference to FIG. 5. 5 is a block diagram illustrating a pitch estimation system using IACF 2 and IACF 3 according to an embodiment of the present invention.

As shown in FIG. 5, the pitch estimation system 50 using IACF 2 and IACF 3 according to an embodiment of the present invention includes a first autocorrelator 510, a second autocorrelator 520, and an interpolation processor 530. ) And a pitch estimator 540.

The first autocorrelator 510 calculates a plurality of candidate pitches by performing autocorrelation on the voiced sound signal in the time domain.

Here, the plurality of candidate pitches may be temporal indexes corresponding to peaks above a predetermined threshold among the pitches of the time domain. In this case, the threshold may be set in consideration of the overall size of the pitches of the voiced sound signal.

The second autocorrelator 520 performs autocorrelation only on adjacent indices of the plurality of candidate pitches in the frequency domain.

That is, when the plurality of candidate pitches are converted into the frequency domain, they do not correspond completely to the time domain, and thus the second autocorrelator 520 performs autocorrelation on the left and right adjacent indices based on the inverse positions of the candidate pitches. do.

Here, the adjacent index of the candidate pitch may be an index immediately adjacent to the left and right of the candidate pitch.

The interpolation processor 530 performs interpolation on the autocorrelated adjacent index.

In this case, the interpolation processor 530 may use a spline interpolation method, and may use a linear interpolation method of Equation 5.

The pitch estimator 540 determines, as the final pitch, the time index corresponding to the maximum peak among the pitches of the interpolated adjacent indexes.

As such, when the autocorrelation of the time domain and the frequency domain is mixed, the present invention can improve the problem in which the low frequency region or the high frequency region is unambiguously expressed due to the resolution mismatch between the time and frequency domain by using interpolation, and doubles the pitch. Reduce half-error.

In addition, the present invention can provide a high performance even by reducing the FFT coefficient, can reduce the calculation amount of the FFT operation, can significantly reduce the calculation amount for autocorrelation, can reduce the system load, and increase the processing speed have.

While the present invention has been described in detail with reference to the accompanying drawings, it is to be understood that the invention is not limited to the above-described embodiments. Those skilled in the art will appreciate that various modifications, Of course, this is possible. Accordingly, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be determined by the description of the following claims.

Claims (8)

A first autocorrelator which auto-correlates a voiced sound signal below a predetermined reference frequency in the time domain and outputs an autocorrelation value in the time domain;
A second autocorrelator which auto-correlates the voiced sound signal having the reference frequency or more in the frequency domain and outputs the autocorrelation of the frequency domain;
A first interpolation processor for interpolating the autocorrelation of the time domain;
A second interpolation processor for interpolating the autocorrelation of the frequency domain; And
And a pitch estimator that estimates a final pitch by applying weights to the interpolated autocorrelation of the time domain and the autocorrelation of the frequency domain.
Wherein the reference frequency is determined according to a sampling frequency of the first autocorrelator and an FFT coefficient of the second autocorrelator. The method of claim 1, wherein the first and second interpolation processor,
A pitch estimation system using an interpolation method including a spline interpolation method. The method of claim 1, wherein the pitch estimator,
And combining the autocorrelation values of the time domain and the autocorrelation values of the frequency domain by applying weights, and estimating an index corresponding to the maximum peak in the combined autocorrelation values as the final pitch. The method of claim 3, wherein the pitch estimator,
And normalizing the autocorrelation of the time domain and the autocorrelation of the frequency domain, and then applying the weight. A first autocorrelator for autonomously computing the voiced sound signal in a time domain to produce a plurality of candidate pitches;
A second autocorrelator performing autocorrelation only on adjacent indices of the plurality of candidate pitches in the frequency domain;
An interpolation processor configured to interpolate a calculation result of the second autocorrelator in a frequency domain;
Pitch estimator for determining the final pitch of the inverse of the frequency index corresponding to the maximum peak value from the interpolated operation result
Pitch estimation system comprising a. The method of claim 5, wherein the interpolation processor,
A spline interpolation method or a linear interpolation method for the interpolation process. The method of claim 5, wherein the first autocorrelator,
And calculating the index of the peak whose peak according to the autocorrelation operation is greater than or equal to a predetermined threshold value as the candidate pitch. The method of claim 5, wherein the adjacent index,
A pitch estimation system that is a left and right frequency index of a frequency that is an inverse of the candidate pitch.

Images