KR101333162B1 - Tone and speed contorol system and method of audio signal using imdct input - Google Patents

Abstract

The present invention relates to a device and a method of controlling the tone and speed of the audio signals using inverse modified discrete cosine transform (IMDCT) input signals. The device and the method of controlling the tone and speed of the audio signals using IMDCT input signals reduces the computational complexity and memory usage by processing input data X(k) before the input data is converted into a signal in a time domain through IMDCT when the tone and speed of the audio signals are controlled and the tone and the speed changed. The device and the method of controlling the tone and speed of the audio signals using IMDCT input signals comprise a step of determining the size of a window which determines the number of samples for extracting a frequency; a step of extracting, in the size of the selected window, a frequency (k) phase and amplitude of the input data X(k) required for an IMDCT process; a step of converting the extracted frequency, and reconstructing IMDCT input data using the converted frequency and the extracted phase and amplitude; and a step of performing IMDCT interpolation and outputting coded audio signals. [Reference numerals] (41) Window unit;(42) Frequency extracting unit;(43) Phase extracting unit;(44) Amplitude extracting unit;(45) Frequency converting unit;(46) Input data re-setting unit;(48) Synthesis filter bank;(AA) Input IMDCT X(k);(BB) Tune variation rate (R_final)

Description

Tone and Speed Contorol System and Method of Audio Signal Using IMDCT Input

The present invention relates to the pitch and speed of an audio signal, and specifically, to transform the pitch by processing the IMDCT input data X (k) before converting the signal into a time domain through an inverse modified discrete cosine transform (IMDCT). The present invention relates to an apparatus and method for varying the pitch and speed of an audio signal using an IMDCT input signal to reduce the amount of computation and memory.

In general, audio data is compressed to store a large amount of audio data on a CD, a hard disk, a mobile storage medium, or to be transmitted in a wired or wireless manner. Audio data compression methods include compression in the time domain and compression in the frequency domain.

The compression method in the frequency domain not only has a high compression ratio but also a good sound quality, so that the audio signal in the time domain is converted into the frequency domain and compressed using a psychoacoustic model and other methods.

MP3 (MPEG 3) and AAC (Advanced Audio Coding) methods use a method of compressing an audio signal in a frequency domain. In order for a human to listen to an audio signal, the compressed data must be decompressed, restored to a signal in the frequency domain before being compressed, and then converted into a signal in the time domain.

When the audio signal converted to the time domain is played back faster than the normal speed, the tone increases, and when played slowly, the pitch decreases. Therefore, in order to reproduce faster or slower than normal speed without changing the pitch, the pitch is changed by using a method such as Synchronous Overlap and Add (SOLA). When the pitch converted signal is played back at normal speed, the converted pitch is reproduced as it is. The pitch-converted audio signal can be played without changing the pitch according to the playback speed, or the pitch and playback speed can be changed simultaneously.

In the prior art, when the pitch or speed of an audio signal compressed in the frequency domain is to be changed, as shown in FIG. In this process, since the pitch or velocity is changed in the time domain, additional calculation is required and a lot of memory is required to store data of the calculation process.

In the MP3 and AAC methods, the process of compressing the audio signal in the time domain into the signal in the frequency domain is the same as in FIG.

In these schemes, the audio signal is decomposed into frequency components of various bands through a filter bank 21, and the components decomposed in the filter bank 21 are modified discrete cosine transform (MDCT) in the MDCT block 22. Through the conversion from the time domain to the frequency domain.

The signal converted into the frequency domain is quantized in the quantization unit 23 and compressed in a form of low noise and low loss through coding in the coding unit 24. The compressed data is bitstreamed along with side information in the bitstream encoding unit 25 and stored or transmitted.

The general process of converting the signal compressed in the frequency domain to the time domain is the same as in FIG.

The sidestream information and the compressed data are separated from the bitstream encoded by the bitstream decoding unit 31.

Since the side information includes information on a decoding method, the data compressed in the frequency domain may be transformed in the IMDCT block 32 into the time domain through an inverse modified discrete cosine transform (IMDCT).

Since the audio data compressed by the MP3 or AAC method is divided into several frequency bands and compressed and decompressed, the restored data is reconstructed through the synthesis filter bank 33 to be converted into an audio signal in the time domain. In the prior art, through the process as shown in FIG. 3, the audio signal is completely converted into a signal in the time domain, and then the pitch change and speed of the audio signal are changed.

The process of converting the audio signal x (n) in the time domain into the frequency information X (k) through the MDCT process is shown in Equation 1. In Equation 1, w (n) is a window function and is expressed as Equation 2, and N in Equations 1 and 2 means a window size to be analyzed. (In the following equations, N denotes the size of the analysis window.) MDCT is performed on the signal x (n) of the time domain within the analysis window to obtain the data X (k) of the frequency domain as shown in Equation (1).

An IMDCT process for converting the information X (k) transformed into the frequency domain into an audio signal in the time domain is shown in Equation 3. In this process, the frequency information X (k) may be processed to change the frequency of the audio signal, thereby obtaining an effect of changing the pitch.

In the prior art, since the audio signal in the frequency domain is converted into a signal in the time domain and then the pitch or speed is changed, a large amount of calculation is required and a lot of memory is required to store data of the calculation process.

The present invention is to solve the problem that a large amount of computation and memory is required in the process of varying the playback speed without changing the pitch and pitch of the audio signal converted to the frequency domain in the prior art, it is easy to pitch the audio signal in the IMDCT process The purpose of the present invention is to provide an apparatus and method for varying the pitch and speed of an audio signal using an IMDCT input signal by extracting various frequency components, amplitude and phase of each frequency from the IMDCT input data X (k) .

The present invention processes an input signal X (k) of an inverse modified discrete cosine transform (IMDCT) process that transforms a signal in a frequency domain into a signal in a time domain. Its purpose is to provide a pitch and speed variable device and method.

The present invention relates to a pitch and speed variable method of an audio signal converted into a frequency domain, and a preprocessing apparatus and method for appropriately converting an IMDCT input signal to enable frequency conversion in an IMDCT process, which is converted into a signal in a time domain through an IMCDT. The purpose of the present invention is to provide an apparatus and method for varying the pitch and speed of an audio signal using the IMDCT input signal by interpolating the playback speed of the audio with the IMDCT input signal preprocessing step so that the playback speed can be changed without changing the pitch. have.

The present invention extracts various frequency components before converting the frequency domain data of the audio signal converted into the frequency domain through the MDCT into the signal in the time domain through the IMDCT, and enables frequency conversion using the amplitude and phase of each frequency component. It is an object of the present invention to provide an apparatus and method for varying the pitch and speed of an audio signal using an IMDCT input signal by regenerating IMDCT input data.

The present invention provides an audio signal using an IMDCT input signal which can omit the process of pitch conversion in the time domain by changing the pitch and the speed by using the frequency information of the signal converted into the frequency domain in order to increase the compression ratio of the audio signal. Its purpose is to provide a pitch and speed variable device and method.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In accordance with an aspect of the present invention, a device for changing a pitch of an audio signal using an IMDCT input signal may include: a window unit configured to determine a window size of a sample to be processed; the IMDCT input data X (k) at a window size determined by the window unit; A frequency extractor and a phase extractor to extract a frequency and a phase of the amplifier; an amplitude extractor to extract an amplitude; a frequency converter to convert the extracted frequency; IMDCT input data using the converted frequency and the extracted phase and amplitude An input data reconstruction unit configured to reconstruct an IMDCT block for converting the reconstructed input data into a time domain through IMDCT; And a synthesis filter bank for synthesizing the various subband components transformed into the time domain through the IMDCT.

In this case, in order to determine a window required for analyzing and reconstructing X (k), which is the IMDCT input data, the window unit has a relatively small window size in comparison with other regions.

In order to determine a window necessary for analyzing and reconstructing X (k), which is the IMDCT input data, the window unit overlaps and sets an analysis window at a server band, a server band, a frame and a frame boundary, and sets a spectrum within the analysis window. Finding the integer frequency (k _in ) through the calculation is characterized by configuring the analysis window around the frequency.

In accordance with another aspect of the present invention, a method of varying a pitch of an audio signal using an IMDCT input signal is performed by processing data X (k) input to the IMDCT before converting the signal into a signal in a time domain through the IMDCT. Determining a number of samples for frequency extraction; extracting the frequency (k), phase, and amplitude of the input data X (k) required for the IMDCT process with the selected window size; Reconstructing the IMDCT input data using the converted frequency and the extracted phase and amplitude; an IMDCT step of converting the data in the frequency domain into the time domain; synthesizing the time domain signals of various frequencies generated in the IMDCT process Synthesis filter bank step; characterized in that it comprises a.

Here, in the frequency extraction step of the IMDCT input data X (k), the frequency of the IMDCT input data X (k) is divided into an integer part k _in and a fractional part ε and displayed as f = k _in + ε, Use the three frequency components X (k _in -1), X (k _in ), and X (k _in +1) to find all the spectral values present in the window to analyze, and determine the largest spectral value (S _k ) among them. It is characterized by making k _in to be the integer frequency component (k _in ).

And the fractional part ε of the frequency component,

라고 두면,

If you say,

인 경우에 대해서

About

라고 두면,

If you say,

인 경우에 대해서

About

로 정의하고, 그 비율이 특정 문턱값(threshold) λ₀ 과 비교하여 작으면 α, 그 외의 경우엔 β를 선택하여 주파수 성분의 소수부분인(ε)을 얻는 것을 특징으로 한다.Determining which of the two and using α and β is used to determine the ratio of the absolute value of the largest frequency component X (k _in ) _{in the} window and the spectral value S _k of k _in .

If the ratio is small compared to a specific threshold λ ₀ , α is selected, otherwise β is obtained, thereby obtaining (ε), which is a fractional part of the frequency component.

를 추출하기 위해서 추출한 IMDCT 정수부 주파수 성분(k_in)을 이용하여 계산하는 것을 특징으로 한다.And the phase of the IMDCT data X (k)

It is characterized by calculating using the extracted IMDCT integer frequency component (k _in ) to extract.

In the extracting of the amplitude, the amplitude A _k of the IMDCT input data is obtained by using the integer frequency component Kin and the fractional frequency component ε.

), 변환된 주파수 f_shift = f(1+R_f)을 이용하여 IMDCT 입력 X'(k)를 재구성하는 것을 특징으로 한다.In the step of reconstructing the IMDCT input data, the window size (N), frequency (f = k _in + ε), phase (from the window selection process, frequency extraction process, phase extraction process, and frequency conversion process)

), And reconstruct the IMDCT input X '(k) using the converted frequency f _shift = f (1 + R _f ).

According to another aspect of the present invention, there is provided a device for varying the pitch and speed of an audio signal using an IMDCT input signal, including: a window unit configured to determine a window size of a sample to be processed; the IMDCT input data X ( k) a frequency extractor for extracting the frequency and phase and a phase extractor, an amplitude extractor for extracting the amplitude; a frequency converter for converting the extracted frequency; IMDCT using the converted frequency, the extracted phase and amplitude An input data reconstruction unit configured to reconstruct input data; an IMDCT block for converting input data reconstructed through IMDCT into a time domain; a synthesis filter bank for synthesizing various subband components transformed into time domain through IMDCT; in the synthesis filter bank An interpolator for adjusting a sampling interval of an output audio signal to change a reproduction speed and a pitch; And a control unit.

In this case, in order to change the pitch of the audio signal in the IMDCT block, the signal is decomposed and extracted from the IMDCT input data X (k) into sine wave components, and the frequency is changed as much as desired to reconstruct the IMDCT input.

In addition, in order to determine a window required for analyzing and reconstructing X (k), which is IMDCT input data, the window unit may reduce the window size of the frequency domain to be analyzed relatively smaller than other regions.

In order to determine the window required for analyzing and reconstructing X (k), which is the IMDCT input data in the window part, the analysis window is superimposed on the server band, server band, frame, and frame boundary, and the spectrum within the analysis window. It is characterized by finding the integer frequency (k _in ) through the calculation to configure the analysis window around the frequency.

In accordance with another aspect of the present invention, a method of varying the pitch and speed of an audio signal using an IMDCT input signal processes data X (k) input to IMDCT before converting it into a signal in a time domain through IMDCT. Determining a number of samples for frequency extraction to change the pitch; extracting frequency k, phase, and amplitude of input data X (k) required for the IMDCT process with the selected window size; Converting the extracted frequency and reconstructing the IMDCT input data using the converted frequency and the extracted phase and amplitude; outputting a coded audio signal by performing IMDCT processing and interpolation.

Here, the process of separating the frequency, phase, amplitude of the IMDCT input data X (k) in the form of an equation or storing the equation in the form of a look-up table in order to change the frequency as desired in the IMDCT process. It is characterized by including.

In the frequency extraction step of the IMDCT input data X (k), the frequency of the IMDCT input data X (k) is divided into the integer part k _in and the fractional part ε and expressed as f = k _in + ε, Using the frequency components X (k _in -1), X (k _in ), and X (k _in +1), k obtains all the spectral values present in the window to be analyzed and produces the largest spectral value (S _k ) among them. _in is an integer frequency component k _in .

And the fractional part ε of the frequency component,

라고 두면,

If you say,

인 경우에 대해서

About

라고 두면,

If you say,

인 경우에 대해서

About

로 정의하고, 그 비율이 특정 문턱값(threshold) λ₀ 과 비교하여 작으면 α, 그 외의 경우엔 β를 선택하여 주파수 성분의 소수부분인(ε)을 얻는 것을 특징으로 한다.Determining which of the two and using α and β is used to determine the ratio of the absolute value of the largest frequency component X (k _in ) _{in the} window and the spectral value S _k of k _in .

If the ratio is small compared to a specific threshold λ ₀ , α is selected, otherwise β is obtained, thereby obtaining (ε), which is a fractional part of the frequency component.

를 추출하기 위해서 추출한 IMDCT 정수부 주파수 성분(k_in)을 이용하여 계산하는 것을 특징으로 한다.And phase of cosine function of IMDCT data X (k)

It is characterized by calculating using the extracted IMDCT integer frequency component (k _in ) to extract.

In the extracting of the amplitude, an amplitude A _k of the IMDCT input data may be obtained by using the integer frequency component Kin and the fractional frequency component ε.

In order to change the reproduction speed without changing the pitch, the variable speed, the sampling interval of the original signal (t _s ), and the new signal are newly generated when the original speed is 1 in conjunction with the IMDCT processing and interpolation. Sampling interval (t ' _s ) of the signal to be created, (original / variable speed) = t _s / t' _s = R _t Find R _t using the relationship (R _f x R _t ) = 1 and then determine R _f, and then change the frequency component k of the extracted IMDCT input data X (k) by f _shift = f (1 + R _f ).

And pitch and, if you wish to change the playback speed at the same time from the reproduction speed (normal speed / variable speed) = R _t R _t to obtain a frequency change rate according to the number n of half tones is desired to change from the _final R = (1 ± 0.06 ⁿ ) and R _final = R _f x R _t The frequency change rate R _f of the IMCDT preprocessing step is determined from the relationship, and the frequency is changed by using f _shift = f (1 + R _f ).

), 변환된 주파수 f_shift = f(1+R_f), 을 이용하여 IMDCT 입력 X'(k)를 구하는 것을 특징으로 한다.In the step of reconstructing the IMDCT input data, the window size (N), frequency (f = k _in + ε), phase (from the window selection process, frequency extraction process, phase extraction process, and frequency conversion process)

, IMDCT input X '(k) by using the transformed frequency f _shift = f (1 + R _f ), characterized in that.

And it is characterized in that for adjusting the frequency change amount of the frequency converter according to the variable speed, the sampling interval of the interpolation step according to the frequency change amount.

And the adjustment of the sampling interval, the original speed / variable speed, the sampling interval of the original signal (t _s), R _t = between sampling interval of the audio signal (t _'s) are regenerated by an interpolation (the original speed / variable speed ) = t _s / t ' _s is established, and if the frequency variation of the frequency converter in the previous stage of IMDCT is R _{f, the} final pitch is variable speed and (R _f xf) x R _t .

The apparatus and method for changing the pitch and speed of an audio signal using the IMDCT input signal according to the present invention have the following effects.

First, the frequency IMDCT input data X (k) may be processed to change the pitch before converting the audio signal compressed in the frequency domain into a signal in the time domain through IMDCT.

Second, by changing the pitch by processing the IMDCT input data X (k) before converting it to a signal in the time domain, the computational amount is reduced, which not only reduces the CPU burden of the system but also reduces power consumption.

Third, as the pitch conversion process of the time domain becomes unnecessary, the memory for storing data can be reduced, thereby making the hardware system cheap.

Fourth, the frequency component and phase and amplitude of the input data X (k) are separated and displayed in the form of an equation, so that the frequency can be easily converted in the IMDCT process, thereby reducing the amount of calculation.

Fifth, in the process of converting the signal in the time domain to the signal in the frequency domain by changing the pitch using information included in the signal in the frequency domain, the process of converting the pitch in the time domain may be omitted.

Sixth, by varying the size of the frequency extraction window, it is possible to extract various frequencies in detail, thereby improving the pitch conversion sound quality.

Seventh, by forming a window centered on a frequency having a large spectral component, unnecessary windows can be eliminated, thereby reducing the amount of calculation.

Eighth, by overlapping the windows at the server band or frame edge, you can extract the frequencies at those edges to improve sound quality.

Ninth, it is possible to simultaneously change the pitch change, the speed change, the pitch and the speed of the audio signal by interlocking the frequency conversion rate in the preceding step of the IMDCT with the sampling interval of the interpolator.

1 is a configuration diagram for varying the pitch or speed of an audio signal compressed in the frequency domain of the prior art
2 is a configuration diagram for converting an audio signal in a time domain into a signal in a frequency domain in the prior art;
3 is a block diagram illustrating a process of converting a signal converted into a frequency domain of the prior art into a time domain
4A is a block diagram of a pitch variable device of an audio signal using IMDCT according to the present invention;
4b is a block diagram of a pitch and speed variable apparatus of an audio signal using IMDCT according to the present invention
5 is a conceptual diagram of a window configuration according to a frequency band within one frame of IMDCT input data X (k).
6 is a flowchart illustrating a frequency component, phase and amplitude extraction process in a window according to the present invention.
7A, 7B, and 7C are conceptual diagrams of pitch conversion of an original signal according to interpolation frequency

Hereinafter, a preferred embodiment of an apparatus and method for changing the pitch and speed of an audio signal using an IMDCT input signal according to the present invention will be described in detail.

Features and advantages of the device and method for the variable pitch and speed of the audio signal using the IMDCT input signal according to the present invention will be apparent from the detailed description of each embodiment below.

4A is a block diagram of a pitch variable device for an audio signal using IMDCT according to the present invention, and FIG. 4B is a block diagram of a pitch and speed variable device for an audio signal using IMDCT according to the present invention.

The present invention provides a preprocessing step for processing an input signal to be used in the IMDCT step to vary the pitch and speed. The apparatus for varying the pitch of an audio signal using the IMDCT according to the present invention, as shown in FIG. A window section 41 for determining the number of samples to be processed, a frequency extracting section 42 for extracting the frequency and phase of the IMDCT input data X (k) at the window size determined by the window section 41, and a phase. IMDCT input data using the extractor 43, the amplitude extractor 44 for extracting the amplitude, the frequency converter 45 for converting the extracted frequency, and the converted frequency, the extracted phase and amplitude. An input data reconstruction unit 46 for reconstructing, an IMDCT block 47 for converting input data reconstructed through IMDCT into a time domain, and a plurality of subband components converted into time domain through IMDCT Synthesis filter bank 48 is included.

In addition, the apparatus for varying the pitch and speed of an audio signal using IMDCT according to the present invention includes a window unit 41 for determining a window size (number of samples to be processed) of a sample to be processed, and the window unit (as shown in FIG. 4B). A frequency extractor 42 and a phase extractor 43 for extracting a frequency and a phase of the IMDCT input data X (k) at a window size determined by 41), an amplitude extractor 44 for extracting an amplitude, and the extracted A frequency converter 45 for converting a frequency, an input data reconstruction unit 46 for reconstructing IMDCT input data using the converted frequency, an extracted phase, and an amplitude, and a time domain for the input data reconstructed through IMDCT An IMDCT block 47 for converting the data, a synthesis filter bank 48 for synthesizing various subband components transformed in the time domain through IMDCT, and an interpolation unit for changing the playback speed by adjusting the sampling interval of the audio signal (In terpolator 49).

The pitch variable method of the audio signal using the IMDCT input signal according to the present invention has a pre-processing step of processing the input signal to be used in the IMDCT step to change the pitch, to extract the size of the frequency component to finely extract the frequency component Adjusting, extracting a frequency of the IMDCT input data X (k), extracting a phase, extracting an amplitude, converting the extracted frequency according to a pitch change ratio, extracting frequency, Reconstructing the IMDCT input using phase and amplitude, an IMDCT step of converting data in the frequency domain into the time domain, and a synthesis filter bank step of synthesizing time domain signals of various frequencies generated in the IMDCT process.

In addition, the method of varying the pitch and speed of an audio signal using the IMDCT according to the present invention includes a preprocessing step of processing an input signal to be used in the IMDCT step, so as to vary the pitch and the speed, and to extract frequency components in detail. Adjusting the number of samples to be processed; extracting the frequency of the IMDCT input data X (k); extracting the phase; extracting the amplitude; and converting the extracted frequency according to the pitch change ratio. Reconstructing the IMDCT input using the extracted frequency, phase, and amplitude, the IMDCT step of converting the data in the frequency domain to the time domain, and synthesizing the time domain signals of various frequencies generated in the IMDCT process. The filter bank step and the step of varying the playback speed by adjusting the sampling interval in accordance with the rate of change of the playback speed do.

Here, in changing the frequency as desired in the IMDCT process, the calculation amount is reduced and the frequency, phase, and amplitude of the input data X (k) are separated and displayed in the form of an equation or look-up in the previous stage of the IMDCT in proportion to the frequency change amount. ) It includes the process of saving in the form of a table.

In the case of simply decoding the compressed data using IMDCT without pitch conversion, the data X (k) of the encoded frequency domain without undergoing the window process, frequency extraction, amplitude extraction, input data reconstruction, and frequency conversion process proposed by the present invention ) Can be decoded through IMDCT and synthesis filter bank.

The windowing process of the window portion 41 is as follows.

In order to change the pitch of the input signal, the frequency component of the IMDCT input data X (k) in the frequency domain should be extracted as closely as possible to reduce the loss of the frequency component. The accuracy of the extraction is determined by the size (N) of the window, which determines how many (N) X (k) samples are collected and extracted.

The method of extracting frequency components is to extract the largest frequency component within N sample windows. It is good to reduce the amount of computation to make the window as large as possible within the range of overlapping frequencies. If the window size is reduced, the frequency components can be extracted finely, but there is a problem that the amount of calculation increases.

Although the frequency band of the audio signal is known as 20 KHz, when analyzing the frequency characteristics of the audio signal, the frequency component of the low frequency region is higher than the frequency component of the high frequency region. In order to reduce the amount of calculation and extract the frequency components in detail, as shown in FIG. 5, when extracting the frequency from the IMDCT input data X (k), the window size of the low frequency region is reduced and the window of the high frequency region is increased.

을 사용하며, 여기서, f와

는 IMDCT 입력 데이터로부터 추출한 특정 주파수와 위상을 의미하며, k는 MDCT에서 주파수 인덱스, A_k는 주파수 인덱스의 진폭(amplitude)이다.The input data X (k) used in the IMDCT step is a complex form as shown in Equation 1, but the present invention

, Where f and

Is the specific frequency and phase extracted from the IMDCT input data, k is the frequency index in the MDCT, A _k is the amplitude (frequency) of the frequency index.

And separating the frequency component of the IMDCT input data X (k), and the amplitude and phase of the frequency in a simple equation form to change the frequency as desired in the IMDCT process.

In the frequency extraction step, the frequency component of the IMDCT input data is f = k _in + ε, and the frequency f is divided into the integer part k _in and the fractional part ε, and three frequency components X (k) neighboring in the window are obtained. -1), X (k), X (k + 1) is used to calculate the spectral value using Equation 4, and k of X (k) having the largest spectral value in the window is an integer frequency component k _in . do.

To know the value of the frequency component more accurately, the fractional frequency component is compared with the ratio of the absolute value of the integer frequency (k _in ) to the spectral value with the threshold value (λ ₀ ) and X (k _in ± 1) or X (k _in Calculate the fractional part using ± 2).

The phase of the extracted frequency is obtained by using the constant frequency (k _in ), the constant frequency component X (k _in ), and the neighboring frequency component X (k _in -1), and the frequency (f = k _in extracted in the step). The amplitude of the extracted frequency is obtained using + ε), X (k _in ), X (k _in -1), fractional frequency (ε), and window size (N).

And extracted IMDCT to convert the frequency (f _shift) in association with changes in the frequency f causes the pitch change in order to vary the pitch by using the phase and amplitude of each frequency and the frequency which forms the input data f _shift = f as described above ( 1 + R _f ) where R _f is the frequency conversion ratio, where positive values raise the pitch and negative values decrease the pitch.

According to the present invention, the speed change, the pitch change, the pitch and the play speed can be simultaneously changed by interlocking the pitch change through the frequency conversion in the step of IMDCT and the sampling interval of the interpolator.

In configuring the window, the window is set by overlapping the server band and the frame in order to extract the integer frequency components present in the server band and server band, and the frame and frame boundary.

The spectrum is calculated using Equation 4 in the server band or frame, and the integer frequency component is found and an analysis window composed of a certain number of samples centered on several integer frequencies (k _in ) having a large value.

The problem of how many integer frequency components are selected in one frame or subband depends on the number of samples constituting the frame or subband.

In the case of the MP3 scheme, the window is determined by the integer frequency k _in within 5 subbands within one subband.

When the audio signal in the time domain is converted into the signal in the frequency domain through MDCT, X (k) to be used as an IMDCT input is expressed as the sum of terms equal to the window size (N) as shown in Equation 1. If the window is large, such as MP3 or AAC, MDCT sine wave of a single frequency (f) as shown in Equation 5, X (k) can be approximated to one term as shown in Equation 6.

)과 진폭(A_k), MDCT의 입력으로 사용된 신호의 단일 주파수(f)로 표현됨을 알 수 있다. 시간영역의 복잡한 모양의 오디오 신호도 결국 여러 주파수의 정현파 합으로 생각할 수 있으므로 IMDCT 입력으로 사용될 X(k)의 각 주파수 성분 즉 인덱스(k)에 대한 정보(진폭, 주파수, 위상)를 추출하여 추출된 주파수를 변화 시켜 IMDCT 단계를 거치면 음정을 변화시킬 수 있다.Analyzing Equation 6, the IMDCT input is the MDCT frequency index (k) and the phase of the frequency index (

), Amplitude (A _k ), and the single frequency (f) of the signal used as the input of the MDCT. Since even a complex audio signal in the time domain can be thought of as a sine wave sum of several frequencies, it extracts and extracts information (amplitude, frequency, and phase) of each frequency component of X (k), that is, index (k), to be used as an IMDCT input. The pitch can be changed by changing the frequency to the IMDCT stage.

The frequency, phase, and amplitude extraction processes of the frequency extractor 42, the phase extractor 43, and the amplitude extractor 44 are the same as those of FIG. 6.

The frequency of the IMDCT input data X (k) can be divided into an integer portion k _in and a fractional portion ε and expressed as f = k _in + ε. Using the three neighboring frequency components X (k _in -1), X (k _in ), and X (k _in +1), all the spectral values 61 existing in the window to be analyzed are obtained and the largest spectral value is obtained. The frequency index k to make (S _k ) is defined as the integer frequency component k _in . (62)

를 구한다.(63) To determine whether the fractional frequency component is to be obtained using the immediately neighboring component X (k _in ± 1) of the integer frequency component X (k _in ) or the next component X (k _in ± 2). Ratio of spectral values centered on the integer frequency index k _in and k _in

Find (63)

Calculate the fractional frequency component ε ₁ using Equation 7 by using the component X (k _in ± 1) immediately adjacent to the integer frequency component. (64, 65) By 2 at the integer frequency component X (k _in ) Another fractional frequency component ε ₂ is obtained by using Equation 8 using the separated frequency component X (k _in ± 2). (66, 67)

인 경우에 대해서

About

인 경우에 대해서

About

가 특정 문턱 값보다 작으면 수학식 7을 이용하여 구한 소수부 주파수성분(ε₁)을 선택하고, 클 경우는 수학식 8을 이용하여 구한 소수부(ε₂)를 선택한다. (68, 69)Ratio of spectral values centered on the integer frequency k _in and k _in

If is less than a specific threshold value, the fractional frequency component ε ₁ obtained using Equation 7 is selected, and if it is large, the fractional portion ε ₂ obtained using Equation 8 is selected. (68, 69)

Using the extracted integer frequency (k _in ) and fractional frequency (ε) of the IMDCT input data X (k), the frequency f is converted into f = k _in Obtained as + ε. (71)

)은 정수부 주파수 성분 X(k_in)과 바로 인접한 성분 X(k_in±1)과 소수부 주파수 성분(ε)을 이용하여 수학식 9를 이용하여 구한다. (70)Phase of cosine function of integer part IMDCT input data X (k)

) Is obtained using Equation 9 using the component X (k _in ± 1) and the fractional frequency component ε immediately adjacent to the integer frequency component X (k _in ). (70)

And the amplitude extraction process in the amplitude extraction unit according to the present invention is as follows. (72) Substituting k _in and k _in -1 into X (k) of Equation 6, respectively, Equations 10 and 11 are obtained. Using the equations (10) and (11), an amplitude A _k as shown in equation (12) can be obtained through simple manipulation. In Equations 10 and 11, f uses extracted frequency information (k _in + ε).

), 진폭(A_k)을 분석할 수 있다.In the above process, the frequency, phase, and amplitude representing the window in one window were obtained. According to the MP3, AAC, and AC-3 methods using the MDCT method, there are several windows that require the above analysis in one frame. By utilizing the concept of the present invention, various frequency (f), phase (

), The amplitude (A _k ) can be analyzed.

When the extraction of the frequency components for one frame is completed, if the pitch is changed without changing the playback speed, i.e., if the sampling interval of the original audio signal and the sampling interval of the interpolator are the same, the extracted frequency is _shifted by f _shift = f (1+ R _f ) Where R _f is the rate of change of frequency. In general, if you raise or lower the semitone, the frequency changes by 6% of the original frequency. To increase or decrease the n-half pitch, change the frequency as f _shift = f {1 ± (0.06) ⁿ }.

If the pitch and playback speed are to be changed at the same time, find R _t from the playback speed (original speed / variable speed) = R _t , and change the frequency ratio according to the number of semitones n to be changed. R _final = (1 ± 0.06 ⁿ ) and R _final = R _f x Rate of change from R _t R _f = (1 ± 0.06 ⁿ ) / R _t Finally, f _shift = f (1 + R _f ) is obtained.

The process of regenerating IMDCT input data of the input data reconstruction unit 46 is as follows.

), 정수 주파수 성분(k_in), 진폭 정보(A_k)를 이용하여 수학식 13과 같이 IMDCT 입력 데이터 X'(k)를 다시 생성한다. 수학식 13에서 k는 IMDCT 영역의 주파수 성분이다.Frequency component (f = k _in + ε) extracted through the above process, frequency change rate (R _f ) according to pitch change, phase (

), The IMDCT input data X '(k) is regenerated using the integer frequency component k _in and the amplitude information A _k as shown in Equation (13). In Equation 13, k is a frequency component of the IMDCT region.

here

When the frame consisting of several windows is subjected to an IMDCT process for each window using Equation 13, an audio signal of a time domain in which the pitch is changed for the frame can be obtained. Using this concept, if the audio signal in the time domain is properly interpolated as needed, the effect of pitch variation, speed variation, pitch and velocity can be obtained.

The concept of change in speed and pitch change in the interpolation step will be described in detail with reference to FIG. 7.

Assume FIG. 7A as an original signal with no change in pitch.

FIG. 7B shows a frequency transformed signal, ie, a pitch transformed signal, through the IMDCT process.

In FIGS. 7A and 7B, T ₀ / T _sh is referred to as a frequency conversion ratio R _f . If R _f is greater than 1, the frequency will be higher than the original audio signal, resulting in higher pitch.

The interval t and _s, t _'s ratio t _s / t of a' _s for sampling the signal in Figure 7b and Figure 7c are defined as R _t ratio of the sampling interval. The ratio R _t of the sampling interval may be thought of as the ratio of the reproduction speed (original speed / variable speed).

The slower playback speed is when (original speed / variable speed) is greater than 1.

If a shorter sampling interval, it is possible to obtain a number of samples of data at a given time, the ratio R _t of sampling interval with respect to the signals that are not to change the pitch Greater than 1 Slow playback and low pitch. Using this concept, you can change the playback speed and pitch by adjusting the sampling interval ratio.

If the original frequency f is converted into the R _f ratio through the IMDCT preprocessing and IMDCT, the frequency before the interpolation step becomes (R _f xf). If the ratio between the sampling interval of the converted signal and the sampling interval of the interpolation step is R _t, the frequency of the final signal that has undergone the interpolation step is (R _f xf) x R _t .

After obtaining the R _t from the pitch and speed To by changing the reproduction speed without a pitch change in the adjustment device (original speed / variable _{speed) = t s / t 's} = R t of the audio signal using the IMDCT (R _f x R _t ) = 1 to determine the frequency change rate R _f of the IMDCT preprocessing step. Since the sampling interval t _s of the original signal is a known value, t ' _s can be obtained which can change the playback speed without changing the pitch.

In the case of changing the pitch of the audio signal using the IMDCT, the pitch change rate R _f is set to the frequency change rate R _f of the IMDCT preprocessing step, and the sampling interval rate is not changed.

If R _f is greater than 1, the pitch is higher; if it is less than 1, the pitch is lower. Typically, the half-step is the frequency conversion factor in accordance with the half-step count (n) of a wish to change to is imported to ± 6% change in the frequency side R _f = (1 ± 0.06 ⁿ ) and regenerate the IMDCT input X '(k) using Equation 13.

In the case in pitch and speed change device of the audio signal using the IMDCT you change the pitch and playback rate at the same time, the number of halftone wants to obtain the R _t (Fig original speed / variable speed) = R _t from the reproduction speed, changing n According to frequency change rate R _final = (1 ± 0.06 ⁿ ) and R _final = R _f The frequency change rate R _f of the IMCDT preprocessing step is determined from x R _t to regenerate the IMDCT input X '(k) using Equation 13, and the sampling interval t' _s of the signal to be regenerated is (original speed / variable speed). = t _s / t ' _s = R _t Obtain it using

The apparatus and method for varying the pitch and speed of an audio signal using the IMDCT input signal according to the present invention are to convert the frequency data X (k) input before converting the signal into a time domain through an Inverse Modified Discrete Cosine Transform (IMDCT). In order to easily change the frequency in the IMDCT process, the frequency component and amplitude of the input data X (k) are separated and displayed in the form of an equation. It includes an interpolation process that enables the simultaneous adjustment of the pitch, playback speed, pitch and playback speed by adjusting the sampling interval of the audio signal changed into the time domain.

Using the above concept, not only the pitch change but also the arbitrary frequency and the arbitrary playback speed can be obtained.

As described above, it will be understood that the present invention is implemented in a modified form without departing from the essential characteristics of the present invention.

It is therefore to be understood that the specified embodiments are to be considered in an illustrative rather than a restrictive sense and that the scope of the invention is indicated by the appended claims rather than by the foregoing description and that all such differences falling within the scope of equivalents thereof are intended to be embraced therein It should be interpreted.

41. Window section 42. Frequency extraction section
43. Phase Extraction Section 44. Amplitude Extraction Section
45. Frequency converter 46. Input data reconstruction unit
47. IMDCT Block 48. Synthetic Filter Bank
49. Interpolator

Claims (24)

A window unit for determining the number of samples to be processed;
A frequency extractor, a phase extractor, and an amplitude extractor to extract a frequency and a phase of the IMDCT input data X (k) at a window size determined by the window unit;
A frequency converter for converting the extracted frequency;
An input data reconstruction unit configured to reconstruct IMDCT input data using the converted frequency, the extracted phase and amplitude;
An IMDCT block for transforming the reconstructed input data into a time domain through an inverse modified discrete cosine transform (IMDCT);
Synthesis filter bank for synthesizing the various subband components transformed into the time domain through the IMDCT; Pitch variable device of an audio signal using an IMDCT input signal comprising a. The method of claim 1, wherein in order to determine a window necessary for analyzing and reconstructing X (k) which is IMDCT input data in the window unit,
A pitch variable device for an audio signal using an IMDCT input signal, wherein a window size of a frequency domain to be analyzed is relatively smaller than other regions. The method of claim 1, wherein in order to determine a window necessary for analyzing and reconstructing X (k) which is IMDCT input data in the window unit,
Set up the analysis window by overlapping the server band, server band, frame and frame boundary.
A device for changing the pitch of an audio signal using an IMDCT input signal, characterized by finding an integer frequency (k _in ) by calculating a spectrum _{in an} analysis window and configuring an analysis window around the frequency. In order to change the pitch by processing the data (X (k)) input to the IMDCT before converting it into a signal in the time domain through the IMDCT,
Determining a window size to determine the number of samples for frequency extraction;
Extracting frequency (k), phase, and amplitude of input data X (k) necessary for the IMDCT process with the selected window size;
Converting the extracted frequency and reconstructing the IMDCT input data using the converted frequency and the extracted phase and amplitude;
An IMDCT step of converting data in the frequency domain into the time domain;
Synthesis filter bank step of synthesizing the time-domain signal of the various frequencies produced in the IMDCT process; pitch control method of the audio signal using the IMDCT input signal comprising a. The method of claim 4, wherein in the frequency extraction of the IMDCT input data X (k),
The frequency of the IMDCT input data X (k) is divided by the integer part (k _in ) and the fractional part (ε) and expressed as f = k _in + ε, and the three neighboring frequency components X (k _in -1) and X (k _in ), Where k _in is the integer frequency component (k _in ) that obtains all the spectral values existing in the window analyzed using X (k _in +1) and makes the largest spectral value (S _k ). A pitch variable method of an audio signal using an IMDCT input signal. 6. The fractional portion ε of the frequency component according to claim 5,

If you say,

About

If you say,

About

Determining which of the two and using α and β is used to determine the ratio of the absolute value of the largest frequency component X (k _in ) _{in the} window and the spectral value S _k of k _in .

If the ratio is small compared to the specific threshold λ ₀ , audio is selected using α , otherwise β is obtained to obtain the fractional part of the frequency component (ε). Pitch variable method of signal. 5. The phase of claim 4 wherein the phase of IMDCT data X (k).

A method of varying the pitch of an audio signal using an IMDCT input signal, characterized in that it is calculated using the extracted IMDCT integer frequency component (k _in ). The method of claim 4, wherein in the extracting of the amplitude,
A method for varying the pitch of an audio signal using an IMDCT input signal, characterized by obtaining the amplitude A _k of the IMDCT input data using the integer frequency component (Kin) and the fractional frequency (ε) frequency component. The method of claim 4, wherein in the step of reconstructing the IMDCT input data,
Window size (N), frequency (f = k _in + ε), phase (from the window selection process, frequency extraction process, phase extraction process, frequency conversion process)

), IMDCT input X '(k) by using the converted frequency f _shift = f (1 + R _f ), characterized in that the pitch of the audio signal using the IMDCT input signal. A window unit for determining a window size of a sample to be processed;
A frequency extractor, a phase extractor, and an amplitude extractor to extract a frequency and a phase of the IMDCT input data X (k) at a window size determined by the window unit;
A frequency converter for converting the extracted frequency;
An input data reconstruction unit configured to reconstruct IMDCT input data using the converted frequency, the extracted phase and amplitude;
An IMDCT block for converting input data reconstructed through IMDCT into a time domain;
A synthesis filter bank for synthesizing various subband components transformed into time domain through IMDCT;
And an interpolator configured to change a playback speed by adjusting a sampling interval of an audio signal output from the synthesis filter bank, and an audio signal and a speed varying device using an IMDCT input signal. 11. The method of claim 10, wherein in order to change the pitch of the audio signal in the IMDCT block to extract the sine wave component from the IMDCT input data X (k), and to convert the frequency as desired by reconfiguring the IMDCT input, characterized in that Pitch and speed variable device of audio signal using IMDCT input signal. The method of claim 10, wherein in order to determine a window necessary for analyzing and reconstructing X (k) which is IMDCT input data in the window unit,
An apparatus for changing the pitch and speed of an audio signal using an IMDCT input signal, wherein a window size of a frequency domain to be analyzed is relatively smaller than that of other regions. The method of claim 10, wherein the analysis window is superimposed on the server band, server band, frame and frame boundary to determine a window required for analyzing and reconstructing X (k), which is IMDCT input data in the window unit,
An apparatus for varying the pitch and speed of an audio signal using an IMDCT input signal, characterized by finding an integer frequency (k _in ) by calculating a spectrum _{in an} analysis window and configuring an analysis window around the frequency. In order to change the pitch by processing the data (X (k)) input to the IMDCT before converting it into a signal in the time domain through the IMDCT,
Determining a window size to determine the number of samples for frequency extraction;
Extracting frequency (k), phase, and amplitude of input data X (k) necessary for the IMDCT process with the selected window size;
Converting the extracted frequency and reconstructing the IMDCT input data using the converted frequency and the extracted phase and amplitude;
Outputting a coded audio signal by performing IMDCT processing and interpolation; and a method of varying the pitch and speed of the audio signal using the IMDCT input signal. 15. The method of claim 14, wherein the frequency, phase, and amplitude of the IMDCT input data X (k) are separated and displayed in the form of an equation or stored in the form of a look-up table in order to change the frequency as desired in the IMDCT process. Method of varying the pitch and speed of an audio signal using an IMDCT input signal, characterized in that it comprises a process of using. 15. The method of claim 14, wherein in the step of extracting the frequency of the IMDCT input data X (k),
The frequency of the IMDCT input data X (k) is divided by the integer part (k _in ) and the fractional part (ε) and expressed as f = k _in + ε, and the three neighboring frequency components X (k _in -1) and X (k _in ), Where k _in is the integer frequency component (k _in ) that obtains all the spectral values existing in the window analyzed using X (k _in +1) and makes the largest spectral value (S _k ). A pitch and speed variable method of an audio signal using an IMDCT input signal. 17. The method of claim 16, wherein the fractional part epsilon of the frequency component is

If you say,

About

If you say,

About

Determining which of the two types is used and determining which of α and β uses the ratio of the absolute value of the largest frequency component X (k _in ) _{in the} window to the spectral value of k _in .

If the ratio is small compared to a specific threshold λ ₀ , then α is selected, otherwise β is obtained to obtain a fraction (ε) of the frequency component. How to change the pitch and speed of the audio signal. 15. The phase of claim 14, wherein the phase of the IMDCT data X (k).

A method of varying the pitch and speed of an audio signal using an IMDCT input signal, characterized in that it is calculated using the extracted IMDCT integer frequency component (k _in ). The method of claim 14, wherein in the step of extracting the amplitude,
A method for varying the pitch and speed of an audio signal using an IMDCT input signal, characterized by obtaining the amplitude A _k of the IMDCT input data using the integer frequency component (Kin) and the fractional frequency (ε) frequency component. 15. The method of claim 14,
A method of varying the pitch and speed of an audio signal using an IMDCT input signal, wherein the frequency converter and the interpolation unit are interlocked according to a variable speed and a frequency change amount to adjust a frequency change amount and a sampling interval. The method of claim 20, wherein in order to change the reproduction speed without changing the pitch,
The sampling interval of the variable speed, the original signal when the in the IMDCT processing and interpolation in connection with the step of outputting a coded audio signal to the original rate by 1 (t _s), the sampling interval of the newly created signal (t _'s), (Original speed / variable speed) = t _s / t ' _s = R _t Find R _t using the relationship (R _f Determine R _f so that x R _t ) = 1
A method of varying the pitch and speed of an audio signal using an IMDCT input signal, wherein the frequency component k of the extracted IMDCT input data X (k) is changed by f _shift = f (1 + R _f ). The frequency change ratio R _final = according to claim 20, wherein when the pitch and the playback speed are to be changed simultaneously, R _t is obtained from the playback speed (original speed / variable speed) = R _t and the number of semitones to be changed n is _final. Determine (1 ± 0.06 ⁿ ), determine the frequency change rate R _f of the IMCDT preprocessing step from the relationship R _final = R _f x R _{t and} change the frequency using f _shift = f (1 + R _f ) Pitch and speed variable method of the audio signal using the IMDCT input signal. 21. The method of claim 20, wherein in reconstructing the IMDCT input data:
Window size (N), frequency (f = k _in + ε), phase (from the window selection process, frequency extraction process, phase extraction process, frequency conversion process)

And an IMDCT input X '(k) using the converted frequency f _shift = f (1 + R _f ),. The method of claim 20, wherein the adjustment of the sampling interval,
Between the original speed / variable speed, the sampling interval of the original signal (t _s ), and the sampling interval of the audio signal regenerated by interpolation (t ' _s ), R _t = (original speed / variable speed) = t _s / t' _s If the relation is established and the frequency variation of the frequency converter in the previous stage of IMDCT is R _{f, the} final pitch is the variable speed and (R _f xf) x R _t is a pitch and speed variable method of an audio signal using the IMDCT input signal characterized in that it is determined.

Images