KR100641453B1 - Time Scale Modification method - Google Patents

Abstract

The audio speed control method according to the present invention minimizes the range and interval of the sample to maintain the quality of the basic search required by the TSM, and applies the optimized AMDF algorithm that provides a large amount of computation, thereby reducing the speed of the audio in a short time. Processing a step of finding a predetermined range having a difference of a pitch or a minimum sample value repeated by the optimized AMDF method, and expanding or reducing the frame without distortion of the signal by performing an OLA process on the pitch or the predetermined range is performed. .

The present invention produces high quality TSM results with low computation in controlling the speed of voice and music in real time.

In more detail, the present invention has the effect of suggesting an audio speed adjusting method for lowering the amount of computation as much as possible so that real-time speed control can be easily applied to any system and nevertheless not affecting the quality.

AMDF, OLA

Description

Audio Scale Adjustment Method {Time Scale Modification method}

1 is a diagram illustrating an upsampling / downsampling method which is one of conventional audio speed adjusting methods.

2 is a diagram illustrating an OLA method, which is one of conventional audio speed adjusting methods.

3 is a flowchart illustrating an audio speed adjusting method according to the spirit of the present invention.

The present invention relates to an audio speed adjusting method which can adjust the speed of audio (voice, music, etc.) from 0.5 to 2 times using a small amount of calculation.

More specifically, the present invention belongs to the field of software, algorithm, audio & speech processing, and can be used in language devices or programs, and in addition to the time of digital TV. The present invention relates to an audio speed control method that can control audio speed in various products such as time shift function and playback speed control of an MP3 player without significant damage to sound.

Algorithms for controlling the speed of audio such as voice or music can be largely divided into a method of recombining a sample and a processing method for each frame.

A representative sample recombination method is an up-sampling / down-sampling method, and a representative frame-by-frame method is an OLA and SOLA algorithm proposed by Salim Roucos of the United States in 1985. As shown in FIG. 1, the upsampling / downsampling method has a small amount of calculation and is simple, but the tone is greatly damaged, so it is not easy to recognize audio (voice or music) at 0.5x or 2.0x. On the other hand, the OLA and SOLA algorithms, which are representative of the frame-by-frame processing method, are mainly used as compared to the upsampling / downsampling methods due to the small tonal damage.

Overlapping and Add (OLA) shown in Fig. 2 is less computational and easier to recognize audio such as voice or music at 0.5x or 2.0x speed than upsampling / downsampling, but it is difficult to be applied to a real product due to distortion of a large signal. SOLA proposed together with OLA to solve such disadvantages of OLA has excellent sound quality, but it has a disadvantage that it is not easy to apply to Real Time TSM system due to large amount of computation. The basic process of SOLA is the same as that of OLA, but the difference is that the formula for finding the processing position of the OLA is compared by all the positions.

More specifically, TSM's frame-by-frame processing method, which means speed control of voice and music, has developed many algorithms such as PSOLA, which finds and processes the pitch of voice or music, and WSOLA, which performs OLA processing for signal similarity. It is under development.

The TSM stands for Time Scale Modification and mainly refers to an algorithm for controlling the speed of voice or music without a great change in timbre.

The TSM can be applied in various fields such as language and broadcasting. Here, when real-time TSM is required, the TSM processing speed is as important as quality.

Currently, TSM algorithms are actively commercialized for language use in MP3 players and PC programs, and several companies have patents in addition to the above mentioned algorithms in various places. Among them, Cosmotan Co., Ltd., a developer of commercial TSM programs in Korea, implements and patents the method using WSOLA and licenses it to various company products.

However, in order to apply the various algorithms to a real product, a method of processing a high quality TSM at an accurate magnification with a low calculation amount is required.

An object of the present invention is to generate high quality TSM results with low computation amount in controlling the speed of voice and music in real time.

More specifically, it is an object of the present invention to propose an audio speed adjusting method for lowering the amount of computation as much as possible so that real-time speed control can be easily applied to any system and nevertheless not affecting the quality.

The audio speed control method according to the present invention minimizes the range and interval of the sample to maintain the basic search quality required by the TSM, and the speed of the audio is rapidly applied by applying the optimized AMDF algorithm that provides a large amount of computation. Processing a step of finding a predetermined range having a difference of a pitch or a minimum sample value repeated by the optimized AMDF method, and expanding or reducing the frame without distortion of the signal by performing an OLA process on the pitch or the predetermined range is performed. .

The present invention produces high quality TSM results with low computation in controlling the speed of audio such as voice or music in real time.

In more detail, the present invention has the effect of suggesting an audio speed adjusting method for lowering the amount of computation as much as possible so that real-time speed control can be easily applied to any system and nevertheless not affecting the quality.

An object of the present invention is to develop a new algorithm and program to apply a good quality TSM function in real time even in a low speed processor or DSP. In the future, a language function of an MP3 player and a mobile phone or a time shift function of a digital TV is provided. Applicable to the back.

The basic pitch of audio, e.g. speech, can be found at 100Hz to 650Hz, which means that the search range for the pitch is Pmin (5/3 × (sample rate / 1000)) and Pmax (25/3 × ( Sample rate / 1000). Performing AMDF by reducing the search range of the pitch is a method commonly used in speech and the like. Here, it is possible to process the AMDF by increasing the range to a large range for accuracy and the like, and a search range of a larger range of pitch may be determined depending on the use. However, increasing the pitch search range may be a factor to increase the AMDF operation. Therefore, except in special cases, it is preferable to use the range defined above. AMDF will be described later in detail.

One feature of the present invention is that the voice signal responds more sensitively to the speed process than the music, so that the speed processing of the voice and the music is performed quickly by applying a pitch search algorithm optimized for the pitch of the voice signal.

The basic pitch search algorithm used in the present invention is an absolute magnitude difference function (AMDF) and is one of the least computational algorithms among various pitch search algorithms such as an autocorrelation method.

In general, when a loss of a sound source occurs, pitch information of a previous frame is required to obtain a residual signal which is a difference between an actual value and a predicted value of the lost sound source. The AMDF is used to find the pitch.

C (P) = ΣS (i)-S (P + i) | (from i = 0 to i = P) i + = 1

The above formula expresses AMDF. S (i) means the value of the audio sample of the buffer, and can be easily obtained by only adding and subtracting the pitch as shown in the above equation.

More specifically, the value of P such that the value of C (P) becomes the minimum value becomes the pitch value of the sound source sample. However, at this time, the C (P) value has a large value as the value of i increases due to the sigma operation, so that the correct C (P) value can be found only after dividing by the number of pitches. There is a problem that the division operation requires a considerable amount of computation.

In other words, even this simple math operation requires a large amount of computation because a large number of samples must be processed in order to implement a TSM for real time, and the processing speed also has to be delayed.

C (P) = ΣS (i) -S (P + i) | (from i = 0 to i = Pavg) i + = Pavg / 6

The above equation represents the AMDF algorithm applied to the idea of the present invention. The present invention proposes an efficient pitch search method by optimizing the conventional AMDF algorithm as shown in the above equation. The optimized AMDF method according to the spirit of the present invention maintains the equation of the conventional AMDF while minimizing the range and spacing of the comparative sample, thereby maintaining the quality of the basic pitch search required in the TSM while providing a large amount of computation.

In more detail, by performing the process of subtracting the sample size of the sound source in the pitch interval as much as Pavg, it is not necessary to perform the division operation that must be performed in parallel when performing the conventional AMDF.

In more detail, in order to obtain the minimum value of C (P) according to the conventional pitch value, the pitch value can be calculated by dividing the pitch value by the value of C (P). However, in the present invention, since the sum and difference operations are performed as much as Pavg regardless of the Pmax or Pmin value, the value of C (P) can be found without the division operation.

In addition, in the conventional AMDF algorithm, the operation was performed while increasing the value of i uniformly by 1, but in the present invention, the operation speed increases because the operation is performed while skipping Pavg by the number divided by a certain number. For example, in the case of finding the pitch while increasing the value of i by Pavg / 6, the number of operations performed to find the minimum value of C (P) is drastically reduced, thereby reducing the amount of computation and improving the processing speed. You can do it.

Hereinafter, AMDF, which is a feature of the present invention, will be described in detail. The present invention is not limited to the embodiments set forth below, and the scope of the invention will be presented by the addition, limitation, deletion, addition, etc. of components within the scope obvious to those skilled in the art to which the present invention pertains. Could be.

An optimized AMDF, which is a feature of the present invention, is used as an algorithm for finding pitch in TSM. AMDF according to the spirit of the present invention by reducing the amount of calculation by adjusting the search range, the comparison range, the comparison interval of the pitch in the equation of the conventional AMDF to thereby significantly improve the processing speed.

The search range of the pitch means a value between Pmax and Pmin as described above. The values of Pmax and Pmin may have various values according to the definition. More preferably, Pmax has a value of 25/3 × (sample rate / 1000) and Pmin has a value of 5/3 × (sample rate / 1000). This can reduce the amount of computation.

When deciding the range of pitches used in AMDF, it is generally desirable to make an accurate comparison using all the number of pitches to find, but to compare the number of pitches used in each operation as a whole, there must be a consistent range. do.

In other words. Conventionally, when searching for the minimum AMDF value, each C (P) value needs to be divided by the number of pitches for accurate comparison. However, in the present invention, it is possible to compare the AMDF values without dividing operation by defining the size of the pitch comparison range as Pavg. In a preferred embodiment, the amount of computation can be reduced by defining Pavg as 5 × (sample rate / 1000).

On the other hand, the reason for finding pitch by performing AMDF mainly on voice is that voice is more sensitive to small signal distortion when performing TSM than music. In addition, most of the speed control function is used because the voice is mainly.

However, this voice-oriented TSM does not seem to act negatively on the TSM of music. Even when the pitch search range is narrowed to the voice center, the time required for decoding the codec mainly used before the TSM is required to operate the TSM with respect to the real time to search the pitch. This is because it still has a burdensome calculation amount.

In the present invention, a method for implementing the AMDF required for the TSM through the minimum amount of computation is implemented. For this purpose, the comparison interval is defined as a delta value instead of one sample interval to perform an operation. As an example of the delta value, Pavg / 6 may be defined, and if the value of Pavg is defined as 5 × (sample rate / 1000), the proposed embodiment, the value of the delta is 5/6 × ( Sample rate / 1000). By defining the delta value, we can reduce the huge amount of sample comparison and optimize the computation.

For example, assuming a sampling rate of 48 kHz, the delta value would be 5/6 × (48000/1000) = 40 and Pavg would be 5 × (sample Rate / 1000) = 240. In this case, 240 subtraction and addition operations should be performed when applying 1 to i instead of delta to calculate AMDF. However, using a delta value requires only six subtraction and addition operations, resulting in a one-fourth reduction in throughput.

If the amount of computation needs to be reduced further, the delta value can be defined as Pavg / α. That is, the delta value is expressed as 5 / α × (sample rate / 1000), and α may be used by taking a value smaller than 6 between 2 and 5. However, since decreasing the value of α increases the distortion of the signal, it is preferable to use an α value of 6 or more unless there is a special case.

According to another aspect of the present invention, by applying the concept of PSOLA, OLA processing by pitch value can reproduce more natural reconstruction sound.

In other words. The above-described optimized AMDF method finds a range having a difference between a pitch value or a minimum sample value, and applies a method of adding or reducing the pitch value or the predetermined range by OLA processing the pitch value or the predetermined range. do.

By repeating this process, you can control the speed of audio such as voice or music in the range of 0.5 times to 2.0 times without damage to the tone. The speed ratio between 0.5 times and 1.0 times and between 1.0 times and 2.0 times can be adjusted by defining how many frames the AMDF and OLA processes are performed once.

It will be described in more detail below. Although the present invention is based on the basic algorithm of PSOLA, the invention can be easily commercialized by inventing and applying optimized AMDF.

By finding the position of the pitch or minimum AMDF value, the OLA algorithm can reduce two pitches to one or two pitches to three. In addition, the speed ratio can be freely adjusted according to how often the reduction or enlargement is performed in units of frames.

For example, if you think about how to set the 1.7x speed, you can use the optimized AMDF and OLA method to reduce the frame rate to 7x out of 10 frames.

The speed ratio ranges from 0.5 times to 2.0 times. In case of 0.5x speed, it is possible to apply optimized AMDF and OLA method to enlargement for all frames, and in case of 2.0x speed, it is possible to apply optimized AMDF and OLA method to reduced size for all frames. The execution of the optimized AMDF and OLA scheme is shown in FIG. 3. Hereinafter, FIG. 3 will be described in detail.

3 is a flowchart illustrating an audio speed adjusting method according to the spirit of the present invention.

Referring to FIG. 3, first, a sample of one frame of a file to be adjusted in speed is read by an audio speed controller (S100). Since the AMDF and the OLA method are changed according to the processing method of the frame recognized in the above step, the processing method of the frame according to the speed ratio is determined (S110). The method is a method of expanding, contracting or immutating a frame.

Considering the case of expanding the frame first, the optimized pitch is found using the optimized AMDF (S120). Next, using the OLA to expand the two pitches retrieved in the process to three (S130). Read Pointer reads samples in increments of one pitch, and Write Pointer stores pitches read from the Read Pointer, extended pitches using OLA, and samples for the two pitches in the buffer. (S140).

Next, the sum of the length of the sample accumulated in the read pointer and Pmax is compared with the size of the frame (S150). As a result of the comparison, if the sum of the sample length and Pmax accumulated in the read pointer is smaller than the size of the frame, the process returns to step S120 to search for the pitch through the optimized AMDF. On the other hand, if the size of the frame is small, the end of the frame has been reached and a new frame needs to be found.

Before searching for the new frame, it is determined whether the file is finished (S200). If the file to be extended ends, the process ends. If the file remains, the process proceeds to step S100 to search for a new frame.

When the speed ratio is unchanged in the step S110, it is not necessary to expand or reduce the frame, so only the end of the file is determined in step S200.

On the other hand, when the speed ratio is reduced in the step S110, as in the case of expansion, the pitch is found with the optimized AMDF (S160), and the two pitches are reduced to one using the OLA (S170). The read pointer reads the samples by two pitches, and the write pointer stores the samples by one pitch in the buffer (S180).

Thereafter, when the sum of the read pointer and Pmax is smaller than the frame size as in step S150, the process proceeds to step S160. Otherwise, it is determined whether the file is to be terminated (S200). When the file ends in step S200, the process is terminated. Otherwise, a new frame is searched for.

The present invention produces high quality TSM results with low computation in controlling the speed of audio such as voice or music in real time.

In more detail, the present invention has the effect of suggesting an audio speed adjusting method for lowering the amount of computation as much as possible so that real-time speed control can be easily applied to any system and nevertheless not affecting the quality.

Therefore, the present invention can be used as a language function or a time shift function in an actual digital TV, a mobile phone, an MP3 player, or the like.

Claims (10)

Set the comparison range of audio sample S (i) for AMD Magnitude Difference Function (AMF) from i = 0 to Pavg (Pavg is the set value), set the comparison interval to the Delta value, and set this set range and interval Searching for the pitch of audio based on the applied optimized AMDF algorithm; Finding a range having a difference in a pitch or a minimum sample value repeated by the optimized AMDF algorithm, and expanding or contracting an audio frame by processing the pitch or the range with overlap and add (OLA); Audio speed control method comprising a. The method of claim 1, The optimized AMDF algorithm is the audio speed control method, characterized in that for processing the speed of speech and music by applying the optimized AMDF algorithm to the pitch of the speech signal. The method of claim 1, The method of claim 8, wherein the pitch is searched by adjusting a pitch search range, a comparison range, and a comparison interval in the optimized AMDF algorithm based pitch search. The method of claim 1, The search range of the pitch is Pmax [Pmax = 25/3 × (Sample Rate / 1000)] ~ Pmin [Pmin = 5/3 × (Sample Rate / 1000)]. The method of claim 1, And calculating a difference of sound source samples having a pitch interval in the AMDF algorithm by Pavg (Pavg ≠ P) regardless of the pitch size (P). The method of claim 1, And the Pavg value is set to 5 × (sample rate / 1000). The method of claim 1, The comparison interval is a method of adjusting the audio speed, characterized in that the operation is performed by applying a delta (Delta) value rather than one sample interval. The method of claim 1, And the delta value is set to Pavg / α. The method of claim 1, The Pavg value is set to 5 × (sample rate / 1000), and the delta value is set to 5 / α × (sample rate / 1000). The method of claim 1, And the delta value is set to Pavg / 6.

Images