KR20030015579A - time-scale modification method of audio signals of which playback time is substantially acculately proportional to a designated playback-time-varying ratio and apparatus for the same - Google Patents

Abstract

PURPOSE: A method and a device for correcting a time scale of an audio signal to obtain a producing time precisely proportional to an assigned transmission rate are provided to obtain precise synchronization of audio signals with video signals by the correction of the time scale to a desired and ideal reproducing time regardless of the size of the audio signal. CONSTITUTION: A method for correcting a time scale of an audio signal to obtain a producing time precisely proportional to an assigned transmission rate includes the steps of calculating synchronization lags whenever correcting a time scale by frame and calculating an accumulation value of the synchronization lags(S30), and correcting the time scale of audio signals by employing a new transmission rate for reducing the accumulated value when the accumulated value exceeds a predetermined allowance range(S34-S36).

Description

{Time-scale modification method of audio signals of which playback time is substantially acculately proportional to a designated playback-time-varying ratio and apparatus for the same}

The present invention relates to an improvement of a technique for correcting a time scale of a signal. More particularly, the time scale of an audio signal has a reproduction time that is substantially proportional to a transmission rate specified by a user while maintaining the pitch information of the original audio signal. It relates to a method and a device for this.

Audio signal processing methods for reproducing an audio signal at a higher speed or a lower speed than a normal speed are largely classified according to whether signal processing is performed in a frequency domain or a time domain. As a method of processing a signal in the time domain, an algorithm based on the principle of time scale modification (TSM) of an audio signal has been widely used. The time scale correction algorithms, based on the overlap and addition (OLA) method introduced by Roucus and Wilgus in 1985, provide the time scale of an audio signal so that it can be reproduced at a desired speed while maintaining the original pitch information. It has been improved in the direction of correcting. Based on the OLA algorithm, Synchronized OLA (SOLA) or OLA method based on waveform similarity that finds the point of highest correlation between adjacent frames, that is, waveform similarity, and overlaps adjacent frames based on the point. That is, improved algorithms such as the overlap and add technique based onwaveform similarity (WSOLA) method have been introduced. In addition, GLS-TSM (global and local search time scale modification), SOLA algorithm (EDSOLA) by edge detection method, PSOLA (pitch-SOLA) proposed by Moulines and Charpentier, etc. have. SOLA, in particular, WSOLA algorithm is evaluated as being able to effectively overcome the weaknesses of the OLA algorithm that causes a lot of distortion of the pitch information of the original audio signal, it is currently evaluated as the most attention algorithm.

The basic idea of the OLA algorithm is to cut the original audio signal x (·) into a number of overlapping frames, modify the distance between adjacent frames according to the specified transmission rate (i.e. time scale) α _o (α _o > 1). Is a height, compression when α _o <1), and weights are added to adjacent frames to newly synthesize a time-scale-corrected signal y (·). Figure 1 shows how an audio signal is cut into frames and then synthesized again when the time scale of the audio signal is modified by applying the OLA algorithm. The original audio signal x (·) shown in Fig. 1A is a digital audio signal. When the transmission rate α _o is α _o > 1, the original audio signal is synthesized as a signal y (·) of which the overall length (that is, reproduction time) is extended as shown in FIG. 1C, and α _o <1 In the case of, the overall length is synthesized into the compressed signal y (·) as shown in FIG.

Specifically, in the state in which the original audio signal x (·) is stored in the first memory (not shown) (Fig. 1 (a)), the frames are read one by one and written out in sequence to the second memory (not shown). In this process, the time scale is modified. When reading in units of frames from the first memory, as shown in FIG. 1B, the frames are divided into a plurality of consecutive frames F ₀ , F ₁ , F ₂ , F ₃ , ..., and adjacent frames. (F _m-1 , F _m ) are read in duplicate by N-SA samples. Here, N represents the number of samples constituting one frame, and SA is an integer value smaller than N, and represents the sample interval of the starting point of adjacent frames of the original audio signal. In this manner, each of the frames read out from the first memory by dividing the frames is divided into adjacent frames, as shown in (c) or (d) of FIG. 1, in order to modify the time scale according to the specified shift ratio α _o . The second memory is written so that the sample interval of the starting point is SS, that is, N-SS samples are overlapped between adjacent frames. And when repositioning the interval between adjacent frames in this way, as shown in (e) of Figure 1, the samples of the overlapping section (10a, 10b) of the adjacent frame is added by applying a weight, and the previous frame (F _m- Samples of the section 10c of the current frame F _m not overlapping with ₁ ) are copied as they are. In this case, SS has a relationship of SS = α _o SA and is appropriately determined in consideration of the maximum value of the transmission rate, the number of samples constituting one frame, and the minimum number of samples to overlap between adjacent frames. And a sample interval of a starting point of adjacent frames of the time-scale modified audio signal y (·).

Since the OLA algorithm corrects the time scale in proportion to the specified speed change rate, the reproduction speed or reproduction time of the time scale-modified audio signal is also changed in proportion to the specified speed change rate. This characteristic of the OLA algorithm is that, in shifting and reproducing multimedia data in which an audio signal and a video signal are integrated, if the audio signal is shifted by applying the OLA algorithm, the synchronization of the video signal and the audio signal can be guaranteed accurately. There is a need. However, since the OLA algorithm does not consider any correlation such as waveform similarity between adjacent frames when modifying the time scale of the audio signal, the OLA algorithm has a disadvantage of distorting the pitch information of the original audio signal. The spectral characteristics or pitch periods of an audio signal cannot be guaranteed to remain intact by simply overlapping adjacent frames at any point. Further clicks, burst of noise, reverberation, etc. may appear in the time scale corrected signal. As a result, the quality of the reproduction sound of the time-scale corrected signal is significantly worse than that of the original audio signal.

There is a need for a method capable of modifying the time scale while maintaining the pitch information of the original audio signal as much as possible. As a representative example of such a method, an SOLA or WSOLA algorithm has been proposed. The SOLA to WSOLA algorithm basically follows the OLA algorithm, but when the original audio signal is divided into a plurality of consecutive frames in order to minimize the distortion of the pitch information of the original audio signal, the SOLA to WSOLA algorithm is most closely examined. It is a method of minimizing the sound quality degradation of the pitch information of the original audio signal by overlapping two adjacent frames at a location where good similarity, that is, a best correlation is obtained. That is, when the SOLA or WSOLA algorithm divides the original signal into a plurality of frames and reads from the first memory, each frame is not always read from the mSA-th sample, which is an integer multiple of SA, but the maximum correlation between adjacent frames. NK _m samples are read from the point where the displacement (which is called the synchronization lag K _m ) is applied, i.e., the mSA + K _m th sample. When each frame thus read is written into the second memory, the interval of the start point of the adjacent frame always maintains SS sample intervals as in the above OLA method. As a result, the time-scale modified signal stored in the second memory is concatenated to have the maximum correlation between adjacent frames, so that the pitch information of the original audio signal is kept almost the same.

According to this method, however, unlike the OLA algorithm, the length of each frame recorded in the second memory varies depending on the size of the synchronization delay K _m , so that the overall sample length of the time-scale-corrected audio signal (that is, the playback time) It is to be noted that is not exactly the same as the product of the original audio signal and the specified transmission rate. In other words, the speed ratio is actually the result of applying a value different from the speed ratio SS / SA of the OLA algorithm, that is, [SS + (average value of K _m )] / SA. In addition, it can be seen that the actual transmission rate is applied to (SS + K _m ) / SA for each frame. If the sign of the synchronization delay K _m for each successive frame (F _m-1 , F _m , F _{m + 1} , ...) in a section of the audio signal is the same, then the absolute value of the The cumulative value does not cancel and continues to increase, so that the reproduction length (i.e., reproduction speed) of the time-scale-corrected signal corresponding to the interval is largely different from the reproduction length (i.e., reproduction speed) in proportion to the specified speed ratio. Will have Such irregularities in the playback speed may not be a big problem when shifting and playing only audio signals. However, the SOLA or WSOLA algorithm is applied to shift and reproduce multimedia signals in which video and audio signals are integrated. In the case of correcting the time scale, the video signal and the audio signal are out of sync. This is because the conventional video signal shifting method corrects the number of video frames in proportion to the designated speed ratio, so that the shifted video signal is reproduced at a constant speed with a size that is exactly proportional to the designated speed ratio. If the degree of inconsistency is high, it causes a fatal problem in which the speaker's mouth on the video screen does not match the playback sound of the speaker.

In view of the above, the present invention provides a method of correcting the time scale of an audio signal such that a signal obtained by time-scaling the original audio signal has a reproduction time that is almost exactly proportional to a specified transmission rate while preserving the pitch information almost intact. It is a primary object to provide.

It is also a second object of the present invention to provide an apparatus which makes it possible to implement such a method of correcting the time scale of an audio signal.

Figure 1 shows how an audio signal is cut into frames and then synthesized again when the time scale of the audio signal is modified by applying the OLA algorithm.

FIG. 2 illustrates the concept of modifying the time scale of an audio signal according to the time scale method of the present invention when the specified shift ratio α _o is much larger than 1, that is, α _o > SS / (SS-L). Drawing.

FIG. 3 illustrates the concept of modifying the time scale of an audio signal according to the time scale method of the present invention when the specified shift ratio α _o is slightly larger than 1, that is, α _o <SS / (SS-L). Drawing.

4 is a block diagram illustrating a configuration of an apparatus for executing a time scale correction method of an audio signal according to the present invention.

5A and 5B are flowcharts illustrating a procedure for executing a method for correcting a time scale of an audio signal according to the present invention.

FIG. 6 illustrates a speed change correction concept for forcibly converging a reproduction time of a time scale modified audio signal to a reproduction time ideally proportional to a specified transmission rate in a process of executing a time scale correction method of an audio signal according to the present invention. do.

7 is a waveform diagram of an original audio signal having a reproduction length of 20 seconds.

FIG. 8A is a waveform diagram of a signal obtained when the audio signal of FIG. 7 is time-scale corrected using the WSOLA algorithm to which only the scan range correction concept described above is applied when the specified shift ratio α _o is 1.2.

FIG. 8B is a waveform diagram of a signal obtained when the audio signal of FIG. 7 is time-scale corrected using the WSOLA algorithm to which a scan range correction and a shift rate correction concept are applied together when the designated shift ratio α _o is 1.2. .

FIG. 9A is a waveform diagram of a signal obtained by time-scale correcting an audio signal of FIG. 7 using a WSOLA algorithm to which only a scan range correction concept is applied when a designated shift ratio α _o is 2.0.

FIG. 9B is a waveform diagram of a signal obtained by time-scaling the audio signal of FIG. 7 using the WSOLA algorithm that applies a scan range correction and a shift rate correction concept when the designated shift ratio α _o is 2.0.

** Explanation of symbols for main parts of drawings **

100: signal processor

110: variable speed engine

120: first memory

130: second memory

According to an aspect of the present invention for achieving the first object,

The audio signal is divided into a plurality of consecutive frames, and adjacent frames are separated by a first number of samples, and the interval of the adjacent frames is compressed or extended so that a second number of samples is overlapped between the separated adjacent frames. And modifying a time scale, wherein the second sample number is determined by further reflecting a synchronization delay obtained at a point of maximum correlation between adjacent frames by a value obtained by multiplying the first sample rate by a user-specified initial transmission rate. In the time scale correction method of an audio signal,

In particular, each time the time scale is modified by one frame, the synchronization delay is calculated, and the cumulative value of the synchronization delay is calculated by adding up the previously accumulated synchronization delay, and when the cumulative value exceeds a predetermined tolerance range, the cumulative value can be reduced. Provided is a time scale correction method of an audio signal, wherein the time scale of the audio signal is modified by applying a new speed ratio.

The tolerance range is preferably set within a range in which the degree of synchronization mismatch between the playback picture and the playback sound is unnaturally felt when playing a multimedia signal in which an audio signal and a video signal are integrated.

In addition, when the cumulative value is reduced by the time scale correction to which the new speed change rate is applied and falls within the reset range defined within the tolerance range, the new speed change rate is preferably restored to the initial speed change rate and applied. .

In the correction method, the maximum correlation point is determined at the point with the highest waveform similarity between two adjacent frames within a predetermined scan range.

Further, the scan range is determined to be a range sufficient to find the maximum waveform similarity between two adjacent frames in consideration of the upper limit value of the initial transmission rate, etc., wherein the start point (mSA) of the current frame of the audio signal is outside the scan range. It is desirable to define the scan range to be located at.

According to another aspect of the present invention for achieving the first object,

In order to maintain the original pitch information of the audio signal while using overlap and add principle, successive frames are used to correct the time scale of the audio signal at a user-specified transmission rate. In a method of modifying the time scale of the audio signal using a predetermined algorithm of a method of concatenating two adjacent frames at the maximum point,

Whenever the time scale is modified by one frame by applying the predetermined algorithm, the best cross-correlation between the current frame of the audio signal and the last frame of the time scale-corrected signal is determined within a predetermined scan range. Finding a synchronization lag that is a sample interval between a representative point and a center point of the scan range;

Calculating a cumulative value of the new synchronization delay by adding the found synchronization delay to the previous accumulated value of the synchronization delays; And

Comparing the calculated cumulative delay value with a predetermined tolerance range, if the cumulative value is out of the tolerance range, the applied transmission rate can be reduced from the next frame and converged within the tolerance range from the subsequent frame. There is provided a time scale correction method of an audio signal, wherein the time scale is corrected by correcting with a new speed ratio.

In the above correction method, preferably, the predetermined algorithm is SOLA, WSOLA, or an equivalent algorithm that defines a point having the highest waveform similarity while sliding two adjacent frames within the scan range.

In the above correction method, the difference SS-SA of the starting point sample interval SA of the adjacent frame of the audio signal is calculated from the starting point sample interval SS of the adjacent frame of the time scale corrected signal, and the scan range is | SS-SA | > L, the scan range is defined as a range including L samples left and right about the SA m skipped sample mSA from the start of the current frame, that is, mSA-L to mSA + L, otherwise It is preferable to define mSA- (SS-SA) to mSA + 2L- (SS-SA). The variables L and m represent half of the scan range and frame index, respectively.

In the correction method, the cumulative value of the synchronization delay is periodically calculated even while the time scale of the audio signal is corrected by applying the new shift rate, and the subsequent period depends on whether the calculated cumulative value converges within the tolerance range. It is preferable to further include the step of adjusting the speed change to be applied from.

Further, the correction method, when the cumulative value is reduced by the time scale correction to which the new speed change rate is applied and falls within a reset range defined as a narrower range within the tolerance range, the user sets the new speed change rate. It is preferable to further include the step of applying again to the speed change rate.

Further, in the correction method, the default value of the correction mode is set to off, the correction mode is changed to on when the magnitude of the accumulated value of the synchronization delay exceeds the tolerance range, and the correction mode is on. If the cumulative value of the synchronization delay increases compared to the cumulative value of the synchronization delay up to the previous period, check that the cumulative value of the synchronization delay exceeds the tolerance range in the next period, and if it decreases, turn on the correction mode. It is preferable to further include the step of maintaining so that the transmission rate applied in the current cycle is applied as it is.

According to another aspect of the present invention for achieving the first object,

The original audio signal is stored in the first memory, the original audio signal is read out one frame from the first memory, and the second audio is rearranged so that the time scale of adjacent frames is modified according to the initial transmission rate set by the user. In the time scale correction method of an audio signal stored in

Read one by one frame from the original audio signal of the first memory, the current frame is to read N + L samples from the mSA-L th sample so that a predetermined sample overlaps with the previous frame, The variable N is the number of samples constituting the unit frame, the variable SA is defined as SS / α, the variable SS is defined as the number of samples between the start points of the successive frames of the time-scale modified audio signal, α represents an applied transmission rate for each frame period, the initial transmission rate or a modified transmission rate, the variable m represents a frame index, and L represents a number of samples corresponding to half of a predetermined scan range;

Calculating a waveform similarity between the last frame of the time-scale-modified audio signal and the read current frame while sliding the current frame within the scan range;

Each frame period, the method comprising calculating a synchronization delay K _m, the synchronization delay K _m is the number of samples between the time scale of having the maximum value among the calculated waveform similarity point and the cycle correction signal starting point (mSS) Defined by the difference of;

NK _m samples to which the synchronization delay K _m calculated in the read current frame is applied are added to the last frame of the second memory from the mSS-th sample point, and the duplicated samples of both the current frame and the last frame are weighted. Summing by applying;

Each time a new synchronization delay K _m is calculated, adding a cumulative value of the synchronization delays up to the previous frame period to newly calculate the synchronization delays up to the current frame period; And

Checking whether the calculated new synchronization delay cumulative value is out of a predetermined tolerance range, and if it is confirmed that the calculated new synchronization delay cumulative value is out of a predetermined tolerance range, forcing the synchronization delay cumulative value to fall within the tolerance range by adjusting a value of the applied transmission rate from the next frame period. Provided is a time scale correction method of an audio signal, comprising: a.

In the correction method, if the cumulative synchronization delay value increases in the negative direction and falls outside the tolerance range, the applied shift ratio α is corrected to a value smaller than the initial transmission ratio α _o , and the accumulated synchronization delay error is positive. It is preferable to correct the applied transmission rate α to a value larger than the initial transmission rate α _o when it is increased in the direction of and falls outside the tolerance range.

In the correction method, a part of the tolerance range is defined as a reset range so that the synchronization delay accumulated value is within the tolerance range, and the synchronization delay accumulated value is out of the tolerance range and then modified. As a result of correcting the time scale by applying the speed change rate, when the synchronization delay cumulative value decreases and falls within the reset range, it is preferable to restore the applied speed change rate to the initial speed change rate α _o .

Further, in the above correction method, the scan range is a range including L samples from left to right around the mSS point when | α _o |> SS / (SS-L), that is, mSS-L to mSS +. When defined as L and | α _o | ≤SS / (SS-L), it is preferred to define mSS- (SS-SA) to mSS + 2L- (SS-SA).

On the other hand, according to an aspect of the present invention for achieving the second object,

The audio signal is time-scaled according to a user-specified transmission rate, but the time of the audio signal is based on the principle of finding and reflecting the best correlation while sliding two adjacent frames within a predetermined scan range. In the device for modifying the scale,

Memory means for storing the original audio signal and the audio signal corrected in time scale based on a transmission rate set by a user; And

The audio signal is divided into a plurality of consecutive frames, and adjacent frames are separated by a first number of samples, and the interval of the adjacent frames is compressed or extended so that a second number of samples is overlapped between the divided adjacent frames. A point indicating the best cross-correlation between the current frame of the audio signal and the last frame of the time-scale-corrected signal within a predetermined scan range, particularly when the time-scale is corrected one frame at a time. And calculating the synchronization delay which is the sample interval between the center point of the scan range and adding the synchronization delay accumulated up to the previous frame period, calculating the accumulation value of the synchronization delay, and comparing the calculated accumulation delay with the predetermined tolerance range. If the cumulative value is out of the tolerance range, the shift is applied from the next frame. And a signal processing means for correcting the time scale by correcting the rate to a new speed change rate that can reduce the cumulative value of the synchronization delay to converge within the tolerance range. .

In the apparatus, the signal processing means calculates the difference SS-SA of the starting point sample interval SA of the adjacent frame of the audio signal from the starting point sample interval SS of the adjacent frame of the time scale corrected signal, and the scan range is | SS-SA | > L, the scan range is defined as a range including L samples left and right around the sample mSA skipped SA from the start of the current frame, that is, mSA-L to mSA + L, otherwise MSA- (SS-SA) to mSA + 2L- (SS-SA). Wherein the integers L and m represent half the scan range and the frame index, respectively.

Further, in the apparatus, the signal processing means periodically calculates a cumulative value of the synchronization delay while the time shift of the audio signal is corrected by applying the new shift rate, and the calculated cumulative value falls within the tolerance range. Depending on whether it is converging or not, it is advisable to readjust the transmission rate to apply from a later cycle.

Further, in the above apparatus, the signal processing means is adapted to reduce the cumulative value by a time scale correction to which the new speed change rate is applied, and to enter the reset range defined as a narrower range within the tolerance range. It is desirable to recover and apply to the transmission rate set by the user.

According to the present invention as described above, the time scale of the audio signal is corrected based on the initial transmission rate designated by the user, and the cumulative error of the playback time occurring in such a correction process is checked every cycle, and the tolerance is set in advance. When the range is exceeded, the magnitude of the shift ratio applied from the next period is corrected to force the cumulative error of the reproduction time to immediately converge within the tolerance range. As a result, the playback time (i.e., playback speed) of the time-scale-corrected signal has almost no error with the ideal playback time (i.e., playback speed), which is always proportional to the initial transmission rate specified by the user at any point in time. Therefore, when the present invention is applied to the variable speed reproduction of the multimedia signal in which the video signal and the audio signal are integrated, the synchronization of the reproduction sound and the reproduction screen can be maintained accurately.

Hereinafter, a method of processing an audio signal according to the present invention will be described in more detail with reference to the accompanying drawings.

4 is a block diagram exemplarily illustrating a configuration of an apparatus for applying a time scale correction method of an audio signal according to the present invention. The method of the present invention includes a memory processor 120 and 130 capable of storing digital audio signals, and a signal processor 100 (or programmed to perform signal processing for modifying a time scale by reading digital audio signals stored in the memory). A program having such a signal processing function may be executed in an apparatus having a signal processor 100 capable of loading the program from the data storage 140 such as a hard disk and executing the program. The signal processor 100 may be, for example, a digital signal processor (DSP), a microcomputer, or a central processing unit (CPU).

The original audio signal x (·) is a digital signal expressing the intensity of sound with respect to time as an integer stream, which is preferably normalized by its maximum value and made into a bit stream. This audio signal is stored in the first memory 120 prior to the time scale correction. The speed change engine 110 is executed by the signal processor 100 as a program capable of time scale correction according to the present invention. The signal processor 100 sequentially reads the audio signals stored in the first memory 120 one frame at a time, performs a process for time-scale correction according to the specified shift rate α _o , and stores each processed frame in a second memory ( 130) sequentially.

5 is a flowchart illustrating the concept of a time scale correction method of the present invention executed by the speed change engine 110. The time scale correction method of the present invention will be described in more detail with reference to this flowchart.

First, the number N of samples constituting a frame, the variable SS, and the like are set to appropriate constant values A and B based on the concept described above. It also defines the scan range for searching the maximum correlation. The initial scan range sets the lower limit K _min and the upper limit K _max to mSS-L to mSS + L, respectively. Where variable m is the frame index, variable L is the number of samples that make up half of the scan range, the maximum value of the variable rate, the number of samples that make up one frame, and the minimum number of samples that need to overlap between adjacent frames The value is appropriately determined in consideration of the number and the like. In addition, the value α _o of the speed change designated by the user through the input means (not shown) is assigned to the speed change variable α, and the variable Err indicating the cumulative value of the synchronization delay K _m for each frame is initialized to 0, and the speed change is corrected. The correction mode (rev_mode) indicating whether or not is set to off. Further, the value of the variable SA is calculated in advance using the relation SA = SS / α (step S10). When a new speed change α is input in the time scale correction process, the transmission engine 110 is interrupted and the above initialization is repeated based on the new speed change rate.

Next, it is determined whether or not the scan range is corrected according to the size of the designated shift rate, and when the correction is necessary, the initially set scan range is corrected (steps S12 and S14).

To this end, first, the difference between the value of the variable SS and the value of the variable SA (lag = SS-SA) is compared with the size L of half of the initially set scan range, and the final scan range is determined according to the comparison result. ). The value of half L of the scan range is the absolute value of the difference | SS-SA | If it is larger, it is not necessary to correct the scan range, so the initial set scan range is maintained. However, when the magnitude of the absolute value | SS-SA | is smaller than L, the scan range is modified as follows in order to correct the time scale in proportion to the designated speed ratio.

K _min = mSS-lag = mSS-(SS-SA)

K _max = mSS + 2L-lag = mSS + 2L-(SS-SA)

The reason why the scan range needs to be corrected is that in the original scan range, the time scale correction effect corresponding to the designated shift rate cannot be obtained. If there is no limit of the scan range, the point with the maximum correlation is the point before the frame sliding to perform the maximum correlation, that is, the overlap point in the original signal ((32) in FIG. 3, which is the original maximum correlation). Will also be referred to as a point). By the way, | SS-SA | ≤ L, i.e., if the designated transmission rate α _o ≤ SS / (SS-L), the original maximum correlation point 32 is located in the original scan range 34 (Fig. 3 (b)). ), The actual maximum correlation point may also be found at a point coinciding with the original maximum correlation point 32. At this point 32, the synchronization delay K _m becomes 0, and the actually applied transmission rate becomes 1, so that the desired time scale correction effect cannot be obtained. Therefore, in order to perform the time scale correction corresponding to the designated shift rate, it is necessary to correct the preset scan range 34 so that the original maximum correlation point 32 is located outside the scan range. As an example of a method of modifying the scan range, the entire original scan range 34 is moved by at least L-lag (but lag = SS-SA) to the right as shown in FIG. The new scan range 36 thus obtained is mSS-lag to mSS + 2L-lag, and the original maximum correlation point 32 is always outside the new scan range 36 so that the actual applied speed ratio is Converging to 1 does not occur.

After performing such a preceding action, the signal processor 100 reads the first frame F ₀ of N samples from the first memory 120 in which the original audio signal x (·) is stored, and reads the first frame F ₀ into the second memory 130. Signal processing for time scale correction is started from the copy as it is (step S16). Since one frame is processed, the frame index variable m is set to 1 (step S18).

Next, the loop iterates once and processes one frame at a time. The current frame F _m of the original audio signal x (·) is taken and the synchronization delay K _m is calculated based on the highest correlation between the frame and the time scale modified signal y (·). Calculation of the synchronization delay K _m is performed by finding the point with the current frame F _m y (m * SS) to slide on the periphery of the current frame F _m and the time scale modified signal with the highest correlation. At the point with the highest correlation, the waveform similarity of both frames is the highest. Thereafter, the current frame F _m is positioned at the point, and the second memory 130 is subjected to time scale correction. At this time, the portion overlapping with the previous frame is added by applying a weight, and the remaining portion of the current frame F _m is simply copied. We run this loop repeatedly, modifying the timescale by one frame. This will be described in more detail as follows.

First, the signal processor 100 reads one frame from the second frame from the first memory 120 and calculates a synchronization delay of the frame. Specifically, when the second frame F ₁ is read out, the starting point of the frame becomes the SA-L-th sample, and from this point, N + L samples are read. In general terms, the current frame F _m read out from the first memory 120 by the signal processor 100 to find the synchronization delay K _m is composed of mSA-L to mSA + Nth samples (FIG. 2). Or (a) of FIG. 3). That is, at least L samples are additionally read out from the N samples (step S19). The synchronization delay K _m is obtained for the current frame F _m thus read (step S20). The synchronization delay K _m of the current frame F _m represents the number of samples between the point having the maximum correlation with the last frame of the time scale correction signal y (·) and the center point mSS of the scan range. Therefore, to calculate the synchronization delay K _m , _first, the correlation between the current frame F _m and the last frame F _m-1 of the time-scale modified signal is calculated. The correlation is calculated by using the following correlation equation while sliding the current frame F _m and the last frame F _m-1 of the time-scale modified signal within the set scan range.

Here, Lo denotes the number of samples of an overlapping portion between the adjacent frame x (mSS + j) and y (mSA + j + K _m ) when added as a time scale corrected signal.

Among the calculated correlation values, the point having the maximum correlation value is found using the following equation, and the sample interval between the point and the center point mSS of the scan range is determined as the synchronization delay K _m (see FIG. 2 or FIG. b)).

The maximum correlation is calculated using the correlation calculation below.

In addition, when the synchronization delay K _m is determined, the signal processor 100 changes only the NK _m samples (20b of FIG. 2 or 30b of FIG. 3) from the mSA + K _m th sample to the time scale modified signal of the second memory 130. Is added from the mSS-th sample point of p and the remaining samples (20a in FIG. 2 or 30b in FIG. 3) are discarded. At this time, the previous frame of the time-scale modified signal and the current frame of the original audio signal are expressed as follows.

Here, g (j) represents a weighting function for summing by applying the weight of overlapping intervals. As a representative example, g (j) may be a linear ramp function as follows, but an exponential function or other appropriate function may be used. You can also choose.

g (j) = 0, j <0;

g (j) = j / Lo, 0 <j <Lo;

g (j) = 1, j> Lo

Each time one frame is processed, the value of the frame index m is increased by one (step S24). Then, the above loop is repeatedly executed until the end of the original audio signal x (·) is reached (step S26). As a result, the original audio signal x (·) is basically synthesized into a signal y (·) corrected in time scale according to the transmission rate α set by the user.

However, the above processing alone can almost maintain the pitch information of the original audio signal x (·), but there is no guarantee that the playback time of the time-scale corrected audio signal, that is, the playback speed, is exactly proportional to the specified shift rate α _o . In particular, the closer the speed ratio α _o is to 1, the greater the difference from the reproduction time which is almost exactly proportional to the specified transmission rate α _o (this is called the ideal reproduction time). In order to solve this problem, further processing is required so that the actual reproduction time of the time scale corrected signal y (·) has only a slight error compared to the ideal reproduction time at any point in time. This will be described in more detail.

According to the conventional SOLA or WSOLA algorithm, when the designated shift ratio α _o is close to 1, the cumulative value of the synchronization delay K _m also increases as the processed frames increase. If the value of the synchronization delay K _m as described above is large, it is highly likely that the inaccuracy of the time scale is large, and therefore, the synchronization delay K _m is preferably found within the limited scan range mSS-L to mSS + L. In general, the probability that the sign of the synchronization delay K _m of each frame becomes negative or positive, but the magnitude thereof is irregular and cannot be predicted. Fortunately, if the synchronization delay K _m of each frame evenly changes the negative and positive values throughout the audio signal, the accumulated synchronization delay value does not grow so much in any interval, so that it can be synchronized with the video signal naturally. However, there is no guarantee that this will always happen. In particular, if the designated shift ratio α _o is close to 1, the problem of synchronization mismatch with the video signal is more significant. In this case, the synchronization delay K _m of each frame has a very high probability of having the same code in succession. Therefore, the cumulative value of the synchronization delay K _m tends to increase without being offset as the number of frames to be processed increases. As a result, after a certain point of time, the cumulative value of the synchronization delay causes a phenomenon that the degree of synchronization mismatch between the video signal and the audio signal is unacceptably large.

The reason why such a unidirectional synchronization delay error appears is as follows. According to the basic concept of the SOLA algorithm or the WSOLA algorithm, the larger the speed ratio, the longer the overlapping interval between adjacent frames of the original audio signal (compare FIG. 2A and FIG. 3A). The synchronization delay K _m is not constant.

Based on this fact, first consider the case where the designated shift ratio α _o is much larger than 1 (see FIG. 2), that is, α _o > SS / (SS-L). As mentioned above, the current frame F _m read from the first memory 120 to find the maximum correlation point consists of N + L samples from mSA-L to mSA + N th (FIG. 2 (a)). Reference). The current frame thus read is slid within the scan frame 24 of the mSS-L to mSS + L and the last frame of the time-scale-corrected signal to find the point having the maximum correlation, that is, the synchronization delay K _m (Fig. 2). (b)). When the synchronization delay K _m is determined, only NK _m samples 20b from the mSA + K _m th samples are added from the mSS point of the time scale modified signal, and the remaining samples 20a are discarded. At this time, if there is no limit of the scan range, theoretically, the maximum correlation point is the overlap point 22 in the original signal [[Check whether 22 points in FIG. 2 (b) coincide with the right end point of SA. Desired), since the point 22 is located outside the scan range 24 due to limiting the scan range 24 to mSS-L to mSS + L, the actual maximum correlation point is not There must be one point. The probability that the sign of the synchronization delay K _m representing the actual maximum correlation point is negative or positive is random. Therefore, in this case, the synchronization delay error may not increase continuously in one direction.

On the other hand, the situation is different when the designated shift ratio α _o is slightly larger than 1, that is, when α _o ≤ SS / (SS-L) (see FIG. 3). In order to find the synchronization delay K _m of the current frame, N + L samples 30a and 30b are taken from the original signal (see FIG. 3 (a)), and the synchronization delay K _m is obtained within the scan range 34 (Figure 3). 3 (b)), when the synchronization delay K _m of the current frame is determined, reflecting this value, add only mSA + K _m th samples to NK _m samples 30b from the mSS th point of the time-scale modified signal. Discarding the remaining sample 30a (see FIG. 3 (d)) is the same as the above case. However, in the case of α _o ≤ SS / (SS-L), the synchronization delay K _m of each frame has a much higher probability of having a positive sign than the probability of having a negative sign. This is because the first scan range 34 defined in mSS-L to mSS + L is modified to the new scan range 36 mSS-lag to mSS + 2L-lag (steps S12 and S14).

In the above description, the case where the specified speed ratio is larger than 1, that is, the playback time is extended, is described as an example. However, even when the specified speed ratio is smaller than 1, when the value is slightly smaller than 1, the scan range needs to be corrected. . As a result, when the absolute value of the designated transmission rate is smaller than SS / (SS-L), it may be considered that correction of the scan range is necessary.

However, if the modified scan range 36 is applied, it is now possible to obtain a shift effect corresponding to the designated shift ratio, but the maximum correlation point is located on the right side of the center point mSS of the original scan range 34. The probability of existence is much higher. That is, the synchronization delay K _m has a higher probability of having a positive sign, so that the cumulative value of the synchronization delay K _m of each frame period becomes larger. As a result, as the time scale correction proceeds, the playback time (i.e., the playback speed) of the time scale corrected signal becomes more and more different than the ideal playback time. Problems will arise.

To solve this problem, periodically check the cumulative value of the synchronization delay K _m and if the value is out of the preset tolerance range, correct the shift ratio to force the accumulation value of the synchronization delay K _m to converge within the tolerance range. (See FIG. 5 (b)). This will be described in detail.

Whenever the synchronization delay K _m is calculated in step S20, the value is cumulatively calculated to obtain a cumulative value Err of the synchronization delay K _m (step S30). The formula is expressed as follows. The cumulative value Err of the synchronization delay K _m calculated every frame period means an error between the actual reproduction time up to the period and the ideal reproduction time. Therefore, if the cumulative value Err is large, it means that the degree of synchronization mismatch between the video signal and the audio signal is large.

Err = ΣK _m

Next, the setting state of the correction mode is checked (step S32). In the calibration mode, since the initial value is set to off, the first loop will always check whether the cumulative value Err exceeds the tolerance range (step S34) (see FIG. 6). The tolerance range E _min to E _max may be determined based on an allowable degree of synchronization mismatch between the video signal and the audio signal in consideration of shift reproduction of the multimedia signal.

As a result of checking in step S34, if the cumulative value Err is within the tolerance range, the deviation is small enough to be negligible compared to the ideal reproduction time (i.e., the ideal reproduction speed), so there is no need for special processing and the time for the next frame immediately. Since the scale correction may be performed, the process of checking whether data of the original audio signal x (·) remains in the first memory 120 (step S26). However, if the error should not be tolerated as such, the shift rate correction processing steps (steps S38 to S44) to reduce the cumulative value Err may be performed.

However, as a result of checking in step S34, if the cumulative value Err is out of the tolerance range, it is necessary to take measures to reduce the cumulative value Err from the next frame period. The method of reducing the cumulative value Err is to modify the transmission rate applied from the next frame period to a value that can contribute to the reduction of the cumulative value Err. For this correction, first, the speed ratio is corrected while the correction mode is set to ON (step S36).

Next, the sign of the accumulated value Err is checked (step S38). If the sign of the cumulative value Err is negative, the actual playback time (or playback speed) of the time-scale corrected signal y (·) is longer than the ideal playback time (or playback speed) that is exactly proportional to the initially specified transmission rate α _o ( Or slower), the shift engine 110 corrects the shift rate to be applied from the next frame period to a value smaller than the present. Thus, the corrected speed change rate is applied to the next frame so that the cumulative value Err is reduced (step S40). Conversely, if the sign of the cumulative value Err is positive, the actual playback time (or playback speed) is shorter (or faster) than the ideal playback time (or playback speed), so that the transmission rate is made larger than the present value. The absolute value of the accumulated value Err is similarly reduced from the next frame period (step S42).

The cumulative value Err can be reduced quickly if the correction rate of the speed change ratio is made too large, while noise may be introduced or the cumulative value Err may be reversed to increase rapidly. On the contrary, if the correction degree of the shift ratio is made too small, the cumulative value Err may not be reduced quickly. In consideration of this point, it is necessary to correct the shift ratio. FIG. 5B shows a case where the correction degree is set to, for example, about 10% of the existing shift rate.

After the shift ratio is modified in this manner, a step of newly calculating the value of the variable SA according to the modified shift ratio is performed (step S44). After this process, the execution sequence goes to step S26 to check whether there is a next frame of the original audio signal to be processed.

On the other hand, if it is determined that the correction mode is set to ON as a result of the check of the correction mode check step (step S32) in the course of repeatedly executing the loop, the reset range R _{min is} defined in the tolerance range. ~ R _max ) is checked (step S46). The reset range may be set to an appropriate value based on the purpose of preventing load increase of the system, in particular, the signal processor 100 due to frequent shift rate correction. For example, it may be set to about 25% of the width of the tolerance range.

Assume that the cumulative value Err is greater than the tolerance range. In this case, by executing the step S40 or S42, the existing speed ratio is corrected to a new speed ratio that contributes to reducing the cumulative value Err. When the modified shift ratio is applied from the next frame, the cumulative value Err is reduced so that the size can finally be small enough to fall within the reset range. Even in this case, if the continuously changed speed ratio is applied, the cumulative value Err is likely to increase again as the sign is changed. Further, immediately after the cumulative value Err is brought into the tolerance range, restoring the modified shift rate back to the originally set shift rate, considering that the sign of the synchronization delay K _m of the next frame period cannot be predicted, It may be inappropriate. Frequent modifications of the transmission rate may force the signal processor 100 to perform various processing such as readjusting related variable values, which may negatively affect the normal operation of a regeneration system (not shown) due to overload. Therefore, it is desirable to make the shift ratio correction only when necessary.

As a result of the introduction of the reset range, the value of the applied transmission rate changes in the following manner. Initially, the speed ratio set by the user is applied. However, when the cumulative value Err becomes large and goes out of the tolerance range, the corrected speed ratio is applied. As a result of the application of the corrected shift rate, the cumulative value Err decreases again to fall within the tolerance range and the corrected shift rate is still applied until the value decreases into the reset range. Then, when the value of the cumulative value Err falls within the reset range, from then on, the original speed ratio is restored instead of the corrected speed ratio, and the correction is stopped. In addition, in the next loop, the correction mode is set to off so that the step of checking whether the accumulated value Err exceeds the tolerance range (step S60) can be executed (step S48).

On the other hand, when the correction mode is on and the cumulative value Err does not fall within the reset range, the cumulative value Err is in a state where at least the magnitude thereof is out of the reset range, and the cumulative value Err is allowed in the previous loop. It means that the speed ratio correction has been made beyond the error range. In this case, the change direction of the cumulative value Err is checked (step S56). If the value of the accumulated value Err increases as a result of the check, the correction mode is set to off to check whether the value of the accumulated value Err of the next frame may finally be out of the tolerance range. On the contrary, the cumulative value Err may be decreased again in the next loop, so it is not necessary to forcibly correct the cumulative value. In this case, therefore, the process proceeds to step S26 without any special processing so that the correction mode is kept on so that the shift rate correction is not performed even in the next loop (steps S50 and S52).

In this way, by introducing a shift rate correction routine (steps S30 to S52) that immediately reduces the error of the playback time generated every frame through the shift rate correction, even if the synchronization delay K _m has a random value every cycle, The actual reproduction time (or reproduction speed) of the scale-modified audio signal is forced to have a value at any point not exceeding half of the tolerance range from the ideal reproduction time (or reproduction speed). As such, when the cumulative value Err of the synchronization delay K _m converges within the tolerance range, the video signal and the audio signal can be synchronized and reproduced at the time of shift reproduction of the multimedia signal.

The audio signal corrected in time scale in this manner is provided to the audio signal reproducing means (not shown) and reproduced. For real time reproduction, it is desirable to construct a plurality of frames into one packet for reproduction processing. In order to reproduce time-scale-modified audio signals in packet units, a plurality of output buffers are provided in the audio signal reproducing means, and each packet is cyclically recorded in the plurality of output buffers. For example, suppose there are four output buffers: Buf _o , Buf ₁ , Buf ₂ , and Buf _3. Consecutive packets are in the order of 'Buf _o → Buf ₁ → Buf ₂ → Buf ₃ → Buf _o ....' Record it. Each packet is reproduced through a speaker (not shown) through post-processing such as conversion to an analog signal and proper amplification in the order recorded in the output buffer.

The above mentioned things were confirmed in the actual test. 7 shows a waveform of an original audio signal having a reproduction length of 20 seconds.

Consider first the case where the specified speed shift α _o is 1.2. α _o <SS / (SS-L) Next, the scan range needs to be modified, and the actual applied scan range is mSS- (SS-SA) to mSS + 2L- (SS-SA). FIG. 8A is a waveform diagram of a signal obtained when the audio signal of FIG. 7 is time-scale corrected using the WSOLA algorithm applying only the scan range correction concept described above when the designated shift ratio α _o is 1.2. FIG. 8 (B) is a waveform diagram of a signal obtained when the audio signal of FIG. 7 is time-scale-corrected using the WSOLA algorithm which applies the concept of scan range correction and shift rate correction when the specified shift ratio α _o is 1.2.

Since the specified shift rate α _o is 1.2, the ideal reproduction time of the time-scale-corrected signal is exactly 24 seconds (= 20 seconds x 1.2). In the waveform of Fig. 8A, the actual reproduction time was measured to be 25.2 seconds. There is an error of about 1.2 seconds from the desired reproduction time. Even in the shortest time of 20 seconds, this error is continuously increased as described above. In the case of multimedia signals with a long running time, the synchronization mismatch between the audio signal and the video signal is increased over time. The degree is so great that it can be easily predicted that viewing will be impossible. This error is due to the fact that shift ratio correction is not applied.

In contrast, the waveform of FIG. 8 (b) was measured at a value close to the actual reproduction time of 24 seconds. In other words, if the shift rate correction is further applied, the time scale is always modified to the desired ideal reproduction time regardless of the size of the original audio signal, so that the synchronization with the video signal can be obtained almost accurately.

Next, consider the case where the designated shift ratio α _o is 2.0. α _o > SS / (SS-L) Then, no correction of the scan range is necessary, and the actual scan range is mSS-L to mSS + L. FIG. 9A is a waveform diagram of a signal obtained by time-scale correcting an audio signal of FIG. 7 using a WSOLA algorithm applying only a scan range correction concept when a designated shift ratio α _o is 2.0, and FIG. 9B. Is a waveform diagram of a signal obtained by time-scale correcting an audio signal of FIG. 7 using a WSOLA algorithm applying a concept of scan range correction and shift rate correction when a designated shift ratio α _o is 2.0.

When the specified shift ratio α _o is 2.0, the ideal reproduction time of the time scale corrected signal is 40 seconds. The signal of FIG. 9A has a large error of about 3.2 seconds when its reproduction time is measured about 36.8 seconds and compared to the ideal reproduction time. In contrast, the signal (b) of FIG. 9 is measured about 39.8 seconds, and it can be seen that only 0.2 seconds differs from the ideal reproduction time.

Although only the case where the specified shift ratio is larger than 1, i.e., the playing time is increased, the same effect can be obtained even when the playing time is reduced.

As can be seen from the above, according to the present invention, it is basically possible to increase or decrease the playback time (that is, the playback speed) of the audio signal according to the transmission rate specified by the user, as well as the playback time of the time-scale modified signal. This can be changed almost in proportion to the specified speed ratio. Therefore, when the present invention is applied to the variable speed reproduction of the multimedia signal, the video signal and the audio signal can be reproduced in perfect synchronization.

Although the present invention has been described above according to an embodiment of the present invention, it will be apparent to those skilled in the art that various modifications may be made without departing from the spirit of the present invention.

Claims (20)

The audio signal is divided into a plurality of consecutive frames, and adjacent frames are separated to overlap by the first number of samples, and the interval of the adjacent frames is compressed or extended so that a second number of samples is overlapped between the divided adjacent frames. And modifying a time scale, wherein the second sample number is determined by further reflecting a synchronization delay obtained at a point of maximum correlation between adjacent frames by a value obtained by multiplying the first sample rate by a user-specified initial transmission rate. In the time scale correction method of an audio signal, In particular, each time the time scale is modified by one frame, the synchronization delay is calculated, and the cumulative value of the synchronization delay is calculated by adding up the previously accumulated synchronization delay, and when the cumulative value exceeds a predetermined tolerance range, the cumulative value can be reduced. And modifying the time scale of the audio signal by applying a new shift rate. The method of claim 1, wherein the tolerance range is set within a range in which a degree of synchronization mismatch between a playback picture and a playback sound is unnaturally felt when playing a multimedia signal in which an audio signal and a video signal are integrated. How to correct time scale of audio signal. The method of claim 1, wherein when the cumulative value is decreased by a time scale correction using the new speed change rate and falls within a reset range defined within the tolerance range, the new speed change rate is restored to the initial speed change rate and applied. Time scale correction method of the audio signal, characterized in that. The method of claim 1, wherein the maximum correlation point is determined at a point where waveform similarity between two adjacent frames is highest within a predetermined scan range. The method of claim 4, wherein the scan range is set to a sufficient range to find the maximum waveform similarity between two adjacent frames in consideration of the upper limit value of the initial transmission rate, etc., wherein the start point (mSA) of the current frame of the audio signal is And determining the scan range to be located outside the scan range. The method of claim 4, wherein the scan range is composed of a predetermined number of left and right samples based on a point shifted by a sample interval SA between start points of consecutive frames of the audio signal from a start point of a current frame. How to correct time scale of audio signal. Using the overlap and add principle of successive frames to modify the time scale of the audio signal at a user-specified transmission rate, the correlation is maximized to maintain the original pitch information of the audio signal. In a method of modifying the time scale of the audio signal using a predetermined algorithm of a method of concatenating two adjacent frames at the in point, Whenever the time scale is modified by one frame by applying the predetermined algorithm, the best cross-correlation between the current frame of the audio signal and the last frame of the time scale-corrected signal is determined within a predetermined scan range. Finding a synchronization lag that is a sample interval between a representative point and a center point of the scan range; Calculating a cumulative value of the new synchronization delay by adding the found synchronization delay to the previous accumulated value of the synchronization delays; And Comparing the calculated cumulative delay value with a predetermined tolerance range, if the cumulative value is out of the tolerance range, the applied transmission rate can be reduced from the next frame and converged within the tolerance range from the subsequent frame. A method of correcting a time scale of an audio signal, characterized by correcting a time scale by correcting with a new speed ratio. 8. The method of claim 7, wherein the predetermined algorithm is SOLA, WSOLA, or equivalent algorithm defining the maximum correlation as the point where the waveform similarity is highest while sliding two adjacent frames within the scan range. How to correct time scale of audio signal. 8. The method of claim 7, wherein a difference SS-SA of a start point sample interval SA of the adjacent frame of the audio signal is calculated from the start point sample interval SS of the adjacent frame of the timescale corrected signal, and the scan range is | SS-SA | > L, the scan range is defined as a range including L samples left and right about the SA m skipped sample mSA from the start of the current frame, that is, mSA-L to mSA + L, otherwise MSA- (SS-SA) to mSA + 2L- (SS-SA), wherein the variables L and m represent half of the scan range and a frame index, respectively. Way. 8. The method of claim 7, wherein the cumulative value of the synchronization delay is periodically calculated even while the time scale of the audio signal is corrected by applying the new shift rate, and then, depending on whether the calculated cumulative value converges within the tolerance range. And adjusting the speed ratio to be applied from the period. 8. The method of claim 7, wherein when the cumulative value is reduced by a time scale correction using the new speed change rate and falls within a reset range defined as a narrower range within the tolerance range, the new speed change rate is set by the user. And recovering and applying the speed ratio again. 12. The correction mode according to claim 7 or 11, wherein a default value of the correction mode is set to off, and when the magnitude of the accumulated value of the synchronization delay exceeds the tolerance range, the correction mode is changed to on, and the correction mode is turned on. If the cumulative value of the synchronization delay increases compared to the cumulative value of the synchronization delay up to the previous period, if the cumulative value of the synchronization delay exceeds the tolerance range in the next period, and if it decreases, the correction mode And keeping the ON so that the transmission rate applied to the current period is applied as it is. The original audio signal is stored in the first memory, the original audio signal is read out one frame from the first memory, and the second audio is rearranged so that the time scale of adjacent frames is modified according to the initial transmission rate set by the user. In the time scale correction method of an audio signal stored in Read one by one frame from the original audio signal of the first memory, the current frame is to read N + L samples from the mSA-L th sample so that a predetermined sample overlaps with the previous frame, The variable N is the number of samples constituting the unit frame, the variable SA is defined as SS / α, the variable SS is defined as the number of samples between the start points of the successive frames of the time-scale modified audio signal, α represents an applied transmission rate for each frame period, the initial transmission rate or a modified transmission rate, the variable m represents a frame index, and L represents a number of samples corresponding to half of a predetermined scan range; Calculating a waveform similarity between the last frame of the time-scale-modified audio signal and the read current frame while sliding the current frame within the scan range; Each frame period, the method comprising calculating a synchronization delay K _m, the synchronization delay K _m is the number of samples between the time scale of having the maximum value among the calculated waveform similarity point and the cycle correction signal starting point (mSS) Defined by the difference of; NK _m samples to which the synchronization delay K _m calculated in the read current frame is applied are added to the last frame of the second memory from the mSS-th sample point, and the duplicated samples of both the current frame and the last frame are weighted. Summing by applying; Each time a new synchronization delay K _m is calculated, adding a cumulative value of the synchronization delays up to the previous frame period to newly calculate the synchronization delays up to the current frame period; And Checking whether the calculated new synchronization delay cumulative value is out of a predetermined tolerance range, and if it is confirmed that the calculated new synchronization delay cumulative value is out of a predetermined tolerance range, forcing the synchronization delay cumulative value to fall within the tolerance range by adjusting a value of the applied transmission rate from the next frame period. Time scale correction method of an audio signal, characterized in that it comprises a. 15. The method of claim 13, wherein if the cumulative synchronization delay value increases in a negative direction and falls outside the tolerance range, the applied shift ratio α is corrected to a value smaller than the initial shift ratio α _o , and the cumulative value of the synchronization delay is positive. And applying the shift ratio α to a value larger than the initial shift ratio α _o when increasing in a direction out of the tolerance range. 15. The shift ratio according to claim 14, wherein a part of the tolerance range is defined as a reset range in order to force the accumulated accumulation delay value to be within the tolerance range, and after the synchronization delay accumulation value is out of the tolerance range, And correcting the time scale as a result of applying the following to restore the applied speed change rate to the initial speed change rate α _o when the synchronization delay accumulated value falls within the reset range. The method according to claim 13, wherein the scan range is mSS-L to mSS + L, which includes L samples left and right around the mSS point when | α _o |> SS / (SS-L). Defined, and mSS- (SS-SA) to mSS + 2L- (SS-SA) when | α _o | ≤SS / (SS-L). The audio signal is time-scaled according to a user-specified transmission rate, but the time of the audio signal is based on the principle of finding and reflecting the best correlation while sliding two adjacent frames within a predetermined scan range. In the device for modifying the scale, Memory means for storing the original audio signal and the audio signal corrected in time scale based on a transmission rate set by a user; And The audio signal is divided into a plurality of consecutive frames, and adjacent frames are separated by a first number of samples, and the interval of the adjacent frames is compressed or extended so that a second number of samples is overlapped between the divided adjacent frames. A point indicating the best cross-correlation between the current frame of the audio signal and the last frame of the time-scale-corrected signal within a predetermined scan range, particularly when the time-scale is corrected one frame at a time. And calculating the synchronization delay which is the sample interval between the center point of the scan range and adding the synchronization delay accumulated up to the previous frame period, calculating the accumulation value of the synchronization delay, and comparing the calculated accumulation delay with the predetermined tolerance range. If the cumulative value is out of the tolerance range, the shift is applied from the next frame. And a signal processing means for correcting the time scale by correcting the rate to a new speed change rate that can reduce the cumulative value of the synchronization delay to converge within the tolerance range. 18. The method of claim 17, wherein in the signal processing means, a difference SS-SA of a start point sample interval SA of an adjacent frame of the audio signal is calculated from a start point sample interval SS of an adjacent frame of the time scale-corrected signal, and the scan range The | SS-SA | > L, the scan range is defined as a range including L samples left and right about the SA m skipped sample mSA from the start of the current frame, that is, mSA-L to mSA + L, otherwise Are defined as mSA- (SS-SA) to mSA + 2L- (SS-SA), wherein the integers L and m represent half of the scan range and frame index, respectively. Time scale corrector. 18. The method according to claim 17, wherein the signal processing means periodically calculates a cumulative value of the synchronization delay while the time shift of the audio signal is corrected by applying the new shift rate so that the calculated cumulative value converges within the tolerance range. And adjusting the speed change rate to be applied from a subsequent period according to whether or not the audio signal is applied. 18. The method according to claim 17, wherein the signal processing means reduces the cumulative value by a time scale correction to which the new speed change rate is applied, and enters the new speed change rate when it enters a reset range defined as a narrower range within the tolerance range. Apparatus for correcting the time scale of the audio signal, characterized in that for recovering and applying again to the shift rate set by the user.