KR20030000400A - Method and apparatus for real- time modification of audio play speed - Google Patents

Abstract

PURPOSE: A method and an apparatus for changing speed of voice regeneration in real time are provided to minimize deterioration of tone quality, damage of voice signals, and delay of regenerating time. CONSTITUTION: An apparatus for changing speed of voice regeneration in real time includes a user interface(7) receiving information required by a user, a digital voice signal storage medium(5) having stored digital voice signals, a voice signal processor(1) receiving the digital voice signals and converting into PCM data, a direct current component removing part(2) removing direct current component of the voice signals by using a weighted average to prevent tone quality from deteriorating according to regeneration speed control of the converted voice signals, a speed rate changing part(3), a real time regeneration control part(4) executing asking to control real time regeneration of the whole voice signals without damage of the tone quality.

Description

Method and apparatus for real-time conversion of voice playback speed {Method and apparatus for real-time modification of audio play speed}

The present invention relates to a method and apparatus for reproducing a digital voice signal in real time, and more particularly, to a multi-tasking (SLA) algorithm, which is a time scale modification technique. The present invention relates to a voice playback speed real-time conversion method and apparatus capable of reproducing a voice signal in real time without changing the tone due to loss or distortion of the voice signal.

In general, the method using the SOLA algorithm is the simplest, the least amount of calculation, the excellent sound quality, and the method suitable for real-time playback of the speech signal correction method on the time axis.

The SOLA algorithm was first introduced by Roucos, S. & Wilgus A. in the 1985 IEEE International Conference on Acoustics, Speech, and Signal Processing, Volume 2 (pp.493-496) as "High Quality Time-Scale Modification for Speech." .

The SOLA algorithm finds a position having the highest cross-correlation coefficient while moving the starting point of a new voice signal frame with respect to the entire preceding voice signal in a specific section, and converts the analog signal into a digital signal at that position. It is a method of overlapping and averaging frames to create a new voice signal like the sampling method used when switching to. In other words, this method finds the optimal similarity between successive frames in the proximity section.

Here, the cross-correlation coefficient is a value calculated by applying a cross-correlation function to two voice signals, and has a value between '-1' and '1'. This means that the correlation between the signals is very high. For reference, the correlation function is a function defined to mathematically express the correlation between two voice signals, an autocorrelation function for the same signal and a cross correlation function for two different signals. do. Such a correlation function is a function indicating a correlation of signals in the time domain, and a function indicating a distribution of power or energy included in the signal in the frequency domain. This relationship is called Wiener-Khintchine's law. The cross-correlation function (R_xy (t)) of two signals x (t) and y (t) is defined as x (t) at time t and y (t +) at time t +. The product is averaged over a sufficiently long time.

As such, the frames combined through the method using the SOLA algorithm may protect the pitch, magnitude, and phase, which have a time-dependent characteristic of the voice signal, and also extract the pitch. It is a simple and efficient method because it does not require extraction, calculation in the frequency domain, and phase unwrapping.

In general, in a reproducing apparatus such as a tape player or an MP3 player, when the reproduction speed is changed, the tone of the digital voice signal stored in the storage medium is changed and reproduced. In other words, if you change the speed faster, you can hear high-pitched voices. The reason for this phenomenon is that when the speed of the voice signal is changed, the frequency and pitch components of the stored voice signal are changed.

As a conventional method for preventing such a phenomenon from occurring, first, a period of an input digital voice signal is found using an average size difference function (AMDF method), and the voiced sound and the unvoiced sound are separated through the found period. There is a method of modulating the length of the voice signal by copying or removing a part of the separated voiced sound, and performing synthesis with the separated voiced sound. Second, there is a method of copying and adding or removing a part of the digital signal in the time axis.

However, various implementations may occur when implemented using the conventional methods as described above.

If you do not find the correct period when you find the period of the input digital voice signal by using the difference function of the average size (AMDF method), further distortion occurs, and how to copy or add a part of the digital signal on the time base In the case of rapidly changing the playback speed, the loss of the digital voice signal is excessive, and in severe cases, the word may be skipped and played back. In addition, in the conventional method, when the playback speed is changed in real time, all audio signals to be reproduced must be processed by blocking them to a certain size. In this case, excessive loss of the voice signal at the connection part of the voice signal processed in units of blocks is performed. Alternatively, the reverberation or the tick distance may be clicked due to distortion, and a time delay may occur for a predetermined time.

SUMMARY OF THE INVENTION An object of the present invention is to solve a conventional problem, and when reproducing a digitized and stored voice signal using SOLA algorithm, which is a time base conversion technique, and a multitasking technique, the reproduction speed of the voice signal can be changed in real time. It is to provide a voice playback speed real-time conversion method and apparatus that can prevent the occurrence of alteration and loss of the sound to occur, and can adjust the playback speed of the voice signal in real time to sound like a person talking quickly or slowly.

1 is a system configuration according to the present invention.

2 is a configuration of a job processing on a system made through the control of a real time playback control unit.

3 is a flowchart of operation in the voice signal processing unit.

4 is a flowchart of operation in the direct current component removing unit;

5 is a flowchart of work in the reproduction speed converting section.

6 is a block diagram of a method for preventing the loss and distortion of speech.

<Description of Drawing>

Audio signal processing section 1, DC component removal section 2, playback speed converting section 3, real time playback control section 4, digital audio signal storage media 5, digital audio signal playback apparatus 6, user interface (7), storage of the audio signal sample 20, connection portion 30 of the audio signal, an initial playback start command (A), and a command to change the playback speed during playback (A ')

The present invention relates to a method and apparatus for reproducing a digital voice signal in real time by varying the playback speed within a range of 0.5 times to 2 times the speed. The voice signal according to the present invention is not only a voice produced by a person, but also a musical instrument. Refers to all sound signals generated from.

The overall configuration of the present invention will be described with reference to FIG.

The system for reproducing in real time by varying the playback speed of the digital audio signal according to the present invention, the information required by the user (selection of the file you want to play, the playback speed to be changed, playback start, playback stop, stop, etc.) Receives a digital voice signal from the user interface (7), a digital voice signal storage medium (5) for storing digital voice signals, and the digital voice signal storage medium (5) and receive a specific pulse code modulation (PCM) data format. Voice signal processing unit (1) for converting the signal, DC component removal unit (2) for removing the DC component of the voice signal by using the weighted average value of the input digital voice signal in order to prevent the sound quality degradation caused by the reproduction speed of the voice signal To change the playback speed of the voice signal, a playback speed change is performed by using a time-base conversion technology, SOLA (Synchronized Overlap-and-Add) algorithm. 3, the real-time playback control unit 4 and the real-time playback control unit 4 performing multitasking to control the entire inputted audio signal to be reproduced in real time without damage to sound quality. And a digital audio signal reproducing apparatus 6 for outputting the audio signal through the reproduction speed converting section 3 through the control.

Here, the audio of a record or a radio takes an analog method of transmitting and reproducing a sound intensity as a current intensity. On the other hand, a combination of a binary signal of '1' or '0', that is, a negative signal is converted into a digital signal and transmitted, called a PCM (Pulse Code Modulation) method. In the present invention, the SOLA algorithm, which is a time-base conversion technique used for converting a reproduction speed of a voice signal, has been described in the related art.

The operation of the voice playback speed real-time conversion device according to the present invention configured as described above will be described with reference to FIG. 2 for the overall business processing in view of the control operation of the real-time playback control section 4.

The system of the present invention first receives information required by the user (selection of a file that the user wants to play, playback speed to change, playback start, playback stop, stop, etc.) in the user interface 7. The inputted information is operated by the control of the real-time playback control section 4, which controls the system as a whole. The system of the present invention, when changing the playback speed, receives a command (A) for the initial start of playback of the audio signal. The processor 1 reads the voice signal from the digital voice signal storage medium 5 in units of blocks of a predetermined size, and then converts the voice signal into a specific PCM data format. The converted voice signal data is transmitted to the DC component removal unit 2 and the reproduction speed conversion unit 3, and then the DC component removal unit 2 removes the DC component of the audio signal, and reproduces the speed conversion unit 3. In this example, the voice signal is converted using the SOLA algorithm, which is a time base conversion technique. In this way, the DC component removing unit 2 and the reproduction speed converting unit 3 perform the DC component removal and speed conversion processing while the audio signal is transmitted from the audio signal processing unit 1. The converted voice signal is output through the digital voice signal reproducing apparatus 6 such as a sound card.

However, if a 'pause' or 'pause' command is received from the real-time playback control section 4 during the operation in the audio signal processing section 1, the operation is paused to pause the playback of the audio signal or terminate the operation. .

In addition, when a command (A ') of changing the speed rate is input to the user interface 7 during the reproduction of the voice signal, the real-time reproduction is performed without passing through the voice signal processing unit 1 to change the speed in real time. After the operation is performed by the DC component removing unit 2 and the reproduction speed converting unit 3 under the control of the control unit 4, the audio signal is output through the digital audio signal reproducing apparatus 6.

In the system of the present invention configured as described above, the real-time conversion method of the voice reproduction speed will be described with reference to the work flow chart for each component of the system.

(1) voice signal processing unit

3 is a flowchart for job processing in the audio signal processing unit 1.

In the audio signal processor 1, as shown in FIG.

In the digital audio signal storage medium 5 designated by the real-time playback control section 4, the digital audio signal stored in the digital audio signal storage medium 5 in accordance with a command to start playback of the audio signal stored in a specific file or a specific address on the memory. Examining the data format of the data format (whether or not it is changed to the PCM data format) [100];

Reading the data in units of blocks of a predetermined size when the change to the PCM data format is possible after the inspection [101];

However, if the change to the PCM data format is impossible after the irradiation, the step [101 '] of notifying the real time reproduction control section 4 is performed.

Step [102] of converting the read data into the PCM data format is performed. In the present invention, the sampling rate is converted into the PCM data format of 11.025 kHz, 16 bits to 22.05 kHz, and 16 bits.

The converted data is divided into blocks of a certain size, which is easy for real-time playback, and then the audio signal in block unit is transmitted to the DC component removing unit 2 until a command such as playing or stopping playback or pausing of the entire audio signal is received. It consists of a step [103]. Here, the speed rate is transmitted to the playback speed conversion section 3 (not shown).

Here, the reason for processing the whole voice signal into blocks is to perform multitasking of the real-time playback control unit 4. When one work unit occupies resources of the system for a long time, a time delay occurs due to a bottleneck. To prevent it.

(2) DC component removal unit

4 is a flowchart for processing a job in the direct current component removing unit 2.

The DC component removing unit 2 is for removing DC components of the audio signal in a block unit transmitted from the audio signal processing unit 1 under the control of the real time reproduction control unit 4.

Assigning a minimum value (Min) and a maximum value (Max) to a fixed value of minimum value = -32767 and maximum value = 32767 in a block unit voice signal transmitted from the voice signal processing unit 1 [200];

Searching the input voice signal samples to find a minimum value R_Min of a sample value smaller than a minimum value Min and a maximum value R_Max of a sample value larger than a maximum value [201];

A step [202] of calculating the average value of each sample of the input voice signal block and adding the weight value (= 0.5) to the calculated average value is performed.

In determining the weighted average value, if the value obtained by subtracting the weighted average value from the minimum value (Real Min) of the sample value is smaller than the minimum value (Min) assigned to the initial value, the weighted average value is added to the weighted average value. If the value obtained by subtracting the weighted average value from the maximum value (Real Max) of the sample value is larger than the first assigned maximum value (Max), the weighted average value is obtained by adding the minimum value (Min) initially assigned to the weighted average value [ 203].

This is expressed as a formula as follows.

IF (R_Min-weighted average) <Min

THEN weighted average = weighted average + Max

IF (R_Max-Weighted Average)> Max

THEN weighted average = weighted average + Min

After the weighted average value is determined, a step of removing the DC component by subtracting the weighted average value from each sample is performed [204].

And transmitting the audio signal in the block unit from which the DC component has been removed to the reproduction speed converting section 3 [205].

The method of removing the DC component of the audio signal by using the weighted average value improves the reproduction sound quality by removing the DC component before the audio signal is corrected in the time domain.

(3) Play speed converter

5 is a flowchart for job processing in the reproduction speed converting section 3.

First, the equation used in the playback speed converter 3 and a description thereof are as follows.

In Equation 1, Equation 1 is used to calculate a step time applied when a maximum correlation value is searched using a window size and a speed rate to be converted. to be. Equation 2 is a formula for calculating a time duration of one sample by referring to the frequency of the input voice signal, and Equation 3 is an input frame step to apply when patching input frames to be processed for speed conversion. The formula to calculate the size. Equation 4 is an equation for calculating the size of one window for applying the Hanning window function, and Equation 5 is an equation for calculating the size, minimum, and maximum range for processing the SOLA algorithm overlapping and addition. Equation 6 is an equation for calculating the total reproduction time of the input voice signal block, and Equation 7 is an equation representing a Hanning window function.

The playback speed conversion section 3 controlled by the real time playback control section 4 performs the playback speed conversion process on the audio signal block from which the DC component has been removed using the SOLA algorithm.

Defining and calculating various time scale parameters such as a window size of a voice signal sample, [300],

In the present invention, the window time of the input voice signal is fixed at 0.032 (32 ms), and the synchronization time (+/- Sync. Time), which is a search time element for synchronization, is fixed at 0.01 (10 ms). do.

After defining and calculating various parameters, a step [301] of calculating the total end time of the audio signal in the block unit in which the DC component is removed is performed by the DC component removing unit 2. Here, the end time can be obtained using Equation 6.

After calculating the total reproduction time of the audio signal, a comparison step [302] is performed between the currently processed audio signal reproduction time (Current Processing Time) and the total reproduction time (End Time) of the input audio signal block.

Here, the steps from step [303] are repeated until the currently processed voice signal reproduction time reaches the entire reproduction time of the input voice signal block. When the total reproduction time is reached [303 '], the audio signal is immediately output through the digital audio signal reproducing apparatus 6.

Using the cross-correlation function and its coefficients to search for the most appropriate position among the speech signal frames [303], Equation 1 is used.

After searching, we apply Haning Window Function [304],

Here, the hanning window function is a technique for smoothing equal interval data using weights of 0.25, 0.5, and 0.25, and is a function applied to reduce a sudden blocking effect of a window. Equations 4 and 7 are used to apply the Hanning window function.

After the application of the Hanning window function, a step of overlapping and adding the speech signal using the window size and the searched overlap and add positions [305] is performed.

A hanning window function and a synthesis step [306] of two superimposed and added speech signals are performed.

Here, the synthesis of the Hanning window function and the superimposed and added speech signals are synthesized using a sampling method used to convert an analog signal into a digital signal call.

The synthesized speech signal frames are updated into the memory buffer and the current processed speech signal reproduction time (Current Processing Time) for output through the digital speech signal reproduction apparatus 6 [307],

After the update, the process returns to step 302 to process the next frame [308], and the comparison operation between the currently processed audio signal reproduction time and the entire reproduction time of the input audio signal block is performed again.

Finally, a method for preventing the loss and distortion of speech will be described with reference to FIG.

If the end frame of the voice signal block is processed, as shown in FIG. 6, when the real time processing is performed, a loss or distortion of the voice signal of the connection part 30 may occur. For example, after storing the voice signal samples as much as 1/2 of the window size, as shown in the figure, when the next block is processed, it is connected to the front part of the next block.

As described above, according to the method and apparatus for real-time conversion of the voice reproducing speed, the digital voice signal input to the voice reproducing apparatus can be reproduced by changing the speed in real time. By minimizing the loss of the audio signal and the playback time delay that can occur due to the blocking process, the playback speed of the audio signal can be changed in real time. Therefore, when a user needs to play back or slow down in real time, such as an MP3 flare or a foreign language learning, which uses an enhanced playback audio signal to play an MPEG-compressed audio signal, a large amount of audio signal data can be played back quickly or slowly. You can get audible effects.

Claims (17)

An apparatus for reproducing in real time by varying the reproduction speed of a digital audio signal, A user interface for receiving information required by the user, Digital voice signal storage medium for storing the digital voice signal, A voice signal processor for receiving a digital voice signal from the digital voice signal storage medium and converting the digital voice signal into a PCM data format; DC component removal unit for removing the DC component of the voice signal using a weighted average value to prevent sound quality degradation due to the control of the playback speed of the voice signal converted by the voice signal processor, Playback speed conversion unit for converting the playback speed of the speech signal from which the DC component is removed using the SOLA algorithm, which is a time base conversion technology, through the DC component removal unit; A real time playback control unit for performing a multi task to control the entire voice signal to be reproduced in real time without damage to sound quality; And a digital voice signal reproducing apparatus for outputting a voice signal through a reproducing rate converting unit under the control of the real time reproducing control unit. The apparatus of claim 1, wherein the entire speech signal is processed in block units (1024 * N; N is an integer greater than 0). The apparatus of claim 1, wherein the reproduction speed of the speech signal reproduced through the digital speech signal reproducing apparatus is variably reproduced within a range of 0.5 times to 2 times speed. The method of claim 1, wherein the command of the initial playback start input to the user interface is performed from the voice signal processor under the control of the real-time playback controller, and the command of changing the playback speed input during the playback of the voice signal is controlled by the control of the real-time playback controller. The voice playback speed real-time conversion device, characterized in that the operation is started in the removal unit and the playback speed conversion unit. The apparatus of claim 1, wherein frames of voice signals in block units are connected to each other in order to prevent loss and distortion of the voice signals. In the method for reproducing in real time by varying the playback speed of the digital audio signal, 1) receiving the information required by the user through the user interface, 2) reading the voice signal from the digital voice signal storage medium designated by the real-time playback control unit and converting the voice signal into a PCM data format; 3) removing the DC component by using a weighted average value in the DC component removing unit to prevent the sound quality degradation due to the conversion of the reproduction speed of the voice signal transmitted from the voice signal processor; 4) converting the playback speed by using the SOLA algorithm in the playback speed converter to convert the playback speed of the voice signal from which the DC component is removed by the DC component remover; 5) The voice playback speed real-time conversion method, characterized in that it comprises the step of outputting the audio signal through the digital voice signal playback apparatus under the control of the real-time playback control unit. In claim 6, step 2) is Checking whether the input digital voice signal is changed into the PCM data format [100]; Reading data in block units of a predetermined size when the change to the PCM data format is possible [101]; Converting the read data into the PCM data format [102]; And dividing the converted data into blocks of a predetermined size, and then transmitting the converted data to a DC component removing unit [103]. In claim 6, step 3) Allocating a fixed value of a minimum value of −32767 and a maximum value of 32767 in a transmitted voice signal in a block unit [200]; Retrieving the minimum value R_Min of the sample value smaller than the minimum value Min and the maximum value R_Max of the sample value larger than the maximum value Max among the voice signal samples [201]; Calculating a weighted average value by adding a weighted value to the calculated average value after calculating an average value of each sample of the audio signal [202]; If the value obtained by subtracting the weighted average value from the minimum value (Real Min) of the sample value is smaller than the minimum value (Min) assigned first, the weighted average value is obtained by adding the maximum value (Max) initially assigned to the weighted average value. If the value obtained by subtracting the weighted average value from the value (Real Max) is larger than the initially assigned maximum value (Max), adding a minimum value (Min) initially assigned to the weighted average value to obtain a weighted average value [203]; Removing the direct current component by subtracting the weighted average value from each sample of the audio signal [204]; And transmitting the audio signal in the block unit from which the DC component has been removed to the reproduction speed conversion unit. In claim 6, step 4) Defining and calculating various time element parameters such as a window size of a voice signal sample [300]; Calculating a total reproduction time of the audio signal [301]; Comparing the currently processed voice signal reproduction time with the total reproduction time of the input voice signal block [302]; Retrieving a most suitable position among voice signal frames using a cross-correlation function and its coefficients [303]; Applying a hanning window function [304]; Superimposing and adding the speech signals using the window size and the searched overlap and add positions [305]; Synthesizing two speech signals having a hanning window function and an overlapping and addition process [306]; And [307] recording the synthesized speech signal frames into a memory buffer and updating the currently processed speech signal reproduction time. 10. The method according to any one of claims 6 to 9, wherein the entire speech signal is processed in block units (1024 * N; N is an integer greater than 0). The method according to any one of claims 6 to 9, wherein the reproduction speed of the audio signal reproduced through the digital audio signal reproducing apparatus is variably reproduced within the range of 0.5 times to 2 times the speed. The method of claim 6, wherein the command of the initial playback start input to the user interface is performed from the voice signal processor under the control of the real-time playback control, the command of changing the playback speed input during the playback of the voice signal is controlled by the control of the real-time playback control unit The operation of the voice playback speed, characterized in that the work is started in the removal unit and the playback speed conversion unit. 7. The method of claim 6, wherein frames of voice signals in block units are connected to each other in order to prevent loss and distortion of the voice signals. 8. The method of claim 6 or 7, wherein the sampling rate is 11.025 kHz, 16 bits to 22.05 kHz, 16 bits when the voice signal processor converts the data into a PCM data format. The method according to claim 6 or 8, characterized in that the weight in calculating the weighted average value in the DC component remover is 0.5, the voice playback speed real-time conversion method. The method of claim 6 or 9, wherein the playback speed converter is configured to fix the window size to 0.032 (32ms) and the synchronization time to 0.01 (10ms). 10. The method of claim 6 or 9, wherein the synthesis of the two speech signals superimposed and added by the Hanning window function in the reproduction speed converting unit uses a sampling method such as converting an analog signal into a digital signal. .