KR19990068417A - language studying system which can change the tempo and key of voice data - Google Patents

Abstract

The present invention relates to a method for encoding / decoding a language learning apparatus and a language learning apparatus. The present invention relates to a method for encoding / decoding a speech learning apparatus, and to varying speed without damaging sound quality by receiving compressed voice data in an MP3 manner. It is. To this end, the encoding / decoding method of the language learning apparatus according to the present invention comprises: a first step of compressing and encoding audio data in an MP3 method; A second step of decoding the compressed data; And a third step for varying the pitch and speed of the decoded data and the character representation of the amplifier and speech data sentences. The language learning apparatus according to the present invention comprises: a modified encoder for compressing and encoding audio data in an MP3 manner; A modified decoder for decoding the data compressed by the encoder; A buffer installed to be connected to the encoder and to buffer the encoded audio data encoded in the MP3 method for the repetitive learning mode; A speed / pitch variable for processing the speed and the pitch variable of the voice data from the decoder; A text / image processor for text representation and context description of voice data sentences; And a controller for controlling the modified encoder, the modified decoder, the buffer, the amplifier, the character / image processor, and the speed / pitch variable.

Description

Language studying system which can change the tempo and key of voice data}

The present invention relates to a language learning system, and more particularly, it can receive and restore compressed voice data in an MP3 manner, and can change the speed without damaging the sound quality. The speed variable, pitch conversion, and character caption function of language learning data are provided. The present invention relates to a method for encoding / decoding a language learning apparatus that can be implemented, and to a language learning apparatus.

In general, the hardest part of learning a foreign language is that you can't exactly understand what it says. It is very difficult to understand the content, especially if you speak fast. However, if you talk slowly and repeatedly, you can easily grasp the context. In order to make it easier to understand the words, pictures and video tapes may be used as auxiliary, but when the speed of the words is fast, these may not be very helpful for the improvement of the listening ability.

Conventional language learning has been made to listen to the learning dialogue recorded only at a predetermined speed using a CD or the like on a cassette tape or a personal computer. In the case of changing the speed of the horse by changing the speed of the driving motor mounted on such a conventional device, the speed conversion area is very narrow, which has a disadvantage in that the learning effect is weak.

The audio data stored in various media is repeatedly listened to to improve the listening ability. Conventional speech reproducing apparatus is to adjust the speed of the voice by driving a mechanical motor. If the speed of the motor is changed without any signal processing, the speed can be changed to some extent without damaging the sound quality, but if the speed required by the language learner is changed, the original sound is damaged and it is difficult to hear.

Analog and digital forms coexist as a method of storing voice data for language learning. The reason why the variable speed is difficult in the conventional method is that the signal is processed analogously.

In the method of storing voice data in a digital form, there are DPCM and ADPCM methods for processing signals in the time domain, and the method for compressing data through the signal processing in the frequency domain is MPEG. Audio compression and AC3 are widely used. Signal processing in the time domain has a small amount of computation but not a high compression ratio. Compression in the frequency domain is more computational but has a higher compression rate. Recently, a method of compressing data in the frequency domain has been widely used due to an improvement in the operating speed of a semiconductor chip. Among them, MPEG audio compression method is widely used in the field where high sound quality such as music is required.

References regarding MPEG compression include a, b, c; References for speed / pitch variability include d, e, f; References regarding the character expression method are g and h as follows.

a. ISO / IEC IS 11172 (MPEG-1) Standard

b. Karlheinz Brandenburg & Gerhard Stool, "ISO-MPEG-1 Audio: A Generic Standard for Coding of High-Quality Digital Audio", J. Audio Eng. Soc., Vol. 42 No. 10, 1994 pp. 780-792

c. Davis Yen Pan, "Digital Audio Compression", Digital Technical Journal Vol. 5. No.2 Spring 1993, pp.1-14

d. E. Hardam, "High Quality Time Scale Modification of Speach Signals Using Fast Synchronized-Overlap-Add Algorithms", IEEE, 1990, pp. 409-412

e. J.L. Wayman and D.L.Wilson, "Some Improvements on the Synchronized-Overlap-Add Method of Time Scale Modification for use in Real-Time Speech Compression and Noise Filtering", IEEE Trans. Acoust., Speech, Signal Processing, Vol. ASSP-36, No. 1, pp. 139-140, January. 1988.

f. J.Makhoul and A.E-Jaroudi, "Time-Scale Modification in Medium to Low Rate Speech Coding," IEEE Int. Conf., Acoust, Speech, Signal Processing, 1986, pp. 1705-1708

g. ISO-646 standard

h. ISO-2022 standard

MPEG1 Audio from the Moving Picture Experts Group (MPEG) is an ISO / IEC standard for high-quality, high-efficiency, high-compression stereo coding. The official standard number is ISO / IEC IS 11172 (MPEG-1). , c] As with other standards, the MPEG1 Audio method is made up of standards only, and the implementation is left to the discretion of the user.

The MPEG1 Audio compression method uses a psycho-acoustic model that models the human auditory structure. In the psychoacoustic model, the human auditory structure recognizes the time domain signal by transforming it into the frequency domain, where sensitivity or audible limits are different for each frequency band. In addition, when there is a signal with a large energy in a specific frequency band, a masking effect that does not hear the weak signal of the peripheral band occurs. After determining the quantization level that is masked by the masking phenomenon to generate an unrecognizable quantization noise, the data can be compressed by bit allocation using the same. The MPEG1 audio compression method is coded by dividing it into layers 1, 2, and 3 according to the target bit rate, and the compression rate varies from 1/4 to 1/12. Among them, the layer3 method having the highest compression ratio is called MP3 (MPEG1 Layer3). The MP3 [Refs. A, b, c] method uses the Modified Discrete Cosine Transform (MDCT) and Huffman Coding in addition to the 32 subbands, so that the compression ratio can be increased to 1/12 and has a good sound quality.

Table 1 below is a diagram showing the performance of layer3 of MPEG1 Audio.

Note) ※ M-Mono, S-Stereo ※ Quality Factor: 5-very good, 4-good, 3-not bad, 2-bad, 1-very bad

※ Actual Delay is 3 times of theoretical Delay

Synchronized OverLap and Add (SOLA) algorithms are generally used in the pitch and rate varying techniques of conventional voice signals.

1 is a view showing an example of the implementation of the SOLA algorithm of the MP3 encoding / decoding method according to the present invention. This figure represents a waveform example for processing an SOLA algorithm. As shown in this figure, the SOLA algorithm cuts out the corresponding blocks while overlapping the voice data at regular intervals using a window of a constant size, and rearranges the blocks at intervals required by the speed change, and then adds them different from the original. It is an algorithm that can synthesize the rate-varying speech signal. However, simply changing the interval and then adding the signals between different blocks will cause the sound quality to deteriorate, resulting in a different sound signal. In order to prevent this, when the blocks are rearranged, a cross-correlation for judging the similarity of two signals can be determined by giving a fine-tuning interval based on the required interval, and fine-tuning corresponding to the largest similarity value. By synthesizing two block signals by moving the interval, the same sound quality as the original sound can be maintained.

In FIG. 1, a) represents the original signal, and b) c) d) e) shows the waveform cut out while overlapping the original signal using a window having a constant size. h) i) j) combines the signals of c) d) e) with the fine-tuned intervals between the solid lines in the figure to combine with the signals b) c) d), respectively, A variable speed signal can be obtained.

Although the processing of the SOLA algorithm used for the variable voice speed [Ref. D, e, f] is executed by the method shown in Fig. 1, it is difficult to process in real time because it is complicated and must perform many operations.

In the digital compression method performed through the above-described method, the digital compression method does not use a high compression rate method when storing in a digital form. The reason is that it is difficult to restore the compressed data in real time.

In addition, storing voice data without any compression requires a large memory, which is a problem in a portable learner. Thus, since a small amount of voice data is stored in the memory of the portable learner, a large amount of language learning contents cannot be stored. In addition, even when the voice data is downloaded to the personal computer, if the compressor method is not used, a problem due to communication time occurs.

In addition, a conventional simple encoding / decoding operation has a problem that data is damaged and its sound quality is changed, thereby making it impossible to achieve the purpose of language learning.

In addition, the conventional language learner does not provide a learning method considering the ability of the language learner. In other words, if you play at a higher speed than normal speed for advanced users, the learning effect is more doubled, and for beginners and underachievers, the language data should be played very slowly, but the conventional language learner provides such a function properly. There is a problem that you are not doing.

Accordingly, an object of the present invention is to solve the above problems, it is possible to vary the speed without damaging the sound quality by receiving the voice data compressed by the MP3 method in a fixed device such as a portable device or a PC, language learning It is to provide a language learning apparatus and a method of encoding / decoding the same, in which a speed change of data, a pitch conversion, and a character caption function can be implemented.

1 is a diagram showing an example of the implementation of the SOLA algorithm of the MP3 encoding / decoding method according to the present invention.

2-schematic configuration diagram of a language learning apparatus according to the present invention.

3-detail of the encoder of FIG.

4A to 2B illustrate the bit-stream format of the language learning apparatus of FIG.

4B-an ancillary data format of the language learning apparatus of FIG.

5-detail of the decoder of FIG.

6 is a flowchart of an MP3 encoding / decoding method according to the present invention.

7-A diagram for obtaining sample values.

8 is a flowchart illustrating a pitch conversion and a speed converter.

In order to achieve the above object, an encoding / decoding method of a language learning apparatus according to the present invention comprises: a first step of compressing and encoding audio data using an MP3 method; A second step of decoding the compressed data; And a third step of varying the pitch and speed of the decoded data.

In the first step, the audio data is received and encoded in MP3 format, and additional information necessary for the language learner is added. The additional information includes information on voice sentence, text sentence, image / character, voice speed, external control, encryption, and the like. This step can be implemented on-line or off-line.

In the second step, the language learning data encoded in the MP3 format is stored in a buffer and then decoded. In decoding, instead of using the existing MP3 encoder, it is possible to additionally support the 11.025 / 22.05 KHz sampling frequency through appropriate modification for the language learner, and to emphasize the voice or to emphasize the audio data with the mixed audio and music. To emphasize. The audio data for language learning can guarantee sufficient sound quality with 11.025 / 22.05KHz sampling frequency and 16-bit data, so that the compression rate can be increased compared to 44.1KHz sampling frequency data by supporting 11.025 / 22.05KHz sampling frequency. The system controller receives academic control information and generates an overall control signal. Learner control information includes speed, pitch, amplification, text / image, repetition, and the like. In addition, the modified MP3 decoder extracts text / image related information from input data and sends the same to the third stage text / image processor.

The third step receives the decoded voice data or general audio data and changes the speed and pitch without distortion of the sound quality. In the present invention, the SOLA algorithm is used for the speed / pitch change, but other algorithms may be used. You can also use the down-sampling method if you have a slight degradation of the quality when passing the data from the modified MP3 decoder to the speed / pitch variable. The down-sampling method has an advantage of reducing memory capacity when implementing the present invention using an integrated circuit, and is required for a fixed device having a slow calculation speed.

The language learning apparatus according to the present invention comprises: an encoder for compressing audio data in an MP3 manner; A data formatter for mixing coded data and language learner control information; A buffer for storing encoded data; Modified MP3 Dec, which performs the functions necessary to implement the language learner while restoring to the MP3 method; A speed / pitch variable for varying the speed and pitch of speech; A system controller that receives the learner control information and controls the learner; An amplifier for adjusting the volume of voice; And a text / image processor for processing text and images.

Accordingly, the speed can be varied within a certain range without damaging the sound quality by receiving the voice data compressed by the MP3 method, and the speed change of the language learning data, the pitch conversion, and the character caption function can be implemented.

In addition, the price of the language learning apparatus is relatively low price, the speed can be changed in consideration of the learning ability of the language learners and the character caption function can be provided.

Hereinafter, the encoding / decoding method of the language learning apparatus according to the present invention will be described in more detail with reference to the accompanying drawings.

2 is a schematic configuration diagram of a language learning apparatus according to the present invention. As shown in the figure, the language learning apparatus is composed of an MP3 encoder, a data formatter, a buffer, an amplifier, a modified MP3 decoder, a pitch and speed variable, an image and a character processor, a system controller, and the like.

The MP3 encoder of the present invention is composed of a conventional MP3 encoder and a data formatter. A variety of additional information (voice data sentence, text information constituting voice data sentence, video / character, voice speed, encryption, external control) is added to the encoded data in the MP3 encoder to make language learning materials. This process can be implemented on-line or off-line. In particular, the encoding process supports the 11.025 / 22.05KHz sampling frequency which is not supported by the conventional MP3 encoder. It adds control data, text and background image insertion block to general MP3 encoder, so that language learning data can be processed off-line.

The decoder of the language learner of the present invention is composed of a modified MP3 decoder, a pitch and speed variable, an image and text processor, a system controller, a BUFFER to support repetitive learning, and an amplifier. Therefore, the decoder has a function of variable speed and pitch conversion of text, background image, and voice signal, and operates in the five modes described below. That is, the five modes include a speed variable or pitch variable drive mode using language learning voice data, an MP3 decoder function mode for music data, a mode of performing a voice speed variable or pitch conversion by attenuating high frequency components of a music signal, It consists of a mode in which a low frequency component of data mixed with music and voice is attenuated, and a mode in which a music signal is mainly listened, and a function mode for performing a background image or text processing while performing the four mode operations.

Looking at the five operating modes in sequence as follows.

1. Mode to operate variable speed and pitch conversion device using contents for language learning

The language learner encoder inserts characters, background images, and control data into language data encoded by a conventional MP3 algorithm, and the language learner decoder processes speed variations, pitch transformations, and character and background images of language data.

In 1) and 2) of FIG. 2, additional data for various language learning are added at the time of encoding, and 3) shows a buffer for supporting a repeat listening mode. This buffer stores the compressed data before it is restored. 4) decodes the audio data for language learning using the modified MP3 decoder and operates as follows.

Since there are few high-frequency components in speech, only low-frequency signals need to be used when operating in language learning mode. Therefore, first of all, the value is calculated only for the lower 8 or 16 subband samples among the 32 subband samples processed in the MP3 decoding process, and the upper 24 or 16 subband samples are multiplied by 0 or a small value. Decrease. As a result, only the low frequency signal remains, and the calculation amount is reduced. The above considerations for low frequency components need to be synthesized using only 8 or 16 subband samples in the low frequency region during subband calculation or subband synthesis. Therefore, although the conventional MP3 decoder supports only 32KHz / 44KHz / 48KHz, it is possible to recover the 11.025KHZ / 22.05KHz sampled data for the language mode. 5) is a system control block that receives the repetition learning mode key and stores the start address of the sentence stored in the memory and sends the repetition control command to the buffer. 6, FIG. 2 performs pitch and speed varying processing of a voice signal. 2, 7) shows an amplifier for amplifying a voice signal, and 8) 9) a block for processing additional data for audiovisual education to perform image and text processing.

2. MP3 decoder function mode for music data

This mode performs the same functions as a MP3 decoder program on a portable MP3 player or a PC. It is mainly used for listening to music data and performs a general MP3 decoder function. In this operation mode, the 1) 2) block of the encoder is the same as the first operation mode, and 4) performs the conventional MP3 decoding process, and 5) 8) 9) corresponding to the system controller and the character / image processor. Is the same as the first mode.

3. Speed variable or pitch conversion driving mode using low frequency component by attenuating high frequency component of music signal

It is an operation designed to change the speed or pitch of voice data with background music.If a foreign key related to speed change or pitch conversion is input at the request of the user, the speed variable or pitch conversion is executed using only the low frequency components of the music data. do. In this mode of operation, the entire system of Fig. 2 is operated. The SOLA algorithm used for variable pitch and velocity in the present invention is possible in the low frequency band, which is the voice data region, and thus removes high frequency components. In the method of removing high frequency used in the present invention, only the lower eight of the sub-bands are decoded, and the sub-bands corresponding to the 24 high frequency components are not decoded.

The energy of the voice data is concentrated in the low frequency region, but the energy of the music signal is spread evenly over the entire frequency band. Therefore, when the upper 24 subbands are removed in FIG. 6, the total energy is attenuated and the volume is reduced. Therefore, the processed low frequency part must be amplified by 7) of FIG. 6 to be similar to the volume of the original signal. The high frequency component is attenuated, and the low frequency component is used to perform the speed and pitch conversion functions in the speed / pitch variable, and the volume is amplified by using the attenuated total energy.

4. Mode that listens to music signals mainly by attenuating low frequency components of data mixed with music and voice.

In contrast to mode 3, the audio frequency band of the music signal is attenuated, the high frequency component is amplified, and smoothed with a filter to listen to cristalized music. This operation mode operates similarly to mode 3 but does not use the speed / pitch variable because it mainly listens to music.

5. Character expression processing mode of background image or voice data sentence while performing 1,2,3,4 mode operation

The operation mode for text representation of speech data sentences and background image processing is a method for enhancing the audio-visual education effect of the audio data sentence information, the text information constituting the voice data sentences, and video additional data 2) of FIG. 6 of the encoder. It is inserted in, and separated from the modified MP3 DEC of 4), and implemented through the system control block of 5) and the image and text processor of 8) 9).

The character expression processing of the voice data sentence uses the additional data inserted in the encoder to find the start information of the character, and finds the number of characters for the voice data sentence using the continuous additional data.

Background image is divided into still image and moving image. Still images make up a simple background, and videos are used for the author's lecture and situation. The image processing finds the moving / still image indicator designated at the time of encoding and refers to the image start address Lookup-table stored in advance using address data that appears continuously. The data referenced in the lookup-table indicates the actual starting address of the image and the image data is processed.

A method of implementing an encoder for a language learner will be described with reference to FIG. 3.

The language learner encoder consists of a conventional MP3 encoder and a data formatr for inserting additional data. Each operation of the MP3 encoder is represented by reference a, b, and c. The additional data insertion is performed by adding the audio data sentence information and the voice data sentence for the repetitive language learning mode of FIG. 4 b to the ancillary data of FIG. It contains the characters, background image and data for selecting the language and instructions of the external CPU. It also supports various sampling frequencies for language learners. 3 is a representation of the encoder component of FIG. 2, and data insertion is performed after the bitstream-formating process of the MP3 encoding process of FIG. The encoding process is performed on-line and off-line, and the linguistic data and the music data are mixed with additional data and encoded in the data formatter. Information inserted in the encoding of the present invention is shown in Fig. 4B.

4A) shows a bit-stream of mp3 format, and FIG. 4B) shows a format for expressing language learning sentences, text and image indicators, and control data in the ancillary data portion of FIG. 4A).

The voice data sentence information indicator is set to process language learning in sentence units. When the format analysis process finds the information indicator of the voice data sentence in the ancillary data part, the start and end of the voice data sentence unit and the number of frames constituting the sentence Since it can be seen that it can facilitate repeated learning in units of voice data sentences.

The nationality indicator or country code indicates the language of the characters that make up the voice data sentence, and the start / end indicators of the characters are found during the format analysis process, and the character table address that appears continuously is the lookup-table start where the text data for language learning is stored. Point to a street address. MP3 consists of 32 frames per second on average, so the speed of a character can be expressed as 1 when pronounced at a rate of 1/32 beats, or 2 when pronounced at a rate of 1/16 beats. The speed at which each character is pronounced is expressed by the number of frame syncs. The background image expresses a still image and a moving image, and the starting address of the image indicates the starting address of each image by an indirect address designation method using a lookup-table. The external cpu command directives allow you to use cpu commands as part of control of specific cpu operations.

Next, an implementation plan of the decoder of the language learner will be described with reference to FIG. 5.

The learner decoder consists of a repeating learning storage buffer, a modified MP3 decoder, a speed / pitch variabler, an amplifier, a character / image processor, and a system controller. 5, 1) stores the voice data in sentence units as a repetition learning buffer. The modified MP3 decoder comprises a conventional MP3 decoder corresponding to 3) 4) 5) 6) 8) 9) of FIG. 5, an additional data analyzer of 2), and a 7) frequency band selector. The speed / pitch variable corresponding to 10) of FIG. 5 is for pitch and speed-variable processing of speech data for language, and 11) is an amplifier for energy compensation of a voice signal according to frequency band selection. 12) 15) 16) of FIG. 5 is a text / image processor, and 13) 14 is a system controller. The conventional MP3 decoder operation corresponding to 3) 4) 5) 6) 8) 9) of FIG. 5 is summarized in references a, b, and c.

6 is a flowchart illustrating the operation of the entire language learner decoder. The left side of the flowchart is difficult because it needs to be processed in real time simultaneously with the modified MP3 decoding process and the voice speed / pitch variable algorithm. Therefore, the present invention improves the processing method for real-time processing of the conventional MP3 and voice speed / pitch variable algorithm, and implements a language learner for processing additional data.

In step 1) of FIG. 6, encoded data corresponding to 1) of FIG. 4 is input. In step 2) of FIG. 6, the additional data separator separates the inserted additional data of the main data and the ancillary data. The main data performs steps 3) to 13) on the left side of the flowchart of FIG. 6 to perform the above operation modes 1 to 4. If there is an indicator in the additional data separated in step 2) of FIG. 6, 14) to 21) of FIG. 6 are performed, and the voice data sentence, the character representation of the sentence constituting the voice data using continuous information following the indicator, Process additional data such as background image. If there is a start indicator and a stop indicator of the voice data sentence of the separated additional data part, it means that the sentence is not finished based on the start indicator. If there is no start indicator, the music data can be listened to by executing the conventional MP3 decoding. Let's do it.

Hereinafter, a modified MP3 decoder implementation will be described in detail.

In step 1) of FIG. 6, data encoded in the form a) of FIG. 4 is input, and each frame includes sync, side information, and main information. However, the main information does not match the start and end of each frame sync. Therefore, the main data can be read continuously only by saving the main information separately except sync and side information from the input. The prior art requires a memory for buffering more than the maximum main information length of at least one frame.

In FIG. 4A, the main information of one frame is composed of two granules. In the present invention, one granule is buffered and then decoded and then processed by another granule. Therefore, buffering is performed when necessary for each granule or smaller size unit, so the memory capacity for buffering can be reduced by half.

In step 4) of FIG. 6, huffman decoding is performed. Huffman decoding obtains a sample from main data using a huffman code table specified in table-select [2: 0], which is a side-information of a bit-stream. Read bit by bit from the main data and continue reading until the value corresponds to the huffman code.

The Huffman decoder reads data bit by bit, comparing the data in the huffman code table one by one, because the huffman-encoded data is continuously connected, and continues reading the data until a matching value is found. The conventional technology reads bit by bit from the bitstream and compares whether the value is "0" or "1" to perform decoding. In the process 4) of FIG. 6 of the present invention, 8 bits (1 byte) of data are read and not cut out by one bit, and the AND-mask of Table 2 is applied to the bits to be read, and then the value is determined to be "0". The above operation is implemented quickly and easily by using a conditional branch, which can be branched to a prearranged address depending on whether the comparison result is 0, thereby reducing the execution time of the huffman decoding, thereby enabling real-time processing.

Table 2 is a diagram showing and-mask.

For MP3 decoding, the following is performed on the sample values (is) obtained by using the huffman code table in step 5) of FIG. Variables represented in Equations 1, 2, and 3 are based on references a, b, and c.

xr (i) = is (i) ^4/3 * gain * s cale

gain = 2 ^{0.25 * (global_gain [gr] -64-8 * subblock_gain [window] [gr])}

s cale = 2 ^{0.25 * (1+ s calefac _ s cale [gr]) * 2 * (-s calefac [cb] [window] [gr] -preflag [gr] * pretab [cb])}

is (i): huffman decoding output

In order to find xr from is, the equations 1,2 and 3 are transformed into equations 4,5,6,7,8 and used for real time processing.

gain = gain1 * gain2

gain1 = 2 ^{0.25 * (global_gain [gr] -64}

gain2 = 2 ^{0.25 * (-8 * subblock_gain [window] [gr])} = 2 ^{-0.5 * (-4 * subblock_gain [window] [gr])}

s cale = 2 ^{0.25 * (1 + s calefac_ s cale [gr]) * 2 * (-s calefac [cb] [window] [gr] -preflag [gr] * pretab [cb])}

s cale = 2 ^{-0.5 * (1 + s calefac_ s cale [gr]) * (s calefac [cb] [window] [gr] + preflag [gr] * pretab [cb])}

In order to obtain gain1, exponent calculation of 2 ^{0.25 * integer} is required. Therefore, the execution time is reduced by storing in Lookup-table form. The global_gain is 8 bits and its value varies from 0 to 255. Therefore, if you create and save a 256-byte ROM table corresponding to each value, you can get 2 ^{0.25 * (global_gain [gr] -64)} by using the global_gain value. According to the invention placed in place of -64 to -210 global gain fine adjustment interval. on the other hand, in order to obtain a scale gain2 and stores the calculation result of the second ^{-0.5 * constants} in table form in the range of 2 ^{-0.5 * integer} from 0 Only up to 64, so we can get a 2-bit ^{* integer} just by creating a lookup-table of 64-bit width for each value.

The result of calculating is (i) ^4/3 is stored in a table, and the value of is (i) ^4/3 is obtained according to the value of is (i). In the MP3 decoder, is is a value ranging from 0 to 8191. Most of the is values are within 1024, and more than that is rarely generated. If the is value is more than 1024, the range of the is value is divided at regular intervals to store the representative value and the slope, and linear interpolation using the representative value and the slope to approximate the other values. Then you just need a little more lookup-table than 1K word. In addition, if you want to reduce the capacity of Lookup-Table, you can store only 512 or 256 instead of 1024 and approximate the rest by the above method. In FIG. 7, the value of is ^4/3 when is changes from 1 to 8191 is shown. In FIG. 7, it can be seen that the straight line is maintained almost within a certain section. Therefore, the straight line equation can be used to approximate the rest of the values as the slope and the representative value.

In the case of a language learner, voice data has few high frequency components. Therefore, sampling at 11.025 KHZ or 22.05 KHZ is sufficient. To support this case in encoding, modify the psychoacoustic model, or sample the voice data at 44.1KHZ and set the upper (high frequency) subband value to zero using the frequency band selector of 9). In case of 11.025KHZ, only the lower (low frequency) 8 subband values should be taken and the remaining high subband values should be 0. In case of 22.05KHZ, only the lower (low frequency) 16 subband values should be taken and the remaining upper subband values should be 0. . Then, the compression rate can be increased by assigning a value such that the upper subband value becomes 0 during the compression process. In this case, since the synthesis is performed only on the lower subbands without performing the synthesis process on the upper zero subbands, significant gains can be obtained in terms of calculation amount. Through this process, 128Kbps transmission rate is usually required, but high-quality language learning is possible even at 64Kbps transmission rate for voice data.

13) of FIG. 6 is provided for changing the speed / pitch of the language learner, and an implementation thereof will be described with reference to FIG. 8.

8 is a flowchart illustrating a process of performing variable speed and pitch conversion of voice data. 8) in FIG. 8, PCM data restored by the MP3 decoder is input as a data input process. The MP3 decoder restores and outputs one of the 11.025 / 22.05 / 32 / 44.1 / 48 KHz sampling rates. In order to process MP3 decoding and variable voice speed in real time, a lot of memory is required, but we devise a method to reduce the amount of data to be processed in order to reduce the amount of data memory and increase the processing speed. The 11.025 / 22.5 KHz sampling rates which are not supported by the conventional MP3 will be described as follows. Human voices have a relatively low frequency, so down-sampling to 11.025 / 22.05KHz is enough to process the signal. For down-sampling, one method is used for every 4 (11.025 KHz) or 2 (22.05 KHz) samples. Down-sampling reduces the memory size, and in case of voice data, one (11.025KHZ) or 2 (22.05KHz) data output from the MP3 decoder is used to process the SOLA algorithm used in the speed / pitch converter of the present invention. This is an advantage of solving the synchronization problem. In particular, when implementing the SOLA algorithm as an integrated circuit in this way, the memory capacity is greatly reduced, which brings a significant effect. Since 1 granule (576 samples) is restored and output from the MP3 decoder, when down sampling, only 576/4 = 144 (11.025KHz) and 576/2 = 288 (22.05KHz) samples are stored and processed in the speed / pitch variable.

Referring to 2) of FIG. 8, an approximation value such as a synthesis reference point, a buffer, and a window size is obtained in relation to a velocity transformation and a pitch transformation. When the variable speed control key is input from the outside, it is obtained by using the Lookup-table which stores the approximate synthesized reference point (win_var) corresponding to the variable voice speed. In general, the SOLA algorithm requires about twice the buffer size for cross-corelation calculations. However, in the present invention, in order to minimize the buffer size, the data processing method is configured to process by 1/3 (win_fix) of the window size so that the buffer size only needs 4/3 of the window size.

The fine tuning interval for the composite reference point due to the speed variation is obtained by cross-correlation, but when the speed is slow, the fine tuning interval is set to be biased to the left from the composite reference point. To speed up, set the fine tuning interval to the right at the composite reference point. The method of deflecting the fine tuning interval to the left and right does not need to measure the similarity for all the fine tuning intervals of the left and right, so the calculation amount is significantly reduced.

8 is a process of cutting the input data to a predetermined size while moving the window at a predetermined interval in the same manner as the conventional SOLA.

Step 4) of FIG. 8 is a process of obtaining an optimized fine tuning interval based on the synthesis reference point. If two waveforms are attached at an optimized fine tuning interval point where the similarity of the two waveforms is large, although some sound quality deterioration is caused, generally, You will get satisfactory speed conversion results. By adjusting the fine tuning interval, cross-correlation (Rxy) is obtained at the interval where two window blocks to measure similarity overlap. Finding the fine tuning interval where Rxy is the maximum and synthesizing the two signals at that location results in a near-circular waveform with less distortion and tempo change.

The maximum computation time of the SOLA algorithm is spent in calculating Rxy. A large overlap section at a given window size means that it takes a lot of time to find one Rxy due to the large amount of data to be calculated. Therefore, it is necessary to shorten this section to shorten the calculation time. The reduction method of overlap section used in the present invention takes into account the characteristics of the waveform—periodicity. That is, since the waveforms of the two window blocks are generated from one waveform, it is assumed that a large Rxy value in the overlap part maintains the same shape, so that the similarity will be maintained in the rest of the overlap part.

Even if Rxy is found in the overlapped partial interval, Rxy still has a large amount of calculation because Rxy performs a complex operation of addition, multiplication, and division. So we devise a way to reduce the number of operations in Rxy.

^{Rxy = (ΣXY) 2 / ΣX} 2 * ΣY 2

In general, since the sign of the voice waveform data is kept the same sign for a certain period of time, extracting only a few samples at a constant rate and calculating the formula can have the same effect as calculating the entire overlapped section.

We devise a way to speed up the division operation. If the data consists of 16-bit samples, normalize the samples X and Y to 2 ¹⁶ , and then subtract the denominators from the numerator of Equation 9, and correspond to the "leading non zero" -1 of the data values of X and Y. By shifting the Rxy values to the right as much as the value, we can see that the result close to 0 increases the similarity. The normalization process and "leading-nonzero" -1 are easily implemented with the shift operation, and the rest are subtracted. In addition, when two signal products of a molecular term are continuously negative in obtaining Rxy, since the data corresponding to the two window blocks have different codes, the fine tuning interval may be changed and the following may be performed.

In the present invention, a time reduction method was devised to find the fine tuning interval. The Rxy value of the different result will be generated at each position of the fine tuning interval, but tends to have a relatively approximate Rxy value at the fine tuning interval positions in the immediate neighboring case. Using this, we devise a way to search by retrieving fine-tuning interval points by 2-point instead of 1-point. In addition, if more than a certain amount of Rxy occurs by applying the threshold value, it is selected as the optimum fine tuning interval, which reduces the time for obtaining the fine tuning interval.

In the process 5) of FIG. 8, two consecutive blocks are synthesized based on the fine adjustment interval found in FIG. 9 4).

In step 6) of FIG. 8, a buffer space for data output is set. If a large amount of data is stored and then goes to the sound output, the entire system operates more stably by reducing the time load on the speed / pitch variable. The actual buffer size depends on the sampling rate at which the data is output.

The additional data processing unit will be described.

The additional data processor supports voice data sentence information, text representation information of voice data sentence, still / video information, system control and repetitive learning functions. Additional data used by the language learner is represented in b) of FIG. 4. The start indicator of the voice data sentence uses hex (01), which is the start of heading character, and the hex (04), which is the character of "End of Transmission", as the end indicator. The start and end indicators of a voice data sentence are indicators of the beginning and end of a sentence of voice data. Stores data in a repeat mode buffer for repeat mode operation.

The method of expressing the characters constituting the voice data sentence is currently defined in ISO [Ref. G, h]. ISO defines internationally used ASCII codes to represent characters in the ISO-646 and ISO-2022 standards. IOS-646 and ISO-2022 are used for the text representation of the language learner. The additional data used to express the characters constituting a sentence in the language learner includes the national indicator / country code indicating the language used and the start / end indicator of the characters constituting the voice data sentence and the characters constituting one sentence. There is a number. Nationality indicator to distinguish nationality uses hex (06) which corresponds to “acknowledge” of ASCII code and assigns 8bit to country code to distinguish countries. Regardless of the country code, the beginning of the characters that make up the voice data sentence use hexa (02), the "start of text" of the ASCII code, as the charater header, and the "end of text" of the ASCII code to indicate the end of the character. Hexa (03) is recognized as the end of the entire string. Additional data following the start indicator of a letter means a lookup-table address that contains only the letter. When the character of the language is recognized, the number of characters constituting the voice data sentence included in the additional data is displayed on the actual screen by using the lookup-table and the font lookup-table in which the characters are stored.

The speed indicator is used to indicate the pronunciation speed of the character and the time signature of the music, and is assigned to hex (07) corresponding to the "Bell" of the ASCII code. The speed is assigned 1/32 beats (0.031 seconds) to 1 for one frame sync for one character, or 2 for 1/16 beats (0.063 seconds). The system controller counts the number of frame syncs expressed in speed and processes them one by one.

The image of the language learner is divided into a video and a still image.In the case of a still image, the start address of each video is stored in the lookup-table and the start address of each first video data of the various videos in the lookup-table. The address is used to access the actual video data storage medium.

An external cpu command indicator is illustrated in FIG. 12 to add branch cpu commands to encoded data to set branches, interrpts, and the like at a specific frame moment, or to describe the starting moment or operation of the processor for effect sound processing and image effect processing. The external cpu command directive is assigned to hex (1a), which is the "Substitute" in ASCII code.

Accordingly, digital variable signal processing techniques are employed to achieve speed variation in a wide range of areas. In addition, due to the development of semiconductor and personal computer technologies, even highly compressed data can be processed in real time even with high decoding techniques. In addition, the development of a real-time processing language learning system for variable pitch and speed of MP3 data, which is not available in the past, pursues text, background and video effects as a function of an additional language learning system, and solves the problem of synchronization between an MP3 audio signal and a character. It can be solved.

The language learner can use various storage media and is a new concept language learning system that is not restricted by time and space. In addition, the use of language learners online will rapidly increase in view of the rapidly evolving network as a new language learner concept that can construct a low cost language learning system using a PC. The present invention can be used with reference to various processing parts of the present invention not only for processing voice and music data for MP3, but also for processing of music data as well as speed and pitch change of voice signals implemented by other compression algorithms.

In the above-described and illustrated examples, the method of encoding / decoding MP3 data according to the present invention has been described as being used in a language learning system. However, the method of encoding / decoding MP3 data according to the present invention is not limited thereto. Of course, it can be applied to the fields that need to change the speed of sound and pitch such as demand and audio on demand.

Therefore, according to the MP3 data encoding / decoding method of the language learning apparatus according to the present invention and the language learning apparatus thereby, the voice learning apparatus has a high sound quality, high efficiency, a high function, and a large capacity, and receives voice data compressed by the MP3 method without any damage to sound quality. It can be variable, and the effect of speed variation, pitch conversion, and character caption of language learning data can be realized.

In addition, the price of the language learning apparatus becomes relatively high, and the speed can be changed in consideration of the learning ability of the language learner, and the character caption function can be obtained.

Claims (5)

In the encoding / decoding method of a language learning apparatus having a modified MP3 encoder for compressing and encoding audio data and a modified MP3 decoder for decoding the encoded data, A first step of the modified MP3 encoder compressing and encoding audio and the additional data by an MP3 method; A second step of separating the additional data from the compressed data by the modified MP3 decoder and decoding the main data; And a third step of varying the pitch and speed of the decoded main data and expressing a character of the additional data. The method of claim 1, Decoding using linear interpolation to reduce storage capacity when calculating is ^4/3 in the modified MP3 decoder; Implementing encoding / decoding by transforming the lower / higher subbands with a frequency band selector to support 11.025 KHz and 22.05 KHz in sampling frequency for language learning voice data; Amplifying the energy of music or voice selected by the frequency band selector; And And storing the compressed data through the buffer device for repetitive learning in units of sentences. The method according to claim 1 or 2, Supporting 11.025 / 22.05KHz by downsampling of a speed / pitch variable that performs voice rate and pitch conversion and reducing memory; Improving the speed of storing the synthesized reference point as a lookup-table; Deflecting around the composite reference point of the fine tuning interval according to the speed change; And A method of encoding / decoding a language learner further comprising using a normalization calculation, a subtraction operation, and non-leading zero for cross-correlation (Rxy). A modified MP3 encoder for compressing and encoding audio data; A modified MP3 decoder for decoding data encoded by the modified MP3 encoder; An iterative learning buffer for storing compressed data; An amplifier for amplifying the energy attenuated by the frequency band selector; Subband selector to support 11.025 / 22.05KHz; A speed / pitch variable for down-sampling PCM voice data from the modified MP3 decoder and processing speed and pitch variation of the voice signal; And And a system controller for controlling the encoder, the decoder, the buffer, the frequency band selector, and the speed / pitch variable. 5. The apparatus of claim 4, wherein the modified MP3 encoder comprises a data formatter, the data formatter comprising: a speech data sentence start / end indicator; Frame number of speech data sentence construct; Nationality indicator; country code; Character start / end indicators for voice data sentences; Character table address; Number of characters for spoken data sentences; Pronunciation speed indicator; Number of frames for indicating the pronunciation speed of one character; Freeze / movie indicator; Stop / start video address; External cpu command indicators; A language learning device comprising inserting additional data such as a cpu command and encoding the same.