KR20050117607A - Sync signal insertion/detection method and apparatus for synchronization between audio contents and text - Google Patents

Abstract

A method and apparatus for inserting a synchronization signal into an audio file for synchronizing and outputting text upon reproduction of an audio file is disclosed. First, information about the size of the first portion of the frame is obtained from the second portion of the frame. Then, based on the obtained information, the start position and size of the third part of the frame are determined, and at least a part of the synchronization signal is inserted into the third part of the frame. Therefore, the synchronization signal can be effectively inserted into the audio file without damaging the audio content.

Description

SYNC SIGNAL INSERTION / DETECTION METHOD AND APPARATUS FOR SYNCHRONIZATION BETWEEN AUDIO CONTENTS AND TEXT}

The present invention relates to audio content, and more particularly, to a method and apparatus for synchronizing between digital audio content and text in a portable digital playback device.

In recent years, in response to the development of computer technology, a technology for reproducing audio content using a computer has been rapidly developed. Accordingly, attention has been paid to a function of visually displaying the content of the audio content while playing the audio content. For example, a technology for displaying the lyrics on the screen while playing audio content related to the song corresponds to this.

Referring to FIG. 10, a configuration of simultaneously displaying audio content and content content in the prior art will be described.

First, an audio content to be played back and a text file storing the content of the audio content are provided. FIG. 10 is a diagram illustrating a reconstruction of a text file storing contents of conventional audio content in the form of a table. The text file stores not only the contents of the audio contents, but also the playback time for visually displaying the contents of the audio contents. In the example of FIG. 10, while the compressed voice or music file is being reproduced, a reproduction time point that informs the time to output characters to the liquid crystal is stored in units of 1/1000 second.

For example, at a playback time of 0000040 ms, audio content is played back, and a character string corresponding to the audio content is visually output through a predetermined display. As the audio content is played back, at the playback time 0001055 ms, at the same time as the audio content is played, a string "while playing music or an audio file" is output.

That is, the playback time is monitored while the audio content is being played back, so that the output string is output when the playback time coincides with the playback time of the output string shown in the table.

The structure of the text file as described above is substantially similar to the structure of, for example, a ".smi file" for outputting subtitles in a video, and is preferably used when sufficient resources are provided, such as a computer.

However, there is a limit to the resources available when synchronizing text with digital audio content in a digital portable playback device. It is not practical to monitor the playback time in ms of audio content in a digital portable playback device. Therefore, there is a difficulty in outputting text in accordance with such minute reproduction time. Therefore, the above-described method is not suitable when playing back in a digital portable playback apparatus.

In order to achieve a specific purpose, the digital content may contain a watermark. In general, the watermarking technology refers to a technology for storing information of a work which is not recognized by the general public in a sound recording for copyright protection of a work and for determining whether the work is forged or forged. Since watermarking technology conceals user-defined information in the actual sound source of the copyrighted work, it is common to use robust watermarks that are robust against signal processing attacks, compression conversion, etc., and that are difficult to remove for malicious purposes. to be.

Since watermarking inserts data into a sound source of digital content, a large amount of computation and computation must be performed because a fairly complicated calculation process must be performed to detect hidden information again. For this reason, since watermarking technology usually consumes a considerable amount of resources to implement a DSP, it is difficult to use a portable digital playback device such as a portable MP3 player using such a DSP. In addition, an additional function that consumes a lot of resources is undesirable in view of the limited battery life of the portable playback device. In particular, since most audio data is in a format for compressing target content, conventional watermarking techniques are not available.

As a technique for hiding information in compressed data, MP3Stego (Computer Laboratory, Cambridge, August, 1998) proposed by F. Petitcolas is disclosed. This technique has a problem in that high-speed insertion processing is not possible because data is concealed during the compression of the sound source.

Also, Non-Invertible Watermarking Methods For MPEG Encoded Audio (Security and watermarking of Multimedia Contents, January 1999) proposed by L. Qia and K. Nahrstedt has a high possibility of altering the sound source of MP3 and has a limited amount of concealable information. There is this.

Also, A compressed-domain watermarking algorithm for MPEG Audio Layer 3 (ACM Multimedia 2001, Septemper 30-October 5, Ottawa, Ontario, Canada) proposed by DK Koukopoulos and YC Stamatiou can be used for high-speed extraction, but not for fast insertion. There is a problem.

Disclosure of Invention The present invention has been made to solve the above-described problems, and an object thereof is to provide a synchronization signal insertion method and apparatus for synchronizing audio content with text, which enables high-speed insertion / processing while minimizing the effect on sound quality. It is done.

Another object of the present invention is to provide a synchronization signal detecting method and apparatus for detecting a synchronization signal from an audio file into which a synchronization signal is inserted.

In order to achieve the above object, the present invention obtains information about the size of the first portion of the frame from the second portion of the frame, based on the obtained information, the starting position of the third portion of the frame And determining a size, and inserting at least a portion of a synchronization signal into the third portion of the frame, the first portion storing audio content, the second having at least information about the size of the first portion. A method of inserting a synchronization signal into an audio file includes a portion and a plurality of frames each having a third portion whose starting position and size are determined according to the size of the first portion.

The sync signal also includes information about the position of the text corresponding to the first portion of the frame.

The inserting of at least a portion of the synchronization signal into the third portion of the frame may include determining whether to insert a synchronization signal into the third portion of the frame, and in response to determining whether to insert a synchronization signal; And inserting text information corresponding to the first portion of the frame into the third portion of the frame.

In the inserting of at least a part of the synchronization signal into the third portion of the frame, the synchronization signal insertion region in the third portion may be compared with the magnitude of the synchronization signal to insert the synchronization signal in the third portion. If the area is smaller than the size of the synchronization signal, inserting a portion of the synchronization signal with the same size as the synchronization signal insertion area into the third portion.

In addition, the audio content may be generated by converting the text to text-to-speech (TTS).

Further, the third portion includes an area indicating the presence of a synchronization signal and an area indicating the content of the synchronization signal.

On the other hand, the present invention is based on the information on the size of the first portion, the step of extracting information about the start position and size of the third portion, by analyzing the third portion, to determine the presence of a synchronization signal And in response to determining the presence of the synchronization signal, obtaining at least a portion of the synchronization signal from the third portion, the first portion having audio content stored therein, the first having at least information about the size of the first portion. A method of detecting a synchronization signal from an audio file includes a plurality of frames each having a second portion and a third portion whose starting position and size are determined according to the size of the first portion.

On the other hand, the present invention is based on the information on the size of the first part, extracting information about the start position and size of the third part, and analyzing the third part, the synchronization to determine the presence of the synchronization signal A first portion storing audio content having a signal presence determining portion and a synchronizing signal obtaining portion for obtaining at least a portion of the synchronization signal from the third portion in response to determining the presence of the synchronization signal; An apparatus for detecting a synchronization signal from an audio file comprising a plurality of frames each having a second portion having information about and a third portion whose starting position and size are determined according to the size of the first portion.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram illustrating an overall process for synchronizing music files and lyrics text in a digital portable playback device.

Referring to FIG. 1, first, a music file and lyrics text are input to a text synchronization program. Using the inputted information, the text synchronization program directly receives a time point at which each lyrics should be output from the user. As shown in FIG. 10, the information input from the user may include information to which text to be output and playback time are connected. The text synchronizing program inserts information indicating a predetermined position of lyrics text for lyrics output corresponding to a predetermined position of a music file according to the synchronization signal insertion method according to the present invention.

Then, when the music file is played back by the portable playback device, if a sync signal is detected during music file playback, the sync signal is analyzed and the character string corresponding to the predetermined position of the text displayed by the sync signal is played. Output through the display means of.

Meanwhile, the process of synchronizing the audio file and the lyrics text according to the present invention may be generated using a text-to-speech (TTS) engine. 2 is a conceptual diagram illustrating a process of synchronizing text with a voice file generated by the TTS technology.

TTS is a technology that synthesizes text into speech files. In converting text characters into speech files, TTS is constructed by phoneme DB with minimum pronunciation unit for each language. The phoneme DB is synthesized to generate a voice signal.

In the configuration of the present invention described above with reference to FIG. 1, the position of text for synchronizing with an audio file must be directly input from a user. However, in the case of speech synthesis by TTS, a text file corresponding to the voice file is generated at the same time. Because the location of the text is automatically detected, no separate user input process is necessary.

Hereinafter, although the format of the music file is described as MP3 in the embodiment of the present invention, in the case of music files stored according to other audio file formats such as WMA, AAC, and AC3, the synchronization signal insertion method according to the present invention may be applied or applied. Those skilled in the art will appreciate.

3 is a diagram illustrating a structure of an MP3 frame. Referring to FIG. 3, the structure of the MP3 frame will be described. The MP3 audio file includes a plurality of frames, and each frame includes a sink 301, side information 303, and main data, each of which has 12 bits of sync bits. 305, and stuffing bits 307.

The main data 305 is losslessly compressed and stored with audio content according to Huffman Coding. The lossless compressed main data 305 is stored in units of bytes, and as a result of Huffman coding, excess bits are generated in which the content of the audio content is not included at all. Such surplus bits are called stuffing bits 307. That is, these bits are empty spaces that are not used at all when playing music. Since the stuffing bit 307 is a bit for making the size of the frame including the main data 305 in units of bytes, the size of the stuffing bit 307 is equal to the size of the main data 305 generated by Huffman coding audio content. Is determined accordingly.

As will be described in more detail below, the present invention inserts a synchronization signal into the stuffing space by using the structural characteristics of the frame.

4 is a flowchart illustrating a synchronization signal insertion process according to a first embodiment of the present invention. Referring to FIG. 4, first, when an MP3 audio file is selected, the MP3 audio file is divided in units of frames (S501).

For each divided frame, frame analysis is performed (S503). The frame analysis analyzes the sink 301 and the sub information 303 to obtain information about the start position and the size of the main data 305. Then, based on the size of the main data 305, the size and position of the stuffing bit 307 is obtained.

Based on the size of the main data 305, if the stuffing bit 307 does not exist, but it is determined that a space for inserting the synchronization signal is necessary, the stuffing bit 307 is generated (S507). In this case, one byte is newly allocated for the stuffing space, and thus, all subsequent frames are reconstructed so as to push backward by one byte (S509).

Then, it is determined whether or not a synchronization signal should be inserted into the frame (S511). When the sync signal is to be inserted, the sync signal is inserted into the stuffing bit (S513). Since the size of the synchronization signal is generally larger than the number of bits of the stuffing space, a part of the synchronization signal is inserted into one stuffing space, rather than all of one sync signal into one stuffing space. That is, one synchronization signal is inserted into the plurality of stuffing spaces. The synchronization signal inserted into the stuffing space includes a portion indicating the presence of the synchronization signal and a portion indicating the position of the text and the number of characters of the output text.

By repeating the above process for each frame, a synchronization signal is inserted into an audio file consisting of frames.

Next, a second embodiment of the present invention will be described with reference to FIGS. 5 and 6. 5 is a flowchart illustrating a synchronization signal insertion process according to a second embodiment of the present invention.

Although not shown in FIG. 5, the members shown by reference numerals 501 to 509 of FIG. 4 exist the same as before 611 of FIG. 5, but are omitted for convenience of illustration and description, and the configuration and operation of FIG. 4 are omitted. Same as those shown and described by reference numerals 501 to 509.

At 611, it is determined whether the synchronization signal needs to be inserted. If it is necessary to insert the synchronization signal, the synchronization signal is inserted into the stuffing space (S613).

When the synchronization signal does not need to be inserted, text is inserted into the stuffing space (S615).

When the sync signal is to be inserted, the sync signal is inserted into the stuffing bit (S615). As described above with reference to FIG. 4, since the size of the synchronization signal is generally larger than the number of bits of the stuffing space, the entire synchronization signal is not inserted into one stuffing space, but a part of the synchronization signal is stuffed into one stuffing. Insert it into the space. That is, one synchronization signal is inserted into the plurality of stuffing spaces. It is sufficient that the synchronization signal inserted into the stuffing space includes only a portion indicating the presence of the synchronization signal. In the reproduction of the audio file, since the information stored in the stuffing area of the previous frames of the frame in which the synchronization signal was detected is a piece of text information, combining these results in the text to be output to the display upon detection of the presence of the synchronization signal. to be.

By repeating the above process for each frame, a synchronization signal is inserted into an audio file consisting of frames.

FIG. 6 is a schematic diagram illustrating an audio file in which a synchronization signal is inserted according to a second embodiment of the present invention in units of frames. In FIG. 6, an audio file is divided into frames and schematically illustrated. For each frame, in the stuffing area, the playback time of a frame containing text information or a synchronization signal and including a synchronization signal is a point in time at which the text inserted in the previous frame is outputted. .

Hereinafter, a synchronization signal detection process according to the present invention will be described.

7 is a schematic diagram schematically illustrating a synchronization signal detection process according to the present invention.

Referring to FIG. 7, the synchronization signal detecting apparatus according to the present invention includes a memory 701, a frame analyzer 703, a stuffing bit identifier 705, a synchronization signal and text configurator 707, an LCD controller 709, and LCD 711.

The memory 701 stores MP3 audio files. In response to the playback command for the MP3 audio file, the information of the MP3 audio file is read from the memory 701 and transmitted to the frame analyzer 703 in the form of an MP3 stream.

The frame analyzer 703 divides the audio file transmitted in the form of an MP3 stream in units of frames.

The stuffing bit identifier 705 then extracts the size of the audio content using the sync and sub information for each frame. Since the bit size and location of the stuffing area can be known based on the size of the audio content, the bit size and location of the stuffing area can be determined accordingly. Thereafter, information about the presence of the stuffing bit and the location and size (if any) is sent to the sync signal and text composer 707.

The synchronizing signal and text configurator 707 analyzes the contents of the detected synchronizing signal to determine the position of the text in the text file indicated by the synchronizing signal and the length of the character string to be displayed, and the corresponding portion of the character string to the text file. Read from On the other hand, in the case of the second embodiment in which the text is included in the MP3 audio file, when there is no synchronization signal, the bit content of the stuffing space is read out and stored in a separate memory space continuously, and the synchronization signal When the presence of is detected, the content stored in the memory space is output as text. After output as text, the contents are removed from the memory space. Thereafter, the character string composed of the text is transmitted to the LCD controller 709.

Thereafter, the LCD controller 709 controls the LCD 711 to erase the character string currently output to the LCD 711 and output a new character string. In this case, if it is necessary to output text longer than a string that can be printed on the LCD at the same time, the character string can be automatically scrolled from right to left, and this scrolling process is known to those skilled in the art.

The sync signal detecting apparatus of FIG. 7 may be implemented in a digital portable playback apparatus as shown in FIGS. 8 and 9. Although it is common to implement in the DSP, the text synchronization operation is controlled by all external devices in MICOM, so if there are enough resources in MICOM, it is advantageous to implement in MICOM as shown in FIG. Since the processing speed and memory required to implement synchronization by the method proposed by the present invention are very small, it is possible to process in MICOM.

8 is an internal configuration diagram when the synchronization signal detection apparatus for text synchronization according to the present invention is implemented in the DSP of the portable digital playback apparatus, and FIG. 9 is an internal configuration diagram when the DSP is implemented in the portable digital playback apparatus. .

8 and 9 are internal configuration diagrams of a general playback apparatus. When a user presses a play button, the microcomputer brings a file name to be played back. After getting the name of the file to play, the data of the file is read and transferred to the buffer, and the DSP decodes the compressed data in the buffer to play music through the speaker.

In this process, when the present invention which displays the voice information of the lyrics or the file to be reproduced on the liquid crystal is inserted, the overall structure is changed as follows. The process of importing a file to be played in Micom is the same. After retrieving the file to be reproduced, the data read from the file is transferred to the buffer, and the synchronization signal detector determines whether the transferred data has a synchronization signal. At this time, when the sync signal is found in the sync signal detector, the controller of the Mikong finds the sync signal and tells what the content of the found sync signal is. The microcomputer's LCD controller sends the information from the sync signal detector to the liquid crystal display.

The difference between FIG. 8 and FIG. 9 differs only in where the sync signal detector is located. The overall execution procedure is the same regardless of the shape of the portable reproducing apparatus.

The invention has been described with reference to specific embodiments for specific applications. Those of ordinary skill in the art, having access to the present teachings, may know additional variations, applications, and embodiments within the scope.

Accordingly, the appended claims are intended to cover any and all such applications, modifications, and embodiments within the spirit of the invention.

The present invention provides a function capable of displaying lyrics or voice content of music automatically reproduced while reproducing a music file or an audio file by adding a text synchronization device to the digital portable player.

The present invention detects a synchronization signal hidden in a music file in real time while the compressed file is being reproduced, and displays it on the LCD screen in synchronization with the time point at which the content file is currently played. Therefore, the user can check the content currently being played back through the LCD screen of the playback device. In addition, by hiding all the information in the digital content until the text information and the time at which the text should be output, the user does not need to additionally store the text file or other information in his portable playback device.

The present invention can be used effectively in a digital portable playback device for language learning because it can comprehensively utilize the contents of teaching materials for foreign language learning as well as lyrics of general music.

The devices that display text in the existing playback devices randomly output text information according to the playback time. Therefore, the actual playback content does not match the output on the liquid crystal display. The function is not implemented.

According to the present invention, it is possible to accurately output the contents reproduced on the LCD screen while the various contents of the compressed format file are reproduced.

1 is a conceptual diagram illustrating an overall process for synchronizing music files and lyrics text in a digital portable playback device.

2 is a conceptual diagram illustrating a process of synchronizing text with a voice file generated by the TTS technology.

3 is a diagram illustrating a structure of an MP3 frame.

4 is a flowchart illustrating a synchronization signal insertion process according to a first embodiment of the present invention.

5 is a flowchart illustrating a synchronization signal insertion process according to a second embodiment of the present invention.

FIG. 6 is a schematic diagram illustrating an audio file in which a synchronization signal is inserted according to a second embodiment of the present invention in units of frames.

7 is a schematic diagram schematically illustrating a synchronization signal detection process according to the present invention.

Claims (11)

A plurality of frames each having a first portion in which audio content is stored, a second portion having at least information about the size of the first portion, and a third portion whose starting position and size are determined according to the size of the first portion; In the method for inserting a synchronization signal into an audio file comprising: Obtaining information about the size of the first portion of the frame from the second portion of the frame; Determining a starting position and size of a third portion of the frame based on the obtained information; And And inserting at least a portion of the synchronization signal into the third portion of the frame. The method of claim 1, And the synchronization signal includes information about a position of text corresponding to the first portion of the frame. The method of claim 1, Inserting at least a portion of a sync signal into the third portion of the frame, Determining whether to insert a synchronization signal into the third portion of the frame; And And in response to the non-insertion determination of the synchronization signal, inserting text information corresponding to the first portion of the frame into the third portion of the frame. The method according to any one of claims 1 to 3, Inserting at least a portion of a sync signal into the third portion of the frame, When the synchronization signal insertion region in the third portion is compared with the magnitude of the synchronization signal, and the synchronization signal insertion region in the third portion is smaller than the size of the synchronization signal, And inserting a portion of the synchronization signal into the third portion. The method of claim 1, And the audio content is generated by text-to-speech (TTS) conversion of the text. The method of claim 1, And the third portion includes an area indicating the presence of a synchronization signal and an area indicating the content of the synchronization signal. A plurality of frames each having a first portion in which audio content is stored, a second portion having at least information about the size of the first portion, and a third portion whose starting position and size are determined according to the size of the first portion; In the method for detecting a synchronization signal from an audio file comprising: Extracting information about a start position and a size of the third part based on the information about the size of the first part; Analyzing the third portion to determine whether a synchronization signal is present; And In response to determining the presence of the synchronization signal, obtaining at least a portion of the synchronization signal from the third portion. The method of claim 7, wherein And in response to determining the absence of a synchronization signal, extracting text information from the third portion. The method of claim 7, wherein And analyzing the content of the synchronization signal, and then selecting a position of a corresponding text based on the analysis. The method according to claim 7 to 9, And combining at least a portion of the synchronization signal with at least a portion of the synchronization signal of a subsequent frame if at least a portion of the synchronization signal obtained from the third portion is not the same as the synchronization signal. Signal detection method. A plurality of frames each having a first portion in which audio content is stored, a second portion having at least information about the size of the first portion, and a third portion whose starting position and size are determined according to the size of the first portion; An apparatus for detecting a synchronization signal from an audio file comprising: On the basis of the information on the size of the first portion, information on the start position and size of the third portion is extracted, and the third portion is analyzed to determine whether there is a synchronization signal presence plate to determine the presence of a synchronization signal. government; And And a synchronizing signal acquiring unit for acquiring at least a portion of the synchronizing signal from the third portion in response to the determination of the existence of the synchronizing signal.