KR20060119533A - Computer recordable medium recording multimedia file for audio/video syncronization and syncronizing device of audio/video - Google Patents

Abstract

A recording medium readable by a computer in which a multimedia file composing program for audio/video synchronization and an audio/video synchronization apparatus are provided to download and reproduce a multimedia data file to synchronize audio data with video data, reduce a calculation load of the audio/video synchronization apparatus, and efficiently use a memory space. A computer readable recording medium includes a data input module(105) for receiving image data samples and audio data, an index generating module(110) for sequentially giving index information to the image data samples, and an index matching module(120) for dividing a representation region transformed into a predetermined characteristic value by the audio data into predetermined sections and generating information about matching the index information to the divided representation regions. The computer readable recording medium further includes an audio analysis module(125) for transforming the audio data into the representation region and giving index information to the representation region according to the matching information, and a file generating module(130) for preparing a file operation field and sequentially recording the audio data, a tag data stream and the index information given according to the matching information to generate a multimedia data file.

Description

Computer recordable medium recording multimedia file for audio / video syncronization and syncronizing device of audio / video}

1A is a graph showing a typical wave waveform showing negative pressure over time in the prior art.

1B is a graph showing the frequency level of sound appearing at a specific time in the prior art.

2 is a block diagram schematically showing the components of a computer on which an audio / video synchronization multimedia file creation program according to the present invention is executed;

3 is a diagram illustrating a structure of a data field recorded by a file generation module of a computer on which an audio / video synchronization multimedia file creation program according to the present invention is executed;

4 is a diagram illustrating an example in which a file generation module of a computer on which an audio / video synchronization multimedia file creation program according to the present invention is executed records an index field following an initial index field;

5 is a block diagram schematically illustrating the components of an audio / video synchronization device according to an embodiment of the present invention;

6 is a diagram exemplarily illustrating a form in which an audio / video synchronization device processes an audio / video synchronization multimedia data file according to an embodiment of the present invention;

<Explanation of symbols for main parts of drawing>

100: Computer running the multimedia file creation program for audio / video synchronization

105: data input module 110: index generation module

115: decryption module 120: index matching module

125: audio analysis module 130: file generation module

135: control unit 140: storage unit

145: display unit 150: audio output unit

The present invention relates to a multimedia data file for audio / video synchronization and an audio / video synchronizing apparatus, and more particularly, to a computer in which software for creating multimedia data file for audio / video synchronization is programmed.

At present, personal handheld devices such as mobile phones, PDAs (Personal Digital Assistants), smart phones, etc. are recognized as essential living devices in the real world, and in addition to basic communication functions, such personal handheld devices include e-mail functions, game functions, and camera photography It provides various functions such as function, voice recording function, music playback function, still image or video playback function.

In particular, recently, a personal portable terminal product has been developed that provides a function of displaying a predetermined image as the music is played as well as a music playback function such as an MP-1 (MPEG-1 Audio Layer 3) player. As the music is played, a technology of synchronizing and displaying images corresponding thereto is used.

Conventionally, in order to extract the characteristics of the music mainly uses the wave waveform of the music, for reference, domestic patent application No. As described in 2002-0023140, techniques for synchronizing multiple images and music using maximum wave pitch values per frame in digital audio devices have been proposed.

FIG. 1A is a graph showing a general wave waveform (a, b) showing a sound pressure over a conventional time, and FIG. 1B shows a sound pressure (c, d, e, f) for each frequency appearing at a specific time in the prior art. One graph.

As can be seen in Figures 1a and 1b, the prior art is dependent on the wave waveform according to the sound pressure, the wave waveform shown in Figure 1a is the hourly sound pressure (a, b) output to the left and right speakers It is displaying. In addition, the wave waveform shown in FIG. 1B expresses the sound pressures c, d, e, and f for each frequency using only a few (4) samples.

A general audio / video synchronizing apparatus decodes audio data for each frame, and simultaneously analyzes a wave waveform according to sound pressure and extracts its characteristics.

In addition, the audio / video synchronizing apparatus performs the function of indexing the image data samples according to the grade given in advance for each characteristic and synchronizing and outputting the corresponding image data samples in real time when the characteristics of the audio data are extracted.

However, in this case, as the audio data is reproduced, the computational process that the audio / video synchronizing apparatus needs to perform in real time becomes excessive, and therefore, a considerable load is placed on the hardware resources of the mobile communication terminal equipped with the audio / video synchronizing apparatus.

In addition, when music and multiple images are synchronized with a wave pitch value in real time, since the calculation is performed uniformly regardless of the type of audio data, the moving speed of the image may be too high. Since the difference greatly varies from frame to frame, a problem may occur in which synchronization is unnaturally implemented.

The audio / video synchronization device of the mobile communication terminal executes the multimedia data file by analyzing the presentation characteristics of the audio data and inserting indexing information into the file itself by matching the image data samples according to the level of the analyzed presentation characteristics. It is an object of the present invention to provide a computer-readable recording medium recording a multimedia data file creation program for audio / video synchronization in which audio data and video data are naturally synchronized by downloading and reproducing.

The present invention also provides an audio / video synchronization device by selecting audio data, image data samples, and synchronization periods on an external computer, and generating a new multimedia data file by inserting indexing information between audio data and image data samples in a minimum size. It is another object of the present invention to provide a computer-readable recording medium on which a multimedia data file aptitude program for audio / video synchronization is recorded so that the storage space of the memory can be efficiently used while reducing the computational burden of the computer.

In addition, the present invention analyzes a multimedia data file for audio / video synchronization previously generated by inserting all necessary indexing information in a minimum size, and simultaneously reproduces audio data and indexed image data samples based on the analyzed period information. It is another object of the present invention to provide an audio / video synchronizing apparatus that processes synchronization naturally without a computational burden.

In order to achieve the above object, a computer-readable recording medium that records an audio / video synchronization multimedia file creation program according to the present invention includes a data input module for receiving an image data sample and audio data; An index generation module for sequentially assigning index information to the input image data samples; An index matching module configured to classify the representation area into a predetermined section by processing the audio data into a predetermined characteristic value, and generate matching information between the index information and the divided representation area section; An audio analysis module for converting the audio data into a representation region of the characteristic value in units of a predetermined period and assigning index information according to the matching information; And a file generation module providing a work field for generating the multimedia data file and sequentially recording the index data according to the audio data, the tag data string, and the matching information to generate one multimedia data file. And a multimedia file creation program for audio / video synchronization.

In addition, a computer-readable recording medium recording an audio / video synchronization multimedia file creation program according to the present invention decodes the audio data decoded into a predetermined format into Pulse Code Modulation (PCM) data and decodes the PCM data. It further comprises a decoding module for transmitting to the audio analysis module.

In addition, the data input module of the computer-readable recording medium recording the audio / video synchronization multimedia file creation program according to the present invention provides a user panel to be used as the image data sample from the image data stored on the computer Characterized in that the data is selected.

In addition, the data input module of the computer-readable recording medium recording the audio / video synchronization multimedia file creation program according to the present invention provides a user panel to the audio data to be processed the signal from the audio data stored on the computer It is characterized by being selected.

The data input module of the computer-readable recording medium recording the audio / video synchronization multimedia file creation program according to the present invention may receive the predetermined period and transmit the predetermined period to the audio analysis module.

The file generation module of the computer-readable recording medium recording the audio / video synchronization multimedia file creation program according to the present invention is characterized by recording a tag name and tag size in the tag data string.

The file generation module of the computer-readable recording medium recording the audio / video synchronization multimedia file creation program according to the present invention is characterized by recording the predetermined period in the tag data string.

Further, the file generation module of the computer-readable recording medium recording the audio / video synchronization multimedia file creation program according to the present invention sequentially records the index values, and the first index value is recorded as a predetermined value. The index values sequentially recorded after the first index value may be recorded as an amount of change from the index value preceding the first index value.

The file generation module of the computer-readable recording medium recording the audio / video synchronization multimedia file creation program according to the present invention may further record the image data sample in the work field.

In order to achieve the above another object, an audio / video synchronizing apparatus according to the present invention includes an image tuning index sequence for synchronizing audio data, image data samples, and image data samples at predetermined intervals as the audio data is reproduced. Storage means for receiving and storing a multimedia data file in which a tag data string is recorded to distinguish the predetermined data field from the audio data; Decoding means for decoding and transmitting the audio data into Pulse Code Modulation (PCM) data according to the predetermined period; Audio output means for converting the PCM data from the decoding means into an analog signal and outputting the converted analog signal; And an image for sequentially reading the image tuning index sequence according to the predetermined period as the decoding means is operated and synchronizing and displaying the image data samples matching the read indexes with the PCM data output through the audio output means. It characterized in that it comprises a synchronization means.

Further, in the image tuning index sequence of the audio / video synchronizing apparatus according to the present invention, an initial index value is recorded as a predetermined value, and index values sequentially recorded after the first index value are indexes that precede the self. And the image synchronizing means reads the image tuning index sequence by sequentially reducing the next index value by calculating the change amount to the first index value.

Hereinafter, a computer-readable recording medium and an audio / video synchronizing apparatus for recording an audio / video synchronization multimedia file creation program according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

First, the computer is a device for creating a multimedia data file to be used in the audio / video synchronization device in advance, the storage medium is mounted and the audio / video synchronization multimedia file creation program according to the invention is installed and executed A programmed computer that carries out the process indicated by the file creation program.

2 is a block diagram schematically showing the components of a computer 100 on which an audio / video synchronization multimedia file creation program (hereinafter referred to as "file creation program" according to the present invention) is executed. .

2, the computer 100 in which the file creation program according to the present invention is executed includes a data input module 105, an index generation module 110, a decoding module 115, an index matching module 120, and an audio analysis module. It comprises a 125, a file generation module 130, a control unit 135, a storage unit 140, a display unit 145 and an audio output unit 150, the data input module 105, index generation The module 110, the decoding module 115, the index matching module 120, the audio analysis module 125 and the file generation module 130 is a configuration of the computer 100 functioned by the file creation program according to the present invention. Element.

First, the storage 140 is provided with image data and audio data.

The audio data may be "wav (wave file)", "wma (windows media audio)", "mp3 (mpeg audio layer-3)", "ogg (compressed music file supported by VBR)", " mid (SMF file) " may be used. In an embodiment of the present invention, an “mp3” format is used.

As the image data, still images such as "BMP", "GIF", "JPG", "PCX", "TGA", "TIF", "WMF" format, or "AVI", "MPG", " MOV "," RM "," WMV "format and the like can be used, in the embodiment of the present invention" JPG "format is used.

The controller 135 includes a kernel to provide an interface with each component, to process an interrupt, and to assign a request processing time and an operation rank so that each component is operated in sequence. In addition, the controller 135 manages the address of the storage 140 to control data input / output.

The data input module 105 provides a user panel to select a jpeg file to be used as an image data sample according to the present invention from among jpeg files previously stored in the storage 140, and stored in the storage 140. Among the mp3 files, one mp3 file to be processed by the file creation program according to the present invention is selected.

The user panel provided by the data input module 105 interworks with the audio output unit 150 and the display unit 145 provided in the computer 100 to output mp3 files as a selection target to a speaker or to display jpeg files. By outputting the image, the user can check and select image data or audio data.

At this time, the number of jpeg files to be used as image data samples is adjusted with the number.

In addition, the data input module 105 receives the period information through the user panel, for example, "0.5 seconds", "1 second", etc. may receive the period information.

The period information means a unit of time in which an mp3 file selected through the data input module 105 is synchronized with the selected image data samples.

The index generation module 110 sequentially assigns indexes to the selected image data samples. For example, if ten image data samples are selected, the indexes are numbered from "1" to "10" in the order of the image data samples. Can be priced as.

These indices are selected for each period as the selected mp3 file is played and used to synchronize the corresponding image data samples. Therefore, if the total playing time of the mp3 file is 2 minutes and the period information is set to "0.5 second", the index should be selected a total of 240 times (120 seconds ÷ 0.5 seconds).

The decoding module 115 decodes the audio data in the mp3 file format into PCM (Pulse Code Modulation) data, and transmits the decoded PCM data to the audio analysis module 125. Here, the PCM method is a method of quantizing a frequency waveform of a music signal and expressing its amplitude value in binary. Usually, it is used for recording and reproducing digital music data by sampling at 44.1 KHz and 16-bit quantization method.

The audio analysis module 125 converts the PCM data transmitted from the decoding module 115 into a representation region of the characteristic value on the basis of the selected time period. The index matching module 120 converts the number of the image data samples. The subdivisions are set by dividing the entire sections of the expression region according to the present invention, and matching information between each subregion and the index is generated.

For example, matching information may be generated as an index number "1" in the first section of the expression area and an index number "2" ... in the second section.

As described above, if ten image data samples are selected, the partial region is preferably divided into ten to match the index of the image data samples.

As described above, the audio analysis module 125 converts the PCM data transmitted from the decoding module 115 into the expression region of the characteristic value in units of the selected time period.

For example, the audio analysis module 125 converts the PCM data into a representation region of a sound pressure or a frequency value of a wave waveform and applies the matching information of the region of each sound pressure section set by the index matching module 120. Select the index accordingly.

Through this process, for example, the mp3 data is analyzed for frequency characteristics in units of 0.5 seconds of playback, and image indexes of image data samples matching the class of frequency characteristics are selected to enable 0.5 second image synchronization.

3 is a diagram illustrating a structure of a data field recorded by the file generation module 130 of the computer 100 on which the multimedia file creation program for audio / video synchronization according to the present invention is executed.

The file generation module 130 provides a work field for generating an audio / video synchronization multimedia data file according to the present invention, records the mp3 audio data, and records a tag data string in a next field.

According to Fig. 3, the tag data string is for distinguishing audio data from other data fields. The tag name (A), the tag size (B), and the period information (C) are recorded.

The file generation module 130 records the selected indexes D, D1, D2, D3, and D4 following the tag data string. As illustrated above, the total playing time of the mp3 file is 2 minutes, and the period information is " 0.5 seconds ", the index is screened a total of 240 times (120 seconds ÷ 0.5 seconds), so a total of 240 selected indexes are recorded sequentially.

The present invention is to reduce the computational process burden on the audio / video synchronization device provided on the mobile communication terminal when the above-described process is processed on the mobile communication terminal, the computer that executes the file creation program according to the present invention ( Through operation 100, operations required for synchronization are processed in advance, and all indexes are selected and generated as a single multimedia file, so that the mobile communication terminal only needs to play it.

However, if all 240 indexes are recorded in the multimedia file, the file capacity becomes large. Therefore, when the multimedia data file for audio / video synchronization according to the present invention is downloaded to the mobile communication terminal, inefficiency may occur in terms of memory.

Accordingly, in the file generation module 130 sequentially recording the index values, the first index value D is recorded as a 1 byte size value, followed by the indexes D1, D2, D3, and D4. Calculates and records only the amount of change from the preceding index including the first index D.

In general, when synchronizing audio data and video data, the amount of change of an image is rarely changed rapidly. Therefore, the index is likely to be recorded linearly, and thus the numerical difference (change amount) between front and rear indexes is small. .

The file generation module 130 records the amount of change between the indexes in units of bits (refer to FIG. 3, and confirms that four bits, that is, four next indexes are recorded in one byte after the first index D). The data section in which the entire index is recorded can be greatly reduced.

4 is a diagram illustrating an example in which the file generation module 130 of the computer 100 on which the multimedia file creation program for audio / video synchronization according to the present invention is executed records an index field following an initial index field. .

According to Fig. 4, four bits are sequentially recorded in the byte field after the first index D. The first bit "D1" is "00", which means "no difference from the leading index", and the second bit. "D2" is "01", which means "1 is increased from the leading index value".

In addition, the third bit "D3" is "10", which means "1 is decreased in the leading index value", and the fourth bit "D4" is "11", and "2" is increased in the leading index value (the fourth bit is the environment). Various values can be applied).

For example, when recording only the change amount between indexes in bit units, a data field of about 30 bytes per minute is required for a 0.5 second cycle, and an index information is about 60 bytes per minute for a 0.25 second period. Can be recorded. For reference, since the first index is allocated 1 byte, the numerical range that can be expressed is 0 to 255. This means that 256 image data samples can be selected.

In addition, the file generation module 130 may record image data samples together in the tag data string, and the image data samples are preferably recorded before and after the recording field of the index.

When each data is recorded in the work field through this process, the file generation module 130 finally stores the data in the storage 140 to generate a multimedia data file for audio / video synchronization according to the present invention.

The multimedia data file for audio / video synchronization according to the present invention can be downloaded to a mobile communication terminal equipped with an audio / video synchronization device via a communication port, a transmission / recording medium such as a memory card.

Hereinafter, an audio / video synchronizing apparatus to which an audio / video synchronization multimedia data file generated by the above-described process is applied will be described.

An audio / video synchronization device according to an embodiment of the present invention is provided in a mobile communication terminal.

5 is a block diagram schematically illustrating the components of an audio / video synchronizing apparatus 200 according to an embodiment of the present invention, and FIG. 6 is an audio / video synchronizing apparatus 200 according to an embodiment of the present invention. A diagram illustrating an example of processing a multimedia data file for video / video synchronization.

Referring to FIG. 5, the audio / video synchronizing apparatus 200 according to the embodiment of the present invention includes a storage means 210, a decoding means 220, an audio output means 240, a video output means 250, and an image synchronization means. 230, wherein the storage means 210 includes a multimedia data file for audio / video synchronization according to the present invention described with reference to FIGS. 2 to 4 (hereinafter, "Multimedia data file according to the present invention"). Is downloaded and stored.

The decoding means 220 reads the multimedia data file according to the present invention from the storage means 210 and analyzes the tag data string field. The decoding means 220 separates the audio data by the tag name of the tag data string and checks the period.

The decoding means 220 decodes the audio data into pulse code modulation (PCM) data on the basis of the identified period and transmits the audio data to the audio output means 240.

The audio output means 240 converts the PCM data from the decoding means 220 into an analog signal and outputs it through a speaker.

The image synchronizing means 230 starts an operation together with the decoding means 220 to read the multimedia data file according to the present invention from the storage means 210 and analyzes the tag data string below the tag name field.

The image synchronizing means 230 checks the period and finds the starting point of the index sequence based on the tag size. Subsequently, the image synchronizing means 230 sequentially reads the numerical values from the initial index field according to the identified period, and selects and transmits the image data sample F1 matching the read index to the video output means 250. .

In this case, the image synchronizing means 230 calculates the amount of change expressed by bits to the preceding index value in the case of the index following the initial index or less, and reduces the image index to match the reduced index. Will be screened.

The video output means 250 decodes the image data sample transferred from the image synchronous means 230 and outputs it on the screen.

As can be seen in Figure 6, the decoding means 220 and the image synchronization means 230 simultaneously access and analyze the multimedia data file according to the present invention from the storage means 210, based on the same period (F2) Audio data F3 and image data sample F1, respectively.

In addition, the decoding means 220 and the image synchronizing means 230 transmit audio data and image data samples processed at the same period to the audio output means 240 and the video output means 250, respectively. You can skip the calculation process and play the synchronized music and images automatically.

In addition, the decoding unit 220 and the image synchronization unit 230 may skip the operation of analyzing the reproduced audio data frame by frame and indexing the corresponding image data sample according to the analyzed musical characteristics. Therefore, since the decoding means 220 and the image synchronizing means 230 perform data processing independently of each other, it is possible to considerably reduce the burden of the calculation process in this respect.

The present invention has been described above with reference to the preferred embodiments, which are merely examples and are not intended to limit the present invention, and those skilled in the art to which the present invention pertains do not depart from the essential characteristics of the present invention. It will be appreciated that various modifications and applications are not possible that are not illustrated above. For example, each component specifically shown in the embodiment of the present invention can be modified. And differences relating to such modifications and applications will have to be construed as being included in the scope of the invention defined in the appended claims.

The present invention has the following effects.

First, according to the present invention, a method of synchronizing audio / video data by analyzing and indexing an existing multimedia data file in real time on a mobile communication terminal, and analyzing and indexing data and inserting index information into multimedia data The process of generating the data is performed by an external computer, and the mobile communication terminal simultaneously reproduces each data based on periodic information, thereby minimizing the use of hardware resources of the mobile communication terminal and automatically synchronizing music and images. .

In addition, according to the present invention, there is no need for a separate device or complicated software for synchronizing the music and the image of the multimedia data, the advantage that can be easily implemented in the conventional mobile communication terminal regardless of its type have.

In addition, according to the present invention, since the user can freely create a multimedia data file for audio / video synchronization by selecting image data samples, audio data and synchronization periods, etc., and downloading them to a mobile communication terminal. It is possible to maximize the effect of the appreciation, it is possible to produce a variety of multimedia data files to reflect the taste of the individual.

Claims (11)

A recording medium having recorded thereon a program which is executed on a computer to create a multimedia data file. A data input module configured to receive image data samples and audio data; An index generation module for sequentially assigning index information to the input image data samples; An index matching module configured to classify the representation area into a predetermined section by processing the audio data into a predetermined characteristic value, and generate matching information between the index information and the divided representation area section; An audio analysis module for converting the audio data into a representation region of the characteristic value in units of a predetermined period and assigning index information according to the matching information; And By providing a work field for generating the multimedia data file to sequentially record the index data according to the audio data, the tag data string and the matching information to generate a multimedia data file to function as a file generation module A computer-readable recording medium that records a multimedia file creation program for audio / video synchronization. The method of claim 1, And a decoding module for decoding the audio data decoded in a predetermined format into PCM (Pulse Code Modulation) data and transferring the PCM data to the audio analysis module. A computer-readable recording medium that records the data. The method of claim 1, wherein the data input module A computer-readable recording medium having recorded thereon a multimedia file creation program for audio / video synchronization, characterized by providing a user panel and selecting image data to be used as the image data sample from image data stored on the computer. The method of claim 1, wherein the data input module A computer-readable recording medium having recorded thereon a multimedia file creation program for audio / video synchronization, characterized in that a user panel is provided to receive the audio data to be processed from the audio data stored on the computer. The method of claim 1, wherein the data input module And a computer-readable recording medium having recorded thereon the predetermined period and transmitting the predetermined period to the audio analysis module. The method of claim 1, wherein the file generation module And a tag name and a tag size are recorded in the tag data string. A computer-readable recording medium having recorded thereon an audio / video synchronization multimedia file creation program. The method of claim 1, wherein the file generation module And a predetermined period is recorded in the tag data string. A computer-readable recording medium having recorded thereon an audio / video synchronization multimedia file creation program. The method of claim 1, wherein the file generation module In recording the index values sequentially, the first index value is recorded as a predetermined value, and the index values sequentially recorded after the first index value are recorded as an amount of change from the index value preceding it. A computer-readable recording medium that records a multimedia file creation program for audio / video synchronization. The method of claim 1, wherein the file generation module And recording the sample of the image data in the work field. A computer-readable recording medium having recorded thereon an audio / video synchronization multimedia file creation program. Audio data, image data samples, and image synchronization index sequence for synchronizing the image data samples at predetermined intervals as the audio data is reproduced, tag data strings for distinguishing the predetermined period and other data fields from the audio data. Storage means for receiving and storing the received multimedia data file; Decoding means for decoding and transmitting the audio data into Pulse Code Modulation (PCM) data according to the predetermined period; Audio output means for converting the PCM data from the decoding means into an analog signal and outputting the converted analog signal; As the decoding means is operated, image synchronization for sequentially reading the image tuning index sequence sequence according to the predetermined period and synchronizing and displaying the image data samples matching the read indexes with the PCM data output through the audio output means. Means for audio / video synchronization. The method of claim 10, In the image tuning index sequence, an initial index value is recorded as a predetermined value, and index values sequentially recorded after the first index value are an order sequence recorded as an amount of change from the index value prior to itself. And the image synchronizing means reads the image tuning index sequence by sequentially calculating the change amount to the first index value and subsequently reducing the next index value.

Images