KR100462826B1 - A portable multimedia playing device of synchronizing independently produced at least two multimedia data, a method for controlling the device, and a system of providing the multimedia data with the device - Google Patents

Abstract

PURPOSE: A portable multimedia data reproduction device for synchronization reproducing of more than two independent multimedia data and a reproduction control method therefor, and a system for supplying multimedia data are provided to manufacture combined data of audio data or video data for being coupled with independent data such as existing audio data, and to synchronize the combined data with the independent data, thereby enabling a user to easily combine contents. CONSTITUTION: A data storage(218) stores independent data and combined data including synchronization information necessary for synchronization with the independent data. A user input processor(210) receives a user selection related to a reproduction of the independent data and the combined data. A data synchronization controller(220) synchronizes the independent data with the combined data. A data reproducer(202) reproduces the synchronized independent data and the combined data. The data synchronization controller(220) comprises as follows. A data analyzer(208) detects a specific part corresponding to the second identifier among the independent data. A synchronous controller(206) controls such that reproduction times of the independent data and the combined data can be synchronized.

Description

A portable multimedia data playback device for synchronized playback of two or more independent multimedia data, and a method for controlling the playback, and a system for providing multimedia data. , AND A SYSTEM OF PROVIDING THE MULTIMEDIA DATA WITH THE DEVICE}

The present invention relates to a portable multimedia data reproducing apparatus for synchronizing and reproducing audio data and video data produced independently of each other, a control method thereof, and a system for providing multimedia data, and more particularly, existing audio data and the like. The present invention relates to an apparatus and a control method for producing combination data such as audio data or video data for reproduction in combination with independent data, and synchronizing with the independent data for reproduction.

In particular, in the field of music, audio data, music video, and the like are produced by various players, but such multimedia contents are generally produced by combining audio and video and distributed as a single content. In addition, due to the spread of over-the-air television or music cable TV, users have high expectations for the level of content, and on the other hand, the rapid spread of portable communication devices and portable electronic devices allows users to use such content anytime and anywhere. The user's desire to do so is also very high. A method of obtaining audio or video content may be first viewed on demand streaming. This is a kind of paid broadcasting. The advantage of this method is that you can enjoy the vast amount of multimedia data stored in the server at any time. This method has a problem in that a communication fee and information fee charged to a user for using a streaming service are high, rather than technical difficulties. In order to solve this problem, a user may enjoy having a data file stored in a storage device. In this case, the storage capacity of the currently available portable storage device is relatively small to store multimedia data such as music video, but the price is relatively expensive. Also, users can carry and use a DVD or video CD player. In the following, the device is not sufficiently miniaturized, and in reality, multimedia contents such as music videos are also not widely spread in a form that can be played on a portable terminal. There is a problem that the price of a portable electronic device equipped with a large-capacity small hard disk for users to carry and use such a large amount of multimedia data is too expensive. In addition, if an essential portable electronic device such as a mobile phone is utilized, it is possible to conveniently enjoy multimedia contents without carrying a separate portable multimedia data storage device. In addition, the opportunity for users to easily connect to the wireless network anytime, anywhere is expanding, and content providers can use the contents in a form that can be used immediately in a wireless environment, where payment for multimedia content is generally recognized among general users. It is desirable to have a wireless network connection service that can distribute and receive a fair price, and can receive a fair royalty fee from the copyright holder's point of view. As a result, the amount and quality of the contents can be expected to increase, but the usage fee of the wireless network is excessive, and it is economically expensive to download the entire multimedia contents such as music video, and due to the limitation of transmission speed, Since the data acquisition takes too much time, its utility is falling significantly.

Currently, a service for storing and displaying a dance motion such as an avatar or storing and playing music is provided separately in a portable multimedia device, but a service for integrating and playing two contents is not provided. In addition, the so-called MOD (Music On Demand) service, which provides music or music videos, is restricted from activation because it does not provide differentiated services from the offline use of music compared to the burden of excessive communication fees charged for the use of contents. It is true. In the case of a personal computer (PC), a service that integrates and plays back two data is provided, but so far, there is no method for precisely synchronizing the process of integrating the two data. Playback starts, so in some cases, the synchronization between the two data is not matched and the integrated playback cannot be performed properly. This is because piracy of data is so prevalent that most music files have the same intro, so even if you don't sync, you won't have a big problem. If you have already purchased the purchased audio or video file on the storage device, you can enjoy the multimedia file by purchasing, receiving, synchronizing and playing only the corresponding video file or audio file without purchasing or receiving the same audio file again. There will be. In particular, in order to solve the problem of time or cost when receiving multimedia data using a wireless communication network, the music audio file can be transferred to a portable device using a PC by receiving a music audio file that has been stored or purchased by wire. After moving and storing, the video data file is transmitted to the portable device through a wireless communication network, and then synchronized and synchronized with the two files to be integrated and reproduced as a multimedia file. This allows you to enjoy multimedia files without having to spend extra money on it. If you sell video data to display dance movements that are personalized according to commercially valuable music audio content, and then play them on a portable device, The first handheld device must have a simple user interface, so that when a user selects specific data to instruct it to play, or in addition, the integrated playback mode is instructed to play specific data, the synchronized audio And video data are preferably reproduced together. However, since the computing power of portable devices is very weak, unlike PC, it takes considerable time for the player to be loaded and ready for playback even if the playback instruction signal is input. The second legitimate data file is usually a file that has been copied for private use, such as ripping from the original file and a legally purchased CD. The difference may occur in the time of the intro part which is a noise state before it starts to be reproduced. Therefore, if the original file or the individual copy file by the right holder in accordance with the scope of the copyright system is used instead of the file generated by repeated piracy, the delay difference needs to be corrected. In order to increase portability, a multimedia device's player may be combined with a separate player, and a time delay between contents may occur during integrated playback according to a communication method or a control method between devices. There has been an increasing demand for more accurate synchronization methods than integrated playback of multimedia data.

The multimedia playback system according to the present invention has been devised to solve the above problems, and provides a combination data to a user having independent data in a cheaper, faster and more convenient way so that they can enjoy them in combination. do. In particular, it is an object of the present invention to provide a simple way for users to easily combine the content obtained by connecting to the network to enjoy in a new form.

The multimedia reproducing apparatus according to the present invention has an object of allowing the user to combine the independent data and the combination data in various combinations so that the user can feel more emotion and interest. In particular, it aims to improve user's haptic utility by providing combination data so that existing independent data can be enjoyed in an improved form.

According to the present invention, a network server storing video data or audio data and providing the same to a user and a data transmission network transmitting data stored in the server to a multimedia device carried by the user and receiving data are synchronized with video data or audio data stored in advance. In reproducing the data, the network server stores data having a relatively small amount of data and transmits the data to the portable multimedia data reproducing apparatus of the user according to a user's selection, and the portable multimedia data reproducing apparatus has a relatively large amount of data in advance. The purpose of this is to store and play the data in combination with the received data.

The user stores and carries information on the composition of characters including various 3D avatars and audio data such as MP3 files in a portable multimedia data reproducing apparatus in advance and choreographs a 3D avatar suitable for each audio data in a service server of a wireless network. Store control signals to control the operation. According to the present invention, a large amount of audio data and structure data of a 3D avatar are transmitted from a personal computer to a portable multimedia data reproducing apparatus and stored, and only certain choreography control operations are transmitted anytime and anywhere through a wireless network. The purpose of this is to enable users to enjoy various contents such as music videos at low rates by saving network charges and time required for transmission.

If mobile internet or wireless LAN is widely used to improve the transmission rate and transmission speed of wireless network, it is burdensome to receive the entire music video at any time. All audio data having a relatively small amount compared to the signal or video data may be transmitted through the wireless network. Even in this case, it is not necessary to duplicate the configuration information of the same 3D avatar every time one audio data is transmitted. Also, if the storage capacity of the portable multimedia data reproducing apparatus is large enough, various types of choreograph videos corresponding to specific audio may be stored in advance, and audio data may be transmitted and enjoyed through the wireless network.

In addition, the present invention may sell video data and audio data in a server located in a wired network and store the same in a portable multimedia data reproducing apparatus. In this case, as described above, the present invention provides a sophisticated multimedia data reproducing apparatus. An object of the present invention is to ensure a perfect playback quality with a synchronization device and method. In addition, an object of the present invention is to increase the value of the service by integrating in any part when utilizing a randomly selected part of the stored data as a multimedia ringtone and the like. It is an object of the present invention to provide a method and apparatus that can be applied even when the combined provision through data synchronization uses valuable audio, multiple graphic data, and the like.

1 is a block diagram showing a schematic configuration of a portable multimedia playback apparatus according to the present invention.

FIG. 2 is a block diagram showing a software configuration of the portable multimedia playback apparatus shown in FIG.

3 is a flowchart illustrating an operation method for synchronizing basic data and additional data according to an embodiment of the present invention.

4 is a block diagram showing a schematic configuration of a portable multimedia playback system according to an embodiment of the present invention.

<Explanation of symbols for main parts of drawing>

100: portable multimedia playback device

102: microprocessor 104: memory

106: keypad 108: LCD panel

110: speaker 112: data transceiver

114: mobile phone antenna 116: external storage device

118: external output device

200: Portable Multimedia Playback Device Software Configuration

202: data player 204: playback method control unit

206: synchronization control unit 208: data analysis unit

210: user input handler 212: video codec

214: Audio codec 216: Graphics data processor

218: data storage 220: data synchronization controller

400: portable multimedia playback system

402 content server 404 mobile ISP network

406: cell phone

408: PDA (Portable Multimedia Player)

410: mobile phone (portable multimedia playback device)

A portable multimedia data reproducing apparatus according to a preferred embodiment of the present invention, which is not intended to be used in combination with other data, is independently generated and synchronized with the independent data, which is generated for playing in combination with the independent data. A data storage unit for storing the combination data including the synchronization information required for the; A user input processor for inputting a user's selection related to reproduction of the independent data and the combined data; A data synchronization controller for synchronizing the combination data with the independent data selected by the user for playback; And a data player for reproducing the synchronized independent data and combination data. In the present invention, the multiple graphic data is data provided to provide a video to a user, and there is no correlation between frames. Means data consisting of graphic data representing a plurality of frames constituting a continuous operation or movement. The data player includes a microprocessor unit for managing the operation and data of the entire apparatus, a decoder for recovering a digital data signal to an analog control signal, and a display and audio output unit for outputting content. The user input processor processor may process an optional playback input that receives a selection of whether to play integrated playback in the form of multimedia data or to play only independent or combined data of audio or video. The user input processor may receive and process a selection of the user's basic data and the corresponding combination data, and receive a selection of either the independent data or the combination data of the user and store the multimedia content data stored in the data storage unit. It also performs the function of distinguishing combinatorial data or independent data that can be interworked with each other.

In addition, independent data according to a preferred embodiment of the present invention-the independent data is generated independently, not intended to be used in combination with other data, and the combined data is generated for reproduction in combination with the independent data and the independent A method of controlling a portable multimedia data reproducing apparatus for synchronizing and playing back one or more combination data, comprising: a reception point for receiving a playback start control signal in a data stream of the combination data; Setting up; Storing and retaining pattern information of the independent data corresponding to the reception point of the combination data in a predetermined recording means; Analyzing the data stream of the independent data and comparing it with the pattern information stored in the recording means; Synchronizing a reproduction time of the independent data with a reproduction time of the combination data when the same or similar pattern is found in the data stream of the independent data as a result of the comparison; And reproducing the independent data and the combined data. Further, a control of a portable multimedia data reproducing apparatus for synchronizing and reproducing independent data and one or more combination data according to an embodiment of the present invention. The method, wherein the independent data is generated independently, not intended to be used in combination with other data, and the combined data is generated for playing in combination with the independent data and includes synchronization information necessary for synchronization with the independent data; Maintaining a data storage for recording association identifier information for additional data reproduced with predetermined independent data, wherein the association identifier information is generated by a user selection input for independent data and additional data reproduced together; Receiving a selection input for independent data reproduced from a user; Retrieving association identifier information of the combination data reproduced together with the independent data by referring to the data storage unit; And controlling combination data corresponding to the retrieved association identifier information to be reproduced together with the independent data. In addition, by synchronizing the independent data with one or more combination data according to an embodiment of the present invention, A control method of a portable multimedia data reproducing apparatus for reproducing, wherein the independent data is generated independently, not intended to be used in combination with other data, and the combination data is generated for reproduction in combination with the independent data and associated with the independent data. Including synchronization information necessary for synchronization of the data storage unit, wherein the data storage unit records association identifier information for the additional data reproduced with the predetermined independent data; and the association identifier information includes the independent data and the additional data reproduced together. For Generated by the select input; Receiving a selection input for independent data reproduced from a user; Retrieving association identifier information of the combination data reproduced together with the independent data by referring to the data storage unit; And controlling combination data corresponding to the retrieved association identifier information to be reproduced together with the independent data. The independent data capable of interworking with a plurality of multimedia content data stored in a data storage unit may be controlled. Steps to separate the combination data may be added. When the user designates either the independent data or the combination data, the portable multimedia data reproducing apparatus automatically retrieves and loads the corresponding data that can be linked thereto by applying a predetermined keyword search algorithm or other algorithm. Similar to the method applied in, it also loads the independent data and the combination data according to the user's selection input. For example, when the user has the independent data which is the audio data of "Letter bottle.mp3" and the combination data which is the moving picture clip of "Letter bottle.asf", the portable multimedia data reproducing apparatus according to the present invention is inputted from the user. It is operable to receive a selection for " Letter bottle.mp3 " and to collectively reproduce the combination data " light bottle.asf " through the keyword search algorithm. In addition, the portable multimedia data player according to an embodiment of the present invention. Considering the ease of use of the mobile device, the user initially makes a selection input, stores the association of the selected data, and when the user selects any one of them after the second round, Instead of waiting for a selection, run the search algorithm to automatically load the corresponding data. Can be operated. In this case, when the user selects "light bottle.asf" as combination data to be output in correspondence with the above-described independent data of "light bottle.mp3", the identifier information of the combination data corresponding to the independent data is indexed to the recording means. The combination data may be implemented to be integrated and reproduced together by storing and analyzing the identifier information indexed when the independent data is reproduced. In the step of reading a reproduction reference point for synchronizing with the independent data from the combined data, the combined data may be used. In general, one or more playback reference points are pre-designated to have relevant information added to the content source data. In this case, which playback reference point is specifically selected is independent of the pattern information of the independent data corresponding to the playback reference points in the event comparison step. The first pattern information found in the data You may select the playback reference point on the combined data equivalent basis. Alternatively, even when the reproduction reference points are given priority to each other and a reproduction command is received, a data search operation may be performed to determine the reproduction reference point prior to the output of the audio or video data. In this case, when the pattern information first found among the pattern information of the independent data has a low priority in a predetermined range, a predetermined judgment algorithm is applied according to a search time or priority of the reproduction reference point, and the data of the predetermined reproduction reference point and the corresponding part is applied. May be implemented to continue the search until it finds the location of the stored memory. In the data reproducing step, the reproducing output of the independent data may be started and the reproducing output of the combination data may be started during the continuous reproducing, or the reproducing reference point is judged until the reproducing reference point is judged before the independent data is output to the speaker or the like. It is also possible to operate to read and analyze data from the memory only in the range necessary for the data, and to simultaneously reproduce the independent data and the combined data after the reproduction reference points are finally determined. In another embodiment of the present invention, the combined data may have an intro portion. If the amplitude of the independent data is more than a certain level, the noise or intro silence part corresponding to the character's first motion has elapsed. The melody or content portion of the data is regarded as being started, and control is performed to start the combination data. The portable multimedia data reproducing apparatus used in the present specification includes a PCS and a cellular mobile phone, a smart phone, a personal digital assistant (PDA), It is a concept that collectively refers to a portable electronic playback device having a predetermined computing power such as an MP3 player and having a function of playing multimedia data and delivering it to a user.

In addition, a character widely used in the present specification is a character personified by a person or an animal, as well as an object such as a rug, a dancing matchstick or a dice, or an object such as a wave or a cloud if it can induce a user's emotion. The movement of the alphabet is a concept encompassing the motion or movement of the letters.

Hereinafter, a portable multimedia playback apparatus and a control method thereof according to an exemplary embodiment of the present invention will be described with reference to the accompanying drawings.

1 is a block diagram illustrating a schematic configuration of an embodiment of a portable multimedia data reproducing apparatus. FIG. 1 illustrates a configuration of a portable multimedia data reproducing apparatus. For convenience of description, the portable multimedia data reproducing apparatus includes a keypad input unit similar to a mobile phone, and includes an LCD panel, an audio speaker, and the like. Explain.

Referring to FIG. 1, the portable multimedia playback apparatus 100 includes a microprocessor 102, a keypad 106, an LCD panel 108, a memory 104, a speaker 110, and a data transceiver 112. The keypad 106 is used as a means for receiving a user input such as selecting or playing content. It is also used to receive user input, such as selective playback of multiple additional data.

The LCD panel 108 is used to convey one or more video data or multiple graphic data or a combination of several visible data to the user.

The speaker 110 is used to deliver audio data or data that produces various sound effects to the user.

The microprocessor 102 is a control unit for controlling synchronization and reproduction of multimedia contents. The microprocessor 102 controls synchronization between independent data and one or more combination data and controls hardware such as the LCD panel 108 and the speaker 110 to control predetermined multimedia. The memory 104 is for storing a predetermined control program for performing a series of control methods performed by the microprocessor 102. The memory 104 stores synchronization information and an identifier between the aforementioned independent data and the combination data. Data for synchronization including information and data including content information may also be implemented to be stored together.

The data transceiver 112 may include both a connection with a content server through an external network or a communication interface with an external device. According to the type of the data transceiver may be classified as follows. According to an embodiment of the present invention, when the portable multimedia data reproducing apparatus is a mobile phone, it may be connected to the wireless carrier's network through the wireless Internet of the mobile phone (114). Through this connection, the playback apparatus can access a dedicated content server in the operator's network and receive independent data and combination data. In another embodiment of the data transmission and reception unit according to the present invention, an external storage device 116, for example, may be connected to a PC using an interface such as serial or USB. The portable multimedia data reproducing apparatus may receive independent data or combination data through the external storage device 116. In another embodiment of the data transmission and reception unit according to the present invention, when the portable multimedia data reproducing apparatus does not include an LCD panel or a speaker module and cannot reproduce multimedia contents by itself, the multimedia contents reproduced by connecting to another external output device are reproduced. The information can be output to another external output device 118. That is, content data and synchronization control may be performed by the portable multimedia data reproducing apparatus 100 and only output data may be sent to the external device 118 for output.

In the portable multimedia data reproducing apparatus according to the present invention, the player may be implemented in different hardware. For example, if you store and play back independent data on a small device with a microprocessor and memory connected to the phone or via a serial or USB interface, and you store and play back combination data on a mobile phone, the standalone data player can be implemented as separate removable portable hardware. Can be. In this case, the data synchronization controller may be implemented in a mobile phone or may be implemented in separate hardware. FIG. 2 is a block diagram illustrating a software configuration of a control program for controlling the portable multimedia data reproducing apparatus shown in FIG. 1. )to be. 2 is a block diagram of a control program executed by a data synchronization controller, and may be stored in a memory of a portable multimedia data reproducing apparatus or an internal memory of a microprocessor. Referring to FIG. The control program according to the present invention includes a user input processor 210, a video codec 212 and an audio codec 214, a graphic data processor 216, a data synchronization controller 220, and a data player 202. Can be configured.

The user input processor 210 is a module that delivers an integrated user command, such as content selection and playback, to the data synchronization controller 220.

The video codec 212 is a module that converts video data into information that can be directly displayed on display means such as the LCD panel 108.

The audio codec 214 is a module that converts audio data into information that can be reproduced through the speaker 110 or the earphone jack.

The graphic data processor 216 is a module for converting 3D avatars, animations, text subtitles, etc. into information that can be displayed on display means such as the LCD panel 108. The data synchronization controller 220 is independent data and combination data. This module controls the synchronization by using the synchronization information. The data synchronization controller 220 may be implemented in a modular format that can be integrated and controlled by the video codec 212, the audio codec 214, and the data player 202 connected to the graphic data processor 216.

The data player 202 receives the content information converted through the codecs 212 and 214 and the graphic data processor 216 under the control of the data synchronization controller 220, and displays the speaker 110 and the display means such as the LCD panel 108. A module for delivering to a user through the present invention. In one embodiment of the present invention, new audio data is played by combining two audio and video data that are independently created and independently. For example, if there is music audio data "A" and music audio data "B" having rhythms such as "A" and "I", the character's video data performing a constant choreograph movement in three quarters of a character It is not necessarily intended to be used in combination with specific music audio data, or may not be intended to be used in combination with music audio data from the beginning. However, if these audio data and video data are combined and synchronized with each other, they can be useful multimedia data. For example, you can repeatedly show the choreography of a character who works with the same beat to the output of three quarters of music audio data, or you can store a lot of choreographed movements of the same beat and extract it according to a certain rule or randomly. It is also possible to display them with changes. In the latter case, the continuity will be reduced, but it can show a variety of choreography movements. A specific method of synchronizing data that exists independently in this way will be described later. In another embodiment of the present invention, the independent data may be generated from existing MP3 files, which are not intended to be reproduced in conjunction with other data. It may be the same music audio data, and the combination data may be video or multi-graphic data produced for use in combination with the independent data, and may include information necessary for synchronization. Such combination data may be generated using individual specific music data as independent data, but may also be produced to combine music audio data of a group having the same characteristics. In other words, if the music is audio data of a predetermined time or a genre, a dance motion or the like choreographed can be used as the combination data, regardless of who plays any song. As described above, the portable multimedia according to the present invention can be used. The data reproducing apparatus may perform a function of distinguishing one or more independent data and one or more combination data recorded in a predetermined data storage device of internal or external type. That is, a search process for searching for specific combination data to be integrated and reproduced together with the reproduction of specific independent data, an indexing process for indexing the association identifier information of the combination data to be reproduced together with the specific independent data, and recording the result in a predetermined memory means As described above, when the user designates one of the independent data or the combination data, the portable multimedia data reproducing apparatus may display corresponding data that can be linked thereto based on the file name or the MP3 file format. In the case of sound source data, a method of automatically searching and loading by applying a predetermined keyword search algorithm based on information such as the name of a song and a singer name included in an ID3 tag, and (2) allowing a user to make a selection input and identify the selected data. Indexed identifier information for storing the information in memory means When the user selects any one of them after the second round, a method of automatically loading corresponding data without waiting for selection of the user's corresponding data may be used. FIG. 3 illustrates an embodiment of the present invention. The flowchart illustrates a method of operating a data synchronization controller or the like for synchronizing independent data with combination data. The independent data and combination data may be data independently created. For example, in the case of using choreographic motions of 3D characters as combination data, the combination data includes information about polygons, texture information, ray-casting ray-tracing information, and shading information for constructing a 3D avatar. , And control information indicating the movement of the polygon groups representing the parts of the body obtained by a technique such as motion capturing to control individual choreography movements. For example, data on the position of each joint and the change value thereof correspond. In addition, the synchronization information corresponding to each synchronization method described below is included. The user input processor 210 illustrated in FIG. 2 receives a user's command and transmits the command to the data synchronization controller 220 (302). For example, if a user selects and plays a video of a character dancing according to MP3 music, a selection and playback command is transmitted to the data synchronization controller 220, and the data synchronization controller 220 loads the selected content into a memory to display the content. Analyze what multimedia data is composed (304). It analyzes independent data, which may be music audio data, and finds a part that matches or is as close as possible to the identifier included in the synchronizing information of the combination data. You can select data decoded through the codec, or select data for various analysis techniques, such as analog data or electrical signals that are actually being played or are being played back. In some cases, the data analysis step may be moved to a later work step. The data synchronization controller 220 designates specific MP3 data as independent data and specific character video data as combination data, and loads the MP3 data. Control to be processed through the codec (308). In addition, the character video information as the combination data is loaded and controlled to be processed through the graphic data processor (310). At this time, independent data is continuously played from the beginning to the end without any special user's stop command. The combined data is selected from one of several synchronization methods and processed and reproduced in synchronization with independent data (312).

In another embodiment of the present invention, a step of checking whether the MP3 audio codec and the video codec or the graphic data processor are initialized is applied. Even if the audio player and the video player are instructed to start playback at the same time, when the time required for loading the audio player and the time required for loading the video player are different, the time when the audio data is output through the speaker and the time when the video data is displayed are actually different. Differences can occur, but this can happen frequently with portable multimedia data playback devices. If the data is controlled to be reproduced without confirming the ready state of the corresponding data processor, the corresponding data may be delayed and synchronization may fail. Therefore, the data synchronization controller 220 controls the audio codec, the video codec, and the graphic processor to be initialized to check whether the processor of the corresponding data is ready for reproduction and to start data processing.

The user input processor 210 according to another embodiment of the present invention initially allows a user to input a selection in consideration of the ease of use of the portable multimedia data reproducing apparatus, and associates the selected data with the data storage unit ( If the user selects any one of them after the second round, the data may be implemented to automatically load the corresponding data by operating a search algorithm without waiting for the user to select the corresponding data. According to another embodiment of the present invention, since the independent data is basically continuously reproduced with respect to time, the reproducing apparatus may maintain elapsed time information, that is, time stamp information, of the portion currently being reproduced based on the start point. On the other hand, the combination data designates specific points in the combination data stream as reproduction reference points and manages data of a data stream section from the reproduction reference point to the next reproduction reference point, respectively. The combination data of a specific section includes time stamp information on how long the independent data corresponding to each reproduction reference point, which is the starting point of the corresponding section, is output on the stream of independent data based on the starting point of the independent data, as synchronization information. Include. In addition, the combination data of the corresponding section includes time stamp information on the interval reproduction time indicating how long the reproduction data is reproduced from the reproduction reference point. The data synchronization controller 220 may synchronize by comparing the time stamp information of the combination data with the time information of the independent data. That is, the data synchronization controller 220 reads the reproduction reference point start time that occurs first after listing the reproduction reference point start times of the time stamps. If the playback time of the independent data is checked periodically and the read time point coincides with the read start point start time, the corresponding combination data is played back from the play reference point (320). Thereafter, the data synchronization controller 220 reads the next playback reference point start time again and repeatedly checks whether it matches the time information of the independent data again. In addition, the combination data, which has already started playing, memorizes each playing time in the data synchronization controller 220, and stops playing the corresponding combination data when the playing time is over. Such playback reference point start time confirmation and playback time confirmation can be independently confirmed regardless of their order. In this embodiment, the song "Love" of the singer "A" is reproduced as independent data and the combination data is used as the combination data. The choreography gesture of the 3D avatar may be played. Choreographed "A" choreography in Section 1, "B" in Section 2, "C" in Chorus, and "D" just before the end of the song. When it is assumed that the choreography is to be changed, the time stamp method described above can be used. The combination data includes choreographic motion data of "A", "B", "C", and "D", and corresponding playback reference time information on when each choreography motion should be played on the time axis of the data stream of independent data. Is in. Accordingly, the data synchronization controller 220 can reproduce the 3D avatar choreography motion that is changed through the time information of the independent data and the combination data in synchronization with the independent data. Can be played continuously and repeatedly for. That is, a synchronization method in which the combination data is automatically played back from the beginning after the playback time starts from one playback reference point start time and the combination data ends, may be used. While playing, it can be used as a synchronizing method for combination data in which drums or other musical instruments are repeatedly played in accordance with the beat of MP3 music data, such as a three-quarter beat or a quarter-time beat as combination data. In addition, it can be used for a method in which a background image is repeatedly changed while an animation is reproduced as independent data. To this end, the first method of recognizing the reproduction reference point may be a synchronization method based on the time stamp described above, or a synchronization method using audio data pattern analysis described later may be used. In the present embodiment, if one section of the song "Love" is played for two minutes, the choreography operation "A" which is to be shown while the first section is played is a 10-second choreography operation. The choreography should not be interrupted by playing a total of 12 times repeatedly. In this case, if section 1 starts for the first time, playback starts with the synchronization method based on the time stamp, and when the choreography operation "A" ends after 10 seconds, the choreography operation "A" is immediately performed at the end of using the synchronization method by the repeated playback. By repeating this, it is possible to synchronize repeatedly. According to another embodiment of the present invention, a special pattern can be detected in independent data and used to synchronize the special pattern and the combination data. For example, after audio data is processed into PCM data in a codec, a method of extracting a specific beat pattern from the data may be used. In the case of general music data, regular and repetitive instrument notes are included. For example, if you can derive the patterns of the drums and snare drums from the drums, you can derive information about how many beats the music is playing and when it starts. As a method of deriving the sound, a method using a software or hardware dedicated signal processor may be used. In another example, a special pattern is found in the video data. When a specific screen change occurs in consideration of the pattern and the color of the data within a frame, a method of playing the corresponding sound effect combination data may be used. . In this data pattern analysis, each appropriate signal processor or algorithm is used, and the information on which method to analyze the pattern is already included in the data synchronization controller or the combined data. To find out the time signature of the independent data, the rhythm of the elements of the music is detected and used. After converting the song to the PCM data format, it is possible to detect the notes of musical instruments that play rhythms such as drums through a low pass filter, etc. After detecting the pattern, a combination of negative strengths and weaknesses detected through several different band pass filters is detected to detect the obsessive and weak beats. The combination of the rhythm beat pattern, the strong beat, and the weak beat can detect whether the song is a two-beat, three-beat, or four-beat sequence. Then, if it is possible to detect whether it is two quarters or sixths of eight beats through detection of a triplet or the like within one beat, the beat can be accurately determined. Assume that it is a song called "love" of the singer and assume that the combination data is a choreographic motion of the 3D avatar. At this time, it is assumed that the choreography operation is a choreographed movement according to the beat, such as a specific beat, that is, a four-fourth choreography operation or a six-eighth-time choreography operation. In addition, this choreography is not only used in the song "B" by the singer "Gap" but has the advantage that it can be used regardless of the type of song if only the beat is matched in any song. According to the present embodiment, it is possible to synchronize the choreographic motion of the 3D avatar which is the combination data when detecting only a few time line even if the exact time of the independent data is not found. In other words, even if you cannot classify two quarters and six quarters, you can basically detect that the song is a two-beat system, so if you play the two-quarters choreography only at the above mentioned speed, You will be able to enjoy choreography of 3D avatars that match the rhythm.An alternative way of analyzing patterns is to analyze the melody of the music and use it as a synchronization method. In this method, a method of converting PCM data converted through a codec into a form of spectrum represented by an axis as a frequency may be used. One way is to use the Fast Fourier Transform (FFT) method, and each coefficient component from this method can be used to calculate the equal temperament of one octave to detect pitch. The rhythm pattern data is generated using this pitch. By comparing the detected rhythm pattern and the synchronization information of the combination data, the choreographic motion of the modified form can be inserted in the basic choreography motion according to the melody to produce more various effects. This method is more effective when used in the same way as playing the choreographed chords using the rhythm described above.

According to another embodiment of the present invention, the data synchronization controller 220 compares the data stream of the independent data with the synchronization stream of the combination data, and when a match is found, at the time when the match is reproduced from the independent data. The combination data is also controlled to be reproduced together. This method largely compares the independent data with the data stream before being converted by the codec 212 or 214 or the graphics data processor 216, and the data after being converted by the codec 212 or 214 or the graphics data processor 216. This can be distinguished by comparing streams. In the former case, the source data of the independent data is compared with the synchronization data of the combination data. In this case, it can be used when the data of the independent data is already known at the time of producing the combination data. In the latter case, we compare the independent data converted by compression in the codec or processor. This is similar to the method of analyzing and synchronizing the data pattern in the above embodiment, but has the advantage of being simple in implementation because it is a method of deriving a matching stream rather than finding a specific pattern from independent data.

According to another embodiment of the present invention, even when the data rate of the combined data and the independent data does not match, the reproduction method controller 204 may match and reproduce the data. In the present embodiment, for example, a song "love" is assumed to be four quarters of time and four seconds of a quarter time are played for 4 seconds. If the choreographed choreography of the four-quarters choreographed movement of the combined data is 3.2 seconds, the choreographed movement of the choreographed choreography will be quickly terminated by 0.8 sec. In this case, the time can be adjusted relatively. The song "Love" used as independent data is played back as it is, and the choreographed operation of the combination data is extended. In other words, if the timeline is adjusted so that the 0.8 seconds of the choreography motion is actually played for 1 second, the 3.2 seconds of the choreography motion are played for 4 seconds, so that the song and song speed match. In this case, as a detailed method of adjusting the time lime of the combined data, if the combined data is audio data, a method of adjusting the bit rate according to the ratio may be used. In other words, if the audio data of the original combination data is 44,100 Hz, if the data is reproduced at 35,280 Hz (44,100 x 3.2 / 4.0), 3.2 seconds of data can be reproduced for 4 seconds. If the choreography motion of the 3D avatar as in this example, the frame rate can be adjusted. For example, suppose that the choreographed motion of the original combination data has 15 frames of data per second. If the choreographed motion is played at 12 frames per second (15x3.2 / 4.0), the choreographed motion of 3.2 seconds can be played for 4 seconds. .

It will be apparent that any other synchronization method not described above may be used as long as it can be used to derive and reproduce the reproduction reference point of the combination data in the independent data in a specific manner. As described above, the reproduction reference point of the combination data corresponding to each other and the reproduction reference point of the independent data are determined and reproduced together. In this case, the data synchronization controller 220 directly controls the respective codecs 212 and 214 or the graphic data processor 216 or outputs data output from the codecs 212 and 214 or the graphic data processor 216 to the data player 202. A method of collecting and reproducing the same may be used. According to another embodiment of the present invention, the combination data may be manufactured to be combined with specific audio data. For example, a dance movement choreographed to audio data entitled "A" of the singer "A" can be used as the combination data. Alternatively, the audio data may be sound data added to maximize the effect of the audio data or a combination of the two.

In another embodiment of the present invention, there may be various types of combination data that can be combined with the independent data, and the user may select which data to be added to the independent data for viewing. To this end, a user input processor such as a keypad may be additionally provided.

The portable multimedia data reproducing apparatus according to another embodiment of the present invention may further include a receiver for receiving all or part of independent data or all or part of a combination data from a communication network. In particular, anytime, anywhere (ubiquitous) may further include a receiver for interfacing with a wired and wireless communication network to support such a reception function, and in particular, a large amount of information such as information on the composition of characters and music audio data is provided separately through a PC or a wired communication network Received and stored data, such as a control signal for choreographic movement of the character and synchronization information for matching the movement of the character with the music audio data can be received and used from time to time through the wireless network. In this way, when the information received through the wireless network is combined with the information on the composition of the character stored in advance, the choreographed motion of the character may be updated from time to time or stored in various types to enjoy more diverse multimedia data. Of course, it takes a lot of time and expensive fees, but music audio data or information on the composition of the new character may be downloaded separately or together with the above data through the wireless network. To support this, service providers must maintain a data server linked to a wireless network.

FIG. 4 is a block diagram illustrating a schematic configuration of a multimedia playback system according to an embodiment of the present invention. In this example, a playback system is installed in a wireless provider network. The portable multimedia data reproducing apparatus will be described in detail using an example of a mobile communication terminal. In FIG. 4, the mobile multimedia data reproducing apparatus 406, 408, and 410 are connected to the cellular, PCS, and mobile Internet through the data transmitting and receiving unit 112 of FIG. 1. Method, and can be connected to the mobile Internet 404 provided by a mobile communication provider using a third generation mobile communication method such as W-CDMA. The playback device may access the content server 402 installed in the wireless provider network through the mobile Internet 404. The portable multimedia data reproducing apparatus 406, 408, or 410 may download respective independent data and combination data from the content server 402. Types of content data that can be provided by the content server 402 include video, MP3 music, WMA music, 3D avatars, 2D animations, and the like, and the content data may be used as independent data or combination data. Furthermore, additional instrument performance data or chorus sound source data corresponding to music independent data, additional 3D avatar motion control data corresponding to 3D avatar independent data, 3D avatar camera motion control data, 2D animation motion control data, as well as video and audio data Not only the actual data of the back but also the data describing the motion and the like may be used as the combination data. In this case, the combination data stored in the content server 404 may include synchronization information by combining with the previously intended independent data, and the data stream used by the data synchronization controller 220 of the portable multimedia data reproducing apparatus. Information about the reproduction reference point may be further provided. On the other hand, the independent data stored in the data storage unit 218 of the portable multimedia data reproducing apparatus may not have synchronization information because it is not specifically scheduled to be played in combination with other data. In order to overcome the shortcomings of the video, MP3 files, 2D or 3D avatars are transmitted from an external storage device via a PC, and only 2D or 3D avatar motion control data is downloaded from the content server 404 via the mobile Internet and synchronized with it. Can be played.

According to another embodiment of the present invention, the multimedia data distribution device is stored in a portable multimedia data reproducing apparatus in a network server connected to a mobile communication network such as a cellular method, a PCS method, a mobile Internet method, and a third generation mobile communication method such as W-CDMA. Providing a series of data including control signals and synchronization information for controlling the movement or movement of a person, animal, or thing, and converting them into video data or continuous frame information through a network such as a wireless network at the user's request A multimedia combination data distribution device may be provided. In some cases, the apparatus for distributing multimedia combination data may further include a user manager configured to provide billing data or perform a billing operation while managing a user's usage history.

As described above, according to the present invention, the user can add and reproduce the combination data to the independent data in various combinations to independently compose and enjoy more interesting and impressive contents. According to the present invention, the independent data or combination Users who have only data can enjoy combining them by providing corresponding independent or combination data in a cheaper, faster and more convenient way.

In particular, according to the present invention, a network server that stores video data or audio data and provides the user with the necessary data can be received and used anytime, anywhere, thereby expanding data usage opportunities and improving user convenience. have.

Particularly, according to the present invention, when the wireless network is expensive, part of the entire multimedia data is secured by using the wireless network, and some of the existing data is recycled to reduce the burden and time of the user's network fee. Particularly, according to the present invention, some of the data required by the already widely used portable receiving device can be transmitted and synchronized and enjoyed through the wireless network interface, thereby allowing the user to receive more rich data from time to time. Not only can you enjoy multimedia content, you can avoid the burden and hassle of purchasing or carrying a separate multimedia player.

In addition, according to the present invention, it is possible to improve the quality of the service provided to the user because it provides the accuracy and flexibility to synchronize the necessary parts for data integration.

Claims (19)

In the portable multimedia data playback device, A data storage unit for storing independent data, which is not intended to be used in combination with other data, and combination data including synchronization information generated for reproduction in combination with the independent data and necessary for synchronization with the independent data; ; A user input processor configured to receive a user selection related to reproduction of the independent data and the combined data; A data synchronization controller for synchronizing the combination data with the independent data selected by the user for playback; And A data player for reproducing the synchronized independent data and combination data; Including; The synchronization information includes a first identifier of one or more specific operating portions of the combination data and a second identifier for identifying one or more specific portions of independent data corresponding thereto; The data synchronization controller, A data analyzer configured to detect the specific part of the independent data corresponding to the second identifier; And A synchronization controller configured to control the reproduction time of the independent data and the combined data with reference to the specific portion and the second identifier of the detected independent data; Portable multimedia data playback device comprising a. The method of claim 1, The independent data is audio data, The combination data comprises one or more video data for which a predetermined character reproduces the same movements for a predetermined time that matches the rhythm of the independent data, And the data player is operable to repeatedly play one or more of the video data. The method of claim 1, And said combination data is data for controlling choreography operation of said character created to be used in accordance with graphic data or predetermined data relating to the configuration of a predetermined character. The method of claim 1 The data analyzer detects a signal having an amplitude greater than or equal to a predetermined value by analyzing the intro of the independent data, And the synchronization controller controls the data player to reproduce the combined data in response to the detection of the signal. The method of claim 1, The second identifier is time stamp information, The data analyzer compares and analyzes the time stamp information included in the synchronization information with time stamp information of the independent data stored in the data storage unit. The synchronization controller determines the reproduction time and reproduction time of the combination data such that the independent data and the combination data are synchronized according to the analysis result of the data analysis unit, and adjusts the data player according to the determined reproduction time and the reproduction time. Portable multimedia data playback device, characterized in that it is operable to control. The method of claim 1, The second identifier comprises one or more of melody, rhythm, or amplitude pattern information of one or more specific portions of the independent data, The data analyzer compares the at least one of melody, rhythm, or amplitude pattern information of the independent data stored in the data storage with the second identifier and detects the same or similar part. The synchronization controller determines a reproduction time and reproduction time of the combination data such that the independent data and the combination data are synchronized according to a detection result of the data analysis unit, and adjusts the data reproducer according to the determined reproduction time and the reproduction time. Portable multimedia data playback device, characterized in that it is operable to control. The method of claim 1, The second identifier is data stream information of one or more specific portions of the independent data, The data analyzer compares the second identifier with a data stream of the independent data stored in the data storage, and detects a specific portion of the independent independent data that is matched; Determine a reproduction time and reproduction time of the combination data to synchronize the independent data and the combination data according to the detection result of the data analysis unit, and control the data reproducer according to the determined reproduction time and the reproduction time Portable multimedia data playback device, characterized in that. The method according to any one of claims 4 to 7, And the data synchronizing controller further comprises a reproducing method control unit for performing data degradation or frame skipping on the reproduction speed of the combination data or the portion of the combination data actually reproduced. The method according to any one of claims 4 to 7, The data storage unit may further store related data including information on the independent data and the combination data selected at the first reproduction input, input from the user input processor, When the user input processor receives a selection input for any one of the independent data and the combination data included in the associated data from the user, the user input processor refers to the related independent data stored in the data storage unit to the selected independent data. And a control signal for reproducing the combination data or the independent data for the selected combination data to the data synchronization controller. The method according to any one of claims 4 to 7, The data storage unit stores one or more combination data related to one independent data, The user input processor receives a selection input for the specific combination data to be reproduced from the one or more combination data from the user, And the data player reproduces the selected combination data and the independent data in synchronization. The method of claim 1, A receiver for receiving at least a portion of graphic data on the configuration of a predetermined character included in the combination data or data controlling the choreographic operation of the character written to match and use the independent data through a wireless communication network. Portable multimedia data playback device further comprises. The method according to any one of claims 4 to 7, A receiver for receiving at least a portion of the graphic data on the composition of the character included in the combination data or data controlling the choreographic operation of the character written to match and use the independent data through a wireless communication network. Portable multimedia data playback device further comprises. The method according to any one of claims 4 to 7, The data storage unit further stores time information until the video codec, audio codec, or graphics processor is loaded and ready to play, The user input processor receives a selection input from the user on whether to play the independent data and the combination data separately or in combination; When the integrated reproduction selection input is received from the user, the data synchronization controller is configured to refer to the time information stored in the data storage unit to play the video codec, the audio codec, or the video codec required to play the independent data and the combined data. And reproducing a playback start time of the first loaded player according to the maximum time information of one or more players of the graphic processor. In the multimedia data providing system, Database means for storing combined data corresponding to independent data of audio data, the combined data being generated for reproduction in combination with the independent data and including synchronization information necessary for synchronization with the independent data, wherein the combined data is determined by a predetermined character. One or more video data for playing the same operations consistent with the rhythm of the independent data for a predetermined time; And Network communication unit for transmitting at least a portion of the combination data stored in the database means to the portable multimedia data playback device of the user Including, The portable multimedia data playback device, A data storage unit for storing the audio data and the video data transmitted from the system; A user input processor configured to receive a user selection related to playing the audio data and the video data; A data synchronization controller for synchronizing the audio data with the video data selected by the user for playback; A data player for reproducing the synchronized audio data and video data, the data player operative to replay one or more of the video data repeatedly; And A user management unit for recording user information for identifying the user, managing a providing history of the combination data according to the user's request, and controlling a predetermined charging system according to the providing history; Multimedia data providing system comprising a. delete A control method of a portable multimedia data reproducing apparatus for synchronizing and playing back independent data and one or more combination data, The independent data is audio data generated independently, not intended to be used in combination with other data, The combination data is generated for reproduction in combination with the independent data and reproduces synchronization information necessary for synchronizing with the independent data and the same operations in which a predetermined character matches the rhythm of the independent data for a predetermined time. Contains one or more video data, Setting a reception point to receive a reproduction start control signal in the data stream of the video data; Storing and maintaining pattern information of the audio data corresponding to the reception point of the video data in a predetermined recording means; Analyzing a data stream of the audio data and comparing it with pattern information stored in the recording means; Synchronizing a playback time of the audio data with a playback time of the video data when a pattern identical or similar to the pattern information is found in the data stream of the audio data as a result of the comparison; And Playing the audio data and the video data, and repeatedly playing one or more of the video data. Portable multimedia data playback device control method comprising a. A control method of a portable multimedia data reproducing apparatus for synchronizing and playing back independent data and one or more combination data, The independent data is audio data generated independently, not intended to be used in combination with other data, The combination data is generated for reproduction in combination with the independent data and reproduces synchronization information necessary for synchronizing with the independent data and the same operations in which a predetermined character matches the rhythm of the independent data for a predetermined time. Contains one or more video data, Maintaining a data storage for recording association identifier information for additional data reproduced with the audio data, wherein the association identifier information is generated by a user selection input for the audio data and additional data reproduced together; Receiving a selection input for audio data reproduced from a user; Retrieving association identifier information of video data reproduced together with the audio data by referring to the data storage unit; And Controlling video data corresponding to the retrieved association identifier information to be reproduced together with the audio data, and repeatedly playing one or more of the video data Portable multimedia data playback device control method comprising a. A control method of a portable multimedia data reproducing apparatus for synchronizing and playing back independent data and one or more combination data, The independent data is audio data generated independently, not intended to be used in combination with other data, The combination data is generated for reproduction in combination with the independent data and reproduces synchronization information necessary for synchronizing with the independent data and the same operations in which a predetermined character matches the rhythm of the independent data for a predetermined time. Contains one or more video data, Maintaining storage means for recording one or more audio data or one or more video data; Receiving a selection input for audio data reproduced from a user; Extracting a predetermined keyword for the audio data, wherein the keyword is extracted from a file name or tag information of the audio data; Retrieving video data including the extracted keyword as a file name or tag information by referring to the storage means; And Playing the retrieved video data together with the audio data, and controlling one or more of the video data to be repeatedly played. Portable multimedia data playback device control method comprising a. A computer-readable recording medium for recording a program for executing the method of any one of claims 16 to 18.