KR20140066641A - Server, multimedia apparatus, and control method of thereof - Google Patents

Abstract

A multimedia apparatus according to the present invention includes a communication unit connected to a server; and a control unit transmitting a replay start request of multimedia contents to the server, receiving sync information for replay synchronization of the multimedia contents determined according to the replay start request, and sync-playing the multimedia contents based on the received synchronization information. Therefore, when the multimedia apparatus is used, it is possible to improve the accuracy of the synchronized replay of the multimedia contents in a plurality of multimedia apparatuses.

Description

[0001] DESCRIPTION [0002] SERVER, MULTIMEDIA APPARATUS, AND CONTROL METHOD OF THEREOF [0003]

The present invention relates to a server, a multimedia device, and a control method thereof, and more particularly, to a server, a multimedia device, and a control method thereof for providing synchronous reproduction of multimedia contents in a plurality of multimedia devices.

All of the existing embedded multimedia devices are provided with media players and the core of the player is the audio / video (A / V) synchronization engine. Therefore, in order to realize the function of the synchronous reproduction media of the multiplayer, the interactive control and the clock control of the multiplayer synchronous reproduction must be added during the audio / video (A / V) synchronization control.

The following defects exist in the existing multi-player synchronous playback media technology.

1. Most of the player audio / video (A / V) synchronization engines in the embedded multimedia devices use the system clock of the local device, so the system clocks on each device may be different. Therefore, it is not possible to ensure consistency of a plurality of device player clocks, and as a result, the multi-player can not be synchronously reproduced accurately.

2. When playing a plurality of player's players, if one player's player is operated, the player of the other player can not synchronously respond to the situation without an external hardware control device, none.

Accordingly, it is an object of the present invention to provide a server, a multimedia device, and a control method thereof that can improve the accuracy of synchronous reproduction of multimedia contents in a plurality of multimedia devices when using the multimedia device.

According to another aspect of the present invention, there is provided a method of controlling a server, comprising: receiving a request to start playback of multimedia contents from one of multimedia devices; Determining synchronization information for playback synchronization of multimedia contents between a plurality of multimedia devices based on a system clock of the server; And transmitting synchronization information corresponding to the determined system clock to a plurality of multimedia devices.

The transmitting to the plurality of multimedia devices may include periodically transmitting synchronization information determined based on the system clock.

And receiving the reproduction delay time and adjusting the system clock based on the reproduction delay time when a reproduction delay of multimedia contents occurs in any of the multimedia devices of the plurality of multimedia devices.

Receiving the playback pause time and adjusting the system clock based on the playback pause time when the multimedia content of any one of the plurality of multimedia devices has been paused to play the multimedia content have.

And receiving the playback start time and adjusting the system clock based on the playback start time when the playback of the multimedia content occurs again after the playback of the multimedia content is restarted in any of the multimedia devices of the plurality of multimedia devices .

And receiving the time search time and adjusting the system clock based on the time search time when a time search of the multimedia contents occurs in any of the multimedia devices of the plurality of multimedia devices.

And stopping the reproduction of the multimedia contents according to the reproduction stop request when the reproduction stop of the multimedia contents occurs in any of the multimedia devices of the plurality of multimedia devices.

According to an embodiment of the present invention, there is provided a server, comprising: a communication unit connected to a plurality of multimedia devices; The method includes receiving a request to start playback of multimedia content from one of the plurality of multimedia devices, determining synchronization information for playback synchronization of the multimedia contents between the plurality of multimedia devices based on the system clock of the server, And a control unit for controlling synchronization information to be transmitted to a plurality of multimedia devices.

The control unit may periodically transmit synchronization information determined based on the system clock to the plurality of multimedia devices, and when a reproduction delay of the multimedia contents occurs in any one of the plurality of multimedia devices, And adjust the system clock based on the reproduction delay time.

The controller may receive the playback pause time and adjust the system clock based on the playback pause time when the multimedia content of any one of the plurality of multimedia devices experiences a playback pause of the multimedia content And when the multimedia content of any one of the multimedia devices has been replayed after the playback of the multimedia contents has been paused, the playback start time may be received and the system clock may be adjusted based on the playback start time .

The control unit may receive the time search time and adjust the system clock based on the time search time when a time search of multimedia contents occurs in any of the multimedia devices of the plurality of multimedia devices, The multimedia content playback apparatus may receive the playback stop request and stop the playback of the multimedia content according to the playback stop request when any one of the multimedia devices stops playing the multimedia content.

According to another aspect of the present invention, there is provided a method of controlling a multimedia device, including: transmitting a request to start playback of multimedia content to a server; Receiving synchronization information for playback synchronization of multimedia contents determined in response to a playback start request from a server; And synchronously reproducing the multimedia contents based on the received synchronization information.

The receiving of the synchronization information from the server may include periodically receiving the synchronization information determined based on the system clock of the server, and when the reproduction delay of the multimedia contents occurs, And transmitting the delay time.

And transmitting the playback pause time to the server when the playback of the multimedia content is paused. In the case where the playback is restarted after the pause of the playback of the multimedia content, To the mobile terminal.

The method may further include transmitting the time search time to the server when the time search of the multimedia content occurs, transmitting the stop request to the server when the stop of the playback of the multimedia content occurs . ≪ / RTI >

According to another aspect of the present invention, there is provided a multimedia device comprising: a communication unit connected to a server; A control unit for transmitting a request to start playback of multimedia content to a server, receiving synchronization information for playback synchronization of multimedia content determined in response to a playback start request from a server, and controlling synchronization reproduction of multimedia contents based on the received synchronization information, And the like.

The controller may periodically receive the synchronization information determined based on the system clock of the server, and may transmit the playback delay time to the server when the playback delay of the multimedia content occurs.

Wherein the control unit is operable to transmit the playback pause time to the server when the multimedia contents are temporarily stopped, and wherein, when the playback of the multimedia contents is paused after the playback of the multimedia contents, The start time can be transmitted.

In addition, the controller may transmit the time search time to the server when the time search of the multimedia content occurs, and may transmit the playback stop request to the server when the playback of the multimedia content occurs.

The above object can be achieved according to the present invention, more specifically by the following synchronous reproduction method.

In the synchronous playback method, the synchronous playback method is a method in which any one of a plurality of target multimedia devices is selected as a main control device, a total amount of target multimedia devices is arranged on the main control device, And the IP address and the port of the synchronous playback server are connected to the synchronous playback server. When each device player is connected to the synchronous playback server on the main control device and successfully connected to the synchronous playback server, When the synchronous playback server receives the playback start request message sent from all the device players, it confirms the current playback time curPlayTime according to the current playback system clock curSysTick, which is maintained by itself, and transmits the playback start clock Message And upon receiving the playback start clock message from each device player, starts to synchronously play back the multimedia file according to curPlayTime in the message.

In the synchronous playback method, the synchronous playback server periodically sends a playback system clock notification message to each player and carries the curPlayTime message during the synchronous playback process.

In the synchronous playback method, when an arbitrary device player finds that an audio delay or a video delay occurs in itself according to curPlayTime transmitted from the synchronous playback server, Sending a playback delay message, and the synchronous playback server adjusting the current playback system clock from curSysTick to curSysTick-latetime after receiving the latency.

In the synchronous playback method, when one device player pauses playback in the synchronous playback process, the device player sends a synchronous playback pause message to the synchronous playback server, and the synchronous playback server receives the message and then pauses When a playback system clock notification message periodically transmitted from a synchronous playback server is received in each device player, curPlayTime in the message is always pausedTime (pausedTime) And suspending playback if it is found.

When each device player pauses the reproduction and activates any one device player, the activated device player sends a synchronization reproduction activation message to the synchronization reproduction server, and the synchronization reproduction server receives the message, the playback time of the active playback is determined by calculating the playback time curPlayTime = pausedTick-startPlayTick after the active playback according to the recorded pause playback system clock pausedTick and the playback start system clock startPlayTick recording the curSysTick, and the playback system clock notification message curPlayTime is incremented sequentially in the pausedTick-startPlayTick, and each device player receives the message and changes from pause to playback when it finds that the curPlayTime is increased.

In the synchronous playback method, when a device player of any one of the device players performs a time search in the synchronous playback process, the device player sends a synchronous time search message carrying a synchronous time search time length skiptime to the synchronous playback server, After receiving the message, transmits the message to all device players when it finds that it is not in the current time search state, and after receiving the message from each device player, it jumps from the current playback time curPlayTime to the position of curPlayTime + skiptime The synchronous playback server further includes adjusting the current playback time curPlayTime to curPlayTime + skiptime while the playback is being performed.

In the synchronous playback method, when any one of the device players in the synchronous playback process goes forward (FF) or backward (REW) playback, the device player advances (FF) or backward (REW) message carrying the speed trickspeed of the REW and the synchronous playback server sends the synchronous forward FF and the REW message to the synchronous playback server after receiving the message, And if you are in that state, you will run step B, otherwise you will run step A,

A. The synchronous playback server transmits a synchronous fast forward (FF) and a fast reverse (REW) message to all device players and installs the current playback speed curPlaySpeed as trickspeed. After receiving the message from each device player, (FF) or backward (REW) playback, enters the C stage,

B. The synchronous playback server installs the next playback speed nextPlaySpeed as trickspeed,

C. After completing a time search operation forward or backward in any one device player, it sends a time search complete message to the synchronous playback server and waits for blocking at the same time, and the synchronous playback server transmits a time search complete message (FF) and Rewind (REW), it confirms the completion of the time search operation and judges whether curPlaySpeed and nextPlaySpeed are matched. If they match, it executes step E directly. Otherwise, it executes step D In addition,

D. When confirming that a synchronous playback server has received a new synchronous fast forward (FF) or fast reverse (REW) message, it first sends a next playback speed notification message to all device players, and nextPlaySpeed = trickspeed in the message,

E. The synchronous playback server sends a time search completion response message to all device players,

F. When the device player of any one of the devices receives the time search completion response message sent from the synchronous playback server, the device searcher proceeds to the next time search operation.

In the synchronous playback method, when one of the device players stops playback in the synchronous playback process, the device player sends a synchronous playback stop message to the synchronous playback server, and after the synchronous playback server receives the message, To each device player, and one of the device players further includes stopping the playback process of the device player after receiving the message.

In the synchronous playback method, when a main control device down or a network shutdown situation occurs in the synchronous playback process, each device player can not receive a playback system clock notification message periodically sent from the synchronous playback server, Automatically logging out,

The controller device can not receive the playback system clock notification message periodically transmitted from the synchronous playback server, so that the controlled device can not receive the playback system clock notification message periodically from the synchronous playback server, And further includes automatically logging out the player during the playback process.

After the controlled device player automatically logs out the playback process, the controlled device player proceeds to a time search in the future according to the current playing time curPlayTime sent from the synchronizing playback server at the time of rebooting, and the length of the time search is set to curPlayTime + a The control device player compares the current playback time cURPlayTime1 of the controlled device player with the current playback time cURPlayTime2 of the synchronous playback server, and if it is curPlayTime1 + 10 &amp;le; curPlayTime2, the controlled device player advances the time search in the future and the length of time search is curPlayTime2 + The control device player switches to the operation of comparing the current playback time cURPlayTime1 of the controlled device player with the current playback time cURPlayTime2 of the synchronous playback server after the completion of the time search and if curPlayTime1 <curPlayTime2 <curPlayTime1 + 10,And the delay time length in the message is latetime = curPlayTime2-curPlayTime1, and if curPlayTime1≥curPlayTime2, the controlled device player waits for a time length of curPlayTime1-curPlayTime2 and plays back again, of which a is the default .

The synchronous playback method sends the display timestamp of the audio frame or the video frame to the synchronous playback server before the intermediate player displays the audio or video of the intermediate frame and the synchronous playback server transmits the display timestamp of the received maximum audio frame Calculating the difference value of the display timestamp of the minimum audio frame and calculating the difference between the display timestamp of the received maximum video frame and the display timestamp of the minimum video frame.

The synchronous playback server is a synchronous playback server located on one target multimedia device. The synchronous playback server forms a connection with each of the target multimedia device players and is successfully connected to the synchronous playback server, The current playback time curPlayTime is determined according to the current playback system clock curSysTick which is maintained at the time of receiving the playback start request message sent from all the target multimedia device players, and the playback start clock message , And the message includes a synchronous playback processing module for causing each target multimedia device player to carry out curPlayTime to start synchronous playback of the multimedia file according to the curPlayTime.

The synchronous playback processing module periodically sends a playback system clock notification message carrying curPlayTime to each target multimedia device player in a synchronous playback process, and the curPlaytime is obtained from the curSysTick.

Wherein the synchronous reproduction processing module, when receiving a synchronous reproduction delay message sent from an arbitrary one of the target multimedia device players in the synchronous reproduction process, changes the current reproduction system clock from the curSysTick to the curSysTick- lt; RTI ID = 0.0 &gt; latetime. &lt; / RTI &gt;

The synchronous reproduction processing module records pauseedTime, which is a pause playback time when receiving a synchronous reproduction pause message sent from an arbitrary one of the target multimedia device players in a synchronous reproduction process, and sets pausedTime to curPlayTime, A clock pausedTick is recorded, and upon receipt of a synchronous reproduction enabling message sent from any one of the target multimedia device players, the current reproduction system clock curSysTick is recorded, and the activated reproduction is performed according to the recorded paused reproduction system clock pausedTick and the start reproduction system clock startPlayTick And the subsequent playback time curPlayTime = pausedTick-startPlayTick.

When receiving a synchronous time search message carrying a synchronous time search time length skiptime sent from an arbitrary one of the target multimedia device players in the synchronous playback process, the synchronous playback processing module detects that the current time is not in the current time search state , Adjusting the current playback time from curPlayTime to curPlayTime + skiptime, and transmitting the message to all the target multimedia device players, causing each target multimedia device player to jump from the current playback time curPlayTime to the curPlayTime + skiptime position.

The synchronous reproduction processing module may include a synchronization forward FF that carries forward trickspeed of FF or REW sent from an arbitrary target multimedia device player during a synchronous reproduction process, (FF) or a backward (REW) state. If it is determined that the current state is the state, the next playback speed nextPlaySpeed is set to trickspeed, and if not, all the target multimedia device players (FF) and backward (REW) messages are transmitted, and the current playback speed curPlaySpeed is set to trickspeed. Also, when receiving the time search completion message sent from all the target multimedia device players, whether curPlaySpeed and nextPlaySpeed are matched And if it is coincident with all the target multimedia device players, Sends a response message to the destination multimedia device, and if not, dispatches the next playback speed notification message to all target multimedia device players with nextPlaySpeed = trickspeed, and then sends a time search completion response message to all the target multimedia device players.

The synchronous reproduction processing module may further include transmitting the message to each target multimedia device player when receiving a synchronous reproduction stop message sent from an arbitrary target multimedia device player in a synchronous reproduction process.

The multimedia device player includes a synchronous connection module for establishing a connection with the synchronous playback server and sending a connection success indication to the synchronous playback module after being successfully connected to the synchronous playback module, And a synchronous playback module that starts synchronous playback of the multimedia file according to curPlayTime in the message when receiving the playback start clock message sent from the synchronous playback server.

If the synchronous playback module finds that an audio delay or a video delay occurs in its own according to the curPlayTime periodically transmitted from the synchronous playback server in the synchronous playback process, the synchronous playback module transmits a synchronous playback delay message carrying a delay time length to the synchronous playback server So that the synchronous playback server can adjust the current playback system clock from curSysTick to curSysTick-latetime.

The synchronous playback module sends a synchronous playback pause message to the synchronous playback server when the playback is paused by itself in the synchronous playback process and the curPlayTime in the playback system clock notification message periodically transmitted from the synchronous playback server is always pausedTime And suspending playback if found.

The synchronous reproduction module sends a synchronous reproduction activation message to the synchronous reproduction server when it is found to be activated, and furthermore, the curPlayTime in the reproduction system clock notification message periodically transmitted from the synchronous reproduction server sequentially increases in the pausedTick-startPlayTick And changing from pause to playback when started.

The synchronous playback module sends a synchronous time search message carrying a synchronous time duration length skiptime to the synchronous playback server when performing a time search in the synchronous playback process, and upon receiving a synchronous time search message sent from the synchronous playback server, And advancing playback from the playback time curPlayTime to the position curPlayTime + skiptime.

Wherein the synchronous playback module is configured to synchronize the playback of the synchronous playback file with the synchronous playback server when carrying out a fast forward (FF) or a fast reverse (REW) FF) and a REW message, completes one time forward or backward time search operation, sends a time search completion message to the synchronous playback server, waits for a blocking time, and sends a time search completion response message And then proceeds to the next time search operation.

The synchronous reproduction module further includes sending a synchronous reproduction stop message to the synchronous reproduction server when the reproduction is stopped in the synchronous reproduction process.

If the synchronous playback module fails to receive the playback system clock notification message periodically transmitted from the synchronous playback server in the synchronous playback process, the synchronous playback module logs out the playback process, and when the playback is rebooted, the current playback time curPlayTime And the length of the time search is curPlayTime + a. After completion of the time search, its own current playback time curPlayTime1 and the current playback time of the synchronous playback server curPlayTime2 are compared. If curPlayTime1 + 10 < The time search length is curPlayTime2 + a seconds. After completion of the time search, the current playback time curPlayTime1 of the synchronous playback server is switched to the current playback time curPlayTime2 of the synchronous playback server. If curPlayTime1 <curPlayTime2 <curPlayTime1 + 10, Send latetime = curPlayTime2-curPlayTime1 the synchronous reproduction delay message, and if curPlayTime1≥curPlayTime2 but is played again after waiting as long as the curPlayTime1-curPlayTime2 of which a further include the default value.

The present invention can improve the accuracy of multi-player synchronous playback and does not require additional hardware resources, thus saving cost. The present invention can be easily recognized by all multimedia playback devices, for example, a tablet PC, a smart phone, a video wall, and the like.

Meanwhile, the present invention can provide more various new features and functions for a home digital television, and an intermediate home digital television can synchronously reproduce the same video file on a large screen by using the scheme configuration according to the present invention. This is similar to the operation of a home theater, so that a very good condition can be provided for high-resolution video reproduction in the future.

In addition, the present invention does not significantly increase the system load for general playback devices. For example, in a case where the present invention is applied to a digital television video wall, when 2, 4, 6, or 8 digital television player synchronous playbacks are displayed on a video wall, the maximum CPU utilization is 18% The maximum memory increase usage rate is 2%, and the maximum synchronous reproduction error between each television player is within 5 milliseconds.

1 is a flowchart illustrating a method of synchronously reproducing a multimedia file by a multi-player according to an embodiment of the present invention.
Figure 2 is a flow chart of a method by which a multiplayer synchronously pauses playback according to an embodiment of the present invention.
Figure 3 is a flow chart of a method by which a multiplayer synchronously activates playback in accordance with an embodiment of the present invention.
4 is a method flow diagram of synchronous time search of a multiplayer according to an embodiment of the present invention.
FIG. 5 is a flowchart of a synchronous fast forward (FF) and fast reverse (REW) reproduction method of a multi-player according to an embodiment of the present invention.
6 is a flowchart of a method for synchronously stopping playback of a multi-player according to an embodiment of the present invention.
7 is a flowchart of a method of abnormality processing of a device in a reproducing process according to an embodiment of the present invention.
8 is a flowchart of a synchronization detection method of a reproduction process according to an embodiment of the present invention.
9 is a configuration diagram of a synchronous playback server according to an embodiment of the present invention.
10 is a configuration diagram of a multimedia device player according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may be embodied in many different forms and is not limited to the embodiments described herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and the same or similar components are denoted by the same reference numerals throughout the specification.

First, the network configuration environment of the present invention will be described.

A plurality of target multimedia devices may be connected to the same internal network, the internal network may be a home or office LAN, or may be an internal network of a direct connection without any placement. Switches in the network support the Internet Group Management Protocol (IGMP).

1 is a flowchart illustrating a method of synchronously reproducing a multimedia file by a multi-player according to an embodiment of the present invention. As shown in Fig. 1, concrete steps are as follows.

Step 101: Select any one of N (N? 2) target multimedia devices to be the main control device, allocate the total number N of the target multimedia devices on the main control device, and operate the synchronous playback server Variable, that is, a unicast Internet Protocol (IP) address of the synchronous playback server, a unicast port of the synchronous playback server, a broadcast IP address of the synchronous playback server, and a broadcast port of the synchronous playback server. On the controlled device, the operation parameters of the synchronous playback server on the main control device, that is, the unicast IP address of the synchronous playback server, the unicast port of the synchronous playback server, the broadcast IP address of the synchronous playback server, .

Step 102: Each device player connects to the synchronous playback server on the main control device and sends a Transmission Control Protocol (TCP) message, i.e., CMD_BAVE (playback start request message), to the synchronous playback server when it is successfully connected .

Each device player uses the unicast IP address and port of the synchronous playback server as the destination IP address and port to form a connection with the synchronous playback server.

Step 103: When receiving the N CMD_BAVE messages from the synchronous playback server, all the device players send a User Datagram Protocol (UDP) message, that is, C [curPlayTime] [startPlayTime] E (playback start clock message) .

Here, curPlayTime is the current playback time of the multimedia file, and startPlayTime is the playback time of the first start of the multimedia file, and generally the corresponding value is zero.

And keeps one current playback system clock curSysTick on the synchronous playback server. At the same time, the synchronous playback server records the start playback system clock startPlayTick. That is, it records the clock when the multimedia file is reproduced for the first time. If an abnormal situation such as a pause is not generated in the synchronous playback process, curPlayTime = curSysTick-startPlayTick.

Step 104: Each device player receives the UDP message and synchronously reproduces the multimedia file according to curPlayTime in the message.

Step 105: During synchronous playback, the synchronous playback server periodically sends a system clock notification message carrying curPlayTime and startPlayTime through the UDP multicast method.

Here, the period may be 1 millisecond. The boundary of the system clock of the player audio / video synchronization is generally 10 milliseconds, and the UDP method has the possibility of packet loss, but it can secure the normal reproduction of each device player and is the same as the reproduction clock of the local equipment. Therefore, the reproduction process of a plurality of device players is also the same.

On the other hand, when an audio delay or a video delay occurs in an arbitrary device player in the synchronous playback process, a TCP message, that is, L [latetime] E (synchronous playback delay message) . The synchronous playback server adjusts the current playback system clock from curSysTick to curSysTick-latetime after receiving the latency. Where latetime is the delay time length of the audio or video of the device player. This ensures that each device player can still be synchronously reproduced under the condition that Audio Late or Video Late is generated.

Here, the device player determines whether an audio delay or a video delay occurs through the following steps.

The device player compares the difference between the display timestamp in the current playback audio frame or video frame and the curPlayTime sent from the synchronous playback server and determines that no audio delay or video delay occurs when the difference is within the preset range If not, it is determined that an audio delay or a video delay has occurred.

Figure 2 is a flow chart of a method by which a multiplayer synchronously pauses playback according to an embodiment of the present invention. Specific steps as shown in Fig. 2 are as follows.

Step 201: When one device player pauses playback in a synchronous playback process of a plurality of device players, the device player sends a TCP message, i.e., CMD_PE (synchronous playback pause message) to the synchronous playback server.

Step 202: After receiving the CMD_PE message from the synchronous playback server, the pause playback time pausedTime is recorded, and the paused time is set to curPlayTime to record the pause playback system clock pausedTick.

Step 203: Upon receiving a playback system clock notification message periodically transmitted from the synchronous playback server in each device player, playback is paused if curPlayTime in the message is found to be always pausedTime.

Thus, curPlayTime in the playback system clock notification message periodically transmitted from the synchronous playback server under the condition of pausedTime! = 0 (i.e., pause of playback) is always pausedTime, and all device players pause playback because curPlayTime does not change continuously do.

Figure 3 is a flow chart of a method by which a multiplayer synchronously activates playback in accordance with an embodiment of the present invention. As shown in Fig. 3, specific steps are as follows.

Step 301: When all device players activate any one device player under a state of being in a pause state, the activated device player sends a TCP message, i.e., CMD_RE (synchronous reproduction activation message), to the synchronous reproduction server.

Step 302: After receiving the TCP message CMD_RE, the synchronous playback server records the current playback system clock curSysTick, and according to the recorded pause playback system clock pausedTick and the playback start system clock startPlayTick, the playback time after active playback curPlayTime = (curSysTick -StartPlayTick) - (curSysTick-pausedTick) = pausedTick-startPlayTick, and set pausedTime to 0 at the same time.

PausedTime = 0 indicates that it is not in the pause playback state.

The curPlayTime in the playback system clock notification message periodically transmitted from the next synchronous playback server starts from the pausedTick-startPlayTick, and as the system clock curSysTick steadily increases, each device player changes from pause to playback.

4 is a method flow diagram of synchronous time search of a multiplayer according to an embodiment of the present invention. The concrete steps as shown in Fig. 4 are as follows.

Step 401: When a device player of any one of the plurality of device players performs a time skip during the synchronous playback process, the device player transmits a TCP message to the synchronous playback server, that is, S [skiptime] E ). Here, skiptime is the time length of the synchronous time search.

Step 402: It checks whether the value of isSeekingFlag is 1 by checking the value of isSeekingFlag, which is a time retrieval notation, which is maintained by itself after receiving a TCP message S [skiptime] E from the synchronous playback server. If the value of isSeekingFlag is 1 The step 403 is executed, and if not, the step 404 is executed.

Step 403: The synchronous playback server sends a time search refusal message to the device player and completes the process.

If isSeekingFlag = 1, this indicates that the previous synchronization time search process has not been completed.

Step 404: The synchronous playback server sets isSeekingFlag to 1 and sends a TCP message, i.e., S [skiptime] E (synchronization time search message), to all device players.

Step 405: After receiving the S [skiptime] E message, each device player jumps from the current playback time curPlayTime to the curPlayTime + skiptime position and proceeds to play back a TCP message, that is, a FINISH_SKAE (audio search complete message) Or FINISH_SKVE (video search complete message) is sent out. At the same time, the synchronous playback server adjusts the current playback time curPlayTime to curPlayTime + skiptime.

Step 406: The synchronous playback server receives N TCP messages, i.e., FINISH_SKAE and / or FINISH_SKVE, and sets isSeekingFlag to 0, the present synchronous time search process is completed.

FIG. 5 is a flowchart of a synchronous fast forward (FF) and fast reverse (REW) reproduction method of a multi-player according to an embodiment of the present invention. The concrete steps as shown in FIG. 5 are as follows.

Step 501: When a device player of one of a plurality of devices proceeds to FF or REW playback during synchronous playback of a plurality of device players, the device player transmits a TCP message to the synchronous playback server, that is, T [ trickspeed] E (Sync Forward (FF) Rewind (REW) message). Where rickspeed indicates the speed of forward (FF) or backward (REW).

Step 502: After receiving the TCP message T [trickspeed] E, the synchronous playback server determines whether the current playback speed curPlaySpeed is 1 or 2, and if yes, it executes step 503, (505).

In general, there are two normal playing states of the player. One is curPlaySpeed = 1, at which time the player plays at normal speed. The other is curPlaySpeed = 2, where the player plays at twice the normal speed.

Step 503: The synchronous playback server sends a TCP message T [trickspeed] E to all device players and installs curPlaySpeed as trickspeed.

Step 504: Each device player receives a TCP message T [trickspeed] E and proceeds to forward (FF) or backward (REW) playback according to trickspeed and proceeds to step 506.

Forward (FF) or backward (REW) playback plays one forward (FF) backward (REW) playback thread and the forward (FF) backward (REW) Proceeds either one time forward search or backward time search.

Step 505: The synchronous playback server installs the next playback speed nextPlaySpeed as trickspeed.

If CurPlaySpeed is not equal to 1 or 2, it means that it is already in a fast forward (FF) or backward (REW) state and you only need to modify nextPlaySpeed to trickspeed.

Step 506: After completing one time forward or backward time search operation in any one device player, a TCP message, i.e., a time search complete message (NTE) is sent to the synchronous playback server to fast forward (FF) backward REW) Wait for playback thread blocking.

Step 507: Upon receipt of N TCP messages, i.e., NTE, in the synchronous playback server, completion of one time search operation during the fast forward (FF) backward rewind operation is determined and it is determined whether curPlaySpeed and nextPlaySpeed match Step 509 is performed directly if a match is found and step 508 is executed if a match is not found.

Step 508: When determining that the synchronous playback server has received a new T [trickspeed] E message, it first sends a TCP message, T [nextPlaySpeed] E (next play speed notification message), to all device players. Where nextPlaySpeed = trickspeed.

Step 509: The synchronous playback server sends a response TCP message, i.e. NTE, to all device players.

Step 510: If any one device player receives a TCP message sent from the synchronous playback server, i.e., NTE, it advances the FF (fast forward) rewind (REW) playback thread and proceeds to the next time search operation.

6 is a flowchart of a method for synchronously stopping playback of a multi-player according to an embodiment of the present invention. The concrete steps as shown in FIG. 6 are as follows.

Step 601: When a device player of any one of the plurality of device players stops playback in the synchronous playback process, the device player sends a TCP message, i.e., QTE (synchronous playback stop message), to the synchronous playback server.

Step 602: After receiving the TCP message, that is, the QTE, the synchronous playback server transmits the message to each device player.

Step 603: Any one device player stops its playback process after receiving the TCP message, i.e. QTE.

7 is a flowchart of a method of abnormality processing of a device in a reproducing process according to an embodiment of the present invention. Specific steps as shown in Fig. 7 are as follows.

Step 701: When a down or network interruption occurs in the main control device during the synchronous playback of a plurality of device players, each device player can not receive the playback system clock notification message periodically transmitted from the synchronous playback server, The player automatically logs out the playback process.

At this time, the network environment may be rearranged and the main control device and the synchronous playback server may be operated.

Step 702: When a network blocking situation occurs in any one of the controlled devices in a plurality of player player synchronous playback processes, the controlled device player can receive a playback system clock notification message periodically transmitted from the synchronous playback server And the controlled device player automatically logs out the reproduction process.

Next, after completing the arrangement of the network environment of the controlled device, replaying is replayed, the next time search is performed according to the current playback time curPlayTime sent through the latest playback system clock notification message from the synchronized playback server, curPlayTime + a. After completion of the time search, the following processing is performed.

1) The current playback time cURPlayTime1 of the controlled device player and the current playback time cURPlayTime2 of the synchronous playback server are compared.

2) If curPlayTime1 + 10 &amp;le; curPlayTime2, the controlled device player proceeds to search for the future time according to curPlayTime2 and the length of time search is curPlayTime2 + a sec.

3) If curPlayTime1 <curPlayTime2 <curPlayTime1 + 10, the controlled device player sends a TCP message to the synchronous playback server, ie, L [latetime] E, latetime = curPlayTime2-curPlayTime1.

4) If curPlayTime1≥curPlayTime2, the controlled device player waits for the time of curPlayTime1-curPlayTime2 and plays it again to complete the present process.

Here, 3? A? 10, preferably a = 5, and the unit is seconds.

8 is a flowchart of a synchronization detection method of a reproduction process according to an embodiment of the present invention. As shown in Fig. 8, concrete steps are as follows.

Here, 3? A? 10 and a = 5 are preferable, and the unit is seconds.

8 is a flowchart of a synchronization detection method of a reproduction process according to an embodiment of the present invention. The concrete steps as shown in FIG. 8 are as follows.

Step 801: The mediator player sends a display timestamp of the audio frame or the video frame to the synchronous playback server via the TCP message before displaying the audio or video of the intermediate frame.

In the audio frame, the TCP message to be transmitted is APTS [audioPTS] E (audio display timestamp message), of which audioPTS is the display timestamp of the audio frame. For a video frame, the outgoing TCP message is VPTS [videoPTS] E (video display timestamp message), of which videoPTS is the display timestamp of the video frame.

Step 802: N audio frame display timestamps, i.e., audioPTS-1, audioPTS-2, ..., audioPTS-N and N video frame display timestamps sent from N device players at the synchronous playback server When receiving videoPTS-1, videoPTS-2, ..., videoPTS-N, the maximum audio frame display timestamp and the minimum audio frame display timestamp, MAX (audioPTS), MIN (audioPTS) And finds the maximum video frame display timestamp and the minimum video frame display timestamp, MAX (videoPTS), MIN (videoPTS).

Step 803: The synchronous playback server calculates MAX (audioPTS) MIN (audioPTS), which is the difference between the maximum audio frame display timestamp and the minimum audio frame display timestamp, and outputs the maximum video frame display timestamp and minimum video (VideoPTS)? IN (videoPTS), which is the difference value of the frame display timestamp, to obtain the synchronization accuracy of audio and video of synchronous reproduction.

In an actual application, audioPTS-MAX-audioPTS-MIN <5 msec, videoPTS-MAX-videoPTS-MIN <5 msec according to the result of synchronous detection test of synchronous playback process for eight device players.

9 is a configuration diagram of a synchronous playback server according to an embodiment of the present invention. The synchronous playback server is located on the main control device and the main control device is selected from a plurality of target multimedia devices. As shown in FIG. 9, the synchronous playback server includes a synchronous connection module 91 and a synchronous playback processing module 92. The synchronous connection module 91 establishes a connection with each target multimedia device player, and after successful connection, sends a connection success indication to the synchronous playback processing module 92.

The synchronous reproduction processing module 92 receives the connection success instruction sent from the synchronous connection module 91 and according to the current reproduction system clock curSysTick which is maintained when receiving the reproduction start request message sent from all the target multimedia device players The current playback time curPlayTime is determined and a playback start clock message carrying curPlayTime is transmitted to all the target multimedia device players, thereby causing each target multimedia device player to start synchronized playback of the multimedia file according to the curPlayTime. In the synchronous playback process, a playback system clock notification message carrying curPlayTime is periodically sent to each target multimedia device player, and the curPlaytime is obtained from the curSysTick. In the synchronous playback process, when receiving a synchronous playback delay message sent from any one of the target multimedia device players, the current playback system clock is adjusted from curSysTick to curSysTick-latetime according to the delay time latency carried in the message. When receiving a synchronous reproduction pause message sent from an arbitrary one of the target multimedia device players in the synchronous reproduction process, recording the pauseedTime as the pause playback time, recording the pause playback system clock pausedTick as pausedTime as curPlayTime, And the reproduction start time after the activation reproduction according to the recorded pause playback system clock pausedTick and the start playback system clock startPlayTick curPlayTime = Calculate pausedTick-startPlayTick. When receiving a synchronous time search message carrying a synchronous time search time length skiptime sent from an arbitrary one of the target multimedia device players in the synchronous playback process, if it is found that the current time is not in the time search state, the current playback time is set to curPlayTime curPlayTime + skiptime, and transmits the message to all the target multimedia device players so that each target multimedia device player jumps from the current playback time curPlayTime to the curPlayTime + skiptime position to proceed with playback. (REW) message carrying forward trickspeed (FF) or backward rewind (REW) sent from an arbitrary one of the target multimedia device players in the synchronous reproduction process, (FF) or backward (REW) state. If so, the next playback speed nextPlaySpeed is set to trickspeed. Otherwise, all of the target multimedia device players are fast forwarded (FF) REW) message, and installs the current playback speed curPlaySpeed as trickspeed. When receiving the time search completion message sent from all the target multimedia device players, it judges whether curPlaySpeed and nextPlaySpeed are matched. If they match, Send a response with a search complete message, Sends a next playback rate notification message to all target multimedia device players with nextPlaySpeed = trickspeed, and then sends a time search completion response message to all target multimedia device players. In the synchronous playback process, when receiving a synchronous playback stop message sent from an arbitrary target multimedia device player, the message is transmitted to each target multimedia device player.

10 is a configuration diagram of a multimedia device player according to an embodiment of the present invention. As shown in FIG. 10, the multimedia player includes a synchronous connection module 101 and a synchronous playback module 102. among them,

 The synchronous connection module 101 forms a connection with the synchronous playback server and sends a connection success indication to the synchronous playback module 102 after being successfully connected.

The synchronous playback module 102 sends a playback start request message to the synchronous playback server after receiving the connection success indication sent from the synchronous connection module 101, and when receiving the playback start clock message sent from the synchronous playback server, The synchronous playback of the multimedia file starts according to the curPlayTime in the &quot; curPlayTime &quot; In the synchronous playback process, if it is found that an audio delay or a video delay occurs in itself according to the curPlayTime periodically transmitted from the synchronous playback server, the synchronous playback delay message is transmitted to the synchronous playback server, and the message carries the delay time latency Allows the synchronous playback server to adjust the current playback system clock from curSysTick to curSysTick-latetime. When the reproduction is paused by itself in the synchronous reproduction process, the synchronous reproduction pause message is sent to the synchronous reproduction server, and when it is found that curPlayTime is always pausedTime in the reproduction system clock notification message periodically transmitted from the synchronous reproduction server, do. If it finds that it is activated, it sends a synchronous playback enable message to the synchronous playback server, and if curPlayTime in the playback system clock notification message periodically sent from the synchronous playback server starts to increase sequentially from the pausedTick-startPlayTick, . When a time search is performed in a synchronous playback process, a synchronous time search message is sent to a synchronous playback server, the message carries a synchronous time search time length skiptime, and when receiving a synchronous time search message sent from the synchronous playback server, the current playback time curPlayTime To the curPlayTime + skiptime position to play back. (FF) or backward (REW) speed to synchronous playback server when playing forward (FF) or backward (REW) playback during synchronous playback. REW) message and completes one time forward or backward time search operation, sends a time search complete message to the synchronous playback server, waits for blocking, and upon receiving a time search complete response message sent from the synchronous playback server, Proceed with the time search operation. When the reproduction is terminated in the synchronous reproduction process, the synchronous reproduction stop message is sent to the synchronous reproduction server. When a playback system clock notification message periodically transmitted from the synchronous playback server is not received in the synchronous playback process, when the playback process is logged out and the playback is rebooted, a time advance search is performed according to the current playback time curPlayTime newly transmitted from the synchronous playback server The length of the time search is curPlayTime + a. After completion of the time search, the current playback time cURPlayTime1 of the synchronous playback server is compared with the current playback time cURPlayTime2 of the synchronous playback server. If curPlayTime1 + 10 & Seconds, and after completion of the time search, the current playback time cURPlayTime1 of the synchronous playback server is compared with the current playback time cURPlayTime2 of the synchronous playback server. If curPlayTime1 <curPlayTime2 <curPlayTime1 + 10, the synchronous playback delay message is sent to the synchronous playback server And the delay time in the message is latetime = curPlayTime2-curPlayTime1. If curPlayTime1≥curPlayTime is satisfied, it waits for the time of curPlayTime1-curPlayTime2 and plays it again, of which a is the default value.

Hereinafter, Figs. 11 to 14 show an embodiment of the configuration shown in Figs. The contents described in Figs. 1 to 10 can be similarly applied to Figs.

11 is a block diagram of a server according to an embodiment of the present invention. The configuration shown in Fig. 11 can be explained in connection with the configuration shown in Fig. As shown in FIG. 11, the server 200 includes a communication unit 210 and a control unit 220, and may be connected to the multimedia devices 100a, 100b,..., 100n.

The communication unit 210 connects with the plurality of multimedia devices 100a, 100b, ..., 100n.

The control unit 220 receives a request to start playback of the multimedia content from one of the plurality of multimedia devices 100a, 100b, ..., 100n and transmits the synchronization start request to the synchronous connection module 91 as shown in FIG. May be implemented. The control unit 220 determines synchronization information for playback synchronization of the multimedia contents among the plurality of multimedia devices 100a, 100b, ..., 100n based on the system clock of the server 200, And controls the synchronization information to be transmitted to the plurality of multimedia devices 100a, 100b, ..., 100n. At this time, determining and transmitting the synchronization information may be implemented including the synchronous playback processing module 92, as shown in FIG.

12 is a block diagram of a multimedia device according to an embodiment of the present invention. The configuration shown in Fig. 12 can be explained in connection with the configuration shown in Fig. 12, the multimedia device 100 includes a communication unit 110, a control unit 120, and a display unit 130, and may be connected to the server 200. As shown in FIG.

The communication unit 110 is connected to the server 200.

The control unit 120 transmits a request to start playback of the multimedia content to the server 200, and may be implemented by including the synchronous connection module 101 as shown in FIG. The control unit 120 receives the synchronization information for reproduction synchronization of the multimedia contents determined in response to the reproduction start request from the server 200 and controls to synchronously reproduce the multimedia contents based on the received synchronization information. At this time, synchronous playback of the multimedia contents by receiving the synchronization information may be implemented by including the synchronous playback module 102 as shown in FIG.

13 is a flowchart illustrating a method of controlling a server according to an embodiment of the present invention. The configuration shown in Fig. 13 can be explained in connection with Figs. 1 to 8. First, in operation S210, a playback start request of multimedia contents is received from one of the multimedia devices, and operation S210 may be implemented including operation 102, as shown in FIG. Next, in operation S220, synchronization information for playback synchronization of the multimedia contents among a plurality of multimedia devices is determined based on the system clock of the server. Finally, in operation S230, synchronization information corresponding to the determined system clock is transmitted to a plurality of multimedia devices, wherein operation S230 may be implemented including operation 103, as shown in Fig.

14 is a flowchart illustrating a method of controlling a multimedia device according to an embodiment of the present invention. The configuration shown in Fig. 14 can be explained in connection with Figs. 1 to 8. Fig. First, in operation S110, a playback start request of multimedia contents is transmitted to the server, and operation S110 may be implemented including operation 102, as shown in FIG. Next, in operation S120, synchronization information for reproduction synchronization of the multimedia contents determined in response to the reproduction start request from the server is received. Finally, in operation S130, the multimedia content is synchronously reproduced based on the received synchronization information, and operation S130 may be implemented including operation 104, as shown in FIG.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.

100: Multimedia device
110:
120:
200: Server
210:
220:

Claims (28)

In a server control method,
The method comprising: receiving a request to start playback of multimedia content from one of the plurality of multimedia devices;
Determining synchronization information for playback synchronization of the multimedia contents among the plurality of multimedia devices based on a system clock of the server;
And transmitting synchronization information corresponding to the determined system clock to the plurality of multimedia devices. The method according to claim 1,
Wherein the step of transmitting to the plurality of multimedia devices comprises:
And periodically transmitting synchronization information determined based on the system clock. The method according to claim 1,
Receiving a reproduction delay time and adjusting the system clock based on the reproduction delay time when a reproduction delay of multimedia contents occurs in any of the multimedia devices of the plurality of multimedia devices; . The method according to claim 1,
Receiving a playback pause time and adjusting the system clock based on the playback pause time when a multimedia content of any one of the plurality of multimedia devices occurs in a playback pause state of the multimedia content, / RTI &gt; The method according to claim 1,
Receiving the playback start time and adjusting the system clock based on the playback start time when the multimedia content of any one of the plurality of multimedia devices reproduces again after pausing playback of the multimedia content; How to control the server. The method according to claim 1,
Receiving the time search time and adjusting the system clock based on the time search time when a time search of multimedia contents occurs in any of the multimedia devices of the plurality of multimedia devices . The method according to claim 1,
Receiving a playback stop request when the playback of the multimedia content is stopped in one of the plurality of multimedia devices and stopping the playback of the multimedia content according to the playback stop request; . In the server,
A communication unit for connecting with a plurality of multimedia devices;
Receiving a request to start playback of multimedia content from one of the plurality of multimedia devices, determining synchronization information for playback synchronization of the multimedia content between the plurality of multimedia devices based on a system clock of the server, And controlling the synchronization information corresponding to the system clock to be transmitted to the plurality of multimedia devices. 9. The method of claim 8,
Wherein the control unit periodically transmits synchronization information determined based on the system clock to the plurality of multimedia devices. 9. The method of claim 8,
Wherein the control unit receives the reproduction delay time and adjusts the system clock based on the reproduction delay time when a reproduction delay of the multimedia contents occurs in any one of the multimedia devices. 9. The method of claim 8,
Wherein the controller is configured to receive the playback pause time when the multimedia content of one of the multimedia devices has been paused to play the multimedia content and to adjust the system clock based on the playback pause time, . 9. The method of claim 8,
Wherein the controller receives the playback start time and adjusts the system clock based on the playback start time when the playback of the multimedia content is restarted after the playback of the multimedia content is restarted in any one of the plurality of multimedia devices server. 9. The method of claim 8,
Wherein the controller receives the time search time and adjusts the system clock based on the time search time when a time search of the multimedia contents occurs in any of the multimedia devices of the plurality of multimedia devices. 9. The method of claim 8,
Wherein the control unit receives the playback stop request when the multimedia content of one of the plurality of multimedia devices stops playing the multimedia content and stops the playback of the multimedia content according to the playback stop request. A method for controlling a multimedia device,
Transmitting a playback start request of the multimedia contents to the server;
Receiving synchronization information for playback synchronization of the multimedia content determined in response to the playback start request from the server;
And synchronously reproducing the multimedia contents based on the received synchronization information. 16. The method of claim 15,
Wherein the step of receiving the synchronization information from the server comprises:
And periodically receiving synchronization information determined based on a system clock of the server. 16. The method of claim 15,
And transmitting the playback delay time to the server when a playback delay of the multimedia content occurs. 16. The method of claim 15,
And transmitting the playback pause time to the server when the playback of the multimedia content is paused. 16. The method of claim 15,
And transmitting the playback start time to the server when playback of the multimedia content occurs after playback of the multimedia content. 15. The method according to claim 15,
And transmitting the time search time to the server when the time search of the multimedia contents occurs. 16. The method of claim 15,
And transmitting the playback stop request to the server when the playback of the multimedia content is stopped. In a multimedia device,
A communication unit for connecting to a server;
The multimedia content management method according to claim 1, further comprising: transmitting a request to start playback of multimedia content to the server; receiving synchronization information for playback synchronization of the multimedia content determined in response to the playback start request from the server; And controls to synchronously reproduce the multimedia data. 23. The method of claim 22,
Wherein the control unit periodically receives synchronization information determined based on a system clock of the server. 23. The method of claim 22,
Wherein the control unit transmits the playback delay time to the server when a playback delay of the multimedia content occurs. 23. The method of claim 22,
Wherein the controller transmits the playback pause time to the server when the pause of the multimedia content occurs. 23. The method of claim 22,
Wherein the control unit transmits the playback start time to the server when the playback of the multimedia content occurs after the playback is paused. The method of claim 22,
Wherein the controller transmits the time search time to the server when the time search of the multimedia contents occurs. 23. The method of claim 22,
Wherein the controller transmits the playback stop request to the server when the playback of the multimedia contents is stopped.

Images