TWI587696B - Method for synchronization of data display - Google Patents

Description

Method of synchronous data playback

The present invention relates to a data sharing method, and in particular to a method for synchronous playback of data.

With the popularity of mobile devices, applications such as wireless network transmission or cloud services have become inseparable from the daily lives of most people. In order to enhance the interaction between mobile devices and other devices, the Wi-Fi Alliance proposes Miracast or Wi-Fi Display, a technology for wireless video sharing.

In the Miracast specification, video and audio playback devices are divided into two categories, one is called a source and the other is called a sink. The receiving end is further divided into a primary sink and a secondary sink. The difference is that the main receiver can support image or audio output, and the secondary receiver only supports sound data output. When the devices at both ends establish a wireless connection, the device functions and network conditions are coordinated to select a suitable audio and video transmission format, and a video stream connection protocol is established to control a series of video streams. Play and terminate.

Therefore, the devices at both ends can be played through the above-mentioned audio-visual sharing technology. The same audio and video data, but the audio and video data after a series of compression, encryption, decryption and decompression and other data processing will have a propagation delay (propagation delay), so that audio and video data can not be played simultaneously.

The invention relates to a method for synchronously playing data, which can correct the playing time according to the time difference by calculating the time difference, so as to achieve synchronized playing data of the devices at both ends.

According to an aspect of the present invention, a method for synchronous playback of data is provided, comprising the following steps. The local device plays the first media segment and transmits the first media segment to the external device. When the external device plays the first media segment, the local device turns on the radio unit or the image capturing unit to receive the first media segment played by the external device. The device of the present unit calculates a time difference between when the external device plays the first media segment and when the first device plays the first media segment, and the device of the present device can selectively transmit the information including the time difference to the external device. The local device transmits the second media segment to the external device. The local device and the external device synchronously play the second media segment according to the calculated time difference.

According to an aspect of the present invention, a method for synchronous playback of data is provided, comprising the following steps. The local device plays the first media segment and transmits the first media segment to the external device. When the first device plays the first media segment, the external device turns on the radio unit to receive the first media segment played by the local device. The external device calculates a time difference between when the external device plays the first media segment and when the local device plays the first media segment. The external device transmits information including the time difference to the local device. The local device transmits the second media segment to the external device. The unit and the external unit are based on The time difference is synchronized to play the second media segment.

In order to better understand the above and other aspects of the present invention, the preferred embodiments are described below, and in conjunction with the drawings, the detailed description is as follows:

100‧‧‧Main unit

102‧‧‧External devices

C1~C6‧‧‧Media clip

Tc1~Tc7‧‧‧

FIG. 1 is a flow chart of a method for synchronous playback of data according to an embodiment of the invention.

Figure 2A~2D shows a schematic diagram of a device for synchronous playback of data.

FIG. 3 is a timing diagram of synchronous playback of data according to an embodiment of the invention.

FIG. 4 is a timing diagram of synchronous playback of data according to an embodiment of the invention.

FIG. 5 is a timing diagram of synchronous playback of data according to an embodiment of the invention.

FIG. 6 is a timing diagram of synchronous playback of data according to an embodiment of the invention.

FIG. 7 is a timing diagram of synchronous playback of data according to an embodiment of the invention.

FIG. 8 is a flow chart showing a method for synchronously playing data according to another embodiment of the present invention.

FIG. 9 is a timing diagram of synchronous playback of data according to an embodiment of the invention.

FIG. 10 is a timing diagram of synchronous playback of data according to an embodiment of the invention.

In an example of the embodiment, a method for synchronous data playback is provided, which can be applied to data transmission and synchronous playback between electronic products such as televisions, set-top boxes, projectors, smart phones, tablet devices, and notebook computers. . Wherein, any device can be used as a transmitting end of the transmission data, which is referred to as a local device in the present case; Any device can be used as a receiving end for receiving data, which is referred to as an external device in this case. In an embodiment, the device can be a smart phone or a tablet device, and the external device can be a television or a projector. In addition, the device of the present invention may be a smart phone, and the external device may be a tablet device or the like, which is not limited by the present invention. The method can transmit data through a wireless local area network, and can also transmit data through a wired network (cable). Through wireless sharing technology, users can avoid the troubles of connecting wires and converters with different specifications. The interoperability is high, and multimedia materials such as pictures, texts, sounds and images can be shared among different devices. Currently known wireless sharing technologies are air play, WiDi, WiGig, WHDI, MirrorLink and Miracast.

The embodiments are described in detail below, and the embodiments are only intended to be illustrative and not intended to limit the scope of the invention.

First embodiment

Please refer to FIG. 1 and FIG. 2A to FIG. 2D. FIG. 1 is a flow chart showing a method for synchronously playing data according to an embodiment of the present invention, and FIG. 2A to FIG. 2D are diagrams showing a device for synchronous playback of data. In step S110, the local device 100 plays a first media segment and transmits the first media segment to an external device 102. As shown in FIG. 2A, when the own device 100 plays a media segment "A", the external device 102 cannot play the media segment "A" instantaneously due to the transmission delay.

In step S120, when the external device 102 plays the first media segment, the local device 100 turns on the radio unit or the image capturing unit to receive the first media segment played by the external device 102. As shown in FIG. 2B, when the external device 102 plays the media segment "A", the device 100 can be transmitted through a microphone (not shown) or photographed. The device (not shown) receives the media segment played by the external device 102. At this time, the local device 100 is playing another media segment "B", so the local device 100 and the external device 102 have not synchronized playing the data.

In step S130, the local device 100 calculates a time difference according to the playing of the first media segment by the external device 102 and the playback of the first media segment by the local device 100. For the calculation of the time difference, please refer to Figures 3 to 7, which will be illustrated later.

In step S140, the local device 100 transmits a second media segment to the external device 102. In step S150, the home device 100 and the external device 102 synchronously play the second media segment according to the time difference. As shown in FIG. 2C, after the time difference generated by the local device 100 and the external device 102 due to the transmission delay is calculated, the local device 100 corrects the time course of the playback according to the time difference, for example, the local device 100 repeatedly plays the media segment "B" at least once. Until the external device 102 plays the media segment "B". Alternatively, as shown in FIG. 2D, the external device 102 corrects the time course of the playback based on the time difference. For example, the external device 102 skips playing the media segment "B" and directly plays the media segment "C" being played by the local device 100.

Referring to FIG. 3, the horizontal axis of the coordinates indicates the time at which each of the media segments C1 to C4 is played, and the vertical axis indicates the order in which the media segments C1 to C4 are played on different devices. Any of the media segments C1 C C4 can be used as the first media segment for calculating the time difference for the device 100 , for example, calculating the time when the external device 102 plays the media segment C1 and the local device 100 plays the media segment C1 to calculate the time difference (Tshift=Tc3- Tc1).

Next, referring to FIG. 4, after the time difference (Tshift=Tc3-Tc1) is calculated, the local device 100 replays the media segment C2 located at the time point Tc2 at the next time point Tc4 according to the time difference, and so on, the local device 100 The media segment arranged at an earlier time point can be played back at the next time point according to the time difference, so that the home device 100 plays the data synchronously with the external device 102.

Next, referring to FIG. 5, after the time difference (Tshift=Tc3-Tc1) is calculated, the local device 100 replays the media segment C3 located at the time point Tc3 at least once according to the time difference at the next time point Tc4 to reduce the local device 100. The time difference from the playback of the external device 102 can reduce the amount of data for buffering. That is, when the external device 102 plays the first media segment (for example, the media segment C1), and the local device 100 is playing a buffered media segment (for example, the media segment C3), the local device 100 repeatedly plays the buffered media segment at least once according to the time difference. In order to reduce the time difference between the playback of the home device 100 and the external device 102.

In addition, in FIG. 5, after the home device 100 repeatedly plays the buffer media segment once according to the time difference (Tshift=Tc3-Tc1), if there is still a time difference between the device 100 and the external device 102, the device 100 can also be used. A buffer time difference (Tshift = Tc6 - Tc5) when the external device 102 plays another buffered media segment (e.g., media segment C4) and the host device 100 plays the other buffered media segment (e.g., media segment C4) is recalculated. At this time, the own device 100 replays the buffer media segment (for example, the media segment C5) located at the time point Tc6 at least once at the next time point Tc7 according to the buffer time difference, so as to reduce the time difference between the playback of the host device 100 and the external device 102. If the front device 100 and the external device 102 are still When there is a time difference, the above steps are repeated until the home device 100 plays the data in synchronization with the external device 102.

In an embodiment, the media segments C1 C C5 may respectively include a sound information or a sound/image information. The device 100 of the present unit can capture the sound information of any of the media segments C1 C C5 through the radio unit for comparison. Therefore, the home device 100 can calculate the time difference by comparing the sound information of the first media segment to the sound information of the first media segment and the external device 102 playing the sound information of the first media segment.

In another embodiment, the media segments C1 C C5 may respectively include an image information or a sound/image information. The device 100 of the device can capture the image information of any of the media segments C1 C C5 through the image capturing unit for comparison. Therefore, the home device 100 can calculate the time difference by comparing the video information of the first media segment with the video information of the first device segment and the video information of the first media segment by the external device 102.

In still another embodiment, the media segments C1 C C5 may include a timing information. When the local device 100 transmits any of the media segments C1 C C5 to the external device 102, the local device 100 can encrypt the time series information into the media segments C1 C C5 through the encryption unit. In addition, when the local device 100 receives any of the media segments C1 C C5 played by the external device 102, the local device 100 can decrypt the timing information from any of the media segments C1 C C5 through the decryption unit to learn that the media segments C1 C C5 are played. Timing for comparison. Therefore, the device 100 of the present embodiment can calculate the time difference based on the time series information.

The above-described sequential information encryption technology can add an encoded information to a sound information or a sound/image information, and transmit the sound information or sound/image information to the external device 102 along with the sound information. When the external device 102 plays the sound information or the sound/image information, the encoded information will be sent out by the high frequency signal that is not recognizable by the human ear, so that the device 100 can accurately recognize the time series information. Thus, when the external device 102 plays the sound information or the sound/video information of the media segment, the human ear does not receive the high frequency signal, but the radio unit of the device 100 of the present device can still receive the high frequency signal to calculate the time difference. At the same time, it can avoid the interference between the sound information and the timing information when the signal is transmitted.

Referring to FIG. 6, when the time difference (Tshift=Tc3-Tc1) is calculated, the local device 100 can transmit information including the time difference to the external device 102. The external device 102 corrects the time course of the playback according to the time difference. For example, the external device 102 originally plays the media segment C2 at the next time point Tc4, but according to the time difference, it is known that the local device 100 is to play the media segment C4 at the next time point Tc4, so the external device 102 Skip playing media clips C2 and C3 and directly play the media clip C4 (ie, the second media clip) that the host device 100 is about to play. Therefore, the front device 100 and the external device 102 can play back the data in synchronization.

Next, referring to FIG. 7, when the time difference (Tshift=Tc3-Tc1) is calculated, the own device 100 transmits information including the time difference to the external device 102. At this time, the external device 102 skips playing the media segment C2 according to the time difference, and directly plays a buffered media segment C3 at the next time point Tc4, so as to reduce the time difference between the playback of the host device 100 and the external device 102, and reduce the amount of data for buffering. If there is still a time difference between the device 100 and the external device 102, the device 100 can recalculate the external device 102 to play the buffered media segment (eg, the media segment C4) and the host device 100 to play the buffered media segment (eg, the media segment C4). A buffer time difference (Tshift = Tc5 - Tc4). Next, the own device 100 transmits information including the buffer time difference to the external device 102. At this time, the external device 102 skips playing the media segment C5 according to the buffer time difference, and directly plays another media segment C6 that the own device 100 is about to play. If there is still a time difference between the own device 100 and the external device 102, the above steps are repeated until the home device 100 plays the data in synchronization with the external device 102.

Second embodiment

Please refer to FIG. 8 , which is a flowchart of a method for synchronous playback of data according to another embodiment of the present invention. Steps S210, S240, and S250 are the same as steps S110, S140, and S150 of the first embodiment. The difference between this embodiment and the first embodiment is that, in step S220, when the first device 100 plays the first media segment. The external device 102 turns on a radio unit to receive the first media segment played by the local device 100. For example, the external device 102 can receive the first media segment played by the local device 100 through a microphone (not shown). In step S230, the external device 102 calculates a time difference between the playback of the first media segment by the external device 102 and the playback of the first media segment by the local device 100, and transmits information including the time difference to the local device 100.

For the calculation of the time difference, please refer to Figures 9 and 10, which will be described in detail. In an embodiment, the media segments C1 C C5 may respectively include a voice resource. News or a sound/image information. The external device 102 can capture the sound information of any of the media segments C1 C C5 through the radio unit for comparison. Therefore, the external device 102 can calculate the time difference by comparing the sound information of the first media segment to the sound information of the first media segment and the external device 102 playing the sound information of the first media segment.

In another embodiment, the media segments C1 C C5 may respectively include an image information or a sound/image information. The external device 102 can capture the image information of any of the media segments C1 C C5 through the image capturing unit for comparison. Therefore, the external device 102 can calculate the time difference by comparing the video information of the first media segment with the video information of the first media segment and the video information of the first media segment by the external device 102.

In still another embodiment, the media segments C1 C C5 may include a timing information. When the local device 100 transmits any of the media segments C1 C C5 to the external device 102, the local device 100 can encrypt the time series information into the media segments C1 C C5 through the encryption unit. In addition, when the external device 102 receives any of the media segments C1 C C5 played by the device 100, the external device 102 can decrypt the timing information from any of the media segments C1 C C5 through the decryption unit to learn that the media segments C1 C C5 play. Timing for comparison. Therefore, the external device 102 can calculate the time difference based on the time series information.

The above-described sequential information encryption technology can add an encoded information to a sound information or a sound/image information, and transmit the sound information or sound/image information to the external device 102 along with the sound information. When the device 100 of the headquarters plays sound information or sound/image information, the coded information will be sent out with high frequency signals that are not recognizable by the human ear for external use. The device 102 accurately recognizes the timing information. Thus, when the local device 100 plays the sound information or the audio/video information of the media segment, the human ear does not receive the high frequency signal, but the radio unit of the external device 102 can still receive the high frequency signal to calculate the time difference. At the same time, it can avoid the interference between the sound information and the timing information when the signal is transmitted.

Referring to FIG. 9, when the time difference (Tshift=Tc3-Tc1) is calculated, the external device 102 transmits information including the time difference to the home device 100. The own device 100 corrects the time course of the playback according to the time difference. For example, the local device 100 originally plays the media segment C4 at the next time point Tc4, but according to the time difference, the external device 102 is required to play the media segment C2 at the next time point Tc4. Therefore, the device 100 of the present embodiment replays the media segment C2 located at the time point Tc2 at the next time point Tc4 in accordance with the time difference, so that the own device 100 and the external device 102 play the material in synchronization.

Next, referring to FIG. 10, when the time difference (Tshift=Tc3-Tc1) is calculated, the external device 102 transmits information including the time difference to the own device 100. At this time, the own device 100 replays a buffered media segment C3 located at the time point Tc3 at least once at the next time point Tc4 according to the time difference, so as to reduce the time difference between the playback of the host device 100 and the external device 102, and reduce the amount of data for buffering. That is, when the external device 102 plays the first media segment (for example, the media segment C1), and the local device 100 is playing a buffered media segment (for example, the media segment C3), the local device 100 repeatedly plays the buffered media segment at least once according to the time difference. In order to reduce the time difference between the playback of the home device 100 and the external device 102.

In addition, in FIG. 10, if there is still a time difference between the local device 100 and the external device 102, the external device 102 may recalculate the external device 102 to play the buffered media segment (eg, the media segment C4) and the local device 100 to play the A buffer time difference (Tshift = Tc6 - Tc5) when buffering a media segment (e.g., media segment C4). Next, the external device 102 transmits information including the buffer time difference to the home device 100. At this time, the own device 100 replays the buffer media segment (for example, the media segment C5) located at the time point Tc6 at least once according to the buffer time at the next time point Tc7 to reduce the time difference between the playback of the host device 100 and the external device 102. If there is still a time difference between the own device 100 and the external device 102, the above steps are repeated until the home device 100 plays the data in synchronization with the external device 102.

In this embodiment, if the buffer space of the external device 102 is large enough, more media segments can be received early and stored, and the method can be implemented as shown in FIGS. 6 and 7. The difference is only in this embodiment. The external device 102 calculates the time difference instead of transmitting the time difference by the own device 100 before transmitting it to the external device 102. Therefore, the external device 102 can directly play the second media segment according to the time difference, so that the local device 100 and the external device 102 simultaneously play the data. In addition, the external device 102 can also play at least one buffered media segment (eg, media segments C3, C4) according to the time difference to reduce the time difference. Next, the external device 102 recalculates a buffer time difference between the playback of the buffered media segment (eg, media segment C4) by the external device 102 and the playback of the buffered media segment (eg, media segment C4) by the native device 100. At this time, the external device 102 skips playing the media segment C5 according to the buffer time difference, and directly plays the video. Another media segment C6 that the device 100 is about to play. If there is still a time difference between the own device 100 and the external device 102, the above steps are repeated until the home device 100 plays the data in synchronization with the external device 102.

The method for synchronously playing data disclosed in the above embodiments of the present invention is to calculate the time difference and correct the time course of the playback according to the time difference, so as to achieve synchronous playback of the data by the local device and the external device without adding an extra complicated calculation process or transmission protocol. It is suitable for any existing audio-visual sharing device and is more convenient to use.

In conclusion, the present invention has been disclosed in the above preferred embodiments, and is not intended to limit the present invention. A person skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined by the scope of the appended claims.

S110, S120, S130, S140, S150‧‧‧ steps

Claims (6)

A method for synchronizing data playback is applicable to a device and an external device, wherein the device can sequentially play a first media segment, a first buffer media segment and a second media segment, the method comprising: playing the local device The first media segment transmits the first media segment to the external device; when the external device plays the first media segment, the local device turns on a radio unit or an image capturing unit to receive the external device to play The first media segment; the local device calculates a time difference when the external device plays the first media segment and the local device plays the first media segment; the local device transmits the first buffer media segment to the external device, The device of the present unit plays the first buffered media segment at least twice according to the time difference; the local device transmits the second media segment to the external device; and the local device plays the second media segment synchronously with the external device according to the time difference. The method for synchronously playing data according to claim 1, wherein the device of the present unit plays the sound information of the first media segment and the sound information of the first media segment by the external device. At the time, the time difference is calculated. The method for synchronously playing data according to claim 1, wherein the device of the present unit plays the image information of the first media segment and the image information of the first media segment by the external device. At the time, the time difference is calculated. The method for synchronously playing data according to claim 1, wherein the first device transmits the first media segment to the external device, and further includes encrypting a time series information to the first media segment, and the local device receives The first media segment played by the external device further includes decrypting the timing information, and calculating the time difference according to the timing information. A method for synchronizing data playback is applicable to a device and an external device. The device can sequentially play a first media segment, a first buffer media segment and a second media segment, including: the local device plays the first a media segment, and transmitting the first media segment to the external device; when the first device plays the first media segment, the external device turns on a radio unit to receive the first media segment played by the local device; The external device calculates a time difference between when the external device plays the first media segment and the local device plays the first media segment; the external device transmits information including the time difference to the local device; and the local device transmits the first buffer media Fragment to the external device, the local device plays the first buffer media segment at least twice according to the time difference; and the local device transmits the second media segment to the external device; and the external device according to the time difference and the local device The second media segment is played synchronously. The method for synchronously playing data according to claim 5, wherein the first device transmits the first media segment to the external device, and further includes encrypting a time series information to the first media segment, and the external device receives When the first media segment is played by the local device, the method further includes decrypting the timing information, and according to the timing information The time difference is calculated.