CN109218795B - Multi-device playing progress synchronization method and device and terminal device - Google Patents
Multi-device playing progress synchronization method and device and terminal device Download PDFInfo
- Publication number
- CN109218795B CN109218795B CN201811447231.1A CN201811447231A CN109218795B CN 109218795 B CN109218795 B CN 109218795B CN 201811447231 A CN201811447231 A CN 201811447231A CN 109218795 B CN109218795 B CN 109218795B
- Authority
- CN
- China
- Prior art keywords
- playing
- slave
- progress
- master
- playing device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000001360 synchronised effect Effects 0.000 claims abstract description 19
- 238000010586 diagram Methods 0.000 description 10
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 101000969688 Homo sapiens Macrophage-expressed gene 1 protein Proteins 0.000 description 1
- 102100021285 Macrophage-expressed gene 1 protein Human genes 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 208000019116 sleep disease Diseases 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4392—Processing of audio elementary streams involving audio buffer management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The application provides a multi-device playing progress synchronization method, a multi-device playing progress synchronization device and a terminal device, and particularly relates to a method and a terminal device for distributing a master playing device and a slave playing device in a multi-device playing system, wherein when the playing progresses of the slave playing device and the master playing device are asynchronous, the pointer direction of a data buffer area of the slave playing device is changed according to the playing progress difference value of the slave playing device and the master playing device. After the pointer direction is changed, the pipeline time of the slave playing device is correspondingly changed. For audio and video data to be written into the data buffer area, if the timestamp of the audio and video data is smaller than the changed pipeline time, the audio and video data is discarded, and then the audio and video data is quickly synchronized with the main playing device; and on the contrary, writing the data into the corresponding audio/video cache region to further realize the synchronization of the waiting and main playing equipment. According to the method for synchronizing the pointers, the pointers are not required to be played by means of key frame data and external equipment, so that accurate synchronization can be achieved, and meanwhile cost can be greatly reduced.
Description
Technical Field
The present disclosure relates to the field of communications technologies, and in particular, to a method and an apparatus for synchronizing playing schedules of multiple devices, and a terminal device.
Background
In exhibition halls, presentation halls, etc., multiple devices are often used to simultaneously play the same media file for advertisement display, educational propaganda, etc., wherein the types of multimedia files may include video files and audio files.
When each device plays video files, the clock adopted for audio and video synchronization is relatively independent, and the clock is linearly increased and cannot be changed externally. Therefore, in order to implement synchronous skip of playing progress for multiple devices, a commonly adopted method is that when the playing progress of one or more devices is asynchronous with that of other devices, a user performs fast forward or fast backward operation on the playing progress of a target device through a key of a remote controller. However, the position obtained after the fast forward or fast backward is usually a key frame position near the target position, and accurate synchronization cannot be realized, thereby reducing user experience. In order to achieve more accurate synchronization of the playing progress of multiple devices, when multiple devices are to play the same source synchronously, the multiple devices are generally selected to be externally connected to one HDMI (High Definition Multimedia Interface) device at the same time. Fig. 1 is a schematic structural diagram of a multi-device playing system in the prior art. As shown in fig. 1, in this system, playback devices 10 are each connected to an HDMI device 20, and in this system, the playback device 10 is merely a display device, and synchronization of playback progress is regulated by the output of the HDMI device. However, the cost of HDMI equipment is high, and the playback cost is increased by the above playback method.
Therefore, the existing multi-device playing progress synchronization mode has the contradiction between accurate synchronization and high cost.
Disclosure of Invention
The embodiment of the invention provides a method and a device for synchronizing the playing progress of multiple devices and a terminal device, which are used for reducing the cost and ensuring the accurate synchronization of the playing progress of the multiple devices.
According to a first aspect of an embodiment of the present invention, a method for synchronizing a playing progress of multiple devices is provided, where the method includes:
acquiring the playing progress of a master playing device and a slave playing device;
judging whether the playing progress of the slave playing device is synchronous with that of the master playing device;
if not, changing the pointer direction of a data buffer area in the slave playing equipment according to the playing progress difference value of the slave playing equipment and the master playing equipment;
the data buffer area is composed of a plurality of data buffer blocks; the pointer points to the slave play device and is used for indicating that data is read from the data cache block to the audio and video play module in the slave play device.
According to a second aspect of the embodiments of the present invention, there is provided a multi-device playing progress synchronization apparatus, including:
the playing progress obtaining module: the method and the device are used for acquiring the playing progress of the master playing device and the slave playing device.
The playing progress judging module: judging whether the playing progress of the slave playing device is synchronous with that of the master playing device;
the pointer pointing adjustment module: if the data is not synchronized, changing the pointer direction of a data buffer area in the slave playing device according to the playing progress difference value of the slave playing device and the master playing device;
the data buffer area is composed of a plurality of data buffer blocks; the pointer points to the slave play device and is used for indicating that data is read from the data cache block to the audio and video play module in the slave play device.
According to a third aspect of the embodiments of the present invention, a terminal device is provided, where the terminal device includes the multi-device playing progress synchronization apparatus provided in the second aspect of the embodiments of the present invention, and an audio/video playing apparatus connected to the multi-device playing progress synchronization apparatus.
As can be seen from the above technical solutions, in the multi-device playing progress synchronization method, apparatus, and terminal device provided in the embodiments of the present invention, a master playing device and a slave playing device are allocated in a multi-device playing system, and when a device serving as a slave playing device finds that the playing progress of the slave playing device is not synchronized with that of the master playing device, a pointer direction of an internal data buffer area of the device is changed according to a playing progress difference value with the master playing device. The change of the pointer direction correspondingly changes the pipeline time of the slave playing device according to the fact that the pipeline time is equal to the product of the value of the pointer to the corresponding data cache block and the capacity of the data cache block. Therefore, for the audio and video data to be written into the data buffer area, if the time stamp of the audio and video data is smaller than the changed pipeline time, the audio and video data is discarded, and the fast synchronization with the main playing device is further realized; and if the timestamp is greater than the changed pipeline time, writing the timestamp into a corresponding audio/video cache region, and further realizing the synchronization of waiting and main playing equipment. The above-mentioned synchronous process through changing pointer point to need not to play with the help of key frame data and peripheral hardware, and then can also greatly reduce cost when can realizing accurate synchronization.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic structural diagram of a multi-device playing system in the prior art.
Fig. 2 is a schematic flowchart of a method for synchronizing a playing progress of multiple devices according to an embodiment of the present application;
fig. 3 is a schematic diagram of a basic structure of an audio data read/write buffer in a Gstreamer system according to an embodiment of the present application;
fig. 4 is a schematic diagram illustrating changes of a pipeline clock and a system clock in a slave playback device when a progress of the slave playback device is ahead of that of a master playback device according to an embodiment of the present application;
fig. 5 is a schematic diagram illustrating changes of a pipeline clock and a system clock in a slave playback device when the slave playback device is behind the master playback device according to an embodiment of the present application;
fig. 6 is a schematic flowchart of another method for synchronizing a playing progress of multiple devices according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a multi-device playing progress synchronization apparatus according to an embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The method, the device and the terminal device for synchronizing the playing progress of the multiple devices provided by the embodiment are mainly applied to devices which can have the following audio and video playing modes: firstly, storing a decoded film source into a corresponding data cache region, wherein the data cache region consists of a plurality of data cache blocks; and then, reading data from each data cache block to an audio and video playing module in the playing equipment.
Because the clock adopted by the internal audio and video synchronization of the existing terminal equipment is linearly increased and the external part of the existing terminal equipment cannot be changed, the existing audio and video synchronization technology cannot be used for realizing the synchronization of the playing contents of a plurality of pieces of equipment, a clock-controllable multi-equipment playing progress synchronization method is provided based on the embodiment, and the method provided by the embodiment is described in detail below. Fig. 2 is a schematic flowchart of a method for synchronizing a playing progress of multiple devices according to an embodiment of the present application. As shown in fig. 2, the method comprises the steps of:
s110: and acquiring the playing progress of the master playing device and the slave playing device.
In this embodiment, a plurality of playback devices are connected in communication, where a playback device whose playback progress is defined as a standard progress is defined as a master playback device, and the other playback devices are slave playback devices, and of course, the roles of the master playback device and the slave playback devices may also be exchanged in a specific playback process. Further, the playing progress of the master playing device can be actively sent to each slave playing device by the master playing device, and the slave playing devices execute the subsequent synchronization steps after receiving the sent playing progress; alternatively, the slave playback device may send the request to the master playback device at regular time intervals.
The present embodiment takes as an example a device installed with a gsstreamer system, where the gsstreamer is an open source multimedia framework for constructing streaming media applications, and the goal is to simplify the development of audio/video applications, and can be used to process multimedia data in multiple formats such as MP3, Ogg, MPEG1, MPEG2, AVI, Quicktime, and so on.
Fig. 3 is a schematic diagram of a basic structure of an audio data read/write buffer in a Gstreamer system according to an embodiment of the present application. As shown in fig. 3, the GstAudioRingBuffer in the Gstreamer system is a ring audio data read/write buffer that provides the functions of writing decoded audio data into and reading audio data from the audio playback device. Specifically, the ring buffer is composed of 20 segments, where writeSeg is used to indicate that the decoded audio data is to be written in the next segment, and segment done is used to indicate that the read thread is to read data from the next segment and write the data into the audio playing device. Assuming that a segment can store 20ms of audio data, when the segment is equal to 2, it indicates that the audio playing device has played 2 segments of audio data at this time, and the corresponding pipeline time is 40 ms.
Based on the above characteristics, the playing progress of the playing device in this embodiment may refer to the pipeline time and may also refer to the segdone value.
S120: and judging whether the playing progress of the slave playing equipment is synchronous with that of the master playing equipment.
Specifically, a threshold value of the difference between the two schedules can be set, and when the difference is within a certain threshold value range, such as 20ms, the two schedules are considered to be synchronous; otherwise, step S130 is executed.
S130: and if the data is not synchronized, changing the pointer direction of the data buffer area in the slave playing equipment according to the playing progress difference value of the slave playing equipment and the master playing equipment.
In this embodiment, the pointer points to an audio/video playing module for indicating that data is to be read from a fourth data cache block to the slave playing device; in addition, the present embodiment further sets an analog clock corresponding to the pointer direction: a pipe time, wherein the size of the pipe time is equal to the product of the value of the pointer pointing to the corresponding data cache block and the data cache block size.
Specifically, when the device plays the progress and the pointer points to the identifier, the pointer point can be modified directly according to the difference between the progress and the progress. When the playing progress is identified by the pipeline time, firstly, calculating a target pointer direction of a data buffer area in the slave playing device according to the pipeline time of the slave playing device and the master playing device and the capacity of a data cache block in the slave playing device; then, according to the target pointer direction, the pointer direction is changed.
Taking the Gstreamer system as an example, when the slave playing device lags behind the playing progress of the master playing device, if the playing progress of the master playing device is 400ms (segdone value is 20), the current playing progress of the master playing device is 200ms (segdone value is 10). After the slave player receives the playing progress of the master playing device, the slave player finds that the playing progress of the slave player is 10 segments behind, then adds 10 to the segment done of the slave player, at this time, the decoded audio and video data of the slave player lags behind, according to the writing characteristic of the audio data, if the audio timestamp to be written is less than the current pipeline time, the audio timestamp to be written lags behind the pipeline time and needs to be discarded, and if the audio timestamp is greater than the current pipeline time, the audio timestamp is written into the corresponding segment. Therefore, the slave player can directly discard the decoded audio and video data and quickly synchronize to 400ms in a frame loss mode, so that the slave player can be synchronized with the master player.
Similarly, when the playing progress of the slave playing device is ahead of the master playing device, the slave player subtracts the corresponding value from its segment according to the playing progress difference between the slave player and the master playing device, so that the decoded audio/video data of the slave player is ahead, and the decoded audio/video data is directly written in the corresponding segment, and the data in the corresponding segment is erased after the segment value is reduced, meanwhile, the segment value is continuously increased on the basis of modification, that is, the pipeline clock continuously increases linearly, and normal playing is started until the audio data exists in the corresponding segment, that is, the pipeline clock value and the pipeline clock value of the master player are within a certain threshold range, and then the waiting is finished. Therefore, by changing the segdone value of the data buffer, the slave player can achieve synchronization of the master player and the slave player in a waiting mode.
Therefore, when the slave playing device confirms that the playing progress is not synchronous with the master playing device, the pointer direction of the annular buffer area of the audio data is modified, so that the current device can be forced to be synchronous to the target progress.
According to the fact that the value of the pipeline clock is equal to the product of the value of the pointer pointing to the corresponding data cache block and the capacity of the data cache block, the pipeline clock provided by the embodiment has the property of linearly increasing the system clock, but simultaneously provides an interface of an external modification clock and a progress query function. Wherein, the main member of pipeline clock includes: last _ clock _ time (starting pipe time), current _ clock _ time (current pipe time), last _ syslock _ time (starting system time), and current _ syslock _ time (current system time), and lock synchronization control.
Fig. 4 is a schematic diagram illustrating changes of a pipeline clock and a system clock in a slave playback device when a progress of the slave playback device is ahead of that of a master playback device according to an embodiment of the present application. When the playback device starts playing back the film source, its pipe clock starts, recording the current time as 0s, which is used as last _ clock _ time. Assuming that the current system clock is 100s, last _ sysclock _ time is updated synchronously. When a new time comes, for example, the system clock is 110s, that is, current _ syslock _ time is 110, and the current pipe time is: current _ clock _ time ═ last _ clock _ time + (current _ syslock _ time-last _ syslock _ time) ═ 10 s.
When the slave playing device is actively paused, the current system clock and the pipe clock time are obtained based on the above method, for example, when the system clock is paused for 130s in fig. 4, the pipe clock is obtained for 27s, the pipe clock time is kept unchanged until a playing message is received, for example, the playing is resumed when the system clock is 137s, the pipe clock is still 27s, and when the system clock is 140s, the pipe clock is updated for 30 s.
And when the progress of the slave playing device is ahead of that of the master playing device, for example, the playing of the slave playing device is advanced by 3s, at this time, the current _ clock _ time of the pipeline clock is reduced by 3s, as shown in fig. 4, when the system clock is 120s, the pipeline clock should be 20s at the moment, reduced by 3s, and changed to 17s, and then the pipeline clock continues to increase, and when the system clock is 130s, the pipeline clock is 27 s.
If the current playing device lags behind other devices, as shown in fig. 4, the system clock receives a message that the current progress lags 3s when it goes to 120s, at this time, the current _ clock _ time value of the pipeline clock is increased, that is, the current _ clock _ time is changed from 20 to 23s, and then the pipeline clock continues to increase, and when the system clock is 130s, the pipeline clock is 33 s.
Fig. 5 is a schematic diagram illustrating changes of a pipe clock and a system clock in a slave playback device when the slave playback device is behind the master playback device according to an embodiment of the present application. As shown in fig. 5, when the system clock goes to 120s, a message that the current progress lags the main playback device 3s is received, and at this time, the current _ clock _ time value of the pipeline clock is increased, that is, the current _ clock _ time is changed from 20 to 23s, and then the pipeline clock continues to increase, and when the system clock is 130s, the pipeline clock is 33 s.
Further, in the multi-device playing process, a scenario may also occur, for example, when the current master playing device is playing normally, and the slave playing device user performs a fast forward operation, if it is detected that the playing progress of the slave playing device is the position fast forwarded by the user at this time, but because the actual position after fast forward is small, if the pipeline clock is operated at this time, a problem that the clock is a negative value after modification occurs.
In view of the above problem, when the pointer pointing to the data buffer in the slave playback device is changed, the embodiment further provides the following steps:
s131: and judging whether the pipeline time of the slave playing equipment is a negative value or not after the pointer direction is changed.
S132: if the value is negative, the slave playing device pauses playing.
S133: and after the slave playing equipment suspends the playing for a preset time, acquiring the current playing progress of the master playing equipment.
For example, the playing advance time is detected again after sleeping for 5ms each time until the advance time is less than the current playing progress.
S134: and changing the pointer direction of the data buffer area in the slave playing equipment according to the difference value between the playing progress of the slave playing equipment and the current playing progress of the master playing equipment.
Further, if the playing device is also used for playing video data, the video player can perform audio and video synchronization inside the device with the audio clock as a reference. Specifically, the following synchronization method may be adopted:
firstly, whether the timestamp of the video data decoded from the playing device is larger than the current pipeline time is judged, wherein the pipeline time is equal to the product of the value of the current pointer pointing to the corresponding data cache block and the capacity of the data cache block. And then, if the time is longer than the current pipeline time, writing the decoded video data into a corresponding video buffer area, namely waiting for the thread to be suspended. Otherwise, adopting frame loss mode. Thus, the synchronization of the video and the audio from the inside of the playing device is realized.
Further, during the playing process of the playing device, there may be another synchronization method provided in this embodiment for increasing the synchronization speed, for example, the master playing device is fast-forwarded for 50s, and correspondingly, all the slave playing devices should execute fast-forwarding operation. Fig. 6 is a flowchart illustrating another method for synchronizing a playing progress of multiple devices according to an embodiment of the present application. As shown in fig. 6, the method specifically includes the following steps:
s210: and acquiring the playing progress of the master playing device and the slave playing device.
S220: and judging whether the playing progress difference value of the slave playing device and the master playing device is greater than a first threshold value.
If the first threshold is exceeded, for example, 5S, the coarse synchronization steps in steps S230 to S250 are executed. If the first threshold is not exceeded, step S280 is executed to determine whether it is greater than the second threshold, and if so, step S270 is executed.
S230: and if the target playing progress is larger than the first threshold, determining the target playing progress of the slave playing equipment according to the playing progress difference value of the slave playing equipment and the master playing equipment.
In this embodiment, the method adopted for performing the rough synchronization step is to find a key frame near the target playing progress, and since the time corresponding to the target playing progress may be a non-key frame, such as a P frame and a B frame, if the P frame and the B frame are transmitted, an abnormal problem such as a coding failure or a screen display is caused. The distribution of the key frames in the film source is not fixed, some film sources may have 1 key frame at an interval of one or two seconds, but some film sources may have one key frame at an interval of 10s or more. Therefore, the searched key frame is different from the target position by several seconds, for example, if the target playing progress is 15s, the actual returned key frame may be 6 s.
In addition, the rough synchronization itself consumes time, and for the above reasons, when the target playing progress is determined, the present embodiment adds a preset time length to the playing progress difference between the slave playing device and the master playing device, so as to serve as a step length that the playing progress of the slave playing device needs to be changed when being adjusted to the target playing progress. Wherein, according to the test result, the preset time is designed to be 5 s.
S240: and searching the time stamp of the key frame closest to the target playing progress from the video data time index table of the slave playing equipment.
The time index table of video data is mainly composed of the timestamp pts and the offset position offset of the key frame, i.e., < pts, offset >. The time index table is generally directly parsed by demux, such as MP4, avi format, and some cannot be parsed, and needs to be dynamically added during playing, such as ts format.
S250: and synchronizing the playing progress of the slave playing equipment to the offset position corresponding to the time stamp.
I.e. the data is parsed starting at the offset position.
S260: and judging whether the difference value between the synchronized playing progress of the slave playing equipment and the playing progress of the master playing equipment is smaller than a second threshold value.
If the value is less than the second threshold value, the two are considered to be synchronized, such as within 20 ms; if the threshold value is greater than the second threshold value, step S270 is executed.
S270: and if the current time is not less than the second threshold, changing the pointer direction of the data buffer area in the slave playing device according to the playing progress difference value of the slave playing device and the master playing device.
Further, for more accurate synchronization, when the distance between the master and slave playback devices is long or the network quality is poor, there is a situation of data transmission delay, and in addition, there is a certain delay time for the playback device to read and display data, and for this problem, this example provides that after obtaining the current playback progress of the master playback device, the following operations are also executed:
and taking the sum of the current playing progress of the main playing device, the network delay time between the main playing device and the slave playing device and the display delay time of the slave player as the playing progress of the main playing device.
Based on the inventive concept of the above method, the present embodiment further provides a multi-device playing progress synchronization apparatus. Fig. 7 is a schematic structural diagram of a multi-device playing progress synchronization apparatus according to an embodiment of the present application. As shown in fig. 7, the apparatus includes:
the play progress acquisition module 710: the method and the device are used for acquiring the playing progress of the master playing device and the slave playing device.
The play progress determination module 720: judging whether the playing progress of the slave playing device is synchronous with that of the master playing device;
the pointer pointing adjustment module 730: and if the data is not synchronized, changing the pointer direction of the data buffer area in the slave playing device according to the playing progress difference value of the slave playing device and the master playing device.
The data buffer area is composed of a plurality of data buffer blocks; the pointer points to the slave play device and is used for indicating that data is read from the data cache block to the audio and video play module in the slave play device.
Based on the above device, this embodiment also provides a terminal device, and the terminal device in this embodiment may be a television, a computer-television all-in-one machine, and the like. Specifically, the terminal device includes the multi-device playing progress synchronizing device provided in the above embodiment, and an audio/video playing device connected to the multi-device playing progress synchronizing device. The audio and video playing device can comprise a loudspeaker for outputting audio signals and a screen which provides an output interface between the terminal and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, they are described in relative terms, as long as they are described in partial descriptions of method embodiments. The above-described embodiments of the apparatus and system are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The foregoing is merely a detailed description of the invention, and it should be noted that modifications and adaptations by those skilled in the art may be made without departing from the principles of the invention, and should be considered as within the scope of the invention.
Claims (8)
1. A multi-device playing progress synchronization method is characterized by comprising the following steps:
acquiring the playing progress of a master playing device and a slave playing device;
judging whether the playing progress difference value of the slave playing device and the master playing device is larger than a first threshold value;
if the target playing progress is larger than the first threshold, determining the target playing progress of the slave playing equipment according to the playing progress difference value of the slave playing equipment and the master playing equipment;
searching a time stamp of a key frame closest to the target playing progress from a video data time index table of the slave playing equipment;
synchronizing the playing progress of the slave playing equipment to an offset position corresponding to the timestamp;
judging whether the synchronous playing progress of the slave playing equipment and the synchronous playing progress of the master playing equipment are smaller than a second threshold value;
if the current time is not less than the second threshold, changing the pointer direction of the data buffer area in the slave playing device according to the playing progress difference value of the slave playing device and the master playing device;
the data buffer area is composed of a plurality of data buffer blocks, and the stored data is decoded data; the pointer points to the slave play device and is used for indicating that data is read from the data cache block to the audio and video play module in the slave play device.
2. The method of claim 1, wherein determining the target playing progress of the slave playing device according to the playing progress difference between the slave playing device and the master playing device comprises:
and adding a preset time length to the playing progress difference value of the slave playing device and the master playing device to serve as a step length of the slave playing device to be adjusted to the target playing progress to be changed.
3. The method of claim 1, wherein changing the pointer pointing of the data buffer in the slave playback device comprises:
judging whether the pipeline time of the slave playing device is a negative value or not after the pointer direction is changed, wherein the pipeline time is equal to the product of the value of the pointer pointing to the corresponding data cache block and the capacity of the data cache block;
if the value is negative, the slave playing equipment pauses playing;
after the slave playing device pauses playing for a preset time, acquiring the current playing progress of the master playing device;
and changing the pointer direction of the data buffer area in the slave playing equipment according to the difference value between the playing progress of the slave playing equipment and the current playing progress of the master playing equipment.
4. The method of claim 1, wherein changing the pointer direction of the data buffer in the slave playback device according to the difference between the playback progress of the slave playback device and the playback progress of the master playback device comprises:
calculating the target pointer direction of a data buffer area in the slave playing device according to the playing progress difference value between the slave playing device and the master playing device and the capacity of a data cache block in the slave playing device;
and changing the pointer direction of the slave playing equipment according to the target pointer direction.
5. The method of claim 1, wherein after changing the pointer pointing to the data buffer in the slave playback device, the method further comprises:
judging whether the timestamp of the decoded video data in the slave playing device is greater than the current pipeline time, wherein the pipeline time is equal to the product of the value of the current pointer pointing to the corresponding data cache block and the capacity of the data cache block;
and if the channel time is longer than the current channel time, writing the decoded video data into a corresponding video buffer area.
6. The method of claim 1, wherein obtaining the playing progress of the main playing device comprises:
acquiring the current playing progress of the main playing equipment;
and taking the sum of the current playing progress of the master playing device, the network delay time between the master playing device and the slave playing device and the display delay time of the slave playing device as the playing progress of the slave playing device.
7. A multi-device playing progress synchronization apparatus, the apparatus comprising:
the playing progress obtaining module: the system comprises a main playing device and a slave playing device, wherein the main playing device and the slave playing device are used for playing data;
the first playing progress judging module: judging whether the playing progress difference value of the slave playing device and the master playing device is larger than a first threshold value;
the second playing progress judging module: judging whether the playing progress difference value of the slave playing device and the master playing device is larger than a second threshold value;
a key frame adjusting module, configured to determine a target playing progress of the slave playing device according to a playing progress difference between the slave playing device and the master playing device if the playing progress difference between the slave playing device and the master playing device is greater than a first threshold, search a time index of a key frame closest to the target playing progress from a video data time index table of the slave playing device, and synchronize the playing progress of the slave playing device to an offset position corresponding to the time index;
the pointer pointing adjustment module: the device is used for the slave playing device after being adjusted by the key frame adjusting module, and if the playing progress difference value between the slave playing device and the master playing device is larger than a second threshold value, the pointer direction of a data buffer area in the slave playing device is changed according to the playing progress difference value between the slave playing device and the master playing device;
the data buffer area is composed of a plurality of data buffer blocks, and the stored data is decoded data; the pointer points to the slave play device and is used for indicating that data is read from the data cache block to the audio and video play module in the slave play device.
8. A terminal device, characterized in that the terminal device comprises the multi-device playing progress synchronizing device of claim 7, and an audio/video playing device connected with the multi-device playing progress synchronizing device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811447231.1A CN109218795B (en) | 2018-11-29 | 2018-11-29 | Multi-device playing progress synchronization method and device and terminal device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811447231.1A CN109218795B (en) | 2018-11-29 | 2018-11-29 | Multi-device playing progress synchronization method and device and terminal device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109218795A CN109218795A (en) | 2019-01-15 |
| CN109218795B true CN109218795B (en) | 2021-09-24 |
Family
ID=64994528
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811447231.1A Active CN109218795B (en) | 2018-11-29 | 2018-11-29 | Multi-device playing progress synchronization method and device and terminal device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109218795B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112118527A (en) * | 2019-06-19 | 2020-12-22 | 华为技术有限公司 | Multimedia information processing method, device and storage medium |
| CN111327911A (en) * | 2020-03-12 | 2020-06-23 | 山东云缦智能科技有限公司 | Method for synchronizing video frames in multi-view live scene |
| CN111541926B (en) * | 2020-05-08 | 2022-05-03 | 浙江大华技术股份有限公司 | Multimedia file playing method and device, storage medium and electronic device |
| CN112040288B (en) * | 2020-09-04 | 2022-07-12 | 华人运通(上海)云计算科技有限公司 | Multimedia system |
| CN114157893B (en) | 2020-09-08 | 2023-06-30 | 京东方科技集团股份有限公司 | Method and device for synchronously playing videos among multiple devices |
| CN112188258B (en) * | 2020-09-22 | 2021-12-17 | 上海商泰汽车信息系统有限公司 | Pause synchronization method and device, storage medium, user side and multi-device playing system |
| CN112702620A (en) * | 2020-12-29 | 2021-04-23 | 四川长虹电器股份有限公司 | Method for realizing multi-screen synchronous playing of player based on gstreamer |
| CN112887769B (en) * | 2021-01-21 | 2023-09-19 | 青岛海信传媒网络技术有限公司 | Display equipment |
| CN113055728A (en) * | 2021-03-31 | 2021-06-29 | 海信电子科技(武汉)有限公司 | Multi-device synchronous playing method and display device |
| WO2023070374A1 (en) * | 2021-10-27 | 2023-05-04 | 京东方科技集团股份有限公司 | Audio and video synchronization method and apparatus, storage medium, and electronic device |
| CN114827681B (en) * | 2022-04-24 | 2024-03-22 | 咪咕视讯科技有限公司 | Video synchronization method, device, electronic equipment, terminal equipment and storage medium |
| CN115426514A (en) * | 2022-07-22 | 2022-12-02 | 北京罗克维尔斯科技有限公司 | Cross-device audio and video synchronization method, device, equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101106724A (en) * | 2006-07-12 | 2008-01-16 | 联发科技股份有限公司 | Method and system for synchronizing audio and video data |
| CN103391467A (en) * | 2013-07-16 | 2013-11-13 | 四川省迪特尔电子有限公司 | Method for achieving synchronization of decoding and displaying of audio and video of network set-top box |
| CN103839558A (en) * | 2012-11-23 | 2014-06-04 | 三星电子（中国）研发中心 | Method and server for synchronous playing and player |
| CN107360475A (en) * | 2017-08-18 | 2017-11-17 | 苏州科可瑞尔航空技术有限公司 | Audio video synchronization plays and control method, control terminal and playback terminal |
| CN107959659A (en) * | 2016-10-17 | 2018-04-24 | 杭州海康威视数字技术股份有限公司 | A kind of flow medium play control method, device and electronic equipment |
| CN108540829A (en) * | 2018-04-12 | 2018-09-14 | 青岛海信电器股份有限公司 | A kind of more playback equipment synchronous broadcast methods, system and terminal |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| RU2014110047A (en) * | 2011-08-16 | 2015-09-27 | Дэстини Софтвэар Продакшнз Инк. | VIDEO RENDERING BASED ON SCENARIO |
| US20140059134A1 (en) * | 2012-08-24 | 2014-02-27 | Realtek Semiconductor Corp. | Multimedia delivery system and related computer program product for providing statistical graph related to auxiliary multimedia data |
| US20150062353A1 (en) * | 2013-08-30 | 2015-03-05 | Microsoft Corporation | Audio video playback synchronization for encoded media |
| US9826015B2 (en) * | 2013-09-04 | 2017-11-21 | Qualcomm Incorporated | Dynamic and automatic control of latency buffering for audio/video streaming |
| CN108156498B (en) * | 2017-11-13 | 2020-11-20 | 海信视像科技股份有限公司 | Audio and video synchronization method and device |
| CN108200481B (en) * | 2017-12-07 | 2020-12-15 | 北京佳讯飞鸿电气股份有限公司 | RTP-PS stream processing method, device, equipment and storage medium |
| CN108737874A (en) * | 2018-06-05 | 2018-11-02 | 武汉斗鱼网络科技有限公司 | A kind of video broadcasting method and electronic equipment |
-
2018
- 2018-11-29 CN CN201811447231.1A patent/CN109218795B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101106724A (en) * | 2006-07-12 | 2008-01-16 | 联发科技股份有限公司 | Method and system for synchronizing audio and video data |
| CN103839558A (en) * | 2012-11-23 | 2014-06-04 | 三星电子（中国）研发中心 | Method and server for synchronous playing and player |
| CN103391467A (en) * | 2013-07-16 | 2013-11-13 | 四川省迪特尔电子有限公司 | Method for achieving synchronization of decoding and displaying of audio and video of network set-top box |
| CN107959659A (en) * | 2016-10-17 | 2018-04-24 | 杭州海康威视数字技术股份有限公司 | A kind of flow medium play control method, device and electronic equipment |
| CN107360475A (en) * | 2017-08-18 | 2017-11-17 | 苏州科可瑞尔航空技术有限公司 | Audio video synchronization plays and control method, control terminal and playback terminal |
| CN108540829A (en) * | 2018-04-12 | 2018-09-14 | 青岛海信电器股份有限公司 | A kind of more playback equipment synchronous broadcast methods, system and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109218795A (en) | 2019-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109218795B (en) | Multi-device playing progress synchronization method and device and terminal device | |
| KR102486777B1 (en) | Establishment and use of temporal mapping based on interpolation using low-rate fingerprinting to facilitate frame-accurate content modification | |
| EP3850859B1 (en) | Dynamically adjusting video to improve synchronization with audio | |
| KR20210020166A (en) | Dynamic control of fingerprinting rate to facilitate temporally accurate media content modification | |
| US20200404399A1 (en) | Methods, systems, and media for synchronizing audio and video content on multiple media devices | |
| CN110581973B (en) | Data playback method and device, terminal equipment and storage medium | |
| US11812103B2 (en) | Dynamic playout of transition frames while transitioning between playout of media streams | |
| US8739041B2 (en) | Extensible video insertion control | |
| US20210029397A1 (en) | Adaptive switching in a whole home entertainment system | |
| KR20090106973A (en) | System and method for synchronization of television signals associated with multiple broadcast networks | |
| KR102311314B1 (en) | Advance preparation for content modification based on expected latency in obtaining new content | |
| KR20170061100A (en) | Method and apparatus of cynchronizing media | |
| US11943497B2 (en) | Network-based audio playback | |
| TW202046742A (en) | Content-modification system with responsive transmission of reference fingerprint data feature | |
| US11388474B2 (en) | Server-side scene change content stitching | |
| US7254310B2 (en) | Clock-slaving in a programmable video recorder | |
| CN112073791A (en) | Playing synchronization method and device, storage medium and user side | |
| CN114339326B (en) | Sound and picture synchronization method, device and system based on video playing | |
| CN115086708B (en) | Video playing method and device, electronic equipment and storage medium | |
| JPH10308936A (en) | Video distribution system data distribution method and medium | |
| US20120065751A1 (en) | Signal processing apparatus and signal processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 266555 Qingdao economic and Technological Development Zone, Shandong, Hong Kong Road, No. 218 Applicant after: Hisense Visual Technology Co., Ltd. Address before: 266555 Qingdao economic and Technological Development Zone, Shandong, Hong Kong Road, No. 218 Applicant before: QINGDAO HISENSE ELECTRONICS Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |