CN114143584B - Broadcasting system and method for realizing synchronous broadcasting among multiple terminals - Google Patents
Broadcasting system and method for realizing synchronous broadcasting among multiple terminals Download PDFInfo
- Publication number
- CN114143584B CN114143584B CN202111156509.1A CN202111156509A CN114143584B CN 114143584 B CN114143584 B CN 114143584B CN 202111156509 A CN202111156509 A CN 202111156509A CN 114143584 B CN114143584 B CN 114143584B
- Authority
- CN
- China
- Prior art keywords
- terminal
- playing
- synchronous
- time stamp
- broadcasting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 82
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000006243 chemical reaction Methods 0.000 claims abstract description 7
- 238000005538 encapsulation Methods 0.000 claims description 18
- 108700041286 delta Proteins 0.000 claims description 4
- 238000004806 packaging method and process Methods 0.000 description 15
- 238000012545 processing Methods 0.000 description 8
- 238000009826 distribution Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000011056 performance test Methods 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Computer Security & Cryptography (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a broadcasting method and a system for realizing synchronous broadcasting among multiple terminals, wherein the method comprises the following steps: s1, marking a reference time stamp by the source end content, wherein the reference time stamp is used for ensuring that the reference time stamp of the audio and video data is accurately restored and obtained in a direct or conversion mode for each frame of audio and video data after the terminal receives the data; s2, connecting the synchronous server, judging whether the connection is successful or not, and if the connection is not successful, prompting an error; if the connection is successful, playing the terminal; s3, a main terminal playing step, after the main terminal starts playing, reporting the current playing progress and state information to the synchronous service; s4, when the slave terminal starts to play, the slave terminal firstly pulls the current playing progress and state information of the master terminal from the synchronous service, simultaneously pulls the media stream, and then starts to play according to the data offset of the target setting starting playing of the synchronous playing with the master terminal.
Description
Technical Field
The invention belongs to the technical field of streaming media production and playing, and particularly relates to a playing system and method for realizing synchronous playing among multiple terminals.
Background
When playing the same program source, if playing through a plurality of different types of intelligent terminal devices, users often want to realize synchronous playing, that is, the pictures seen by different playing devices at any moment are synchronous.
In general, to achieve picture synchronization between terminals, each terminal needs to clearly know that, at the moment of the common reference clock C, a picture with an original timestamp T needs to be played, so that the pictures are synchronized. The original time stamp refers to that in the production link, the audio/video data producer defines a common reference time axis according to the playing effect, as shown in fig. 1, the audio frames 0 to M are arranged in time sequence, and the video frames 0 to N are also arranged in time sequence. In reality, however, referring to fig. 2, since the transmission of streaming media often goes through many links including content production (1 audio data and 1 video data), format encapsulation (2 channels), distribution channels (3 channels), and playback terminals (3 channels), the following problems are faced:
(1) On different terminals, the clocks cannot be exactly synchronized, that is to say lack a common reference clock C for synchronization; the cost of the terminal increases if accurate timing is to be achieved.
(2) Time stamping is the most important means of identifying video frames in current video technology. However, in an actual use environment, the same content is often packaged into different packaging formats and then distributed, and because the time expression modes of the different packaging formats are often different, that is, the time stamp of the frame is T in the production link, after the packaging is completed, the frame is transformed according to the definition of the packaging format, for example, the frame becomes an increment number from 0, some time units are changed, some overflow occurs due to the data precision, and the like), so that the terminal cannot find the corresponding video frame according to the time stamp.
(3) The distribution link may generate unpredictable delay, so that the time for the same frame data to reach the terminal may be greatly different.
The above factors cause that the content played on different terminal devices is fast or slow, and synchronization cannot be realized.
Disclosure of Invention
The invention aims to solve the technical problem of providing a broadcasting method and a broadcasting system for realizing synchronous broadcasting among multiple terminals, which are used for eliminating the broadcasting time difference among the multiple terminals as much as possible on the premise of not carrying out accurate time service synchronization on the terminals.
In order to solve the technical problems, the invention adopts the following technical scheme:
an aspect of the present invention provides a broadcasting method for implementing synchronous broadcasting between multiple terminals, which is characterized by comprising the following steps:
s1, marking a reference time stamp by the source end content, wherein the reference time stamp is used for ensuring that the reference time stamp of the audio and video data is accurately restored and obtained in a direct or conversion mode for each frame of audio and video data after the terminal receives the data;
s2, connecting the synchronous server, judging whether the connection is successful or not, and if the connection is not successful, prompting an error; if the connection is successful, playing the terminal;
s3, a main terminal playing step, after the main terminal starts playing, reporting the current playing progress and state information to the synchronous service;
s4, when the slave terminal starts to play, the slave terminal firstly pulls the current playing progress and state information of the master terminal from the synchronous service, simultaneously pulls the media stream, and then starts to play according to the data offset of the target setting starting playing of the synchronous playing with the master terminal.
Preferably, in S1, in the packaging step, an original reference timestamp is attached to each frame of audio and video data in the media packaging format, where the reference timestamp is continuously increased according to the broadcasting sequence.
Preferably, in S1, a reference timestamp is periodically stamped in the code stream, and the terminal calculates and restores to obtain a reference timestamp corresponding to a certain frame by using the timestamp defined by the encapsulation format itself.
Preferably, in S3, the master terminal transmits the status information at a certain time interval.
Preferably, in S3, the master terminal transmits the status information when the status is changed.
In yet another aspect, the embodiment of the present invention provides a broadcasting system for implementing synchronous broadcasting between multiple terminals, including a content creation terminal, a synchronization server, a master terminal and at least one slave terminal,
the content making end is used for marking the content of the source end reference time stamp, so that the terminal can accurately restore each frame of audio and video data in a direct or conversion mode after receiving the data to obtain the reference time stamp of the audio and video data;
the synchronous server is used for connecting with the master terminal and the slave terminal, and playing the terminal after the connection is successful;
the main terminal is used for reporting the current playing progress and state information to the synchronous service after playing is started;
when the slave terminal starts playing, the slave synchronous server firstly pulls the current playing progress and state information of the corresponding master terminal, simultaneously pulls the media stream, and then starts playing according to the data offset of the target setting of playing synchronously with the master terminal.
Preferably, in the content making end, in the packaging link, an original reference time stamp is respectively attached to each frame of audio/video data in the media packaging format, and the reference time stamps are continuously increased according to the broadcasting sequence.
Preferably, in the content making end, a reference time stamp is periodically stamped in the code stream, and the terminal calculates and restores to obtain a reference time stamp corresponding to a certain frame by using the time stamp defined by the encapsulation format.
Preferably, the master terminal transmits the status information at certain time intervals.
Preferably, the master terminal transmits the status information when the status is changed.
The invention has the following beneficial effects:
(1) The streaming media synchronous broadcasting of different types of terminals and different distribution channels can be realized;
(2) Cross-terminal interactions (including but not limited to subtitles, advertisements, etc.) may be implemented.
Drawings
FIG. 1 is a diagram of a prior art audio/video frame set reference time;
FIG. 2 is a schematic diagram of a production, distribution and consumption flow of audio and video content common in the prior art;
FIG. 3 is a flowchart illustrating steps of a method for implementing synchronous playing between multiple terminals according to an embodiment of the present invention;
fig. 4 is a data flow chart of a broadcasting method for implementing synchronous broadcasting between multiple terminals according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a broadcasting system for implementing synchronous broadcasting between multiple terminals according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 3, a broadcasting method for implementing synchronous broadcasting between multiple terminals according to an embodiment of the present invention includes the following steps:
s1, marking a time stamp by a source end content, wherein the time stamp is used for ensuring that after the terminal receives data, the reference time stamp of the audio and video data is accurately restored and obtained in a direct or conversion mode for each frame of audio and video data;
s2, connecting the synchronous server, judging whether the connection is successful or not, and if the connection is not successful, prompting an error; if the connection is successful, playing the terminal; the master terminal and the slave terminal can perform synchronous play adjustment by connecting the synchronous server.
S3, a main terminal playing step, after the main terminal starts playing, reporting the current playing progress and state information to the synchronous service;
s4, when the slave terminal starts to play, the slave terminal firstly pulls the current playing progress and state information of the master terminal from the synchronous service, simultaneously pulls the media stream, and then starts to play according to the data offset of the target setting starting playing of the synchronous playing with the master terminal.
In S1, the reference timestamp will now serve as a common reference between different terminals. The generated content may be distributed through a conventional content distribution channel, such as a CDN. For the setting of the reference timestamp, different ways may be chosen.
In an embodiment of the present invention, the timestamp is set in the following manner: in the packaging link, each frame of audio and video data is respectively attached with an original reference time stamp T in the media packaging format, and the reference time stamps simultaneously meet the condition of continuously increasing according to the broadcasting sequence while meeting the packaging format specification. If the encapsulation format cannot attach a reference timestamp for each frame, some optimization approach may be used.
In another embodiment of the present invention, the reference timestamp is set in the following manner: the reference time stamp T is periodically marked in the code stream, and the terminal can calculate and restore to obtain the reference time stamp corresponding to a certain frame by using the time stamp (for distinguishing the original reference time stamp, the reference time stamp is denoted by P hereinafter) defined by the encapsulation format. For example, a sequence of video frames F 0 ，F 1 ，F 2 ，F 3 ，F 4 ，F 5 F is known to 0 The time stamp is P 0 And its corresponding reference timestamp is T 0 Then video frame F 5 (time stamp is P 5 ) Corresponding reference timestamp T 5 ＝T 0 +(P 5 -P 0 )。
In S3 and S4, the playing progress includes the marking information (reference timestamp) corresponding to the audio and video frame currently played in the encapsulation format, the clock of the current terminal device, and the like. The play status information includes events that have a significant impact on the play progress, such as play, buffer, pause, etc.
In the embodiment of the invention, the terminal can select to send the playing state at a certain time interval so as to reduce the data transmission quantity with the server. In other embodiments of the present invention, the data may be selected to be sent only when the playing status is changed (e.g., the playing is paused, the playing speed is changed, the playing resumes to normal, etc.), so as to further reduce the data amount.
In the embodiment of the present invention, S4, in the process of performing synchronization judgment, since each step of video processing needs to consume a certain amount of time and the consumed time is different on different terminal devices, the terminalFirstly, performing a performance test on time consumption of each module in the self audio/video processing flow to obtain accurate time consumption data of each module. According to the data, for the data frames with any given time stamp, the overall time consumption of processing and rendering can be calculated, so that a direct basis is provided for accurate fixed-point playing; each terminal should also acquire time consuming interactions with the synchronization service; based on these factors, a corresponding advance processing is added when positioning is performed. Referring to fig. 4, the specific implementation procedure is as follows: the main terminal reports the state to the synchronous server, which comprises the following steps: (1) play position P0, local clock T a (0) And a play state S0; (2) the synchronous server records the following information to a database, including a main playing end, a local clock T of the synchronous server when receiving the message b (0) Master terminal clock T a (0) Master terminal play state S0, master terminal delay RTT0; (3) estimating the playing delay delta 1 of the local machine from the terminal; (4) the slave terminal requests the state of the master terminal and records the current local clock as T c (0) The method comprises the steps of carrying out a first treatment on the surface of the (5) The synchronization server records the time T of receiving the request b (1) The method comprises the steps of carrying out a first treatment on the surface of the (6) The synchronous server reads the data from the database and records the time T of sending the request b (2) The method comprises the steps of carrying out a first treatment on the surface of the (7) Recording the time Tc (1) of receiving the reply from the terminal, and calculating the time delay RTTt1= ((T) c (1)-Tc(0))-(T b (2)-T b (1) A) and/or 2, and calculates the time point p1=p0+rtt0+ (T) at which the slave terminal should request to play b (2)-T b (0) +rtt1, play delay P1' =p1- δ1; (8) p1' and T c (1) And respectively serving as a starting point of a stream time stamp and a starting point of a local clock to check and judge whether the progress is correct in the subsequent playing process.
Referring to fig. 5, a broadcasting system for implementing synchronous broadcasting between multiple terminals according to an embodiment of the present invention includes a content making end, a streaming media server, a synchronization server, a master terminal and at least one slave terminal, where the content making end is used to make a source reference timestamp mark, and ensure that after receiving data, the terminal accurately restores each frame of audio and video data directly or through conversion to obtain a reference timestamp of the audio and video data; the streaming media server is used for transmitting the content from the content making end to the terminal; the synchronous server is used for connecting with the master terminal and the slave terminal, and playing the terminal after the connection is successful; the main terminal is used for reporting the current playing progress and state information to the synchronous service after playing is started; when the slave terminal starts playing, the slave synchronous server firstly pulls the current playing progress and state information of the corresponding master terminal, simultaneously pulls the media stream, and then starts playing according to the data offset of the target setting of playing synchronously with the master terminal.
In an embodiment of the present invention, the reference timestamp is set in the following manner: in the packaging link, each frame of audio and video data is respectively attached with an original reference time stamp T in the media packaging format, and the reference time stamps simultaneously meet the condition of continuously increasing according to the broadcasting sequence while meeting the packaging format specification. If the encapsulation format cannot attach a reference timestamp for each frame, some optimization approach may be used.
In another embodiment of the present invention, the reference timestamp is set in the following manner: the reference time stamp T is periodically marked in the code stream, and the terminal can calculate and restore to obtain the reference time stamp corresponding to a certain frame by using the time stamp (for distinguishing the original reference time stamp, the reference time stamp is denoted by P hereinafter) defined by the encapsulation format. For example, a sequence of video frames F 0 ，F 1 ，F 2 ，F 3 ，F 4 ，F 5 F is known to 0 The time stamp is P 0 And its corresponding reference timestamp is T 0 Then video frame F 5 (time stamp is P 5 ) Corresponding reference timestamp T 5 ＝T 0 +(P 5 -P 0 )。
In the master terminal and the slave terminal, the playing progress includes the marking information (reference timestamp) corresponding to the audio and video frame currently played in the encapsulation format, the clock of the current terminal device, and the like. The play status information includes events that have a significant impact on the play progress, such as play, buffer, pause, etc.
In the embodiment of the invention, the terminal can select to send the playing state at a certain time interval so as to reduce the data transmission quantity with the server. In other embodiments of the present invention, the data may be selected to be sent only when the playing status is changed (e.g., the playing is paused, the playing speed is changed, the playing resumes to normal, etc.), so as to further reduce the data amount.
In the embodiment of the invention, in the process of synchronous judgment from the terminal, each step of video processing needs to consume a certain time and the consumed time is different on different terminal equipment, so that the terminal needs to perform a performance test on the consumed time of each module in the self audio/video processing flow at first so as to acquire accurate consumed time data of each module. According to the data, for the data frames with any given time stamp, the overall time consumption of processing and rendering can be calculated, so that a direct basis is provided for accurate fixed-point playing; each terminal should also acquire time consuming interactions with the synchronization service; based on these factors, a corresponding advance processing is added when positioning is performed. Referring to fig. 4, the specific implementation procedure is as follows: the main terminal reports the state to the synchronous server, which comprises the following steps: (1) play position P0, local clock T a (0) And a play state S0; (2) the synchronous server records the following information to a database, including a main playing end, a local clock T of the synchronous server when receiving the message b (0) Master terminal clock T a (0) Master terminal play state S0, master terminal delay RTT0; (3) estimating the playing delay delta 1 of the local machine from the terminal; (4) the slave terminal requests the state of the master terminal and records the current local clock as T c (0) The method comprises the steps of carrying out a first treatment on the surface of the (5) The synchronization server records the time T of receiving the request b (1) The method comprises the steps of carrying out a first treatment on the surface of the (6) The synchronous server reads the data from the database and records the time T of sending the request b (2) The method comprises the steps of carrying out a first treatment on the surface of the (7) Recording the time Tc (1) of receiving the reply from the terminal, and calculating the time delay RTTt1= ((T) c (1)-Tc(0))-(T b (2)-T b (1) A) and/or 2, and calculates the time point p1=p0+rtt0+ (T) at which the slave terminal should request to play b (2)-T b (0) +rtt1, play delay P1' =p1- δ1; (8) p1' and T c (1) And respectively serving as a starting point of a stream time stamp and a starting point of a local clock to check and judge whether the progress is correct in the subsequent playing process.
Application example 1: start playing successively
The terminal A starts to play the program source (namely, A is used as a main terminal) and starts to report the playing progress (comprising a timestamp corresponding to the audio and video frame currently played in the encapsulation format, a system clock of the current terminal, the playing state of the current terminal and the like) to the synchronous service at fixed time;
terminal B (B is a slave terminal) selects to play the same program source and requires to synchronize with A;
b obtains the playing progress of A from the synchronous server, and calculates the playing start point according to the progress information of A to play
And continuously receiving the playing progress from the A in the playing process of the B, and ensuring the synchronization with the audio and video of the A.
Application example 2: synchronization during playback
Two terminals (A, B) independently start playing the same live program source and start reporting the playing progress (comprising a timestamp corresponding to a currently played audio and video frame in a packaging format, a system clock of the current terminal, a playing state of the current terminal and the like) to a synchronous server at fixed time;
wherein the B terminal (slave terminal) is set halfway to follow the a terminal (master terminal);
b obtains the playing progress of A from the synchronous server, and adjusts the current playing progress according to the progress information of A so as to ensure the synchronization with the audio and video of A;
and continuously receiving the playing progress from the A in the playing process of the B, and ensuring the synchronization with the audio and video of the A.
Application example 3: during the playing process, interruption or other forms of change occur in the playing of A
Two terminals (A is a master terminal and B is a slave terminal) synchronously play the same program source, and start to report the playing progress (comprising the timestamp corresponding to the audio and video frame played currently in the encapsulation format; the system clock of the current terminal; the playing state of the current terminal, etc.) to the synchronous service at fixed time;
the playing is paused on the terminal A, and the A reports the playing state to the synchronous server as paused;
the synchronization server informs the terminal B that B is also immediately paused;
the playing is restored on the terminal A, and the playing state is reported to the synchronous server by the terminal B;
and the synchronous server informs the terminal B, and the B starts playing from the appointed position according to the playing progress of the current A.
Application example 4: multiple slave terminals
The terminal A, B independently starts playing the same live program source and starts reporting the playing progress (comprising a timestamp corresponding to the currently played audio and video frame in the encapsulation format, a system clock of the current terminal, the playing state of the current terminal and the like) to the synchronous server at fixed time;
the terminal B is set to follow the slave terminal of the terminal A midway, the playing progress of the terminal A is obtained from the synchronous server, and the current playing progress is continuously adjusted according to the progress information of the terminal A so as to ensure the synchronization with the audio and video of the terminal A;
the terminal C is set to follow the terminal A, acquires the playing progress of the terminal A from the synchronous server, and sets the playing starting position according to the progress information of the terminal A so as to ensure the synchronization with the audio and video of the terminal A; and continuously maintaining synchronous operation in the playing process.
It should be understood that the exemplary embodiments described herein are illustrative and not limiting. Although one or more embodiments of the present invention have been described with reference to the accompanying drawings, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims.
Claims (10)
1. A broadcasting method for realizing synchronous broadcasting among multiple terminals is characterized by comprising the following steps:
s1, marking a reference time stamp by the source end content, wherein the reference time stamp is used for ensuring that the reference time stamp of the audio and video data is accurately restored and obtained in a direct or conversion mode for each frame of audio and video data after the terminal receives the data;
s2, connecting the synchronous server, judging whether the connection is successful or not, and if the connection is not successful, prompting an error; if the connection is successful, playing the terminal;
s3, a main terminal playing step, after the main terminal starts playing, reporting the current playing progress and state information to the synchronous service;
s4, when the slave terminal starts to play, the slave terminal firstly pulls the current playing progress and state information of the master terminal from the synchronous service, simultaneously pulls the media stream, and then starts to play according to the data offset of the target setting starting playing of the synchronous playing with the master terminal;
and S3, reporting state information to a synchronous server by the main terminal, wherein the step comprises the following steps of: (1) play position P0, local clock T a (0) And a play state S0; (2) the synchronous server records the following information to a database, including a main playing end, a local clock T of the synchronous server when receiving the message b (0) Master terminal clock T a (0) Master terminal play state S0, master terminal delay RTT0; (3) estimating the playing delay delta 1 of the local machine from the terminal; (4) the slave terminal requests the state of the master terminal and records the current local clock as T c (0) The method comprises the steps of carrying out a first treatment on the surface of the (5) The synchronization server records the time T of receiving the request b (1) The method comprises the steps of carrying out a first treatment on the surface of the (6) The synchronous server reads the data from the database and records the time T of sending the request b (2) The method comprises the steps of carrying out a first treatment on the surface of the (7) Recording the time Tc (1) of receiving the reply from the terminal, and calculating the time delay RTTt1= ((T) c (1)-Tc(0))-(T b (2)-T b (1) A) and/or 2, and calculates the time point p1=p0+rtt0+ (T) at which the slave terminal should request to play b (2)-T b (0) +rtt1, play delay P1' =p1- δ1; (8) p1' and T c (1) And respectively serving as a starting point of a stream time stamp and a starting point of a local clock to check and judge whether the progress is correct in the subsequent playing process.
2. The broadcasting method for realizing synchronous broadcasting among multiple terminals as claimed in claim 1, wherein in S1, in the encapsulation procedure, each frame of audio/video data is respectively attached with its original reference time stamp in the media encapsulation format, and the reference time stamps satisfy continuous increasing in broadcasting order.
3. The broadcasting method for realizing synchronous broadcasting among multiple terminals as claimed in claim 1, wherein in S1, reference time stamps are periodically stamped in the code stream, and the terminal calculates and restores to obtain a reference time stamp corresponding to a certain frame by using the time stamp defined by the encapsulation format itself.
4. A method of playing back a plurality of terminals synchronously according to any one of claims 1 to 3, wherein in S3 the master terminal transmits status information at regular intervals.
5. A method of playing back a plurality of terminals synchronously according to any one of claims 1 to 3, wherein in S3, the master terminal transmits status information when the status is changed.
6. A broadcasting system for realizing synchronous broadcasting among multiple terminals is characterized by comprising a content making end, a synchronous server, a master terminal and at least one slave terminal,
the content making end is used for marking the reference time stamp of the source end content, so that the terminal can accurately restore each frame of audio and video data in a direct or conversion mode after receiving the data to obtain the reference time stamp of the audio and video data;
the synchronous server is used for connecting with the master terminal and the slave terminal, and playing the terminal after the connection is successful;
the main terminal is used for reporting the current playing progress and state information to the synchronous service after playing is started;
when the slave terminal starts playing, the slave synchronous server firstly pulls the current playing progress and state information of the corresponding master terminal, simultaneously pulls the media stream, and then starts playing according to the data offset of the target setting of playing synchronously with the master terminal;
the main terminal reports state information to the synchronous server, which comprises the following steps: (1) play position P0, local clock T a (0) And a play state S0; (2) the synchronous server records the following information to a database, including a main playing end, when the synchronous server is local when receiving the messageClock T b (0) Master terminal clock T a (0) Master terminal play state S0, master terminal delay RTT0; (3) estimating the playing delay delta 1 of the local machine from the terminal; (4) the slave terminal requests the state of the master terminal and records the current local clock as T c (0) The method comprises the steps of carrying out a first treatment on the surface of the (5) The synchronization server records the time T of receiving the request b (1) The method comprises the steps of carrying out a first treatment on the surface of the (6) The synchronous server reads the data from the database and records the time T of sending the request b (2) The method comprises the steps of carrying out a first treatment on the surface of the (7) Recording the time Tc (1) of receiving the reply from the terminal, and calculating the time delay RTTt1= ((T) c (1)-Tc(0))-(T b (2)-T b (1) A) and/or 2, and calculates the time point p1=p0+rtt0+ (T) at which the slave terminal should request to play b (2)-T b (0) +rtt1, play delay P1' =p1- δ1; (8) p1' and T c (1) And respectively serving as a starting point of a stream time stamp and a starting point of a local clock to check and judge whether the progress is correct in the subsequent playing process.
7. The system for implementing synchronous playing among multiple terminals as claimed in claim 6, wherein in the content creation terminal, in the encapsulation step, each frame of audio/video data is respectively attached with its original reference time stamp in the media encapsulation format, and the reference time stamps are continuously incremented in the playing order.
8. The broadcasting system for realizing synchronous broadcasting among multiple terminals as claimed in claim 6, wherein in the content making terminal, reference time stamps are periodically stamped in the code stream, and the terminal calculates and restores to obtain the reference time stamp corresponding to a certain frame by using the time stamp defined by the encapsulation format itself.
9. A playout system for enabling synchronized playback between multiple terminals as claimed in any one of claims 6 to 8, wherein the master terminal transmits status information at regular intervals.
10. A playout system for enabling synchronized playback between multiple terminals as claimed in any one of claims 6 to 8, wherein the master terminal transmits status information when a status change occurs.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111156509.1A CN114143584B (en) | 2021-09-29 | 2021-09-29 | Broadcasting system and method for realizing synchronous broadcasting among multiple terminals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111156509.1A CN114143584B (en) | 2021-09-29 | 2021-09-29 | Broadcasting system and method for realizing synchronous broadcasting among multiple terminals |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114143584A CN114143584A (en) | 2022-03-04 |
| CN114143584B true CN114143584B (en) | 2024-03-26 |
Family
ID=80394079
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111156509.1A Active CN114143584B (en) | 2021-09-29 | 2021-09-29 | Broadcasting system and method for realizing synchronous broadcasting among multiple terminals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114143584B (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101207809A (en) * | 2006-12-19 | 2008-06-25 | 浙江工业大学 | Method and system for transmitting real time flow media based on video frequency frame splitting |
| CN103209360A (en) * | 2013-03-04 | 2013-07-17 | 北京小米科技有限责任公司 | Multi-party video sharing method and device |
| CN103533388A (en) * | 2013-10-25 | 2014-01-22 | 潘伟洲 | Method and system for synchronously playing multimedia files |
| CN103561356A (en) * | 2013-05-31 | 2014-02-05 | 华为技术有限公司 | Multi-terminal synchronous playing method and device |
| CN103839558A (en) * | 2012-11-23 | 2014-06-04 | 三星电子（中国）研发中心 | Method and server for synchronous playing and player |
| CN104378675A (en) * | 2014-12-08 | 2015-02-25 | 厦门雅迅网络股份有限公司 | Multichannel audio-video synchronized playing processing method |
| CN104602037A (en) * | 2015-02-10 | 2015-05-06 | 福州瑞芯微电子有限公司 | Method, device, terminal and system for controlling synchronous playing of multiple devices |
| CN104683823A (en) * | 2013-11-29 | 2015-06-03 | 红板凳科技股份有限公司 | Multi-screen linked audio and video synchronizing system |
| WO2015188629A1 (en) * | 2014-06-13 | 2015-12-17 | Tencent Technology (Shenzhen) Company Limited | Method and system for client-server real-time interaction based on streaming media |
| CN106162236A (en) * | 2015-03-23 | 2016-11-23 | 腾讯科技（深圳）有限公司 | A kind of method and device of sharing video frequency |
| CN107682715A (en) * | 2016-08-01 | 2018-02-09 | 腾讯科技（深圳）有限公司 | Video synchronization method and device |
| CN110035311A (en) * | 2019-04-04 | 2019-07-19 | 网宿科技股份有限公司 | A kind of methods, devices and systems that message flow and audio/video flow is played simultaneously |
| CN111541926A (en) * | 2020-05-08 | 2020-08-14 | 浙江大华技术股份有限公司 | Multimedia file playing method and device, storage medium and electronic device |
| CN112383795A (en) * | 2021-01-12 | 2021-02-19 | 北京清奇科技有限公司 | Video synchronous playing system |
| KR20210078985A (en) * | 2019-12-19 | 2021-06-29 | 애드커넥티드 주식회사 | Method of display playback synchronization of digital contents in multiple connected devices and apparatus using the same |
| CN113179441A (en) * | 2021-03-26 | 2021-07-27 | 卡莱特云科技股份有限公司 | Synchronous playing method, device, system and computer equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7424209B2 (en) * | 2003-02-13 | 2008-09-09 | Microsoft Corporation | System and method for real-time data archival |
-
2021
- 2021-09-29 CN CN202111156509.1A patent/CN114143584B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101207809A (en) * | 2006-12-19 | 2008-06-25 | 浙江工业大学 | Method and system for transmitting real time flow media based on video frequency frame splitting |
| CN103839558A (en) * | 2012-11-23 | 2014-06-04 | 三星电子（中国）研发中心 | Method and server for synchronous playing and player |
| CN103209360A (en) * | 2013-03-04 | 2013-07-17 | 北京小米科技有限责任公司 | Multi-party video sharing method and device |
| CN103561356A (en) * | 2013-05-31 | 2014-02-05 | 华为技术有限公司 | Multi-terminal synchronous playing method and device |
| CN103533388A (en) * | 2013-10-25 | 2014-01-22 | 潘伟洲 | Method and system for synchronously playing multimedia files |
| CN104683823A (en) * | 2013-11-29 | 2015-06-03 | 红板凳科技股份有限公司 | Multi-screen linked audio and video synchronizing system |
| WO2015188629A1 (en) * | 2014-06-13 | 2015-12-17 | Tencent Technology (Shenzhen) Company Limited | Method and system for client-server real-time interaction based on streaming media |
| CN104378675A (en) * | 2014-12-08 | 2015-02-25 | 厦门雅迅网络股份有限公司 | Multichannel audio-video synchronized playing processing method |
| CN104602037A (en) * | 2015-02-10 | 2015-05-06 | 福州瑞芯微电子有限公司 | Method, device, terminal and system for controlling synchronous playing of multiple devices |
| CN106162236A (en) * | 2015-03-23 | 2016-11-23 | 腾讯科技（深圳）有限公司 | A kind of method and device of sharing video frequency |
| CN107682715A (en) * | 2016-08-01 | 2018-02-09 | 腾讯科技（深圳）有限公司 | Video synchronization method and device |
| CN110035311A (en) * | 2019-04-04 | 2019-07-19 | 网宿科技股份有限公司 | A kind of methods, devices and systems that message flow and audio/video flow is played simultaneously |
| KR20210078985A (en) * | 2019-12-19 | 2021-06-29 | 애드커넥티드 주식회사 | Method of display playback synchronization of digital contents in multiple connected devices and apparatus using the same |
| CN111541926A (en) * | 2020-05-08 | 2020-08-14 | 浙江大华技术股份有限公司 | Multimedia file playing method and device, storage medium and electronic device |
| CN112383795A (en) * | 2021-01-12 | 2021-02-19 | 北京清奇科技有限公司 | Video synchronous playing system |
| CN113179441A (en) * | 2021-03-26 | 2021-07-27 | 卡莱特云科技股份有限公司 | Synchronous playing method, device, system and computer equipment |
Non-Patent Citations (1)
| Title |
|---|
| 非对称无线传输信道的音视频同步优化设计;马宏雷;刘儿兀;朱军;;计算机仿真(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114143584A (en) | 2022-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11368732B2 (en) | Synchronizing program presentation | |
| CN102857796B (en) | Synchronization and automation in an ITV enviroment | |
| CN101854533B (en) | Frequency channel switching method, device and system | |
| TWI760981B (en) | Dynamic playout of transition frames while transitioning between playout of media streams | |
| CN102752669B (en) | The transfer processing method of multichannel real time flow medium file and system, receiving system | |
| CN101419827B (en) | Method for synchronzing audio and video data in avi file | |
| KR100902013B1 (en) | Tiled-display system and synchronization method in the system | |
| KR100482287B1 (en) | Apparatus and method for injection of synchronized stream data in digital broadcasting environment | |
| CN103155584A (en) | Method for synchronizing multimedia flows and corresponding device | |
| US20220038758A1 (en) | Advanced Preparation for Content Revision Based on Expected Latency in Obtaining New Content | |
| CN113179441A (en) | Synchronous playing method, device, system and computer equipment | |
| CN105656616A (en) | Inter-multi-device data synchronization method and device, transmitting end and receiving end | |
| KR102566550B1 (en) | Method of display playback synchronization of digital contents in multiple connected devices and apparatus using the same | |
| CN106060580A (en) | Synchronous image display method and device | |
| CN114143584B (en) | Broadcasting system and method for realizing synchronous broadcasting among multiple terminals | |
| CN107465952A (en) | The method and system that a kind of broadcasted content synchronously plays with side information | |
| CN100544448C (en) | A kind of clock system of mobile multimedia network | |
| CN101237276B (en) | Mobile phone TV single-frequency network synchronization system and method based on mobile network | |
| JP7162295B2 (en) | Stream distribution system and program with segment file division function | |
| CN100571119C (en) | A kind of mobile network of utilization realizes the method for clock synchronization | |
| CN116347113A (en) | Live broadcast recording method for high-availability television time shifting and review based on HLS protocol | |
| CN101202918A (en) | Method of terminal correcting clock | |
| CN116567308A (en) | Method, device, equipment and storage medium for synchronizing multi-stream network video | |
| CN102891748A (en) | Method and system for sending follow message | |
| TH22722A (en) | Linker, transmit processor, and recording / playback device in a stream with a field structure suitable for conveying television information. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |