CN114422810B - Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director - Google Patents
Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director Download PDFInfo
- Publication number
- CN114422810B CN114422810B CN202111330933.3A CN202111330933A CN114422810B CN 114422810 B CN114422810 B CN 114422810B CN 202111330933 A CN202111330933 A CN 202111330933A CN 114422810 B CN114422810 B CN 114422810B
- Authority
- CN
- China
- Prior art keywords
- time
- live broadcast
- director
- machine position
- mapping relation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
- H04N21/26241—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the time of distribution, e.g. the best time of the day for inserting an advertisement or airing a children program
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Business, Economics & Management (AREA)
- Marketing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention relates to a multi-channel live broadcast synchronous calibration method based on a mobile terminal broadcasting guide table, which comprises the steps that the mobile terminal broadcasting guide table creates a broadcasting guide example, creates a plurality of mobile terminal camera positions, generates machine position serial codes and forms two-dimensional codes; the director establishes a long connection with the long connection service; the live broadcast client adds the director through the two-dimensional code generated by scanning, establishes long connection between the machine position and a long connection server, and establishes a time mapping relation between the machine position and the director according to example information; and each machine position respectively sends the established mapping relation to a streaming media server, the live broadcast client performs stream pushing operation on the streaming media server, and the streaming media server performs decoding, timestamp calibration and synchronous operation after receiving a machine position push message and performs recoding. The method and the device furthest reduce the influence of the network environment on the multi-channel live broadcast synchronization problem, and control the multi-channel live broadcast stream synchronization problem at millisecond level so as to meet the live broadcast synchronization requirement under the environment with poor network conditions.
Description
Technical Field
The invention relates to the technical field of IPTV, in particular to a multichannel live broadcast synchronous calibration method based on a mobile terminal director.
Background
With the rapid development of the network informatization process, short videos and live broadcasts are gradually merged into the daily life of people, and the production and life modes of people are changed silently. Meanwhile, the development of segment industries in various fields is promoted, such as: live tape, news numbers, etc. Video has become the wind gap for mobile internet. In recent years, new crown disasters are abused, and on-line teaching, on-line inquiry and on-line meeting become the mainstream forms of teaching, life and office under the background of normalized epidemic prevention and control. This puts higher demands on the live video processing technology.
On-line teaching, on-line meeting, on-line operation and the like are all the applications of the live broadcast form of multiple machine positions and multiple terminals in each scene, and the requirements on the real-time performance and the synchronism among all paths of video streams are very high. Especially for remote operation, real-time and synchronous operation is required to be controlled in millisecond level. Multiple cameras are bound to the problem of synchronization of multiple paths of live broadcast videos, the playing of the traditional multiple paths of live broadcast videos is not processed by a synchronization related mechanism, so that the synchronization of the multiple paths of live broadcast completely depends on a network environment, and due to the fact that the network may have various problems such as delay, jitter, congestion and packet loss, the abnormal terminal causes synchronization problems of different degrees. In this case, if the synchronization calibration process is not performed, the user experience is very poor, and a serious adverse effect may be caused.
Disclosure of Invention
In order to overcome the defects, the invention aims to provide a method for multi-channel live broadcast synchronous calibration based on a mobile terminal director, provides a multi-channel live broadcast synchronous calibration idea, and aims to solve the problem of multi-channel live broadcast asynchronism in various network environments.
The technical scheme adopted by the invention for solving the technical problems is as follows: a method for multi-channel live broadcast synchronous calibration based on a mobile terminal broadcast director comprises the following core design steps of establishing a time mapping relation between a live broadcast client and the broadcast director, synchronously calibrating a live broadcast stream frame timestamp according to the mapping relation, then synchronizing and converging according to a multi-channel live broadcast stream synchronization strategy, reducing the influence of a network environment on multi-channel live broadcast synchronization problems to the maximum extent, controlling the multi-channel live broadcast asynchronous problems within a certain range, and comprising the following specific steps:
firstly, a mobile terminal director platform creates a director example, completes registration on a streaming media server, creates a multi-station unique serial code which comprises example information, and generates a two-dimensional code;
secondly, the director and the long connection server establish long connection;
thirdly, scanning codes by the live broadcast client to join in the director, establishing long connection between the live broadcast client and the long connection server, and analyzing the two-dimension codes to obtain instance information;
fourthly, the live broadcast client acquires the current time stamp TS clt1 The same asAccording to the example information, a request is sent to the broadcasting station through long connection, and the time stamp TS of the current broadcasting station is obtained dtr ；
Fifthly, the live client acquires the time TS of the director dtr Then, local time TS is obtained again clt2 ；
Sixthly, acquiring time TS twice according to the live broadcast client clt1 、TS clt2 Calculating RTT (round trip time), and establishing a mapping relation between the time of the live broadcast client and the time of the program director, wherein the mapping formula is as follows:
seventhly, sending a time mapping relation established between the live broadcast client and the director to a streaming media server, wherein the time mapping relation contains a unique identifier generated by the director obtained through analysis, and the streaming media server caches the mapping relation;
eighthly, the live broadcast client performs live broadcast stream pushing operation to the streaming media server, wherein the live broadcast client contains the only identification of the live broadcast client;
ninthly, after receiving the push data of the live broadcast client, the streaming media service acquires the mapping relation between the time of the client and the time of the director from a cache according to the unique identifier of the client, decodes the data of the live broadcast frame, extracts the information of a time stamp TS (timestamp), calculates a new time stamp by taking the time of the director as a standard according to the acquired mapping relation, and rewrites the new time stamp;
and step ten, the streaming media server performs frame data synchronization, confluence and coding operations on the multi-channel messages according to the new timestamp, and pushes the multi-channel messages to a Content Delivery Network (CDN), so as to complete the multi-channel live streaming data synchronization operation.
The invention has the following beneficial effects:
the method and the system establish a time mapping relation between a live broadcast client and a broadcasting guide station, synchronously calibrate live broadcast stream frame timestamps according to the mapping relation, and then realize synchronization and confluence according to a multi-path live broadcast stream synchronization strategy; the influence of the network environment on the multi-channel live broadcast synchronization problem is reduced to the maximum extent, and the multi-channel live broadcast stream synchronization problem is controlled to be millisecond level so as to meet the live broadcast synchronization requirement under the environment with poor network conditions.
Drawings
FIG. 1 is a timing diagram of the process of the present invention.
Fig. 2 is a schematic diagram of the multi-channel live stream synchronous stream merging of the present invention.
Detailed Description
The present invention will now be described in further detail with reference to the accompanying drawings.
A method based on multi-channel live broadcast synchronous calibration of a mobile terminal broadcasting guide station is characterized in that research and development construction are carried out by taking the mobile terminal broadcasting guide station as a basic platform, related functions of broadcasting guide, live broadcast stream pushing and live broadcast watching are integrated into the same application, a plurality of mobile terminals install the application, and the specific implementation mode is as follows:
firstly, a mobile terminal director platform creates a director example, completes registration in a streaming media server, creates a plurality of mobile terminal camera positions, generates a two-dimensional code of each position serial code (unique identification containing director example information), and creates two live broadcast clients, namely a position A and a position B for description;
step two, the director establishes a long connection with a long connection server through a WSS (WebSocket) protocol;
step three, the two machine positions generate two-dimensional codes through scanning in the step one and add the two-dimensional codes into the director, long connection between the machine positions and a long connection server is established, the two-dimensional codes are analyzed, and example information is obtained;
step four, establishing a time mapping relation between the machine position and the broadcasting station according to the example information, comprising the following steps:
4.1, recording the current time TS by the machine position clt1 According to the example information, a request is sent to the broadcasting-directing station through long connection to obtain the current broadcasting-directing station time TS dtr After the time of the director is obtained, the current time TS of the current position is obtained again clt2 ；
4.2 obtaining time TS according to two machine positions clt1 、TS clt2 Calculating RTT (round trip time), and establishing a mapping relation between the machine time and the time of the director, wherein the mapping formula is as followsThe following:
the example of the mapping relation between the RTT calculated by the station A and the time of the broadcasting station is established is as follows:
A. calculating RTT
The local time is respectively acquired before and after the acquisition of the director time: 1629967263000, i.e. TS clt1 =1629967263000,TS clt2 =1629967263600, the time for the director to return to this instance is: 1629967266000, i.e. TS dtr =1629967266000, then:
b. establishing a mapping relation with the time of the director
And b, according to the effective data obtained by the step a and the RTT calculation result, considering the RTT influence, and estimating the machine position time:
the established mapping relationship is:
step five, each machine position respectively sends the established mapping relation to a streaming media server, meanwhile, the string code generated in the step one is included, the streaming media server caches the mapping relation, the caching time is 5 hours, the live broadcast lasts for 5 hours at most, and the examples of the caching relation are as follows:
step six, the two machine positions respectively carry out stream pushing operation on the stream media server, the sender position only identifies, and the live stream data message is pushed;
step seven, after receiving the machine position push message, the streaming media server performs decoding, timestamp calibration and synchronization operations, and the operation method comprises the following steps:
7.1, obtaining the unique serial code of the machine position, obtaining the mapping relation between the time of the local position and the time of the broadcasting station in the data cache according to the unique serial code, and mapping the time stamp TS of the local position clt Mapping director time stamp TS dtr ；
7.2, decoding the direct broadcasting stream frame data, extracting the timestamp TS (timestamp) information, and recalculating the brand-new timestamp TS according to the mapping relation between the time of obtaining the local machine position and the time of the broadcasting station new The calculation method is as follows:
new time stamp TS to be calculated new Rewriting the time stamp of the current frame to realize time stamp calibration operation, wherein the frame data time stamp calibration example carries out the following operation based on the example calculation result in the above steps:
assume that the timestamp TS of one frame is extracted as: 1629967304000, the new timestamp of this frame is:
7.3, performing synchronous operation processing on the multi-channel live broadcast streams, as shown in 2, wherein the live broadcast stream data is composed of uninterrupted frame data queues, processing according to a first-in first-out principle, performing confluence operation on frame data with the same time stamp in each live broadcast data queue in the current director example, the same time stamp is not absolute, judging that the frame data in the time stamp is less than 1/2 frame time length from top to bottom, judging that the frame data of the time stamp are confluent, supposing that the frame rate of the current live broadcast is 20 frames/second, the time length of each frame is 50ms, the 1/2 frame is +/-25 ms, namely performing confluence operation on the frame data in the +/-25 ms of the multi-channel live broadcast queue, and finishing frame data synchronization logic, namely, controlling the time difference in a 50ms level by multi-channel video synchronization operation;
step eight, after the synchronous calibration and confluence of the multi-channel live streaming data are completed in the streaming media server, recoding is carried out, and the streaming is pushed to the CDN for acceleration and the like;
step nine, completing a synchronous processing flow of the multi-path live broadcast streams, and enabling the user side to conduct synchronous viewing experience on the multi-path live broadcast video guided by the mobile side;
step ten, after the director/live broadcast is finished, sending an instruction to delete the time mapping relation between the local machine position and the director station.
The present invention is not limited to the above embodiments, and any structural changes made by the present invention shall be understood as falling within the scope of the present invention, wherein the structural changes are the same as or similar to the technical solutions of the present invention.
The techniques, shapes, and configurations not described in detail in the present invention are all known techniques.
Claims (4)
1. A multi-channel live broadcast synchronous calibration method based on a mobile terminal broadcast director is characterized by comprising the following steps: comprises the steps of (a) preparing a substrate,
the mobile terminal director platform creates a director example, completes registration in a streaming media server, creates a plurality of mobile terminal camera positions, generates each camera position serial code and forms a two-dimensional code;
the method comprises the steps that a long connection is established between a broadcasting director and a long connection server;
the live broadcast client adds the two-dimensional code generated by scanning into a broadcast guide, establishes long connection between a machine position and a long connection server, analyzes the two-dimensional code and acquires instance information;
establishing a time mapping relation between a machine position and a broadcasting guide station according to the example information, wherein the time mapping relation is obtained by a live broadcasting client side clt1 Meanwhile, according to the example information, a request is sent to the broadcasting guide station through long connection, and the time stamp TS of the broadcasting guide station is obtained dtr Live broadcast client side obtains broadcasting guide time TS dtr Thereafter, local time TS is acquired again clt2 Obtaining time TS twice according to live broadcast client clt1 、TS clt2 Calculating the Round Trip Time (RTT), and establishing a mapping relation between the live client time and the director time, wherein the mapping formula is as follows:
wherein TS clt For local bit time, TS dtr Is the director's station time;
each machine position sends the established mapping relation to a streaming media server respectively, and the streaming media server caches the mapping relation, wherein the mapping relation contains the serial codes of each machine position;
the live broadcast client side carries out stream pushing operation on the stream media server, sends the unique identifier of the machine position and pushes a live broadcast stream data message;
after receiving the machine position push message, the streaming media server performs decoding, timestamp calibration and synchronization operations, wherein,
the timestamp calibration mode is that a timestamp is extracted, a new timestamp is calculated according to the mapping relation between the time of the local machine position and the time of the broadcasting station, and rewriting is carried out;
new timestamp TS new The calculation method of (c) is as follows:
the method of the synchronous operation is that live broadcast stream data is composed of uninterrupted frame data queues, the live broadcast stream data is processed according to the principle of first-in first-out, the frame data with the same time stamp in each live broadcast data queue in the present director example is subjected to confluence operation, the same time stamp is not absolute, the frame data in the time period when the time stamp is less than 1/2 frame time is judged to be the frame data of the present time stamp, and the frame data are all subjected to confluence;
after the synchronous calibration and confluence of the multi-channel live streaming data are finished in the streaming media server, recoding is carried out, and the multi-channel live streaming data are pushed to a content distribution network to finish the synchronous operation of the multi-channel live streaming data.
2. The method according to claim 1, wherein the method comprises the following steps: the method for calculating the round trip time RTT comprises the steps of respectively acquiring local Time (TS) before and after acquiring the time of the broadcasting director clt1 、TS clt2 Then:
3. the method according to claim 1, wherein the method comprises the following steps: and after the synchronous operation flow of the multi-channel live broadcast stream data is completed, the user side can perform synchronous viewing experience on the multi-channel live broadcast video guided by the mobile terminal.
4. The method according to claim 1, wherein the method comprises the following steps: and when the direct broadcasting of the live broadcasting client is finished or the direct broadcasting is finished, the direct broadcasting station sends an instruction to delete the time mapping relation between the local station and the direct broadcasting station.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111330933.3A CN114422810B (en) | 2021-11-11 | 2021-11-11 | Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111330933.3A CN114422810B (en) | 2021-11-11 | 2021-11-11 | Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114422810A CN114422810A (en) | 2022-04-29 |
| CN114422810B true CN114422810B (en) | 2022-10-04 |
Family
ID=81265947
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111330933.3A Active CN114422810B (en) | 2021-11-11 | 2021-11-11 | Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114422810B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116866624B (en) * | 2023-06-09 | 2024-03-26 | 北京医百科技有限公司 | Method and system for copying and sharing configuration information of guide table |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018095130A1 (en) * | 2016-11-24 | 2018-05-31 | 广州华多网络科技有限公司 | Method and system for determining duration of live streaming |
| CN112351294A (en) * | 2020-10-27 | 2021-02-09 | 广州赞赏信息科技有限公司 | Method and system for frame synchronization among multiple machine positions of cloud director |
| CN112866733A (en) * | 2021-01-05 | 2021-05-28 | 广东中兴新支点技术有限公司 | Cloud director synchronization system and method of multiple live devices |
| CN113163222A (en) * | 2021-03-31 | 2021-07-23 | 杭州奥点科技股份有限公司 | Video frame synchronization method, system, equipment and readable storage medium |
| CN113453086A (en) * | 2021-03-29 | 2021-09-28 | 北京国际云转播科技有限公司 | Multi-machine-position synchronous cloud broadcasting guiding method, system, storage medium and video encoder |
| CN113542897A (en) * | 2021-05-19 | 2021-10-22 | 广州速启科技有限责任公司 | Audio and video live broadcast method suitable for multi-view live broadcast |
-
2021
- 2021-11-11 CN CN202111330933.3A patent/CN114422810B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018095130A1 (en) * | 2016-11-24 | 2018-05-31 | 广州华多网络科技有限公司 | Method and system for determining duration of live streaming |
| CN112351294A (en) * | 2020-10-27 | 2021-02-09 | 广州赞赏信息科技有限公司 | Method and system for frame synchronization among multiple machine positions of cloud director |
| CN112866733A (en) * | 2021-01-05 | 2021-05-28 | 广东中兴新支点技术有限公司 | Cloud director synchronization system and method of multiple live devices |
| CN113453086A (en) * | 2021-03-29 | 2021-09-28 | 北京国际云转播科技有限公司 | Multi-machine-position synchronous cloud broadcasting guiding method, system, storage medium and video encoder |
| CN113163222A (en) * | 2021-03-31 | 2021-07-23 | 杭州奥点科技股份有限公司 | Video frame synchronization method, system, equipment and readable storage medium |
| CN113542897A (en) * | 2021-05-19 | 2021-10-22 | 广州速启科技有限责任公司 | Audio and video live broadcast method suitable for multi-view live broadcast |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114422810A (en) | 2022-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11758209B2 (en) | Video distribution synchronization | |
| JP6317872B2 (en) | Decoder for synchronizing the rendering of content received over different networks and method therefor | |
| CN100588250C (en) | Method and system for rebuilding free viewpoint of multi-view video streaming | |
| EP2670157B1 (en) | Fingerprint-based inter-destination media synchronization | |
| JP2024519363A (en) | Game data processing method, device, computer device, and computer program | |
| EP3298790A1 (en) | Providing low&high quality streams | |
| US11516518B2 (en) | Live streaming with live video production and commentary | |
| CN101917389A (en) | Network television direct broadcasting system | |
| US10778361B1 (en) | Stream synchronization | |
| CN109361945A (en) | The meeting audiovisual system and its control method of a kind of quick transmission and synchronization | |
| CN112291498B (en) | Audio and video data transmission method and device and storage medium | |
| CN101917613A (en) | Acquiring and coding service system of streaming media | |
| CN110602522B (en) | Multi-path real-time live webRTC stream synthesis method | |
| CN112004102A (en) | Multi-camera picture synchronization method based on IP live stream | |
| CN108494792A (en) | A kind of flash player plays the converting system and its working method of hls video flowings | |
| CN114422810B (en) | Multipath live broadcast synchronous calibration method based on mobile terminal broadcast director | |
| CN113194278A (en) | Conference control method and device and computer readable storage medium | |
| Tang et al. | Audio and video mixing method to enhance WebRTC | |
| CN114885198B (en) | Mixed network-oriented accompanying sound and video collaborative presentation system | |
| US20230328308A1 (en) | Synchronization of multiple content streams | |
| Westerink et al. | A live intranet distance learning system using MPEG-4 over RTP/RTSP | |
| Montagud et al. | Introduction to media synchronization (MediaSync) | |
| US20190191195A1 (en) | A method for transmitting real time based digital video signals in networks | |
| Wang et al. | Pre-synchronization of AVS audio-video based on Real-Time Transport Protocol | |
| CN113766342A (en) | Subtitle synthesis method and related device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |