CN102202070A - Fast inter-stream synchronization mechanism for streaming media - Google Patents
Fast inter-stream synchronization mechanism for streaming media Download PDFInfo
- Publication number
- CN102202070A CN102202070A CN2010101297869A CN201010129786A CN102202070A CN 102202070 A CN102202070 A CN 102202070A CN 2010101297869 A CN2010101297869 A CN 2010101297869A CN 201010129786 A CN201010129786 A CN 201010129786A CN 102202070 A CN102202070 A CN 102202070A
- Authority
- CN
- China
- Prior art keywords
- streaming media
- rtp
- timestamp
- media
- rtcp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The multimedia synchronization technology is a key technology in the streaming media technology. In general design of a streaming media client, the establishment of synchronous information depends on an RTCP (realtime transport control protocol) packet typically borne on the UDP (user datagram protocol), thereby bringing unpredictability to the first playback time of a media object. In order to shorten the waiting time before playback, the invention provides a mechanism for quickly determining the synchronization relationship between media streams based on the RTP (realtime transport protocol)/RTCP and RTSP (real time streaming protocol). The mechanism provided by the invention not only can reduce the complexity of inter-stream synchronization of the streaming media client, but also can accelerate the first playback time of the media object so that a user can watch the program content as soon as possible and good user experience can be provided.
Description
Technical field
The invention belongs to the computer media simultaneous techniques.
Background technology
The multimedia simultaneous techniques is the technology of a key in the stream media technology, its objective is when the user shows multimedia messages, keeps in the multimedia object and intrinsic time-domain constraints relation between object.Multimedia comprises two homochronousness synchronously: a class is the stream inter-sync, and its main task is the relational tense relation that guarantees Media Stream inside, just transmits each multimedia object by the regular hour requirement, and can carry out continuous representing in client.Another kind of is synchronous between stream, and main task is the time relationship of safeguarding between media object, as time relationship between the time relationship between the Voice ﹠ Video (being that lip is synchronous), audio frequency and video and the literal or the like.
The RTP/RTCP agreement is important part in the stream media protocol stack, it has born the transmission task of media data, the RTSP agreement then is responsible for effective transmission of control media data in the stream media protocol stack, RTSP and RTP/RTCP agreement are the cores of whole stream media protocol stack.
Summary of the invention
The present invention is primarily aimed at the IP network environment, and based on RTP/RTCP agreement and RTSP agreement, utilize the mapping relations of RTP timestamp and NTP (Network Time Protocol) timestamp in RTP timestamp, the RTCP bag and in conjunction with the Range header field in the PLAY method response message of RTSP and the mapping relations of RTP-Info header field, determine the synchronized relation between Media Stream apace, can also eliminate simultaneously the synchronous dependence between each Media Stream, make that the number of synchronous complexity and Media Stream has nothing to do between stream.This scheme following several advantage arranged:
1) the Streaming Media client just can be set up the corresponding relation of RTP timestamp and NTP timestamp when session begins, thereby can shift to an earlier date the playback zero-time of media object.
2) has anti-packet loss ability.Because the RTSP control channel that passes through of the synchronized relation between medium is set up,, can not have influence on first playback duration of medium in the RTP session even therefore losing of RTCP SR bag arranged yet.
3) when having many Media Streams to exist in the session, the synchronous no dependence between each Media Stream, thus make that the number of synchronous complexity and Media Stream has nothing to do between stream.
4) because the initial value of the local virtual NTP timestamp that this scheme is mapped to the RTP timestamp is 0, therefore, this virtual NTP timestamp can be directly as the absolute demonstration timestamp of this Media Stream, thereby reduced Streaming Media client amount of calculation.
Embodiment
The Streaming Media client is after the response message that receives the PLAY method, the rtptime parameter extraction of each stream in the RTP-Info header field is come out, and 0 corresponding with NTP timestamp form respectively, the RTP timestamp that is about to first bag of each Media Stream is mapped on the virtual NTP timestamp 0, in RTP conversation procedure next, first RTCP SR bag that utilization is received calculates the actual NTP timestamp and the difference diff of NTP timestamp 0, and the NTP timestamp that wraps of RTCP SR subsequently all needs to deduct earlier this diff and sets up mapping relations with the RTP timestamp again.That is to say when session begins, the Streaming Media client at first utilizes the rtptime parameter in the RTP-Info header field to set up corresponding relation between RTP timestamp and the local virtual NTP timestamp, utilize first RTCP SR bag of receiving subsequently then, set up the mapping relations of the actual NTP timestamp of server end and this virtual NTP timestamp, thereby set up the mapping relations of the actual NTP timestamp of RTP timestamp and server end indirectly, follow-up RTCP SR bag then is used to calibrate this mapping relations, as shown in Figure 2, provided the synchronizing process of two Media Streams among the figure, the situation of many streams similarly.
Narration Streaming Media client is implemented various VCR controls back to Media Stream and is realized synchronous detailed step between stream below.
1, sends RTSP PLAY request for the first time
After the Streaming Media client utilizes the methods such as DESCRIBE, SETUP of RTSP agreement to finish session negotiation, can send RTSP PLAY method request streaming media server to streaming media server and begin to send media data, synchronous in order to realize between stream, receive the success response (being that conditional code is 2xx) of this PLAY method in the Streaming Media client after, should carry out following operation in proper order:
(1) extracts the rtptime parameter that respectively flows in the RTP-Info header field of this PLAY response;
(2) the rtptime parameter value with each stream corresponds on the NTP timestamp 0;
Calculate the actual NTP timestamp and the difference diff of NTP timestamp 0 when (3) receiving each first RTCP SR bag that flows;
(4) receive that when the individual RTCP SR of each n (n＞1) that flows wrapped, the NTP timestamp of the reality of this SR bag deducted diff earlier, set up corresponding relation with the RTP timestamp of this SR bag then;
If synchronous problem between not considering to flow, Streaming Media client are after executing the operation of the 2nd step, just can begin the decoding and the playback of media object, the mapping relations that the 3rd step and the operation of the 4th step then are used to calibrate RTP timestamp and NTP timestamp.
2, send the RTSP PLAY request that carries the Range header field in the playing process
In the playing programs process, if play when the Streaming Media client-requested is selected, then can send the RTSP PLAY request that carries the Range header field to streaming media server, the time range of the media data that the request of wherein carrying in the Range header field sends, after in order to realize selecting between the stream of media data synchronously, receive the success response of this PLAY method in the Streaming Media client after, should carry out following operation in proper order:
(1) empties each socket buffering area that flows in the RTP session;
(2) extract rtptime parameter and the seq parameter that respectively flows in the RTP-Info header field of this PLAY response;
(3) the rtptime parameter value with each stream corresponds on the NTP timestamp 0;
Calculate the actual NTP timestamp and the difference diff of NTP timestamp 0 when (4) receiving each first RTCP SR bag that flows;
When (5) receiving each second of flowing and the bag of RTCP SR subsequently, the NTP timestamp of the reality of this SR bag deducts diff earlier, sets up corresponding relation with the RTP timestamp of this SR bag then;
Wherein the purpose of the 1st step operation be discarded in play when selecting before the Streaming Media client received but still untreated RTP bag and RTCP wrap, RTP bag before and after seq parameter of each stream that is extracted in the 2nd step operation then is used for distinguishing when selecting (because the RTP bag before when having part and being trapped in selecting of network, receive after this PLAY responds just arrive the Streaming Media client).If synchronous problem between not considering to flow, Streaming Media client are after carrying out the operation of the 3rd step, just can begin the decoding and the playback of media object, the effect of the 4th step and the operation of the 5th step is with 1 trifle.
3, RTSP PAUSE request back sends the RTSP PLAY request of no Range header field
The Streaming Media client is after having sent the time-out request, recover to play if wish, then can send the RTSP PLAY request of a no Range header field, under the normal condition, the streaming media server end can be replied the success response of this PLAY request, and wherein can carry a Range header field, be used to indicate the reproduction time scope of Media Stream subsequently.In this case,, and realize suspending the synchronous of back media data, should carry out following operation in proper order for media client before not influencing time-out has received but the playback of untreated media data still:
(1) extracts the rtptime parameter that respectively flows in the RTP-Info header field of this PLAY response;
(2) extract NPT initial time stamp in the Range header field of this PLAY response, and be converted into NTP timestamp form, be designated as ntp_start;
(3) the rtptime parameter value with each stream corresponds on the NTP timestamp ntp_start;
Calculate the actual NTP timestamp and the difference diff of NTP timestamp 0 when (4) receiving each first RTCP SR bag that flows;
When (5) receiving each second of flowing and the bag of RTCP SR subsequently, the NTP timestamp of the reality of this SR bag deducts diff earlier, sets up corresponding relation with the RTP timestamp of this SR bag then;
If synchronous problem between not considering to flow, Streaming Media client are after executing the operation of the 3rd step, just can begin the decoding and the playback of media data, the effect of the 4th step and the operation of the 5th step is with 1 trifle.
4, RTSP PAUSE request back sends the RTSP PLAY request that carries the Range header field
The Streaming Media client is after having sent the time-out request, if play when next wishing to select, then need to send a RTSP PLAY request that carries the Range header field, wherein carry time range to be played in the Range header field, under the normal condition, the streaming media server end can be replied the success response of this PLAY request, at this moment, after in order to realize selecting between the stream of media data synchronously, performed operation should with 2 trifles in identical, this no longer superfluous chatting.
Claims (1)
1. when the RTSP flow media session begins, the Streaming Media client at first utilizes the rtptime parameter in the RTP-Info header field to set up corresponding relation between RTP timestamp and the local virtual NTP timestamp, utilize first RTCP SR bag of receiving subsequently then, set up the mapping relations of the actual NTP timestamp of server end and this virtual NTP timestamp, thereby set up the mapping relations of the actual NTP timestamp of RTP timestamp and server end indirectly, follow-up RTCP SR bag then is used to calibrate this mapping relations.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101297869A CN102202070A (en) | 2010-03-23 | 2010-03-23 | Fast inter-stream synchronization mechanism for streaming media |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101297869A CN102202070A (en) | 2010-03-23 | 2010-03-23 | Fast inter-stream synchronization mechanism for streaming media |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102202070A true CN102202070A (en) | 2011-09-28 |
Family
ID=44662465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101297869A Pending CN102202070A (en) | 2010-03-23 | 2010-03-23 | Fast inter-stream synchronization mechanism for streaming media |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102202070A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021159329A1 (en) * | 2020-02-12 | 2021-08-19 | 深圳元戎启行科技有限公司 | Streaming media network latency determination method and apparatus, computer device, readable storage medium, and remote driving system |
| CN115174979A (en) * | 2022-06-20 | 2022-10-11 | 阿里巴巴（中国）有限公司 | Streaming media transmission network, transmission control method, device, equipment and storage medium |
-
2010
- 2010-03-23 CN CN2010101297869A patent/CN102202070A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021159329A1 (en) * | 2020-02-12 | 2021-08-19 | 深圳元戎启行科技有限公司 | Streaming media network latency determination method and apparatus, computer device, readable storage medium, and remote driving system |
| CN115174979A (en) * | 2022-06-20 | 2022-10-11 | 阿里巴巴（中国）有限公司 | Streaming media transmission network, transmission control method, device, equipment and storage medium |
| CN115174979B (en) * | 2022-06-20 | 2023-12-29 | 阿里巴巴（中国）有限公司 | Streaming media transmission network, transmission control method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6783293B2 (en) | Synchronizing multiple over-the-top streaming clients | |
| US10034037B2 (en) | Fingerprint-based inter-destination media synchronization | |
| EP2752023B1 (en) | Method to match input and output timestamps in a video encoder and advertisement inserter | |
| CN103733631B (en) | Media content transceiving method and transceiving apparatus using same | |
| Howson et al. | Second screen TV synchronization | |
| CN103607664B (en) | A kind of audio and video synchronization method of embedded multimedia playing system | |
| CN102421035A (en) | Method and device for synchronizing audio and video of digital television | |
| KR100482287B1 (en) | Apparatus and method for injection of synchronized stream data in digital broadcasting environment | |
| CN101179484A (en) | Method and system of synchronizing different media stream | |
| CN103338386A (en) | Audio and video synchronization method based on simplified timestamps | |
| JP2001320413A (en) | Data transmitter and method | |
| CN103546662A (en) | Audio and video synchronizing method in network monitoring system | |
| US10503460B2 (en) | Method for synchronizing an alternative audio stream | |
| CN101902649A (en) | Audio-video synchronization control method based on H.264 standard | |
| CN112929713B (en) | Data synchronization method, device, terminal and storage medium | |
| JP6197211B2 (en) | Audiovisual distribution system, audiovisual distribution method, and program | |
| CN102202070A (en) | Fast inter-stream synchronization mechanism for streaming media | |
| CN103596033A (en) | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback | |
| US20170048291A1 (en) | Synchronising playing of streaming content on plural streaming clients | |
| CN100562107C (en) | The constructive method of RTP blender | |
| JP2008245061A (en) | Pcr reproduction system in ip stream transmission | |
| CN102752668A (en) | Synchronous control device for network video server | |
| CN101938633A (en) | Interactive set top box (STB) based implementation method of embedded streaming media play module | |
| KR20230154051A (en) | Method for providing time-synchronized multi-stream data transmission | |
| WO2012066832A1 (en) | Ip receiver device, ip transmitter device, and ts receiver device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| DD01 | Delivery of document by public notice |
Addressee: Feng Siya Document name: Notification of Publication of the Application for Invention |
|
| DD01 | Delivery of document by public notice |
Addressee: Feng Siya Document name: Notification of before Expiration of Request of Examination as to Substance |
|
| DD01 | Delivery of document by public notice |
Addressee: Feng Siya Document name: Notification that Application Deemed to be Withdrawn |
|
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20110928 |