CN101951506A - System and method for realizing synchronous transmitting and receiving of scalable video coding service - Google Patents
System and method for realizing synchronous transmitting and receiving of scalable video coding service Download PDFInfo
- Publication number
- CN101951506A CN101951506A CN2010102908817A CN201010290881A CN101951506A CN 101951506 A CN101951506 A CN 101951506A CN 2010102908817 A CN2010102908817 A CN 2010102908817A CN 201010290881 A CN201010290881 A CN 201010290881A CN 101951506 A CN101951506 A CN 101951506A
- Authority
- CN
- China
- Prior art keywords
- stream
- timestamp
- multimedia broadcasting
- media
- mobile multimedia
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234327—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/633—Control signals issued by server directed to the network components or client
- H04N21/6332—Control signals issued by server directed to the network components or client directed to client
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/24—Systems for the transmission of television signals using pulse code modulation
Abstract
The invention provides a method for realizing synchronous transmitting of a scalable video coding service in the mobile multi-media broadcasting. The video service is coded to generate multipath media streams which are respectively sealed in different multiplexing subframes in a broadcasting channel frame by taking a media unit as a unit; a mobile multi-media broadcasting timestamp of each media unit is sealed in the multiplexing subframe; the mobile multi-media broadcasting timestamps of the media streams at the same sampling time are synchronized; the position information of the multiplexing subframe in which the media stream are located is sealed in a broadcasting channel frame and is transmitted to a terminal; and the media stream comprises a base-layer code stream and an enhancement layer code stream corresponding to the base-layer code stream generated by video stream coding of the video service, and also comprises an audio stream of the video service. The invention also provides a system for realizing synchronous transmitting of the scalable video coding service in the mobile multi-media broadcasting and a receiving method and a receiving terminal of the scalable video coding service, and is aligned with the received media data according to the mobile multi-media broadcasting timestamp.
Description
Technical field
The present invention relates to transmission technology in the mobile multimedia broadcast system, relate in particular to a kind of China Mobile multimedia broadcasting (CMMB of system, China Mobile Multimedia Broadcasting) transmits scalable video (SVC in, when Scalable Video Coding) professional, realize the system and method that the scalable video service synchronization sends, receives.
Background technology
Along with development of Communication Technique, the application of China mobile multi-media broadcasting technology is more and more wider.At present, China Mobile multimedia broadcasting system standard has been stipulated in the broadcasting service frequency range, frame structure, chnnel coding and the modulation etc. of mobile multimedia broadcast system broadcast channel transmission signals.Regulation in the CMMB standard " Mobile Multimedia Broadcasting part 2: multiplexing " adopts multiplexed sub frame to come stream medium datas such as encapsulate video, audio frequency, transmits.
Scalable video (SVC, Scalable Video Coding) is a kind of video spatial scalable coded system.Encoder is encoded to video content source, produces the code stream of many levels, and basic layer bit stream can be decoded separately, and enhancement layer bitstream comprises the additional information that is used to improve the low-lever code current mass, need and comprise that the low layer of basic layer is decoded together.The SVC technology can provide gradable telescopic business, and realizing has other differentiated service in poor service, realizes that the ability of various terminals is adaptive, has plurality of advantages, and therefore, it is necessary transmitting the SVC business in the CMMB system.
Application number is 200910088679.3 patent " classification transmission, method of reseptance and device in the mobile multimedia broadcast system ", provided a kind of method that SVC transmits that in CMMB, realizes, according to this method, basic layer bit stream in the SVC video traffic and enhancement layer bitstream can be carried out the layering transmission, basic layer bit stream and enhancement layer bitstream are packaged in respectively in the different multiplexed sub frames in the broadcast channel frame by the layer under it, are packaged in the positional information of the multiplexed sub frame at each layer bit stream place of described video flowing in the described broadcast channel frame simultaneously and are sent to receiving terminal.In terminal, the positional information of the multiplexed sub frame at each layer bit stream place of video flowing in the video traffic in the monitoring broadcast channel frame, receiving terminal receives basic layer bit stream according to the video flow processing ability of self, or receive basic layer bit stream and corresponding enhancement layer bitstream, enhancement layer bitstream to basic layer bit stream or basic layer bit stream and correspondence is decoded, and exports the video data of basic layer bit stream or the video data after basic layer bit stream and the enhancement layer bitstream merging.
Each layering of SVC separately transmits at different multiplexed sub frames, and the synchronous, collaborative of each layering is the problem that must solve.How multiplexed frame under the multiplexed sub frame at each layer bit stream place is one synchronously and must solves Ah 's problem not simultaneously.
In the SVC video, the access unit on the specific time point can be encoded in a plurality of layerings such as a frame of video, and they are separated to transmit at different multiplexed sub frames; In end side, the different layered encoded unit that needs to come by different multiplexed sub frame transmission merges, and merges as a plurality of layerings with same frame of video, carries out video decode then and represents.When merging, each hierarchical coding unit is synchronously necessary, and what could guarantee the participation merging is the coding unit of same access unit, guarantees the success of merging, merges to guarantee that successfully subsequent decoding represents success.
The present invention need solve, when in CMMB, transmitting SVC exactly and dividing layer service, how to guarantee between above-mentioned a plurality of SVC layering synchronously.
Summary of the invention
The technical problem to be solved in the present invention is when transmission SVC divides layer service in CMMB, the system and method that provides a kind of SVC of realization audio video synchronization to send, receive, between the SVC individual-layer data that guarantees separately in different multiplexed sub frames, to transmit synchronously, thereby the normal enforcement of SVC business among the assurance CMMB.
In order to address the above problem, the invention provides and realize the method that the scalable video service synchronization sends in a kind of Mobile Multimedia Broadcasting, comprising:
The video traffic coding is generated multipath media stream, with described multipath media stream is that unit is encapsulated in respectively in the different multiplexed sub frames in the broadcast channel frame with the media units, also carry the Mobile Multimedia Broadcasting timestamp of each media units that encapsulates in this multiplexed sub frame in the described multiplexed sub frame, and the Mobile Multimedia Broadcasting timestamp of the Media Stream of same sampling instant is synchronous, and the positional information of the multiplexed sub frame at described Media Stream place is packaged in the described broadcast channel frame, described broadcast channel frame is sent to receiving terminal, described Media Stream comprises basic layer bit stream and the corresponding enhancement layer bitstream thereof that the video flowing coding of described video traffic generates, perhaps, the audio stream that comprises video traffic, basic layer bit stream and corresponding enhancement layer bitstream thereof that the video flowing coding of video traffic generates.
Further, said method also can have following characteristics, as follows described Media Stream is encapsulated in the described multiplexed sub frame:
Described multipath media stream is encapsulated as multichannel real-time transport protocol (rtp) code stream, wherein, every road RTP code stream is followed one road RTCP Real-time Transport Control Protocol (RTCP) code stream, and described RTCP code stream guarantees NTP (Network Time Protocol) (NTP) time synchronized of the Media Stream of same sampling instant;
Extract the Media Stream that encapsulates in the described RTP code stream, with described Media Stream is that unit is encapsulated in respectively in the different multiplexed sub frames in the broadcast channel frame with the media units, the RTP timestamp of described Media Stream is converted to the NTP time, again be converted to the Mobile Multimedia Broadcasting timestamp under the benchmark unified time NTP time, described Mobile Multimedia Broadcasting timestamp be encapsulated in the multiplexed sub frame at its corresponding media units place.
Further, said method also can have following characteristics, as follows the RTP timestamp is converted to the Mobile Multimedia Broadcasting timestamp:
To each media units, take out the RTP timestamp of its place RTP bag, the relevant temporal information of the RTCP bag that transmits in conjunction with the RTCP code stream of its place RTP code stream correspondence, the NTP time of calculating this media units;
The NTP time and the Mobile Multimedia Broadcasting time scale of this media units are multiplied each other, obtain the Mobile Multimedia Broadcasting timestamp of described media units.
Further, said method also can have following characteristics, and the Mobile Multimedia Broadcasting timestamp of the Media Stream of described same sampling instant is meant synchronously: the difference between the Mobile Multimedia Broadcasting timestamp value of the Media Stream of same sampling instant is in default timestamp tolerance scope.
The present invention also provides and realizes the system that the scalable video service synchronization sends in a kind of Mobile Multimedia Broadcasting, and described system comprises encoding device and front end transmitting apparatus, wherein:
Described encoding device is used for the video traffic coding is generated multipath media stream; Described Media Stream comprises basic layer bit stream and the corresponding enhancement layer bitstream thereof that the video flowing coding of described video traffic generates, perhaps, the basic layer bit stream and the corresponding enhancement layer bitstream thereof that comprise the video flowing coding generation of the audio stream of described video traffic, described video traffic;
Described front end transmitting apparatus, being used for described Media Stream is that unit is encapsulated in respectively in the different multiplexed sub frames of broadcast channel frame with the media units, the Mobile Multimedia Broadcasting timestamp of each media units of encapsulating in this multiplexed sub frame is encapsulated in the described multiplexed sub frame, and the Mobile Multimedia Broadcasting timestamp of the Media Stream of same sampling instant is synchronous, and the positional information of the multiplexed sub frame at described Media Stream place is packaged in the described broadcast channel frame, described broadcast channel frame is sent to receiving terminal.
Further, said system also can have following characteristics, and described encoding device comprises coding unit and encapsulation unit, wherein:
Described coding unit is used for the video traffic coding is generated multipath media stream;
Described encapsulation unit, be used for described multipath media stream is encapsulated as multichannel real-time transport protocol (rtp) code stream, wherein, every road RTP code stream is followed one road RTCP Real-time Transport Control Protocol (RTCP) code stream, and described RTCP code stream guarantees NTP (Network Time Protocol) (NTP) time synchronized of the Media Stream of same sampling instant;
Described front end transmitting apparatus comprises first encapsulation unit, second encapsulation unit, converting unit, and the 3rd encapsulation unit and transmitting element, wherein:
First encapsulation unit is used for extracting the Media Stream that described RTP code stream encapsulates, and is that unit is encapsulated in respectively in the different multiplexed sub frames in the broadcast channel frame with the media units with described Media Stream;
Described second encapsulation unit is used for the positional information of the multiplexed sub frame at described Media Stream place is encapsulated in described broadcast channel frame;
Described converting unit is used for the RTP timestamp of described Media Stream is converted to the NTP time, is converted to the Mobile Multimedia Broadcasting timestamp under the benchmark unified time NTP time again;
Described the 3rd encapsulation unit is used for described Mobile Multimedia Broadcasting timestamp is encapsulated into the multiplexed sub frame at its corresponding media units place;
Described transmitting element is used for described broadcast channel frame is sent to receiving terminal.
Further, said system also can have following characteristics, and described converting unit comprises first converting unit and second converting unit, wherein:
Described first converting unit is used for each media units, takes out the RTP timestamp of its place RTP bag, the relevant temporal information of the RTCP bag that transmits in conjunction with the RTCP code stream of its place RTP code stream correspondence, the NTP time of calculating this media units;
Described second converting unit is used for the NTP time and the Mobile Multimedia Broadcasting time scale of described media units are multiplied each other, and obtains the Mobile Multimedia Broadcasting timestamp of described media units.
Further, said system also can have following characteristics, and the Mobile Multimedia Broadcasting timestamp of the Media Stream of described same sampling instant is meant synchronously: the difference between the Mobile Multimedia Broadcasting timestamp value of the Media Stream of same sampling instant is in default timestamp tolerance scope.
The present invention also provides and realizes the scalable video service receiving method in a kind of Mobile Multimedia Broadcasting, comprising:
The positional information of the multiplexed sub frame at each layer bit stream place of video flowing in the video traffic in the receiving terminal monitoring broadcast channel frame;
Receiving terminal receives basic layer bit stream according to the video flow processing ability of self, perhaps, receives basic layer bit stream and corresponding enhancement layer bitstream;
When receiving the enhancement layer bitstream of basic layer bit stream and correspondence, after each layer bit stream merged according to Mobile Multimedia Broadcasting timestamp alignment, basic layer bit stream and corresponding enhancement layer bitstream are decoded, export the video data after basic layer bit stream and enhancement layer bitstream merge.
Further, said method also can have following characteristics, and basic layer bit stream of described reception and corresponding enhancement layer bitstream comprise each layer bit stream according to Mobile Multimedia Broadcasting timestamp alignment merging:
Receiving terminal will receive basic layer bit stream and enhancement layer bitstream, deposit in the buffering area;
From buffering area, take out the basic layer bit stream data that belong to a video access unit, Mobile Multimedia Broadcasting timestamp with described basic layer bit stream data is a benchmark, the synchronous enhancement layer bitstream data of Mobile Multimedia Broadcasting timestamp of taking-up and described basic layer bit stream data merge its data as same video access unit.
Further, said method also can have following characteristics, the synchronous enhancement layer bitstream data of the Mobile Multimedia Broadcasting timestamp of described and described basic layer bit stream data are meant, with the enhancement layer bitstream data of difference in default timestamp tolerance scope of the Mobile Multimedia Broadcasting timestamp of described basic layer bit stream data.
The present invention also provides and realizes the scalable video service receiving device in a kind of Mobile Multimedia Broadcasting, comprising:
Monitoring unit, the positional information of the multiplexed sub frame at each layer bit stream place of video flowing in the video traffic in the monitoring broadcast channel frame;
Receiving element receives basic layer bit stream according to self video flow processing ability, perhaps, receives basic layer bit stream and corresponding enhancement layer bitstream;
The alignment merge cells is used for each layer bit stream is merged according to the alignment of Mobile Multimedia Broadcasting timestamp;
Decoding unit is decoded to basic layer bit stream and corresponding enhancement layer bitstream, exports the video data after basic layer bit stream and enhancement layer bitstream merge.
Further, said apparatus also can have following characteristics, described receiving element, and the basic layer bit stream and the enhancement layer bitstream that also are used for receiving deposit buffering area in;
Described alignment merge cells, be used for taking out the basic layer bit stream data that belong to a video access unit from buffering area, Mobile Multimedia Broadcasting timestamp with described basic layer bit stream data is a benchmark, the synchronous enhancement layer bitstream data of Mobile Multimedia Broadcasting timestamp of taking-up and described basic layer bit stream data merge its data as same video access unit.
Further, said apparatus also can have following characteristics, the synchronous enhancement layer bitstream data of the Mobile Multimedia Broadcasting timestamp of described taking-up and described basic layer bit stream data are meant, the enhancement layer bitstream data of the difference between the Mobile Multimedia Broadcasting timestamp of taking-up and described basic layer bit stream data in default timestamp tolerance scope.
The method of the invention is stamped the synchronous CMMB airtime and is stabbed in the SVC of different layerings data, guaranteed between the different individual-layer datas synchronously.
Description of drawings
Fig. 1 is the composition structural representation of CMMB channel frame;
Fig. 2 is a system schematic of the present invention;
Fig. 3 is the composition structural representation of broadcast channel frame involved in the present invention;
Fig. 4 is a terminal processes functional schematic involved in the present invention, when the SVC frame of video is synthetic among the figure, participates in synthetic basic layer video unit and enhancement-layer video unit and must keep synchronous.
Embodiment
Basic thought of the present invention is to stamp synchronous CMMB timestamp on the enhancement layer bitstream of the audio stream of same sampling instant, basic layer bit stream and correspondence, thereby realize the SVC service synchronization.
The invention provides and realize the method that the scalable video service synchronization sends in a kind of Mobile Multimedia Broadcasting, comprising:
The video traffic coding is generated multipath media stream, with described multipath media stream is that unit is encapsulated in respectively in the different multiplexed sub frames in the broadcast channel frame with the media units, also carry the Mobile Multimedia Broadcasting timestamp of each media units that encapsulates in this multiplexed sub frame in the described multiplexed sub frame, and the Mobile Multimedia Broadcasting timestamp of the Media Stream of same sampling instant is synchronous, and the positional information of the multiplexed sub frame at described Media Stream place is packaged in the described broadcast channel frame, described broadcast channel frame is sent to receiving terminal, described Media Stream comprises basic layer bit stream and the corresponding enhancement layer bitstream thereof that the video flowing coding generates, perhaps, comprise audio stream, basic layer bit stream that the video flowing coding generates and corresponding enhancement layer bitstream thereof.
Wherein, as follows described Media Stream is encapsulated in the described multiplexed sub frame:
Described multipath media stream is encapsulated as multichannel real-time transport protocol (rtp) code stream, wherein, every road RTP code stream is followed one road RTCP Real-time Transport Control Protocol (RTCP) code stream, and described RTCP code stream guarantees NTP (Network Time Protocol) (NTP) time synchronized of the Media Stream of same sampling instant;
Extract the Media Stream that encapsulates in the described RTP code stream, with described Media Stream is that unit is encapsulated in respectively in the different multiplexed sub frames in the broadcast channel frame with the media units, the RTP timestamp of described Media Stream is converted to the NTP time, again be converted to the Mobile Multimedia Broadcasting timestamp under the benchmark unified time NTP time, described Mobile Multimedia Broadcasting timestamp be encapsulated in the multiplexed sub frame at its corresponding media units place.
Wherein, as follows the RTP timestamp is converted to the Mobile Multimedia Broadcasting timestamp:
To each media units, take out the RTP timestamp of its place RTP bag, the relevant temporal information of the RTCP bag that transmits in conjunction with the RTCP code stream of its place RTP code stream correspondence, the NTP time of calculating this media units;
The NTP time and the Mobile Multimedia Broadcasting time scale of this media units are multiplied each other, obtain the Mobile Multimedia Broadcasting timestamp of described media units.
Wherein, the Mobile Multimedia Broadcasting timestamp of described media units comprises relative reproduction time two parts of initial reproduction time and each media units correspondence.In the same multiplexed sub frame, the initial reproduction time of each media units is identical.
Wherein, the Mobile Multimedia Broadcasting timestamp of the Media Stream of described same sampling instant is meant synchronously: the difference between the Mobile Multimedia Broadcasting timestamp value of the Media Stream of same sampling instant is in default timestamp tolerance scope.
The present invention also provides and realizes the system that the scalable video service synchronization sends in a kind of Mobile Multimedia Broadcasting, and described system comprises encoding device and front end transmitting apparatus, wherein:
Described encoding device is used for the video traffic coding is generated multipath media stream; Described Media Stream comprises basic layer bit stream and the corresponding enhancement layer bitstream thereof that the video flowing coding generates, and perhaps, comprises basic layer bit stream and corresponding enhancement layer bitstream thereof that audio stream, video flowing coding generate;
Described front end transmitting apparatus, being used for described Media Stream is that unit is encapsulated in respectively in the different multiplexed sub frames of broadcast channel frame with the media units, the Mobile Multimedia Broadcasting timestamp of each media units of encapsulating in this multiplexed sub frame is encapsulated in the described multiplexed sub frame, and the Mobile Multimedia Broadcasting timestamp of the Media Stream of same sampling instant is synchronous, and the positional information of the multiplexed sub frame at described Media Stream place is packaged in the described broadcast channel frame, described broadcast channel frame is sent to receiving terminal.
Wherein, described encoding device comprises coding unit and encapsulation unit, wherein:
Described coding unit is used for the video traffic coding is generated multipath media stream;
Described encapsulation unit, be used for described multipath media stream is encapsulated as multichannel real-time transport protocol (rtp) code stream, wherein, every road RTP code stream is followed one road RTCP Real-time Transport Control Protocol (RTCP) code stream, and described RTCP code stream guarantees NTP (Network Time Protocol) (NTP) time synchronized of the Media Stream of same sampling instant;
Described front end transmitting apparatus comprises first encapsulation unit, second encapsulation unit, converting unit, and the 3rd encapsulation unit and transmitting element, wherein:
First encapsulation unit is used for extracting the Media Stream that described RTP code stream encapsulates, and is that unit is encapsulated in respectively in the different multiplexed sub frames in the broadcast channel frame with the media units with described Media Stream;
Described second encapsulation unit is used for the positional information of the multiplexed sub frame at described Media Stream place is encapsulated in described broadcast channel frame;
Described converting unit is used for the RTP timestamp of described Media Stream is converted to the NTP time, is converted to the Mobile Multimedia Broadcasting timestamp under the benchmark unified time NTP time again;
Described the 3rd encapsulation unit is used for described Mobile Multimedia Broadcasting timestamp is encapsulated into the multiplexed sub frame at its corresponding media units place;
Described transmitting element is used for described broadcast channel frame is sent to receiving terminal.
Wherein, described converting unit comprises first converting unit and second converting unit, wherein:
Described first converting unit is used for each media units, takes out the RTP timestamp of its place RTP bag, the relevant temporal information of the RTCP bag that transmits in conjunction with the RTCP code stream of its place RTP code stream correspondence, the NTP time of calculating this media units;
Described second converting unit is used for the NTP time and the Mobile Multimedia Broadcasting time scale of described media units are multiplied each other, and obtains the Mobile Multimedia Broadcasting timestamp of described media units.
The present invention also provides and realizes the scalable video service receiving method in a kind of Mobile Multimedia Broadcasting, comprising:
The positional information of the multiplexed sub frame at each layer bit stream place of video flowing in the video traffic in the receiving terminal monitoring broadcast channel frame;
Receiving terminal receives basic layer bit stream according to the video flow processing ability of self, perhaps, receives basic layer bit stream and corresponding enhancement layer bitstream;
When receiving the enhancement layer bitstream of basic layer bit stream and correspondence, after each layer bit stream merged according to Mobile Multimedia Broadcasting timestamp alignment, basic layer bit stream and corresponding enhancement layer bitstream are decoded, export the video data after basic layer bit stream and enhancement layer bitstream merge.
Wherein, basic layer bit stream of described reception and corresponding enhancement layer bitstream comprise each layer bit stream according to Mobile Multimedia Broadcasting timestamp alignment merging:
Receiving terminal will receive basic layer bit stream and enhancement layer bitstream, deposit in the buffering area;
From buffering area, take out the basic layer bit stream data that belong to a video access unit, Mobile Multimedia Broadcasting timestamp with described basic layer bit stream data is a benchmark, the synchronous enhancement layer bitstream data of Mobile Multimedia Broadcasting timestamp of taking-up and described basic layer bit stream data merge its data as same video access unit.
Wherein, the synchronous enhancement layer bitstream data of the Mobile Multimedia Broadcasting timestamp of described and described basic layer bit stream data are meant that the difference between the Mobile Multimedia Broadcasting timestamp of enhancement layer bitstream data and the Mobile Multimedia Broadcasting timestamp of described basic layer bit stream data is in default timestamp tolerance scope.
The present invention also provides and realizes the scalable video service receiving device in a kind of Mobile Multimedia Broadcasting, comprising:
Monitoring unit, the positional information of the multiplexed sub frame at each layer bit stream place of video flowing in the video traffic in the monitoring broadcast channel frame;
Receiving element receives basic layer bit stream according to self video flow processing ability, perhaps, receives basic layer bit stream and corresponding enhancement layer bitstream;
The alignment merge cells is used for each layer bit stream is merged according to the alignment of Mobile Multimedia Broadcasting timestamp;
Decoding unit is decoded to basic layer bit stream and corresponding enhancement layer bitstream, exports the video data after basic layer bit stream and enhancement layer bitstream merge.
Wherein, described receiving element, the basic layer bit stream and the enhancement layer bitstream that also are used for receiving deposit buffering area in;
Described alignment merge cells, be used for taking out the basic layer bit stream data that belong to a video access unit from buffering area, Mobile Multimedia Broadcasting timestamp with described basic layer bit stream data is a benchmark, the synchronous enhancement layer bitstream data of Mobile Multimedia Broadcasting timestamp of taking-up and described basic layer bit stream data merge its data as same video access unit.
Wherein, the synchronous enhancement layer bitstream data of the Mobile Multimedia Broadcasting timestamp of described taking-up and described basic layer bit stream data are meant, the enhancement layer bitstream data of the difference between the Mobile Multimedia Broadcasting timestamp of taking-up and described basic layer bit stream data in default timestamp tolerance scope.
The invention provides a kind of system that realizes the professional a plurality of hierarchical synchronization of scalable video, comprise encoding device, front end transmitting apparatus and terminal, wherein:
Described encoding device, be used for video source is encoded, generation comprises the SVC code stream of basic layer and some enhancement layers, the SVC code stream of a plurality of layerings sends after being encapsulated as multichannel RTP (RTP) code stream, for each road RTP code stream, follow one road RTCP (RTCP Real-time Transport Control Protocol) code stream, described RTCP code stream be used to guarantee audio frequency and video business in the basic layer on NTP (NTP (Network Time Protocol)) synchronously, temporal synchronously with the SVC business of a plurality of layerings at NTP (NTP (Network Time Protocol)), the enhancement layer bitstream that promptly guarantees basic layer bit stream of same sampling instant and correspondence is temporal synchronously at NTP, if there is audio stream, the audio stream that also will guarantee same sampling instant and basic layer bit stream and corresponding enhancement layer bitstream are temporal synchronous at NTP.
Described front end transmitting apparatus, be used for RTP code stream and RTCP code stream that received code equipment sends, from the RTP code stream, extract basic layer bit stream of SVC and enhancement layer bitstream, is that unit is packaged in respectively in the different multiplexed sub frames in the broadcast channel frame with the basic layer bit stream of SVC and enhancement layer bitstream with the video unit by the layer under it, also carry the CMMB airtime of each video unit that encapsulates in this multiplexed sub frame in the multiplexed sub frame and stab, the positional information with the multiplexed sub frame at each layer bit stream place of described video flowing is packaged in the described broadcast channel frame simultaneously.On this equipment, the CMMB airtime that the RTP timestamp of media data converts under benchmark unified time is stabbed, it is synchronous that control SVC each minute layer service stabbed in the CMMB airtime, and the media data that needs are sent stabs broadcast transmission together with the CMMB airtime of this media data.
Described terminal is used for monitoring the positional information of the multiplexed sub frame at each layer bit stream place of video flowing in the broadcast channel frame video traffic, and receives basic layer bit stream according to self video flow processing ability, or receives basic layer bit stream and corresponding enhancement layer bitstream.Merge after stabbing synchronously by the CMMB airtime different layered encoded unit, carry out video decode then and represent.
The present invention also provides a kind of method that realizes the professional a plurality of hierarchical synchronization of scalable video, comprising:
Encoding device will be encoded and be sent after the SVC code stream of a plurality of layerings of generating is encapsulated as multichannel RTP code stream, for each road RTP code stream, follow one road RTCP code stream.The RTCP code stream guarantees that basic layer SVC data, enhancement layer SVC data and the voice data in the same sampling instant is temporal synchronously at NTP;
RTP code stream and RTCP code stream that front end transmitting apparatus received code equipment sends, from the RTP code stream, extract packaged SVC business datum, the basic layer bit stream of SVC and enhancement layer bitstream are packaged in respectively in the different multiplexed sub frames in the broadcast channel frame by the layer under it, and the positional information with the multiplexed sub frame at each layer bit stream place of described video flowing is packaged in the described broadcast channel frame simultaneously.On this equipment, the CMMB airtime that the RTP timestamp of media data converts under benchmark unified time is stabbed, it is synchronous that control SVC each minute layer service stabbed in the CMMB airtime, and the media data that needs are sent stabs broadcast transmission together with the CMMB airtime of this media data.
The positional information of the multiplexed sub frame at each layer bit stream place of video flowing in the video traffic in the terminal monitoring broadcast channel frame, and receive basic layer bit stream according to self video flow processing ability, or receive basic layer bit stream and corresponding enhancement layer bitstream, the CMMB airtime is pressed in different layered encoded unit stab alignment, merge synchronously, carry out video decode then and represent.
On the front end transmitting apparatus, it is synchronous that SVC each minute layer service stabbed in the CMMB airtime, carries out by the following method:
For pairing each multiplexed sub frame of each layering of SVC,
A1, the RTP that receives input and RTCP stream are according to the relevant temporal information of the RTCP bag that the pairing RTP of video unit wraps, the pairing RTCP code stream of RTP bag place RTP code stream transmits, the NTP time of calculating this video unit correspondence;
B1, for each video unit, directly be multiplied by the CMMB time scale with the above-mentioned NTP time, the result is stabbed maximum number of digits according to the CMMB airtime rounds, the CMMB airtime that obtains this video unit stabs.
For each multiplexed sub frame, media units at all band timestamps that will be encapsulated into this multiplexed sub frame, get an initial reproduction time, stabbed the initial reproduction time that is decomposed into multiplexed sub frame and relative reproduction time two parts of each unit correspondence the CMMB airtime of each video unit, the relative reproduction time of initial reproduction time and each unit correspondence is encapsulated in the multiplexed sub frame.
The time scale of CMMB described in the step B1, the CMMB chronomere number that the expression per second takes place, according to CMMB standard " Mobile Multimedia Broadcasting part 2: multiplexing ", the CMMB time scale is 22500.It is 32 that the CMMB airtime of this standard code is stabbed maximum number of digits.
The CMMB airtime stabs and is similar to the RTP timestamp, is that a kind of relative time stabs.Initial reproduction time in the multiplexed sub frame has been indicated each the interior audio unit of this multiplexed sub frame or the reproduction time benchmark of video unit, the relative reproduction time of each audio unit or video unit has been indicated the skew of the reproduction time of each audio unit in this multiplexed sub frame or video unit with respect to initial reproduction time, can determine that according to initial reproduction time and relative reproduction time the relative time of each audio unit in the multiplexed sub frame or video unit broadcast concerns.But, when the business as SVC is transmitted with multiplexed subframe stream, because it is that the relative time that is suitable in a kind of single channel multiplexed sub frame stream stabs that the CMMB airtime stabs, its timestamp of multiplexed sub frame between the multiplexed subframe stream lacks common benchmark, shortage is at the original bench mark of the timestamp of the SVC business of transmitting with multiplexed subframe stream, thereby multiplexed subframe can't realize synchronously between flowing.
The present invention is multiplied by the CMMB airtime that the CMMB time scale converses the video unit correspondence with the NTP time and stabs by direct, because its NTP time synchronized of video unit of same sampling instant, thus its CMMB airtime of video unit that can guarantee same sampling instant stab synchronously.This CMMB airtime stabs to unite with relative reproduction time with initial reproduction time and represents, the initial reproduction time in the same multiplexed sub frame is identical, and the initial reproduction time of different multiplexed sub frames can be identical or different.
Among the present invention, can determine that the CMMB airtime of SVC business stabs initial value (also can use other times to stab initial value as the CMMB airtime) with the NTP time of first video unit in handled first multiplexed sub frame, and determine that by this conversion method the CMMB airtime of subsequent video unit stabs.
Among the present invention, with a kind of like this method that is easy to carry out, set up the common timestamp benchmark of the SVC business of transmitting with multiplexed subframe stream, each video unit in the multiplexed subframe of same business, it stabbed according to the CMMB airtime that said method converses, stab the deviation of initial value with respect to the above-mentioned CMMB airtime, determined that this unit stabs the reproduction time skew of initial value with respect to the CMMB airtime.The present invention can guarantee the unification that each the layering business datum of SVC on the same NTP time point was stabbed in the CMMB airtime, thereby synchronous assurance means between the different layerings of SVC are provided.
According to this method, but pairing each the multiplexed sub frame independent process of each layering of SVC, and coupling does not rely on each other.
For said method, the data that transmit in the SVC stratified flow are except video data, and when also comprising voice data, the timestamp of voice data is by the method processing same with video data.
On the terminal, the carrying out by the following method synchronously of SVC each minute layer service:
A2, each layering business datum of received SVC stabbed together with its time corresponding put into buffering area, this buffering area can be one, also can be one of each layering, and buffer size should be able to be held the transmission time difference of each individual-layer data.For some reason, the SVC individual-layer data of synchronization can not guarantee fully to receive simultaneously that the poor threshold value time of reception of establishing each layering business datum is Td, and buffering area should be able to hold the data of all layerings of SVC in the Td time range;
B2, in particular moment, as the decoding of specific interval constantly, get data to be combined from buffering area, alignment merges.Getting the base layer data that belongs to a video access unit earlier, is benchmark with the timestamp of base layer data, takes out the enhancement data corresponding with the base layer data timestamp, and its data as same video access unit are merged.
In step B2, for a certain reason, the temporal information that the RTP that encoding device sends, RTCP are entrained, may not guarantee that base layer data, enhancement data and voice data in the same sampling instant are temporal in full accord at NTP, the CMMB airtime of changing out on headend equipment like this stabs also just can not be in full accord, and have a little deviation, when terminal is looked for corresponding enhancement data according to the timestamp of base layer data, also establish a timestamp tolerance.
For making the purpose, technical solutions and advantages of the present invention clearer, by the following examples, the present invention is described in more detail.
Fig. 2 is a system schematic of the present invention.
Encoding device is basic layer bit stream and at least one enhancement layer bitstream with the SVC video coding, and basic layer bit stream can be decoded separately, and enhancement layer bitstream comprises the additional information that is used to improve the low-lever code current mass, need and comprise that the low layer of basic layer is decoded together.
On timeline, video data is a base unit with the video access unit, and typical video access unit is as a frame of video.Under the SVC mode, a video access unit on the particular point in time can be encoded as a plurality of layers data, and these data of a plurality of layers can be divided into multichannel and transmit; Receiving terminal can receive multiplex a plurality of layers data as required simultaneously, and it is merged by affiliated video access unit, and decoding represents.In the present embodiment, suppose that encoding device is the H264SVC encoder, its coding output video data elementary cell is NALU (Network Abstraction Layer Unit, a network abstraction layer unit), and the some NALU on the same time point form a video access unit.
For ease of describing, in the present embodiment, suppose that broadcasting service has comprised SVC video and one road audio frequency, the SVC video adopts the space delamination pattern, be encoded as one road QVGA (QuarterVGA, 320 * 240 pixel images) basic stream and one road VGA (Video Graphic Array, 640X480 pixel image) strengthen video flowing, and audio frequency is encoded as one road audio stream.Need to prove that the method for the invention is equally applicable to the situation of multichannel enhanced flow and other SVC coding mode.
Encoding device will be encoded and be sent after the basic layer bit stream, enhancement layer bitstream and the audio stream that generate are encapsulated as one road RTP code stream separately.For each road RTP code stream, follow to send one road RTCP code stream.The timestamp of every road RTP code stream can not interdepend, and has separately independently RTP time scale and initial time stamp, and time scale is represented chronomere's number of medium in 1 second.By RTCP, guarantee that basic layer SVC data, enhancement layer SVC data and the voice data in the same sampling instant is temporal synchronously at NTP.For example, the time scale of audio frequency is 48000, and the time scale of video is 90000.For certain video access unit, after base layer data was encapsulated in the RTP bag of basic stream, the RTP timestamp can be Tbase, after enhancement data is encapsulated in the RTP bag of enhanced flow, the RTP timestamp can be Text, can be Raudio with audio frequency RTP on its synchronization point bag timestamp.But this each self-corresponding RTCP code stream of several roads RTP code stream, SR in the RTCP message (Sender Report) has carried the lining with reference to NTP time and corresponding reference RTP timestamp, should guarantee that basic layer SVC data, enhancement layer SVC data and voice data in the same sampling instant are temporal synchronously at NTP, promptly above-mentioned Tbase, the pairing separately NTP time of Text, Taudio should be consistent.For the flexibility that realizes, " the NTP time should be consistent " herein can be the pairing separately NTP time approximately equal of Tbase, Text, Taudio, allows a deviate, and this deviate can be decided as required.
RTP code stream and RTCP code stream that front end transmitting apparatus received code equipment sends, from the RTP code stream, extract packaged SVC business datum, basic layer bit stream and enhancement layer bitstream are packaged in respectively in the different multiplexed sub frames in the broadcast channel frame by the layer under it, and the positional information with the multiplexed sub frame at each layer bit stream place of described video flowing is packaged in the described broadcast channel frame simultaneously.
Fig. 3 is the composition structural representation of broadcast channel frame of the present invention.As shown in Figure 3,40 time slots are arranged on certain frequency F, wherein time slot 0 (multiplexed frame 0) is used for communicating control information, and time slot 1 to time slot 39 is used for transport service information.The front end transmitting apparatus is configured to the basic layer bit stream of the video among the video traffic S, audio frequency and data message in the multiplexed frame 1, takies time slot 1 to time slot 4, and multiplexed sub frame number is 1.The video enhancement layer bitstream is configured to multiplexed frame 2, takies time slot 5 to time slot 6, and multiplexed sub frame number is 1.No longer transmit other business in multiplexed frame 1 and the multiplexed frame 2.Control information and electron service instruction (ESG in business, Electronic Service Guide) descriptor of the multiplexed frame position of each layer bit stream of interpolation carrying (basic layer bit stream and each corresponding enhancement layer bitstream thereof) in the information, this descriptor indicates that video traffic S comprises two multiplexed sub frames: the multiplexed sub frame 1 of multiplexed frame 1 is professional basic layer bit stream data, and the multiplexed sub frame 1 of multiplexed frame 2 is the enhancement layer bitstream data.
The front end transmitting apparatus obtains the basic layer bit stream V1 of video traffic S, is encapsulated in the multiplexed sub frame 1 of multiplexed frame 1, obtains the enhancement layer bitstream V2 of video traffic S, is encapsulated in the multiplexed sub frame 1 of multiplexed frame 2.The front end transmitting apparatus obtains audio code stream and the data segment information of video traffic S, adds in the multiplexed sub frame 1 of multiplexed frame 1, promptly is carried in the same multiplexed sub frame with basic layer bit stream.The descriptor of the multiplexed frame position of each layer bit stream is carried in the multiplexed frame 0 in the broadcast channel frame, with the multiplexed frame position of each layer bit stream of indication receiving terminal, to make things convenient for its receiving video service S.
When audio, video data is encapsulated in the multiplexed sub frame, so that audio unit or video unit are that unit encapsulates one by one.For each audio unit or video unit, all there is the CMMB airtime of a correspondence to stab, according to the CMMB standard, this timestamp is that the interior initial reproduction time of multiplexed sub frame and the relative reproduction time of each unit correspondence combine.The relative reproduction time of initial reproduction time and each unit correspondence also is encapsulated in the multiplexed sub frame, together sends with media data.
On the front end transmitting apparatus, the RTP timestamp of the media data that received need be converted to the CMMB airtime to stab, what SVC each minute layer service stabbed in the CMMB airtime in the same sampling instant of control in this process is synchronous.
On the front end transmitting apparatus, it is synchronous that SVC each minute layer service stabbed in the CMMB airtime, carries out by the following method:
For pairing each multiplexed sub frame of each layering of SVC:
The RTP of A, reception input and RTCP stream for each video unit, take out the RTP timestamp of its place RTP bag, the relevant temporal information that the RTCP that transmits in conjunction with the pairing RTCP code stream of its place RTP code stream wraps, the NTP time of calculating this video unit correspondence;
B, directly be multiplied by the CMMB time scale with the NTP time, the CMMB airtime that obtains this video unit stabs.To multiplexed sub frame, get an initial reproduction time, the CMMB airtime of video unit is stabbed the relative reproduction time two parts that are decomposed into initial reproduction time and each unit correspondence, the relative reproduction time of initial reproduction time and each unit correspondence is encapsulated in the multiplexed sub frame goes.
The calculating of the NTP time of video unit correspondence can be adopted following method in the steps A:
(1) for each video unit, take out the RTP timestamp of its place RTP bag, in the RTCP message of correspondence, take out reference NTP time and the corresponding reference RTP timestamp that carries SR (Sender Report) lining;
(2) do difference with the reference RTP timestamp in the RTCP code stream of RTP timestamp in the RTP bag and correspondence, again divided by timescale (time scale), obtain one poor with reference to absolute time, add the time with this difference with reference to NTP, just obtain the corresponding NTP time of RTP bag, that is the NTP time of video unit correspondence.
Above-mentioned timescale is the markers of RTP medium itself or claims time scale or claim time scale, chronomere's number of representing medium in 1 second, medium different time unit can be different, as the RTP video, the clock of 90000Hz commonly used is a chronomere, and the clock number 90000 in 1 second is exactly the timescale of video; Audio frequency sample rate commonly used is a chronomere, and its timescale is exactly the hits in 1 second, is 48000/ second as sample rate, and then the hits 48000 in 1 second is the timescale of audio frequency.
The time scale of CMMB described in the step B, the CMMB chronomere number that the expression per second takes place according to CMMB standard " Mobile Multimedia Broadcasting part 2: multiplexing ", is 22500.
According to this method, but pairing each the multiplexed sub frame independent process of each layering of SVC, and coupling does not rely on each other.The CMMB standard is regulation not, how the initial value that the professional CMMB airtime stabs determines (the initial reproduction time in the multiplexed sub frame is at each multiplexed sub frame rather than at business), between the SVC individual-layer data by multiplexed subframe transmission, do not have the method that existing assurance is synchronous.This method is multiplied by the CMMB airtime that the CMMB time scale converses SVC video unit correspondence with the NTP time and stabs by direct, determined that with the NTP time of first SVC video unit in handled first multiplexed sub frame the professional CMMB airtime stabs initial value, and determined that by this conversion method the CMMB airtime of follow-up SVC video unit stabs.For present embodiment, for certain video access unit, base layer data RTP timestamp is Tbase, enhancement data RTP timestamp is Text, with the audio frequency RTP timestamp Raudio on its synchronization point, the RTCP message that encoder sends, guarantee Tbase, Text, the pairing separately NTP time unanimity of Taudio (equal or approximately equal of pairing separately NTP time, allow a little deviate), after the method for the invention processing, the base layer data of video flowing, voice data on enhancement data and the synchronous points, when going in being encapsulated into multiplexed sub frame, it is consistent (equate or approximately equal, allow a little deviate) that its corresponding CMMB airtime stabs, although they are encapsulated in different multiplexed sub frames, multiplexed sub frame is an independent process separately.This method can guarantee the unification that each the layering business datum of SVC on the same time point was stabbed in the CMMB airtime, thereby synchronous assurance means between the different layerings of SVC are provided.
As shown in Figure 4, when receiving on the terminal, the multiplexed frame 0 in the terminal monitoring broadcast channel frame, receiving control information and ESG information, with correct receiving multimedia broadcast business, listen to simultaneously, also comprise the positional information of the multiplexed sub frame at each layer bit stream place in the SVC business.Terminal is according to self needs, and as its video flow processing ability or Network Transmission situation, decision receives basic layer bit stream, or receives basic layer bit stream and corresponding enhancement layer bitstream.In the present embodiment, establishing terminal is the net book that can handle the VGA video, need receive the basic stream of QVGA and the enhanced flow of VGA simultaneously and process, and decoding represents the VGA video.After terminal receives the basic stream and enhanced flow that transmits from different multiplexed sub frames, video unit is extracted from multiplexed sub frame, parse H264 basic coding unit NALU, the NALU that will belong to different layerings stabs alignment, synchronous by the CMMB airtime, the NALU that will belong to same video access unit afterwards merges, and carries out video decode then and represents.
On the terminal, the carrying out by the following method synchronously of SVC each minute layer service:
A, each video unit all have the corresponding CMMB airtime to stab, and this timestamp is that the interior initial reproduction time of multiplexed sub frame and the relative reproduction time of each video unit correspondence combine.NALU data that video unit comprised are stabbed together with its time corresponding put into buffering area, buffering area can be basic layer, enhancement layer separately independently.For some reason, the SVC individual-layer data of synchronization can not guarantee to receive simultaneously fully, if the time of reception of each layering business datum, the threshold value of difference was Td, promptly allow to belong to the data of same video access unit, time difference between data of receiving the earliest and the data of receiving the latest is Td, and then buffering area should be able to hold the data of all layerings of SVC in the Td time range;
B, every a Fixed Time Interval, terminal is just got a video access unit from aforementioned buffering area, after the NALU alignment of data of the different layerings of video access unit is merged, sends into decoder.Be to guarantee the decoder smoothing processing, can after buffer data is no less than the tolerance data of aforesaid Td time, just begin to carry out the operation of fetching data the first time.Get the base layer data that belongs to a video access unit earlier, timestamp with base layer data is a benchmark, take out the enhancement data corresponding with the base layer data timestamp, with its data as same video access unit, merge with base layer data, data after merging as a complete video access unit, are sent into decoder decode.
In step B, for a certain reason, the temporal information that the RTP that encoding device sends, RTCP are entrained, may not guarantee that base layer data, enhancement data and voice data in the same sampling instant are temporal in full accord at NTP, the CMMB airtime of changing out on headend equipment like this stabs also just can not be in full accord, and have a little deviation, when terminal is looked for corresponding enhancement data according to the timestamp of base layer data, also establish a timestamp tolerance.
The present invention not only is applicable to the CMMB system, also is fit to other mobile multimedia broadcast systems.
The above is preferred embodiment of the present invention only, is not to be used to limit protection scope of the present invention.
Claims (14)
1. realize the method that the scalable video service synchronization sends in a Mobile Multimedia Broadcasting, it is characterized in that, comprising:
The video traffic coding is generated multipath media stream, with described multipath media stream is that unit is encapsulated in respectively in the different multiplexed sub frames in the broadcast channel frame with the media units, also carry the Mobile Multimedia Broadcasting timestamp of each media units that encapsulates in this multiplexed sub frame in the described multiplexed sub frame, and the Mobile Multimedia Broadcasting timestamp of the Media Stream of same sampling instant is synchronous, and the positional information of the multiplexed sub frame at described Media Stream place is packaged in the described broadcast channel frame, described broadcast channel frame is sent to receiving terminal, described Media Stream comprises basic layer bit stream and the corresponding enhancement layer bitstream thereof that the video flowing coding of described video traffic generates, perhaps, the audio stream that comprises video traffic, basic layer bit stream and corresponding enhancement layer bitstream thereof that the video flowing coding of video traffic generates.
2. the method for claim 1 is characterized in that, as follows described Media Stream is encapsulated in the described multiplexed sub frame:
Described multipath media stream is encapsulated as multichannel real-time transport protocol (rtp) code stream, wherein, every road RTP code stream is followed one road RTCP Real-time Transport Control Protocol (RTCP) code stream, and described RTCP code stream guarantees NTP (Network Time Protocol) (NTP) time synchronized of the Media Stream of same sampling instant;
Extract the Media Stream that encapsulates in the described RTP code stream, with described Media Stream is that unit is encapsulated in respectively in the different multiplexed sub frames in the broadcast channel frame with the media units, the RTP timestamp of described Media Stream is converted to the NTP time, again be converted to the Mobile Multimedia Broadcasting timestamp under the benchmark unified time NTP time, described Mobile Multimedia Broadcasting timestamp be encapsulated in the multiplexed sub frame at its corresponding media units place.
3. method as claimed in claim 2 is characterized in that, as follows the RTP timestamp is converted to the Mobile Multimedia Broadcasting timestamp:
To each media units, take out the RTP timestamp of its place RTP bag, the relevant temporal information of the RTCP bag that transmits in conjunction with the RTCP code stream of its place RTP code stream correspondence, the NTP time of calculating this media units;
The NTP time and the Mobile Multimedia Broadcasting time scale of this media units are multiplied each other, obtain the Mobile Multimedia Broadcasting timestamp of described media units.
4. the method for claim 1, it is characterized in that the Mobile Multimedia Broadcasting timestamp of the Media Stream of described same sampling instant is meant synchronously: the difference between the Mobile Multimedia Broadcasting timestamp value of the Media Stream of same sampling instant is in default timestamp tolerance scope.
5. realize the system that the scalable video service synchronization sends in a Mobile Multimedia Broadcasting, it is characterized in that described system comprises encoding device and front end transmitting apparatus, wherein:
Described encoding device is used for the video traffic coding is generated multipath media stream; Described Media Stream comprises basic layer bit stream and the corresponding enhancement layer bitstream thereof that the video flowing coding of described video traffic generates, perhaps, the basic layer bit stream and the corresponding enhancement layer bitstream thereof that comprise the video flowing coding generation of the audio stream of described video traffic, described video traffic;
Described front end transmitting apparatus, being used for described Media Stream is that unit is encapsulated in respectively in the different multiplexed sub frames of broadcast channel frame with the media units, the Mobile Multimedia Broadcasting timestamp of each media units of encapsulating in this multiplexed sub frame is encapsulated in the described multiplexed sub frame, and the Mobile Multimedia Broadcasting timestamp of the Media Stream of same sampling instant is synchronous, and the positional information of the multiplexed sub frame at described Media Stream place is packaged in the described broadcast channel frame, described broadcast channel frame is sent to receiving terminal.
6. system as claimed in claim 5 is characterized in that,
Described encoding device comprises coding unit and encapsulation unit, wherein:
Described coding unit is used for the video traffic coding is generated multipath media stream;
Described encapsulation unit, be used for described multipath media stream is encapsulated as multichannel real-time transport protocol (rtp) code stream, wherein, every road RTP code stream is followed one road RTCP Real-time Transport Control Protocol (RTCP) code stream, and described RTCP code stream guarantees NTP (Network Time Protocol) (NTP) time synchronized of the Media Stream of same sampling instant;
Described front end transmitting apparatus comprises first encapsulation unit, second encapsulation unit, converting unit, and the 3rd encapsulation unit and transmitting element, wherein:
First encapsulation unit is used for extracting the Media Stream that described RTP code stream encapsulates, and is that unit is encapsulated in respectively in the different multiplexed sub frames in the broadcast channel frame with the media units with described Media Stream;
Described second encapsulation unit is used for the positional information of the multiplexed sub frame at described Media Stream place is encapsulated in described broadcast channel frame;
Described converting unit is used for the RTP timestamp of described Media Stream is converted to the NTP time, is converted to the Mobile Multimedia Broadcasting timestamp under the benchmark unified time NTP time again;
Described the 3rd encapsulation unit is used for described Mobile Multimedia Broadcasting timestamp is encapsulated into the multiplexed sub frame at its corresponding media units place;
Described transmitting element is used for described broadcast channel frame is sent to receiving terminal.
7. system as claimed in claim 6 is characterized in that,
Described converting unit comprises first converting unit and second converting unit, wherein:
Described first converting unit is used for each media units, takes out the RTP timestamp of its place RTP bag, the relevant temporal information of the RTCP bag that transmits in conjunction with the RTCP code stream of its place RTP code stream correspondence, the NTP time of calculating this media units;
Described second converting unit is used for the NTP time and the Mobile Multimedia Broadcasting time scale of described media units are multiplied each other, and obtains the Mobile Multimedia Broadcasting timestamp of described media units.
8. as claim 5,6 or 7 described systems, it is characterized in that the Mobile Multimedia Broadcasting timestamp of the Media Stream of described same sampling instant is meant synchronously: the difference between the Mobile Multimedia Broadcasting timestamp value of the Media Stream of same sampling instant is in default timestamp tolerance scope.
9. realize the scalable video service receiving method in a Mobile Multimedia Broadcasting, it is characterized in that, comprising:
The positional information of the multiplexed sub frame at each layer bit stream place of video flowing in the video traffic in the receiving terminal monitoring broadcast channel frame;
Receiving terminal receives basic layer bit stream according to the video flow processing ability of self, perhaps, receives basic layer bit stream and corresponding enhancement layer bitstream;
When receiving the enhancement layer bitstream of basic layer bit stream and correspondence, after each layer bit stream merged according to Mobile Multimedia Broadcasting timestamp alignment, basic layer bit stream and corresponding enhancement layer bitstream are decoded, export the video data after basic layer bit stream and enhancement layer bitstream merge.
10. method as claimed in claim 9 is characterized in that, basic layer bit stream of described reception and corresponding enhancement layer bitstream comprise each layer bit stream according to Mobile Multimedia Broadcasting timestamp alignment merging:
Receiving terminal will receive basic layer bit stream and enhancement layer bitstream, deposit in the buffering area;
From buffering area, take out the basic layer bit stream data that belong to a video access unit, Mobile Multimedia Broadcasting timestamp with described basic layer bit stream data is a benchmark, the synchronous enhancement layer bitstream data of Mobile Multimedia Broadcasting timestamp of taking-up and described basic layer bit stream data merge its data as same video access unit.
11. method as claimed in claim 10 is characterized in that,
The synchronous enhancement layer bitstream data of the Mobile Multimedia Broadcasting timestamp of described and described basic layer bit stream data are meant, with the enhancement layer bitstream data of difference in default timestamp tolerance scope of the Mobile Multimedia Broadcasting timestamp of described basic layer bit stream data.
12. realize the scalable video service receiving device in the Mobile Multimedia Broadcasting, it is characterized in that, comprising:
Monitoring unit, the positional information of the multiplexed sub frame at each layer bit stream place of video flowing in the video traffic in the monitoring broadcast channel frame;
Receiving element receives basic layer bit stream according to self video flow processing ability, perhaps, receives basic layer bit stream and corresponding enhancement layer bitstream;
The alignment merge cells is used for each layer bit stream is merged according to the alignment of Mobile Multimedia Broadcasting timestamp;
Decoding unit is decoded to basic layer bit stream and corresponding enhancement layer bitstream, exports the video data after basic layer bit stream and enhancement layer bitstream merge.
13. device as claimed in claim 12 is characterized in that,
Described receiving element, the basic layer bit stream and the enhancement layer bitstream that also are used for receiving deposit buffering area in;
Described alignment merge cells, be used for taking out the basic layer bit stream data that belong to a video access unit from buffering area, Mobile Multimedia Broadcasting timestamp with described basic layer bit stream data is a benchmark, the synchronous enhancement layer bitstream data of Mobile Multimedia Broadcasting timestamp of taking-up and described basic layer bit stream data merge its data as same video access unit.
14. device as claimed in claim 13 is characterized in that,
The synchronous enhancement layer bitstream data of the Mobile Multimedia Broadcasting timestamp of described taking-up and described basic layer bit stream data are meant, the enhancement layer bitstream data of the difference between the Mobile Multimedia Broadcasting timestamp of taking-up and described basic layer bit stream data in default timestamp tolerance scope.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010290881.7A CN101951506B (en) | 2010-09-17 | 2010-09-17 | System and method for realizing synchronous transmitting and receiving of scalable video coding service |
| PCT/CN2011/076622 WO2012034442A1 (en) | 2010-09-17 | 2011-06-30 | System and method for realizing synchronous transmission and reception of scalable video coding service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010290881.7A CN101951506B (en) | 2010-09-17 | 2010-09-17 | System and method for realizing synchronous transmitting and receiving of scalable video coding service |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101951506A true CN101951506A (en) | 2011-01-19 |
| CN101951506B CN101951506B (en) | 2014-03-12 |
Family
ID=43454846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010290881.7A Expired - Fee Related CN101951506B (en) | 2010-09-17 | 2010-09-17 | System and method for realizing synchronous transmitting and receiving of scalable video coding service |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN101951506B (en) |
| WO (1) | WO2012034442A1 (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012034442A1 (en) * | 2010-09-17 | 2012-03-22 | 中兴通讯股份有限公司 | System and method for realizing synchronous transmission and reception of scalable video coding service |
| WO2012068898A1 (en) * | 2010-11-24 | 2012-05-31 | 中兴通讯股份有限公司 | Method, apparatus and system for synchronizing tiered service in mobile multimedia broadcasting |
| CN102510488A (en) * | 2011-11-04 | 2012-06-20 | 北京播思软件技术有限公司 | Method and device for synchronizing video and audio by utilizing broadcasting characteristics |
| CN102665108A (en) * | 2012-04-10 | 2012-09-12 | 中国联合网络通信集团有限公司 | Processing method, processing device and processing system of mobile video service |
| CN102761776A (en) * | 2012-08-01 | 2012-10-31 | 重庆大学 | Video and audio synchronizing method of P2PVoD (peer-to-peer video on demand) system based on SVC (scalable video coding) |
| CN105611222A (en) * | 2015-12-25 | 2016-05-25 | 北京紫荆视通科技有限公司 | Voice data processing method, device and system and controlled device |
| WO2016192413A1 (en) * | 2015-06-04 | 2016-12-08 | 中兴通讯股份有限公司 | Bitstream alignment and synchronous processing method and system, receiving terminal and communication system |
| CN106231317A (en) * | 2016-09-29 | 2016-12-14 | 三星电子（中国）研发中心 | Video processing, coding/decoding method and device, VR terminal, audio/video player system |
| CN106507112A (en) * | 2015-09-07 | 2017-03-15 | 中兴通讯股份有限公司 | Code flow processing method, apparatus and system |
| CN112383816A (en) * | 2020-11-03 | 2021-02-19 | 广州长嘉电子有限公司 | ATSC system signal analysis method and system based on android system intervention |
| CN112564837A (en) * | 2019-09-25 | 2021-03-26 | 杭州海康威视数字技术股份有限公司 | Multi-path data flow synchronization method and multi-path data flow synchronization step-by-step transmission system |
| CN112599138A (en) * | 2020-12-08 | 2021-04-02 | 北京百瑞互联技术有限公司 | Multi-PCM signal coding method, device and medium of LC3 audio coder |
| CN112825513A (en) * | 2019-11-21 | 2021-05-21 | 深圳市中兴微电子技术有限公司 | Method, device and equipment for transmitting multi-channel data and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1868213A (en) * | 2003-09-02 | 2006-11-22 | 索尼株式会社 | Content receiving apparatus, video/audio output timing control method, and content providing system |
| CN101359974A (en) * | 2007-07-31 | 2009-02-04 | 北京新岸线移动多媒体技术有限公司 | High-efficient source adaptation method suitable for LDPC block coding in T-MMB system |
| CN101449584A (en) * | 2006-05-26 | 2009-06-03 | 英国电讯有限公司 | Video processing |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101179736B (en) * | 2006-11-08 | 2010-07-14 | 中兴通讯股份有限公司 | Method for converting transmission stream program to China mobile multimedia broadcasting program |
| CN101394555B (en) * | 2008-10-24 | 2010-06-09 | 清华大学 | High error tolerant low time delay video transmission method and device suitable for deep space communication |
| CN101742246B (en) * | 2009-12-01 | 2012-12-12 | 中广传播集团有限公司 | System and method for realizing interactive service of mobile multimedia broadcast |
| CN101951506B (en) * | 2010-09-17 | 2014-03-12 | 中兴通讯股份有限公司 | System and method for realizing synchronous transmitting and receiving of scalable video coding service |
-
2010
- 2010-09-17 CN CN201010290881.7A patent/CN101951506B/en not_active Expired - Fee Related
-
2011
- 2011-06-30 WO PCT/CN2011/076622 patent/WO2012034442A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1868213A (en) * | 2003-09-02 | 2006-11-22 | 索尼株式会社 | Content receiving apparatus, video/audio output timing control method, and content providing system |
| CN101449584A (en) * | 2006-05-26 | 2009-06-03 | 英国电讯有限公司 | Video processing |
| CN101359974A (en) * | 2007-07-31 | 2009-02-04 | 北京新岸线移动多媒体技术有限公司 | High-efficient source adaptation method suitable for LDPC block coding in T-MMB system |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012034442A1 (en) * | 2010-09-17 | 2012-03-22 | 中兴通讯股份有限公司 | System and method for realizing synchronous transmission and reception of scalable video coding service |
| WO2012068898A1 (en) * | 2010-11-24 | 2012-05-31 | 中兴通讯股份有限公司 | Method, apparatus and system for synchronizing tiered service in mobile multimedia broadcasting |
| CN102510488A (en) * | 2011-11-04 | 2012-06-20 | 北京播思软件技术有限公司 | Method and device for synchronizing video and audio by utilizing broadcasting characteristics |
| CN102510488B (en) * | 2011-11-04 | 2015-11-11 | 播思通讯技术（北京）有限公司 | A kind of utilize broadcast characteristic to carry out audio-visual synchronization method and device |
| CN102665108A (en) * | 2012-04-10 | 2012-09-12 | 中国联合网络通信集团有限公司 | Processing method, processing device and processing system of mobile video service |
| CN102761776A (en) * | 2012-08-01 | 2012-10-31 | 重庆大学 | Video and audio synchronizing method of P2PVoD (peer-to-peer video on demand) system based on SVC (scalable video coding) |
| CN102761776B (en) * | 2012-08-01 | 2015-01-14 | 重庆大学 | Video and audio synchronizing method of P2PVoD (peer-to-peer video on demand) system based on SVC (scalable video coding) |
| US10582234B2 (en) | 2015-06-04 | 2020-03-03 | Zte Corporation | Bitstream alignment and synchronous processing method transmitting terminal, receiving terminal and communication system |
| WO2016192413A1 (en) * | 2015-06-04 | 2016-12-08 | 中兴通讯股份有限公司 | Bitstream alignment and synchronous processing method and system, receiving terminal and communication system |
| CN106507112A (en) * | 2015-09-07 | 2017-03-15 | 中兴通讯股份有限公司 | Code flow processing method, apparatus and system |
| WO2017041522A1 (en) * | 2015-09-07 | 2017-03-16 | 中兴通讯股份有限公司 | Streaming processing method, device and system |
| CN106507112B (en) * | 2015-09-07 | 2020-05-12 | 中兴通讯股份有限公司 | Code stream processing method, device and system |
| CN105611222A (en) * | 2015-12-25 | 2016-05-25 | 北京紫荆视通科技有限公司 | Voice data processing method, device and system and controlled device |
| CN106231317A (en) * | 2016-09-29 | 2016-12-14 | 三星电子（中国）研发中心 | Video processing, coding/decoding method and device, VR terminal, audio/video player system |
| CN112564837A (en) * | 2019-09-25 | 2021-03-26 | 杭州海康威视数字技术股份有限公司 | Multi-path data flow synchronization method and multi-path data flow synchronization step-by-step transmission system |
| CN112564837B (en) * | 2019-09-25 | 2022-05-06 | 杭州海康威视数字技术股份有限公司 | Multi-path data flow synchronization method and multi-path data flow synchronization step-by-step transmission system |
| CN112825513A (en) * | 2019-11-21 | 2021-05-21 | 深圳市中兴微电子技术有限公司 | Method, device and equipment for transmitting multi-channel data and storage medium |
| CN112825513B (en) * | 2019-11-21 | 2023-08-22 | 深圳市中兴微电子技术有限公司 | Method, device, equipment and storage medium for transmitting multipath data |
| CN112383816A (en) * | 2020-11-03 | 2021-02-19 | 广州长嘉电子有限公司 | ATSC system signal analysis method and system based on android system intervention |
| CN112599138A (en) * | 2020-12-08 | 2021-04-02 | 北京百瑞互联技术有限公司 | Multi-PCM signal coding method, device and medium of LC3 audio coder |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101951506B (en) | 2014-03-12 |
| WO2012034442A1 (en) | 2012-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101951506B (en) | System and method for realizing synchronous transmitting and receiving of scalable video coding service | |
| KR101639358B1 (en) | Transmission apparatus and method, and reception apparatus and method for providing 3d service using the content and additional image seperately transmitted with the reference image transmitted in real time | |
| CN101282482B (en) | Apparatus, system and method for synchronously playing video data and audio data | |
| CN101690200B (en) | Method and apparatus for providing high-quality multimedia service in dmb system | |
| US8009742B2 (en) | Method and system for retransmitting internet protocol packet for terrestrial digital multimedia broadcasting service | |
| CN103155584A (en) | Method for synchronizing multimedia flows and corresponding device | |
| CN101945261A (en) | Hierarchical delivery and receiving method and device in mobile multimedia broadcasting system | |
| CN104038844A (en) | Mobile live broadcast system based on MPEG-2 standard | |
| CN102714754A (en) | Remote re-multiplexing of transport streams | |
| CN1972453B (en) | A data flow packaging method of mobile multimedia broadcasting system | |
| CN105745899A (en) | Apparatus for transmitting broadcast signals, apparatus for receiving broadcast signals, method for transmitting broadcast signals and method for receiving broadcast signals | |
| CN106134203A (en) | Broadcast singal dispensing device, broadcast receiver, broadcast singal sending method and broadcast signal received method | |
| CN101729908A (en) | Synchronous multiplexing method for video and audio of transmission stream | |
| CN101150383A (en) | IP data transmission method | |
| CN101729887B (en) | Data transmission method and data transmission device of digital broadcasting system | |
| CN102098542A (en) | Multi-media data transmission method, system and device | |
| CN101754024B (en) | Multiplexing device and multiplexing method | |
| CN102404560B (en) | Method and system for implementing collaborative transmission of SVC (Scalable Video Coding) service | |
| CN102480634B (en) | The method, apparatus and system that in Mobile Multimedia Broadcasting, classified service is synchronous | |
| US20100315486A1 (en) | Stereoscopic video service providing/receiving method and apparatus in digital broadcasting system | |
| CN101453653B (en) | Method for spreading digital audio and video parameter set | |
| CN102572521B (en) | Method and system for automatically distributing code rate of basic video code stream of transmission stream | |
| CN100561931C (en) | Adopt elongated frame to improve the mobile multimedia broadcast system and the method thereof of transmission efficiency | |
| KR20060089508A (en) | Apparatus and method for creating transport stream for efficient transmission of timing information, and dmb transmission system using it | |
| KR100950771B1 (en) | Apparatus and method for transmitting and receiving the broadcasting signal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140312 Termination date: 20170917 |