CN105991586B - A kind of Multimedia Clock coordinate synchronization method under heterogeneous network - Google Patents

Abstract

The present invention provides a kind of Multimedia Clock coordinate synchronization methods under heterogeneous network, this method lacks the compatible mechanism of multi-clock for existing multimedia system and agreement, causing in isomery converged network can not content and the problem of service in precise integration heterogeneous networks, using data package module, data transmission blocks, data reception module, data parsing and presentation module, compatible multiple network clock synchronization mechanism under heterogeneous network is realized by increasing Multimedia Clock coordinate synchronization mechanism, using unified encapsulation format multi-medium data, the signaling table of the corresponding relationship of different clocks is provided, both it is conducive to integrate multiple network resources, improve the rich and stability of multimedia service under isomery converged network, it is also convenient for the unified management of multimedia content, reduce the time of Data duplication encapsulation and processing caused by due to clock synchronization mechanism difference And the wasting of resources, better service experience is brought to user.

Description

A kind of Multimedia Clock coordinate synchronization method under heterogeneous network

Technical field

The present invention relates to a kind of in heterogeneous network realizes the mechanism of Multimedia Clock coordinate synchronization, more specifically, relating to And it is a kind of in Two-way Network and the Multimedia Clock coordination of realization the whole network in the mixing that unidirectional network forms, heterogeneous network and synchronous machine System.

Background technique

With the maturation and commonly used, especially fine definition, high resolution digital video industry of digital multimedia technology The extensively and profoundly development of business, while user has experienced better digital media service, also proposes digital media business Higher requirement.Due to the extensive use of digital media technology, synchronous between being flowed in multimedia programming and stationary problem also day in flowing Benefit highlights.More importantly in the existing unidirectional network using television network broadcast as representative and using broadband network as representative In Two-way Network, using different multimedia time synchronization mechanisms and agreement, causes two kinds of network multimedia services incompatible, make User cannot enjoy isomery converged network business bring advantage and convenience.

Current most of multi-media protocols both define the Time Synchronization Mechanism of respective complete set, can be acceptable Relatively successfully resolution protocol inner multimedia system clock stationary problem in range, such as the common MPEG2-TS of digital TV field Agreement combines PCR (Program Clock in Media Stream using 27M system clock STC (System Time Clock) Reference) synchronous with PTS (Presentation Time Stamp) the information progress clock of multimedia service；Broadband network In common HTTP, combined using Coordinated Universal Time(UTC) UTC (Coordinated Universal Time) in Real-time Transport Protocol it is unified The clock that Network Time Protocol NTP (Network Time Protocol) time stamp completes multimedia service is synchronous.Even so, In view of public users are indifferent to the particular technique of multimedia service, service quality and effect are only focused on, so in actual conditions Middle user is often in isomery hybrid network, and it is desirable that enjoys indifference, stable multimedia service.Problem is not Same network system, in other words in different agreement clock synchronization mechanism incompatibility, cause user the same time can only enjoy certain A kind of business, and the wider media business of isomery hybrid network bring and higher-quality media services can not be enjoyed.It is more Net fusion has become trend, and heterogeneous networks have respective advantage, complement each other, and isomery converged network should be user and provide more Good service, however different clock synchronization mechanisms is difficult to realize this target, multimedia system are difficult to accurately to match, are whole Close the content and service in heterogeneous networks.

On the other hand, the rich of digital media resource meets the growing media consumption demand of user, media money Content association between source also provides the foundation for the integration of media resource.Nowadays multimedia service is no longer by physical network Limitation, the multimedia service for comparing high quality can be enjoyed in heterogeneous networks, but accordingly produces another and asks Topic, identical data content need to be packaged into different media formats in heterogeneous networks different agreement, not only increase additional Workload, and different media formats cause user that must rely on physical condition ability since clock information is mutually incompatible Obtain multimedia service, do not meet Digital Media consumption at any time, everywhere, any terminal, the trend of any content.

Better method is that multi-medium data uses unified encapsulation format, which can be compatible with existing a variety of Clock information, while the Time Synchronization Mechanism of Two-way Network and unidirectional network, media should be compatible in the multimedia system of heterogeneous network Transmitting terminal all sends data, calibration time stamp using the transport protocol based on IP (Internet Protocol), while providing difference The signaling table of the corresponding relationship of clock, clock protocols used by client can adjust as needed and mechanism.For example, in width When band network disconnects, user is not available Network Time Protocol completion clock and synchronizes, but still can be by other clock synchronization protocol TV programme are watched using broadcasting network.Its difficulty is: existing multimedia system is typically all to correspond to certain particular network, Need to design the multimedia system under reasonable isomery hybrid network, to compatible heterogeneous networks characteristic；Existing mature protocol is all The demand for not accounting for heterogeneous network needs to redesign media encapsulation format and biography according to the requirement of compatible different clocks mechanism Defeated agreement.

Summary of the invention

For the defects in the prior art, the object of the present invention is to provide the Multimedia Clock coordinations under a kind of heterogeneous network Synchronous method solves existing multimedia to increase the multi-clock relation mechanism that existing multimedia system and agreement lack The problem of system and agreement can not provide isomery converged network association service since synchronised clock is incompatible, and be a new generation IP-based heterogeneous network provides clock synchronization solutions.

To achieve the above object, the invention adopts the following technical scheme:

A kind of Multimedia Clock coordinate synchronization method under heterogeneous network, the method use four resume modules: data Package module, data transmission blocks, data reception module, data parsing and presentation module, in which:

Original multi-medium data is carried out unified format and is encapsulated into MPU number by the data package module, the module According to, MPU data include the other timestamp information of frame level, by the relevant Information encapsulation of all media in file, including multimedia envelope File and signaling file are filled, is submitted necessary information for data parsing with module is presented, especially the compatibility timestamp information of standard；

The data transmission blocks, the data which sends include clock associated description signaling, the matchmaker for marking timestamp Volume data, packets fields format is corresponding with the data format of data package module, guarantees the text that data package module generates Part necessary information can correctly be transmitted to client, while provide for data receiver and recovery module unpacking with data recovery operation Reference format；

The data receiver and recovery module, the module receive, restore media data and signaling information, calibrate receiving end sheet Ground clock encapsulates data the various media informations that module is provided with sending module and is correctly restored locally, guide data Parsing is realized correctly output with presentation module and is presented；

The data parsing parses signaling table clock relevant information with module, the module is presented, and guidance decoder correctly solves Code is presented with browser, and by data transmission blocks and data receiver and recovery module, which encapsulates data module and include File content and related signaling control completely correctly presented by receiving end, the advantage of this system is intuitively presented to use Family.

Preferably, the data package module, in which: MPU format is based on ISOBMFF (ISO Base Media File Format), increase new box type on this basis, while being divided into time series data and non-sequential data, to all original Data type carries out the encapsulation of unified format, and wherein time series data predominant package audio-video source data, add other than media track Add new hint track, and the PTS clock information of media content is provided in hint track sample, rather than the coding of time series data It is uncorrelated to clock with presenting, so not needing setting PTS clock information；PTS clock information is calculated by encoder clock source It arrives, corresponds to PTS clock information in data frame level when media data is packaged into MPU format in this way.

Preferably, the data transmission blocks, wherein each field in transport protocol part of MMT all encapsulates lattice with unified media The time series data field of formula MPU is consistent, reaches the consistency of multi-medium data storage and transmission and is quickly converted, transmits more matchmakers While volume data, NTP service is also being carried out, and receiving end local clock is carried out periodical correction, can prevent error time Cumulative effect, guarantee the quality of multimedia service.

Preferably, the data transmission blocks utilize MMT agreement when receiving end is unsatisfactory for the condition of NTP service operation In CRI signaling table, provide the corresponding relationship of NTP and PCR, receiving end is converted by receiving the NTP time stamp in stream according to this table PCR information out reaches clock synchronization and multimedia synchronous, CRI signaling table periodically updates, protect to correct local clock Demonstrate,prove the precise synchronization of whole network clock.

Preferably, the data receiver and recovery module, with the NTP clock or clock time stamp in obtained Media Stream For correcting the clock of receiving end, make the time synchronization of the whole network, meanwhile, the other presentation time stamp guidance solution of frame level in media data Code device is correctly decoded.

Preferably, it is same to carry out clock according to STC mechanism when being in suspension state for the data parsing and presentation module Step: parsing CRI signaling table obtains PCR information, corrects local system clock, while utilizing the other PTS of frame level in media data Information knows that decoder decodes；If being in connected state, ignores the PCR and PTS information in data, carried out using Network Time Protocol Clock is synchronous: receiving end services regular calibration local clock using NTP, guides solution using the timeline administrative mechanism in Media Stream Code device decoding.

Compared with prior art, the present invention have it is following the utility model has the advantages that

Using technical solution of the present invention, the compatible machine of multi-clock can be lacked for existing multimedia system and agreement System, cause in isomery converged network can not content and the problem of service in precise integration heterogeneous networks, when by increasing multimedia Clock coordination synchronization mechanism realizes compatible multiple network clock synchronization mechanism under heterogeneous network, using unified encapsulation format multimedia Data provide the signaling table of the corresponding relationship of different clocks, are both conducive to integrate multiple network resources, improve under isomery converged network The rich and stability of multimedia service is also convenient for the unified management of multimedia content, reduces because of clock synchronization mechanism difference Time and the wasting of resources of the encapsulation of Data duplication caused by and and processing, bring better service experience to user.

Detailed description of the invention

Upon reading the detailed description of non-limiting embodiments with reference to the following drawings, other feature of the invention, Objects and advantages will become more apparent upon:

Fig. 1 is clock coordinate synchronization schematic diagram in multimedia system；

Fig. 2 is general MPU (Media Processing Unit) encapsulation format；

Fig. 3 is the synchronous flow chart of receiving end clock；

Fig. 4 is modification schematic diagram consistent with the time series data field of unified media encapsulation format MPU.

Specific embodiment

The present invention is described in detail combined with specific embodiments below.Following embodiment will be helpful to the technology of this field Personnel further understand the present invention, but the invention is not limited in any way.It should be pointed out that the ordinary skill of this field For personnel, without departing from the inventive concept of the premise, various modifications and improvements can be made.These belong to the present invention Protection scope.

Nowadays, the network integration has become trend, advantage integration is carried out using the resource of the whole network, wherein the pass for needing to solve Keyness problem is to design the clock coordinate synchronization mechanism of the whole network.In view of the compatibility to existing network, the whole network clock machine System should be able to be compatible with existing clock protocols and mechanism.

There are two the clock system and agreement of existing mainstream are main, one is MPEG2-TS used in radio data system Standard, clock synchronization section therein have used STC/PCR/PTS mechanism, using transmission regular calibration mode in stream；Another It is NTP used in internet system (Network Time Protocol) agreement, using the side of the outer transmission regular calibration of stream Formula.So the Multimedia Clock synchronization mechanism under heterogeneous network should be compatible with both the above clock protocols, and two kinds of clocks should be Global mutually mapping, determines the clock protocols really used with condition according to actual needs.

Below by taking MMT (MPEG Media Transport) standard as an example, the Multimedia Clock association under heterogeneous network is introduced Adjust synchronization mechanism.Certainly, this mechanism is not limited to MMT standard, also can be applied to other standards and scheme.MMT full name is MPEG Media transport protocol is formulated by international standard group MPEG, and media can be carried out to unified encapsulation, allow media data two-way Net and unidirectional transfers on network, and transmission method and presentation mechanism have been redefined for emerging media application scene.

It requires to modify to existing system and agreement in the different links of multimedia system, below with a set of basic For multimedia system, as shown in Figure 1, being divided into data encapsulation, data transmission, data receiver, data parsing and being presented module 4 Module clock synchronization clock coordination synchronization mechanism is described respectively.

Data encapsulation:

Unified MPU (Media Processing Unit) file is generated, includes the other timestamp information of frame level

By encoder, original audio, video flowing are generated, unified format is carried out and is encapsulated into MPU data.MPU lattice Formula is based on ISOBMFF (ISO Base Media File Format), increases new box type on this basis, is divided into simultaneously Time series data and non-sequential data carry out the encapsulation of unified format to all original data types, as shown in Figure 2.When wherein Ordinal number adds new hint track according to predominant package audio-video source data other than media track, and in hint track sample The PTS clock information of media content is provided, rather than the coding of time series data and presentation are uncorrelated to clock, so not needing to be arranged PTS information.PTS clock information is calculated by encoder 27M clock source, so when media data is packaged into MPU format Through corresponding to PTS clock information in data frame level.Shown in MPU document location Fig. 2, specific MMTHSample's sets PTS information It counts as follows:

The field semantics newly increased are as follows, other field semantics refer to ISO/IEC 23008-1:

This 33 bit field of PTS (presentation_time_stamp)-is the presentation time of cited access unit Stamp.

The coding semantic description of PTS field refers to ISO/IEC 13818-1.

Reserved-reserved bit.

Data are sent:

The transmission of the media data of transmission comprising clock associated description signaling and label time stamp

Due to the extensive use of IP agreement, following isomery converged network should be also based on IP agreement, no matter which kind of All current timestamp can be stamped for data packet when application layer protocol transmitting multimedia data, receiving end can be according to the timestamp It is compared with local clock and completes the multimedia accurate presentation of timing.And current application layer protocol generally all fixes a kind of clock association View, wherein the transport protocol part of MMT also only considered the label of NTP time stamp, in the timing multimedia application of heterogeneous network, It has to consider to be unsatisfactory for the synchronization mechanism with multimedia system when NTP clock source communication condition, and to be compatible with multiple network Clock protocols need to modify this partial content, and 33 PTS fields, record are added after the packet header of original time series data MFU The presentation time stamp of the frame data and 7 reserved fields, it is reserved for the following extension, as shown in Figure 4.Each field of Fig. 4 is all and uniformly The time series data field of media encapsulation format MPU is consistent, can achieve multi-medium data storage with transmission consistency and fastly Speed conversion.While transmitting multi-medium data, NTP service is also being carried out, and receiving end local clock is carried out periodical correction, energy The cumulative effect for enough preventing error time guarantees the quality of multimedia service.

If receiving end is unsatisfactory for the condition of NTP service operation, it is same with multimedia that above-mentioned mechanism will not can solve clock synchronization The problem of step.It can use the CRI signaling table in MMT agreement at this time, the corresponding relationship of NTP and PCR be provided, receiving end can lead to Cross the NTP time stamp received in stream and converse PCR information according to this table, to correct local 27M clock, reach clock it is synchronous with Multimedia synchronous.CRI signaling table periodically updates, and guarantees the precise synchronization of whole network clock.NTP and PCR corresponding relationship in MMT Such as following table, STC_sample corresponds to PCR value, and NTP_timestamp_sample corresponds to NTP value, is specifically detailed in ISO/IEC 23008-1。

Data receiver and recovery:

It receives, restore media data and signaling information, calibrate receiving end local clock

No matter receiving end, which is taken the initiative, obtains or passively receives data, all should correctly recover media data and signaling letter Breath.NTP clock or 27M System Clock time stamp in obtained Media Stream are used to correct the clock of receiving end, make the whole network Time synchronization.Meanwhile the other presentation time stamp guidance decoder of frame level in media data is correctly decoded.

Data parsing and presentation:

Signaling table clock relevant information is parsed, guidance decoder is correctly decoded to be presented with browser

Since client there are many forms of expression, in multimedia system by taking browser as an example, illustrates data parsing and present The treatment process of module clock coordinate synchronization mechanism, can similarly be applied to special-purpose software client, special player etc..It connects Receiving end passes through terminal networking detection module first, confirms local connected state, while interface parameters and result being fed back to clear It lookes inside device.Browser judges locally whether network, if networking, holding and ntp server periodical communication, to correct local UTC clock, while MMTP stream is constantly received and reading, ignore PCR and PTS information in data, utilizes CI (Composition Information the NTP time) and in MPU, instruct that decoder is correctly decoded and browser is correctly presented；It is first if not networking The CRI signaling table for first reading transmitting terminal periodicity sending, obtains the correct UTC of server end by wherein STC_sample field Time calibrates local system clock, while also constantly receiving and reading MMTP stream, obtains MPU data and metadata information, PTS information in middle metadata MFU comprising data frame refers to then on the basis of local system clock referring to PTS temporal information Lead that player is correctly decoded and browser is correctly presented, specific flow chart is shown in Fig. 3.In addition to there are also several for CRI signaling table in MMT Other kinds of signaling table such as CI (Composition Information) is described by the clock corresponding relationship of CRI, is received End can obtain the synchronization of the whole network, to preferably be brought using the rendering and presentation of various signalings control browser for user Completely new experience.

The present invention is both conducive to integrate multiple network resources, and multimedia service is rich and steady under raising isomery converged network It is qualitative, be also convenient for the unified management of multimedia content, reduce Data duplication encapsulation caused by due to clock synchronization mechanism difference and The time of processing and the wasting of resources bring better service experience to user.

Specific embodiments of the present invention are described above.It is to be appreciated that the invention is not limited to above-mentioned Particular implementation, those skilled in the art can make various deformations or amendments within the scope of the claims, this not shadow Ring substantive content of the invention.

Claims (4)

1. a kind of Multimedia Clock coordinate synchronization method under heterogeneous network, which is characterized in that the method uses four modules Processing: data package module, data transmission blocks, data reception module, data parsing and presentation module, in which:

Original multi-medium data is carried out unified format and is encapsulated into MPU data by the data package module, the module, MPU data include the other timestamp information PTS of frame level, by the relevant Information encapsulation of all media in MPU file, including multimedia Package file and signaling file submit necessary information for data parsing with module is presented, especially the other timestamp information of frame level PTS；

The data transmission blocks, the data which sends include clock associated description signaling, the media number for marking timestamp According to wherein packets fields format is corresponding with the data format of data package module, guarantees the MPU that data package module generates File necessary information can correctly be transmitted to client, while unpacking for data receiver with recovery module and mentioning with data recovery operation For reference format；Data, which are sent, is based on IP agreement, and while transmitting multi-medium data, NTP service is also being carried out；

The data receiver and recovery module, which receives, restores media data and signaling information, when calibrating client local Clock encapsulates data the various media informations that module is provided with data transmission blocks and is correctly restored locally, guide data Parsing is realized correctly output with presentation module and is presented；

Data parsing be presented module, which parses signaling table clock relevant information, guidance decoder be correctly decoded with Client is presented, and by data transmission blocks and data receiver and recovery module, which encapsulates data the MPU that module includes File content and related signaling control are completely correctly presented by client；

The data transmission blocks utilize the CRI signaling in MMT agreement when client is unsatisfactory for the condition of NTP service operation Table provides the corresponding relationship of NTP and PCR, and client converses PCR information according to this table by receiving the NTP time stamp in stream, from And local clock is corrected, reach clock synchronization and multimedia synchronous, CRI signaling table periodically update, guarantees whole network clock Precise synchronization；

The data parsing and presentation module, when being in suspension state, it is synchronous to carry out clock according to STC mechanism: parsing CRI believes It enables table obtain PCR information, corrects local system clock, while referring to using the other timestamp information PTS of frame level in media data Lead decoder decoding；If being in connected state, ignores the PCR and the other timestamp information PTS of frame level in data, assisted using NTP It is synchronous that view carries out clock: client services regular calibration local clock using NTP, utilizes the timeline administrative mechanism in Media Stream Guide decoder decoding.

2. the Multimedia Clock coordinate synchronization method under a kind of heterogeneous network according to claim 1, which is characterized in that institute State data package module, in which: MPU format is based on ISOBMFF, when increasing new box type on this basis, while being divided into Ordinal number evidence and non-sequential data carry out the encapsulation of unified format to all original data types, and wherein time series data mainly seals Audio-video source data are filled, new hint track are added other than media track, and provide media content in hint track sample The other timestamp information PTS of frame level, rather than the coding of time series data and present it is uncorrelated to clock, so do not need be arranged frame level Other timestamp information PTS；The other timestamp information PTS of frame level is calculated by encoder clock source, is encapsulated in this way in media data At corresponding to the other timestamp information PTS of frame level in data frame level when MPU format.

3. the Multimedia Clock coordinate synchronization method under a kind of heterogeneous network according to claim 1, which is characterized in that institute Data transmission blocks are stated, wherein time series data word of each field in transport protocol part of MMT all with unified media encapsulation format MPU Section is consistent, reaches the consistency of multi-medium data storage and transmission and is quickly converted, client local clock is carried out the period Property correction, the cumulative effect of error time can be prevented.

4. the Multimedia Clock coordinate synchronization method under a kind of heterogeneous network according to claim 1-3, special Sign is, the data receiver and recovery module, with the NTP clock or 27M System Clock time stamp in obtained Media Stream For correcting the clock of client, make the time synchronization of the whole network, meanwhile, the other timestamp information PTS of frame level in media data draws Decoder is led to be correctly decoded.