CN1767601A - Synchronous broadcast controlling method capable of supporting multi-source stream media - Google Patents

Synchronous broadcast controlling method capable of supporting multi-source stream media Download PDF

Info

Publication number
CN1767601A
CN1767601A CN 200510096215 CN200510096215A CN1767601A CN 1767601 A CN1767601 A CN 1767601A CN 200510096215 CN200510096215 CN 200510096215 CN 200510096215 A CN200510096215 A CN 200510096215A CN 1767601 A CN1767601 A CN 1767601A
Authority
CN
China
Prior art keywords
media
video
synchronous
stream
audio
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 200510096215
Other languages
Chinese (zh)
Other versions
CN100352270C (en
Inventor
郑庆华
刘均
张之明
江丕峻
王晨旭
彭挺
陈洪
杨冬
林杰
伍洋
张帆
吕伟梅
姜山
丁凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Jiaotong University
Original Assignee
Xian Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Jiaotong University filed Critical Xian Jiaotong University
Priority to CNB2005100962159A priority Critical patent/CN100352270C/en
Publication of CN1767601A publication Critical patent/CN1767601A/en
Application granted granted Critical
Publication of CN100352270C publication Critical patent/CN100352270C/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

This invention discloses a synchronous play control method for supporting multi-source flow media, in which, a device realizing said method includes a separator, a decoder set, a multi-source flow media module, a multi-source video flow combination module, an OSD and an audio filter lens, in which, the separator separates the video and audio data in the multiple local media files or the multiple flow media, the decoder set calls related decoders to decode them to be sent to the multi-source flow media synchronous module, which applies a system of multi-size hierarchical synchronous control to control the multi-route media flow synchronously among and in the media objects, the multi-source video combination module combines multiple videos to one route, the OSD module laminates the combined data, volume, the current play time or caption information to output the video, the audio filter lens carries out formation transformation and linear overlap to the multiple audios then to output the audio.

Description

A kind of synchronous broadcast controlling method of supporting multi-source flow media
Affiliated field
The invention belongs to Computer Design and applied technical field, relate to computer software, multimedia technology and look audio signal processing technique, particularly a kind of synchronous broadcast controlling method of supporting multi-source flow media.
Background technology
Development of multimedia technology impels people that complex media is presented improving constantly of demand, and becomes gradually that people obtain and transmission information, the effective means of networked autonomous learning of carrying out.Therefore, each operating system is all made every effort to always and can be reached perfect support to multimedia.
Multi-source flow media is a kind of two-way or the above stream medium data synchronous playing of two-way and complex media type that shows with ad hoc fashion (as " picture-in-picture ", multiwindow) supported.It has further improved the expressive ability and the information content of medium, is subjected to current people's common concern.
The applicant is new through looking into, and retrieves 1 of patent related to the present invention, 4 pieces of domestic literatures, and 4 pieces of external pertinent literatures:
(1) relevant patent is 1:
1. patent name: based on the media player under the LINUX operating system of UPNP agreement
Patent applicant: Hisense Company Limited
(2) Related domestic documents is 3 pieces:
1. title: H.323 Synchronization Control realizes research, communication journal .2004,25 (5) .-67-74
2. title: the buffering area compensation technique of multimedia synchronization system
Periodical name: Chinese journal of computers .2003,26 (4) .-484-490
3. title: a kind of new medium inter-sync control algolithm
Periodical name: computer research and development .2002,39 (12) .-1675-1680
4. title: time-delay is estimated and the Synchronization Control of real time data
Periodical name: communication journal .1999,20 (2) .-46-52
(3) external pertinent literature is 4 pieces:
1.Title:A synchronization and flow control scheme for interactivemultimedia-on demand(MOD)systems
Author(s):Chung-Ming Huang;Chian Wang;Hsu-Yang Kung
Source:Proceedings Seventh International Conference on Parallel andDistributed Systems(Cat.No.PR00568):39-46,2000
2.Title:A performance comparison of single-stream and multi-streamapproaches to live media synchronization
Author(s):Tasaka,S.;Ishibashi,Y.
Source:IEICE Transactions on Communications vol.E81-B,no.11:1988-97,11 1998
3.Title:A synchronization scheme for distributed multimedia informationsystems
Author(s):Lu,GJ.;Pung,H.K.;Chua,T.S.
Editor(s):Ng,C.S.;Yeo,T.S.;Yeo,S.P.
Source:Communications on the Move.Singapore.ICCS/ISITA′92(Cat.No.92TH0479-6):1-5 vol.1,1992
4.Title:A fast,multi-platform open source SMIL player
Conference:ACM Multimedia 2004-proceedings of the 12th ACM
International Conference on Multimedia
Publisher:Association for Computing Machinery
As follows to above-mentioned document analysis:
In patent 1, the inventor has proposed a kind of based on the media player under the LINUX operating system of UPNP agreement, its application program module mainly includes content delivery subsystem, formats subsystem, and audio frequency and video transmission service module, connection management service module and player control service module, it is characterized in that: in application program module, be provided with the UPNP protocol stack.The media player based on the UPnP agreement that this invention is mentioned has realized that mainly media player in the network is found automatically and the function of interoperability, and the function of shared among users use Internet resources is provided.
In the domestic literature [1], proposition has also realized a kind of H.323 multimedia communications system audio-visual synchronization control algolithm, and algorithm is a benchmark with the normal voice broadcast, utilizes H.263 bit stream syntax information, synchronous by adjusting between video playback frame per second realization medium, do not increase the communication redundancy degree.
In the domestic literature [2], this influences the key factor of the continuous play quality of multimedia messages to be primarily aimed at delay variation, by buffering area being set offsetting delay variation playing end, thereby guarantee in the medium and between medium synchronously.
In the domestic literature [3], the media sync that proposes is play scheme, be based on the media sync control of transmit leg, the situation that takies according to buffering area detects step-out, and it is fed back to transmit leg, the control function that the transmit leg utilization provides is adjusted sending frame per second, keeps the synchronous playing of receiving method, medium.
In the domestic literature [4], the Synchronization Control algorithm of proposition lacks the QoS business features at common Packet Based Network, by means of the data transmission mechanism of RTP, by the estimation to message delay and delay jitter, regulates and keep the real-time media data synchronization.
In the foreign literature [1], mainly discussed the media-on-demand system to the mutual response mode of VCR.Mutual at the medium under the distributed multimedia environment, proposed some synchronisation control meanss, but these methods are very not concrete.
In the foreign literature [2], single current and the multithread transmission problem looked between the audio frequency have been discussed.Because the separately transmission of video flowing and audio stream, cause the nonsynchronous problem of labial that in transmission course, produces, paper proposes to implement dynamic shunt transmission control can improve synchronizing quality.
In the foreign literature [3], discussed the influence that stationary problem and user interactions present multithread between stream inter-sync and stream; Propose main synchronously in the stream and rely on medium and present time relationship between the medium, between convection current be synchronously according to the synchronization properties of medium set up and safeguard between them synchronously.
In the foreign literature [4], having discussed a kind of is media player core, cross-platform with new SMIL engine.This player does not carry out Synchronization Control to multi-path media in the SMIL file, a plurality of video windows is not merged yet.
From home and abroad database retrieval and document analysis, mainly be the stream inter-sync of single channel Media Stream to be discussed more, do not relate to about the synchronisation control means and the multi-path media data fusion method of multi-source flow media.Convection current inter-sync method mainly contains two kinds, the one, adjust to play the size of end buffering area, guarantee between medium synchronously; Based on host-host protocol, regulate and safeguard the synchronous of media data.
Summary of the invention
In order to overcome defective or the deficiency that above-mentioned prior art exists, the invention provides the synchronous broadcast controlling method of supporting multi-source flow media, can will carry out Synchronization Control, and after multi-channel video merges, play from the relevant dissimilar medium of the multichannel sequential of media server or media file.
Technical scheme of the present invention is achieved in that
The control of Streaming Media synchronous playing adopts Streaming Media synchronous playing control device to realize, this device comprises separator, Decoder bank, multi-source flow media synchronization module, multisource video stream Fusion Module, OSD module and audio frequency filter; Separator receives from the media data in a plurality of local media files or the online multi-way stream medium, and in real time the video/audio in the Media Stream of every road is separated, and the data after the separation enter respective queue separately; Decoder bank receives video/audio from formation, according to the multimedia coding algorithm that it adopted, call corresponding decoder, finish the decoding of multichannel video/audio concurrently, and the back media data of will decoding is delivered to the multi-source flow media synchronization module in the multi-process mode; After the multi-source flow media synchronization module receives the multi-path media flow data, adopt many granularities hierarchical synchronization controlling mechanism that the multipath media stream between the media object and in the media object is carried out Synchronization Control; Multi-channel video synchronously and multi-path audio-frequency data are delivered to multisource video stream Fusion Module and audio frequency filter respectively; The multisource video Fusion Module adopts based on the video blending algorithm of image co-registration with the multichannel synchronization video road video that permeates, to avoid the overlapping demonstration of a plurality of video windows; In the fusion process, this module can also realize the switching of primary and secondary media object video, and merges again; After the OSD module receives video data after the fusion, volume, current reproduction time or caption information are added on the video line output of going forward side by side; The final synchronous playing control that realizes multi-path video stream; Behind the voice data of audio frequency filter after merge, carry out form and change, as gain, resample, the line linearity of going forward side by side stack finally realizes the synchronous playing control of multichannel voice frequency stream.
At the multi-source flow media synchronization module, adopted a kind of many granularities hierarchical synchronization controlling mechanism, be three levels that stationary problem are decomposed into Media layer, fluid layer and object layer; At fluid layer, according to the priority orders of audio frequency, video, captions, at first, based on presentative time stab PTS realize looking audio frequency synchronously, secondly, based on reference point RP realize captions and video synchronously, fluid layer media data synchronous solved synchronous in the media object; On the synchronous basis of fluid layer, at object layer, adopt pipe communication mechanism between multi-process realized between media object synchronously.
At the multisource video Fusion Module, adopt video blending algorithm based on image co-registration; This algorithm is that a plurality of video windows are carried out image co-registration in a virtual background: at first, frame of video in the multi-path video stream after synchronous is carried out the conversion of chrominance space, the yuv format unification is converted into RGB (RedGreen Blue) form, secondly, select to determine principal and subordinate's video according to the user; The 3rd, according to master slave relation video frame size is carried out mapping transformation, and determine position on virtual background; The 4th, on virtual background, carry out the fusion of Pixel-level, generate the fusion video of " picture-in-picture " effect.
Adopt the present invention, can realize a kind of player of supporting multi-source flow media, this player is supported the broadcast of various Media Streams and local media file; And can support the isomerous multi-source media synchronous playing that is combined to form between them.This player can be widely used in fields such as online courseware VOD, multiple scene monitoring, many playing back videos, many video request programs.
Description of drawings
Fig. 1 is architecture of the present invention and data flow diagram;
Fig. 2 is the description schematic diagram of stationary problem in the multi-source flow media playing process of the present invention;
Fig. 3 is many granularities of the present invention hierarchical synchronization reference model schematic diagram;
Fig. 4 is synchronistic model schematic diagram between media object of the present invention;
The principle schematic that Fig. 5 merges for multisource video stream picture split screen of the present invention;
Fig. 6 is the mapping change procedure schematic diagram of image size of the present invention;
Fig. 7 is a frequency division integration region schematic diagram of the present invention;
Fig. 8 is pixel level of the present invention " picture-in-picture " video fusion process schematic diagram;
Fig. 9 is multisource video stream syncretizing effect of the present invention.
Embodiment
The present invention is described in further detail below in conjunction with accompanying drawing.
The composition of synchronous playing controlling mechanism and working mechanism
With reference to shown in Figure 1, Streaming Media synchronous playing control device comprises separator, Decoder bank, multi-source flow media synchronization module, multisource video stream Fusion Module, OSD module and audio frequency filter.
Separator is used for receiving the media data from a plurality of local media files or online multi-way stream medium, and it is carried out format identification, separation.The media file the beginning part has one piece of data to can be used for discerning media formats, and this part data comprises file type, the number of data streams that comprises, data stream type, the encryption algorithm of data flow according to these information, can be separated into medium independent video flowing, audio stream and caption stream; Data after the separation enter respective queue separately.
Decoder bank receives video/audio from formation, according to the multimedia coding algorithm that it adopted, call corresponding decoder, finish the decoding of multichannel video/audio concurrently in the multi-process mode, video data is decoded into the picture frame of BMP or yuv format, and voice data is decoded into linear format; Decoding back media data is sent to the multi-source flow media synchronization module.
The multi-source flow media synchronization module receive multipath media stream according to after, adopt many granularities hierarchical synchronization controlling mechanism that the multipath media stream between the media object and in the media object is carried out Synchronization Control; In media object (fluid layer), according to the priority orders of audio frequency, video, captions, at first, realize looking the synchronous of audio frequency based on PTS, secondly, realize the synchronous of captions and video based on RP; On the synchronous basis of fluid layer, at object layer, adopt pipe communication mechanism between multi-process realized between media object synchronously.Multi-channel video synchronously and multi-path audio-frequency data are delivered to multisource video stream Fusion Module and audio frequency filter respectively.
The image fusion technology that adopts multisource video stream Fusion Module realizes with the multichannel synchronization video road video that permeates, to avoid the overlapping demonstration of a plurality of video windows; In the fusion process, this module can also realize the switching of primary and secondary media object video, and merges again.After the OSD module receives video data after the fusion, volume, current reproduction time or caption information are added on the video line output of going forward side by side; The final synchronous playing control that realizes multi-path video stream.
Behind the voice data of audio frequency filter after merge, be translated into the form that is fit to input, the line linearity of going forward side by side stack finally realizes the synchronous playing control of multichannel voice frequency stream.
Key technology
The multi-source flow media synchronisation control means of many granularities layering
(1) the multi-source flow media stationary problem is described
With reference to shown in Figure 2, the multi-source flow media stationary problem has media object 1 and 2 two media object among Fig. 2, and each media object comprises video flowing, audio stream, caption stream.T wherein aBe meant the temporal relationship of a Media Stream internal media packet; t bWith t cBeing meant the temporal relationship between the different media flows of a media object inside, is respectively the temporal relationship of video and audio frequency, video and captions in the media object 1 here.According to the strict degree that in synchronizing process, sequential is required, can also be with a step is subdivided into dynamic synchronization and synchronous two kinds of static state more synchronously between stream.t bDescribed sound, video labial are a kind of dynamic synchronization process synchronously, and it requires any moment of audio frequency and video in playing process to keep synchronously; t cDescribed captions and video then be a static synchronizing process synchronously, for specific screen captions, its correspondence one section show time interval, as long as guarantee interval point constantly (or being called reference point) and video flowing synchronously, and to any moment in interval during this period of time, not simultaneous operation.
If be strict synchronism in the media object, then have:
t a=1/fps,t b=0,t c=0
Wherein, fps is a video frame rate.
Synchronized relation between two or more media object is used t MT among Fig. 2 is described MSequential relationship between presentation medium object 1 and the media object 2 sound intermediate frequency data, same, if be strict synchronism between media object, then have:
t M=0
In the multi-source flow media playing process, flow inter-sync synchronization granularity minimum, it is to solve inter-sync basis of media object and prerequisite, and between media object then is in the sequential relationship between a plurality of media object on the inter-sync basis of media object, its synchronization granularity maximum synchronously.Generally speaking, t aDepend on the result of media file itself or Streaming Media transmission, and can satisfy the multi-source flow media synchronisation requirement basically, so, in research emphasis of the present invention be:
By a series of efficient synchronization mechanism, make t b, t cAnd t MThe value minimum.
(2) many granularities hierarchical synchronization reference model
Various stationary problems in the multi-source flow media broadcast have proposed many granularities hierarchical synchronization reference model of playing towards multi-source flow media.
With reference to shown in Figure 3, in the hierarchical synchronization model, we are divided into three layers with different synchronisation requirement: Media layer, fluid layer and object layer, what the right side provided in illustraton of model is the simultaneous operation object of every layer of correspondence.
Media layer: its objective is and realize the synchronous of interior each media units of stream, media units comprises frame of video, audio sample and subtitle segment.At present, the synchronization mechanism of this layer mainly be at multi-medium data in network transmission process, by corresponding buffering area being set, changing the transmitting terminal transmission rate and taking the ARQ feedback mechanism to realize at transmitting terminal or local reception end, this process occurs in the media data Network Transmission stage, do not belong within the category of the present invention, therefore, not not describing synchronously Media layer.For fluid layer and layer, sync, its synchronizing process is transparent.t aIt is the synchronous regime characterising parameter of this layer.
Fluid layer: its objective is and realized between the Media Stream synchronously.Media Stream comprises video flowing, audio stream, caption stream, and according to the priority of audio frequency, video, captions, we have adopted based on synchronization mechanism between the stream of timestamp.Here, for audio frequency, video, timestamp is the PTS of media data, represents the quiet hour mark that a packet expectation is presented, and for captions, timestamp is the RP of captions, i.e. the quiet hour mark that captions expectation is shown, hides.t bAnd t cBe the synchronous regime characterising parameter of this layer, describe respectively look between the audio frequency and video and screen between synchronously.
Object layer: its objective is realize between media object synchronously.We realize having proposed communication function between a plurality of media object processes based on synchronization mechanism between the media object of pipeline communication by the utilization pipeline.t MIt is this layer synchronous regime characterising parameter.
Wherein, Media layer be to realize the synchronous basis of fluid layer synchronously, and fluid layer is synchronously to realize the synchronous prerequisite of object layer.Can see that in three layers of above-mentioned synchronistic model in each layer, sync, we have solved the stationary problem of this layer simultaneous operation object by corresponding synchronous mechanism.
(3) foundation of synchronous base
Because the synchronization granularity of each layer, sync differs greatly, specifically, the synchronization granularity of fluid layer is little, and the synchronization granularity of object layer is big, and therefore, the synchronous base in each layer, sync also has otherness.
The synchronous base of ■ fluid layer
The synchronous main operand of fluid layer is video flowing, audio stream and caption stream etc., determine the synchronous base of fluid layer, is exactly to determine main Media Stream according to the priority relationship of the above-mentioned three kinds of streams in media object inside in fact.
Because people's ear is comparatively responsive to the adjustment of sound pause, repetition or playback rate, therefore usually with audio stream as main Media Stream, and with video flowing and other Media Stream as from Media Stream, realize between the stream of fluid layer synchronous by adjusting reproduction time.
What look audio frequency is a kind of dynamic synchronization process synchronously, synchronization performance is required than higher, and caption stream is the synchronous process of a kind of static state synchronously that it not is very high that synchronization performance is required.Can determine that according to above-mentioned analysis three kinds of main Media Streams have the priority relationship of " audio frequency>video>captions ".So audio stream is the synchronous base of fluid layer.Especially, if the media object that fluid layer is handled does not have audio stream,, can determine that video flowing is a synchronous base according to priority.
The synchronous base of ■ object layer
The synchronous situation of object layer is divided into two classes: one, need not synchronously, promptly form between a plurality of media object in the multi-source flow media broadcast, almost do not have dependence, need not carry out Synchronous Processing, the value in threshold value space is infinitely great.Spread as the multi-source of a plurality of irrelevant media object and to put.Two, precise synchronization is promptly formed between a plurality of media object in the multi-source flow media broadcast, has relatively high expectations synchronously, and the synchronization threshold space is less.As in the sports tournament at the panorama replayed section of the competition field video that a plurality of different angles photograph, its synchronization threshold need be arranged on below the 1s.
According to above-mentioned analysis, we proposed can unique description media object 7 yuan of vector models:
MO=<ID,Ts,Weight,ThRank,MRL,Fusion,Info>
Wherein,
MO:Media Object is meant media object;
ID a: positive integer of this media object of unique identification;
Ts:TimeStamp writes down the play position of this media object, is provided by the interface of fluid layer, is generally the pts value of the main Media Stream in this media object;
Weight: weight is a positive integer, and the height that this media object net synchronization capability of mark requires stipulates that the more little synchronous requirement of its value is just high more, and the weighted value of giving tacit consent to main media object is 1;
ThRank:Threshold Rank, threshold levels is a relative value, represents the height of net synchronization capability requirement between this media object and the main media object.According to the practical application needs, we determine to tabulate as lower threshold level:
ThRank Threshold value space (s) ThRank Threshold value space (s)
-1 3 3~7
1 0~1 4 7~12
2 1~3 5 12~
MRL: refer to the resource path of this media object, it can be local path or network flow path;
Fusion: the sign position that video merges is worth and is Major, Minor or None;
Info: some additional informations that are used to describe this media object.
If object layer will be handled N the stationary problem between media object, we use vector M=(MO 0, MO 1... MO N), write down the synchronous element set of object layer.
In M, we are defined as the media object of weighted value minimum the synchronous base of object layer.
(4) media object inter-sync
The media object inter-sync comprises synchronous with captions, video synchronously that looks audio frequency.
■ is based on the audio sync of looking of PTS
The main thought of looking audio sync based on PTS is: audio frequency is according to normal speed forward, the frame of video control program carries out normal play or frame losing processing according to the output result of synchronous regime trace routine to frame of video, after whenever finishing a frame of video, A/V PTS correction program is done respective handling and the result is fed back to the synchronous regime trace routine audio delay, thereby finishes the correction to current out of step conditions.
If:
Audio_delay: audio delay is the important evidence that the synchronous regime trace routine is judged current synchronous regime, and initial value is 0;
Timer: the play position of current video, initial value are 0;
Playsize: the space size that can insert data among the sound card Buffer;
O_bps: a second is the compressed audio byte number not;
Fps:frame per second, video frame rate;
Th: sound, audio video synchronization threshold value space, value 100ms;
V_pts: current video PTS;
A_pts: current audio frequency PTS;
T1: broadcast time-delay from current audio frequency pts to the current voice data that reads;
T2: the broadcast time-delay of the voice data that deposit the input block of audio stream;
T3: do not send the broadcast time-delay of the voice data of sound card behind the decompress(ion) as yet to;
T4: the broadcast time-delay of the voice data of not playing as yet in the sound card;
AV_delay: the actual time delay that exists between the current in progress sound that calculates by A/V PTS correction program, video, by revising the span factor S, audio_delay is revised;
S: revise the span factor, in algorithm was realized, value was 0.1.Be 1s if we obtain the AV_delay value, then A/V PTS correction program has so just been avoided because of correction value is excessive video playback being produced significantly " shake " phenomenon through repeatedly finishing the whole corrections to this AV_delay.
Sound, audio video synchronization algorithm based on PTS:
Step1 begins, initialization audio_delay=0, timer=0;
Step2 changes Step13 if file finishes, otherwise, obtain can insert among the sound card Buffer space size playsize of data, continue;
The voice data of Step3 decompression playzise size is also sent into sound card and is play;
Step4 revises the value of audio_delay;
audio_delay=audio_delay+playsize/o_bps
Step5 reads not decoded video frames data of a frame, revises the value of audio_delay and timer;
timer=timer+1/fps
audio_delay=audio_delay-1/fps
Step6 synchronous regime trace routine is calculated the audio_delay-T4 value, if should be worth greater than sound, audio video synchronization threshold value space Th value, asynchrony phenomenon then occurred, changes Step12, otherwise, continue;
This frame of video of Step7 frame of video control program decompress(ion), sleep (used time of 1/fps-top-operation) shows;
* * * * * * * * A/V PTS correction program is revised * * * * * * to the audio_delay value
Step8 obtains a_pts and v_pts value, and to revise a_pts be current just in moment of audio plays;
a_pts=a_pts+T1-T2-T3-T4
Step9 calculates the AV_delay value;
AV_delay=a_pts-v_pts
Step10 revises the audio_delay value;
audio_delay=audio_delay+AV_delay*S
The correction of * * * * * * * * * * A/V PTS correction program finishes * * * * * * * * * * * * * *
Step11 returns Step2;
Step12 frame of video control program is this frame of video of decompress(ion) not, does discard processing, returns Step2;
Step13 finishes.
■ is based on captions, the audio video synchronization of " reference point "
The demonstration of captions need keep with the PTS of frame of video synchronously with hiding, each " reference point " all corresponding in fact a video PTS, when video PTS arrived, the captions display control program just can determine whether whether current caption content needs to show or hide.Hence one can see that, keeps synchronously as long as guarantee first reference point and the PTS of video, and other reference points have all kept synchronous with respect to the constant time lag of first reference point because every section caption content the demonstration time all determine.Like this, very simply realized the synchronous of captions and video.
(5) object layer is synchronous
Object layer needs to determine the synchronous interaction information format and safeguard various basic synchronizing informations synchronously.
■ synchronous interaction information format
We determine that object layer synchronous interaction information format is as follows:
SyncMsg=ID+Ts+Weight+ThRank+Action+Param
In following formula, have 6 partial datas and form, wherein preceding 4 partial datas are consistent with information in the media object model, and back two parts data declaration is as follows:
Action: represent the required action of taking of receiving media object process of this message, promptly main media object process is given the fill order from the media object process;
Param: generally be required additional information when carrying out the Action order.
Wherein the corresponding value of Action and Param is as shown in the table:
Action Param Action description
Play Invalid Normal play, this is a default value
Pause Time out Suspend, faster than the operation of taking after the main media object
Stop Invalid Stop
Skip Abandon fragment length Abandon, be slower than the operation of taking after the main media object
Seek Positioning time point The location, the retrieval by window operation (only effective when the user operates) of corresponding main Media Stream
■ basis synchronizing information
In order to realize the synchronous of multi-source media, system need set up and safeguard following synchronous information data:
1. media object chained list: PlayTree
Writing down the essential information of current in progress a plurality of media object, is a dynamic link table.
2. threshold levels matrix: TRM (Threshold Rank Matrix)
TRM=(ThR 21,ThR 31,...ThR N1) T
It is the vector of a N-1 dimension, writes down all from the threshold levels between media object and main media object.
3. depart from matrix: SSM (Synchronization Skew Matrix) synchronously
In theory, the synchronized relation between N media object is the relation of a two dimension, i.e. the matrix S SM of N * N 1:
Ss wherein Ij(i, j=0,1 ..., N) be synchronous deviation value (Ts between i media object and j media object i-Ts j).
Because SSM 1Be an antisymmetric matrix, as long as and we know the data of the first half wherein or the latter half, just can carry out the Synchronous Processing of media object fully, so we can be with SSM 1Be reduced to following formula upper triangular matrix SSM 2:
In fact, can know according to the establishment process of object layer synchronous base, between media object synchronously in, from can not be concerned about synchronously between media object, key is to handle main media object well and from the synchronized relation between the media object, thus, we can also be with SSM 2Simplify following formula and depart from matrix S SM synchronously:
SSM=(ss 21,ss 31,...,ss N1)
It also is the vector of N-1 dimension, and we have obtained between media object the very important matrix S SM that departs from synchronously in the synchronizing process thus.And in this way, we are reduced to the one dimension stationary problem by the two-dimentional stationary problem that object layer is comparatively complicated.
Synchronistic model between the ■ media object
Object layer element set M has synchronously been arranged, synchronous interaction information format SyncMsg, and basic synchronizing information PlayTree, TRM, SSM, the basic thought that object layer is synchronous:
At first, obtain PlayTree, and make up TRM and SSM according to M;
Secondly, be the synchronizing information that mutual each the media object process of interprocess communication is carried out on the basis with SyncMsg, renewal PlayTree, SSM also dynamically safeguards;
At last, depart from program synchronously and detect current asynchronous state according to PlayTree, TRM, SSM information, and carry out Synchronous Processing: main media object is according to normal speed forward, do possibly from the broadcast of media object and to skip or pausing operation (receiving main media object command information), finish thus between media object synchronously.
This shows, between media object synchronously than audio frequency and video synchronously, need the object handled many, granularity is big, dynamic is strong.
With reference to shown in Figure 4, we are for being described as follows based on synchronizing process between the media object of pipeline communication:
(1) the multi-source flow media playing program begins, and according to the relevant information of input medium, determines main media object, parameters such as initialization Ts, Weight, ThRank, establishment PlayTree starts main media object broadcast process and by normal speed broadcast, startup simultaneously is respectively from media object broadcast process.
(2) generate threshold levels matrix T RM by PlayTree.
(3) by the pipeline communication between process, main media object is play all state informations from media object that process obtained, and Ts parameter value among the PlayTree that upgrades in time is finished the Dynamic Maintenance to PlayTree.
(4) finish that three function: I keep and the pipeline communication of main media object process, will in time feed back to main media object process from the state information of media object self; II accepts the command information of main media object process, carries out simultaneous operation; III receives the user to the VCR control information from media object.
(5) generating also according to PlayTree, Dynamic Maintenance departs from matrix S SM synchronously.
(6) the synchronous detecting program is by comparing the value of corresponding position among SSM and the TRM, if promptly find one of them SS from media object J1Value has surpassed its corresponding Th J1Value produces asynchronous information.
(7) asynchronous information is notified main media object process, main media object process adopts pipeline communication to send command information SyncMsg to nonsynchronous from the media object process, finishes simultaneous operation.
(8) user carries out VCR control to main media object.
(9) user is to carrying out VCR control from media object.
For this synchronistic model, need explanation what time following:
√ step (3)~(7) constitute a closed loop procedure, and it is to realize synchronous core loop between media object.
√ is in this synchronistic model, also has a Clock module, it is not system reference time SCR (System Clock Reference), a but timing parameter that obtains jointly by minimum value among the TRM and SCR, be " global clock " of this synchronistic model, the detection and the correction of " unified leadership " synchronous situation thus.
If √ synchronous detecting program testing result has a plurality of asynchronous information, system will at first handle the out of step conditions of its ThRank value lower (priority is higher).If the ThRank value of media object is-1, the synchronous detecting program will be ignored the Synchronous Processing to this media object.
√ master's media object and slightly different from the realization of the user VCR of media object control.For main media object, directly with VCR control information notice from the media object process and take corresponding VCR operation; For from media object, at first this information is sent to main media object process, and information is sent to other from the media object process by main media object process.
√ has the process of a global synchronization when carrying out Seek operation and VCR operation, this operation is realized by main media object process.
Like this, the proposition that we are complete the synchronization solutions of playing towards multi-source flow media, the stationary problem of multi-source flow media has obtained solution.
Video syncretizing mechanism based on image co-registration
The purpose that multisource video stream merges is exactly to make the covering (Overlay) of prospect in selected fusion center point or regional realization prospect and background, and does some smoothnesses and seamlessly transit processing at boundary.
At first, we carry out preliminary treatment to background and prospect, lay a solid foundation for carrying out the pixel level fusion, then, choose center or the zone that will merge in background, use blending algorithm at last and reach our fusion requirement, finish fusion process.
With reference to shown in Figure 5, preliminary treatment has been finished the image of different-format has been carried out the conversion of chrominance space, the standardization of image dimension, the work such as mapping transformation of image size.The relative position of prospect and background has been determined in the selection of merging the position.The pixel level that blending algorithm has been finished image merges.Above step has been finished the basic function of image co-registration, realize the fusion of multi-source stream video on this basis, still, the fusion process of multi-source stream video has more complexity than the fusion process of two images, so, above-mentioned flow process need be expanded (among Fig. 5 shown in the empty frame).
It is the image co-registration processes of a plurality of prospects in a virtual background that multi-source stream video frequency division merges handled, these prospects all do not cover mutually, therefore we also need to finish the selection course of primary and secondary prospect after Video Decoder output forms frame of video, the position fixing process of complicated fusion position etc., and, different for the different video output equipment to the support situation of chrominance space, we need do suitably " reprocessing " to merging output, so that flexible adaptation, as for realization prospect seamlessly transitting to background, in the gradation zone of image, each pixel after the fusion had both comprised the information from background, had the information of prospect again.
■ multi-source stream video blending algorithm
If: in the playing process of multi-source stream, obtain sequence of frames of video F i, i ∈ [1, N], the decoding back is the image file sequence B i, i ∈ [1, N]:
S1. begin
Receive the image file sequence B i, as the input of image co-registration, multi-source stream video frequency division stream merges beginning.
S2. generate virtual background
(be made as U * V), generate the virtual background bgB (background Bitmap) of image co-registration, form is RGB24, each pixel filling value 0xFFFFFF (black) according to final video output equipment resolution sizes.How playback of media files finishes, and changes S12, otherwise continues.
S3. determine the primary and secondary prospect
According to the B that receives i, choose one of them image file B j, j ∈ [1, N] is as the main prospect MajorFB (Major Frontground Bitmap) of image co-registration, and remaining is all as image co-registration time prospect MinorFB k, k ∈ [1, N-1] (Minor Frontground Bitmap).
7 yuan of vector models according to can the unique identification media object have a Fusion territory in this model, it is the sign position that video merges, and value can be Major, Minor or None, and meaning of each value is as follows:
Major: the identification medium object is the main prospect during video merges;
Minor: the identification medium object is the inferior prospect during video merges;
None: the identification medium object does not carry out video and merges.
Generally speaking, the Fusion value of system default master media object is Major, is Minor from the Fusion value of media object, and the Fusion value of media object is accepted user's modification.Like this, just determined a main prospect MajorFB, N-1 time prospect MinorFB k, k ∈ [1, N-1].
If the Fusion value of all media object is None, then do not carry out video and merge, adopt multiple-window mode to show, change S12.
S4. the conversion of chrominance space
Because the MajorFB and the MinorFB that determine kK ∈ [1, N-1] may adopt different chrominance spaces, form is inconsistent, therefore above-mentioned image need be carried out the conversion of chrominance space (picture format), for the ease of further image co-registration processing, by following formula all image transitions to be come to rgb space, its transformation result all is the nothing compression BMP bitmap file form of RGB24.
R G B = 1.000 0 1.140 1.000 - 0.390 - 0.580 1.000 2.030 0 Y U V
S5. the standardization of image dimension
With MajorFB and MinorFB k, the resolution standard of k ∈ [1, N-1] turns to U * V, is MajorFB, MinorFB k, the fusion that k ∈ [1, N-1] and bgB carry out pixel level lays the foundation.
S6. the mapping transformation of image size
Before merging, need be to MajorFB and MinorFB k, k ∈ [1, N-1] carries out the conversion of image size, and this is the process of a pixel mapping transformation, and its transforming function transformation function is designated as f.We underdraw to this process, and the ratio of the wide and height of general pattern is 4: 3, is without loss of generality, and suppose that we need be with original image M ajorFB and MinorFB k, (U * V) be reduced into original 1/z dwindles back image M ajorFB and MinorFB to k ∈ [1, N-1] k, the dimension of k ∈ [1, N-1] is m * n, positive integer z is called the image zoom multiplying power.Might as well establish:
U=zm+1,V=zn+r
So, to MajorFB and MinorFB k, k ∈ [1, N-1] can adopt and abandon part edge pixel (1 * r zone among the figure), and the method converted image size that direct ratio dwindles is wherein to getting in the every z of former pixel * z square formation pixel region a bit as MajorFB zAnd MinorFB Kz, promptly there are following pixel mapping relations in the pixel of k ∈ [1, N-1]:
MajorFB z=f(z,MajorFB)
MinorFB kz=f(z,MinorFB k),k∈[1,N-1]
Reference its transfer process shown in Figure 6:
Like this, just determined prospect MajorFB zOr MinorFB Kz, the integration region m * n of k ∈ [1, N-1] in virtual background bgB.
With reference to shown in Figure 7:
S7. merge the calculating of position
Respectively the size of primary and secondary prospect is carried out conversion according to U, V and image zoom multiplying power z, and obtained the integration region Q:m * n of primary and secondary prospect in background after the conversion thus.But we also need determine MajorFB zAnd MinorFB Kz, the position P:P of first pixel of k ∈ [1, N-1] in bgB (x, y) and P k(x k, y k) afterwards, could determine the fusion position of each prospect in background fully.
In the fusion process of multisource video stream, we adopt " picture-in-picture " form, P k(x k, y k) between use simple linear relationship to represent, that is:
P k+1(x k+1,y k+1)=P k(x k,y k+n)
If MajorFB zDo not exist, promptly the Fusion value of all medium all is Minor among the S3, then with MinorFB KzIn bgB " average layout ".
S8. pixel level " picture-in-picture " video merges
Be output as after supposing to merge , at existing bgB, MajorFB zAnd MinorFB Kz, on the basis of k ∈ [1, N-1], P, Q, provide the process that video merges.
Process with reference to video fusion shown in Figure 8:
Fusion process is designated as F. Be with bgB and MajorFB by pixel level image co-registration process F zAnd MinorFB Kz, k ∈ [1, N-1] carries out the result who merges N time.
If bgB, MajorFB zAnd MinorFB Kz, k ∈ [1, N-1] is that linear address is preserved, that is:
bgB=byte[0..3*U*V-1]
MajorFB z,MinorFB kz=byte[0..3*m*n-1],k∈[1,N-1]
With MinorFB Kz, the fusion process of k ∈ [1, N-1] and bgB is an example, fusion process F can use following arthmetic statement:
* * * * * * * * * * * * * * * * * * * * * * multisource video stream blending algorithm * * * * * * * * * * * * * * * * * * * *
001 for I=0 to N-1 do // carry out continuously N time fusion process
The capable pixel of J+1 among 002 for J=0 to m-1 do // replacement Q
J+1 among 003 for K=0 to n-1 do // replacement Q is capable, K+1 is listed as pairing pixel
004 begin
005 bgB{[(x I+1)*n+y I+1-1]*3}=MinorFB (I+1)z[(J*n+K)*3];
R component in the // replacement pixel
006 bgB{[(x I+1-1)*n+y I+1-1]*3+1}=MinorFB (I+1)z[(J*n+K)*3+1];
G component in the // replacement pixel
007 bgB{[(x I+1-1)*n+y I+1-1]*3+2}=MinorFB (I+1)z[(J*n+K)*3+2];
B component in the // replacement pixel
008 end.
*************************************************************
Arrive this, finished multisource video stream " picture-in-picture " pixel level by F and merged.
S9. reprocessing
The difference of the chrominance space of supporting according to different picture output device is right Carrying out the chrominance space conversion of similar S4 and the image scaled of similar S5 handles.
S10. video output
To in a video window, present to the user by Video Out in " picture-in-picture " mode.
S11. user window control
In the playing process of multisource video, accept user window operation (, switching main prospect and time prospect etc.) as main prospect is full frame, the value in the Fusion territory of modification of program media object is returned S2, the multisource video stream fusion process that system restart is new.
With reference to the design sketch after the video fusion shown in Figure 9.
■ window control strategy
Because video window is smaller, detailed information can't be seen clearly, has introduced the window control strategy for this reason, realizes the flexible switching of primary and secondary prospect media object video.
Strategy: the video window after the fusion Be the window of a U * V dimension, wherein each pixel all corresponding a coordinate p in the video window (u, v), as shown in Figure 8, according to bgB, MajorFB zAnd MinorFB Kz, k ∈ [1, N-1], we can judging point p (u v) is at current main prospect media object MajorFB z, still at inferior prospect media object MinorFB Kz, in k ∈ [1, the N-1] zone, so the value in the Fusion territory by revising the respective media object, restart multi-source stream video blending algorithm, finish the handoff functionality of primary and secondary prospect media object video.
Definition message evFusion produces when the user double-clicks in inferior foreground window, and after the multisource video window received this message, the Fusion value of the current main prospect media object of modification of program was Minor, and the Fusion value of modification time prospect media object is Major.

Claims (3)

1. synchronous broadcast controlling method of supporting multi-source flow media, it is characterized in that: adopt Streaming Media synchronous playing control device to realize, this device comprises separator, Decoder bank, multi-source flow media synchronization module, multisource video stream Fusion Module, OSD module and audio frequency filter; Separator receives from the media data in a plurality of local media files or the online multi-way stream medium, and in real time the video/audio in the Media Stream of every road is separated, and the data after the separation enter respective queue separately; Decoder bank receives video/audio from formation, according to the multimedia coding algorithm that is adopted, call corresponding decoder, finish the decoding of multichannel video/audio concurrently, and the back media data of will decoding is delivered to the multi-source flow media synchronization module in the multi-process mode; After the multi-source flow media synchronization module receives the multi-path media flow data, adopt many granularities hierarchical synchronization control method that the multipath media stream between the media object and in the media object is carried out Synchronization Control; Multi-channel video synchronously and multi-path audio-frequency data are delivered to multisource video stream Fusion Module and audio frequency filter respectively; The multisource video Fusion Module adopts based on the video blending algorithm of image co-registration the multichannel synchronization video road video that permeates; In the fusion process, this module can also realize the switching of primary and secondary media object video, and merges again; After the OSD module receives video data after the fusion, volume, current reproduction time or caption information are added on the video line output of going forward side by side; The final synchronous playing control that realizes multi-path video stream; Behind the voice data of audio frequency filter after merge, carry out form and change, resample, the line linearity of going forward side by side stack finally realizes the synchronous playing control of multichannel voice frequency stream.
2. a kind of synchronous broadcast controlling method of supporting multi-source flow media according to claim 1, it is characterized in that, in a kind of many granularities hierarchical synchronization control method that the multi-source flow media synchronization module adopts, be three levels that stationary problem are decomposed into Media layer, fluid layer and object layer; At fluid layer, according to the priority orders of audio frequency, video, captions, at first, based on presentative time stab PTS realize looking audio frequency synchronously, secondly, based on reference point RP realize captions and video synchronously, fluid layer media data synchronous solved synchronous in the media object; On the synchronous basis of fluid layer, at object layer, adopt pipe communication mechanism between multi-process realize between media object synchronously.
3. a kind of synchronous broadcast controlling method of supporting multi-source flow media according to claim 1, it is characterized in that: the video blending algorithm method based on image co-registration that adopts at the multisource video Fusion Module is that a plurality of video windows are carried out image co-registration in a virtual background: at first, frame of video in the multi-path video stream after synchronous is carried out the conversion of chrominance space, it is Red Green Blue form that the yuv format unification is converted into rgb format, secondly, select to determine principal and subordinate's video according to the user; The 3rd, according to master slave relation video frame size is carried out mapping transformation, and determine the position on virtual background; The 4th, on virtual background, carry out the fusion of Pixel-level, generate the fusion video of " picture-in-picture " effect.
CNB2005100962159A 2005-10-21 2005-10-21 Synchronous broadcast controlling method capable of supporting multi-source stream media Expired - Fee Related CN100352270C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNB2005100962159A CN100352270C (en) 2005-10-21 2005-10-21 Synchronous broadcast controlling method capable of supporting multi-source stream media

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNB2005100962159A CN100352270C (en) 2005-10-21 2005-10-21 Synchronous broadcast controlling method capable of supporting multi-source stream media

Publications (2)

Publication Number Publication Date
CN1767601A true CN1767601A (en) 2006-05-03
CN100352270C CN100352270C (en) 2007-11-28

Family

ID=36743175

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB2005100962159A Expired - Fee Related CN100352270C (en) 2005-10-21 2005-10-21 Synchronous broadcast controlling method capable of supporting multi-source stream media

Country Status (1)

Country Link
CN (1) CN100352270C (en)

Cited By (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008055420A1 (en) * 2006-11-09 2008-05-15 Huawei Technologies Co., Ltd. A synchronizing method between different medium streams and a system
CN101212679B (en) * 2006-12-30 2010-04-14 凌阳科技股份有限公司 AV synchronization method and device for switching multi-channel audio streams during playing of AVI files
CN101068345B (en) * 2007-05-24 2010-06-23 杭州华三通信技术有限公司 Video frequency monitoring method and system and network transmission equipment
CN101840494A (en) * 2010-05-25 2010-09-22 福建新大陆电脑股份有限公司 Device and method for judging bar code boundary
CN101163222B (en) * 2006-10-13 2010-09-29 索尼株式会社 Image apparatus and captured image recording method
CN101883270A (en) * 2010-06-10 2010-11-10 上海海事大学 Method for inserting related media in independent video streams
CN101894336A (en) * 2010-06-28 2010-11-24 深圳市江波龙电子有限公司 Video on demand payment system and payment method thereof
CN101901126A (en) * 2010-07-12 2010-12-01 东北大学 Method for controlling combined large-screen stream media playing computer
CN101034582B (en) * 2007-04-02 2011-01-26 北京暴风网际科技有限公司 System for improving the playing effect of the media file and its method
CN102110430A (en) * 2009-12-23 2011-06-29 富士通株式会社 Method and system for multi-window display
CN102215427A (en) * 2011-05-17 2011-10-12 深圳创维-Rgb电子有限公司 Method and device for synchronously playing multimedia information on television, and television
CN102254560A (en) * 2010-05-19 2011-11-23 安凯(广州)微电子技术有限公司 Audio processing method in mobile digital television recording
CN101577110B (en) * 2009-05-31 2012-04-25 腾讯科技(深圳)有限公司 Method for playing videos and video player
CN101227265B (en) * 2008-01-31 2012-06-13 深圳国人通信有限公司 Method for data synchronization among client ends in equipment monitoring system
CN102522096A (en) * 2011-12-07 2012-06-27 中国航空无线电电子研究所 Synchronous replay system for multi-channel audio and video and method thereof
CN102547482A (en) * 2011-12-30 2012-07-04 北京锐安科技有限公司 Synchronous playing method of multi-path IP (Internet Protocol) audio-video stream
CN102625164A (en) * 2012-04-06 2012-08-01 上海车音网络科技有限公司 Multimedia data processing platform, multimedia reading material, system and method
CN102623032A (en) * 2012-04-01 2012-08-01 深圳英飞拓科技股份有限公司 Method for realizing synchronous playback of hard disk recorder
CN101563915B (en) * 2006-12-20 2012-09-19 Gvbb控股股份有限公司 Embedded audio routing switcher
WO2013044772A1 (en) * 2011-09-28 2013-04-04 歌尔声学股份有限公司 Method, system and device for simultaneously viewing different pictures on identical screen
CN103065503A (en) * 2012-12-27 2013-04-24 重庆大学 Reappearing method and system controlling river reach ship traveling dynamic and signal revealing process
WO2013082965A1 (en) * 2011-12-05 2013-06-13 优视科技有限公司 Streaming media data processing method and apparatus and streaming media data reproducing device
CN103237185A (en) * 2013-05-15 2013-08-07 无锡北斗星通信息科技有限公司 System for carrying out broadcasting control on ATSC (Advanced Television System Committee) signal at television station end
CN103246752A (en) * 2013-05-30 2013-08-14 北京瑞翔恒宇科技有限公司 System for associating ppt document with audio
CN101594487B (en) * 2008-05-30 2013-08-21 新奥特(北京)视频技术有限公司 Method for synchronously changing scene areas of caption machine
CN103686347A (en) * 2012-09-14 2014-03-26 美国博通公司 Adaptable media processing architectures
CN103905783A (en) * 2012-12-25 2014-07-02 杭州海康威视数字技术股份有限公司 Method and device for performing decoding and displaying on video stream
CN101889422B (en) * 2007-12-05 2014-07-09 皇家Kpn公司 Method and system for synchronizing the output of terminals
CN105392024A (en) * 2015-10-28 2016-03-09 中国电子科技集团公司第二十八研究所 Multi-source heterogeneous data synchronous replaying method
CN105578204A (en) * 2014-10-14 2016-05-11 青岛海信电器股份有限公司 Multi-video-data displaying method and apparatus
CN105704506A (en) * 2016-01-19 2016-06-22 北京流金岁月文化传播股份有限公司 Device and method for synchronizing audio and video coding labial sound
CN105721920A (en) * 2014-12-04 2016-06-29 中国移动通信集团公司 Method and apparatus for cooperative playing of media on multi devices, and terminal
CN107333164A (en) * 2016-04-29 2017-11-07 北京学而思教育科技有限公司 A kind of image processing method and device
CN107809659A (en) * 2017-10-27 2018-03-16 北京雷石天地电子技术有限公司 The method and device that a kind of multi-channel video fusion plays
CN108174283A (en) * 2017-12-27 2018-06-15 威创集团股份有限公司 A kind of vision signal source generating method and device
CN108495145A (en) * 2013-04-01 2018-09-04 微软技术许可有限责任公司 Dynamic track switching in media stream
CN108881918A (en) * 2017-11-29 2018-11-23 北京视联动力国际信息技术有限公司 A kind of processing method of multi-channel video and a kind of multimedia terminal
CN108881917A (en) * 2018-07-27 2018-11-23 西安电子科技大学 Parallel decoding quicksort method based on high speed real-time decoding platform
CN109828742A (en) * 2019-02-01 2019-05-31 珠海全志科技股份有限公司 Voice-frequency-multichannel synchronism output method, computer installation and computer readable storage medium
CN110225363A (en) * 2019-07-10 2019-09-10 央视国际网络无锡有限公司 A kind of method that difference live source switches and is superimposed different filters
CN110519477A (en) * 2006-06-23 2019-11-29 埃科360公司 Embedded equipment for multimedia capture

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6715126B1 (en) * 1998-09-16 2004-03-30 International Business Machines Corporation Efficient streaming of synchronized web content from multiple sources
US6816909B1 (en) * 1998-09-16 2004-11-09 International Business Machines Corporation Streaming media player with synchronous events from multiple sources
US6930620B2 (en) * 2002-01-15 2005-08-16 Microsoft Corporation Methods and systems for synchronizing data streams
CN1221132C (en) * 2002-07-30 2005-09-28 华为技术有限公司 Device and method for realizing conversion between various VF flow formats

Cited By (55)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110519477A (en) * 2006-06-23 2019-11-29 埃科360公司 Embedded equipment for multimedia capture
CN101163222B (en) * 2006-10-13 2010-09-29 索尼株式会社 Image apparatus and captured image recording method
CN101371488B (en) * 2006-11-09 2013-01-30 华为技术有限公司 Synchronization method and system between different media flows
WO2008055420A1 (en) * 2006-11-09 2008-05-15 Huawei Technologies Co., Ltd. A synchronizing method between different medium streams and a system
CN101563915B (en) * 2006-12-20 2012-09-19 Gvbb控股股份有限公司 Embedded audio routing switcher
CN101212679B (en) * 2006-12-30 2010-04-14 凌阳科技股份有限公司 AV synchronization method and device for switching multi-channel audio streams during playing of AVI files
CN101034582B (en) * 2007-04-02 2011-01-26 北京暴风网际科技有限公司 System for improving the playing effect of the media file and its method
CN101068345B (en) * 2007-05-24 2010-06-23 杭州华三通信技术有限公司 Video frequency monitoring method and system and network transmission equipment
CN101889422B (en) * 2007-12-05 2014-07-09 皇家Kpn公司 Method and system for synchronizing the output of terminals
CN101227265B (en) * 2008-01-31 2012-06-13 深圳国人通信有限公司 Method for data synchronization among client ends in equipment monitoring system
CN101594487B (en) * 2008-05-30 2013-08-21 新奥特(北京)视频技术有限公司 Method for synchronously changing scene areas of caption machine
CN101577110B (en) * 2009-05-31 2012-04-25 腾讯科技(深圳)有限公司 Method for playing videos and video player
CN102110430A (en) * 2009-12-23 2011-06-29 富士通株式会社 Method and system for multi-window display
CN102254560A (en) * 2010-05-19 2011-11-23 安凯(广州)微电子技术有限公司 Audio processing method in mobile digital television recording
CN102254560B (en) * 2010-05-19 2013-05-08 安凯(广州)微电子技术有限公司 Audio processing method in mobile digital television recording
CN101840494A (en) * 2010-05-25 2010-09-22 福建新大陆电脑股份有限公司 Device and method for judging bar code boundary
CN101883270A (en) * 2010-06-10 2010-11-10 上海海事大学 Method for inserting related media in independent video streams
CN101894336B (en) * 2010-06-28 2013-10-09 深圳市江波龙电子有限公司 Video on demand payment system and payment method thereof
CN101894336A (en) * 2010-06-28 2010-11-24 深圳市江波龙电子有限公司 Video on demand payment system and payment method thereof
CN101901126A (en) * 2010-07-12 2010-12-01 东北大学 Method for controlling combined large-screen stream media playing computer
CN101901126B (en) * 2010-07-12 2012-01-04 东北大学 Method for controlling combined large-screen stream media playing computer
CN102215427A (en) * 2011-05-17 2011-10-12 深圳创维-Rgb电子有限公司 Method and device for synchronously playing multimedia information on television, and television
WO2013044772A1 (en) * 2011-09-28 2013-04-04 歌尔声学股份有限公司 Method, system and device for simultaneously viewing different pictures on identical screen
US8670072B1 (en) 2011-12-05 2014-03-11 Guangzhou Ucweb Computer Technology Co., Ltd Method and apparatus for streaming media data processing, and streaming media playback equipment
WO2013082965A1 (en) * 2011-12-05 2013-06-13 优视科技有限公司 Streaming media data processing method and apparatus and streaming media data reproducing device
CN102522096A (en) * 2011-12-07 2012-06-27 中国航空无线电电子研究所 Synchronous replay system for multi-channel audio and video and method thereof
CN102547482B (en) * 2011-12-30 2014-05-07 北京锐安科技有限公司 Synchronous playing method of multi-path IP (Internet Protocol) audio-video stream
CN102547482A (en) * 2011-12-30 2012-07-04 北京锐安科技有限公司 Synchronous playing method of multi-path IP (Internet Protocol) audio-video stream
CN102623032B (en) * 2012-04-01 2014-09-17 深圳英飞拓科技股份有限公司 Method for realizing synchronous playback of hard disk recorder
CN102623032A (en) * 2012-04-01 2012-08-01 深圳英飞拓科技股份有限公司 Method for realizing synchronous playback of hard disk recorder
CN102625164A (en) * 2012-04-06 2012-08-01 上海车音网络科技有限公司 Multimedia data processing platform, multimedia reading material, system and method
CN103686347A (en) * 2012-09-14 2014-03-26 美国博通公司 Adaptable media processing architectures
CN103905783A (en) * 2012-12-25 2014-07-02 杭州海康威视数字技术股份有限公司 Method and device for performing decoding and displaying on video stream
CN103905783B (en) * 2012-12-25 2017-09-01 杭州海康威视数字技术股份有限公司 The method and apparatus of decoding display is carried out to video flowing
CN103065503B (en) * 2012-12-27 2015-05-20 重庆大学 Reappearing method and system controlling river reach ship traveling dynamic and signal revealing process
CN103065503A (en) * 2012-12-27 2013-04-24 重庆大学 Reappearing method and system controlling river reach ship traveling dynamic and signal revealing process
CN108495145A (en) * 2013-04-01 2018-09-04 微软技术许可有限责任公司 Dynamic track switching in media stream
CN103237185A (en) * 2013-05-15 2013-08-07 无锡北斗星通信息科技有限公司 System for carrying out broadcasting control on ATSC (Advanced Television System Committee) signal at television station end
CN103246752A (en) * 2013-05-30 2013-08-14 北京瑞翔恒宇科技有限公司 System for associating ppt document with audio
CN105578204A (en) * 2014-10-14 2016-05-11 青岛海信电器股份有限公司 Multi-video-data displaying method and apparatus
CN105721920A (en) * 2014-12-04 2016-06-29 中国移动通信集团公司 Method and apparatus for cooperative playing of media on multi devices, and terminal
CN105721920B (en) * 2014-12-04 2019-04-02 中国移动通信集团公司 A kind of media cooperate with the method, apparatus and terminal of broadcasting in more equipment
CN105392024B (en) * 2015-10-28 2018-04-24 中国电子科技集团公司第二十八研究所 A kind of multi-source heterogeneous data synchronized playback method
CN105392024A (en) * 2015-10-28 2016-03-09 中国电子科技集团公司第二十八研究所 Multi-source heterogeneous data synchronous replaying method
CN105704506A (en) * 2016-01-19 2016-06-22 北京流金岁月文化传播股份有限公司 Device and method for synchronizing audio and video coding labial sound
CN107333164A (en) * 2016-04-29 2017-11-07 北京学而思教育科技有限公司 A kind of image processing method and device
CN107333164B (en) * 2016-04-29 2020-05-15 北京学而思教育科技有限公司 Image data processing method and device
CN107809659A (en) * 2017-10-27 2018-03-16 北京雷石天地电子技术有限公司 The method and device that a kind of multi-channel video fusion plays
CN108881918A (en) * 2017-11-29 2018-11-23 北京视联动力国际信息技术有限公司 A kind of processing method of multi-channel video and a kind of multimedia terminal
CN108174283A (en) * 2017-12-27 2018-06-15 威创集团股份有限公司 A kind of vision signal source generating method and device
CN108881917B (en) * 2018-07-27 2019-12-03 西安电子科技大学 Parallel decoding quicksort method based on high speed real-time decoding platform
CN108881917A (en) * 2018-07-27 2018-11-23 西安电子科技大学 Parallel decoding quicksort method based on high speed real-time decoding platform
CN109828742A (en) * 2019-02-01 2019-05-31 珠海全志科技股份有限公司 Voice-frequency-multichannel synchronism output method, computer installation and computer readable storage medium
CN110225363A (en) * 2019-07-10 2019-09-10 央视国际网络无锡有限公司 A kind of method that difference live source switches and is superimposed different filters
CN110225363B (en) * 2019-07-10 2021-08-06 央视国际网络无锡有限公司 Method for switching different live broadcast sources and superposing different filters

Also Published As

Publication number Publication date
CN100352270C (en) 2007-11-28

Similar Documents

Publication Publication Date Title
CN100352270C (en) Synchronous broadcast controlling method capable of supporting multi-source stream media
CN1190081C (en) Method and apparatus for processing, transmitting and receiving dynamic image data
CN1141844C (en) Moving image composing system
CN1299502C (en) Decoding device and decoding method
CN1220359C (en) Communication terminal, server, relay device, broadcast communication system, method and program
CN1178472C (en) Image signal system converter and TV set
CN1174607C (en) Device and method for image coding, device for decoding image
CN1187929C (en) Data receiving device, data receiving method, data transmission method and data storage medium
CN101035279A (en) Method for using the information set in the video resource
CN1714577A (en) Transmission of video
CN1890972A (en) Content distribution/reception device, content transmission/reception system, content distribution/reception method, and content distribution/reception program
CN1276602C (en) Broadcasting system
CN1279753C (en) Video relay device, video distribution system, video relay method
CN1314762A (en) Data transmission method and data transmission system
CN1682539A (en) Apparatus and method for adapting 2D and 3D stereoscopic video signal
CN1269352C (en) Decoding apparatus and decoding method
CN101060617A (en) A video-on-demand control method, customer terminal equipment and switching control device
CN1574944A (en) Dynamic image decoding device
CN101052127A (en) Information-processing apparatus, information-processing method, recording medium and program
CN1692640A (en) Broadcast recording system, recording apparatus, broadcasting apparatus, and recording program for saving storage space of recording medium used for recording contents
CN1852414A (en) Video code-flow gamma characteristic correction method and multi-point control unit
CN1659883A (en) A method and system for remote playback of a DVD
CN1830210A (en) Live streaming broadcast method, live streaming broadcast device, live streaming broadcast system, program, recording medium, broadcast method, and broadcast device
CN1319996A (en) Image code/decode equipment and method, image recording and transmitting equipment
CN1812548A (en) Recorder and method therefor, player and method therefor, program, and recording medium

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
GR01 Patent grant
C14 Grant of patent or utility model
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20071128

Termination date: 20131021

C17 Cessation of patent right