CN101938606B - Method, system and device for propelling multimedia data - Google Patents
Method, system and device for propelling multimedia data Download PDFInfo
- Publication number
- CN101938606B CN101938606B CN 200910088167 CN200910088167A CN101938606B CN 101938606 B CN101938606 B CN 101938606B CN 200910088167 CN200910088167 CN 200910088167 CN 200910088167 A CN200910088167 A CN 200910088167A CN 101938606 B CN101938606 B CN 101938606B
- Authority
- CN
- China
- Prior art keywords
- medium data
- serial number
- data bag
- collecting device
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The embodiment of the invention discloses a method for propelling multimedia data. In the method, at least two multimedia data collection devices package collected multimedia data, package corresponding sequence numbers in the packaged multimedia data packets according to the packaging sequence and transmits the multimedia data packets to a virtual studio server; the virtual studio server adjusts the sequence numbers in the received multimedia data packets, wherein the adjusted sequence numbers contained in the multimedia data packets the time of which are transmitted by the multimedia data collection device are identical; the multimedia data packets with the adjusted sequence numbers are transmitted to a live broadcast server; the live broadcast server combines the data in the multimedia data packets with the consistent sequence number and transmits the combined multimedia data subject to compression coding to a live broadcast client. The embodiment of the invention also discloses a system and a device for propelling multimedia data. The invention can effectively save the bandwidth of a multimedia live broadcast system.
Description
Technical field
The present invention relates to the network application field, relate in particular to a kind of multi-medium data method for pushing, system and equipment.
Background technology
In recent years, along with interview class program becoming much more popular rapidly on TV media, Internet video interview system arises at the historic moment.Traditional Internet video interview system class is similar to the simple amplification of network teleconference system, as to carry out the supporting platform of real time communication as the user who joins virtual studio based on standard agreement video conferencing system H.323.
Traditional video conferencing system generally adopts client end/server end (C/S) network architecture; This network architecture is made up of server and a plurality of client; The video/audio that server is responsible for server end is collected and the video/audio of client send other clients except that this client to, and client receives behind the video/audio that server transmits that carry out data in this locality synthetic and show.
The client terminal quantity that existing Internet video interview system can connect receives the restriction of hardware resources such as system bandwidth, server load; Make system bigger for the number of users quantitative limitation; Want to watch that for many talk show is live; And can not carry out the user of real-time, interactive (, being similar to the preceding spectators of the television set of watching talk show on the TV media) as carry out operations such as real-time communication with the welcome guest, traditional Internet video interview system can not well support.
And the video/audio that server need collect server end and the video/audio of client send other clients except that this client respectively to, the bigger system bandwidth of having wasted.
Summary of the invention
The embodiment of the invention provides a kind of multi-medium data method for pushing, system and equipment, is used to save the bandwidth use of Internet video interview system.
The embodiment of the invention provides a kind of multi-medium data method for pushing, and this method comprises:
At least two multi-medium data collecting devices are packed the multi-medium data that collects, and number be encapsulated in corresponding order in the multi-medium data bag after the packing and this multi-medium data bag is sent to the virtual studio server according to the packing order;
Serial number in the multi-medium data bag that the adjustment of virtual studio server receives, the serial number that the identical multi-medium data bag of adjustment back multi-medium data collecting device transmitting time is comprised is identical; The adjusted multi-medium data bag of serial number is sent to live direct broadcast server;
Live direct broadcast server synthesizes the data in the multi-medium data bag of the serial number unanimity that receives, and sends to live live client behind the multi-medium data compressed encoding after will synthesizing.
The embodiment of the invention provides a kind of multi-medium data supplying system, and this system comprises:
At least two multi-medium data collecting devices; Said multi-medium data collecting device is used for the multi-medium data that collects is packed, and number is encapsulated in corresponding order in the multi-medium data bag after the packing and this multi-medium data bag is sent to the virtual studio server according to the packing order;
The virtual studio server is used for adjusting the serial number of the multi-medium data bag that receives, and the serial number that the identical multi-medium data bag of adjustment back multi-medium data collecting device transmitting time is comprised is identical; The adjusted multi-medium data bag of serial number is sent to live direct broadcast server;
Live direct broadcast server, the data that are used for the multi-medium data bag that the serial number that receives is consistent are synthesized, and send to live live client behind the multi-medium data compressed encoding after will synthesizing;
Live live client is used to receive said multi-medium data.
The embodiment of the invention provides a kind of virtual studio server, and this server comprises:
The packet receiving element is used for the multi-medium data bag that the receiving multimedia data collecting device is sent;
The packet lock unit is used for adjusting the serial number of the multi-medium data bag that said packet receiving element receives, and the serial number that the identical multi-medium data bag of adjustment back multi-medium data collecting device transmitting time is comprised is identical;
The packet retransmission unit is used for the adjusted multi-medium data bag of serial number is sent to live direct broadcast server.
The embodiment of the invention provides a kind of live direct broadcast server, and this server comprises:
The packet receiving element is used to receive the multi-medium data bag that comprises serial number that the virtual studio server is sent;
The multimedia synthesis unit; The data of the multi-medium data bag that the serial number that is used for said packet receiving element is received is consistent are synthesized; Wherein, The consistent multi-medium data bag of said serial number is specially: through in said virtual studio server; Choose a multi-medium data collecting device at least two multi-medium data collecting devices as time reference equipment, the serial number in the multi-medium data bag that adjustment receives, and then the identical multi-medium data bag of transmitting time that is as the criterion of the time that obtains with said time reference equipment;
The live unit of multimedia is used for said multimedia synthesis unit sending to live live client behind the multimedia compressed encoding after synthetic.
In the scheme that the embodiment of the invention provides; At least two multi-medium data collecting devices are packed the multi-medium data that collects, and number be encapsulated in corresponding order in the multi-medium data bag after the packing and this multi-medium data bag is sent to the virtual studio server according to the packing order; Serial number in the multi-medium data bag that the adjustment of virtual studio server receives; The serial number that the identical multi-medium data bag of adjustment back multi-medium data collecting device transmitting time is comprised is identical, and the adjusted multi-medium data bag of serial number is sent to live direct broadcast server; Live direct broadcast server synthesizes the data in the multi-medium data bag of the serial number unanimity that receives, and sends to live live client behind the multi-medium data compressed encoding after will synthesizing.Make that the virtual studio server synthesizes the data in a plurality of multi-medium data bags of the serial number unanimity that receives, compress and the back unification of encoding sends to live live client; And need all not send once for each live live client a plurality of multi-medium data bags, effectively saved system bandwidth.
Description of drawings
The method flow sketch map that Fig. 1 provides for the embodiment of the invention;
Fig. 2 is the L2 cache sketch map in the embodiment of the invention;
Fig. 3 is the synthetic sketch map of the video in the embodiment of the invention;
The system configuration sketch map that Fig. 4 provides for the embodiment of the invention;
The multi-medium data collecting device structural representation that Fig. 5 provides for the embodiment of the invention;
The virtual studio server architecture sketch map that Fig. 6 provides for the embodiment of the invention;
The live direct broadcast server structural representation that Fig. 7 provides for the embodiment of the invention.
Embodiment
For the bandwidth of saving Internet video interview system is used; Make the number of users of Internet video interview system not receive the restriction of server bandwidth etc.; The embodiment of the invention provides a kind of multi-medium data method for pushing, in this method, receive the multi-medium data bag that a plurality of multi-medium data collecting devices send after; Data in the multi-medium data bag that transmitting time is consistent are synthesized, and send to live live client behind the multi-medium data compressed encoding after will synthesizing.
Referring to Fig. 1, the multi-medium data method for pushing that the embodiment of the invention provides specifically may further comprise the steps:
Step 10: at least two multi-medium data collecting devices are packed the multi-medium data that collects, and number be encapsulated in corresponding order in the multi-medium data bag after the packing and this multi-medium data bag is sent to the virtual studio server according to the packing order;
Step 11: the serial number in the multi-medium data bag that the adjustment of virtual studio server receives, the serial number that the identical multi-medium data bag of transmitting time of adjustment back multi-medium data collecting device is comprised is identical; The adjusted multi-medium data bag of serial number is sent to live direct broadcast server;
Step 12: live direct broadcast server synthesizes the data in the multi-medium data bag of the serial number unanimity that receives;
Step 13: send to live live client behind the multi-medium data compressed encoding after live direct broadcast server will synthesize.
In the step 10, when multi-medium data comprised voice data and video data, the mode that the multi-medium data that collects is packed was: with the coding video data that collects, coded data is encapsulated in the video packets of data; The voice data that collects, playing duration is identical with the playing duration of said video data is encoded, coded data is encapsulated in the packets of audio data.Concrete: the mode to the audio, video data coding can be following:
Will be according to video acquisition frame per second fr
vThe M that a collects frame of video is encoded, and M is not less than 1 integer; With the current byte number that collects do
Voice data encode, wherein, fr
aBe audio sample rate, i.e. the per second hits; N is a channel number, the channel number of promptly sampling; Bps is a sampling resolution, i.e. each sampled point figure place.
After adopting aforesaid way to the audio, video data coding; The playing duration of the original video data before the video data encoder corresponding codes in the video packets of data is the playing duration of M frame of video, and the playing duration of the original audio data before the coded audio data corresponding codes in the packets of audio data is identical with the playing duration of M frame of video.Wherein, the value of M can be 1.
Can pack to collecting video data and voice data in two steps:
The first step; The original audio data that collects is split into the data segment of size for
; Utilize audio compress standard to encode (like coding standard G.729) again, obtain the coded data of a packets of audio data.Simultaneously, according to the video acquisition frame per second, utilize video compression standard (like coding standard H.264), to obtain the coded data of a video packets of data to every frame coding video data;
In second step, application data section such as the serial number (Index) that the current video data bag is corresponding, ID and video data encoder are encapsulated in the video packets of data, obtain the complete video packet.Same, application data section such as the serial number (Index) that current packets of audio data is corresponding, ID and coded audio data are encapsulated in the packets of audio data, obtain complete packets of audio data.The Index of first packet is 0, and the Index of follow-up data bag can increase by 1 successively.
Data packet format such as following table:
With the M frame of video is that base unit comes video and voice data packing, is in order to make in the unit interval number-of-packet of video and audio frequency identical, and its serial number Index has corresponding relationship in time, operates to carry out time synchronized after convenient.
In the step 11, the serial number in the multi-medium data bag that the adjustment of virtual studio server receives, its concrete implementation is following:
At first, choose a multi-medium data collecting device at least two multi-medium data collecting devices as time reference equipment;
Then; For other each the multi-medium data collecting devices except that time reference equipment, confirm that this multi-medium data collecting device begins to begin to send to the virtual studio server time migration of multi-medium data bag to virtual studio server transmission multi-medium data bag and time reference equipment;
At last, according to the multi-medium data collecting device time corresponding skew of confirming, adjust the serial number of the multi-medium data bag of this multi-medium data collecting device transmission.
Above-mentioned definite time migration specifically can may further comprise the steps:
Step S01: confirm the corresponding video frame number of propagation delay time (first propagation delay time) between multi-medium data collecting device and virtual studio server according to following formula: Δ 1=t1/T1; Wherein t1 is first propagation delay time, and T1 is the playing duration of the multi-medium data bag of this multi-medium data collecting device transmission;
Step S02: confirm the corresponding video frame number of propagation delay time (second propagation delay time) between time reference equipment and virtual studio server according to following formula: Δ 2=t2/T2; Wherein t2 is second propagation delay time, and F2 is the playing duration of the multi-medium data bag of time reference equipment transmission;
Step S03: when confirming that the virtual studio server receives first multi-medium data bag of multi-medium data collecting device, receive the serial number I that the multi-medium data bag of time reference equipment is comprised recently;
Step S04: confirm that according to following formula time migration is: Δ=I-| Δ 1-Δ 2|.
If the video packets of data that the multi-medium data collecting device sends comprises according to video acquisition frame per second fr
V1The data of the M that a collects frame of video, then T1 equals M/fr
V1
If the video packets of data that time reference equipment sends comprises according to video acquisition frame per second fr
V2The data of the M that a collects frame of video, then T2 equals M/fr
V2Wherein, M is not less than 1 integer.Fr
V1With fr
V2Can equate.Less than the distance threshold value that is provided with in advance, when promptly the time reference device very closely can be ignored apart from the virtual studio server, Δ 2 values were 0 in the distance of time reference equipment and virtual studio server.
Definite mode of first propagation delay time is: before the multi-medium data collecting device is ready for sending first video or packets of audio data, at first send a handshake packet to the virtual studio server, and record local zone time P
1, after the virtual studio server is received handshake packet, returning at once and confirm to wrap, the multi-medium data collecting device receives that the local zone time of confirming bag is P
2, obtain one way first propagation delay time thus:
And first propagation delay time that will obtain is notified to the virtual studio server; The virtual studio server confirms that according to first propagation delay time corresponding video frame number is:
Definite mode of second propagation delay time is identical with definite mode of first propagation delay time, repeats no more here.
According to aforesaid way, the serial number of the multi-medium data bag that adjustment multi-medium data collecting device sends is specially: with the serial number of this multi-medium data bag with calculate this multi-medium data collecting device time corresponding skew and superpose.For example, if time migration Δ=5, then the serial number with the multi-medium data bag adds 5, if time migration Δ=-5, then the serial number with the multi-medium data bag subtracts 5.
The data that the virtual studio server is transmitted to live direct broadcast server are based on that ICP/IP protocol transmits, like this to ensure the order of live direct broadcast server end data bag.
In the step 12, live direct broadcast server synthesizes the data in the multi-medium data bag of the serial number unanimity that receives, and its concrete implementation is following:
The multi-medium data bag that receives is put into the corresponding synthetic buffer unit of multi-medium data collecting device that sends this multi-medium data bag;
According to first polling time interval of setting, from each synthetic buffer unit, read current serial number the earliest and identical multi-medium data bag;
The multi-medium data that the multi-medium data bag that reads is carried synthesizes.
As shown in Figure 2; Live direct broadcast server is corresponding to each multi-medium data collecting device; Have two synthetic buffer units; One of them synthetic buffer unit (V) is used to store the video packets of data that corresponding multi-medium data collecting device sends, and another synthetic buffer unit (A) is used to store the packets of audio data that corresponding multi-medium data collecting device sends, certainly; If the multi-medium data collecting device also sends the packet of other types, then live direct broadcast server can also have the synthetic buffer unit of the packet that is used to store these other types corresponding to this multi-medium data collecting device.The serial number Index of the packet of the numeral storage among Fig. 2.
Synthetic buffer unit can adopt the formation mode of data first in first out, and the serial number of the same memory cell cached data packet of individual queue is identical.For each formation is provided with synthetic pointer, be used in reference to memory cell, first memory cell of synthetic pointer initial directional formation to the synthetic packet of the current needs of storage.
Synthetic method specifically can for: adopt a poll thread control synthetic at interval; Make that polling time interval is Δ t; Whether the data that each poll thread detects when triggering in all synthetic buffer units of current synthetic pointed are here, if do not have herely, wait for that then poll triggers next time; If it is here that 3 polls still do not have, the packet that then uses and not synthesizing to the previous packet of packet; If here, then the audio frequency of looking in the packet of current synthetic pointed synthesized, and current synthetic pointer is moved down the position of a memory cell.
Video data uses length and width to change and the mode of combination is synthesized, and is as shown in Figure 3.Voice data synthesizes audio mixing through simple arithmetic average, as shown in the formula:
Wherein, A
iThe voice data of representing i synthetic buffer unit, Mix is the voice data after synthetic.
In the step 13, with sending to live live client behind the multi-medium data compressed encoding after synthetic, its concrete implementation is following:
Multi-medium data after synthetic is put into the broadcast buffer unit;
According to second polling time interval of setting, from play buffer unit, read current serial number multi-medium data the earliest, the serial number of multi-medium data is meant the serial number in the synthetic employed multi-medium data bag of this multi-medium data;
With sending to live live client through point-to-point (P2P) network behind the multi-medium data compressed encoding that reads.
Still referring to Fig. 2; Live direct broadcast server also has two and plays buffer unit; One of them is play buffer unit (V) and is used to store the video data after synthesizing, and another is play buffer unit (A) and is used to store the voice data after synthesizing, certainly; If the multi-medium data collecting device also sends the packet of other types, then live direct broadcast server can also have the broadcast buffer unit that is used to store these other types data after synthesizing.
Play the formation mode that buffer unit can adopt the data first in first out, the serial number of the multi-medium data of the same memory cell buffer memory of individual queue is identical.For this formation is provided with playing pointer, be used in reference to the current memory cell that needs the data of broadcast of storage, first memory cell of playing pointer initial directional formation.
The data live broadcasting method specifically can for: adopt a poll thread to come controls playing at interval; Make that polling time interval is Δ t; When each poll thread triggers; Send to each live live client through the P2P network behind the multi-medium data compressed encoding in all packets that current playing pointer is pointed to, and current playing pointer is moved down the position of a memory cell.
In order to make live signal more steady; The data that the tolerance network jitter causes arrive uneven; Start when live; Must surpass 50% at the degree of filling of playing buffer unit and just begin to carry out, initial polling time interval is the inverse of video acquisition frame per second, i.e.
Use following strategy to come self adaptation adjustment to play the capacity of buffer unit: when the degree of filling θ of play unit less than 25%, or surpass at 75% o'clock, the polling time interval linearity reduces or increases, to adjust aggregate velocity:
In order to prevent underflow on the buffer cell,, then suspend live degree of filling and surpass 50% up to the broadcast buffer unit if the memory cell that current playing pointer points to does not have data.If do not have after current synthetic pointer moves down idlely must play buffer unit, then suspend and syntheticly surpass 50% up to the degree of filling of playing buffer unit.
Referring to Fig. 4, the embodiment of the invention also provides a kind of multi-medium data supplying system, and this system comprises:
At least two multi-medium data collecting devices 40; Said multi-medium data collecting device is used for the multi-medium data that collects is packed, and number is encapsulated in corresponding order in the multi-medium data bag after the packing and this multi-medium data bag is sent to the virtual studio server according to the packing order;
Virtual studio server 41 is used for adjusting the serial number of the multi-medium data bag that receives, and the serial number that the identical multi-medium data bag of adjustment back transmitting time is comprised is identical; The adjusted multi-medium data bag of serial number is sent to live direct broadcast server;
Live direct broadcast server 42, the data that are used for the multi-medium data bag that the serial number that receives is consistent are synthesized, and send to live live client through the P2P network behind the multi-medium data compressed encoding after will synthesizing;
Live live client 43 is used to receive said multi-medium data.
Said multi-medium data collecting device 40 is used for:
With the coding video data that collects, coded data is encapsulated in the video packets of data; The voice data that collects, playing duration is identical with the playing duration of said video data is encoded, coded data is encapsulated in the packets of audio data.
Said multi-medium data collecting device 40 is used for:
Will be according to video acquisition frame per second fr
vThe M that a collects frame of video is encoded, and M is not less than 1 integer; With the current byte number that collects do
Voice data encode said fr
aBe audio sample rate, said N is a channel number, and said bps is the sampled point figure place.
Said virtual studio server 41 is used for:
Choose a multi-medium data collecting device in said at least two multi-medium data collecting devices as time reference equipment; For other each the multi-medium data collecting devices except that said time reference equipment, confirm that this multi-medium data collecting device begins to begin to send to said virtual studio server the time migration of multi-medium data bag to said virtual studio server transmission multi-medium data bag and said time reference equipment; According to the multi-medium data collecting device time corresponding skew of confirming, adjust the serial number of the multi-medium data bag of this multi-medium data collecting device transmission.
Said virtual studio server 41 is used for:
Confirm the corresponding video frame number of first propagation delay time between this multi-medium data collecting device and said virtual studio server according to following formula: Δ 1=t1/T1; Wherein t1 is first propagation delay time, and T1 is the playing duration of the multi-medium data bag of this multi-medium data collecting device transmission; Confirm the corresponding video frame number of second propagation delay time between said time reference equipment and said virtual studio server according to following formula: Δ 2=t2/T2, wherein t2 is second propagation delay time, F2 is the video acquisition frame per second of said time reference equipment; When confirming to receive first multi-medium data bag of this multi-medium data collecting device, receive the serial number I that the multi-medium data bag of said time reference equipment is comprised recently; Confirm that according to following formula said time migration is: Δ=I-| Δ 1-Δ 2|; Serial number and said time migration that said multi-medium data bag is comprised superpose.
If the video packets of data that this multi-medium data collecting device sends comprises according to video acquisition frame per second fr
V1The data of the M that a collects frame of video, then T1 equals M/fr
V1If the video packets of data that said time reference equipment sends comprises according to video acquisition frame per second fr
V2The data of the M that a collects frame of video, then T2 equals M/fr
V2Said M is not less than 1 integer.
Said multi-medium data collecting device is virtual studio client or virtual studio acquisition server; Said virtual studio client is used for, and gathers the multi-medium data of looking the mutual user of audio frequency with said virtual studio server in real time; Said virtual studio acquisition server is used for, and gathers the multi-medium data of performing in a radio or TV programme the hall.
Said virtual studio server 41 is used for:
As time reference equipment, and said Δ 2 is 0 with said virtual studio acquisition server.
Said live direct broadcast server 42 comprises:
At least two synthetic buffer units are used to store the multi-medium data bag; Each synthetic buffer unit is corresponding with a multi-medium data collecting device in said at least two multi-medium data collecting devices, and the corresponding multi-medium data collecting device of different synthetic buffer unit is different;
Synthesis unit is used for the multi-medium data bag that receives is put into the corresponding synthetic buffer unit of multi-medium data collecting device that sends this multi-medium data bag; According to the polling time interval of setting, from each synthetic buffer unit, read current serial number the earliest and identical multi-medium data bag, the multi-medium data that the multi-medium data bag that reads is carried synthesizes;
Broadcast unit is used for the multi-medium data after synthetic is put into the broadcast buffer unit; Polling time interval according to setting reads current serial number multi-medium data the earliest from said broadcast buffer unit, the serial number of multi-medium data is meant the serial number in the synthetic employed multi-medium data bag of this multi-medium data; With sending to live live client behind the multi-medium data compressed encoding that reads;
Play buffer unit, be used for the buffer memory multi-medium data.
Native system can also comprise:
Web server 44 and database server 45, being used to the interactive client of literal provides literal interactive function;
The interactive client 46 of literal is used for carrying out the literal interaction with Web server;
Long-distance Control server 47, being used to administrative staff provides each server and real-time remote monitoring function.
Referring to Fig. 5, the embodiment of the invention also provides a kind of multi-medium data collecting device, and this equipment comprises:
Said packing data unit 50 comprises:
Video packing unit is used for the coding video data that collects is encapsulated in coded data in the video packets of data;
Audio frequency packing unit is used for the voice data that collects, playing duration is identical with the playing duration of said video data is encoded, and coded data is encapsulated in the packets of audio data.
Said video packing unit is used for:
Will be according to video acquisition frame per second fr
vThe M that a collects frame of video is encoded, and M is not less than 1 integer;
Said audio frequency packing unit is used for:
With the current byte number that collects do
Voice data encode said fr
aBe audio sample rate, said N is a channel number, and said bps is the sampled point figure place.
Referring to Fig. 6, the embodiment of the invention also provides a kind of said virtual studio server, and this server comprises:
Packet receiving element 60 is used for the multi-medium data bag that the receiving multimedia data collecting device is sent;
Packet lock unit 61 is used for adjusting the serial number of the multi-medium data bag that said packet receiving element receives, and the serial number that the adjustment back is comprised by the identical multi-medium data bag of multi-medium data collecting device transmitting time is identical;
Packet retransmission unit 62 is used for the adjusted multi-medium data bag of serial number is sent to live direct broadcast server.
Said packet lock unit 61 comprises:
Reference device is chosen the unit, and a multi-medium data collecting device that is used for choosing said at least two multi-medium data collecting devices is as time reference equipment;
The unit is confirmed in time migration; Be used for for other each the multi-medium data collecting devices except that said time reference equipment, confirm that this multi-medium data collecting device begins to begin to send to said virtual studio server the time migration of multi-medium data bag to said virtual studio server transmission multi-medium data bag and said time reference equipment;
The serial number adjustment unit is used for adjusting the serial number of the multi-medium data bag of this multi-medium data collecting device transmission according to the multi-medium data collecting device time corresponding skew of confirming.
Said time migration confirms that the unit is used for:
Confirm the corresponding video frame number of first propagation delay time between this multi-medium data collecting device and said virtual studio server according to following formula: Δ 1=t1/T1; Wherein t1 is first propagation delay time, and T1 is the playing duration of the multi-medium data bag of this multi-medium data collecting device transmission;
Confirm the corresponding video frame number of second propagation delay time between said time reference equipment and said virtual studio server according to following formula: Δ 2=t2/T2; Wherein t2 is second propagation delay time, and T2 is the playing duration of the multi-medium data bag of said time reference equipment transmission;
When confirming to receive first multi-medium data bag of this multi-medium data collecting device, receive the serial number I that the multi-medium data bag of said time reference equipment is comprised recently; Confirm that according to following formula said time migration is: Δ=I-| Δ 1-Δ 2|;
Said serial number adjustment unit is used for, and serial number and said time migration that said multi-medium data bag is comprised superpose.
If the video packets of data that this multi-medium data collecting device sends comprises according to video acquisition frame per second fr
V1The data of the M that a collects frame of video, then T1 equals M/fr
V1If the video packets of data that said time reference equipment sends comprises according to video acquisition frame per second fr
V2The data of the M that a collects frame of video, then T2 equals M/fr
V2Said M is not less than 1 integer.Said Δ 2 can be 0.
Referring to Fig. 7, the embodiment of the invention also provides a kind of live direct broadcast server, and this server comprises:
The live unit 72 of multimedia is used for said multimedia synthesis unit sending to live live client behind the multimedia compressed encoding after synthetic.
This server also comprises:
At least two synthetic buffer units 73 are used for buffer memory multi-medium data bag;
Said multimedia synthesis unit 71 is used for, and the multi-medium data bag that said packet receiving element receives is put into the corresponding synthetic buffer unit of multi-medium data collecting device that sends this multi-medium data bag; According to the polling time interval of setting, from each synthetic buffer unit, read current serial number the earliest and identical multi-medium data bag, the multi-medium data that the multi-medium data bag that reads is carried synthesizes.
This server also comprises:
Play buffer unit 74, be used for the buffer memory multi-medium data;
The live unit 72 of said multimedia is used for, and the multi-medium data after said multimedia synthesis unit is synthesized is put into the broadcast buffer unit; Polling time interval according to setting reads current serial number multi-medium data the earliest from said broadcast buffer unit, the serial number of multi-medium data is meant the serial number in the synthetic employed multi-medium data bag of this multi-medium data; With sending to live live client behind the multi-medium data compressed encoding that reads.The initial value of said polling time interval
Fr wherein
vBe the video acquisition frame per second.
Said multimedia synthesis unit 71 also is used for:
Adopt following formula that said polling time interval is adjusted:
wherein θ is the degree of filling of said broadcast buffer unit.
To sum up, beneficial effect of the present invention comprises:
In the scheme that the embodiment of the invention provides; At least two multi-medium data collecting devices are packed the multi-medium data that collects, and number be encapsulated in corresponding order in the multi-medium data bag after the packing and this multi-medium data bag is sent to the virtual studio server according to the packing order; Serial number in the multi-medium data bag that the adjustment of virtual studio server receives, the serial number that the identical multi-medium data bag of adjustment back transmitting time is comprised is identical, and the adjusted multi-medium data bag of serial number is sent to live direct broadcast server; Live direct broadcast server synthesizes the data in the multi-medium data bag of the serial number unanimity that receives, and sends to live live client behind the multi-medium data compressed encoding after will synthesizing.Make that the virtual studio server synthesizes the data in a plurality of multi-medium data bags of the transmitting time unanimity that receives, compress and the back unification of encoding sends to live live client; And need all not send once for each live live client a plurality of multi-medium data bags, effectively saved system bandwidth.
Obviously, those skilled in the art can carry out various changes and modification to the present invention and not break away from the spirit and scope of the present invention.Like this, belong within the scope of claim of the present invention and equivalent technologies thereof if of the present invention these are revised with modification, then the present invention also is intended to comprise these changes and modification interior.
Claims (22)
1. a multi-medium data method for pushing is characterized in that, this method comprises:
At least two multi-medium data collecting devices are packed the multi-medium data that collects, and number be encapsulated in corresponding order in the multi-medium data bag after the packing and this multi-medium data bag is sent to the virtual studio server according to the packing order;
Choose a multi-medium data collecting device in said at least two multi-medium data collecting devices as time reference equipment; Serial number in the multi-medium data bag that virtual studio server adjustment receives, the serial number that the identical multi-medium data bag of transmitting time that adjustment back multi-medium data collecting device was as the criterion with time of said time reference equipment is comprised is identical; The adjusted multi-medium data bag of serial number is sent to live direct broadcast server;
Live direct broadcast server synthesizes the data in the multi-medium data bag of the serial number unanimity that receives, and sends to live live client behind the multi-medium data compressed encoding after will synthesizing.
2. the method for claim 1 is characterized in that, if said multi-medium data comprises voice data and video data, the said multi-medium data that collects is packed comprises:
With the coding video data that collects, coded data is encapsulated in the video packets of data;
The voice data that collects, playing duration is identical with the playing duration of said video data is encoded, coded data is encapsulated in the packets of audio data.
3. method as claimed in claim 2 is characterized in that, said the coding video data that collects is comprised:
Will be according to video acquisition frame per second fr
vThe M that a collects frame of video is encoded, and M is not less than 1 integer;
Said voice data that will collect, that playing duration is identical with the playing duration of said video data is encoded and is comprised:
With the current byte number that collects do
Voice data encode said fr
aBe audio sample rate, said N is a channel number, and said bps is the sampled point figure place.
4. the method for claim 1; It is characterized in that; Saidly choose a multi-medium data collecting device in said at least two multi-medium data collecting devices as time reference equipment, the serial number in the multi-medium data bag that the adjustment of virtual studio server receives comprises:
For other each the multi-medium data collecting devices except that said time reference equipment, confirm that this multi-medium data collecting device begins to begin to send to said virtual studio server the time migration of multi-medium data bag to said virtual studio server transmission multi-medium data bag and said time reference equipment;
According to the multi-medium data collecting device time corresponding skew of confirming, adjust the serial number of the multi-medium data bag of this multi-medium data collecting device transmission.
5. method as claimed in claim 4; It is characterized in that said definite this multi-medium data collecting device begins to begin to comprise to the time migration of said virtual studio server transmission multi-medium data bag to said virtual studio server transmission multi-medium data bag and said time reference equipment:
Confirm the corresponding video frame number of first propagation delay time between this multi-medium data collecting device and said virtual studio server according to following formula: Δ 1=t1/T1; Wherein t1 is first propagation delay time, and T1 is the playing duration of the multi-medium data bag of this multi-medium data collecting device transmission;
Confirm the corresponding video frame number of second propagation delay time between said time reference equipment and said virtual studio server according to following formula: Δ 2=t2/T2; Wherein t2 is second propagation delay time, and T2 is the playing duration of the multi-medium data bag of said time reference equipment transmission;
When confirming to receive first multi-medium data bag of this multi-medium data collecting device, receive the serial number I that the multi-medium data bag of said time reference equipment is comprised recently;
Confirm that according to following formula said time migration is: Δ=I-| Δ 1-Δ 2|;
The serial number of the multi-medium data bag that this multi-medium data collecting device of said adjustment sends comprises:
Serial number and said time migration that said multi-medium data is comprised superpose.
6. method as claimed in claim 5 is characterized in that, if the video packets of data that this multi-medium data collecting device sends comprises according to video acquisition frame per second fr
V1The data of the M that a collects frame of video, then T1 equals M/fr
V1
If the video packets of data that said time reference equipment sends comprises according to video acquisition frame per second fr
V2The data of the M that a collects frame of video, then T2 equals M/fr
V2Said M is not less than 1 integer.
7. method as claimed in claim 5 is characterized in that, if the distance of said time reference equipment and said virtual studio server is less than the distance threshold value of setting in advance, the value of then said Δ 2 is 0.
8. like arbitrary described method among the claim 1-7, it is characterized in that said live direct broadcast server synthesizes the data in the multi-medium data bag of the serial number unanimity that receives and comprises:
The multi-medium data bag that receives is put into the corresponding synthetic buffer unit of multi-medium data collecting device that sends this multi-medium data bag;
According to the polling time interval of setting, from each synthetic buffer unit, read current serial number the earliest and identical multi-medium data bag;
The multi-medium data that the multi-medium data bag that reads is carried synthesizes.
9. method as claimed in claim 8 is characterized in that, sends to live live client behind the multi-medium data compressed encoding after said will synthesizing and comprises:
Multi-medium data after synthetic is put into the broadcast buffer unit;
Polling time interval according to setting reads current serial number multi-medium data the earliest from said broadcast buffer unit, the serial number of multi-medium data is meant the serial number in the synthetic employed multi-medium data bag of this multi-medium data;
With sending to live live client behind the multi-medium data compressed encoding that reads.
10. method as claimed in claim 9 is characterized in that, the initial value of said polling time interval
Fr wherein
vBe the video acquisition frame per second.
11. method as claimed in claim 10 is characterized in that, adopts following formula that said polling time interval is adjusted:
12. a multi-medium data supplying system is characterized in that, this system comprises:
At least two multi-medium data collecting devices; Said multi-medium data collecting device is used for the multi-medium data that collects is packed, and number is encapsulated in corresponding order in the multi-medium data bag after the packing and this multi-medium data bag is sent to the virtual studio server according to the packing order;
The virtual studio server; A multi-medium data collecting device that is used for choosing said at least two multi-medium data collecting devices is as time reference equipment; Serial number in the multi-medium data bag that receives of adjustment, the serial number that the identical multi-medium data bag of transmitting time that adjustment back multi-medium data collecting device was as the criterion with time of said time reference equipment is comprised is identical; The adjusted multi-medium data bag of serial number is sent to live direct broadcast server;
Live direct broadcast server, the data that are used for the multi-medium data bag that the serial number that receives is consistent are synthesized, and send to live live client behind the multi-medium data compressed encoding after will synthesizing;
Live live client is used to receive said multi-medium data.
13. system as claimed in claim 12 is characterized in that, said multi-medium data collecting device is used for:
With the coding video data that collects, coded data is encapsulated in the video packets of data;
The voice data that collects, playing duration is identical with the playing duration of said video data is encoded, coded data is encapsulated in the packets of audio data.
14. system as claimed in claim 12 is characterized in that, said virtual studio server is used for:
For other each the multi-medium data collecting devices except that said time reference equipment, confirm that this multi-medium data collecting device begins to begin to send to said virtual studio server the time migration of multi-medium data bag to said virtual studio server transmission multi-medium data bag and said time reference equipment;
According to the multi-medium data collecting device time corresponding skew of confirming, adjust the serial number of the multi-medium data bag of this multi-medium data collecting device transmission;
Said virtual studio acquisition server is used for, and gathers the multi-medium data of performing in a radio or TV programme the hall.
15. system as claimed in claim 12 is characterized in that, said live direct broadcast server comprises:
At least two synthetic buffer units are used to store the multi-medium data bag; Each synthetic buffer unit is corresponding with a multi-medium data collecting device in said at least two multi-medium data collecting devices, and the corresponding multi-medium data collecting device of different synthetic buffer unit is different;
Synthesis unit is used for the multi-medium data bag that receives is put into the corresponding synthetic buffer unit of multi-medium data collecting device that sends this multi-medium data bag; According to first polling time interval of setting, from each synthetic buffer unit, read current serial number the earliest and identical multi-medium data bag, the multi-medium data that the multi-medium data bag that reads is carried synthesizes;
Broadcast unit is used for the multi-medium data after synthetic is put into the broadcast buffer unit; Second polling time interval according to setting reads current serial number multi-medium data the earliest from said broadcast buffer unit, the serial number of multi-medium data is meant the serial number in the synthetic employed multi-medium data bag of this multi-medium data; With sending to live live client behind the multi-medium data compressed encoding that reads;
Play buffer unit, be used to store multi-medium data.
16., it is characterized in that said multi-medium data collecting device is virtual studio client or virtual studio acquisition server like arbitrary described system among the claim 12-15;
Said virtual studio client is used for, and gathers the multi-medium data of looking the mutual user of audio frequency with said virtual studio server in real time.
17. a virtual studio server is characterized in that, this server comprises:
The packet receiving element is used for the multi-medium data bag that the receiving multimedia data collecting device is sent;
The packet lock unit; A multi-medium data collecting device that is used for choosing at least two multi-medium data collecting devices is as time reference equipment; Adjust the serial number in the multi-medium data bag that said packet receiving element receives, the serial number that the identical multi-medium data bag of transmitting time that adjustment back multi-medium data collecting device was as the criterion with time of said time reference equipment is comprised is identical;
The packet retransmission unit is used for the adjusted multi-medium data bag of serial number is sent to live direct broadcast server.
18. server as claimed in claim 17 is characterized in that, said packet lock unit comprises:
The unit is confirmed in time migration; Be used for for other each the multi-medium data collecting devices except that said time reference equipment, confirm that this multi-medium data collecting device begins to begin to send to said virtual studio server the time migration of multi-medium data bag to said virtual studio server transmission multi-medium data bag and said time reference equipment;
The serial number adjustment unit is used for adjusting the serial number of the multi-medium data bag of this multi-medium data collecting device transmission according to the multi-medium data collecting device time corresponding skew of confirming.
19. server as claimed in claim 18 is characterized in that, said time migration confirms that the unit is used for:
Confirm the corresponding video frame number of first propagation delay time between this multi-medium data collecting device and said virtual studio server according to following formula: Δ 1=t1/T1; Wherein t1 is first propagation delay time, and T1 is the playing duration of the multi-medium data bag of this multi-medium data collecting device transmission;
Confirm the corresponding video frame number of second propagation delay time between said time reference equipment and said virtual studio server according to following formula: Δ 2=t2/T2; Wherein t2 is second propagation delay time, and F2 is the playing duration of the multi-medium data bag of said time reference equipment transmission;
When confirming to receive first multi-medium data bag of this multi-medium data collecting device, receive the serial number I that the multi-medium data bag of said time reference equipment is comprised recently;
Confirm that according to following formula said time migration is: Δ=I-| Δ 1-Δ 2|;
Said serial number adjustment unit is used for, and serial number and said time migration that said multi-medium data bag is comprised superpose.
20. a live direct broadcast server is characterized in that, this server comprises:
The packet receiving element is used to receive the multi-medium data bag that comprises serial number that the virtual studio server is sent;
The multimedia synthesis unit; The data of the multi-medium data bag that the serial number that is used for said packet receiving element is received is consistent are synthesized; Wherein, The consistent multi-medium data bag of said serial number is specially: through in said virtual studio server; Choose a multi-medium data collecting device at least two multi-medium data collecting devices as time reference equipment, the serial number in the multi-medium data bag that adjustment receives, and then the identical multi-medium data bag of transmitting time that is as the criterion of the time that obtains with said time reference equipment;
The live unit of multimedia is used for said multimedia synthesis unit sending to live live client behind the multimedia compressed encoding after synthetic.
21. server as claimed in claim 20 is characterized in that, this server also comprises:
At least two synthetic buffer units are used for buffer memory multi-medium data bag;
Said multimedia synthesis unit is used for, and the multi-medium data bag that said packet receiving element receives is put into the corresponding synthetic buffer unit of multi-medium data collecting device that sends this multi-medium data bag; According to the polling time interval of setting, from each synthetic buffer unit, read current serial number the earliest and identical multi-medium data bag, the multi-medium data that the multi-medium data bag that reads is carried synthesizes.
22. server as claimed in claim 21 is characterized in that, this server also comprises:
Play buffer unit, be used for the buffer memory multi-medium data;
The live unit of said multimedia is used for, and the multi-medium data after said multimedia synthesis unit is synthesized is put into the broadcast buffer unit; Polling time interval according to setting reads current serial number multi-medium data the earliest from said broadcast buffer unit, the serial number of multi-medium data is meant the serial number in the synthetic employed multi-medium data bag of this multi-medium data; With sending to live live client behind the multi-medium data compressed encoding that reads.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910088167 CN101938606B (en) | 2009-07-03 | 2009-07-03 | Method, system and device for propelling multimedia data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910088167 CN101938606B (en) | 2009-07-03 | 2009-07-03 | Method, system and device for propelling multimedia data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101938606A CN101938606A (en) | 2011-01-05 |
| CN101938606B true CN101938606B (en) | 2012-12-26 |
Family
ID=43391713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910088167 Expired - Fee Related CN101938606B (en) | 2009-07-03 | 2009-07-03 | Method, system and device for propelling multimedia data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101938606B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102048618B1 (en) * | 2012-06-27 | 2019-11-25 | 구글 엘엘씨 | System and method for determining appropriate content for an event content stream |
| CN103856787B (en) * | 2012-12-04 | 2017-02-22 | 上海文广科技（集团）有限公司 | Commentary video passing-back live system based on public network and live method of commentary video passing-back live system based on public network |
| CN103200424B (en) * | 2013-03-26 | 2016-08-03 | 天脉聚源(北京)传媒科技有限公司 | Media program edit methods and system |
| CN103428555B (en) * | 2013-08-06 | 2018-08-10 | 乐视网信息技术（北京）股份有限公司 | A kind of synthetic method of multimedia file, system and application process |
| CN104506883A (en) * | 2014-12-11 | 2015-04-08 | 成都德芯数字科技有限公司 | Audio and video encoder based on wide area network live broadcast and working method thereof |
| CN106649415B (en) * | 2015-11-03 | 2019-12-20 | 广州市动景计算机科技有限公司 | Client data checking and updating method and device |
| CN107347065A (en) * | 2017-06-29 | 2017-11-14 | 上海鋆创信息技术有限公司 | A kind of data push method and device, a kind of terminal and storage medium |
| CN107995503B (en) * | 2017-11-07 | 2020-07-14 | 西安万像电子科技有限公司 | Audio and video playing method and device |
| JP7191571B2 (en) * | 2018-07-27 | 2022-12-19 | シャープ株式会社 | JOB MANAGEMENT DEVICE, JOB MANAGEMENT SYSTEM, PRINTING DEVICE, MFP, PRINTING SYSTEM AND PRINTING METHOD |
| CN110290290A (en) * | 2019-06-21 | 2019-09-27 | 深圳迪乐普数码科技有限公司 | Implementation method, device, computer equipment and the storage medium of the studio cloud VR |
| CN110933450B (en) * | 2019-10-16 | 2022-02-18 | 咪咕文化科技有限公司 | Multi-channel live broadcast synchronization method, system, edge device, terminal and storage medium |
| CN110958072B (en) * | 2019-11-04 | 2021-11-05 | 北京航星机器制造有限公司 | Multi-node audio and video information synchronous sharing display method |
| CN112804618B (en) * | 2021-01-06 | 2022-07-29 | 昆腾微电子股份有限公司 | Audio output method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101064669A (en) * | 2006-04-29 | 2007-10-31 | 互联天下科技发展（深圳）有限公司 | Network real-time playing multimedia data method |
| CN101166273A (en) * | 2006-10-16 | 2008-04-23 | 华为技术有限公司 | Method, device and system for transmitting video data |
| CN101197770A (en) * | 2007-10-09 | 2008-06-11 | 深圳市丕微科技企业有限公司 | Method for transmitting multimedia data by aid of network |
| CN101282356A (en) * | 2008-05-26 | 2008-10-08 | 张建中 | Method, apparatus and system for real-time data transmission |
-
2009
- 2009-07-03 CN CN 200910088167 patent/CN101938606B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101064669A (en) * | 2006-04-29 | 2007-10-31 | 互联天下科技发展（深圳）有限公司 | Network real-time playing multimedia data method |
| CN101166273A (en) * | 2006-10-16 | 2008-04-23 | 华为技术有限公司 | Method, device and system for transmitting video data |
| CN101197770A (en) * | 2007-10-09 | 2008-06-11 | 深圳市丕微科技企业有限公司 | Method for transmitting multimedia data by aid of network |
| CN101282356A (en) * | 2008-05-26 | 2008-10-08 | 张建中 | Method, apparatus and system for real-time data transmission |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101938606A (en) | 2011-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101938606B (en) | Method, system and device for propelling multimedia data | |
| CN109889543B (en) | Video transmission method, root node, child node, P2P server and system | |
| CN104053014B (en) | Live broadcast system and method based on mobile terminal and mobile terminal | |
| CN107846633A (en) | A kind of live broadcasting method and system | |
| CN101202613B (en) | Terminal for clock synchronization | |
| CN101009824A (en) | A network transfer method for audio/video data | |
| CN101184035B (en) | Broadcast transport stream distribution system, and broadcast transport stream distribution apparatus, user terminal device and distribution method for use in the system | |
| CN101394585A (en) | Quasi-real time stream system in mobile stream media live broadcast service | |
| CN101998116A (en) | Method, system and equipment for realizing multi-view video service | |
| CN102752669A (en) | Transfer processing method and system for multi-channel real-time streaming media file and receiving device | |
| CN104079870A (en) | Video monitoring method and system for single-channel video and multiple-channel audio frequency | |
| WO2011113315A1 (en) | Stream media live service system and implementation method thereof | |
| CN101465996A (en) | Method, equipment and system for displaying network television time | |
| US7587507B2 (en) | Media recording functions in a streaming media server | |
| CN102404646A (en) | Wireless television live broadcasting system and live broadcasting method thereof | |
| CN110062268A (en) | A kind of audio-video sends and receives processing method and processing device with what screen played | |
| CN100473171C (en) | Clock synchronizing method in broadcast network | |
| CN1960435B (en) | Method and system for synchronizing broadcast time of mobile multimedia | |
| CN101662655B (en) | Audio/video on-demand system | |
| CN100544448C (en) | A kind of clock system of mobile multimedia network | |
| CN103139188B (en) | Flow-medium transmission method and system | |
| TW200814743A (en) | Method and apparatus for distributing statistical mutliplex signals to handheld devices | |
| Gualdi et al. | Low-latency live video streaming over low-capacity networks | |
| CN102325158B (en) | Dynamic multi-thread broadcasting transmitting and analysis method | |
| CN100512121C (en) | Method for realizing file downloading in broadcasting network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220621 Address after: 100871 No. 5, the Summer Palace Road, Beijing, Haidian District Patentee after: Peking University Patentee after: New founder holdings development Co.,Ltd. Patentee after: BEIJING FOUNDER ELECTRONICS Co.,Ltd. Address before: 100871 No. 5, the Summer Palace Road, Beijing, Haidian District Patentee before: Peking University Patentee before: PEKING UNIVERSITY FOUNDER GROUP Co.,Ltd. Patentee before: BEIJING FOUNDER ELECTRONICS Co.,Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20121226 |