CN101895718B - Video conference system multi-image broadcast method, and device and system thereof - Google Patents
Video conference system multi-image broadcast method, and device and system thereof Download PDFInfo
- Publication number
- CN101895718B CN101895718B CN 201010232744 CN201010232744A CN101895718B CN 101895718 B CN101895718 B CN 101895718B CN 201010232744 CN201010232744 CN 201010232744 CN 201010232744 A CN201010232744 A CN 201010232744A CN 101895718 B CN101895718 B CN 101895718B
- Authority
- CN
- China
- Prior art keywords
- mcu
- sprite
- meeting
- place
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a video conference system multi-image broadcast method, and a device and a system thereof, which are applied to a video conference system, including a video conference terminal capable of sending conference video code streams and receiving video conference multi-image video code streams, and an MCU (Multipoint Conference Unit) comprising a primary MCU and a secondary MCU. The method comprises the steps that: the MCUs each process the corresponding conference video code streams according to the conference video code streams assigned by the primary MCU in order to obtainmulti-image code streams containing conference sub-images corresponding thereto; and the primary MCU synthesizes the multi-image code streams obtained from the processing of the MCUs into a video conference multi-image video code stream and broadcasts the video conference multi-image video code stream. By adopting the method, the problem in the prior art that multi-image broadcast of video conference fails owing to the limited processing ability of single MCU can be solved.
Description
Technical field
The present invention relates to communication technical field, particularly relate to a kind of video conferencing system multi-image broadcast method and device and system thereof.
Background technology
Video conferencing system realizes virtual meeting by the network communications technology, integrate video, audio frequency, data communication, but the image in each meeting-place of real-time Transmission and sound, so that each meeting-place is from be in logic a meeting, but physically not in a zone, the user who disperses on the geography is gathered in a place, by various ways exchange of information such as image, sound, carries out collaborative work.Space length has been shortened in video conference to the full extent, has made things convenient between the cooperation member in real time, has exchanged intuitively, compares with traditional meeting, can save overspending and does not affect the quality of meeting.This conference model has obtained more and more general application.
MCU (Multi-Point Control Unit, multipoint control unit) is the nucleus equipment in the video conference, main be responsible for and terminal or other MCU between the processing of signaling and forwarding or the processing of code stream.
The multi-image broadcast technology is the common function in the video conferencing system, and general MCU supports this technology.Generally speaking, the MCU video flowing that each terminal sends over of decoding, select a plurality of meeting field picture carry out image scaling, be spliced into a picture, many pictures that then will splice are transmitted to all terminals (meeting-place) in the meeting, make video conference participants can see by a picture picture in a plurality of meeting-place, each meeting-place can be play by the corresponding sprite in this picture.
Usually, the decode resources of MCU is limited.In general, the sprite number is more, and the MCU resource that expends is just more, if need the sprite of processing more, then might cause realizing multi-image broadcast owing to the restriction of MCU decoding capability.
Summary of the invention
The invention provides a kind of video conferencing system multi-image broadcast method and device and system, in order to solve the problem that causes to realize the video conference multi-image broadcast in the prior art because of single MCU limited processing ability.
Video conference multi-image broadcast method provided by the invention, be applied to include the video conference terminal that can send meeting-place video code flow and many picture videos of receiver, video meeting code stream, and the video conferencing system of MCU, described MCU comprises main MCU and auxiliary MCU, the method comprises:
Each MCU is the corresponding meeting-place of the sprite video code flow of its distribution according to main MCU, and corresponding meeting-place video code flow is processed, and obtains including many picture videos code stream of corresponding sprite;
Many picture videos code stream that main MCU obtains each MCU processing synthesizes many picture videos of video conference code stream and broadcasts.
In the said method, each MCU is the corresponding meeting-place of the sprite video code flow of its distribution according to main MCU, corresponding meeting-place video code flow is processed, comprise: after described main MCU receives the meeting-place video code flow, according to the sprite allocative decision, auxiliary MCU distribution sprite for this main MCU and/or appointment, wherein, each MCU is assigned to delegation or multirow sprite, with when the auxiliary MCU for appointment distributes sprite, the meeting-place video code flow that the sprite of corresponding line is corresponding is transmitted to corresponding auxiliary MCU; The MCU that is assigned to video code flow processes the meeting-place video code flow that distributes.
Said method, also comprise: when the many pictures of video conference switch, main MCU is according to the arrangement mode of each meeting-place sprite in the many pictures of video conference after switching, be updated to the sprite that corresponding MCU distributes, and transmit the meeting-place video code flow that receives according to the sprite allocative decision after upgrading.
In the said method, being fixed at multichannel meeting-place video code flow is assigned in the situation that each MCU receives, and also comprises:
Main MCU is that the auxiliary MCU of this main MCU and appointment distributes sprite, and the sprite allocative decision is notified to auxiliary MCU; Wherein, each MCU is assigned to delegation or multirow sprite, and the sprite that different MCU distribute is different from other MCU;
Described each MCU is the meeting-place video code flow of its distribution according to main MCU, and corresponding meeting-place video code flow is processed, and comprising:
Main MCU is or/and after auxiliary MCU receives the meeting-place video code flow, according to described sprite allocative decision, the sprite that is assigned at the meeting-place video code flow of determining to receive and this MCU is transmitted to the MCU that has distributed corresponding sprite not at once with the meeting-place video code flow that receives;
Receive the MCU of the meeting-place video code flow corresponding with the sprite that distributes, corresponding meeting-place video code flow is processed.
Said method also comprises: when the many pictures of video conference switched, main MCU upgrade the sprite allocative decision, and the sprite allocative decision after will upgrading was notified to auxiliary MCU according to the arrangement mode of each meeting-place sprite in the many pictures of video conference after switching.
In the said method, described auxiliary MCU is a plurality of.
MCU provided by the invention, be applied to include the video conference terminal that can send meeting-place video code flow and many picture videos of receiver, video meeting code stream, and the video conferencing system of MCU, described MCU can be used as main MCU or the auxiliary MCU in the described system, when described MCU comprises during as main MCU:
Allocation units are used to each MCU to distribute sprite;
Retransmission unit is connected with described allocation units, is used for the sprite allocative decision according to described allocation units, and the meeting-place video code flow that receives is forwarded to corresponding auxiliary MCU;
Receiving element is used for receiving the many picture videos code stream that includes corresponding meeting-place sprite that obtains after auxiliary MCU processes according to the corresponding meeting-place of the sprite video code flow that is assigned to;
Synthesis unit is connected with described receiving element, is used for that each MCU is processed the many picture videos code stream that obtains and synthesizes many picture videos of video conference code stream;
Radio unit is connected with described synthesis unit, and many picture videos of the video conference code stream after being used for synthesizing is broadcasted.
Among the above-mentioned MCU, described allocation units specifically are used for, and be the auxiliary MCU distribution sprite of this main MCU and/or appointment, and each MCU are assigned to delegation or multirow sprite;
Described retransmission unit specifically is used for, and when the auxiliary MCU for appointment distributes sprite, according to the sprite allocative decision of allocation units, the corresponding meeting-place of the sprite of corresponding line video code flow is transmitted to corresponding auxiliary MCU.
Among the above-mentioned MCU, described allocation units also are used for, and when the many pictures of video conference switched, the arrangement mode according to each meeting-place sprite in the many pictures of video conference after switching was updated to the sprite that corresponding MCU distributes.
Among the above-mentioned MCU, described allocation units specifically are used for, and being fixed at multichannel meeting-place video code flow is assigned in the situation of each MCU reception, be that the auxiliary MCU of this main MCU and appointment distributes sprite, and the sprite allocative decision is notified to auxiliary MCU; Wherein, each MCU is assigned to delegation or multirow sprite, and the sprite that different MCU distribute is different from other MCU;
Described retransmission unit specifically is used for, after receiving the meeting-place video code flow, according to described sprite allocative decision, the sprite that is assigned at the meeting-place video code flow of determining to receive and this MCU is transmitted to the meeting-place video code flow that receives the auxiliary MCU that has distributed corresponding sprite not at once.
Among the above-mentioned MCU, described allocation units also are used for, when the many pictures of video conference switch, according to the arrangement mode of each meeting-place sprite in the many pictures of video conference after switching, upgrade the sprite allocative decision, and the sprite allocative decision after will upgrading is notified to auxiliary MCU.
Above-mentioned MCU also comprises:
Video processing unit, be used for the sprite according to described allocation units distribution, meeting-place video code flow to the corresponding sprite that receives is processed, and obtains comprising many picture videos code stream of corresponding sprite, and should send to described synthesis unit by many picture videos code stream.
Described MCU comprises during as auxiliary MCU:
Receiving element is used for receiving meeting-place video code flow corresponding to sprite that is assigned to this auxiliary MCU;
Video processing unit is connected with receiving element, is used for the meeting-place video code flow that receives is processed, and obtains comprising many picture videos code stream of corresponding sprite;
Transmitting element is connected with described video processing unit, is used for that described video processing unit is processed the many picture videos code stream that comprises corresponding sprite that obtains and sends to described main MCU.
Video conferencing system provided by the invention comprises the video conference terminal that can send meeting-place video code flow and many picture videos of receiver, video meeting code stream, and this video conferencing system also comprises:
Main MCU is used to each MCU to distribute sprite; And, each MCU is processed the many picture videos code stream that obtains synthesize many picture videos of video conference code stream and broadcast;
Auxiliary MCU, be connected with described main MCU, being used for according to main MCU is the corresponding meeting-place of the sprite video code flow of its distribution, and corresponding meeting-place video code flow is processed, obtain including many picture videos code stream of corresponding meeting-place sprite, and described many picture videos code stream is sent to main MCU.
In the above-mentioned video conferencing system, described main MCU specifically is used for, and is the auxiliary MCU distribution sprite of this main MCU and appointment, and wherein, each MCU is assigned to delegation or multirow sprite; And after receiving the meeting-place video code flow, according to the sprite allocative decision, the meeting-place video code flow that the sprite of corresponding line is corresponding is transmitted to corresponding auxiliary MCU.
In the above-mentioned video conferencing system, being fixed at multichannel meeting-place video code flow is assigned in the situation that each MCU receives, and described main MCU specifically is used for, and is that the auxiliary MCU of this main MCU and appointment distributes sprite, and the sprite allocative decision is notified to auxiliary MCU; Wherein, each MCU is assigned to delegation or multirow sprite, and the sprite that different MCU distribute is different from other MCU; And, after receiving the meeting-place video code flow, according to described sprite allocative decision, the sprite that is assigned at the meeting-place video code flow of determining to receive and this MCU is transmitted to the meeting-place video code flow that receives the auxiliary MCU that has distributed corresponding sprite not at once; Or/and
Described auxiliary MCU specifically is used for, after receiving the meeting-place video code flow, according to described sprite allocative decision, the sprite that is assigned at the meeting-place video code flow of determining to receive and this MCU is transmitted to the MCU that has distributed corresponding sprite not at once with the meeting-place video code flow that receives.
In the above-mentioned video conferencing system, described auxiliary MCU is a plurality of.
Useful technique effect of the present invention comprises:
Be that auxiliary MCU distributes the meeting-place video code flow by main MCU, by auxiliary MCU the meeting-place video code flow that is assigned to is processed, obtain including many picture videos code stream of corresponding meeting-place sprite, the many picture videos code stream that by main MCU each MCU processing is obtained at last synthesizes many picture videos of video conference code stream and broadcasts, carry out thereby a plurality of MCU are shared in the processing operation of many pictures, avoided the problem that can't realize the video conference multi-image broadcast that causes because of single MCU Video processing capabilities limits.
Description of drawings
The video conferencing system multi-image broadcast configuration diagram that Fig. 1 provides for one embodiment of the invention;
The video conferencing system multi-image broadcast schematic flow sheet that Fig. 2 provides for one embodiment of the invention;
The related system architecture schematic diagram of video conferencing system multi-image broadcast flow process that Fig. 3 provides for another embodiment of the present invention;
The video conferencing system multi-image broadcast configuration diagram that Fig. 4 provides for another embodiment of the present invention;
The structural representation of the main MCU that Fig. 5 provides for one embodiment of the invention;
The structural representation of the auxiliary MCU that Fig. 6 provides for one embodiment of the invention.
Embodiment
In order to solve the problems referred to above that cause to realize the video conference multi-image broadcast in the prior art because of single MCU coding and decoding video capabilities limits, in the embodiment of the invention, adopt a plurality of MCU stacking, thereby realize multi-image broadcast.Wherein, each MCU can carry out to the video code flow of parton picture encoding and decoding to be processed, the video code flow of each sprite after then each MCU being processed is pieced together frame and is processed, thereby obtain comprising many pictures of each sprite, carrying out multi-image broadcast with the single MCU of use in the prior art compares, the embodiment of the invention for the many pictures of the video conference with multirow sprite, has solved single MCU coding/decoding capability bottleneck problem by sharing the video code flow encoding and decoding by a plurality of MCU.
In the embodiment of the invention, a plurality of MCU are stacked, are jointly shared the encoding and decoding of each meeting-place picture by these MCU that are stacked and process.Among these MCU that are stacked, effect and function according in each comfortable multi-image broadcast handling process can be divided into main MCU and auxiliary MCU.Wherein, main MCU is except the encoding and decoding of burden parton picture are processed, and the video code flow after also each MCU (comprising main MCU and auxiliary MCU) can being processed is pieced together the frame processing, thereby obtains the many pictures of video conference.
Below in conjunction with accompanying drawing the embodiment of the invention is described in detail.
Referring to Fig. 1, the video conferencing system multi-image broadcast configuration diagram that provides for one embodiment of the invention.
As shown in the figure, MCU 1 and MCU 2 are stacked, and wherein, MCU 1 is main MCU, and MCU 2 is auxiliary MCU.Main MCU can be according to the arrangement mode of sprite in many pictures in each meeting-place, the video code flow of selection portion sub-venue is transmitted to auxiliary MCU and carries out the encoding and decoding processing, and can receive the video code flow that includes corresponding meeting-place sprite after auxiliary MCU encoding and decoding are processed, again according to the arrangement mode of sprite in many pictures in each meeting-place, the many picture videos code stream after many picture videos code stream of oneself processing and auxiliary MCU processed is spliced into many picture videos frame and plays.
Main MCU can be by transforming realization to common MCU, as passing through programming mode, so that this MCU can realize the function of above-mentioned main MCU.
The embodiment of the invention can be connected to each MCU in the consolidated network, thereby realization MCU's is stacking.
The below illustrates the flow process of multi-image broadcast take scene shown in Figure 1 as example.As shown in Figure 1, current need to (picture of T1～T4) be placed on more than one and broadcasts in the picture, and the sprite arrangement mode in each meeting-place is shown in the many pictures 12 among Fig. 1 in many pictures with 4 meeting-place.
According to above scene, the flow process that the MCU that is stacked carries out multi-image broadcast can as shown in Figure 2, mainly may further comprise the steps:
Owing to be responsible for FPGA (the Filed Programmable GateArray that coding and decoding video is processed among the MCU, programmable gate array) normally by row frame of video is spliced, and, DSP (the digital signal processor that is used for coding and decoding video among the MCU, digital signal processor) usually carries out encoding and decoding to go the macro block mode, therefore, MCU 1 is according to the position of the sprite in each meeting-place in many pictures, selects to be in to be transmitted to MCU 2 with meeting-place video code flow corresponding to the sprite of delegation and to process.Concrete scene for present embodiment, because the sprite of meeting-place T1, T4 is in same delegation in many pictures, the sprite of meeting-place T3, T2 is in same delegation in many pictures, therefore, MCU 1 can be transmitted to the video code flow of meeting-place T3, T2 MCU 2, also the video code flow of meeting-place T1, T4 can be transmitted to MCU 2.In this flow process, MCU 1 can be transmitted to the video code flow of meeting-place T3, T2 MCU 2.
When MCU 1 is transmitted to MCU 2 at the video code flow with meeting-place T3, T2, can also be with target picture size (being the size of many pictures 11) notice to MCU 2, so that MCU 2 is big or small according to this target picture, and generate many pictures 11 of corresponding size according to the video code flow of meeting-place T3, T2.
In this step, after MCU 2 receives the video code flow of meeting-place T3, T2, can carry out conventional encoding and decoding and process, as, video code flow is decoded, meeting-place name, banner, captions, frame etc. then superpose, the many pictures 11 of splicing place, encoding obtains video code flow 2 again.In like manner, MCU 1 also can be to the processing of decoding of the video code flow of meeting-place T1, T4, the first half of splicing many pictures 12, and encoding obtains video code flow 1 again.Wherein, MCU 2 can carry out according to the size of the many pictures 12 that receive from MCU 1 the coding processing of many pictures 12.
Because the DSP encoding and decoding are processed take macro block as unit usually, so the picture altitude of the handled sprite of each MCU also should be the integral multiple of macro block height.That is to say, when MCU (can comprise main MCU, also can comprise auxiliary MCU) carries out many pictures processing, as the case may be, might need the size of sprite is finely tuned, so that the sprite size is the integral multiple of macro block height.
The video code flow (video code flow 2) of the many pictures 11 that include meeting-place T3, T2 sprite after step 204, MCU 2 will process sends to MCU 1.
In this step, MCU1 can adopt RTP (Real-time Transport Protocol, RTCP Real-time Transport Control Protocol) to the processing of packing of video code flow 1 and video code flow 2.Because the image of video code flow 1 and video code flow 2 does not have correlation, send to all meeting-place so can be directly the present frame of the present frame of video code flow 1 and video code flow 2 be synthesized frame data, thus the realization multi-image broadcast.When carrying out multi-picture splicing according to each video code flow, MCU 1 can be placed on the latter half of many pictures 12 with many pictures 11 according to the arrangement mode of each sprite in many pictures, thereby forms complete many pictures 12.
When the meeting-place that needs broadcasting pictures more, when namely the sprite in many pictures is more, in order further to improve the multi-image broadcast performance, can stacking more MCU, in order to shared the processing of sprite by more MCU.
Fig. 3 shows a kind of schematic diagram that carries out multi-image broadcast by stacking 3 MCU.
As shown in Figure 3, MCU 1, MCU 2 and MCU 3 are stacked, and wherein, MCU 1 is main MCU, and MCU 2 and MCU 3 are auxiliary MCU.
Current need to (picture of T1～T9) be placed on more than one and broadcasts in the picture, and the sprite arrangement mode in each meeting-place is shown in the many pictures 33 among Fig. 3 in many pictures with 9 meeting-place.
According to above scene, the flow process that the MCU that is stacked carries out multi-image broadcast is basic identical with flow process shown in Figure 2, wherein:
Among the above embodiment, when many pictures switch, switch or/and many pictures neutron number of pictures changes such as the meeting-place picture, main MCU is as long as adjust the code stream that sends to auxiliary MCU.For example, for scene shown in Figure 1, if need to be the picture of meeting-place T5 with the Picture switch of meeting-place T3, then MCU 1 be transmitted to MCU 2 with the video code flow of meeting-place T5, T2, and follow-up processing flow can realize with reference to mode shown in Figure 2, do not repeat them here.
Usually, in video conference application, can there be the multi-channel video code stream, if all receive and transmit by main MCU, can increases the weight of the load of main MCU.For addressing this problem, in video conference application, the multi-channel video code stream can be shared on the different MCU and being received, such as, for framework shown in Figure 3, can be assigned among 3 MCU 9 road video code flows (being the video code flow of meeting-place T1～T9) are average and relatively-stationary, as, the video code flow of T1～T3 is assigned to MCU 1, the video code flow of T4～T6 is assigned to MCU 3, the video code flow of T7～T9 is assigned to MCU 2.
For this situation, in another embodiment of the present invention, main MCU can notify the sprite allocative decision to auxiliary MCU in advance, as being notified respectively to each auxiliary MCU by the corresponding meeting-place of the sprite that each MCU processes sign (or the video conference terminal that sends video code flow identifies).In the video conference process, each MCU (comprising main MCU and auxiliary MCU) receives the video code flow that the allocative decision reception is assigned to separately according to video code flow in advance, if finding the video code flow that self receives is exactly that oneself needs the corresponding video code flow of sprite to be processed, then carries out encoding and decoding and process; Otherwise, this video code flow is forwarded to the MCU (may be that main MCU also may be auxiliary MCU) that processes this video code flow.Auxiliary MCU finishes dealing with after the many pictures that include the parton picture of oneself being responsible for, corresponding video code flow is sent to main MCU, finally synthesize the many pictures that include all sprites by main MCU, its specific implementation process can with reference to aforementioned description, not repeat them here.
For example, be similar to scene shown in Figure 3, as shown in Figure 4, video code flow receives allocative decision and is: the video code flow of T1～T3 is assigned to MCU 1, the video code flow of T4～T6 is assigned to MCU3, the video code flow of T7～T9 is assigned to MCU 2; The scheme that sprite distributes is: MCU 1 processes many pictures that T4～T6 consists of, and finally synthesizes many pictures 43; MCU 2 processes many pictures 42 that T7～T9 consists of; MCU 3 processes many pictures 41 that T1～T3 consists of.
When many Picture switchs occur when, switch or/and many pictures neutron number of pictures changes such as the meeting-place picture, main MCU needs only the arrangement mode according to each sprite in the many pictures after switching, adjustment sprite allocative decision and notice get final product to auxiliary MCU, make each MCU share the processing of the sprite of different rows in many pictures.
In the embodiment of the invention, although a plurality of MCU are stacked, but in consolidated network, propagation delay time between these MCU is usually very little, through checking, substantially can be in 30ms, this time delay is with respect to the time-delay of encoding and decoding, substantially can ignore, bring too many impact namely can for the multi-image broadcast performance.
Need to prove that above-described embodiment only is assigned to delegation's sprite take each MCU and is described as example, in fact, main MCU also can distribute to the multirow sprite own or auxiliary MCU processes.In addition, when auxiliary MCU was a plurality of, the auxiliary MCU that main MCU can share many pictures appointment processed, and the MUC of this appointment can be whole auxiliary MCU, also can be the auxiliary MCU of part.
Can find out that by above description main MCU adopts auxiliary MCU to share many pictures, process loads thereby can share many pictures, can avoid causing to realize because of single MCU coding and decoding video bottleneck the problem of video conference multi-image broadcast.In addition, main MCU can directly piece together frame with the code stream of each MCU, does not do secondary coding-decoding operation, and is simple, and do not need to consume extra load.
Based on identical technical conceive, the embodiment of the invention also provides a kind of MCU, can be applicable to aforementioned each flow process.This MCU can be applicable to include the video conference terminal that can send meeting-place video code flow and many picture videos of receiver, video meeting code stream, and the video conferencing system of MCU, and described MCU can be used as main MCU or the auxiliary MCU in this system.
Fig. 5 shows the structure of a kind of main MCU, and this main MCU can comprise: allocation units 51, retransmission unit 52, receiving element 53, synthesis unit 54, radio unit 55, wherein:
Allocation units 51 are used to each MCU to distribute sprite;
Retransmission unit 52 is connected with allocation units 51, is used for the sprite allocative decision according to allocation units 51, and the meeting-place video code flow that receives is forwarded to corresponding auxiliary MCU;
Receiving element 53 is used for receiving the many picture videos code stream that includes corresponding meeting-place sprite that obtains after auxiliary MCU processes according to the corresponding meeting-place of the sprite video code flow that is assigned to;
Synthesis unit 54 is connected with receiving element 53, is used for that each MCU is processed the many picture videos code stream that obtains and synthesizes many picture videos of video conference code stream; Concrete, this unit can synthesize processing according to the sprite allocative decision that allocation units 51 are formulated;
Radio unit 55 is connected with synthesis unit 54, and many picture videos of the video conference code stream after being used for synthesizing is broadcasted.
Allocation units 51 among the above-mentioned main MCU and retransmission unit 52 can have two kinds of implementations:
Mode one: allocation units 51 can when distributing sprite for each MCU, be the auxiliary MCU distribution sprite of this main MCU and/or appointment, and each MCU be assigned to delegation or multirow sprite; Retransmission unit 52 can be after receiving the video code flow that the meeting-place sends, and when distributing sprite for the auxiliary MCU of appointment, according to the sprite allocative decision of allocation units, the corresponding meeting-place of the sprite of corresponding line video code flow is transmitted to corresponding auxiliary MCU.Further, allocation units 51 can be when the many pictures of video conference switch, and the arrangement mode according to each meeting-place sprite in the many pictures of video conference after switching is updated to the sprite that corresponding MCU distributes.
Mode two: allocation units 51 can be fixed at multichannel meeting-place video code flow and be assigned in the situation of each MCU reception, when distributing sprite for each MCU, be that the auxiliary MCU of this main MCU and appointment distributes sprite, and the sprite allocative decision is notified to auxiliary MCU; Wherein, each MCU is assigned to delegation or multirow sprite, and the sprite that different MCU distribute is different from other MCU; Retransmission unit 52 can be after receiving the meeting-place video code flow, according to described sprite allocative decision, the sprite that is assigned at the meeting-place video code flow of determining to receive and this MCU is transmitted to the meeting-place video code flow that receives the auxiliary MCU that has distributed corresponding sprite not at once.Further, allocation units 51 also can when the many pictures of video conference switch, according to the arrangement mode of each meeting-place sprite in the many pictures of video conference after switching, upgrade the sprite allocative decision, and the sprite allocative decision after will upgrading be notified to auxiliary MCU.
Above-mentioned main MCU, also can comprise video processing unit 56, be used for the sprite according to allocation units 51 distribution, meeting-place video code flow to the corresponding sprite that receives is processed, obtain comprising many picture videos code stream of corresponding sprite, and should send to synthesis unit 54 by many picture videos code stream.
Fig. 6 shows the structure of a kind of auxiliary MCU, and this auxiliary MCU can comprise: receiving element 61, video processing unit 62 and transmitting element 63, wherein:
Receiving element 61 is used for receiving meeting-place video code flow corresponding to sprite that is assigned to this auxiliary MCU; This meeting-place video code flow can be to transmit the meeting-place video code flow of coming by other MCU (comprising main MCU or/and other auxiliary MCU), also can be the meeting-place video code flow that this auxiliary MCU directly receives from video conference terminal;
Transmitting element 63 is connected with video processing unit 62, is used for that video processing unit 62 is processed the many picture videos code stream that comprises corresponding sprite that obtains and sends to main MCU.
The specific implementation of each functional unit among the above-mentioned MCU is described with the corresponding steps with reference to aforementioned flow process, does not repeat them here.
Through the above description of the embodiments, those skilled in the art can be well understood to the present invention and can realize by the mode that software adds essential general hardware platform, can certainly pass through hardware, but the former is better execution mode in a lot of situation.Based on such understanding, the part that technical scheme of the present invention contributes to prior art in essence in other words can embody with the form of software product, this computer software product is stored in the storage medium, comprise that some instructions are with so that a station terminal equipment (can be mobile phone, personal computer, server, the perhaps network equipment etc.) carry out the described method of each embodiment of the present invention.
The above only is preferred implementation of the present invention; should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; can also make some improvements and modifications, these improvements and modifications also should be looked protection scope of the present invention.
Claims (14)
1. video conference multi-image broadcast method, be applied to include the video conference terminal that can send meeting-place video code flow and many picture videos of receiver, video meeting code stream, and the video conferencing system of multipoint control unit MCU, it is characterized in that, described MCU comprises main MCU and auxiliary MCU, and the method comprises:
Each MCU is the corresponding meeting-place of the sprite video code flow of its distribution according to main MCU, and corresponding meeting-place video code flow is processed, and obtains including many picture videos code stream of corresponding sprite;
Many picture videos code stream that main MCU obtains each MCU processing synthesizes many picture videos of video conference code stream and broadcasts;
Wherein, each MCU is the corresponding meeting-place of the sprite video code flow of its distribution according to main MCU, and corresponding meeting-place video code flow is processed, and comprising:
After described main MCU receives the meeting-place video code flow, according to the sprite allocative decision, auxiliary MCU distribution sprite for this main MCU and/or appointment, wherein, each MCU is assigned to delegation or multirow sprite, with when the auxiliary MCU for appointment distributes sprite, the meeting-place video code flow that the sprite of corresponding line is corresponding is transmitted to corresponding auxiliary MCU;
The MCU that is assigned to video code flow processes the meeting-place video code flow that distributes.
2. the method for claim 1 is characterized in that, also comprises:
When the many pictures of video conference switch, main MCU is according to the arrangement mode of each meeting-place sprite in the many pictures of video conference after switching, be updated to the sprite that corresponding MCU distributes, and transmit the meeting-place video code flow that receives according to the sprite allocative decision after upgrading.
3. the method for claim 1, it is characterized in that, being fixed at multichannel meeting-place video code flow is assigned in the situation that each MCU receives, and also comprise: main MCU distributes sprite for the auxiliary MCU of this main MCU and appointment, and the sprite allocative decision is notified to auxiliary MCU; Wherein, each MCU is assigned to delegation or multirow sprite, and the sprite that different MCU distribute is different from other MCU;
Described each MCU is the meeting-place video code flow of its distribution according to main MCU, and corresponding meeting-place video code flow is processed, and comprising:
Main MCU is or/and after auxiliary MCU receives the meeting-place video code flow, according to described sprite allocative decision, the sprite that is assigned at the meeting-place video code flow of determining to receive and this MCU is transmitted to the MCU that has distributed corresponding sprite not at once with the meeting-place video code flow that receives;
Receive the MCU of the meeting-place video code flow corresponding with the sprite that distributes, corresponding meeting-place video code flow is processed.
4. method as claimed in claim 3 is characterized in that, also comprises:
When the many pictures of video conference switched, main MCU upgrade the sprite allocative decision, and the sprite allocative decision after will upgrading was notified to auxiliary MCU according to the arrangement mode of each meeting-place sprite in the many pictures of video conference after switching.
5. such as each described method of claim 1 to 4, it is characterized in that described auxiliary MCU is a plurality of.
6. MCU, be applied to include the video conference terminal that can send meeting-place video code flow and many picture videos of receiver, video meeting code stream, and the video conferencing system of MCU, described MCU can be used as main MCU or the auxiliary MCU in the described system, it is characterized in that, when described MCU comprises during as main MCU:
Allocation units are used to each MCU to distribute sprite;
Retransmission unit is connected with described allocation units, is used for the sprite allocative decision according to described allocation units, and the meeting-place video code flow that receives is forwarded to corresponding auxiliary MCU;
Receiving element is used for receiving the many picture videos code stream that includes corresponding meeting-place sprite that obtains after auxiliary MCU processes according to the corresponding meeting-place of the sprite video code flow that is assigned to;
Synthesis unit is connected with described receiving element, is used for that each MCU is processed the many picture videos code stream that obtains and synthesizes many picture videos of video conference code stream;
Radio unit is connected with described synthesis unit, and many picture videos of the video conference code stream after being used for synthesizing is broadcasted;
Wherein, described allocation units specifically are used for, and be the auxiliary MCU distribution sprite of this main MCU and/or appointment, and each MCU are assigned to delegation or multirow sprite;
Described retransmission unit specifically is used for, and when the auxiliary MCU for appointment distributes sprite, according to the sprite allocative decision of allocation units, the corresponding meeting-place of the sprite of corresponding line video code flow is transmitted to corresponding auxiliary MCU.
7. MCU as claimed in claim 6 is characterized in that, described allocation units also are used for, and when the many pictures of video conference switched, the arrangement mode according to each meeting-place sprite in the many pictures of video conference after switching was updated to the sprite that corresponding MCU distributes.
8. MCU as claimed in claim 6, it is characterized in that described allocation units specifically are used for, being fixed at multichannel meeting-place video code flow is assigned in the situation of each MCU reception, be the auxiliary MCU distribution sprite of this main MCU and appointment, and the sprite allocative decision is notified to auxiliary MCU; Wherein, each MCU is assigned to delegation or multirow sprite, and the sprite that different MCU distribute is different from other MCU;
Described retransmission unit specifically is used for, after receiving the meeting-place video code flow, according to described sprite allocative decision, the sprite that is assigned at the meeting-place video code flow of determining to receive and this MCU is transmitted to the meeting-place video code flow that receives the auxiliary MCU that has distributed corresponding sprite not at once.
9. MCU as claimed in claim 8, it is characterized in that, described allocation units also are used for, when the many pictures of video conference switch, arrangement mode according to each meeting-place sprite in the many pictures of video conference after switching, upgrade the sprite allocative decision, and the sprite allocative decision after will upgrading is notified to auxiliary MCU.
10. such as each described MCU of claim 6-9, it is characterized in that, also comprise:
Video processing unit, be used for the sprite according to described allocation units distribution, meeting-place video code flow to the corresponding sprite that receives is processed, and obtains comprising many picture videos code stream of corresponding sprite, and should send to described synthesis unit by many picture videos code stream.
11. MCU as claimed in claim 6 is characterized in that, described MCU comprises during as auxiliary MCU:
Receiving element is used for receiving meeting-place video code flow corresponding to sprite that is assigned to this auxiliary MCU;
Video processing unit is connected with receiving element, is used for the meeting-place video code flow that receives is processed, and obtains comprising many picture videos code stream of corresponding sprite;
Transmitting element is connected with described video processing unit, is used for that described video processing unit is processed the many picture videos code stream that comprises corresponding sprite that obtains and sends to described main MCU.
12. a video conferencing system comprises the video conference terminal that can send meeting-place video code flow and many picture videos of receiver, video meeting code stream, it is characterized in that, also comprises:
Main MCU is used to each MCU to distribute sprite; And, each MCU is processed the many picture videos code stream that obtains synthesize many picture videos of video conference code stream and broadcast, specifically be used for, be that the auxiliary MCU of this main MCU and appointment distributes sprite, wherein, each MCU is assigned to delegation or multirow sprite; And after receiving the meeting-place video code flow, according to the sprite allocative decision, the meeting-place video code flow that the sprite of corresponding line is corresponding is transmitted to corresponding auxiliary MCU
Auxiliary MCU, be connected with described main MCU, being used for according to main MCU is the corresponding meeting-place of the sprite video code flow of its distribution, and corresponding meeting-place video code flow is processed, obtain including many picture videos code stream of corresponding meeting-place sprite, and described many picture videos code stream is sent to main MCU.
13. video conferencing system as claimed in claim 12, it is characterized in that being fixed at multichannel meeting-place video code flow is assigned in the situation of each MCU reception, described main MCU specifically is used for, be the auxiliary MCU distribution sprite of this main MCU and appointment, and the sprite allocative decision is notified to auxiliary MCU; Wherein, each MCU is assigned to delegation or multirow sprite, and the sprite that different MCU distribute is different from other MCU; And, after receiving the meeting-place video code flow, according to described sprite allocative decision, the sprite that is assigned at the meeting-place video code flow of determining to receive and this MCU is transmitted to the meeting-place video code flow that receives the auxiliary MCU that has distributed corresponding sprite not at once; Or/and
Described auxiliary MCU specifically is used for, after receiving the meeting-place video code flow, according to described sprite allocative decision, the sprite that is assigned at the meeting-place video code flow of determining to receive and this MCU is transmitted to the MCU that has distributed corresponding sprite not at once with the meeting-place video code flow that receives.
14. such as claim 12 or 13 each described video conferencing systems, it is characterized in that described auxiliary MCU is a plurality of.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010232744 CN101895718B (en) | 2010-07-21 | 2010-07-21 | Video conference system multi-image broadcast method, and device and system thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010232744 CN101895718B (en) | 2010-07-21 | 2010-07-21 | Video conference system multi-image broadcast method, and device and system thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101895718A CN101895718A (en) | 2010-11-24 |
| CN101895718B true CN101895718B (en) | 2013-10-23 |
Family
ID=43104764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010232744 Active CN101895718B (en) | 2010-07-21 | 2010-07-21 | Video conference system multi-image broadcast method, and device and system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101895718B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102215375B (en) * | 2011-06-24 | 2018-04-27 | 中兴通讯股份有限公司 | The system of selection of the video source of the sprite of more pictures and device in multimedia conferencing |
| CN103686219B (en) * | 2012-09-24 | 2017-09-29 | 华为技术有限公司 | A kind of method, equipment and the system of video conference recorded broadcast |
| CN105704424A (en) * | 2014-11-27 | 2016-06-22 | 中兴通讯股份有限公司 | Multi-image processing method, multi-point control unit, and video system |
| CN104519307B (en) * | 2014-12-12 | 2018-01-12 | 华为软件技术有限公司 | Meeting management system, continuous presence equipment, branch's field device, video meeting implementing method and system |
| US10390076B2 (en) * | 2016-03-29 | 2019-08-20 | Mitsubishi Electric Corporation | Image receiving/reproducing device, image generating/transmitting device, display system, image receiving/reproducing method, image generating/transmitting method, and computer readable medium |
| CN109698927A (en) * | 2017-10-23 | 2019-04-30 | 中兴通讯股份有限公司 | Conference management method, device and storage medium |
| CN110636242B (en) * | 2018-06-22 | 2023-02-17 | 中兴通讯股份有限公司 | Picture switching method in video conference, terminal and MCU |
| CN110489167B (en) * | 2019-07-26 | 2020-07-03 | 广东高云半导体科技股份有限公司 | Double-kernel code stream downloading method and device, computer equipment and storage medium |
| CN110971862B (en) * | 2019-11-04 | 2021-04-23 | 厦门亿联网络技术股份有限公司 | Video conference broadcasting method and device |
| CN112351329A (en) * | 2020-10-14 | 2021-02-09 | 视联动力信息技术股份有限公司 | Subtitle display method and device, terminal equipment and storage medium |
| CN112839197B (en) * | 2020-12-31 | 2023-08-25 | 北京融讯科创技术有限公司 | Image code stream processing method, device, system and storage medium |
| CN112887635A (en) * | 2021-01-11 | 2021-06-01 | 深圳市捷视飞通科技股份有限公司 | Multi-picture splicing method and device, computer equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101388997A (en) * | 2008-10-22 | 2009-03-18 | 杭州华三通信技术有限公司 | Method and system for establishing video conference |
| WO2009156867A2 (en) * | 2008-06-23 | 2009-12-30 | Radvision Ltd. | Systems,methods, and media for providing cascaded multi-point video conferencing units |
-
2010
- 2010-07-21 CN CN 201010232744 patent/CN101895718B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009156867A2 (en) * | 2008-06-23 | 2009-12-30 | Radvision Ltd. | Systems,methods, and media for providing cascaded multi-point video conferencing units |
| CN101388997A (en) * | 2008-10-22 | 2009-03-18 | 杭州华三通信技术有限公司 | Method and system for establishing video conference |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101895718A (en) | 2010-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101895718B (en) | Video conference system multi-image broadcast method, and device and system thereof | |
| CN101316351B (en) | Multi-point control cell cascade connection system and communication method and device thereof | |
| US9462228B2 (en) | Distributed real-time media composer | |
| CN100454820C (en) | MCU cascade system and establishing and communication method for the same | |
| CN100583985C (en) | Method, apparatus and system for switching pictures in video service | |
| CN101370114B (en) | Video and audio processing method, multi-point control unit and video conference system | |
| EP2214410B1 (en) | Method and system for conducting continuous presence conferences | |
| US20130215215A1 (en) | Cloud-based interoperability platform using a software-defined networking architecture | |
| KR101555855B1 (en) | Method and system for conducting video conferences of diverse participating devices | |
| US20120069133A1 (en) | Method and system for video conference control, videoconferencing network equipment, and videoconferencing site | |
| KR20080109780A (en) | Generating and selecting media streams | |
| WO2011140812A1 (en) | Multi-picture synthesis method and system, and media processing device | |
| CN102342066A (en) | REAL-TIME MULTI-MEDIA STREAMING processing BANDWIDTH MANAGEMENT | |
| CN103327287B (en) | A kind of conference signal playing method and device, video conference terminal, mobile device | |
| US8836753B2 (en) | Method, apparatus, and system for processing cascade conference sites in cascade conference | |
| CN102017516A (en) | Systems and methods for media distribution | |
| CN102387338B (en) | Distributed type video processing method and video session system | |
| US9374232B2 (en) | Method and a device for optimizing large scaled video conferences | |
| CN113542660A (en) | Method, system and storage medium for realizing conference multi-picture high-definition display | |
| CN105611219A (en) | Method and device for processing video conference | |
| CN101345844A (en) | Multimedia code stream transmission method and multi-point control cell | |
| CN108156413B (en) | Video conference transmission method and device and MCU | |
| CN100366080C (en) | Method of implementing switching of single picture and multi-pictures in conference television system | |
| WO2014150992A1 (en) | Cloud-based interoperability platform using a software-defined networking architecture | |
| US8976225B2 (en) | Method, computer program and device for managing media stream in video conferencing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 310052 Binjiang District Changhe Road, Zhejiang, China, No. 466, No. Patentee after: Xinhua three Technology Co., Ltd. Address before: 310053 Hangzhou hi tech Industrial Development Zone, Zhejiang province science and Technology Industrial Park, No. 310 and No. six road, HUAWEI, Hangzhou production base Patentee before: Huasan Communication Technology Co., Ltd. |