WO2011023103A1

WO2011023103A1 - Media control server cascaded system, and method and device for controlling multimedia code stream

Info

Publication number: WO2011023103A1
Application number: PCT/CN2010/076323
Authority: WO
Inventors: 林永琪; 陈庆丰; 徐凌峰; 陈显义
Original assignee: 华为终端有限公司
Priority date: 2009-08-31
Filing date: 2010-08-25
Publication date: 2011-03-03
Also published as: CN101998105A; CN101998105B

Abstract

A method for controlling a multimedia code stream in a media control server cascaded system is disclosed in the present invention. In the cascaded system, multiple cascaded channels are established among all media control servers. The method comprises: sending a plurality of multimedia code streams to other media control servers via the pre-established multiple cascaded channels. A media control server and media control server cascaded system are also disclosed in the present invention. In the embodiments of the present invention, multiple cascaded channels can be established for each of the media control servers. Therefore when audio code streams are being forwarded, the operations such as mixing, encoding and the like may not to be performed on the audio code streams, but some audio code streams can be selected from them and can be sent directly to the next equipment in the transmitting path. Thus, a transmission time delay, caused by performing the operations such as mixing, encoding and the like on the audio data by the equipments of each of the middle stages on the transmission path, is reduced. And the multiple video code streams are transmitted simultaneously through multiple cascaded channels, all kinds of the application implements are extended, and the requirements of users are satisfied.

Description

Media control server cascading system and method and device for controlling multimedia stream The entire content of the Chinese patent application of "Method and Apparatus" is incorporated in this application by reference. Technical field

The present invention relates to the field of communication technology, and more specifically, to a media control server system and a method and device for controlling multimedia stream. Background technique

With the continuous development of network technology and broadband construction, multimedia conference systems have been widely used in work and life. The multimedia conference system is a communication method that uses video conferencing technology and equipment to communicate between two or more locations through a transmission channel.

In video conferencing applications, multipoint control units can be cascaded with each other, and terminal conference sites under different multipoint control units (MCUs) can be scheduled in the same conference by holding a cascade conference. However, only one cascading channel is set between each MCU, that is, only one multimedia code stream can be transmitted between each MCU, which brings limitations to some application implementations. Summary of the invention

In view of this, the embodiments of the present invention provide a multipoint control unit cascade system and a method and device for controlling a multimedia stream to solve various application implementations caused by the fact that the multimedia stream can only be transmitted one way at the same time in the prior art. Unachievable problem.

The embodiment of the present invention is implemented as follows:

A multimedia code stream control method of a media control server cascade system, in the cascade system, Multiple cascading channels are established between each media control server, and the method includes: sending multiple multimedia code streams to other media control servers through the multiple pre-established cascading channels.

A media control server, the media control server is located in a media control server cascade system, and the media control server includes:

The cascade channel establishment unit is used to establish multiple cascade channels with other media control servers in the media control server cascade system.

A media control server cascade system includes multiple media control servers, and multiple cascade channels are established between each media control server when multiple video streams need to be transmitted, wherein:

Each media control server obtains multiple video code streams, determines the corresponding relationship between each video code stream and the cascade channel, and transmits the multiple video code streams through the corresponding multiple cascade channels.

A media control server cascade system includes a first media control server, a second media control server, and a third media control server, and multiple cascade channels are established between each media control server when multiple audio streams need to be transmitted, in:

The first media control server is configured to receive multiple local conference site audio code streams, select multiple audio code streams that meet predetermined conditions after decoding, and send them through multiple pre-established transmission channels; the second media control Server, configured to receive multiple channels of local conference site audio code streams and multiple channels of audio code streams sent by the first media control server, decode the multiple channels of local conference site audio code streams, and obtain information from the decoding results and the first Select the multiple audio code streams that meet the predetermined conditions from the multiple audio code streams sent by the media control server, and send them through multiple pre-established transmission channels;

The third media control server is configured to receive multiple channels of local venue audio code streams and multiple channels of audio code streams sent by the second media control server, decode the multiple channels of local venue audio code streams, and decode As a result, among the multiple audio code streams sent by the second media control server, a multiple audio code stream that meets a predetermined condition is selected, processed, and then provided to the local conference venue.

It can be seen from the above technical solutions that, compared with the prior art, in the embodiment of the present invention, each MCU establishes multiple cascaded channels when it needs to transmit multiple multimedia streams. As a result, each MCU is transcribing When the audio code stream is used, it is not necessary to perform operations such as mixing or encoding the audio code stream. Instead, select several audio code streams and send them directly to the next MCU (upper level MCU or lower level MCU) on the transmission path, thereby reducing The transmission delay caused by operations such as mixing and encoding of audio data by the middle levels of the MCU on the transmission path is avoided. In addition, each MCU can simultaneously transmit multiple video streams through multiple cascaded channels, expand various application implementations, and meet user needs.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

Fig. 1 is a schematic structural diagram of an MCU cascade system for audio code stream processing provided by the prior art;

Fig. 2 is a schematic structural diagram of an MCU cascaded system for video bitstream processing provided by the prior art;

Figure 3 is a schematic structural diagram of an MCU cascade system for point-to-point control or camera control provided by the prior art;

FIG. 4 is a schematic structural diagram of an MCU cascade system provided by an embodiment of the present invention; FIG. 5 is a schematic diagram of an application structure for audio code stream processing of the MCU cascade system shown in FIG. 4;

FIG. 6 is a schematic diagram of an application structure of the MCU cascade system shown in FIG. 4 for processing video code streams;

Fig. 7 is a schematic diagram of an application structure of the MCU cascade system shown in Fig. 4 for multi-channel point-to-point viewing or multi-channel point-to-point camera control;

FIG. 8 is a schematic diagram of cascading channels between MCUs in the above-mentioned MCU cascading system provided by an embodiment of the present invention;

FIG. 9 is a schematic diagram of a structure of an MCU provided by an embodiment of the present invention; FIG. 10 is a schematic structural diagram of another MCU provided by an embodiment of the present invention; FIG. 11 is a schematic structural diagram of another MCU provided by an embodiment of the present invention;

FIG. 12 is a schematic structural diagram of another MCU provided by an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of another MCU provided by an embodiment of the present invention;

FIG. 14 is a schematic structural diagram of another MCU provided by an embodiment of the present invention;

Fig. 15 is a flowchart of a method for controlling an audio code stream by the above-mentioned MCU cascade system according to an embodiment of the present invention;

Fig. 16 is a flowchart of a method for controlling a video code stream by the above-mentioned MCU cascade system according to an embodiment of the present invention. DETAILED DESCRIPTION The technical solution provided by the embodiment of the present invention relates to a media control server cascade system and its communication method and device.

For the sake of reference and clarity, the technical terms, shorthands or abbreviations used in this article are summarized as follows:

H.320, ITU-T Recommendation H.320, Narrow-band visual telephone systems and terminal equipment, a standard designated by the International Telecommunications Union Telecommunications Standards Department, this standard specifies multimedia communication systems based on narrowband switching systems;

H.323, ITU-T Recommendation H.323, Packet-based Multimedia Communications Systems, a standard designated by the Telecommunications Standards Department of the International Telecommunication Union, this standard specifies the multimedia communication system architecture based on the packet switching system;

ITU-T, International Telecommunication Union Telecommunication Standardization Sector, namely: International Telecommunication Union Telecommunication Standardization Department;

IP, Internet Protocol, namely: network protocol, protocol;

SIP, Session Initiation Protocol, that is, Session Initiation Protocol;

RM, Resource Manager, is a multipoint conference resource management system;

UDP, User Datagram Protocol, User Datagram Protocol; RDC, Remote Device Control, remote device control;

A meeting place is a place for information exchange. Generally, there is a conference terminal in a meeting place; the local meeting place is under the management of a media control server (only MCU (Multipoint Control Unit, multipoint control unit) is taken as an example below). A place where information exchange is realized between two parties directly or through a media control server;

The cascaded venues are generally managed by different media control servers, and the venues for information interaction through the respective media control servers.

In current video conferencing applications, multipoint control unit MCUs can be cascaded with each other. By calling a cascaded conference, terminal conference sites under different MCUs can be scheduled in the same conference. The cascading method is shown in Figure 1, where MCU2 is hung under MCU1, and MCU3 is hung under MCU respectively.

Among them, the venue of each level of MCU is independently mixed, and then the mixed code stream is sent to the upper level MCU, and the upper level MCU re-decodes and then continues to mix, and so on. As shown in Figure 1, after MCU3 decodes each venue code stream, it performs mixing processing to obtain the mixed code stream and sends it to MCU2, which is decoded, mixed and coded again by MCU2, and then sent to MCU1.

In the process of realizing the creation of the present invention, the inventor found that the above-mentioned prior art requires decoding, mixing and encoding of audio code streams on each level of MCU, which brings delay to the transmission of audio code streams. In addition, multiple encoding and decoding will also cause a certain loss to the audio code stream, and the delay and loss directly affect the sound quality. In addition, when the audio code stream is transmitted between MCUs with large levels of difference, the longer the transmission path and the greater the number of encoding and decoding, the more serious the impact on the sound quality.

In addition, in terms of cascaded multi-screen and TV wall applications, in the existing MCU cascaded system, only one stream between each MCU can be added to the multi-screen or TV wall at the same time, as shown in Figure 2, mounted on Only one of the conference venues (T1, Τ3) under MCU_B can be added to the multi-screen or TV wall on MCU A, and one of the conference venues (T2, Τ4 Μ) mounted under MCU-C can be added to MCU-A. That is to say, the MCU-A cannot simultaneously receive the video code streams of multiple conference sites mounted under its lower-level MCU, which results in application defects and cannot meet user requirements.

In addition, all levels of connected MCUs can only support one point-to-point view at the same time, as shown in Figure 3. There is only one cascade channel between MCU-A and MCU-B, so it can only support T1 to choose T3 or T4, or T2 to choose T3 or T4, but it cannot support T1 to choose T3 and T2 at the same time. Look at T4, or choose T4 for T1 and choose T3 for T2. In addition, the cascaded MCUs can only support one point-to-point camera control at the same time.

In order to solve the above-mentioned problem, the embodiment of the present invention provides an MCU cascade system and a multimedia processing method and MCU of the system.

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

The embodiment of the present invention provides a multipoint control unit MCU cascade system. The system includes multiple MCUs. Figure 4 shows a structure of the system, including multiple MCUs. Each MCU needs to transmit multiple multimedia channels. When streaming, multiple cascading channels are established.

The multimedia code stream may be an audio code stream or a video code stream or an audio and video code stream, or an RDC or Telepresence code stream.

The following is a description of several application examples:

Example one

The cascade system shown in Figure 5 includes multiple MCUs, namely MCU51, MCU52, and MCU53. There are multiple venues mounted under MCU51, MCU52, and MCU53. The venues mounted under MCU51 are T511~T51n, which are mounted on The conference venues under the MCU52 are T521~T52n, and the conference venues under the MCU53 are T531~T53n.

In this embodiment, the cascaded channels between MCU51 and MCU52, and between MCU52 and MCU53 are used to transmit audio code streams. The audio code streams processed by various MCUs are different, which are described below:

The working process of MCU51 is: receiving and decoding audio code streams from conference venues T511 to T51n, selecting an audio code stream that meets predetermined conditions from the decoded audio code stream, and setting it in MCU51 and The multiple cascaded channels between MCU52 are sent to MCU52.

The working process of MCU52 is: receiving the audio code stream of the conference site T521 ~ T52n and the audio code stream sent from MCU51, and decode the audio code stream of the conference site T521 ~ T52n, and the decoded audio code stream and the audio code stream sent from MCU51 The audio code stream that meets the predetermined conditions is selected from the audio code stream, and is sent to the MCU53 through the cascade channel set between the MCU52 and the MCU53. It should be noted that the number of audio code streams meeting the predetermined conditions will be selected according to the actual number of cascaded channels, that is, the number of audio code streams meeting the predetermined conditions will not be greater than the actual number of cascaded channels.

The working process of the MCU53 is: receiving the audio code streams of the venues T531 to T53n and the audio code streams sent by the MCU42, decoding the audio code streams of the venues T531 to T53n, and sending the decoded audio code streams and the MCU52 Select an audio code stream that meets a predetermined condition from the audio code streams, perform audio mixing and encoding processing successively, and then provide it to the local conference site (T531 ~ T53n).

In this embodiment, an audio code stream that meets the predetermined condition refers to an audio code stream with louder sound, specifically, an audio code stream whose sound volume exceeds a predetermined threshold. The sound volume of the audio code stream can reduce the volume of the entire audio The decoded code stream is calculated by the envelope detection algorithm.

It can be seen that, in the embodiment of the present invention, since multiple cascaded channels can be established between MCUs as required, during the audio stream transmission process, the passing MCU may not perform operations such as mixing and encoding the audio stream. Instead, it selects several audio code streams from the multiple audio code streams including the audio code stream and sends them directly to the next MCU (upper MCU or lower MCU) on the transmission path, thereby reducing the number of intermediate audio streams on the transmission path. Transmission delay caused by operations such as mixing and encoding of audio data by MCUs at all levels.

In addition, each MCU in the above-mentioned MCU cascade system can also simultaneously transmit multiple video streams through multiple cascade channels. Specifically, it can be applied to cascaded multi-screen and TV wall settings, and multi-channel point-to-point viewing or control.

In the MCU cascading system disclosed in the embodiment of the present invention, there are multiple cascading channels between MCUs, so the MCU can transmit multiple video streams simultaneously through the multiple cascading channels, thereby solving the problems in the prior art described above. problem. The following will illustrate through examples two and three:

Example two

The TV wall on a certain MCU in the cascade system is equipped with multiple display screens, respectively corresponding to the venues mounted under the MCU cascaded with the MCU. The cascade system shown in Figure 6 includes MCU-A, MCU B, MCU-C and MCU-D, where MCU_A is connected to a multi-screen TV wall, and the multi-screen TV wall has 4 display sub-pictures (picture 1, picture 2, picture 3, and picture 4), and each sub-picture is connected to the cascade The conference venues T1, T2, T3 and T4 correspond to each other, T1 is mounted under MCU_B, T2 is mounted under MCU-C, and T3 and T4 are mounted under MCU-D.

Among them, each MCU establishes a cascade channel with an upper-level MCU or a lower-level MCU, a cascade channel (channel ab) is established between MCU_A and MCU_B, and three cascade channels (channel acl) are established between MCU_A and MCU_C. , Ac2 and ac3), two cascading channels (channels cdl and cd2) are established between MCU_C and MCU-D.

Specifically, the working process of each MCU is roughly the same, but there are some differences at the same time. The working process of each MCU in the system is described in detail below:

The working process of MCU_A is as follows:

After receiving the multi-screen setting message sent by the upper-level device (such as RM), check the conference site (including local conference site and cascade conference site) connected to this MCU. The specific checking method is: From the received multi-screen setting message Obtain its identification from the sub-picture data of the sub-screen, and then determine whether it is a conference site on the MCU according to the identification of each sub-screen conference site, and check the conclusion indication: there is no local conference site, and 4 cascaded conference sites (respectively CsdAB, CsdACl, CsdAC2, CsdAC3); Then, according to the b mark of the venue, determine the cascaded venue corresponding to each sub-screen on the MCU, and determine the result: Screen 1 corresponds to CsdAB, and screens 2, 3 and 4 correspond to CsdACl, CsdAC2, respectively , CsdAC3; Next, assign a corresponding cascade channel to each sub-picture, which can be allocated according to a preset allocation strategy. The result of the assignment is: cascade channel ab is used to transmit screen 1, cascade channel ac 1, ac2, ac3 They are used to transmit screens 2, 3, 4 respectively; after that, the setting information used to indicate the correspondence between the cascade channel and each screen is sent to the other end device (MCU_B and MCU_C) of the cascade channel. The allocation strategy may be:

The first is the cascade channel that already has the sub-picture conference site code stream, and the second is the idle sub-cascade channel. Thirdly, the used channels are preempted by comparing the MCU levels, and the higher level has priority; finally, the main cascade channel is preempted.

After the above operations, MCU-A can receive the video code stream carried on each level of link channel, and transmit it to the corresponding sub-screen for display.

The working process of MCU B is as follows:

Receive the setting information sent by MCU-A, and check the conference site connected to this MCU. The check result indicates: There is a local conference site (T1), and there is no cascaded conference site; then, determine the correspondence between T1 and the cascaded conference site CsdAB, That is, the video code stream from T1 is transmitted to MCU-A through the cascade channel ab. In addition, since there is no cascaded venue connection, there is no need to send setting information.

The working process of MCU C is as follows:

Receive the setting information sent by MCU-A, and check the conference site connected to this MCU. The check result indicates: There are 1 local conference site (T2) and 2 cascaded conference sites (CsdCD1 and CsdCD2); then, make sure that each sub-screen is in this For the conference site corresponding to the MCU, the determination result is: Picture 2 corresponds to T2, Picture 3 and Picture 4 correspond to CsdCD1 and CsdCD2, respectively; Next, the corresponding cascade channel is assigned to each sub-picture, and the result of the assignment is: The cascade channel cdl is used for transmission Screen 2, cascade channels cd2, cd3, are used to transmit screens 3 and 4, respectively; after that, according to the setting information sent by the MCU-A, the corresponding connection between the cascade channel cdl and the cascade channel acl, and the cascade channel are determined The correspondence relationship between cd2 and cascade channel ac2, and the correspondence relationship between cascade channel cd3 and cascade channel ac3, namely: configure the code stream of T2 to the cascade conference site CsdACl, and configure the code streams of the cascade conference site CsdCD1 and CsdCD2 respectively Give CsdAC2 and CsdAC3. Then, the setting information used to indicate the corresponding relationship between the cascade channel and each screen is respectively given to the other end device (MCU-D) of the cascade channel.

The working process of MCU_D is as follows:

Receive the setting information sent by MCU-C, and check the conference site connected to this MCU. The check result indicates: There are two local conference sites (T3 and T4), and there is no cascading conference site; then, determine each sub-picture Looking at the conference site corresponding to this MCU, the determination result is: Screen 3 corresponds to T3, Screen 4 corresponds to T4; Then, according to the setting information sent by MCU_C, configure the code stream of T3 to the cascaded conference site CsdCD2, and set the code stream of T4 Configured to the cascaded conference site CsdCD3.

It can be seen from the working process of the above-mentioned MCUs that the video code stream of venue T1 is transmitted to MCU-A by MCU_B through the cascade channel ab, and the video code stream of venue T2 is transmitted to MCU_A by MCU-C through the cascade channel acl. The video code streams of T3 and Τ4 are transmitted to MCU-C through cascade channels cd2 and cd3 respectively by MCU_D, and are transmitted to MCU-A through cascade channels ac2 and ac3 respectively by MCU-C; MCU_A will be carried on cascade channel ab The video code streams of, acl, ac2 and ac3 are respectively transmitted to pictures 1, 2, 3, and 4 of the RM for display. The MCU can transmit multiple video streams by establishing multiple cascading channels.

In addition, when there are multiple cascade channels between MCUs, they can be divided into two types: main cascade channels and sub-cascade channels. The number of main cascade channels is one, and the number of sub-cascade channels can be one or more. For example: Among the cascading channels acl, ac2 and ac3 between MCU-A and MCU_C, acl is the main cascading channel, ac2 and ac3 are sub-cascading channels; the cascading channel cdl between MCU-C and MCU-D In cd2, cdl is the main cascade channel, and cd2 is the sub-cascade channel.

It can be considered that the function of the main cascade channel is the same as the function of the channel between the MCUs in the prior art, and the sub-cascade channel is a channel for transmitting the spread code stream.

Example three

The multi-cascade channel structure of the MCU cascade system provided by the embodiment of the present invention can also solve the problem that the existing cascade system cannot support multi-channel point-to-point selection.

As shown in Figure 7, MCU-A is mounted with T1, Τ2, and T3, MCU-B is mounted with T4, Τ5, and T6, and MCU-A and MCU-B are provided with multiple cascade channels (abl , Ab2 and ab3 ).

The above-mentioned MCUs (MCU-A and MCU_B) select the appropriate cascade channel to send the corresponding video code stream according to the selection information sent by the local conference site.

Assuming that it is necessary to implement T1 to select T4, T2 to select T5, and T3 to select T6 at the same time, the workflows of MCU-A and MCU-B are as follows: The workflow of MCU A is:

Receive selected viewing information, which is used to indicate: T1 selects T4, T2 selects T5, and T3 selects T6; then, cascade channels are allocated to the selected conference site, and the allocation result is: abl for transmission The video code stream of venue T4, ab2 is used to transmit the video code stream of venue T5, and ab3 is used to transmit the video code stream of venue T6; and the corresponding relationship between the cascade channel and the venue that initiates the selection is established, that is, the connection channels at all levels The transmitted code stream is configured to the conference site that initiated the selection. The configuration result is: configure the code stream of abl to the conference site T1, configure the code stream of ab2 to the conference site T2, and configure the code stream of ab3 to the conference site T3; then, to the MCU — B sends the indication information used to indicate the correspondence between the cascading channel and the selected conference site (that is, the above-mentioned allocation result).

The workflow of MCU B is:

Receive the instruction information sent by MCU-A, where the instruction information is used to indicate: abl is used to transmit the video code stream of site T4, ab2 is used to transmit the video code stream of site T5, and ab3 is used to transmit the video code stream of site T6; To view the information, configure the code stream of the corresponding site to the corresponding cascade channel, and the configuration result is: the code stream of the site T4 is configured to abl, the code stream of the site T5 is configured to ab2, and the code stream of the site T6 is configured to ab3.

It can be seen from the working process of the above-mentioned MCUs that through the cascaded channels abl, ab2 and ab3 between MCU-A and MCU-B, the code streams of venues T4, Τ5, and Τ6 can be transmitted at the same time, thereby realizing venues T1, Τ2. The D of T4, T5, and T6 are selected at the same time as T3, which makes up for the deficiencies of the prior art.

Example four

The multi-cascade channel structure of the MCU cascade system provided by the embodiment of the present invention can also solve the problem that the existing cascade system cannot support the control of multi-channel point-to-point venue equipment. The venue equipment may be a camera, a microphone or other venues. The equipment, control principle and process are basically the same, and the following only takes camera control as an example for description.

Still taking the structure shown in Figure Ί as an example, assuming that T1 controls the camera of Τ4, Τ2 controls the camera of Τ5, and Τ3 controls the camera of Τ6, there are cascade channels abl and ab2 between MCU A and MCU-B. ab3; and allocate cascading channels for each venue, that is, T1 controls T4 through abl Camera, T2 controls the camera of T5 through ab2, and T3 controls the camera of T6 through ab3. Then the workflows of MCU_A and MCU_B are as follows:

The workflow of MCU A is:

Receive a camera control message sent by the controller, such as: a message from T1 to control a T4 camera; then, search for the cascade channel corresponding to the controller, and the search result is: the cascade channel corresponding to T1 is abl; The camera control message is transmitted to MCU_B through the cascade channel.

The workflow of MCU B is:

Receive the camera control message, and forward the control message sent from T1 to T4.

Correspondingly, when MCU-A receives a control message from T2 or T3, the workflow of MCU-A and MCU-B is also similar to receiving a control message from T1.

In all the above embodiments, the MCU can establish multiple cascaded channels in the following ways: Method 1. The MCU initiates a call (setup request) or receives a call according to the dial-in and dial-out mode of the conference site. When connecting channels, assign a call to each cascaded channel to ensure the uniqueness of the call.

As shown in Figure 8, 3 cascading channels (abl, ab2, ab3, respectively) need to be established between MCU-A and MCU_B, and different calls are allocated to each cascading channel, call 1 corresponds to abl, and call 2 corresponds to ab2, call 3 corresponds to ab3, MCU-A can initiate a call in order: call 1, call 2, call 3, MCU-B receives the call, after MUC-A and MCU_B have successfully exchanged capabilities, open the corresponding media The flow channel marks the completion of the establishment of bl, ab2 and ab3 between MCU-A and MCU-B.

It should be noted that, according to different types of cascaded conference sites, there are different ways to ensure the uniqueness of their calls. For example: For an E1 cascaded conference site or a 4E1 cascaded conference site, since E1 is a dedicated line, the line connection can ensure the call ——Correspondence; For IP cascaded conference sites, if the cascaded conference site uses number definition (that is, the conference access number of the opposite MCU), both the sending caller and the receiving caller use the number and site name to define a unique call, for example: When there is only one conference on the MCU, if the cascaded conference site uses IP definition, IP matching will be used to ensure the call is made when sending or receiving calls. answer.

Method 2. Initiate the creation of multiple cascading channels through one call. During capability exchange, distinguish different cascading channels by exchanging different media protocols and ports with the other party.

For example: Corresponding to the distinction between the main cascade channel and the sub-cascade channel, the main cascade channel uses the H264CIF protocol, the port is 100; the sub-cascade channel 1 uses the H264CIF protocol, and the port is 101; the sub-cascade channel 3 uses the H264CIF protocol, The port is 102; …and so on.

Method 3. Multiple media channels of multiple protocols on a UDP connection: MCU can transmit media streams of multiple protocols simultaneously in one channel by customizing the media protocol description header in UDP and extending the TP package. After receiving the media stream, the peer MCU uses the specific PayLoad value to identify the media stream of the specific protocol.

For example: The UDP media protocol description header in the first package is defined as: video, the PayLoad value is 34, and the UDP media protocol description header in the second package is defined as: audio, the PayLoad value is 98; …and so on.

It should be noted that the above-mentioned establishment methods are applicable but not limited to communication protocols such as H323, H320, and SIP.

In the MCU cascade system disclosed in the above embodiments, various MCUs belong to the protection scope of the present invention. The various MCUs are described in detail below.

It should be noted that, for the sake of clarity and brevity, the following embodiments only describe the differences between the MCU disclosed in the embodiments of the present invention and the existing MCU, and omit the same or substantially the same features as the existing MCU (for example: decoder, Encoders, etc.). Figure 9 shows the structure of an MCU, including: a cascade channel establishment unit 91, a code stream acquisition unit 92, a code stream selection unit 93, and a code stream sending unit 94, where:

The cascade channel establishment unit 91 is used to establish multiple cascade channels with other media control servers in the media control server cascade system.

The code stream obtaining unit 92 is configured to obtain decoded multiple audio code streams.

The code stream selection unit 93 is configured to select from the decoded multiple audio code streams in accordance with a predetermined Conditional multi-channel audio stream.

The predetermined condition is: the volume is greater than a predetermined threshold.

The code stream sending unit 94 is configured to send the multiple audio code streams selected by the code stream selecting unit 93 through multiple pre-established cascaded channels.

The decoded multi-channel audio code stream obtained by the code stream obtaining unit 92 may be a local conference site audio code stream, or a local conference site audio code stream and a set code of multiple audio code streams sent by other MCUs through multiple cascaded channels flow.

Figure 10 shows another MCU structure, including: a cascade channel establishment unit 101, a setting information receiving unit 102, a first processing unit 103, a second processing unit 104, and a setting information sending unit 105;

in:

The cascade channel establishment unit 101 is used to establish multiple cascade channels with other media control servers in the media control server cascade system.

The setting information receiving unit 102 is configured to receive the setting information sent by the upper-level device, and the setting information indicates the correspondence between the cascade channel and the sub-screen between the MCU and the upper-level device.

The first processing unit 103 is configured to determine the cascade conference site corresponding to each sub-screen on the MCU, and allocate a corresponding cascade channel to each sub-screen, where the cascade channel is located between the MCU and the lower-level device.

The second processing unit 104 is configured to configure the code stream of the cascade conference site corresponding to each sub-picture on the MCU to the cascade channel corresponding to the sub-picture among the cascade channels between the MCU and the upper-level device.

The setting information sending unit 105 is configured to send the setting information indicating the correspondence between the cascade channel and the sub-screen between the MCU and the lower-level device to the lower-level device.

Figure 11 shows another structure of the MCU, including: a cascade channel establishing unit 111, a selective viewing information receiving unit 112, and a third processing unit 113; where:

The cascade channel establishment unit 111 is used to establish multiple cascade channels with other media control servers in the media control server cascade system. The selective viewing information receiving unit 112 is configured to receive selective viewing information, where the selective viewing information carries the identifier of the selected venue and the identifier of the selected venue.

The third processing unit 113 is configured to allocate a cascade channel for the selected conference site, and configure the code stream of the cascade channel to the corresponding conference site that initiates the selection.

The present invention also discloses another MCU whose structure is as shown in FIG.

The cascade channel establishment unit 121 is used to establish multiple cascade channels with other media control servers in the media control server cascade system.

The instruction receiving unit 122 is configured to receive instruction information indicating the corresponding relationship between the selected conference site and the cascade channel, and the fourth processing unit 123 is configured to configure the code stream of the selected conference site to the corresponding cascade channel according to the corresponding relationship.

Figure 13 shows another MCU structure, including: a cascade channel establishment unit 131, a cascade channel allocation unit 132, a conference site device control message receiving unit 133, and a fifth processing unit 134, where:

The cascade channel establishment unit 131 is used to establish multiple cascade channels with other media control servers in the media control server cascade system.

The cascade channel allocation unit 132 is used to allocate cascade channels to each conference site.

The site equipment control message receiving unit 133 is configured to receive site equipment control messages.

The fifth processing unit 134 is configured to find a cascade channel corresponding to the controlling party, and transmit the conference site device control message to the media control server of the controlled party through the cascading channel.

Figure 14 shows another structure of the MCU, including: a cascade channel establishment unit 141, a site equipment control message receiving unit 142, a site equipment control message forwarding unit 143;

The cascade channel establishment unit 141 is used to establish multiple cascade channels with other media control servers in the media control server cascade system.

The venue equipment control message receiving unit 142 is configured to receive venue equipment control messages through the cascade channel, The site device control message forwarding unit 143 is configured to forward the received site device control message to the controlled party.

It should be noted that the MCU described in all the foregoing embodiments includes a unit for establishing a multi-channel cascade channel, and the working process of the functional unit can refer to the description of the method part above. Regarding the MCU cascading system disclosed in the above embodiment, the method for controlling the multimedia stream by the MCU also belongs to the protection category of the present invention, and the method for controlling the multimedia stream mainly includes the following:

Method 1, ten pairs of audio stream control methods.

Please refer to FIG. 15, which is a flowchart of an audio code stream control method, which includes the following steps: Step S151: Receive decoded multiple audio code streams.

Step S152: Select multiple audio code streams that meet predetermined conditions from the multiple audio code streams, and send them to other MCUs through multiple pre-established cascaded channels.

For the specific process, please refer to the working process of each MCU in the first embodiment above.

It should be noted that the decoded multi-channel audio code stream in the above step S101 is a collection of the local conference site audio code stream and the multi-channel audio code stream sent by other MCUs through multiple cascaded channels.

Methods 2, 40 control methods for video stream.

Please refer to FIG. 15, which is a flowchart of a method for controlling a video code stream, which includes the following steps: Step S161. Obtain multiple video code streams.

Step S162: Determine the corresponding relationship between each video code stream and the cascade channel, and transmit the multiple video code streams through the corresponding multiple cascade channels.

This method can be applied to the setting scenes of cascading multi-pictures and TV walls, or multi-channel point-to-point viewing or multi-channel point-to-point control. In each application scenario, the specific process may be different. Please refer to the previous embodiment. 2. The contents of the third embodiment and the fourth part of the embodiment.

Those skilled in the art can understand that any of many different processes and technologies can be used to represent information, messages, and signals. For example, the messages and information mentioned in the above description can all be expressed as voltage, current, electromagnetic wave, magnetic field or magnetic particles, light field or any combination of the above.

Professionals may further realize that in combination with the various examples described in the embodiments disclosed herein The units and algorithm steps of the examples can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the functions of each example. Composition and steps. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered as going beyond the scope of the present invention.

A person of ordinary skill in the art can understand that all or part of the processes in the method of the foregoing embodiments can be implemented by instructing relevant hardware through a computer program. The program can be stored in a computer readable storage medium. During execution, it may include the procedures of the above-mentioned method embodiments. Wherein, the storage medium may be a magnetic disk, an optical disc, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM), etc. The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.

Various modifications to these embodiments will be obvious to those skilled in the art. The principles defined in this document can be implemented in other embodiments without departing from the spirit or scope of the present invention. . Therefore, the present invention will not be limited to the embodiments shown in this document, but should conform to the widest scope consistent with the principles and novel features disclosed in this document.

Claims

1. A multimedia code stream control method of a media control server cascade system, characterized in that, in the cascade system, multiple cascade channels are established between each media control server, and the method includes: The code stream is sent to other media control servers through multiple pre-established cascading channels.

2. The method according to claim 1, wherein the multi-channel multimedia stream is a multi-channel audio stream of a local conference site, or a multi-channel audio stream including the local conference site and other media control servers.

3. The method according to claim 2, wherein the sending multiple multimedia code streams to other media control servers via pre-established multiple cascading channels comprises:

Selecting multiple audio code streams that meet predetermined conditions from the multiple audio code streams;

The multiple audio code streams that meet the predetermined conditions are sent to other media control servers through multiple pre-established cascading channels.

4. The method according to claim 3, wherein the predetermined condition is: the volume is greater than a predetermined threshold.

5. The method according to claim 1, wherein the multiple multimedia code streams are video code streams, and the multiple multimedia code streams are sent to other media control servers through multiple pre-established cascaded channels It includes: determining the corresponding relationship between each video code stream and the cascaded channel, and transmitting the multiple video code streams through the corresponding multiple cascaded channels.

6. A media control server, characterized in that the media control server is located in a media control server cascade system, and the media control server includes:

7. The media control server according to claim 6, further comprising: a code stream obtaining unit, configured to obtain decoded multiple audio code streams carried in the multiple cascaded channels; A code stream selection unit, configured to select multiple audio code streams that meet a predetermined condition from the decoded multiple audio code streams;

The code stream sending unit is configured to send the multiple audio code streams selected by the code stream selection unit through multiple pre-established cascaded channels.

8. The media control server according to claim 6, wherein the stream to be transmitted is a picture of each conference site.

9. The media control server according to claim 8, further comprising: a setting information sending unit, configured to indicate the level between the local media control server and the lower-level device when there is a lower-level media control server The setting information of the correspondence between the link channel and the sub-picture is sent to the lower-level device.

10. The media control server according to claim 8 or 9, further comprising: a setting information receiving unit, configured to receive setting information sent by a higher-level device, the setting information instructing the media control server to communicate with each other. Describe the correspondence between the cascading channels and the sub-pictures between the upper-level devices;

The first processing unit is used to determine the cascade conference site corresponding to each sub-picture on the MCU, and allocate a corresponding cascade channel to each sub-picture;

The second processing unit is configured to configure the code stream of the cascade conference site corresponding to each sub-picture on the MCU to the cascade channel corresponding to the sub-picture among the cascade channels between the MCU and other MCUs.

11. The media control server according to claim 6, further comprising: a selective viewing information receiving unit, configured to receive selective viewing information, the selective viewing information carrying the selected viewing venue identifier and the selected viewing venue identifier;

The third processing unit is configured to allocate a cascade channel to the selected conference site, and configure the code stream of the cascade channel to the corresponding conference site that initiates the selection.

12. The server according to claim 6, further comprising:

The instruction receiving unit is configured to receive instruction information indicating the correspondence between the selected conference site and the cascade channel; the fourth processing unit is configured to configure the code stream of the selected conference site to the corresponding level according to the corresponding relationship. 联通。 Linked channels.

13. The media control server according to claim 6, further comprising: a cascade channel allocation unit, configured to allocate cascade channels to each conference site;

A site equipment control message receiving unit, configured to receive a site equipment control message;

The fifth processing unit is configured to find a cascade channel corresponding to the controlling party, and transmit the conference site device control message to the media control server of the controlled party through the cascading channel.

14. The media control server according to claim 6, further comprising: a site device control message receiving unit, configured to receive site device control messages through a cascade channel; and a site device control message forwarding unit, configured to transfer all The received conference site device control message is forwarded to the controlled party.

15. A media control server cascade system, characterized in that it includes multiple media control servers, and multiple cascade channels are established between each media control server when multiple video streams need to be transmitted, wherein:

16. A media control server cascade system, characterized in that it includes a first media control server, a second media control server, and a third media control server, and each media control server is established when multiple audio streams need to be transmitted. Multiple cascading channels, of which:

The first media control server is configured to receive multiple local conference site audio code streams, select multiple audio code streams that meet predetermined conditions after decoding, and send them through multiple pre-established transmission channels; the second media control Server, configured to receive multiple channels of local conference site audio code streams and multiple channels of audio code streams sent by the first media control server, decode the multiple channels of local conference site audio code streams, and obtain information from the decoding result and the first Select the multiple audio code streams that meet the predetermined conditions from the multiple audio code streams sent by the media control server, and send them through multiple pre-established transmission channels;

The third media control server is configured to receive multiple channels of local conference site audio code streams and multiple channels of audio code streams sent by the second media control server, and decode the multiple channels of local conference site audio code streams, And from the decoding result and the multiple audio code streams sent by the second media control server, multiple audio code streams that meet the predetermined requirements are selected, processed, and then provided to the local venue.