CN113824919A - Control method of video conference system, multipoint control unit and storage medium - Google Patents
Control method of video conference system, multipoint control unit and storage medium Download PDFInfo
- Publication number
- CN113824919A CN113824919A CN202010564660.8A CN202010564660A CN113824919A CN 113824919 A CN113824919 A CN 113824919A CN 202010564660 A CN202010564660 A CN 202010564660A CN 113824919 A CN113824919 A CN 113824919A
- Authority
- CN
- China
- Prior art keywords
- video conference
- conference terminal
- video
- control unit
- code stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 238000004590 computer program Methods 0.000 claims description 6
- 230000010365 information processing Effects 0.000 claims description 3
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
- H04N7/152—Multipoint control units therefor
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention discloses a control method of a video conference system, a multipoint control unit and a storage medium, wherein the control method of the video conference system is applied to the multipoint control unit and comprises the following steps: responding to a private chat request initiated by a first video conference terminal to a second video conference terminal, and acquiring a code stream of the second video conference terminal through a second multipoint control unit, wherein the second video conference terminal is connected to the second multipoint control unit; sending the code streams of the second video conference terminal and the third video conference terminal to the first video conference terminal; and transmitting the code stream of the first video conference terminal to a second video conference terminal through a second multipoint control unit. Therefore, the first video conference terminal and the second video conference terminal can establish the private chat conference under the multi-level video conference, the audio and video data of the private chat conference cannot influence the progress of the private chat conference as a main conference place, and convenience is brought to the application of the video conference.
Description
Technical Field
The present invention relates to the field of video conference technologies, and in particular, to a control method of a video conference system, a multipoint control unit, and a video conference terminal.
Background
The video conference is a real-time, bidirectional or multiparty video communication conference held by a video conference terminal and a video conference Multipoint Control Unit (MCU), and through the video conference, people can conveniently perform a teleconference, training and teaching at multiple places.
Generally speaking, a main conference terminal as a broadcast source and a video conference terminal for receiving the broadcast source exist in a video conference, and code stream transmission is performed between the video conference terminal and the main conference terminal through a video channel; in the process of video conference, when participants need to communicate with each other, a private chat sub-conference needs to be established in the current video conference, but in the prior art, the establishment of the private chat sub-conference can only be realized under a single-level conference, namely, a main conference terminal serving as a broadcast source and two video conference terminals establishing the private chat sub-conference need to be directly connected with the same multipoint control unit, the private chat sub-conference is established by the multipoint control unit, but the private chat sub-conference cannot be established under the condition that more than two multipoint control units are cascaded.
Disclosure of Invention
The embodiment of the invention mainly aims to provide a control method of a video conference system, aiming at establishing a private chat sub-conference under a multi-level video conference.
In order to achieve the above object, an embodiment of the present invention provides a method for controlling a video conference system, including:
the video conference system comprises a first video conference terminal, a second video conference terminal, a third video conference terminal, a first multipoint control unit and a second multipoint control unit, the control method is applied to the first multipoint control unit, and the control method comprises the following steps:
establishing connection with the first video conference terminal and the third video conference terminal respectively, and receiving code streams from the first video conference terminal and the third video conference terminal;
responding to a private chat request initiated by the first video conference terminal to the second video conference terminal, and acquiring a code stream of the second video conference terminal through the second multipoint control unit, wherein the second video conference terminal is connected to the second multipoint control unit;
sending the code streams of the second video conference terminal and the third video conference terminal to the first video conference terminal;
and transmitting the code stream of the first video conference terminal to the second video conference terminal through the second multipoint control unit.
In order to achieve the above object, an embodiment of the present invention further provides a multipoint control unit, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program, when executed by the processor, implements the control method of the video conference system.
In order to achieve the above object, an embodiment of the present invention further provides a computer-readable storage medium, on which an information processing program is stored, and the information processing program, when executed by a processor, implements the control method of the aforementioned video conference system.
The embodiment of the invention provides a control method, a multi-point control unit and a storage medium of a video conference system, which utilize the control method of the video conference system to establish a private chat sub-conference for two video conference terminals under the current multi-level video conference, for example, a first video conference terminal is directly connected with a first multi-point control unit, a second video conference terminal is directly connected with a second multi-point control unit, the first multi-point control unit and the second multi-point control unit are connected in a cascade way or belong to two different conference places, when the first multi-point control unit receives a private chat request initiated by the first video conference terminal to the second video conference terminal, the first multi-point control unit establishes a sub-conference for the first video conference terminal and the second video conference terminal and controls the exchange of audio and video data of the first video conference terminal and the second video conference terminal between the first multi-point control unit and the second multi-point control unit, and simultaneously, audio and video data of a third video conference terminal serving as a broadcast source are provided for the first video conference terminal, so that the first video conference terminal and the second video conference terminal can establish a private chat conference under a multi-level video conference, the audio and video data of the private chat conference can not influence the third video conference terminal serving as the broadcast source, and convenience is brought to the application of the video conference.
Drawings
Fig. 1 is a schematic diagram of a video conference system connection structure of a cascade structure according to an embodiment of the present invention;
fig. 2 is a flowchart of a control method of a video conference system according to an embodiment of the present invention;
fig. 3 is a flowchart of a control method of a video conference system according to another embodiment of the present invention;
fig. 4 is a flowchart of a control method of a video conference system according to another embodiment of the present invention;
fig. 5 is a schematic diagram of a video conference system connection structure of a cascade structure according to another embodiment of the present invention;
fig. 6 is a flowchart of a control method of a video conference system according to another embodiment of the present invention;
fig. 7 is a schematic diagram of a connection structure of a video conference system including two main conference sites according to another embodiment of the present invention;
fig. 8 is a flowchart of a control method of a video conference system according to another embodiment of the present invention;
fig. 9 is a flowchart of a control method of a video conference system according to another embodiment of the present invention;
fig. 10 is a flowchart of a control method of a video conference system according to another embodiment of the present invention;
fig. 11 is a diagram illustrating a device connection structure of a multipoint control unit according to an embodiment of the present invention.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the following description, suffixes such as "module", "part", or "unit" used to denote elements are used only for facilitating the explanation of the present invention, and have no peculiar meaning in itself. Thus, "module", "component" or "unit" may be used mixedly.
A Multipoint Control Unit (MCU) in the video conference selects a video conference terminal as a main conference terminal, receives the audio and video code stream of the video conference terminal and broadcasts the audio and video code stream to other video conference terminals, and the audio and video code streams of other video conference terminals are also broadcast by the Multipoint Control Unit; in the process of video conference, conference members (video conference terminals) may need to discuss with each other, in order to not affect the normal operation of a main conference place, a multipoint control unit often needs to disconnect video conference terminals of private chat and exchange audio and video data through the multipoint control unit, two video conference terminals needing to establish the private chat are directly connected to the same multipoint control unit to be controlled, along with the development of a multistage conference, the conditions of a plurality of video conference terminals in a cascading group are more and more, however, the private chat sub-conference cannot be established under a cascading structure, and inconvenience is brought to the application of the video conference. In this regard, the following described embodiments of the invention provide a method of establishing a private chat sub-conference in a multi-level video conference.
The present embodiment provides a control method of a video conference system, where the control method is a method for establishing a private chat sub-conference in a multi-level video conference, the multi-level video conference has two or more multipoint control units, and two or more video conference terminals establishing the private chat are at least directly connected to two different multipoint control units.
In one embodiment, referring to fig. 1, the video conferencing system is in a cascaded configuration, consisting of: the first multipoint control unit is directly connected with a first video conference terminal (represented by a terminal A in the figure), the second multipoint control unit is directly connected with a second video conference terminal (represented by a terminal B in the figure), the first multipoint control unit is connected with the second multipoint control unit to form a cascade conference, the cascade conference system further comprises a third video conference terminal (represented by a terminal C in the figure) directly connected to the first multipoint control unit, the third video conference terminal serves as a main conference place, and the first multipoint control unit broadcasts a code stream of the third video conference terminal to each video conference terminal below.
As shown in fig. 2, fig. 2 is a flowchart of a control method of the video conference system in the above embodiment, and the control method is applied to a first multipoint control unit, and includes the following steps:
s110: respectively establishing connection with a first video conference terminal and a third video conference terminal, and receiving code streams from the first video conference terminal and the third video conference terminal;
in this embodiment, the first multipoint control unit is directly connected to the third video conference terminal and selects the third video conference terminal as a main conference place, and the second multipoint control unit is connected to the first multipoint control unit to form a cascade conference, so that from the perspective of the first multipoint control unit, the first multipoint control unit needs to receive and broadcast the code streams of the first video conference terminal and the third video conference terminal, so that the second video conference terminal can view the pictures of the first video conference terminal and the third video conference terminal, and simultaneously receive and broadcast the code stream of the second video conference terminal sent by the second multipoint control unit.
S120, responding to a private chat request initiated by the first video conference terminal to the second video conference terminal, and acquiring a code stream of the second video conference terminal through the second multi-point control unit;
when a first video conference terminal needs to chat with a second video conference terminal in a private mode, the first video conference terminal sends a private chat conference request to a first multipoint control unit, and after the first multipoint control unit receives the private chat conference request initiated by the first video conference terminal, the private chat conference is established for the first video conference terminal and the second video conference terminal.
S130, sending the code streams of the second video conference terminal and the third video conference terminal to the first video conference terminal;
and S140, transmitting the code stream of the first video conference terminal to the second video conference terminal through the second multipoint control unit, and simultaneously transmitting the code stream of the third video conference terminal to the second video conference terminal through the second multipoint control unit.
In order to enable both a first video conference terminal and a second video conference terminal in a private chat sub-conference to receive audio and video code streams of a broadcast source, a first multi-point control unit sends the code stream of a third video conference terminal to the private chat sub-conference, so that the first video conference terminal and the second video conference terminal can obtain pictures and sounds of the third video conference terminal simultaneously in the private chat process, but because the audio and video data of the first video conference terminal and the second video conference terminal are only exchanged between the first video conference terminal and the second video conference terminal in the private chat sub-conference, the private chat of the first video conference terminal and the second video conference terminal does not affect a main conference place.
Through the control method from step S110 to step S140, the establishment of the private chat sub-conference is realized in the cascade video conference shown in fig. 1, and the communication effect of the video conference is greatly improved.
It should be noted that the code stream of the video conference terminal usually includes a video code stream and an audio code stream, and those skilled in the art can know that the video code stream and the audio code stream are exchanged between the video conference terminal and the multipoint control unit through different channels, for example, in the aspect of transmission of the video code stream, a first video channel and a second video channel are usually established between the video conference terminal and the multipoint control unit, the first video channel can be used to transmit a main video code stream, such as camera data, the second video channel can be used to transmit a video code stream with a slightly smaller data amount, such as a computer screen, and part of the video conference terminal also supports a third video channel and even a fourth video channel for transmitting more paths of video data; the video code stream transmission of the embodiment of the invention has the following different control methods according to the difference of the number of video channels supported by the video conference terminal:
referring to fig. 3, if the first video conference terminal supports the third video channel, step S130 specifically includes:
s132, establishing a first video channel, a second video channel and a third video channel with the first video conference terminal;
and S133, sending the code stream of the third video conference terminal to the first video conference terminal through the first video channel and/or the second video channel, and sending the video code stream of the second video conference terminal to the first video conference terminal through the third video channel.
The third video channel is used for transmitting pictures of the private chat sub-conference, so that the video code stream of the private chat sub-conference can be prevented from influencing the video code stream of the third video conference terminal, and the main conference place (namely the third video conference terminal) always needs the first video channel and the second video channel to transmit the code stream; in addition, the first video conference terminal can also synthesize the picture of the main conference place and the picture of the second video conference terminal, play the pictures on one display screen, and also realize the simultaneous watching.
Referring to fig. 4, if the first video conference terminal does not support the third video channel, step S130 specifically includes:
s134, establishing a first video channel with the first video conference terminal;
and S135, synthesizing the images of the second video conference terminal and the third video conference terminal, and sending the synthesized images to the first video conference terminal through the first video channel.
And similarly, for a second video conference terminal which does not support a third video channel, the second multipoint control unit can also synthesize the pictures of the first video conference terminal and the third video conference terminal and then send the pictures to the second multipoint control unit.
Before the first multipoint control unit and the first video conference terminal establish the third video channel, the first multipoint control unit needs to determine whether the first video conference terminal supports the third video channel, and the method is implemented by the following steps:
s131, interacting a terminal capability set with the first video conference terminal, and determining that the first video conference terminal supports the third video channel capability.
The operation is performed through a capability interaction module of the first multi-point control unit, and the capability interaction module is used for determining a video format selected by the call according to the capability types of the main video/the auxiliary video/the private chat video supported by the video conference terminal in the video conference call establishment stage. Therefore, it can be understood that after the image acquisition of the third video conference terminal is completed, the image is compressed by the video coding algorithms such as h.264 and h.265, and then becomes a video code stream, and the video code stream is sent to the first multipoint control unit, and after the first video conference terminal receives the video code stream, the image is output after the video decoding algorithm matched with the coding is performed on the video code stream.
In the aspect of transmitting audio code streams, in the prior art, audio data is usually encoded and compressed, then the audio code streams are transmitted through an audio channel, and finally the audio code streams are decoded and played at a video conference terminal. In step S110, the stage of establishing the cascade video conference in the embodiment of the present invention includes:
and an audio receiving channel and an audio sending channel are established with the first video conference terminal, the audio receiving channel is used for receiving audio data in the code stream of the first video conference terminal, and the audio sending channel is used for sending audio data in the code stream of the third video conference terminal to the first video conference terminal.
Through the audio receiving channel and the audio sending channel, the first video conference terminal can receive and send voice, so that the first video conference terminal participates in the current cascade video conference.
After the private chat sub-conference is established, in order to avoid influence of voice of the first video conference terminal on a main conference place, the first multi-point control unit newly establishes the private chat sub-conference for the first video conference terminal and the second video conference terminal, and as the private chat sub-conference is not completely isolated from the current cascade video conference, the private chat sub-conference also needs to receive a code stream broadcasted by the third video conference terminal, so that the first multi-point control unit needs to adjust a forwarding rule of the first multi-point control unit, and the method specifically comprises the following steps:
and setting the forwarding object of the received code stream of the first video conference terminal as a second video conference terminal, and setting the forwarding object of the received code stream of the second video conference terminal as the first video conference terminal.
Under the current cascade video conference, the audio channels pre-established by the first multipoint control unit and the first video conference terminal are not suitable for private chat sub-conferences, namely, the first multipoint control unit broadcasts the audio code stream of the first video conference terminal to all the participating terminals, if the private chat conference is established at the moment without changing the forwarding rule, the audio code stream of the first video conference terminal can affect the main meeting place, therefore, the first multipoint control unit releases the related resources by adjusting the internal forwarding rule, so that the audio code streams of the first video conference terminal and the second video conference terminal are only transmitted between the first video conference terminal and the second video conference terminal, and the sending channel of the audio code stream of the third conference terminal is kept, namely, channels for sending the audio code stream of the main meeting place to the first video conference terminal and the second video conference terminal are maintained, and the function of watching the main meeting place simultaneously in the private chat process is realized.
In the foregoing procedure of adjusting the forwarding object, the method further includes: mixing the received voice data in the code stream of the first video conference terminal, the received voice data in the code stream of the second video conference terminal and the received voice data in the code stream of the third video conference terminal;
and sending the voice data after the sound mixing to the first video conference terminal and the second video conference terminal.
And the first video conference terminal removes the voice of the first video conference terminal in the audio mixing data based on an Acoustic Echo Cancellation (AEC) function, so that the voice of the second video conference terminal and the voice of the third video conference terminal are played at the first video conference terminal.
The embodiment of the invention is a control method for establishing the private chat sub-conference based on the cascade video conference, and the control method can be used for facilitating the video conference terminals participating in the conference to establish the private chat sub-conference for communication, simultaneously not influencing the main conference, and improving the use experience of the video conference.
In another embodiment, referring to fig. 5, the cascade of video conferencing systems consists of: the video conference system also comprises a third multipoint control unit, wherein a first video conference terminal (represented by a terminal A in the figure) is directly connected with the first multipoint control unit, a second video conference terminal (represented by a terminal B in the figure) is directly connected with the second multipoint control unit, and a third video conference terminal (represented by a terminal C in the figure) is directly connected with the third multipoint control unit, wherein the third multipoint control unit is used as the uppermost multipoint control unit and is respectively connected with the first multipoint control unit and the second multipoint control unit, the third video conference terminal is used as a main conference place, and the third multipoint control unit broadcasts the code stream of the third video conference terminal to each video conference terminal below.
As shown in fig. 6, fig. 6 is a flowchart of a control method of the video conference system in the above embodiment, the control method is applied to a first multipoint control unit, and includes the following steps:
s210, establishing connection with a first video conference terminal and establishing connection with a third video conference terminal through a third multipoint control unit;
s220, receiving a code stream from the first video conference terminal, and receiving a code stream from the third video conference terminal through the third multipoint control unit;
in this embodiment, the first multipoint control unit is connected to the third video conference terminal through the third multipoint control unit, so as to receive a code stream of the third video conference terminal serving as a main conference site and forward the code stream to the first video conference terminal, and meanwhile, the code stream of the first video conference terminal also needs to be broadcast in the current cascade video conference.
S230, responding to a private chat request initiated by the first video conference terminal to the second video conference terminal, and acquiring a code stream of the second video conference terminal through the second multipoint control unit, wherein the second video conference terminal is connected to the second multipoint control unit;
when a first video conference terminal needs to chat with a second video conference terminal in a private mode, the first video conference terminal sends a private chat conference request to a first multipoint control unit, and after the first multipoint control unit receives the private chat conference request initiated by the first video conference terminal, the private chat conference is established for the first video conference terminal and the second video conference terminal. Certainly, since the first multipoint control unit and the second multipoint control unit are not directly connected at this time, data may be transmitted through the third multipoint control unit as an intermediary, or a new channel is established between the first multipoint control unit and the second multipoint control unit for data transmission.
S240, sending the code streams of the second video conference terminal and the third video conference terminal to the first video conference terminal;
and S250, transmitting the code stream of the first video conference terminal to a second video conference terminal through a second multipoint control unit.
The first multipoint control unit acquires the code stream of the third video conference terminal through the third multipoint control unit and sends the code stream to the first video conference terminal, so that the first video conference terminal can simultaneously obtain the picture and the sound of the third video conference terminal in the private chat process, but the private chat of the first video conference terminal and the second video conference terminal only exchanges audio and video data between the first video conference terminal and the second video conference terminal in the private chat sub-conference, so that the private chat of the first video conference terminal and the second video conference terminal does not affect the main conference place. Similarly, for the second video conference terminal, the second multipoint control unit acquires the code stream of the third video conference terminal through the third multipoint control unit and sends the code stream to the second video conference terminal, so that the second video conference terminal can simultaneously acquire the picture and the sound of the third video conference terminal in the private chat process.
It is to be understood that, similar to the first embodiment, the code stream of the video conference terminal generally includes a video code stream and an audio code stream, and the present embodiment only differs from the first embodiment in that the first multipoint control unit is not directly connected to the third video conference terminal, but is connected to the third video conference terminal through the third multipoint control unit, so in terms of video code stream and audio code stream transmission, the present embodiment is the same as the description of the first embodiment, and in order to avoid repetition, the description is omitted here.
In another embodiment, referring to fig. 7, a video conferencing system includes two separate video conferencing sites, consisting of: the first multipoint control unit is directly connected with a first video conference terminal (shown as a terminal A in the figure), the second multipoint control unit is directly connected with a second video conference terminal (shown as a terminal B in the figure), the first multipoint control unit and the second multipoint control belong to two different conference places, and for convenience of description, a fourth video conference terminal (shown as a terminal D in the figure) is introduced to serve as another main conference place, so that the main conference place terminal of the first multipoint control unit is a third video conference terminal (shown as a terminal C in the figure), and the main conference place terminal of the second multipoint control unit is a fourth video conference terminal.
As shown in fig. 8, fig. 8 is a flowchart of a control method of the video conference system in the above embodiment, the control method is applied to the first multipoint control unit, and includes the following steps:
s310: respectively establishing connection with a first video conference terminal and a third video conference terminal, and receiving code streams from the first video conference terminal and the third video conference terminal;
in this embodiment, the first multipoint control unit is directly connected to the third video conference terminal and selects the third video conference terminal as the main conference place, and the first video conference terminal receives the broadcasted code stream of the third video conference terminal.
S320: responding to a private chat request initiated by a first video conference terminal to a second video conference terminal, and acquiring a code stream of the second video conference terminal through a second multi-point control unit;
when a first video conference terminal needs to chat with a second video conference terminal in a private mode, the first video conference terminal sends a private chat request to a first multi-point control unit, the first multi-point control unit establishes a private chat conference for the first video conference terminal and the second video conference terminal after receiving the private chat request initiated by the first video conference terminal, at the moment, the first multi-point control unit establishes connection with the second multi-point control unit in a network mode and the like, and code stream data are transmitted through a newly-established signal; since the private chat sub-conference is a newly established video conference, the broadcast rule of the audio/video code stream of each terminal in the video conference is also applicable to the private chat sub-conference, that is, the audio/video code streams of the first video conference terminal and the second video conference terminal can be received by the other party.
S330, sending the code streams of the second video conference terminal and the third video conference terminal to the first video conference terminal;
and S340, sending the code stream of the first video conference terminal to a second video conference terminal through a second multipoint control unit.
Because the first video conference terminal and the second video conference terminal respectively receive different broadcast sources, only the received code streams of the second video conference terminal and the third video conference terminal are sent to the first video conference terminal from the perspective of the first multipoint control unit, and meanwhile, in order to enable the second video conference terminal to receive the audio and video data of the first video conference terminal, the first multipoint control unit also sends the code stream of the first video conference terminal to the second multipoint control unit, and the second multipoint control unit sends the received code streams of the first video conference terminal and the fourth video conference terminal to the second video conference terminal, so that private chat of the first video conference terminal and the second video conference terminal is realized.
The private chat conference is established for the first video conference terminal and the second video conference terminal which need private chat through the steps, and the private chat conference is a new video conference place, so that the audio and video data of the first video conference terminal and the second video conference terminal are only exchanged between the first video conference terminal and the second video conference terminal, the private chat of the first video conference terminal and the second video conference terminal can not affect respective main meeting places, and the communication effect of the video conference is greatly improved.
It should be noted that the code stream of the video conference terminal usually includes a video code stream and an audio code stream, and those skilled in the art can know that the video code stream and the audio code stream are exchanged between the video conference terminal and the multipoint control unit through different channels, for example, in the aspect of transmission of the video code stream, a first video channel and a second video channel are usually established between the video conference terminal and the multipoint control unit, the first video channel can be used to transmit a main video code stream, such as camera data, the second video channel can be used to transmit a video code stream with a slightly smaller data amount, such as a computer screen, and part of the video conference terminal also supports a third video channel and even a fourth video channel for transmitting more paths of video data; the video code stream transmission of the embodiment of the invention has the following different control methods according to the difference of the number of video channels supported by the video conference terminal:
referring to fig. 9, if the first video conference terminal supports the third video channel, step S330 specifically includes:
s332, establishing a first video channel, a second video channel and a third video channel with the first video conference terminal;
and S333, sending the code stream of the third video conference terminal to the first video conference terminal through the first video channel and/or the second video channel, and sending the video code stream of the second video conference terminal to the first video conference terminal through the third video channel.
The third video channel is used for transmitting pictures of the private chat sub-conference, so that the video code stream of the private chat sub-conference can be prevented from influencing the video code stream of the third video conference terminal, and the main conference place (namely the third video conference terminal) usually needs the first video channel and the second video channel to transmit the code stream; in addition, the first video conference terminal can also synthesize the picture of the main conference place and the picture of the second video conference terminal, play the pictures on one display screen, and also realize the simultaneous watching. Similarly, for the second multipoint control unit, a third video channel may also be established with the second video conference terminal supporting the third video channel, and the video stream of the private chat sub-conference is transmitted through the third video channel.
Referring to fig. 10, if the first video conference terminal does not support the third video channel, step S330 specifically includes:
s334, establishing a first video channel with a first video conference terminal;
and S335, synthesizing the pictures of the second video conference terminal and the third video conference terminal, and sending the synthesized pictures to the first video conference terminal through the first video channel.
And similarly, for a second video conference terminal which does not support a third video channel, the second multipoint control unit can also synthesize the pictures of the first video conference terminal and the third video conference terminal and then send the pictures to the second multipoint control unit.
Before the first multipoint control unit and the first video conference terminal establish the third video channel, the first multipoint control unit needs to determine whether the first video conference terminal supports the third video channel, and the method is implemented by the following steps:
and S331, interacting a terminal capability set with the first video conference terminal, and determining that the first video conference terminal supports the third video channel capability.
The steps are carried out through a capability interaction module of the first multi-point control unit, and the capability interaction module is used for determining the video format selected by the call according to the capability types of the main video/the auxiliary video/the private chat video supported by the video conference terminal in the video conference call establishment stage. Therefore, it can be understood that after the image acquisition of the third video conference terminal is completed, the image is compressed by the video coding algorithms such as h.264 and h.265, and then becomes a video code stream, and the video code stream is sent to the first multipoint control unit, and after the first video conference terminal receives the video code stream, the image is output after the video decoding algorithm matched with the coding is performed on the video code stream.
It is to be understood that the above-described control method for the first multipoint control unit is also applicable to the second multipoint control unit, since the structural configuration of the first video conference terminal-the first multipoint control unit-the third video conference terminal is substantially the same as the structural configuration of the second video conference terminal-the second multipoint control unit-the fourth video conference terminal. It should be noted that there may be other multipoint control units between the second multipoint control unit and the second video conference terminal, so as to form a cascade structure, in this case, the control method of the second multipoint control unit may refer to the steps of the first embodiment, and details are not described herein.
In the aspect of transmitting audio code streams, in the prior art, audio data is usually encoded and compressed, then the audio code streams are transmitted through an audio channel, and finally the audio code streams are decoded and played at a video conference terminal. In the video conference establishing stage in step S310, the embodiment of the present invention includes the following steps:
and an audio receiving channel and an audio sending channel are established with the first video conference terminal, the audio receiving channel is used for receiving audio data in the code stream of the first video conference terminal, and the audio sending channel is used for sending audio data in the code stream of the third video conference terminal to the first video conference terminal.
Through the audio receiving channel and the audio transmitting channel, the first video conference terminal can receive and transmit voice so as to participate in the current video conference.
After the private chat sub-conference is established, in order to avoid that the voice of the first video conference terminal affects the main conference place, the first multi-point control unit newly establishes the private chat sub-conference for the first video conference terminal and the second video conference terminal, because the private chat sub-conference is not completely isolated from the current video conference, the first multi-point control unit is further connected with the third video conference terminal and sends the code stream of the third video conference terminal to the first video conference terminal, and the second multi-point control unit also needs to send the code stream of the fourth video conference terminal to the second video conference terminal, so taking the first multi-point control unit as an example, the first multi-point control unit needs to adjust the forwarding rule of the first multi-point control unit in the processing process of the audio code stream, and specifically comprises the following steps:
and setting the forwarding object of the received code stream of the first video conference terminal as a second video conference terminal, and setting the forwarding object of the received code stream of the second video conference terminal as the first video conference terminal.
Under the video conference of the first multipoint control unit, the audio channel pre-established by the first multipoint control unit and the first video conference terminal is not suitable for private chat conferences, namely, the first multipoint control unit broadcasts the audio code stream of the first video conference terminal to all the participating terminals, if the private chat conference is established at the moment without changing the forwarding rule, the audio code stream of the first video conference terminal can affect the main meeting place, therefore, the first multipoint control unit releases the related resources by adjusting the internal forwarding rule, so that the audio code streams of the first video conference terminal and the second video conference terminal are only transmitted between the first video conference terminal and the second video conference terminal, and the sending channel of the audio code stream of the third conference terminal is kept, namely, a channel for sending the audio code stream of the main meeting place to the first video conference terminal is maintained, and the function of watching the main meeting place simultaneously in the private chat process is realized. Similarly, the second multipoint control unit may also process the audio code stream with reference to the above steps, except that the audio code stream of the third video conference terminal is replaced with the code stream of the fourth video conference terminal.
In the foregoing procedure of adjusting the forwarding object, the method further includes: mixing the received voice data in the code stream of the first video conference terminal, the received voice data in the code stream of the second video conference terminal and the received voice data in the code stream of the third video conference terminal;
and sending the voice data after the sound mixing to the first video conference terminal.
The first multi-point control unit performs audio mixing operation on audio sent by the first video conference terminal, the second video conference terminal and the third video conference terminal, and sends the code stream after audio mixing to the first video conference terminal, and the first video conference terminal removes voice of the first video conference terminal in audio mixing data based on an Acoustic Echo Cancellation (AEC) function, so that voice of the second video conference terminal and voice of the third video conference terminal are played at the first video conference terminal. For the second multipoint control unit, the audio mixing object includes audio data sent by the first video conference terminal, the second video conference terminal and the fourth video conference terminal, and the second video conference terminal plays voices of the first video conference terminal and the fourth video conference terminal after filtering through the AEC function.
The control method provided by the embodiment of the invention is characterized in that under the current multi-level video conference, when a first multi-point control unit receives a private chat conference request initiated by a first video conference terminal to a second video conference terminal, the first multi-point control unit establishes a sub-conference for the first video conference terminal and the second video conference terminal, controls the audio and video data of the first video conference terminal and the second video conference terminal to be exchanged between the first video conference terminal and the second video conference terminal, and simultaneously provides the audio and video data of a third video conference terminal serving as a broadcast source for the first video conference terminal, so that the private chat conference established by the first video conference terminal and the second video conference terminal under the multi-level video conference is realized, and the audio and video data of the private chat cannot influence the third video conference terminal serving as the broadcast source, thereby bringing convenience to the application of the video conference.
Another embodiment of the present invention further provides a multipoint control unit 400, and referring to fig. 11, the multipoint control unit 400 includes a memory 402, a processor 401 and a computer program stored on the memory 402 and executable on the processor 401, and the computer program implements the control method of the video conference system of any one of the foregoing when executed by the processor 401. In the current multi-level video conference by the control method in the embodiment of the present invention, when the multi-point control unit 400 receives a private chat request initiated by a first video conference terminal to a second video conference terminal, the multi-point control unit 400 creates a sub-conference for the first video conference terminal and the second video conference terminal, controls audio and video data of the first video conference terminal and the second video conference terminal to exchange between the first video conference terminal and the second video conference terminal, and provides audio and video data of a third video conference terminal serving as a broadcast source for the first video conference terminal, thereby implementing that the first video conference terminal and the second video conference terminal establish a private chat sub-conference in the multi-level video conference, and the private audio and video data does not affect the third video conference terminal serving as the broadcast source, which brings convenience to the application of the video conference.
Another embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions, which are executed by a processor 401 or a controller, for example, by a processor 401 in the embodiment of the multipoint control unit 400, so that the processor 401 executes the control method in the embodiment, for example, the method steps S110 to S140 in fig. 2, the method steps S131 to S133 in fig. 3, the method steps S134 to S135 in fig. 4, the method steps S210 to S250 in fig. 6, the method steps S310 to S340 in fig. 8, the method steps S331 to S333 in fig. 9, and the method steps S334 to S335 in fig. 10, which are described above.
One of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
While the preferred embodiments of the present invention have been described in detail, it will be understood by those skilled in the art that the foregoing and various other changes, omissions and deviations in the form and detail thereof may be made without departing from the scope of this invention.
Claims (10)
1. A control method of a video conference system including a first video conference terminal, a second video conference terminal, a third video conference terminal, a first multipoint control unit and a second multipoint control unit, the control method being applied to the first multipoint control unit, the control method comprising:
establishing connection with the first video conference terminal and the third video conference terminal respectively, and receiving code streams from the first video conference terminal and the third video conference terminal;
responding to a private chat request initiated by the first video conference terminal to the second video conference terminal, and acquiring a code stream of the second video conference terminal through the second multipoint control unit, wherein the second video conference terminal is connected to the second multipoint control unit;
sending the code streams of the second video conference terminal and the third video conference terminal to the first video conference terminal;
and transmitting the code stream of the first video conference terminal to the second video conference terminal through the second multipoint control unit.
2. The method of claim 1, wherein when the code stream of the first video conference terminal is transmitted to the second video conference terminal through the second multipoint control unit, the code stream of the third video conference terminal is also transmitted to the second video conference terminal through the second multipoint control unit.
3. The method of controlling a video conference system according to claim 1, wherein the video conference system further comprises a third multipoint control unit;
the establishing connection with the first video conference terminal and the third video conference terminal respectively, and receiving code streams from the first video conference terminal and the third video conference terminal, includes:
establishing connection with the first video conference terminal and establishing connection with the third video conference terminal through the third multipoint control unit;
and receiving the code stream from the first video conference terminal, and receiving the code stream from the third video conference terminal through the third multipoint control unit.
4. The method of claim 1, wherein the sending the code streams of the second video conference terminal and the third video conference terminal to the first video conference terminal comprises:
establishing a first video channel, a second video channel and a third video channel with the first video conference terminal;
and transmitting the code stream of the third video conference terminal to the first video conference terminal through the first path of video channel and/or the second path of video channel, and transmitting the video code stream of the second video conference terminal to the first video conference terminal through the third path of video channel.
5. The method of claim 1, wherein the sending the code streams of the second video conference terminal and the third video conference terminal to the first video conference terminal comprises:
establishing a first video channel with the first video conference terminal;
and synthesizing the pictures of the second video conference terminal and the third video conference terminal, and sending the synthesized pictures to the first video conference terminal through the first video channel.
6. The method according to claim 1, wherein the establishing of the connection with the first video conference terminal and the third video conference terminal respectively comprises:
and establishing an audio receiving channel and an audio sending channel with the first video conference terminal, wherein the audio receiving channel is used for receiving audio data in a code stream of the first video conference terminal, and the audio sending channel is used for sending audio data in a code stream of the third video conference terminal to the first video conference terminal.
7. The method of claim 1, wherein when the connection is established with the second multipoint control unit, further comprising:
setting a forwarding object of the received code stream of the first video conference terminal as the second video conference terminal;
and setting a forwarding object of the received code stream of the second video conference terminal as the first video conference terminal.
8. The method of claim 7, wherein when the connection is established with the second multipoint control unit, further comprising:
and mixing the received voice data in the code stream of the first video conference terminal, the received voice data in the code stream of the second video conference terminal and the received voice data in the code stream of the third video conference terminal.
9. A multipoint control unit, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing a control method of a video conferencing system as claimed in any of claims 1 to 8.
10. A computer-readable storage medium, characterized in that an information processing program is stored thereon, which when executed by a processor implements a control method of a video conference system according to any one of claims 1 to 8.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010564660.8A CN113824919A (en) | 2020-06-19 | 2020-06-19 | Control method of video conference system, multipoint control unit and storage medium |
| PCT/CN2021/100684 WO2021254452A1 (en) | 2020-06-19 | 2021-06-17 | Method for controlling video conference system, and multipoint control unit and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010564660.8A CN113824919A (en) | 2020-06-19 | 2020-06-19 | Control method of video conference system, multipoint control unit and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113824919A true CN113824919A (en) | 2021-12-21 |
Family
ID=78924535
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010564660.8A Pending CN113824919A (en) | 2020-06-19 | 2020-06-19 | Control method of video conference system, multipoint control unit and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113824919A (en) |
| WO (1) | WO2021254452A1 (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5680392A (en) * | 1996-01-16 | 1997-10-21 | General Datacomm, Inc. | Multimedia multipoint telecommunications reservation systems |
| US20060164507A1 (en) * | 2005-01-27 | 2006-07-27 | Noam Eshkoli | Method and system for conducting a sub-videoconference from a main videoconference |
| CN101022481A (en) * | 2007-03-21 | 2007-08-22 | 华为技术有限公司 | Method and device for realizing private conversation in multi-point meeting |
| CN110113555A (en) * | 2018-02-01 | 2019-08-09 | 北京视联动力国际信息技术有限公司 | A kind of video conference processing method and system based on view networking |
| CN110971863A (en) * | 2019-11-21 | 2020-04-07 | 厦门亿联网络技术股份有限公司 | Multi-point control unit cross-area conference operation method, device, equipment and system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080016156A1 (en) * | 2006-07-13 | 2008-01-17 | Sean Miceli | Large Scale Real-Time Presentation of a Network Conference Having a Plurality of Conference Participants |
| CN105323536A (en) * | 2014-07-30 | 2016-02-10 | 三亚中兴软件有限责任公司 | Attendee private chat method and device in television conference |
| CN105472306A (en) * | 2014-08-19 | 2016-04-06 | 中兴通讯股份有限公司 | Video conference data sharing method and related device |
| CN110740286A (en) * | 2018-07-18 | 2020-01-31 | 中兴通讯股份有限公司 | video conference control method, multipoint control unit and video conference terminal |
-
2020
- 2020-06-19 CN CN202010564660.8A patent/CN113824919A/en active Pending
-
2021
- 2021-06-17 WO PCT/CN2021/100684 patent/WO2021254452A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5680392A (en) * | 1996-01-16 | 1997-10-21 | General Datacomm, Inc. | Multimedia multipoint telecommunications reservation systems |
| US20060164507A1 (en) * | 2005-01-27 | 2006-07-27 | Noam Eshkoli | Method and system for conducting a sub-videoconference from a main videoconference |
| CN101022481A (en) * | 2007-03-21 | 2007-08-22 | 华为技术有限公司 | Method and device for realizing private conversation in multi-point meeting |
| CN110113555A (en) * | 2018-02-01 | 2019-08-09 | 北京视联动力国际信息技术有限公司 | A kind of video conference processing method and system based on view networking |
| CN110971863A (en) * | 2019-11-21 | 2020-04-07 | 厦门亿联网络技术股份有限公司 | Multi-point control unit cross-area conference operation method, device, equipment and system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021254452A1 (en) | 2021-12-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9462228B2 (en) | Distributed real-time media composer | |
| US7973859B2 (en) | Apparatus, network device and method for video/audio data transmission | |
| RU2533304C2 (en) | Conference call management method and related device and system | |
| US6603501B1 (en) | Videoconferencing using distributed processing | |
| US20040257433A1 (en) | Method and apparatus for video conferencing having dynamic picture layout | |
| JPH0528558B2 (en) | ||
| WO2012031566A1 (en) | Method and device for adjusting display of attendee images in multi-screen video conference | |
| JP2003530787A (en) | Video conference recordings | |
| WO2011140812A1 (en) | Multi-picture synthesis method and system, and media processing device | |
| TWI451746B (en) | Video conference system and video conference method thereof | |
| WO2012083799A1 (en) | Method, device and system for processing cascade conference sites in cascade conference | |
| CN111601068A (en) | Method for realizing multi-MCU cascade centerless video conference | |
| JP2007318556A (en) | Multipoint video conference system in communication network | |
| KR101918676B1 (en) | Videoconferencing Server for Providing Multi-Screen Videoconferencing by Using Plural Videoconferencing Terminals and Camera Tracking Method therefor | |
| US20050094578A1 (en) | Method and apparatus for conducting a video conference | |
| US9270474B2 (en) | Endpoint initiation of multipart conferences | |
| WO2020258976A1 (en) | Conference recording method and apparatus, and conference recording system | |
| CN113824919A (en) | Control method of video conference system, multipoint control unit and storage medium | |
| WO2014026478A1 (en) | Video conference signal processing method, video conference server and video conference system | |
| CN113726534A (en) | Conference control method, conference control device, electronic equipment and storage medium | |
| JP2010093479A (en) | Conference system and conference terminal device | |
| JP2001036881A (en) | Voice transmission system and voice reproduction device | |
| TWI654880B (en) | The method of holding a multiway video conference | |
| JP2008109170A (en) | Voice level adjusting device of remote conference system | |
| CN118301274A (en) | Video conference access method, device, terminal and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |