WO2016101623A1

WO2016101623A1 - Remote interaction method and device in multipoint audio and video communication

Info

Publication number: WO2016101623A1
Application number: PCT/CN2015/086271
Authority: WO
Inventors: 刘峥; 陆平; 张成华; 李闪阁
Original assignee: 中兴通讯股份有限公司
Priority date: 2014-12-24
Filing date: 2015-08-06
Publication date: 2016-06-30
Also published as: CN105791740A

Abstract

Disclosed are a remote interaction method and device in multipoint audio and video communication. The remote interaction method in multipoint audio and video communication comprises: a multipoint control processing unit in multipoint audio and video communication sends, to each participant terminal, an audio interface and/or a video interface corresponding to current interaction and set to guide user input, each participant terminal supporting a user input channel; a service management system in multipoint audio and video communication performs statistics on interaction information fed back by each participant terminal under the guidance of the audio interface and/or the video interface; and the multipoint control processing unit in multipoint audio and video communication sends a statistics result obtained by the service management system in multipoint audio and video communication to each participant terminal. By means of the present invention, remote interaction is implemented in a multipoint audio and video communication system, so that a third-party device does not need to be introduced, operability is improved, and costs are reduced.

Description

Method and device for remote interaction in multi-point audio and video communication

Technical field

The present invention relates to the field of communications, and in particular to a method and device for remote interaction in multi-point audio video communication.

Background technique

With the rapid development of IP networks and multimedia communication technologies, and the impact of the global concept of environmentally friendly low carbon, users can participate in conferences, decision-making, etc. by remote communication via audio and video multimedia technology without having to visit the site. Remote communication has been widely used in various industries.

Conference TV systems, distance education, and telemedicine are typical applications. 1 is a schematic structural diagram of a multipoint audio video communication system according to the related art. As shown in FIG. 1, a multipoint audio video communication system includes: a multipoint audio video communication service management system 1 and a multipoint audio and video communication multipoint control process. Unit 2. In the system shown in FIG. 1, a multipoint audio and video communication multipoint control processing unit 2 is connected to a plurality of multipoint audio and video communication terminals distributed around the country to realize interaction and communication between the participants.

In real-world applications, such as distance education and distance training, questionnaires, quizzes and other usage scenarios are often used. For some usage scenarios that require users to vote, score, and vote, most of the cases require the user to visit the site, which is a great waste of travel expenses and time.

In order to achieve remote voting, scoring, voting, it is necessary to increase the voting function. 2 is a schematic diagram showing the structure of a multipoint audio and video communication system having a voting function according to the related art. As shown in FIG. 2, a third party voting system is added to the system shown in FIG.

The above schemes for introducing a third-party voting system have certain defects: on the one hand, it is necessary to change the networking of the multi-point audio and video communication system, and matching settings are required in various aspects such as resources and configuration, and the professional requirements are high, and the use thereof is used. Efficiency has an impact; on the other hand, the purchase of a third-party voting system will also increase capital investment, which is not convenient for widespread promotion.

Summary of the invention

The embodiments of the present invention provide a method and a device for remote interaction in multi-point audio and video communication, so as to at least solve the defects caused by the introduction of a third-party interactive system in a multi-point audio and video communication system.

According to an embodiment of the present invention, a remote interaction method in an audio video conference is provided, including: the MCU sends an audio and/or video interface corresponding to the current interaction to direct user input to each participating terminal; Each participating terminal supports the user input channel; the conference service management system collects the interactive information fed back by the participating terminals according to the audio and/or video interface; the MCU sends the statistical result obtained by the conference service management system to each participating terminal.

Before the MCU sends the audio and/or video interface to each participant terminal, the MCU obtains the pre-stored audio file and the image file corresponding to the interaction, and synthesizes the audio file and the image file separately to form a user input. Audio and video interface.

The MCU sends the audio and/or video interface input by the user to each participating terminal, and specifically includes: the MCU acquires the type of information supported by each participating terminal, and sends an audio and/or video interface to each participating terminal based on the type of the information.

The MCU will send the audio and/or video interface input by the user to each participating terminal, and specifically includes: the MCU acquires the type of information supported by each participating terminal, and detects whether each participating terminal sets the type of the received information; If yes, it detects whether the information type set by the corresponding conference terminal belongs to the information type supported by the conference terminal, and if yes, sends an audio or video interface to the corresponding conference terminal based on the set information type; otherwise, according to the information type supported by the conference terminal The participating terminal sends an audio or video interface; if not set, the audio and/or video interface is sent to each participating terminal based on the type of information supported by each participating terminal.

The MCU sends the statistics to each participant terminal, including: the MCU synthesizes the statistical result into text, audio, and video, and sends text, audio, and/or video to each participating terminal based on the type of information previously supported by each participating terminal. Statistical results.

The manner in which the MCU obtains the information types supported by the participating terminals includes: the MCU acquires a logical channel established for each participating terminal, and determines the type of information supported by the participating terminal according to the type of the logical channel; the types of the logical channel include: audio, video, and data.

The MCU will direct the audio and/or video interface input by the user to each participating terminal, and further includes: the MCU sets the audio and/or video interface set to direct the user input together with the data, audio and video streams in the current conference as The media stream format is sent to each participating terminal; the MCU sends the statistics to each participant terminal, and further includes: the MCU encodes the statistical result together with the data, audio, and video streams in the current conference into a media stream format, and then sends the result to each participant. terminal.

The types of interaction include: voting, scoring, and voting.

According to another embodiment of the present invention, there is provided a multipoint control unit MCU, comprising: an interaction initiation module, configured to set, in a video conference, an audio and/or video corresponding to the current interaction to guide user input. The interface is sent to each participating terminal; wherein each participating terminal supports a user input channel; the interactive processing module is configured to receive interactive information fed back by each participating terminal according to the audio and/or video interface, and send the interactive information to the meeting. The business management system, and the statistical result obtained by the conference service management system using the interactive information, and the statistical result is sent to each participating terminal.

The interaction initiation module is further configured to obtain a pre-stored audio file and a picture file corresponding to the interaction, and separately synthesize the audio file and the picture file to form an audio and video interface for guiding user input.

The interaction initiation module is specifically configured to obtain an information type supported by each conference terminal, and send an audio and/or video interface to each conference terminal based on the information type.

The interaction initiation module is specifically configured to obtain the type of information supported by each conference terminal, and detect whether each conference terminal sets the type of the received information. If the setting is performed, it is detected whether the information type set by the corresponding conference terminal belongs to the The type of information supported, if yes, sending an audio or video interface to the corresponding conference terminal based on the set information type; otherwise, sending an audio or video interface to the corresponding conference terminal based on the type of information supported by the conference terminal; if not, An audio and/or video interface is sent to each participating terminal based on the type of information supported by each participating terminal.

The interactive processing module is further configured to synthesize the statistical result into text, audio, and video, and send the statistical result of the text, audio, and/or video to each participating terminal based on the type of information supported by each of the participating terminals.

The interaction initiation module is further configured to encode the audio and/or video interface set to direct the user input together with the data, audio and video code streams in the current conference as a media stream format, and then send the format to the conference terminal; the interaction processing module further It is set to encode the statistical result together with the data, audio and video streams in the current conference into the media stream format and then send it to each participating terminal.

According to still another embodiment of the present invention, a remote interaction system in an audio video conference is provided, including: a conference service management system, a conference terminal, and the above-mentioned MCU; the conference terminal is set to receive the data transmitted by the MCU. After the interaction is set to guide the user to input the audio and/or video interface, the interactive operation is performed according to the instructions of the audio and/or video interface, and the interactive information obtained by the operation is sent to the MCU; the conference service management system is set to receive The interactive information sent by each participating terminal sent by the MCU according to the guidance of the audio and/or video interface, and the statistical information is collected, and the statistical result is sent to the MCU.

According to still another embodiment of the present invention, a remote interaction method in multi-point audio video communication is provided, including: a multi-point audio video communication multi-point control processing unit sets the audio corresponding to the current interaction to be set to guide the user input. And/or the video interface is sent to each participating terminal; wherein each participating terminal supports the user input channel; the multi-point audio and video communication multi-point control processing unit receives the interactive information fed back by each participating terminal according to the instruction of the audio and/or video interface; The multi-point audio and video communication multi-point control processing unit sends the interactive information to the multi-point audio video communication service management system; the multi-point audio and video communication multi-point control processing unit receives the statistical data obtained by the multi-point audio and video communication service management system using the interactive information. As a result, the multipoint audio video communication multipoint control processing unit transmits the statistical result to each participating terminal.

The multi-point audio video communication multi-point control processing unit sends the audio and/or video interface to each participating terminal, and further includes: a multi-point audio video communication multi-point control processing unit acquires a pre-stored audio file corresponding to the interaction and The image file, and the audio file and the image file are separately synthesized to form an audio and video interface for guiding user input.

The multi-point audio and video communication multi-point control processing unit sends the audio and/or video interface that directs the user input to each participating terminal, including: the multi-point audio video communication multi-point control processing unit acquires the type of information supported by each participating terminal; The audio video multipoint control processing unit transmits an audio and/or video interface each guiding the user input to each participating terminal based on the type of information supported by each participating terminal.

The multi-point audio and video communication multi-point control processing unit sends an audio and/or video interface guiding the user input to each participating terminal, including: a multi-point audio video communication multi-point control processing unit acquires information types supported by each participating terminal, and detects Whether each participating terminal sets the type of the received information; when the participating terminal sets the type of the received information, it detects whether the type of information set by the corresponding participating terminal belongs to the type of information it supports, and if so, based on the type of the set information Sending an audio and/or video interface to the corresponding conference terminal; if not, transmitting an audio and/or video interface to the corresponding conference terminal based on the type of information supported by the conference terminal; and/or when the conference terminal does not set the type of the reception information, based on The type of information supported by each participating terminal sends an audio and/or video interface to each participating terminal.

The multi-point audio and video communication multi-point control processing unit sends the statistical result to each participating terminal, including: a multi-point audio video communication multi-point control processing unit synthesizes the statistical result into text, audio and video, and is based on the previously acquired each participating terminal The type of information supported, sending statistical results of text, audio and/or video to each participating terminal.

The multi-point audio and video communication multi-point control processing unit acquires the information types supported by the participating terminals, including: the multi-point audio video communication multi-point control processing unit acquires a logical channel established for each participating terminal, and determines the participating terminal according to the type of the logical channel. Types of information supported; types of logical channels include: audio, video, and data.

The multipoint audio video communication multipoint control processing unit transmits an audio and/or video interface directing user input to each of the participating terminals, including: a multipoint audio video communication multipoint control processing unit will be set to direct user input audio and/or The video interface is encoded into the media stream format together with the data, audio and video code streams in the current multi-point audio and video communication, and then sent to each participating terminal; the multi-point audio and video communication multi-point control processing unit sends the statistical result to each participating terminal. The multi-point audio and video communication multi-point control processing unit encodes the statistical result together with the data, audio and video code streams in the current conference into a media stream format, and then sends the result to each participating terminal.

The types of interaction include: voting, scoring, and voting.

The multipoint audio video communication multipoint control processing unit includes a multipoint control unit MCU.

According to still another embodiment of the present invention, a multipoint audio video communication multipoint control processing unit is provided, including: an interaction initiation module, configured to set a guide corresponding to the interaction in the multipoint audio video communication as a guide The audio and/or video interface input by the user is sent to each participating terminal; wherein each participating terminal supports a user input channel; the interactive processing module is configured to receive interactive information fed back by each participating terminal according to the audio and/or video interface guidelines. And transmitting the interactive information to the multi-point audio video communication service management system, and receiving the statistical result obtained by the conference service management system using the interactive information, and transmitting the statistical result to each participating terminal.

The interaction initiation module is configured to obtain an information type supported by each participant terminal, and send an audio and/or video interface to each participant terminal based on the information type.

The interaction initiation module is configured to obtain the type of information supported by each conference terminal, and detect whether each conference terminal sets the type of the received information. If the setting is performed, it is detected whether the information type set by the corresponding conference terminal belongs to the support. The information type, if yes, sends an audio or video interface to the corresponding conference terminal based on the set information type; otherwise, the audio or video interface is sent to the corresponding conference terminal based on the type of information supported by the conference terminal; if not, the The type of information supported by each participating terminal sends an audio and/or video interface to each participating terminal.

The interaction initiation module is further configured to encode the audio and/or video interface set to direct the user input together with the data, audio and video code streams in the current conference as a media stream format, and then send the format to the conference terminal; The management module is further configured to encode the statistical result together with the data, audio and video code streams in the current conference into a media stream format, and then send the result to each conference terminal.

The multipoint audio video communication includes a voice video conference; the multipoint audio video communication multipoint control processing unit includes a multipoint control unit MCU.

Through the embodiments of the present invention, remote interaction is implemented in a multi-point audio video communication system, thereby eliminating the need to introduce third-party devices, improving operability and reducing costs.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a schematic structural diagram of a multipoint audio video communication system according to the related art;

2 is a schematic structural diagram of a multipoint audio video communication system with a voting function according to the related art;

3 is a flow chart of a method for remote interaction in multi-point audio video communication according to Embodiment 1 of the present invention;

4 is a structural block diagram of a multipoint audio video communication multipoint control processing unit according to Embodiment 1 of the present invention;

5 is a flowchart of a method for remote interaction in another multi-point audio video communication according to Embodiment 1 of the present invention;

6 is a structural block diagram of a multipoint audio video communication service management system according to Embodiment 1 of the present invention;

7 is a flowchart of a method for remote interaction in multi-point audio video communication according to Embodiment 2 of the present invention;

8 is a structural block diagram of a multipoint audio video communication multipoint control processing unit according to Embodiment 2 of the present invention;

9 is a flowchart of a method for remote interaction in multi-point audio video communication according to an alternative embodiment 1 of the present invention;

10 is a flowchart of a method for remote interaction in multi-point audio video communication according to an alternative embodiment 2 of the present invention;

11 is a structural block diagram of a multipoint audio video communication multipoint control processing unit according to an alternative embodiment 3 of the embodiment of the present invention;

FIG. 12 is a schematic structural diagram of a remote interaction system in distance education according to an alternative embodiment 4 of the embodiment of the present invention.

detailed description

The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

Embodiment 1

In the embodiment, a method for remote interaction in multi-point audio video communication is provided. FIG. 3 is a flowchart of a method for remote interaction in multi-point audio video communication according to the first embodiment of the present invention, as shown in FIG. The process includes the following steps:

Step S302, the multipoint audio video communication multipoint control processing unit sends the audio and/or video data carrying the first interactive information to the multipoint audio video communication terminal;

Step S304, the multi-point audio video communication multi-point control processing unit receives the audio and/or video data that carries the second interaction information that is sent by the multi-point audio video communication terminal according to the first interaction information;

Step S306, the multipoint audio video communication multipoint control processing unit processes the second interactive information.

In an optional implementation manner of the embodiment of the present invention, the multi-point audio and video communication multi-point control processing unit processing the second interaction information in the step S306 may include: multi-point audio and video communication multi-point control processing unit to multi-point audio video The communication service management system sends the second interaction information, and receives a processing result sent by the multi-point audio video communication service management system according to the second interaction information. With this alternative embodiment, the processing load of the multipoint audio video communication multipoint control processing unit is reduced.

In an optional implementation manner of the embodiment of the present invention, the multi-point audio video communication multi-point control processing unit receives the processing result sent by the multi-point audio video communication service management system according to the second interactive information, and may also provide multi-point audio video. The communication terminal transmits audio and/or video data carrying the above processing result.

In an optional implementation manner of the embodiment of the present invention, the multi-point audio video communication multi-point control processing unit sends the audio and/or video data carrying the first interaction information to the multi-point audio video communication terminal, including: multi-point audio video. The communication multi-point control processing unit acquires the first interaction information, and the first interaction information and the voice in the conference and/or Or video stream encoding, obtaining audio and/or video data carrying the first interactive information, and transmitting the audio and/or video data carrying the first interactive information to the multi-point audio video communication terminal. With this alternative embodiment, the information and the interactive information will be encoded and sent to the multi-point audio video communication terminal.

In an optional implementation manner of the embodiment of the present invention, the multipoint audio video communication multipoint control processing unit sends the audio and/or video data of the bearer processing result to the multipoint audio video communication terminal, including: multipoint audio video communication. The multipoint control processing unit encodes the above processing result and the voice and/or video code stream in the conference to obtain audio and/or video data carrying the processing result, and transmits the audio of the bearer processing result to the multipoint audio video communication terminal and/or Or video data.

In the embodiment, a multi-point audio and video communication multi-point control processing unit is further provided, and the unit is configured to implement the above-mentioned embodiments and preferred embodiments, and the detailed description thereof has been omitted. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.

4 is a structural block diagram of a multipoint audio video communication multipoint control processing unit according to Embodiment 1 of the present invention. As shown in FIG. 4, the multipoint audio video communication multipoint control processing unit includes: a first sending module 410, setting The receiving module 420 is connected to the first sending module 410 and configured to receive the multi-point audio and video communication terminal according to the first interactive information, to send the audio and/or video data that carries the first interactive information to the multi-point audio and video communication terminal. The audio and/or video data carrying the second interactive information; the processing module 430 is connected to the receiving module 420 and configured to process the second interactive information.

In an optional implementation manner of the embodiment of the present invention, the processing module 430 may include: a sending unit, configured to send second interaction information to the multi-point audio video communication service management system; and the receiving unit is connected to the sending unit, and is configured to Receiving a processing result sent by the multi-point audio video communication service management system according to the second interactive information.

In an optional implementation manner of the embodiment of the present invention, the multi-point audio video communication multi-point control processing unit further includes: a second sending module, configured to send the audio carrying the processing result to the multi-point audio video communication terminal and/or Video data.

In an embodiment of the present invention, the first sending module 410 may include: an acquiring unit, configured to acquire first interaction information; and the first encoding unit is connected to the acquiring unit, and is configured to set the first interaction information and The voice and/or video code stream in the conference is encoded to obtain audio and/or video data carrying the first interaction information; the first sending unit is connected to the first coding unit and configured to send the bearer to the multi-point audio video communication terminal. An interactive audio and/or video data.

In an optional implementation manner of the embodiment of the present invention, the foregoing second sending module may include: a second encoding unit configured to encode the processing result and the voice and/or video code stream in the conference to obtain an audio carrying the processing result. And/or video data; the second transmitting unit is connected to the second encoding unit and configured to transmit the audio and/or video data carrying the processing result to the multi-point audio video communication terminal.

In this embodiment, another method for remote interaction in multi-point audio video communication is also provided. FIG. 5 is a flowchart of a method for remote interaction in another multi-point audio video communication according to the first embodiment of the present invention. As shown in 5, the process includes the following steps:

Step S502, the multipoint audio video communication service management system instructs the multipoint audio video communication multipoint control processing unit to send the first interaction information to the multipoint audio video communication terminal;

Step S504, the multi-point audio video communication service management system receives the second interaction information that is sent by the multi-point audio and video communication terminal sent by the multi-point audio and video communication multi-point control processing unit according to the first interaction information;

Step S506, the multi-point audio video communication service management system processes the second interaction information.

In an optional implementation manner of the embodiment of the present invention, after the multi-point audio video communication service management system processes the second interaction information, the multi-point audio video communication service management system may further indicate multi-point audio and video communication. The point control processing unit transmits the processing result to the multipoint audio video communication terminal.

In an optional implementation manner of the embodiment of the present invention, the first interaction information includes voting information and/or scoring information; the second interaction information includes: voting selection and/or scoring score; and multi-point audio video in step S506 The processing of the second interactive information by the communication service management system may include: the multi-point audio video communication service management system performs statistical analysis on the voting selection and/or the scoring score to obtain a voting result and/or a scoring result.

In this embodiment, a multi-point audio and video communication service management system is further provided. The unit is configured to implement the foregoing embodiments and preferred embodiments, and details are not described herein. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.

FIG. 6 is a structural block diagram of a multipoint audio video communication service management system according to Embodiment 1 of the present invention. As shown in FIG. 6, the system includes: a first indication module 610, configured to indicate multipoint audio and video communication multipoint control processing. The unit sends the first interaction information to the multi-point audio and video communication terminal; the receiving module 620 is connected to the first indication module 610, and is configured to receive the multi-point audio and video communication multi-point control processing unit to send the multi-point audio and video communication terminal according to the first The second interactive information of the interactive information feedback; the processing module 630 is connected to the receiving module 620 and configured to process the second interactive information.

In an optional implementation manner of the embodiment of the present invention, the multi-point audio video communication service management system further includes: a second indication module, configured to instruct the multi-point audio and video communication multi-point control processing unit to the multi-point audio and video communication terminal Send the processing result.

In an optional implementation manner of the embodiment of the present invention, the first interaction information includes voting information and/or scoring information; the second interaction information includes at least one of: voting selection and/or scoring score; the processing module is configured to The multi-point audio video communication service management system performs statistical analysis on voting selection and/or scoring scores to obtain voting results and/or scoring results.

Embodiment 2

In this embodiment, a method for remote interaction in multi-point audio video communication is provided. FIG. 7 is a flowchart of a method for remote interaction in multi-point audio video communication according to Embodiment 2 of the present invention, as shown in FIG. The process includes the following steps:

Step S702, the multi-point audio video communication multi-point control processing unit sends an audio and/or video interface corresponding to the current interaction, which is set to guide the user input, to each participating terminal; wherein each participating terminal supports the user input channel;

Step S704, the multi-point audio video communication multi-point control processing unit receives the interaction information fed back by each participating terminal according to the instruction of the audio and/or video interface;

Step S706, the multi-point audio video communication multi-point control processing unit sends the interaction information to the multi-point audio video communication service management system;

Step S708, the multipoint audio and video communication multipoint control processing unit receives the statistical result obtained by using the interactive information by the multipoint audio and video communication service management system;

Step S710, the multi-point audio video communication multi-point control processing unit sends the statistical result to each participating terminal.

In an implementation manner of the embodiment of the present invention, the multi-point audio video communication multi-point control processing unit sends the audio and/or video interface to each participating terminal, and further includes: multi-point audio video communication multi-point control processing unit acquiring The audio file and the image file corresponding to the current interaction are pre-stored, and the audio file and the image file are separately synthesized to form an audio and video interface for guiding user input.

In an implementation manner of the embodiment of the present invention, the multi-point audio video communication multi-point control processing unit sends an audio and/or video interface that directs user input to each participating terminal, including: multi-point audio and video communication. The control processing unit acquires the type of information supported by each participating terminal; the multi-point audio and video multi-point control processing unit transmits the audio and/or video interfaces each guiding the user input to each participating terminal based on the type of information supported by each participating terminal.

In an implementation manner of the embodiment of the present invention, the multi-point audio video communication multi-point control processing unit sends an audio and/or video interface that directs user input to each participating terminal, including: multi-point audio video communication multi-point control processing. The unit acquires the type of information supported by each participating terminal, and detects whether each participating terminal sets the type of the received information; when the participating terminal sets the type of the received information, it detects whether the type of information set by the corresponding participating terminal belongs to its support. The type of information, if yes, sending an audio and/or video interface to the corresponding participant terminal based on the set information type; if not, transmitting an audio and/or video interface to the corresponding participant terminal based on the type of information supported by the participant terminal; and/or when When the participating terminal does not set the type of the received information, the audio and/or video interface is transmitted to each participating terminal based on the type of information supported by each participating terminal.

In an implementation manner of the embodiment of the present invention, the multi-point audio video communication multi-point control processing unit sends the statistical result to each participating terminal, including: a multi-point audio video communication multi-point control processing unit synthesizes the statistical result into text and audio. And video, and based on the type of information supported by the respective participating terminals, the statistical results of text, audio and/or video are sent to the participating terminals.

In an implementation manner of the embodiment of the present invention, the multi-point audio video communication multi-point control processing unit acquires the information type supported by each participating terminal, including: multi-point audio video communication multi-point control processing unit acquisition is established for each participating terminal The logical channel determines the type of information supported by the participant terminal according to the type of the logical channel; the types of the logical channel include: audio, video, and data.

In an implementation manner of the embodiment of the present invention, the multi-point audio video communication multi-point control processing unit sends an audio and/or video interface that directs user input to each participating terminal, including: multi-point audio video communication multi-point control processing. The unit is configured to direct the audio and/or video interface input by the user to be encoded into the media stream format together with the data, audio and video code streams in the current multi-point audio video communication, and then sent to each participating terminal; multi-point audio and video communication is multi-point The control processing unit sends the statistical result to each participating terminal, including: the multi-point audio video communication multi-point control processing unit encodes the statistical result together with the data, audio and video code streams in the current conference into a media stream format, and then sends the result to each participant. terminal.

In an implementation manner of an embodiment of the present invention, the types of interaction include: voting, scoring, and voting.

In an implementation manner of the embodiment of the present invention, the multipoint audio video communication includes an audio video conference; the multipoint audio video communication multipoint control processing unit includes a multipoint control unit MCU.

8 is a structural block diagram of a multipoint audio video communication multipoint control processing unit according to Embodiment 2 of the present invention. As shown in FIG. 8, the method includes: an interaction initiation module 810, which is configured to be in a multipoint audio and video communication. The interaction corresponding to the secondary interaction is to send the audio and/or video interface input by the user to each participating terminal; wherein each participating terminal supports the user input channel; the interaction processing module 820 is configured to receive the respective terminal according to the audio and/or video. The interactive information fed back by the interface, and the interactive information is sent to the multi-point audio and video communication service management system, and the statistical result obtained by the conference service management system using the interactive information is collected, and the statistical result is sent to each participating terminal.

In an optional implementation manner of the embodiment of the present invention, the interaction initiation module 810 is further configured to acquire a pre-stored audio file and a picture file corresponding to the current interaction, and separately synthesize the audio file and the image file for formation. To guide the user input audio and video interface.

In an optional implementation manner of the embodiment of the present invention, the interaction initiation module 810 is configured to obtain an information type supported by each participant terminal, and send an audio and/or video interface to each participant terminal based on the information type.

In an optional implementation manner of the embodiment of the present invention, the interaction initiation module 810 is configured to acquire information types supported by the conference terminals, and detect whether each conference terminal sets the type of the received information, and if configured, Then, it is detected whether the information type set by the corresponding conference terminal belongs to the information type supported by the conference terminal, and if yes, the audio or video interface is sent to the corresponding conference terminal based on the set information type; otherwise, the information type supported by the conference terminal is used to correspond to the conference terminal. Send an audio or video interface; if not set, send an audio and/or video interface to each participating terminal based on the type of information supported by each participating terminal.

In an optional implementation manner of the embodiment of the present invention, the interaction processing module 820 is further configured to synthesize the statistical result into text, audio, and video, and send the message to each participating terminal based on the type of information supported by each participating terminal. Statistical results for text, audio, and/or video.

In an optional implementation manner of the embodiment of the present invention, the interaction initiation module 810 is further configured to encode the audio and/or video interface set to direct user input with the data, audio, and video code streams in the current conference as the media. The stream format is sent to each participant terminal. The interaction processing module 820 is further configured to encode the statistical result together with the data, audio and video code streams in the current conference into a media stream format, and then send the result to each conference terminal.

In an optional implementation manner of the embodiment of the present invention, the multipoint audio video communication includes a voice video conference; the multipoint audio video communication multipoint control processing unit includes a multipoint control unit MCU.

In an embodiment of the invention, multi-point audio video communication may include various types of communication, such as video conferencing, voice conferencing, etc., but is not limited thereto. The following describes an optional implementation manner of the embodiment of the present invention by taking an audio and video conference as an example.

Alternative embodiment 1

In this alternative embodiment, an interaction method in an audio video conference is provided. In the optional implementation manner, the audio and/or video interface input by the user is used as an example to describe an example. As shown in FIG. 9, the method includes the following steps:

Step S902, the multi-point audio and video communication multi-point control processing unit sends an audio and/or video interface corresponding to the current interaction, which is set to guide the user input, to each of the participating multi-point audio and video communication terminals;

In this step, the types of interactions mentioned above include, but are not limited to, voting, scoring, and voting.

In this step, each of the multi-point audio and video communication terminals supports the user input channel; specifically, the multi-point audio and video communication multi-point control processing unit opens the user input channel for each participating terminal when the participating terminals join the conference, so that Each participating terminal supports the user input channel, thereby providing technical support for the secondary input of the end user.

In this step, the manner of obtaining the audio and/or video interface that is set to direct the user input includes: the multi-point audio video communication multi-point control processing unit acquires the pre-stored audio file and the image file corresponding to the current interaction, and The audio file and the image file are separately synthesized to form an audio and video interface for guiding user input.

In this step, the multi-point audio and video communication multi-point control processing unit delivers audio, video or audio and video to each participating terminal, which is determined by the type of information supported by the participating terminal. For example, if the participating terminal only supports audio or video, Then send an audio or video interface to it, and if the participating terminal supports audio and video, send an audio and video interface to it. The manner in which the multi-point audio and video communication multi-point control processing unit acquires the information types supported by the participating terminals includes, but is not limited to, multi-point audio video communication multi-point control processing unit acquires a logical channel established for each participating terminal, according to logic The type of channel determines the type of information supported by the participating terminal; the types of the logical channel include: audio, video, and data.

Or, the user's will is taken into consideration, and the implementation manner is as follows: the multi-point audio and video communication multi-point control processing unit acquires the information types supported by the participating terminals, and detects whether each participating terminal sets the type of the received information;

If the setting is made, it is detected whether the information type set by the corresponding conference terminal belongs to the information type supported by the conference terminal, and if yes, the audio or video interface is sent to the corresponding conference terminal based on the set information type; otherwise, based on the conference terminal support The information type sends an audio or video interface to the corresponding conference terminal;

If no setting is made, an audio and/or video interface is transmitted to each participating terminal based on the type of information supported by each participating terminal.

Step S904, the multi-point audio video communication service management system collects interactive information that each participating terminal feeds back according to the guidance of the audio and/or video interface;

Step S906, the multipoint audio video communication multipoint control processing unit sends the statistical result obtained by the multipoint audio video communication service management system to each participating terminal.

In this step, the multi-point audio video communication multi-point control processing unit synthesizes the statistical result into text, audio and video, and sends text, audio and/or video to each participating terminal based on the type of information supported by each participating terminal. Statistical results.

In this optional implementation manner, in order to effectively combine the interaction process with the existing conference process, the network resources are saved, and the information of the interaction between the multipoint audio and video communication multipoint control processing unit and each participating terminal and the data in the conference, The audio and video streams are encoded together into a media stream format for interaction. The specific performance is:

The multi-point audio video communication multi-point control processing unit encodes the audio and/or video interface set to direct the user input together with the data, audio and video code streams in the current conference into a media stream format and transmits them to each participating terminal.

The multi-point audio and video communication multi-point control processing unit encodes the statistical result together with the data, audio and video code streams in the current conference into a media stream format, and then transmits the result to each participating terminal.

In summary, it can be seen that, in the optional implementation manner, on the basis of not changing the conference system of the user and changing the original usage habits of the user to the conference system, the conference end users can vote as if they are on the scene. An interactive experience of scoring, voting, or voting.

In this optional implementation, the multipoint audio video communication may be a television video conference, and the multipoint audio video communication multipoint control processing unit may be an MCU.

Optional implementation method 2

In this optional implementation, an interaction method in an audio video conference is provided. As shown in FIG. 10, the method includes the following steps:

In step S1002, the conference administrator selects a terminal list in the multi-point audio video communication service management system to hold a video conference. The multi-point audio video communication service management system notification protocol and the channel control module hold a video conference, and call each participating terminal to join the conference.

Among them, the user input channel (UserInput) is supported during the capability negotiation and logical channel opening process. By supporting the user input channel, the user provides the necessary technical support according to the instruction input during the interaction.

In step S1004, the service management system judges whether the voting, scoring, or voting process needs to be enabled according to the needs of the meeting site. If not, the meeting maintains the normal mode to continue, and if necessary, enters the voting, scoring, or voting process, and proceeds to step S1006. .

Step S1006: The multi-point audio video communication multi-point control processing unit separately synthesizes the already saved audio file and the picture file to form an audio and video interface (ie, a user interface UI of the video) that prompts the user to input.

Step S1008: The multipoint audio video communication multipoint control processing unit encodes the audio and video interface together with the data, audio and video code streams in the current conference into a media stream format.

Step S1010: The multipoint audio video communication multipoint control processing unit sends the audio and video code streams to the conference terminal;

For audio conferences or terminals that only support audio capabilities, the video channel will not be opened in step S1002, and therefore the video media stream will not be sent to the corresponding conference terminal.

In step S1012, the participating terminal inputs its own voting, scoring or voting data and submits it according to the guidance of the received audio and/or video interface, using a remote controller, a keyboard, or the like.

Among them, the operation according to the guidance of the received audio and/or video interface is an interactive audio video response (IVR or IVVR) technology.

In step S1014, the multipoint audio and video communication multipoint control processing unit receives the voting, scoring or voting data submitted by all the participating terminals, and submits the data to the multipoint audio and video communication service management system for decision.

In step S1016, the multi-point audio video communication service management system aggregates all the voting, scoring or voting data, makes a decision according to the user's will, and sends the summary result to the multi-point audio and video communication multi-point control processing unit.

In step S1018, the multi-point audio video communication multi-point control processing unit sends the result of voting, scoring or voting to the participating terminal by synthesizing text, audio and video. Among them, for audio conferences and terminals that only support audio capabilities, only audio is sent to the corresponding conference terminal to inform the result.

In step S1020, after the voting, scoring or voting process ends, the conference continues in the normal mode. As long as the voting process needs to be performed in the conference, the process proceeds to the above process until the conference ends.

In summary, it can be seen that the solution according to the embodiment of the present invention is based on the original networking of the audio video conferencing system, and helps the user to vote and score in the scene of multimedia remote field access such as conference, interview, and remote interaction. And express their own opinions and realize the interaction in audio and video conferences.

The solution in the embodiment of the present invention is simple to implement, and does not need to change the current conference system networking of the user, and does not need to add a traditional third-party voting system, and does not need to manually count and vote for the members in the conference. As a result of the scoring, the system uses interactive audio and video response (IVR, IVVR) to allow users to easily vote and score. The system displays the automatic statistics and calculation results and presents them to the user in multimedia mode.

Alternative embodiment three

An embodiment of the present invention provides a multipoint audio and video communication multipoint control processing unit, as shown in FIG.

The interaction initiation module 1110 is configured to, in the multi-point audio video communication, send an audio and/or video interface corresponding to the current interaction to guide the user input to each communication terminal; wherein each communication terminal supports the user input channel ;

The interaction processing module 1120 is configured to receive interaction information that each communication terminal feeds back according to the instruction of the audio and/or video interface, and send the interaction information to the multi-point audio video communication service management system, and receive the multi-point audio video communication service. The management system uses the interaction information to perform statistics and obtains the statistical result, and sends the statistical result to each communication terminal.

Based on the above structural framework and implementation principle, several specific and preferred embodiments under the above structure are given below to refine and optimize the functions of the multipoint audio and video communication multipoint control processing unit of the present invention, specifically the following content:

The interaction initiation module 1110 forms an audio and video interface for guiding user input by acquiring an audio file and a picture file corresponding to the current interaction and synthesizing the audio file and the picture file respectively.

The interaction initiation module 1110 acquires an information type supported by each communication terminal, and sends an audio and/or video interface to each communication terminal based on the information type; or acquires an information type supported by each communication terminal, and detects whether each communication terminal receives the information. Setting the type of information, if it is set, detecting whether the type of information set by the corresponding communication terminal belongs to the type of information it supports, and if so, sending an audio or video interface to the corresponding communication terminal based on the set information type; Otherwise, an audio or video interface is sent to the corresponding communication terminal based on the type of information supported by the communication terminal; if not, the audio and/or video interface is transmitted to each communication terminal based on the type of information supported by each communication terminal.

The interaction processing module 1120 synthesizes the statistical result into text, audio, and video, and transmits the statistical result of the text, audio, and/or video to each communication terminal based on the type of information supported by each of the previously acquired communication terminals.

Optionally, in this embodiment, in order to effectively combine the interaction process with the existing conference process, the network resource is saved, and the information of the multipoint audio and video communication multipoint control processing unit and each communication terminal and the data in the conference are exchanged. The audio and video streams are encoded together into a media stream format for interaction. The specific performance is:

The interaction initiation module 1110, the audio and/or video interface set to direct the user input is encoded into the media stream format together with the data, audio and video code streams in the current communication, and then sent to each communication terminal;

The interaction processing module 1120 encodes the statistical result together with the data, audio and video code streams in the current communication into a media stream format, and then sends the result to each communication terminal.

In summary, it can be seen that the multi-point audio and video communication multi-point control processing unit in the embodiment does not change the user's conference system networking, and does not change the user's original usage habits of the conference system. The multi-point audio and video communication multi-point control processing unit described can enable the communication terminal user to have an interactive experience of voting, scoring or voting as if it were on-site.

Optional implementation four

The embodiment of the present invention provides a remote interaction system in distance education, as shown in FIG. 12, including: a multi-point audio and video communication service management system 1210, a multi-point audio and video communication terminal 1220, and multiple points of the optional embodiment 3. Audio and video communication multipoint control processing unit 1230; wherein:

The remote multi-point audio video communication terminal 1220 is configured to, after receiving the audio and/or video interface corresponding to the current interaction sent by the multi-point audio video communication multi-point control processing unit 1230 to guide the user input, according to the audio and / or the interface of the video interface to perform an interactive operation, and the interactive information obtained by the operation is sent to the multi-point audio video communication multi-point control processing unit 1230;

The multi-point audio video communication service management system 1210 is configured to receive the interaction information that each remote terminal sends back according to the guidance of the audio and/or video interface sent by the multi-point audio and video communication multi-point control processing unit 630, and the interaction information is After the statistics are performed, the statistical results are sent to the multipoint audio video communication multipoint control processing unit 1230.

Further, since the specific composition and implementation principle of the multi-point audio video communication multi-point control processing unit have been described in detail in the second embodiment, the structure and function of the multi-point audio and video communication multi-point control processing unit are not described in detail in this embodiment.

In summary, it can be seen that the system in this embodiment does not change the conference system of the user, does not change the original usage habits of the user to the conference system, and uses the multipoint audio and video communication described in this embodiment. The control processing unit enables the participating end users to have an interactive experience of voting, scoring or voting as if they were on the spot.

It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Industrial applicability

As described above, the method and device for remote interaction in multi-point audio and video communication provided by the embodiments of the present invention have the following beneficial effects: solving the related art, introducing a third-party interactive system to implement an interaction in a multi-point audio and video communication system The resulting defect enables remote interaction in a multi-point audio video communication system, eliminating the need to introduce third-party devices, increasing operability and reducing costs.

Claims

A remote interaction method in an audio video conference, including:

The multi-point control unit MCU sends an audio and/or video interface corresponding to the current interaction to guide the user input to each participating terminal; wherein each participating terminal supports the user input channel;

The conference business management system collects interactive information that each participating terminal feeds back according to the guidance of the audio and/or video interface;

The MCU sends the statistical results obtained by the conference service management system to each participating terminal.
The method of claim 1, wherein the MCU sends the audio and/or video interface to each of the participating terminals before:

The MCU obtains the pre-stored audio file and picture file corresponding to the interaction, and synthesizes the audio file and the picture file separately to form an audio and video interface for guiding user input.
The method of claim 1, wherein the MCU sends an audio and/or video interface that directs user input to each of the participating terminals, including:

The MCU obtains the type of information supported by each participating terminal, and sends an audio and/or video interface to each participating terminal based on the type of the information.
The method of claim 1, wherein the MCU sends an audio and/or video interface that directs user input to each of the participating terminals, including:

The MCU obtains the type of information supported by each participating terminal, and detects whether each participating terminal sets the type of the received information;

If the setting is made, it is detected whether the information type set by the corresponding conference terminal belongs to the information type supported by the conference terminal, and if yes, the audio or video interface is sent to the corresponding conference terminal based on the set information type; otherwise, based on the conference terminal support The information type sends an audio or video interface to the corresponding conference terminal;

If no setting is made, an audio and/or video interface is transmitted to each participating terminal based on the type of information supported by each participating terminal.
The method of claim 1, wherein the MCU sends the statistical result to each participant terminal, specifically:

The MCU synthesizes the statistical results into text, audio, and video, and sends statistical results of text, audio, and/or video to each participating terminal based on the type of information previously supported by each participating terminal.
The method according to claim 3 or 4 or 5, wherein the manner in which the MCU obtains the type of information supported by each participating terminal comprises:

The MCU obtains a logical channel established for each participating terminal, and determines the type of information supported by the participating terminal according to the type of the logical channel; the types of the logical channel include: audio, video, and data.
The method according to any one of claims 1 to 5, wherein

The MCU sends an audio and/or video interface directed to the user to each participating terminal, further comprising: the MCU will use the audio and/or video interface for directing user input along with the data, audio and video streams in the current conference. Encoded into a media stream format and sent to each participating terminal;

The MCU sends the statistical result to each participant terminal, and further includes: the MCU encodes the statistical result together with the data, audio, and video code streams in the current conference into a media stream format, and then sends the result to the conference terminal.
The method of any one of claims 1 to 5, wherein the type of interaction comprises: voting, scoring, and voting.
A multipoint control unit MCU comprising:

The interactive initiating module is configured to send, in the video conference, an audio and/or video interface corresponding to the current interaction, which is set to guide the user input, to each participating terminal; wherein each participating terminal supports the user input channel;

The interaction processing module is configured to receive interaction information fed back by each participating terminal according to the instruction of the audio and/or video interface, and send the interaction information to the conference service management system, and receive the conference service management system to use the interaction information to perform The statistical result obtained by the statistics is sent to each participating terminal.
The MCU of claim 9, wherein the interaction initiation module is further configured to acquire a pre-stored audio file and a picture file corresponding to the current interaction, and separately synthesize the audio file and the picture file to form a The audio and video interface that guides the user input.
The MCU according to claim 9, wherein the interaction initiation module is specifically configured to acquire an information type supported by each participant terminal, and send an audio and/or video interface to each participant terminal based on the information type.
The MCU according to claim 9, wherein the interaction initiation module is specifically configured to acquire an information type supported by each participant terminal, and detect whether each participant terminal sets a type of the received information, and if set, Then, it is detected whether the information type set by the corresponding conference terminal belongs to the information type supported by the conference terminal, and if yes, the audio or video interface is sent to the corresponding conference terminal based on the set information type; otherwise, the information type supported by the conference terminal is used to correspond to the conference terminal. Send an audio or video interface; if not set, send an audio and/or video interface to each participating terminal based on the type of information supported by each participating terminal.
The MCU according to claim 9, wherein

The interaction processing module is further configured to synthesize the statistical result into text, audio, and video, and send the statistical result of the text, audio, and/or video to each participating terminal based on the type of information supported by each of the participating terminals.
The MCU according to any one of claims 9 to 13, wherein

The interaction initiation module is further configured to encode the audio and/or video interface set to direct user input with the data, audio and video code streams in the current conference as a media stream format, and then send the format to the conference terminal;

The interaction processing module is further configured to encode the statistical result together with the data, audio, and video code streams in the current conference into a media stream format, and then send the result to each conference terminal.
A remote interactive system in an audio video conference, comprising: a conference service management system, a conference terminal, and the MCU according to any one of claims 9 to 14;

The participant terminal is configured to perform an interactive operation according to an instruction of the audio and/or video interface after receiving an audio and/or video interface corresponding to the current interaction and being set to guide the user input, and The interaction information obtained by the operation is sent to the MCU;

The conference service management system is configured to receive interaction information that is fed back by the MCUs according to the guidance of the audio and/or video interface, and collect statistics on the interaction information, and send the statistics to the MCU.
A remote interaction method in multi-point audio video communication, comprising:

The multi-point audio and video communication multi-point control processing unit sends an audio and/or video interface corresponding to the current interaction to guide the user input to each participating terminal; wherein each participating terminal supports the user input channel;

The multi-point audio video communication multi-point control processing unit receives interaction information fed back by each participating terminal according to the guidance of the audio and/or video interface;

The multipoint audio video communication multipoint control processing unit transmits the interactive information to a multipoint audio video communication service management system;

The multi-point audio video communication multi-point control processing unit receives the statistical result obtained by the multi-point audio video communication service management system using the interaction information to perform statistics;

The multipoint audio video communication multipoint control processing unit transmits the statistical result to each participating terminal.
The method according to claim 16, wherein the multi-point audio video communication multi-point control processing unit sends the audio and/or video interface to each of the participating terminals, and further includes:

The multi-point audio video communication multi-point control processing unit acquires the pre-stored audio file and picture file corresponding to the current interaction, and separately synthesizes the audio file and the picture file to form an audio and video interface for guiding user input.
The method of claim 16, wherein the multipoint audio video communication multipoint control processing unit transmits an audio and/or video interface directing user input to each of the participating terminals, including:

The multi-point audio and video communication multi-point control processing unit acquires the type of information supported by each participating terminal;

The multi-point audio video multi-point control processing unit transmits an audio and/or video interface that will guide the user input to each participating terminal based on the type of information supported by each participating terminal.
The method of claim 16, wherein the multipoint audio video communication multipoint control processing unit transmits an audio and/or video interface directing user input to each of the participating terminals, including:

The multi-point audio and video communication multi-point control processing unit acquires the type of information supported by each participating terminal, and detects whether each participating terminal sets the type of the received information;

When the participant terminal sets the type of the received information, it detects whether the type of information set by the corresponding conference terminal belongs to the type of information it supports, and if so, sends an audio and/or video interface to the corresponding conference terminal based on the set information type; No, sending an audio and/or video interface to the corresponding participant terminal based on the type of information supported by the participant terminal; and/or

When the participating terminal does not set the type of the received information, the audio and/or video interface is sent to each participating terminal based on the type of information supported by each participating terminal.
The method according to claim 16, wherein the multi-point audio video communication multi-point control processing unit sends the statistical result to each participating terminal, including:

The multi-point audio video communication multi-point control processing unit synthesizes the statistical result into text, audio and video, and transmits the statistical result of text, audio and/or video to each participating terminal based on the type of information supported by each participating terminal.
The method according to any one of claims 18 to 20, wherein the manner in which the multipoint audio video communication multipoint control processing unit acquires the type of information supported by each participating terminal comprises:

The multi-point audio and video communication multi-point control processing unit acquires a logical channel established for each participating terminal, and determines the type of information supported by the participating terminal according to the type of the logical channel; the types of the logical channel include: audio, video, and data.
The method according to any one of claims 16 to 20, wherein

The multi-point audio video communication multi-point control processing unit transmits an audio and/or video interface that directs user input to each participating terminal, including: a multi-point audio video communication multi-point control processing unit that will guide the user input audio and And/or the video interface is encoded into the media stream format together with the data, audio and video code streams in the current multi-point audio and video communication, and then sent to each participating terminal;

The multi-point audio video communication multi-point control processing unit sends the statistical result to each participating terminal, including: the multi-point audio video communication multi-point control processing unit encodes the statistical result together with the data, audio and video code streams in the current conference. It is sent to each participant terminal after being formatted as a media stream.
The method of any one of claims 16 to 20, wherein the type of interaction comprises: voting, scoring, and voting.
The method according to any one of claims 16 to 20, wherein the multipoint audio video communication multipoint control processing unit comprises a multipoint control unit MCU.
A multi-point audio and video communication multi-point control processing unit includes:

The interactive initiating module is configured to send, in the multi-point audio video communication, an audio and/or video interface corresponding to the current interaction, which is set to guide the user input, to each participating terminal; wherein each participating terminal supports the user input channel;

The interaction processing module is configured to receive the interaction information fed back by each participating terminal according to the instruction of the audio and/or video interface, and send the interaction information to the multi-point audio video communication service management system, And receiving the statistical result obtained by the conference service management system by using the interaction information, and sending the statistical result to each conference terminal.
The multipoint audio video communication multipoint control processing unit of claim 25, wherein the interaction initiation module is further configured to acquire a pre-stored audio file and a picture file corresponding to the current interaction, and to the audio file and The image files are separately synthesized to form an audio and video interface for directing user input.
The multipoint audio video communication multipoint control processing unit according to claim 25, wherein the interaction initiation module is configured to acquire an information type supported by each participant terminal, and send audio and/or to each participant terminal based on the information type. Or video interface.
The multipoint audio and video communication multipoint control processing unit according to claim 25, wherein the interaction initiation module is configured to acquire information types supported by each participant terminal, and detect whether each participant terminal sets the type of the received information. If the setting is made, it is detected whether the information type set by the corresponding conference terminal belongs to the information type supported by the conference terminal, and if so, the audio or video interface is sent to the corresponding conference terminal based on the set information type; otherwise, based on the conference terminal The supported information type sends an audio or video interface to the corresponding participant terminal; if not, the audio and/or video interface is sent to each participating terminal based on the type of information supported by each participating terminal.
A multipoint audio video communication multipoint control processing unit according to claim 25, wherein

The interaction processing module is further configured to synthesize the statistical result into text, audio, and video, and send the statistical result of the text, audio, and/or video to each participating terminal based on the type of information supported by each of the participating terminals.
The multipoint audio video communication multipoint control processing unit according to any one of claims 25 to 29, wherein

The interaction initiation module is further configured to encode the audio and/or video interface set to direct user input with the data, audio and video code streams in the current conference as a media stream format, and then send the format to the conference terminal;

The interaction processing module is further configured to encode the statistical result together with the data, audio, and video code streams in the current conference into a media stream format, and then send the result to each conference terminal.
The multipoint audio video communication multipoint control processing unit according to any one of claims 25 to 29, wherein said multipoint audio video communication comprises a voice video conference; said multipoint audio video communication multipoint control processing unit comprises Multipoint control unit MCU.