WO2004098125A1

WO2004098125A1 - Distributed multimedia conference system based on ip web

Info

Publication number: WO2004098125A1
Application number: PCT/CN2003/001080
Authority: WO
Inventors: Sheng An Wang
Original assignee: Sheng An Wang
Priority date: 2003-04-28
Filing date: 2003-12-17
Publication date: 2004-11-11
Also published as: CN1272932C; CN1543123A; AU2003289613A1

Abstract

A distributed multimedia conference system based on IP web, mostly comprising multimedia conference terminal, multipoint control unit and gatekeeper, characterized in that, said gatekeeper divided into tandem gatekeeper and multi user gatekeepers. Each user gatekeeper respectively is connected to the multimedia conference terminals and multipoint control unit, which is managed by said user gatekeeper. Tandem gatekeeper is connected to each user gatekeepers and matches them, managers the entire system. During the course of the conference, each conferred multimedia conference terminal is also connected to the multipoint control unit, which only controls the conference. Multipoint processing function is separated from the multipoint control unit, and is achieved in the each multimedia conference terminal. Furthermore, the invention is also discloses the multimedia conference system that can multicast the image, voice and data, in order to save the network bandwidth.

Description

Distributed multimedia conference system based on IP network

The invention relates to a multimedia conference system running in an IP service-supporting network, and in particular adopts distributed processing technology to realize a large-capacity multimedia conference system. The system is suitable for a multipoint system, that is, a plurality of conferences located in different places. A system for implementing a multimedia conference function is also suitable for a point-to-point system, that is, a system in which multimedia communication is performed by two parties located in different places. Background technique

The multimedia conference system connects a plurality of remotely located multimedia conference terminals through a communication network. Under the control of a multipoint control unit, participants are allowed to communicate in multimedia modes such as image, voice, and data. Multimedia conference system includes multimedia conference terminal and multipoint control unit

(Multi-point Control Unit, MCU) and gatekeeper (GK). Multimedia conference The terminal performs two-way communication with other terminals, multipoint control units and gatekeepers, and supports multimedia communication such as image, voice and data. The multipoint control unit provides control of multiple multimedia conference terminals, and mixes and exchanges images, voice, and data, and supports multipoint conferences between three or more terminals. Generally speaking, a multipoint control unit is composed of two function modules: multipoint control (MC) and multipoint processing (MP). Multipoint control provides control of a multipoint conference, and multipoint processing controls multiple channels of video, audio, and data. The signals are processed, and the multipoint processor is controlled by the multipoint controller. The gatekeeper implements the management functions of the system, including user management, conference management, device management, and bandwidth management.

In the existing multimedia conference system (as shown in FIG. 1), the multipoint control unit implements both the multipoint control function and the multipoint processing function. Therefore, the following problems exist:

1. Because the multipoint processing function is implemented on the multipoint control unit in a centralized manner, the processing load of the multipoint control unit is heavy, especially when there are many system users and multiple conferences are held at the same time, which limits the scale of the system Cannot form a large multimedia conference system;

2. During a multipoint conference, all the images, sounds, and data sent by all participating terminals are transmitted to multiple parties. The point control unit is centralized. After the multipoint processing module performs image and voice processing (multi-screen processing, mixing speed, mixing, etc.), it is forwarded to each participating terminal in a unicast manner, as shown in Figure 1. As shown in the line, this will inevitably cause a large data flow at the node where the multipoint control unit is located, and it will occupy more bandwidth resources, especially when there are a large number of participating terminals, which may cause network congestion in severe cases. Summary of the invention

The technical solution of the present invention is to provide a distributed multimedia conference system based on an IP network, and use it to form a large-capacity multimedia conference system, balance the load of system equipment, make reasonable use of network bandwidth resources, and effectively reduce the network load. The above problems exist in the existing multimedia conference systems.

The technical solution adopted by the present invention is: A distributed multimedia conference system based on an IP network, which mainly includes a multimedia conference terminal, a multipoint control unit, and a gatekeeper, and is characterized in that the gatekeeper is divided into a tandem gatekeeper and a multi-gatekeeper. Each user gatekeeper, each user gatekeeper is connected to the multimedia conference terminal and multipoint control unit under its jurisdiction, the tandem gatekeeper is connected to each user gatekeeper, and coordinate each user gatekeeper to manage the entire system; each multimedia conference terminal Communicate with the user's gatekeeper when it is idle. During the conference, in addition to establishing a communication connection with the user's gatekeeper, it also establishes a communication connection with the multipoint control unit that controls the conference terminal; the multipoint control unit only It has multi-point control function, its multi-point processing function is separated from the multi-point control unit, and the multi-point processing function is realized in each multimedia conference terminal.

In the present invention, by adopting a distributed processing method, the gatekeeper is divided into a tandem gatekeeper and a plurality of user gatekeepers, and a plurality of user gatekeepers jointly implement system management functions under the coordination of the tandem gatekeeper; The multipoint processing function is separated from the multipoint control unit and distributed to the conference terminal, that is, the multipoint control unit and each multimedia conference system terminal only exchange control code streams. There is no exchange of information code streams such as audio and video. Information codes The stream is transmitted between the multimedia terminals through the corresponding routing under the control of the multipoint control unit. When necessary, the present invention sets up multiple multipoint control units to balance the load of the multipoint control unit, which can greatly expand the multimedia conference. The capacity of the system solves the existing set The capacity of Chinese conference system is limited.

In the present invention, all signals between the multimedia conference terminals, the conference terminal and the multipoint control unit, the user gatekeeper, and the tandem gatekeeper are transmitted through the IP network; and each conference terminal transmits audio and video code streams in a multicast manner. Multicast is a unique transmission technology in IP networks. It allows one or more senders (multicast sources) to send a single data packet to multiple receivers at one time. A multicast source sends a data packet with a specific multicast address. Only members of the multicast group can receive the data packet. Multicast can greatly save network bandwidth, because no matter how many receivers, only a single data packet is transmitted on any link in the entire network, it improves data transmission efficiency and reduces the possibility of congestion in the backbone network.

In the present invention, a plurality of user gatekeepers and tandem gatekeepers can adopt a Chinese-machine hot-scoring method to improve their reliability. BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a distributed structure and code stream control of a centralized multimedia conference system in the prior art;

FIG. 2 is a schematic diagram of a logical structure distribution of the present invention; FIG.

3 is a schematic diagram of code stream control according to the present invention;

4 is a schematic diagram of a communication process between devices of the present invention;

FIG. 5 is a block diagram of the structure of a multimedia conference terminal according to the present invention;

FIG. 6 is a principle block diagram of a multipoint control module and a gatekeeper in the multimedia conference terminal in FIG. 5; FIG. 7 is a structural topology diagram of an embodiment of the present invention. detailed description

As shown in FIG. 2, the present invention is cascaded by a tandem gatekeeper 101, multiple user gatekeepers 1021-102η (n greater than or equal to 2), and multiple multipoint control units MCU 1031-103η (η greater than or equal to 2). It is composed of multiple multimedia conference terminals 1041 to 104η (η is greater than or equal to 2), the tandem gatekeeper 101 establishes a communication connection with each user gatekeeper 10Ή-102η, and coordinates the user gatekeeper 1021-102η. Manage the entire system; At the same time, each user gatekeeper 1021-102η is responsible for managing the corresponding multimedia conference terminals 1041-104η and the multipoint control unit MCU 1031-103η, and establishes a two-way communication connection; each multimedia terminal 1041-104η is in Establish a two-way communication connection with the user gatekeeper 1021-102η when it is idle. During the meeting, in addition to establishing a connection with the user gatekeeper 1021-102η, it also establishes a communication connection with the multipoint control unit that controls the conference, and Each multimedia conference terminal 1041-104η also has a multi-point processing MP function, that is, the audio and video signals transmitted by other terminals can be processed for multi-screen, mixing, and mixing speed. Therefore, its main feature is that it can simultaneously implement multi-point processing functions. Multi-point control unit MCU controls the conference terminals 1041-104η during the multi-point conference, switches the image and voice signals, and implements the multi-point control MC function. Multiple MCUs can be set in the system to achieve load balancing.

As shown in FIG. 3, during the conference, the multimedia conference terminal 104n, which is a broadcast venue, performs bidirectional communication with other terminals 1041-104η-1, the multipoint control unit 103η, which controls the conference, and the user gatekeeper 1026 to which it belongs. In addition to supporting multimedia applications such as voice, image, and data, the multipoint processing MP function is added, that is, the audio and video signals transmitted by other terminals can be processed for multi-screen and mixing, and the multimedia conference terminal 104η, which is a broadcast venue, also Multicast transmission of audio and video code streams to other terminals 1041-104η-1, as shown by the solid line in FIG. 3; At the same time, the multipoint control unit 103η is only responsible for controlling the exchange of code streams, as shown by the dotted line in FIG. 3, which does not exist The exchange of audio and video code streams greatly reduces the processing load of the multipoint control unit 103η.

As shown in FIG. 5, each multimedia conference terminal 1041-104η mainly includes a video source 401, an audio source 402, an audio-video input interface 403, an audio-video encoding 408, an audio-video decoding 409, and an audio-video output interface 4 (Η, sound reinforcement equipment). 405. The display device 406, the network interface 410, and the like are characterized in that a multipoint processing function module 407 is added between the audio and video decoding 409 and the audio and video output interface 404, so that each multimedia conference terminal 1041-104η also has a multipoint processing MP. Function: The video source 401 and audio source 402 send the image, voice and data signals of the local conference site to the audio and video input interface 403 to the audio and video encoding 408 for compression encoding and other processing and then send them to the IP network through the network interface 410; After receiving various information such as audio and video by the IP network, after the network interface 410 Send to audio and video decoding 409 for decoding and other processing, and restore it to image, voice and data signals, and then multipoint processing 407 to restore the restored image, voice, and data signal to multi-screen, mixing and other processing, and then output through audio and video The interface 404 connects to the sound reinforcement device 405 and the display device 406.

The working process of the present invention is as follows:

As shown in Figures 2, 3, and 4, after the multimedia conference terminals 1041-104η are activated, they first establish a two-way communication connection with the user gatekeepers 1021-102η to which they belong. As shown in Figure 3, they apply to the user gatekeepers 1021-102η for login. The user gatekeeper authenticates the identity of the terminal. If the terminal passes user authentication, the login is successful and the system can be used; otherwise, the terminal cannot use the system. The user authentication function of the system can ensure the security of system use and prevent unauthorized users from using the system.

Each user gatekeeper 1021-102η obtains the relevant information of the user during the user authentication process, such as the user identification code, user name, user IP address, etc., and each user gatekeeper 1021-102η queries the local database according to the user identification code, and confirms Whether the user is an authorized user. If it is an authorized user, update the relevant information of the user, such as user name, user IP address, user status, and so on. The user gatekeeper uploads the update information of the user to the tandem gatekeeper 101. The tandem gatekeeper 101 dynamically maintains the updated user information database, and then downloads the updated information to other user gatekeepers. status information. Through the above process, the tandem gatekeeper 101 and each user gatekeeper 1021-102η can dynamically monitor, update, and manage user status information of the entire system in real time.

After the multimedia conference terminal is successfully logged in, the system can be used. The process of convening the conference is as follows: The multimedia conference terminal 104n that needs to hold a conference firstly schedules a conference to be held from the user gatekeeper 102η as a broadcast conference site. The user inputs conference information through the conference terminal 104n, including information such as conference name, conference start time, conference duration, conference profile, and participating members. The conference terminal 104η transmits the conference information to the user gatekeeper 102η to which it belongs. The user gatekeeper 102η assigns the conference to the conference according to the distribution of participating members in the system and the load of the MCUs in the system. A multipoint control unit MCU controls the conference by the multipoint control unit MCU. The user gatekeeper adds the relevant information of the conference to the local conference information database, and uploads the updated conference information to the tandem gatekeeper 101. Tandem Gatekeeper 101 at the Update System Meeting At the same time as the information database, the updated conference information is transmitted to other user gatekeepers. Through the above process, the tandem gatekeeper 101 and each user gatekeeper 1021-102η can dynamically update and maintain conference information of the entire system in real time.

When the terminal 104n initiating the conference wants to hold a scheduled conference, it first establishes a two-way communication connection with the multipoint control unit MCU of the conference, and the multipoint control unit MCU calls each participating terminal to join the conference. At this time, the multipoint control unit MCU establishes a two-way communication connection with the participating terminals that have joined the conference, as shown by the dashed lines in FIG. 3. After joining the conference, the participating terminals still maintain communication connections with their respective user gatekeepers in order to report changes in terminal status information to the user gatekeepers in real time. For example, after the conference, the status of each participant terminal from "idle" to "participating", this time, the participating terminal active gatekeeper to which the user belongs to the state change report Gen ^ ^1, gatekeeper update the user status of the local user Information, and upload the updated user status information to the tandem gatekeeper 101 at the same time. The tandem gatekeeper 101 updates the system user status information database, and then downloads the updated user status information to other user gatekeepers, ensuring that the user status information of all gatekeepers is updated dynamically and in real time.

During the conference, the multipoint control unit MCU of this conference can control the multipoint conference according to the requirements of the terminal that has the conference control right. For example, the image and sound of a selected site are broadcasted to other sites, and the speaking right is assigned to the required Speaking venue, etc. The multi-point control unit MCU controls multiple participating terminals as a multi-point control MC. In order to implement the multi-point control function, the multi-point control unit MCU and the participating terminals must maintain a two-way communication connection and exchange control and response information, as shown in the figure. 3 dashed lines and shown in Figure 4.

As shown in FIG. 6, the multipoint processing units 1031 to 103η are composed of a main processor CPU 701, a memory 702, an I / O interface 705, a mass storage medium 704, other physical interfaces 703, and a network interface 706.

During the conference, it is necessary to perform multi-screen and audio mixing on the audio and video signals of multiple terminals, which is called multi-point processing MP. In the present invention, the multipoint processing function is not implemented on the multipoint control unit MCU, but is implemented on each of the conference terminals 1041 to 104η. As shown in FIG. 5, the multipoint processing function is separated from the multipoint control unit MCU. Each of the participating terminals 1041-104η realizes a multipoint processing conference. As shown in FIG. 3, in the distributed conference system of the present invention, a multipoint control unit Only the control and response information is exchanged between the MCU and the participating terminals 1041-104η. As shown by the dashed line in Figure 3, the control and response information occupies a very small bandwidth, and the average value of each channel is on the order of 100b / s. Therefore, the multipoint control unit The communication load and processing load of the MCU are very light. It can support more terminals and form a large-capacity conference system, which solves the problem of limited capacity of the existing centralized conference system.

At the end of the conference of this system, the MCU controlling the multipoint control unit of the conference and the participating terminals 1041-104η are disconnected from the communication connection, and the participating terminals 1041-104η return to the idle state, and only maintain with the respective user gatekeepers 1021-102η Communication connection.

As shown in FIG. 7, it is an embodiment of the present invention: The present invention is used to implement a certain multimedia conference system. The system consists of a tandem gatekeeper, a user gatekeeper, a multipoint control unit MCU, multiple multimedia conference terminals, and corresponding routers. There is no restriction on the physical location of these devices in the network. The number of user gatekeepers, MCUs, conference terminals, routers, and Ethernet segments is unlimited, and can be two or more. The tandem gatekeeper, user gatekeeper, MCU, conference terminal, and router are connected to the local Ethernet through the Ethernet interface. Each Ethernet segment is connected via an IP transmission network via a router. Logically, the tandem gatekeeper and the user gatekeeper are independent of the multipoint control unit MCU and the multimedia conference terminal, but can be included in the multipoint control unit MCU and the multimedia conference terminal during physical implementation. The gatekeeper and user gatekeeper software runs on the MCU server of the multipoint control unit. The specific working process of this embodiment is the same as the implementation process described in the above specific implementation.

In addition, the technical solution of the present invention can also implement a remote teaching system, a remote medical system, and an image monitoring system. Industrial applications,

Compared with the prior art, the present invention has the following beneficial effects:

1. Due to the distributed processing method, the gatekeeper is divided into a tandem gatekeeper and multiple user gatekeepers, and the multipoint processing function is separated from the multipoint control unit to implement multipoint on the multimedia conference terminal. Processing functions, that is, multi-screen processing, mixing speed, mixing and other functions are implemented by the participating terminals, This greatly reduces the processing load of the multi-point control unit. A single multi-point control unit can hold more meetings at the same time and support more multimedia conference terminal work. Second, when setting up a large-scale system, it can be based on the multimedia conference terminal. According to the distribution of multiple multi-point control units, the tandem gatekeeper can reasonably allocate the conferences according to the distribution of the entire network of conferences among the multi-point control units, and realize the load balancing function of the multi-point control units.

2. Because the audio and video code streams are transmitted in a multicast manner, and the multipoint processing function is implemented on each conference terminal, when a conference is held, the image and sound of a conference terminal selected according to the needs is directly sent to all conference terminals. Multicast does not have to be aggregated to the multipoint control unit, and multicast data packets are transmitted to the multimedia conference terminals along the corresponding routes, which saves network bandwidth and avoids network congestion problems that may occur in centralized multimedia conference systems.

Claims

Rights request

1. Distributed multimedia system based on IP network, which mainly includes multimedia conference terminal, multipoint control unit and gatekeeper, which is characterized by:

The gatekeeper is divided into a tandem gatekeeper and a plurality of user gatekeepers. Each user gatekeeper is connected to a multimedia conference terminal and a multipoint control unit under its jurisdiction, and the tandem gatekeeper is connected to each user gatekeeper and coordinated. Each user gatekeeper manages the entire system;

During the conference, each participating multimedia conference terminal is also connected to a multipoint control unit that controls the conference;

The multi-point control unit only has multi-point control functions. The multi-point processing function is separated from the multi-point control unit, and the multi-point processing functions are implemented in each multimedia conference terminal.

2. The distributed multimedia conference system according to claim 1, wherein there are two or more user gatekeepers.

3. The distributed multimedia conference system according to claim 1 or 2, characterized in that: the tandem gatekeeper or a plurality of user gatekeepers can adopt a dual-machine hot backup method to improve reliability.

4. The distributed multimedia conference system according to claim 1, wherein the multipoint control unit only implements a multipoint conference control function.

5. The distributed multimedia conference system according to claim 4, wherein: said multipoint control unit is two or more.

6. The distributed multimedia conference system according to claim 1 or 5, characterized in that: the multipoint control unit realizes conference control load balancing with the support of a tandem gatekeeper and each user gatekeeper.

7. The distributed multimedia conference system according to claim 1, wherein each of the multimedia conference terminals further has a multipoint processing function.

8. The distributed multimedia conference system according to claim 1, wherein: All signals between the multimedia conference terminals, the conference terminal and the multipoint control unit, the user gatekeeper and the tandem gatekeeper are transmitted through the IP network.

9. The distributed multimedia conference system according to claim 1 or 7, wherein each of the multimedia conference terminals transmits an image, a voice, and a data signal in a multicast manner.