WO2017004831A1

WO2017004831A1 - Multimedia meeting implementation method, device, and system

Info

Publication number: WO2017004831A1
Application number: PCT/CN2015/083664
Authority: WO
Inventors: 麻鑫锋
Original assignee: 华为技术有限公司
Priority date: 2015-07-09
Filing date: 2015-07-09
Publication date: 2017-01-12

Abstract

The present invention provides a multimedia meeting implementation method, device, and system. The system, providing a multimedia meeting between two or more meeting rooms, includes a meeting server and two or more meeting terminals. A mobile terminal in a first meeting room of the two or more meeting rooms is configured to acquire first multimedia data of the first meeting room, and transmit, according to a multicast group address of a multicast group of the first meeting room, the first multimedia data. A meeting terminal of the first meeting room is configured to enter the multicast group, receive, via the multimedia group, the first multimedia data, combine the first multimedia data with and self-acquired second multimedia data, and transmit the same to the meeting server. Based on the embodiments of the invention, the meeting terminal can employ the mobile terminal to collect multimedia data of the meeting room and transmit the same to the meeting server. The mobile terminal is often carried by a meeting participant and can be moved freely, and consequently the mobile terminal can be used to collect the multimedia data in the meeting room easily.

Description

Method, device and system for realizing multimedia conference

Technical field

The present invention relates to the field of communications technologies, and in particular, to a method, apparatus, and system for implementing a multimedia conference.

Background technique

With the development of communication technology and enterprise scale, many enterprises need to establish business bases in various places. Therefore, more and more enterprises are now discussing and communicating through multimedia conferences.

A multimedia conference is a group of individuals or groups with different geographical locations. Through transmission lines and multimedia devices, voice, video and file data are transmitted to each other for instant and interactive communication, which can be audio conferences or audio and video conferences. The multimedia conference system mainly includes a conference server located on the network side, such as an MCU (Multi-Point Control Unit), and a conference terminal of each conference site.

Generally, each conference site has a conference terminal, and the conference terminal collects multimedia data of the conference site and sends the multimedia data to the conference server through a multimedia data collection device (such as a microphone or a camera). For example, the voice of the participant (ie, the pickup) is collected by a plurality of microphones and sent to the conference server. If it is an audio-video conference, the video of the participant who is the presenter can be sent to the conference server through the camera.

However, the number of multimedia data collection devices of the conference terminal is generally limited, and the location is relatively fixed. When there are many participants in a certain venue, it is not convenient to collect multimedia data of the participants in need. For example, at present, many conference terminals used in conferences use array microphones to expand the range of pickup. The general pickup range is 3 meters to 5 meters. If the distance between a participant and the conference terminal exceeds this range, then pick up. The sound effect will be poor. If you need to speak, you need to move to the position within the pickup range, which is very inconvenient.

Summary of the invention

The present invention provides a method, device, and system for implementing a multimedia conference, which solves the problem that the multimedia data collection device in the conference terminal cannot be conveniently collected due to the limited number of multimedia data collection devices of the conference terminal and the relatively fixed location. The problem.

The first aspect provides a conference system, where the conference system is configured to perform a multimedia conference between at least two conference sites, where the conference system includes a conference server and at least two conference terminals, and each conference site is configured with a conference terminal. ;

The first site of the at least two sites further includes a multicast group and a mobile terminal;

The mobile terminal is configured to acquire a multicast address of the multicast group of the first site, collect the first multimedia data of the first site, and perform multicast according to the multicast group of the first site. Sending the collected first multimedia data to the multicast group of the first site;

The conference terminal of the first site is configured to collect the second multimedia data of the first site, join the multicast group of the first site, and receive the mobile by using the multicast group of the first site. The first multimedia data sent by the terminal combines the first multimedia data with the second multimedia data, and sends the combined multimedia data to the conference server.

In a first possible implementation manner of the first aspect, the mobile terminal, after collecting the first multimedia data of the first site, is specifically used to collect the first multiple Media data is encapsulated into N data packets, and the N data packets are sent to the multicast group of the first site, where a destination address of each of the N data packets includes the The multicast address of the multicast group of the first site, where N is a positive integer greater than or equal to 2; the conference terminal of the first site receives the N data in the multicast group that passes the first site After the packet, specifically, the data included in the N data packets is integrated into the first multimedia data.

With reference to the first possible implementation manner of the first aspect, in a second possible implementation, the first site includes M mobile terminals, and the M mobile terminals collect M shares of the first multimedia The volume data, the first multimedia data is collected by a mobile terminal, and the conference terminal of the first site receives M×N data packets through the multicast group of the first site, where the M× Each of the N data packets includes a media access control address of a sender transmitting each of the M×N data packets, the M being a positive integer greater than or equal to 2; the first The conference terminal of the site is specifically configured to identify, according to the media access control address of the sender in each of the M×N data packets, the sending of each data packet in the M×N data packets. Party, will be derived from the same mobile terminal The data contained in the N data packets is integrated into one piece of the first multimedia data.

With the second possible implementation of the first aspect, in a third possible implementation, the M mobile terminals of the first site are also used to access the conference terminal of the first site. Subnet; the sender's media access control address is carried in the source MAC address field of each of the M x N data packets.

With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, the M mobile terminals of the first conference site are accessed by using the wifi hotspot provided by the conference terminal of the first conference site Accessing the subnet accessed by the conference terminal of the first site in a network manner; or accessing the network by using a wifi hotspot access network in the same subnet as the conference terminal of the first conference site The subnet to which the conference terminal of a conference site is connected.

With reference to the first aspect, or any one of the first to fourth possible implementations of the first aspect, in the fifth possible implementation, the multimedia conference includes any of an audio conference, a video conference, and an audio and video conference. a conference terminal of the first site, after receiving the first multimedia data and collecting the second multimedia data, specifically for using the first multimedia data and the The audio data in the second multimedia data is subjected to a mixing process, and/or the video data in the first multimedia data and the second multimedia data are subjected to screen combining processing.

A second aspect provides a method for implementing a multimedia conference, where the method is applied to a conference system, where the conference system is configured to perform a multimedia conference between at least two conference sites, where the conference system includes a conference server and at least two a conference terminal, each conference site is configured with a conference terminal, and the first conference site further includes a multicast group and a mobile terminal, and the conference terminal of the first conference site joins the multicast group, and the method includes: the mobile terminal acquiring the location a multicast address of the multicast group of the first site; the mobile terminal collects the first multimedia data of the first site, and according to the multicast address of the multicast group of the first site, The multicast group of the first site sends the collected first multimedia data.

In a first possible implementation manner of the second aspect, the sending, according to the multicast address of the multicast group of the first site, the first multiple collected to the multicast group of the first site Media data, with Body includes:

The mobile terminal encapsulates the collected first multimedia data into N data packets, and sends the N data packets to the multicast group of the first site, where the N data The destination address of each data packet in the packet includes the multicast address of the multicast group of the first site, and the N is a positive integer greater than or equal to 2.

In conjunction with the first possible implementation of the second aspect, in a second possible implementation, each of the N data packets includes a media access control address of the mobile terminal.

With reference to the second possible implementation manner of the second aspect, in a third possible implementation manner, the media access control address of the mobile terminal carries a source MAC address of each data packet in the N data packets. In the field, before the sending the collected first multimedia data to the multicast group of the first site, the method further includes: accessing, by the mobile terminal, the conference terminal accessed by the first conference site Subnet.

With reference to the third possible implementation manner of the second aspect, in a fourth possible implementation manner, the subnet that the mobile terminal accesses the conference terminal of the first conference site specifically includes: the mobile terminal Accessing the network through the wifi hotspot provided by the conference terminal of the first conference site; or the mobile terminal accessing the network through the wifi hotspot of the same subnet as the conference terminal of the first conference site.

A third aspect provides a method for implementing a multimedia conference, where the method is applied to a conference system, where the conference system is configured to perform a multimedia conference between at least two conference sites, where the conference system includes a conference server and at least two a conference terminal, where each conference site is configured with a conference terminal, and the first conference site further includes a multicast group and a mobile terminal for collecting the first multimedia data of the first conference site, where the method includes: the first conference site The conference terminal collects the second multimedia data of the first site, joins the multicast group of the first site, and receives the first multiple sent by the mobile terminal by using the multicast group of the first site. Media data, combining the first multimedia data with the second multimedia data, and transmitting the combined multimedia data to the conference server.

In a first possible implementation manner of the third aspect, the first multimedia data that is received by the conference terminal of the first site by the multicast group of the first site is encapsulated in N data packets. Its The destination address of each of the N data packets includes a multicast address of the multicast group of the first site, where N is a positive integer greater than or equal to 2; Before combining the multimedia data with the second multimedia data, the conference terminal of the first site integrates data included in the N data packets into the first multimedia data.

With reference to the first possible implementation manner of the third aspect, in a second possible implementation, the first site includes M mobile terminals, and the M mobile terminals collect M shares of the first multimedia The volume data, the first multimedia data is collected by a mobile terminal, and the conference terminal of the first site receives M×N data packets through the multicast group of the first site, where the M× Each of the N data packets includes a media access control address of a sender transmitting each of the M×N data packets, the M being a positive integer greater than or equal to 2; the first The conference terminal of the site specifically identifies the sender of each of the M×N data packets according to the media access control address of the sender in each of the M×N data packets, and The data contained in the N data packets originating from the same mobile terminal is integrated into one piece of the first multimedia data.

With reference to the third aspect, or the first or second possible implementation manner of the third aspect, in a third possible implementation, the multimedia conference includes any one of an audio conference, a video conference, and an audio and video conference. The conference terminal of the first site combines the first multimedia data and the second multimedia data, and the conference terminal of the first site will use the first multimedia data. And performing audio mixing processing on the audio data in the second multimedia data, and/or performing video combination processing on the video data in the first multimedia data and the second multimedia data.

A fourth aspect provides a mobile terminal, where the mobile terminal is located in a first conference site of at least two conference sites of a multimedia conference, where the multimedia conference is performed by a conference system including a conference server and at least two conference terminals, each The conference site is configured with a conference terminal, and the first conference site further includes a multicast group, and the conference terminal of the first conference site has joined the multicast group; the mobile terminal includes an acquisition unit and a sending unit; and the collection unit is configured to: Collecting the first multimedia data of the first site; the sending unit, configured to acquire a multicast address of the multicast group of the first site, and according to the obtained first session The multicast address of the multicast group of the field sends the first multimedia data collected by the collecting unit to the multicast group of the first site.

In a first possible implementation manner of the fourth aspect, the sending unit is configured to: encapsulate the first multimedia data collected by the collecting unit into N data packets, and The multicast group of the first site sends the N data packets, where the destination address of each of the N data packets includes a multicast address of the multicast group of the first site, and the N Is a positive integer greater than or equal to 2.

In conjunction with the first possible implementation of the fourth aspect, in a second possible implementation, each of the N data packets includes a media access control address of the mobile terminal.

With reference to the second possible implementation manner of the fourth aspect, in a third possible implementation manner, the media access control address of the mobile terminal carries a source MAC address of each data packet in the N data packets. In the field, the mobile terminal further includes an access unit, configured to access a subnet accessed by the conference terminal of the first conference site.

With reference to the third possible implementation manner of the fourth aspect, in a fourth possible implementation, the access unit is specifically configured to: access a network through a wifi hotspot provided by the conference terminal of the first conference site; or And accessing the network through a wifi hotspot in the same subnet as the conference terminal of the first conference site.

A fifth aspect provides a conference terminal, where the conference terminal is included in a conference system, where the conference system is configured to perform a multimedia conference between at least two conference sites; the conference site where the conference terminal is located further includes a multicast group and Collecting, by the mobile terminal, the first multimedia data of the conference site where the conference terminal is located; the first conference terminal includes: a data acquisition unit, a data processing unit, and a data sending unit;

The data acquisition unit is configured to collect the second multimedia data of the conference site where the conference terminal is located, and add the conference terminal to the multicast group of the conference site where the conference terminal is located, and pass the group of the conference site where the conference terminal is located. Receiving, by the broadcast group, the first multimedia data sent by the mobile terminal;

The data processing unit is configured to combine the first multimedia data collected by the data acquiring unit and the received second multimedia data;

The data sending unit is configured to send the multimedia data obtained by combining the data processing units to the conference server.

In a first possible implementation manner of the fifth aspect, the data acquiring unit, by using the multicast group received by the multicast group of the conference site where the conference terminal is located, is encapsulated in N data packets, where The destination address of each of the N data packets includes a multicast address of a multicast group of the site where the conference terminal is located, where N is a positive integer greater than or equal to 2; the data acquisition unit further And used to integrate data included in the N data packets into the first multimedia data.

With reference to the first possible implementation manner of the fifth aspect, in a second possible implementation manner, the conference site where the conference terminal is located includes M mobile terminals, and the M mobile terminals collect the first Media data, each of the first multimedia data is collected by a mobile terminal, and the data acquiring unit receives M×N data packets by using a multicast group of the conference site where the conference terminal is located, where the M×N Each of the data packets includes a media access control address of a sender transmitting each of the M×N data packets, the M being a positive integer greater than or equal to 2; the data acquisition unit Specifically, according to the media access control address of the sender in each of the M×N data packets, identifying a sender of each of the M×N data packets, which is derived from the same The data contained in the N data packets of the mobile terminal is integrated into one piece of the first multimedia data.

With reference to the fifth aspect, or the first or the second possible implementation manner of the fifth aspect, in a third possible implementation, the multimedia conference includes any one of an audio conference, a video conference, and an audio and video conference. The data processing unit is specifically configured to perform mixing processing on the audio data in the first multimedia data and the second multimedia data, and/or, the first multimedia The data and the video data in the second multimedia data are subjected to screen combination processing.

According to the embodiment of the present invention, the conference terminal can use the mobile terminal of the conference site to collect the multimedia data of the conference site and send it to the conference server. The mobile terminal is often carried by the participants and can be easily moved, so that the multimedia data in the venue can be conveniently collected. In addition, according to the embodiment of the present invention, there is no need for signaling interaction between the conference terminal and the mobile terminal, and there is no need to establish a communication channel between the conference terminal and the mobile terminal, and the resource consumption of the conference terminal and the mobile terminal is relatively small.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.

FIG. 1 is a schematic structural diagram of a networking of a conference system according to an embodiment of the present disclosure;

2 is a schematic flowchart of a method according to Embodiment 1 of the present invention;

3A and 3B are schematic diagrams showing the structure of a mobile terminal 500 according to Embodiment 2 of the present invention;

4 is a schematic structural diagram of a conference terminal 600 according to Embodiment 3 of the present invention;

FIG. 5 is a schematic structural diagram of a mobile terminal 1000 according to Embodiment 4 of the present invention;

FIG. 6 is a schematic structural diagram of a conference terminal 2000 according to Embodiment 5 of the present invention;

detailed description

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

FIG. 1 is a schematic structural diagram of a conference system 100 according to Embodiment 1 of the present invention. The conference system 100 is configured to perform a multimedia conference between the two conference sites, including the conference server 110 and the conference terminal 120 configured in each conference site, such as the conference terminal A in the first conference site and the conference terminal B in the second conference site. The conference terminal 120 may specifically be an IP (Internet Protocol) telephone or a desktop conference terminal. The conference terminal 120 of each site accesses the conference created on the conference server 110, and sends the multimedia data (such as audio data, video data, and the like) of the conference site to the conference server 110, and the conference server 110 receives the conference terminal 120 from a conference site. The multimedia data is sent to the conference terminal 120 of the other conference site.

In addition, the first site shown in FIG. 1 further includes a mobile terminal 130, which is a mobile terminal having a multimedia data collection device (such as a microphone or a camera), and may be a smart phone, a tablet computer, or the like, and the mobile terminal 130. An application (hereinafter referred to as a conference assistant application) may be installed, by which the operations performed by the mobile terminal 130 in the method of Embodiment 1 of the present invention are performed.

In the conference system 100 provided by the first embodiment of the present invention, the first site also includes a multicast group. Correspondingly, the mobile terminal 130 of the first site acquires the multicast address of the multicast group of the first site, and collects the first multimedia data of the first site, and according to the group of the first site The multicast address of the broadcast group sends the collected first multimedia data to the multicast group of the first site; the conference terminal 120 of the first site collects the second multimedia of the first site The first data of the first site is added to the multicast data of the first site, and the first multimedia data sent by the mobile terminal 130 of the first site is received by the multicast group of the first site, and the first The multimedia data is combined with the second multimedia data, and the combined multimedia data is sent to the conference server 110; the conference server 110 sends the multimedia data received from the conference terminal of the first conference site to the A conference terminal of another conference site of at least two sites (such as conference terminal B of the second conference site).

The multicast address may be a MAC (Media Access Control) multicast address, and the mobile terminal sends the first multimedia data to the multicast group according to the MAC multicast address. Multicast for Layer 2 (data link layer).

The multicast address may also be an IP multicast address, and the manner in which the mobile terminal sends the first multimedia data to the multicast group according to the IP multicast address is called a three-layer (IP layer) group. broadcast.

It should be noted that, if the multicast is performed through the Layer 2 multicast mode, the multicast data is only in the same subnet as the sender. Before the broadcast, you need to access the subnet accessed by the conference terminal of the first site.

FIG. 2 is a schematic flowchart of a method according to Embodiment 1 of the present invention. The method according to Embodiment 1 of the present invention is applied to the conference system 100 shown in FIG. 1.

As shown in FIG. 2, the method provided in Embodiment 1 of the present invention includes:

Steps S101-S102: The conference terminal (hereinafter referred to as the first conference terminal) of the first site joins the multicast group of the site and joins the multimedia conference.

In a specific implementation, when the first conference terminal is registered to the management server on the network, the management server allocates a multicast address to the site where the first conference terminal is located, and sends the multicast address to the conference site. The first conference terminal is described. The management server may specifically be an automatic configuration server. (automatic configuration server, ACS).

The first conference terminal joins the multicast group corresponding to the multicast address after receiving the multicast address, and may be added to the multicast group when receiving the multicast address, or is in access The joining of the multicast group in the conference may be performed by adding the multicast group when detecting a user operation for instructing the first conference terminal to listen to the multicast group.

Step S103: The mobile terminal of the first site acquires the multicast address of the multicast group of the first site.

There are various implementations of step S103.

Embodiment 1: The first conference terminal displays the multicast address on the interface, and the user of the mobile terminal manually inputs the multicast address displayed on the interface of the first conference terminal on the mobile terminal. Correspondingly, the mobile terminal acquires the multicast address input by the user.

Embodiment 2: The first conference terminal displays a two-dimensional code including the multicast address on an interface, and the mobile terminal acquires the multicast address by scanning the two-dimensional code.

Embodiment 3: The mobile terminal sends a message carrying the location of the mobile terminal to the management server to request to acquire a multicast address of a conference terminal that is closest to the mobile terminal, where the management server is configured according to the mobile terminal The location and the pre-acquired location of each conference terminal determine a conference terminal that is closest to the mobile terminal, and transmits a multicast address assigned to the conference terminal closest to the mobile terminal to the mobile terminal.

The above three implementation manners are applicable to the Layer 2 multicast mode and the Layer 3 multicast mode. For the Layer 2 multicast mode, the following implementation manner 4 can also be adopted.

Implementation 4: The administrator configures a conference terminal for each subnet, that is, ensures that there is only one conference terminal in each subnet, and conference terminals in multiple different subnets share one MAC multicast address. Correspondingly, the shared MAC multicast address can be directly configured in the mobile terminal as the multicast address of the multicast group of the site, and the mobile terminal needs to send the first to the multicast group. Before the multimedia data, access the subnet to which the conference terminal of the site is connected, and obtain the pre-configured shared MAC multicast address. Subsequently, the first multimedia data that is sent by the mobile terminal to the multicast group according to the shared MAC multicast address is naturally received by the first conference terminal, and is not used by others. The conference terminal received.

In a specific implementation, step S103 may be performed before the triggering operation is detected, or may be performed when the triggering operation is detected, where the triggering operation is used to indicate sending the self-collected to the first conference terminal. The first multimedia data of the first venue.

Step S104: The first conference terminal monitors the first multimedia data sent to the multicast group.

The first conference terminal starts to listen to the multimedia data sent to the multicast group after joining the multicast group, and specifically may start the monitoring at the startup, or start the monitoring when it is determined that the conference has been accessed, or The monitoring is started when a user operation for instructing the first conference terminal to listen to the multicast group is detected.

A preferred solution is that the first conference terminal joins the multicast group when receiving the multicast address in step S101, and starts monitoring when it is determined that the conference has been accessed in step S104.

Steps S105-S106: The mobile terminal collects the first multimedia data of the first site, and sends the collected first multimedia data to the multicast group according to the multicast address.

Specifically, the mobile terminal may perform steps S105-S106 when the triggering operation is detected. If step S103 is also performed when the triggering operation is detected, the mobile terminal sequentially performs steps S103, S105, and S106 when detecting the triggering operation.

In a specific implementation, the mobile terminal encapsulates the collected first multimedia data into N data packets, and sends the N data packets to the multicast group of the first site, where The destination address of each of the N data packets includes a multicast address of the multicast group of the first site, where N is a positive integer greater than or equal to 2.

The multicast address may be a MAC multicast address, and the mobile terminal sends the first multimedia data to the multicast group according to the MAC multicast address, that is, the layer 2 multicast is adopted. Mode for multicasting. Specifically, the mobile terminal encapsulates the first multimedia data into N Layer 2 data packets, where a destination MAC address of the N Layer 2 data packets is the MAC multicast address.

It should be noted that, if the multicast is performed by using the Layer 2 multicast mode, the mobile terminal needs to access the subnet accessed by the conference terminal of the first site before the step S106. The mobile terminal Specifically, the subnet accessed by the conference terminal of the first conference site may be accessed by using the wifi hotspot access network provided by the conference terminal of the first conference site; and the conference with the first conference site may also be utilized. The wifi hotspot of the same subnet accesses the network to access the subnet accessed by the conference terminal of the first site.

The multicast address may also be an IP multicast address. Correspondingly, the mobile terminal sends the first multimedia data to the multicast group according to the IP multicast address, that is, the three-layer group is adopted. Broadcast mode for multicast. Specifically, the mobile terminal encapsulates the first multimedia data into N IP data packets, and the destination IP address of the N IP data packets is the IP multicast address, and further the N IP data packets are encapsulated into N Layer 2 data packets.

The first conference terminal can receive, because the first multimedia data is sent to the multicast group according to the multicast address, and the first conference terminal has joined the multicast group. To the first multimedia data.

The multimedia conference includes any one of an audio conference, a video conference, and an audio and video conference. Correspondingly, the first multimedia data includes at least one of audio data and video data, and the mobile terminal acquires the audio data by using a microphone, and collects the video data by using a camera.

The specific conference scenario may be that multiple users participate in the conference in the conference room where the first conference terminal is located, and one of the users wants to click on the conference running on the mobile terminal when speaking through the mobile terminal. a microphone button on the interface of the assistant application to indicate that the mobile terminal collects audio data to send to the multicast group, or when the user wants to display his own image in the conference, click a camera button on the interface to indicate The mobile terminal collects video data and sends the data to the multicast group. The conference assistant application acquires audio data or video by using an API (Application Programming Interface) provided by an operating system of the mobile terminal by using an API (Application Programming Interface) provided by an operating system of the mobile terminal when detecting a signal triggered by the user operation. data.

Steps S107-S108: The first conference terminal receives the first multimedia data sent by the mobile terminal by using the multicast group of the first conference site, and the first multimedia data and the first Two multi-media The volume data is combined, and the combined multimedia data is sent to the conference server.

Further, the conference server sends the multimedia data received from the first conference terminal to the conference terminal of the other conference sites in the at least two conference sites.

In a specific implementation, the first conference terminal receives, by using the multicast group of the first site, the N data packets that encapsulate the first multimedia data, and correspondingly, the first The conference terminal integrates the data included in the N data packets into the first multimedia data.

The first multimedia data may include audio data. Correspondingly, the specific implementation of step S108 includes: the first conference terminal collects audio data in the first multimedia data and the first conference terminal. The audio data (i.e., the audio data in the second multimedia data) is subjected to a mixing process.

The first multimedia data may include video data. Correspondingly, the specific implementation of step S107 includes: the first conference terminal collects video data in the first multimedia data and the first conference terminal. The video data (ie, the video data in the second multimedia data) is subjected to picture combining processing, such as combining into a multi-picture or picture-in-picture.

The first multimedia data may further include video data (hereinafter referred to as first video data) and audio data (hereinafter referred to as first audio data), and correspondingly, the first conference terminal may further The information carried in the header of the N data packets identifies an audio data packet including the audio data and a video data packet including the video data, and then integrates the audio data included in each audio data packet into the first audio data, and The video data included in each video data package is integrated into the first video data, and the first audio data and the first conference terminal collect audio data for mixing processing, and the first video data and the The video data collected by the first conference terminal is subjected to screen combination processing.

Since the length of the Layer 2 data packet carrying the audio data is often much smaller than the length of the Layer 2 data packet carrying the video data, the value of the data length field in the header of the Layer 2 data packet carrying the audio data is often much smaller than the data carrying the video data. The value of the data length field in the header of the Layer 2 packet, therefore, the first conference terminal can identify each data link layer packet according to the value of the data length field in the header of each data link layer packet. An audio data packet containing audio data and a video data packet containing video data. In addition, a data type field for indicating a type of the multimedia data encapsulated in the layer 2 data packet may be extended in a header of the layer 2 data packet, and different values are represented. Different types, such as audio and video. Correspondingly, the first conference terminal may further identify the audio data packet containing the audio data and the video data included in each data link layer data packet according to the value of the data type field in the packet header of each data link layer data packet. Video packet.

Each of the N data packets further includes a MAC address of the mobile terminal, and the MAC address of the mobile terminal may specifically carry a source MAC address of each of the N data packets. Fields can also be carried in other fields.

The first conference terminal may acquire information (such as the user's name, department, etc.) of the user of the mobile terminal according to the MAC address of the mobile terminal in the data packet from the mobile terminal, and The association information of the first multimedia data from the mobile terminal is sent to the conference server, and the name of the user is superimposed on the screen of the video data from the mobile terminal, and sent to the conference server. Or sending the name of the user to the conference server as part of the description information of the site where the user is located (for example, Bob of the Nanjing conference site is speaking); and the conference server may use the information of the user as the source. The association information of the first multimedia data of the mobile terminal is sent to the conference terminals of the other conference sites in the at least two conference sites. In a specific implementation, the first conference terminal may send a request message carrying a MAC address of the mobile terminal to an information server for managing user information on the network side, where the information server according to the saved MAC address and the Obtaining an identifier of the user of the user of the mobile terminal, such as an account of the user, and acquiring the identifier of the user according to the identifier of the user, such as the name, department, etc. of the user, and Returning the information of the user to the first conference terminal.

In a specific scenario, the first site may include M mobile terminals (M is a positive integer greater than or equal to 2), and the M mobile terminals collect M pieces of the first multimedia data, each of which is described. The first multimedia data is collected by a mobile terminal, and correspondingly, the conference terminal of the first site receives M×N data packets through the multicast group of the first site, and the M×N data packets Each of the data packets includes a MAC address of a sender that transmits each of the M x N data packets.

The conference terminal of the first site may identify a sender of each of the M×N data packets according to a MAC address of a sender in each of the M×N data packets. Will The data contained in the N data packets originating from the same mobile terminal is integrated into one piece of the first multimedia data.

If the mobile terminal performs multicasting through the Layer 3 multicast mode, the mobile terminal and the first conference terminal may be in the same subnet or may be in different subnets. Since the Layer 2 data packet (that is, the Layer 2 data packet encapsulating the Layer 3 data packet) passes through the gateway between the subnets, the value of the source MAC address field in the Layer 2 data packet is changed by the gateway, so The Layer 2 data packet received by the first conference terminal by using the multicast group includes the MAC address of the mobile terminal, and can be implemented in the following two manners.

Manner 1: The mobile terminal carries its own MAC address through a field in the transmitted Layer 2 data packet that is not changed by the gateway between the subnets.

Manner 2: The mobile terminal accesses the subnet where the first conference terminal is located before step S106. The mode 2 can ensure that the Layer 2 data packet sent by the mobile terminal can reach the first conference terminal without using a gateway between the subnets, and the value of the source MAC address field in the Layer 2 data packet is in the transmission process. The first conference terminal may determine that the MAC address carried in the source MAC address field in the received Layer 2 data packet is the MAC address of the mobile terminal. The mobile terminal may access the subnet accessed by the conference terminal of the first conference site by using the wifi hotspot accessing the network provided by the first conference terminal; or by utilizing the first conference The wifi hotspot of the same subnet accesses the network to access the subnet accessed by the conference terminal of the first site. In a specific implementation, the mobile terminal may be automatically added to the subnet where the first conference terminal is located when the triggering operation is detected, or may be added to the child where the first conference terminal is located according to manual operations of the user. network.

If the mobile terminal is multicast by the Layer 2 multicast mode, the mobile terminal needs to access the subnet accessed by the first conference terminal, that is, the mobile terminal and the first conference, before step S106. The terminal is in the same subnet, and the value of the source MAC address field in the Layer 2 data packet sent by the mobile terminal to the multicast group is not in the transmission process before the layer 2 data packet arrives at the first conference terminal. Changing, therefore, the first conference terminal can determine the source in the received layer 2 data packet The MAC address carried in the MAC address field is the MAC address of the mobile terminal.

The audio data in the first multimedia data sent by each mobile terminal to the multicast group is generally encoded in a G.711 format. Correspondingly, the first conference terminal sends audio data from different mobile terminals to different DSP channels of a DSP (digital signal processing) codec of the first conference terminal for decoding, and decoding the decoded Combining the audio data with the audio data collected by the microphone of the first conference terminal, and encoding the mixed audio data after the mixing process by using the DSP codec, and further, passing through the first conference terminal A media channel for transmitting audio data with the conference server transmits the encoded mix audio data to the conference server.

The video data in the first multimedia data sent by each mobile terminal to the multicast group may be unencoded original video data or encoded video data. Correspondingly, if the video data is unencoded original video data, the first conference terminal directly performs screen combination processing on original video data from each mobile terminal and video data collected by a camera of the first conference terminal. If the video data is encoded video data, the first conference terminal respectively decodes the encoded video data from each mobile terminal, and the decoded video data and the video data collected by the camera of the first conference terminal Perform screen combination processing.

According to the method provided by Embodiment 1 of the present invention, the conference terminal can use the mobile terminal at the site to collect the multimedia data of the conference site and send it to the conference server. The mobile terminal is often carried by the participants and can be easily moved, so that the multimedia data in the venue can be conveniently collected. In addition, according to the method provided by Embodiment 1 of the present invention, there is no need for signaling interaction between the conference terminal and the mobile terminal, and there is no need to establish a communication channel between the conference terminal and the mobile terminal, and the resource consumption of the conference terminal and the mobile terminal is relatively small. .

According to Embodiment 1 of the present invention, Embodiment 2 of the present invention provides a mobile terminal 500, where the mobile terminal 500 is located in a first conference site of at least two conference sites of a multimedia conference, and the multimedia conference is included The conference server and the conference system of the at least two conference terminals are configured, and each conference site is configured with a conference terminal, where the conference server is configured to send multimedia data of the conference terminal from the first conference site to the at least two The conference terminal of the other site in the site; the first site also includes a multicast group, and the conference terminal of the first site has joined the multicast group. As shown in FIG. 3A, the mobile terminal 500 includes an acquisition unit 501 and a transmission unit 502.

The collecting unit 501 is configured to collect the first multimedia data of the first site.

The sending unit 502 is configured to obtain the multicast address of the multicast group of the first site, and send the multicast group to the multicast group of the first site according to the obtained multicast address of the multicast group of the first site. The first multimedia data collected by the collecting unit 501, so that the conference terminal of the first site will receive the first multimedia data and the first part that are received by the multicast group of the first site. The second multimedia data of the first site collected by the conference terminal of the conference site is combined and sent to the conference server.

The sending unit 502 is specifically configured to: encapsulate the first multimedia data collected by the collecting unit 501 into N data packets, and send the N data packets to the multicast group of the first site, The destination address of each of the N data packets includes a multicast address of the multicast group of the first site, where N is a positive integer greater than or equal to 2. Each of the N data packets includes a media access control address of the mobile terminal, where the media access control address of the mobile terminal may be specifically carried in each of the N data packets. Source MAC address field.

As shown in FIG. 3B, the mobile terminal may further include an access unit 503, configured to access the mobile terminal to a subnet where the conference terminal is located. Specifically, the wifi hotspot provided by the conference terminal can access the network; or access the network through a wifi hotspot that is in the same subnet as the conference terminal.

The mobile terminal 500 described in Embodiment 2 of the present invention can be used to implement the method described in Embodiment 1 above.

According to the mobile terminal provided by Embodiment 2 of the present invention, the conference terminal can use the mobile terminal of the conference site to collect multimedia data of the conference site and send it to the conference server. The mobile terminal is often carried by the participants and can be easily moved, so that the multimedia data in the venue can be conveniently collected.

According to Embodiment 1 of the present invention, Embodiment 3 of the present invention provides a conference terminal 600. The conference terminal 600 is included in a conference system, and the conference system is configured to perform a multimedia conference between at least two conference sites. The conference system is further configured to include a conference server and a conference terminal of another conference site of the at least two conference sites, where the conference server is configured to be used from the first conference site. The multimedia data of the conference terminal is sent to the conference terminal of the other conference sites of the at least two conference sites; the first conference site further includes a multicast group and a mobile terminal that collects the first multimedia data of the first conference site. As shown in FIG. 4, the conference terminal 600 includes a data acquisition unit 601, a data processing unit 602, and a data transmission unit 603.

The data acquisition unit 601 is configured to collect the second multimedia data of the first site, add the first conference terminal to the multicast group of the first site, and pass the multicast group of the first site. Receiving the first multimedia data sent by the mobile terminal.

The data processing unit 602 is configured to combine the first multimedia data collected by the data acquiring unit 601 and the received second multimedia data. The multimedia conference includes any one of an audio conference, a video conference, and an audio and video conference. The data processing unit 602 is specifically configured to: use audio data in the first multimedia data and the second multimedia data. Performing a mixing process, and/or performing a screen combining process on the video data in the first multimedia data and the second multimedia data.

The data sending unit 603 is configured to send the multimedia data obtained by combining the data processing unit 602 to the conference server.

In a specific implementation, the first multimedia data received by the data acquiring unit 601 through the multicast group of the first site is encapsulated in N data packets, where each of the N data packets The destination address of the data packet includes the multicast address of the multicast group of the first site, where N is a positive integer greater than or equal to 2; correspondingly, the data obtaining unit 601 is further configured to use the N data packets. The included data is integrated into the first multimedia data.

The first site may include M mobile terminals, and the M mobile terminals collect M pieces of the first multimedia data, and each of the first multimedia data is collected by a mobile terminal, and the data acquiring unit 601: Receive, by using the multicast group of the first site, M×N data packets, where the M×N data packets Each of the data packets includes a media access control address of a sender that sends each of the M×N data packets, where M is a positive integer greater than or equal to 2; correspondingly, the data acquisition unit 601 is specific Identifying, according to the media access control address of the sender in each of the M×N data packets, a sender of each of the M×N data packets, which will originate from the same mobile The data contained in the N data packets of the terminal is integrated into one piece of the first multimedia data.

The conference terminal 600 described in Embodiment 3 of the present invention can be used to implement the method described in Embodiment 1 above.

The conference terminal according to the second embodiment of the present invention can use the mobile terminal at the site to collect the multimedia data of the conference site and send it to the conference server. The mobile terminal is often carried by the participants and can be easily moved, so that the multimedia data in the venue can be conveniently collected.

According to Embodiment 1 of the present invention, Embodiment 4 of the present invention provides a mobile terminal 1000. As shown in FIG. 5, the mobile terminal 1000 includes a processor 1010 and a memory 1020, wherein the processor 1010 and the memory 1020 between the two through the bus to complete communication.

The memory 1020 is configured to store computer operation instructions. The memory 1020 may include a high speed RAM memory and may also include a non-volatile memory such as at least one disk memory.

The processor 1010 is configured to execute computer operation instructions stored in the memory 1020. The processor 1010 may be a central processing unit (CPU) and is a computer core unit.

The processor 1010 executes the computer operation instructions to cause the mobile terminal 1000 to perform the operations performed by the mobile terminal in the method described in Embodiment 1 above.

According to the mobile terminal provided by Embodiment 4 of the present invention, the conference terminal can use the mobile terminal of the conference site to collect the multimedia data of the conference site and send it to the conference server. The mobile terminal is often carried by the participants and can be easily moved, so that the multimedia data in the venue can be conveniently collected.

According to Embodiment 1 of the present invention, Embodiment 5 of the present invention provides a conference terminal 2000, as shown in FIG. 6. As shown, the conference terminal 2000 includes a processor 2010 and a memory 2020 in which communication between the processor 2010 and the memory 2020 is accomplished by a bus.

The memory 2020 is configured to store computer operating instructions. The memory 2020 may include a high speed RAM memory and may also include a non-volatile memory such as at least one disk memory.

The processor 2010 is configured to execute computer operation instructions stored in the memory 2020. The processor 2010 may specifically be a central processing unit (CPU) and is a computer core unit.

The processor 2010 executes the computer operation instruction to cause the mobile terminal 2000 to perform the operations performed by the mobile terminal in the method described in Embodiment 1 above.

The conference terminal according to Embodiment 5 of the present invention can use the mobile terminal at the site to collect multimedia data of the conference site and send it to the conference server. The mobile terminal is often carried by the participants and can be easily moved, so that the multimedia data in the venue can be conveniently collected.

A person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium. The storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalents, improvements, etc., which are within the spirit and scope of the present invention, should be included in the protection of the present invention. Within the scope.

Claims

A conference system, the conference system is configured to perform a multimedia conference between at least two conference sites, where the conference system includes a conference server and at least two conference terminals, and each conference site is correspondingly configured with one conference terminal;

It is characterized in that

The first site of the at least two sites further includes a multicast group and a mobile terminal;

The mobile terminal is configured to acquire a multicast address of the multicast group of the first site, collect the first multimedia data of the first site, and perform multicast according to the multicast group of the first site. Sending the collected first multimedia data to the multicast group of the first site;

The conference terminal of the first site is configured to collect the second multimedia data of the first site, join the multicast group of the first site, and receive the mobile by using the multicast group of the first site. The first multimedia data sent by the terminal combines the first multimedia data with the second multimedia data, and sends the combined multimedia data to the conference server.
The conference system of claim 1 wherein:

After collecting the first multimedia data of the first site, the mobile terminal is specifically configured to encapsulate the collected first multimedia data into N data packets, and The multicast group of the first site sends the N data packets, where the destination address of each of the N data packets includes a multicast address of the multicast group of the first site, N is a positive integer greater than or equal to 2;

After receiving the N data packets by the multicast group in the first site, the conference terminal of the first site is specifically configured to integrate data included in the N data packets into the first Multimedia data.
The conference system according to claim 2, wherein the first site includes M mobile terminals, and the M mobile terminals collect M pieces of the first multimedia data, each of the first The multimedia data is collected by a mobile terminal, and the conference terminal of the first site receives M×N data packets through the multicast group of the first site, and each data in the M×N data packets The packet includes a media access control address of a sender that sends each of the M×N data packets, where the M is a positive integer greater than or equal to 2;

The conference terminal of the first site is specifically configured to identify each of the M×N data packets according to a media access control address of a sender in each of the M×N data packets. The sender of the data packet integrates the data contained in the N data packets originating from the same mobile terminal into one piece of the first multimedia data.
A conference system according to claim 3, wherein

The M mobile terminals of the first site are also used to access a subnet accessed by the conference terminal of the first site;

The sender's media access control address is carried in a source MAC address field of each of the M x N data packets.
A conference system according to claim 4, wherein

The M mobile terminals of the first site access the subnet accessed by the conference terminal of the first conference site by using the wifi hotspot access network provided by the conference terminal of the first conference site; or And accessing the subnet accessed by the conference terminal of the first conference site by using a wifi hotspot accessing network in the same subnet as the conference terminal of the first conference site.
The conference system according to any one of claims 1 to 5, wherein the multimedia conference comprises any one of an audio conference, a video conference, and an audio and video conference;

After receiving the first multimedia data and collecting the second multimedia data, the conference terminal of the first site is specifically configured to use the first multimedia data and the second The audio data in the multimedia data is subjected to a mixing process, and/or the video data in the first multimedia data and the second multimedia data are subjected to screen combining processing.
A method for implementing a multimedia conference, the method being applied to a conference system, the conference system is configured to perform a multimedia conference between at least two conference sites, where the conference system includes a conference server and at least two conference terminals, each conference site Corresponding to the configuration of a conference terminal, the first conference site further includes a multicast group and a mobile terminal, and the conference terminal of the first conference site joins the multicast group, the party The law includes:

Obtaining, by the mobile terminal, a multicast address of the multicast group of the first site;

The mobile terminal collects the first multimedia data of the first site, and sends the collected information to the multicast group of the first site according to the multicast address of the multicast group of the first site. The first multimedia data.
The method according to claim 7, wherein the transmitting the collected first multimedia to the multicast group of the first site according to the multicast address of the multicast group of the first site Volume data, including:

The mobile terminal encapsulates the collected first multimedia data into N data packets, and sends the N data packets to the multicast group of the first site, where the N data The destination address of each data packet in the packet includes the multicast address of the multicast group of the first site, and the N is a positive integer greater than or equal to 2.
The method of claim 8 wherein each of said N data packets includes a media access control address of said mobile terminal.
The method of claim 9, wherein the media access control address of the mobile terminal is carried in a source MAC address field of each of the N data packets;

Before the sending the first multimedia data to the multicast group of the first site, the method further includes: the mobile terminal accessing a subnet accessed by the conference terminal of the first site.
The method of claim 10, wherein the accessing the subnet to which the mobile terminal accesses the conference terminal of the first conference site comprises:

The mobile terminal accesses the network through the wifi hotspot provided by the conference terminal of the first conference site; or

The mobile terminal accesses the network through a wifi hotspot of the same subnet as the conference terminal of the first conference site.
A method for implementing a multimedia conference, the method being applied to a conference system, the conference system The system is configured to perform a multimedia conference between at least two conference sites, where the conference system includes a conference server and at least two conference terminals, and each conference site is configured with a conference terminal, where the first conference site further includes a multicast group and a mobile terminal, configured to collect the first multimedia data of the first site, where the method includes:

The conference terminal of the first site collects the second multimedia data of the first site, joins the multicast group of the first site, and receives the multicast group sent by the mobile terminal by using the multicast group of the first site. The first multimedia data combines the first multimedia data with the second multimedia data, and sends the combined multimedia data to the conference server.
The method according to claim 12, wherein the first multimedia data received by the conference terminal of the first site by the multicast group of the first site is encapsulated in N data packets, The destination address of each of the N data packets includes a multicast address of the multicast group of the first site, where N is a positive integer greater than or equal to 2;

Before the combining the first multimedia data and the second multimedia data, the conference terminal of the first site integrates the data included in the N data packets into the first A multimedia data.
The method of claim 13, wherein the first site comprises M mobile terminals, and the M mobile terminals collect M pieces of the first multimedia data, each of the first plurality of The media data is collected by a mobile terminal, and the conference terminal of the first site receives M×N data packets through the multicast group of the first site, and each data packet of the M×N data packets And a media access control address of a sender that sends each of the M×N data packets, where M is a positive integer greater than or equal to 2;

The conference terminal of the first site specifically identifies each of the M×N data packets according to a media access control address of a sender in each of the M×N data packets. The sender integrates the data contained in the N data packets originating from the same mobile terminal into one piece of the first multimedia data.
The method of any of claims 12-14, wherein the multimedia session It includes any of audio conferences, video conferences, and audio and video conferences;

The combining, by the conference terminal of the first site, the first multimedia data and the second multimedia data, the conference terminal of the first site, the first multimedia data and the conference The audio data in the second multimedia data is subjected to a mixing process, and/or the video data in the first multimedia data and the second multimedia data are subjected to screen combining processing.
A mobile terminal is located in a first conference site of at least two conference sites of a multimedia conference, and the multimedia conference is performed by a conference system including a conference server and at least two conference terminals, and each conference site is configured with a conference terminal. The first site also includes a multicast group, and the conference terminal of the first site has joined the multicast group; the mobile terminal includes an acquisition unit and a sending unit;

The collecting unit is configured to collect first multimedia data of the first site;

The sending unit is configured to acquire a multicast address of the multicast group of the first site, and send the multicast group to the first site according to the obtained multicast address of the multicast group of the first site Sending the first multimedia data collected by the collection unit.
The mobile terminal according to claim 16, wherein the sending unit is configured to package the first multimedia data collected by the collecting unit into N data packets, and The multicast group of the first site sends the N data packets, where the destination address of each of the N data packets includes a multicast address of the multicast group of the first site, and the N Is a positive integer greater than or equal to 2.
The mobile terminal of claim 17, wherein each of the N data packets includes a media access control address of the mobile terminal.
The mobile terminal according to claim 18, wherein the media access control address of the mobile terminal is carried in a source MAC address field of each of the N data packets;

The mobile terminal further includes an access unit, configured to access a subnet accessed by the conference terminal of the first conference site.
The mobile terminal according to claim 19, wherein the access unit is specifically configured to:

Accessing the network through the wifi hotspot provided by the conference terminal of the first conference site; or

The network is accessed through a wifi hotspot that is in the same subnet as the conference terminal of the first conference site.
A conference terminal, the conference terminal is included in a conference system, and the conference system is configured to perform a multimedia conference between at least two conference sites. The conference site where the conference terminal is located further includes a multicast group and a collection center. a mobile terminal of the first multimedia data of the conference site where the conference terminal is located; the first conference terminal includes: a data acquisition unit, a data processing unit, and a data sending unit;

The data acquisition unit is configured to collect the second multimedia data of the conference site where the conference terminal is located, and add the conference terminal to the multicast group of the conference site where the conference terminal is located, and pass the group of the conference site where the conference terminal is located. Receiving, by the broadcast group, the first multimedia data sent by the mobile terminal;

The data processing unit is configured to combine the first multimedia data collected by the data acquiring unit and the received second multimedia data;

The data sending unit is configured to send the multimedia data obtained by combining the data processing units to the conference server.
The conference terminal according to claim 21, wherein the first multimedia data received by the data acquisition unit through the multicast group of the conference site where the conference terminal is located is encapsulated in N data packets, wherein The destination address of each of the N data packets includes a multicast address of a multicast group of the site where the conference terminal is located, where N is a positive integer greater than or equal to 2;

The data acquiring unit is further configured to integrate data included in the N data packets into the first multimedia data.
The conference terminal according to claim 22, wherein the conference site where the conference terminal is located includes M mobile terminals, and the M mobile terminals collect M pieces of the first multimedia data, each of which is A multimedia data is collected by a mobile terminal, and the data acquiring unit receives M×N data packets through a multicast group of the conference site where the conference terminal is located, and each of the M×N data packets The packet includes a media access control address of a sender that transmits each of the M×N data packets, where M is a positive integer greater than or equal to 2;

The data acquiring unit specifically identifies, according to a media access control address of a sender in each of the M×N data packets, a sender of each of the M×N data packets, The data contained in the N data packets originating from the same mobile terminal is integrated into one piece of the first multimedia data.
The method according to any one of claims 21 to 23, wherein the multimedia conference comprises any one of an audio conference, a video conference, and an audio and video conference;

The data processing unit is specifically configured to perform mixing processing on the audio data in the first multimedia data and the second multimedia data, and/or, the first multimedia data and The video data in the second multimedia data is subjected to screen combination processing.