CN107682657B - WebRTC-based multi-user voice video call method and system - Google Patents

Abstract

The invention discloses a multi-user voice and video call method based on WebRTC, which comprises the following steps: a user can simultaneously initiate a multi-channel call request by appointing a connection Room number Room ID and a Room size n; the users establish P2P connection by two users, and finally, multi-user video voice communication is realized; based on a WebRTC communication mechanism, the system has the characteristics of flexible operation, quick response and low delay; in the process of establishing the P2P connection, a series of signaling exchanges and SDP negotiations need to be performed; in the face of a complex network environment, mature NAT traversal technology can enable users in non-identical local area networks to directly communicate. The method is developed based on the Android platform, can be widely applied to various mobile devices, improves the applicability and flexibility, and is suitable for small-sized multi-user voice video calls.

Description

WebRTC-based multi-user voice video call method and system

Technical Field

The invention relates to the field of video call, in particular to a multi-user voice video call method and a multi-user voice video call system based on WebRTC.

Background

With the rapid development of internet technology and communication technology, the communication modes and communication contents of people are greatly enriched and developed. In the information era with faster and faster pace, the traditional text-based communication mode is low in efficiency and sometimes cannot accurately express the intention of people. Therefore, communication systems supporting voice and video are increasingly prevalent. For early web instant messaging, it is still necessary to download bulky and insecure plug-ins. But the WebRTC makes up the defects of the traditional instant messaging.

WebRTC, Web Real-Time Communication, is a technology that supports a Web browser to perform Real-Time voice and video calls. Its immediate application is to allow developers to implement video calls or other point-to-point data transfers. WebRTC has a full set of audio-video solutions and the code is open-source. Furthermore, WebRTC is also fully platform supported. The method is not only limited to a webpage end, but also provides an interface used by mobile development. This provides a basis for secondary development of mobile-end applications.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide a multi-user voice and video call method based on WebRTC, which abandons the traditional streaming media communication mode, reduces communication delay, deals with complex network environment, improves user experience, enriches user communication forms and reduces system maintenance cost; the method is simple and flexible to operate, has high information transmission and response speed, and is suitable for most mobile terminals.

Another object of the present invention is to provide a WebRTC-based multi-user voice and video call system.

The purpose of the invention is realized by the following technical scheme:

a multi-user voice and video call method based on WebRTC comprises the following steps:

step 1, a first user appoints to connect a Room number Room ID and a Room size n and initiates n-1 paths of communication;

step 2, the first user collects network information and local session information, and sends the connection data to a server to wait for a receiving end to establish connection with the server;

step 3, the second user appoints the same Room number, Room ID and Room size, n, initiates n-1 calls, and simultaneously the second user collects network information and local session information as connection data;

step 4, the second user selects one of the calls as a receiving end, responds to the call request of the first user and establishes connection with the first user; meanwhile, the rest n-1-1 paths of calls wait for a new receiving end to establish connection with the new receiving end;

and 5, repeating the steps 3 and 4, respectively establishing connection between the third user and the first user and between the third user and the second user, and circulating in sequence to finally realize interconnection and intercommunication between the n clients.

The step 1 specifically comprises the following steps: a first user appoints to connect a Room number Room ID and a Room size n, and the Room number Room ID and the Room size n are used as an initiating end of n-1 paths of communication, and the first user initiates the communication and waits for a receiving end to establish connection with the receiving end; the first user forms a Client which is identified by a unique Client Id; each call is called an Instance, is identified by a unique Instance ID, is the minimum unit of P2P connection, and starts to acquire local audio and video data according to a specified coding format.

The step 2 specifically comprises the following steps: the first user collects network information through an NAT penetration technology in each path of session Instance to be used for cross-network segment communication; and meanwhile, collecting local media description, sending the collected network information and the local media description to a server, transferring the network information and the local media description by the server, and waiting for the receiving end to obtain the network information and the local media description.

The local media description comprises relevant parameters of the audio and video.

The step 3 specifically comprises the following steps: the second user obtains connection data from the server and the number m of clients currently waiting for connecting the user by specifying the Room number Room ID and the Room size n (at present, since only one Client initiates a request, m is 1), and simultaneously collects network information and local media description information.

The step 4 specifically comprises the following steps: randomly selecting m instances as receiving ends from the local n-1 instances by the second user Client, responding to the m clients, and establishing P2P connection; meanwhile, the rest n-1-m instances are used as an initiating end to initiate a call request to wait for a new receiving end to establish connection with the new receiving end.

The step 5 specifically comprises the following steps: repeating the step 3 and the step 4, the third user Client firstly acquires the connection data and the number m (m is 2) of the current user clients waiting to be connected, respectively responding to the first and second user clients by the local connection data, and establishing P2P connection with the first and second user clients; by analogy, after the n user clients establish connection through the same Room number Room ID and the same Room size n, the n users are interconnected and intercommunicated.

The other purpose of the invention is realized by the following technical scheme:

a multi-user voice and video call system based on WebRTC comprises a local audio and video stream acquisition module, a P2P connection management module and a multi-user management module; wherein

The local audio and video stream acquisition module is responsible for acquiring and encoding local audio and video streams; the client starts a camera and a microphone, acquires audio and video streams through a specified coding format, and creates an audio track and a video track;

the P2P connection management module is responsible for signaling interaction between the two clients and further establishes WebRTC connection; each P2P connection is managed by an Instance, with a unique Instance ID;

the multi-person management module is responsible for coordinating the organization and management of the n-1 calls in the multi-person environment; in the current Client, n-1 instances can acquire the number m of users who have added into a room in the current situation by requesting a server, wherein m is less than n; then, in n-1 calls in the local Client, namely in n-1 instances, m calls are randomly selected as receiving ends connected by P2P, and simultaneously the following conditions are met: for each Client, only one of the instances that responds; the remaining n-1-m paths of calls serve as new initiators to wait for the receiving end to establish a connection with them.

Compared with the prior art, the invention has the following advantages and beneficial effects:

1. the Android operating system is widely applied, and is very popular not only in a mobile phone terminal but also in a tablet personal computer and wearable equipment. According to the method, the WebRTC is used for developing the Android application, so that the communication means of the Android are enriched, and the WebRTC is well popularized.

2. NAT technology, that is, network address translation technology, is a technology for translating an internal network private IP address into an external network address that can be legally propagated over the Internet. Due to the scarcity of public network IP addresses, most computers are in a network environment behind a NAT. However, for the WebRTC protocol, NAT can cause it to fail in an environment across network segments. Therefore, in order to solve the above problem, the present invention needs to achieve NAT traversal, which is a technique for freely communicating between the outside and the inside of the NAT, and preferably solves the above technical problem by means of NAT traversal. The STUN protocol or TURN protocol can solve most of NAT traversal problems well. The ICE protocol is a comprehensive NAT traversal solution formed by combining two parts, the STUN protocol and the TURN protocol.

The multimedia session description SDP is mainly applied to session initialization and signaling interaction between a client and a client in WebRTC. The two sides send the collected SDP information to the server, and then the server transfers the SDP information to the other side. This is done by an offer/answer operation to the PeerConnection object.

The WebSocket protocol is an instant messaging protocol. The method is essentially Socket connection established on a TCP protocol, and encapsulation is carried out on an application layer, so that an interface is simplified and an interface is called. The WebSocket protocol can enable a full-duplex high-speed data channel to be established between the client and the server. During the communication, data is transmitted using text-based messages. It also has great advantages in terms of transmission stability and transmission data amount compared to polling and long connections.

And signaling is applied to the process of coordinating communication. In order to establish WebRTC communication, a series of signaling interactions are required between the client and the client. Although the WebRTC protocol can be applied to enable two hosts in a network to communicate directly, i.e., P2P. But this does not mean that WebRTC does not require a server. In the process of establishing a channel for data transmission, a server must participate. And the signaling plays this role.

2. The invention realizes multi-user voice video call based on the Android platform of WebRTC, and the users participating in the call establish P2P connection pairwise, so that an expensive and complicated streaming media server is abandoned, and the maintenance cost is reduced.

3. The invention has the same status among all users and flexible mechanism for joining and leaving the communication.

4. The technical scheme of the invention has the characteristics of low delay, simple and flexible operation, network segment crossing and suitability for the Android platform. In a small-scale call scene, the method has good user experience.

Drawings

Fig. 1 is a schematic diagram of a multi-person communication scheme.

Fig. 2 is a flow chart of P2P connection establishment.

Fig. 3 is a local audio video stream capture flow diagram.

Fig. 4 is a schematic diagram of a multi-person call management structure.

Detailed Description

The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited thereto.

A multi-user voice and video call method based on WebRTC specifically comprises the following steps:

step 1, a first user appoints to connect a Room number Room ID and a Room size n, and initiates n-1 paths of communication.

And 2, the user collects the network information and the local session information, and sends the connection data to the server to wait for the receiving end to establish connection with the server.

And 3, the second user appoints the same Room number, Room ID and Room size, n, and initiates n-1 calls. And meanwhile, collecting network information and local session information as connection data.

And 4, the second user selects one of the calls as a receiving end, responds to the call request of the first user and establishes connection with the first user. While the remaining n-1-1 calls wait for a new receiver to establish a connection with it.

And 5, repeating the step 3 and the step 4, and finally realizing the interconnection and intercommunication between the plurality of clients by establishing … … the connection between the third user and the first user and the connection between the third user and the second user respectively.

Further, the step 1 is further specifically: the first user specifies to connect the Room number Room ID to the Room size n and initiates an n-1 call. At this time, each call is designated as the originating end of the P2P connection, and waits for the receiving end to establish a connection with it. The user constitutes a Client identified by a unique Client Id. Each call is called an Instance, identified by a unique Instance ID, and is the smallest unit of P2P connection. The Instance can be used as an initiating end or a receiving end, and the multi-person management module is designated as the role of the initiating end or the receiving end. Meanwhile, according to the specified coding format, the camera and the microphone are started to collect local audio and video data. And loading a renderer to display the local video data.

Further, the step 2 is further specifically: in each Instance call: and starting to communicate with the punching server, and acquiring network information such as the IP address of the local public network and port information through the NAT traversal technology. And saved as an ICECandidate instance, used for cross-network segment communications. And meanwhile, local media description information SDP, such as related parameters of audio and video, is collected. The two pieces of information need to call setlocalciccondidate () and setlocalddp () methods to save to the local PeerConnection instance; and meanwhile, an HTTP POST request is sent to the server, the two parts of data are stored in the server, and the data are waited for being acquired by a receiving end.

Further, the step 3 is further specifically: the second user Client initiates n-1 calls, namely n-1 instances, by appointing the connection Room number Room ID and the Room size n. Firstly, acquiring the number m of user clients which have joined the room currently. Since only the first user joins the room at this time, m is 1. And according to the specified encoding format, the user starts a camera and a microphone to acquire local audio and video data. And loading a renderer to display the local video data. Simultaneously accessing a punching server to acquire network information; local video information is collected. These two pieces of information are stored as an ICECandidate Instance and an SDP Instance, and are stored as local connection data in the Peerconnection Instance of each Instance.

Further, the step 4 is further specifically: and the second user Client randomly selects m instances from the local n-1 instances to designate as a receiving end role. And respectively responding to the m clients by using the local connection data, and acquiring the connection data stored in the server by the other party so as to establish P2P connection with one Instance in each Client. Then, in the n-1 calls of the second user, the n-1-m lines which do not respond to the receiving end, namely the remaining n-1-m instances, are selected. These instances are respectively designated as the role of the sender. And sending the collected local connection data to the server in the same way, and waiting for the receiving end to establish connection with the server.

Further, the step 5 is further specifically: and repeating the step 3 and the step 4. The third user Client similarly obtains the number m of clients that have joined the room, where m is 2. After the local connection data is collected successfully, the Client randomly selects m instances to respectively respond to one Instance in the m clients, so that the clients respectively establish P2P connection. Similarly, when n users, namely n clients, establish connection successively through the same connection Room number Room ID and Room size n, the n users will be interconnected and intercommunicated with each other.

As shown in fig. 1 and 4, the multi-user voice and video call method based on WebRTC is implemented based on a multi-user voice and video call system based on WebRTC, and the multi-user voice and video call system based on WebRTC includes the following modules:

and the local audio and video stream acquisition module is responsible for acquiring the local audio and video stream, coding and the like. As shown in fig. 3, the client starts a camera and a microphone, and obtains audio/video stream AudioSource/VideoSource by specifying a coding format. And creates an audio track and a video track AudioTrack/VideoTrack. These two tracks need to be stored in the same MediaStream instance. Finally, the MediaStream instance is associated with the PeerConnection instance.

The P2P is connected to the management module and is responsible for signaling interaction between the two clients, thereby establishing the WebRTC connection. Each P2P connection is managed by an Instance, with a unique Instance ID. As in fig. 2, establishing a P2P connection requires the following steps:

the first step is as follows: and initiating a call request to the server by the initiating terminal. After the server checks that the Room number of the connection Room and the related parameters are correct, the origin Instance will create a PeerConnection Instance. The PeerConnection Instance is the core Instance implementing P2P and is also a key part of Instance, holding all information about the connection.

The second step is that: the originating end Instance collects and saves local SDP information.

The third step: the SDP information is sent to the server-this action is called Offer.

The fourth step: the receiving end accesses the server by the same parameter to obtain the SDP information of the initiating end and collects the local SDP information. Likewise, these pieces of information are stored in the Peerconnection Instance of the receiving end Instance.

The fifth step: the receiving end Instance responds with the local SDP message to the originating end-this action is called answer.

And a sixth step: when the initiating terminal and the receiving terminal respectively have the SDP information of the opposite side, the P2P connection based on WebRTC is established accordingly.

The receiving end Instance acquires the connection data from the server, and can request through Http Post, which is a "pull" action. The receiving end responds to the initiating end, and pushes data through WebSocket actively, which is an action of "pushing". If the two ends are respectively in different networks, the STUN/TURN/ICE server is firstly accessed to obtain the information such as the IP address and the port exposed on the Internet, and the information is stored as an ICE Candidate example as a part of signaling interaction, so that the communication is directly established between the hosts in different network environments.

And the multi-person management module is responsible for coordinating the organization and management of the n-1 calls in the multi-person environment. In the current Client, there are n-1 instances. By requesting from the server, the number m of users (m < n) who have joined the room in the current context can be obtained. And then randomly selecting m channels of calls as receiving ends connected with P2P from n-1 channels of calls in the local Client, namely n-1 instances. And it is necessary to ensure that: for each Client, only one of the instances responds. The remaining n-1-m paths of calls serve as new initiators to wait for the receiving end to establish a connection with them. The module is responsible for assigning the role of each Instance in the Client, and ensuring that the P2P connection is established orderly among a plurality of users. When the P2P connection of a certain path is interrupted, the module will reclaim the connection resources and reset the role, thereby ensuring the exit of the reconnection of the user.

The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims (6)

1. A multi-user voice and video call method based on WebRTC is characterized by comprising the following steps:

step 1, a first user appoints to connect a Room number Room ID and a Room size n and initiates n-1 paths of communication; the method specifically comprises the following steps: a first user appoints a room number RoomID and a room size n to be connected, the room number RoomID and the room size n are used as an initiating end of n-1-path communication, communication is initiated, and a receiving end is waited to establish connection with the receiving end; the first user forms a Client which is identified by a unique Client Id; each call is called an Instance, is identified by a unique Instance ID and is the minimum unit of P2P connection, and simultaneously starts to acquire local audio and video data according to a specified coding format;

step 2, the first user collects network information and local session information, and sends the connection data to a server to wait for a receiving end to establish connection with the server; the method specifically comprises the following steps: the first user collects network information through an NAT penetration technology in each path of session Instance to be used for cross-network segment communication; meanwhile, local media description is collected, and the collected network information and the local media description are sent to a server, transferred by the server and waited for being obtained by a receiving end;

step 3, the second user appoints the same Room number, Room ID and Room size, n, initiates n-1 calls, and simultaneously the second user collects network information and local session information as connection data;

step 4, the second user selects one of the calls as a receiving end, responds to the call request of the first user and establishes connection with the first user; meanwhile, the rest n-1-1 paths of calls wait for a new receiving end to establish connection with the new receiving end;

and 5, repeating the steps 3 and 4, respectively establishing connection between the third user and the first user and between the third user and the second user, and circulating in sequence to finally realize interconnection and intercommunication between the n clients.

2. The WebRTC-based multi-person voice-video call method according to claim 1, wherein the local media description comprises parameters related to audio and video.

3. The WebRTC-based multi-user voice and video call method according to claim 1, wherein the step 3 specifically comprises: and the second user acquires the connection data from the server and the number m of clients currently waiting for connecting the user by specifying the number of the connected rooms from ID and the size n of the Room, and simultaneously acquires the network information and the local media description information.

4. The WebRTC-based multi-user voice and video call method according to claim 1, wherein the step 4 is specifically: randomly selecting m instances as receiving ends from the local n-1 instances by the second user Client, responding to the m clients, and establishing P2P connection; meanwhile, the rest n-1-m instances are used as an initiating end to initiate a call request to wait for a new receiving end to establish connection with the new receiving end.

5. The WebRTC-based multi-user voice and video call method according to claim 1, wherein the step 5 specifically comprises: repeating the step 3 and the step 4, the third user Client firstly acquires the connection data and the number m of the current user clients waiting to be connected, respectively responds to the first and second user clients with the local connection data, and establishes P2P connection with the first and second user clients; by analogy, after the n user clients establish connection through the same Room number Room ID and the same Room size n, the n users are interconnected and intercommunicated.

6. A many people voice video conversation system based on WebRTC, its characterized in that: the system comprises a local audio and video stream acquisition module, a P2P connection management module and a multi-person management module; wherein

The local audio and video stream acquisition module is responsible for acquiring and encoding local audio and video streams; the client starts a camera and a microphone, acquires audio and video streams through a specified coding format, and creates an audio track and a video track;

the P2P connection management module is responsible for signaling interaction between the two clients and further establishes WebRTC connection; each P2P connection is managed by an Instance, with a unique Instance ID;

the multi-person management module is responsible for coordinating the organization and management of the n-1 calls in the multi-person environment; in the current Client, n-1 instances can acquire the number m of users who have added into a room in the current situation by requesting a server, wherein m is less than n; then, in n-1 calls in the local Client, namely in n-1 instances, m calls are randomly selected as receiving ends connected by P2P, and simultaneously the following conditions are met: for each Client, only one of the instances that responds; the remaining n-1-m paths of calls serve as new initiators to wait for the receiving end to establish a connection with them.

Images