Disclosure of Invention
Aiming at the defects in the prior art, the invention provides the audio and video communication system and the method based on the browser, which can realize video call or voice chat through the browser or a webpage and are convenient for users to use.
In a first aspect, an audio and video communication system based on a browser comprises a first server, a second server and a plurality of computer devices supporting a WebRTC standard; wherein the computer equipment is provided with a browser;
the SDP information and the candidate information are transmitted between the computer devices through the second server to establish an audio and video transmission channel between the computer devices; audio and video communication is carried out between the computer devices through the established audio and video transmission channel;
the first server is used for being accessed by the computer equipment and obtaining the candidate information of the computer equipment.
Preferably, the number of the computer devices is two, and the two computer devices comprise a first computer device and a second computer device;
the method for establishing the audio and video transmission channel by the first computer equipment and the second computer equipment comprises the following steps:
the method comprises the steps that a first computer device creates a Peerconnection object based on a WebRTC standard, packages audio and video data and adds the packaged audio and video data into the Peerconnection object;
the first computer equipment calls a request method of the PeerConnection object to create a request SDP object for the request, and the request SDP object is sent to the second computer equipment through the second server;
when the second computer equipment receives the request SDP object, calling a response method of the PeerConnection object to create a response SDP object for response, and sending the response SDP object to the first computer equipment through the second server;
the first computer device saves the received response SDP object;
the first computer equipment acquires candidate information thereof through the first server and sends the candidate information to the second computer equipment through the second server;
the second computer equipment acquires candidate information thereof through the first server and sends the candidate information to the first computer equipment through the second server;
and establishing the audio and video transmission channel.
Preferably, the method for audio and video communication between computer devices through the established audio and video transmission channel includes:
the first computer equipment sends the audio and video stream to be transmitted to the second computer equipment;
and when the second computer equipment receives the audio and video stream, generating a MediaStream object for identifying the audio and video stream through the additive stream call interface of the PeerConnection object, and rendering the MediaStream object on the second computer equipment.
Preferably, the candidate information comprises an IP address of the computer device;
the method for acquiring candidate information by the computer equipment comprises the following steps:
the computer device accessing a first server;
and the first server sends the candidate information of the computer equipment to the computer equipment through the candidate client interface of the PeerConnection object.
Preferably, the first server is a stun server.
In a second aspect, a browser-based audio/video communication method includes the following steps:
the SDP information and the candidate information are transmitted between the computer devices through the second server to establish an audio and video transmission channel between the computer devices;
audio and video communication is carried out between the computer devices through the established audio and video transmission channel;
the computer equipment supports the WebRTC standard, and a browser is arranged on the computer equipment; the candidate information is obtained by the computer device accessing the first server.
Preferably, the number of the computer devices is two, and the two computer devices comprise a first computer device and a second computer device;
the transmitting the SDP information and the candidate information between the computer devices through the second server to establish the audio/video transmission channel between the computer devices specifically includes:
the method comprises the steps that a first computer device creates a Peerconnection object based on a WebRTC standard, packages audio and video data and adds the packaged audio and video data into the Peerconnection object;
the first computer equipment calls a request method of the PeerConnection object to create a request SDP object for the request, and the request SDP object is sent to the second computer equipment through the second server;
when the second computer equipment receives the request SDP object, calling a response method of the PeerConnection object to create a response SDP object for response, and sending the response SDP object to the first computer equipment through the second server;
the first computer device saves the received response SDP object;
the first computer equipment acquires candidate information thereof through the first server and sends the candidate information to the second computer equipment through the second server;
the second computer equipment acquires candidate information thereof through the first server and sends the candidate information to the first computer equipment through the second server;
and establishing the audio and video transmission channel.
Preferably, the audio and video communication between the computer devices through the established audio and video transmission channel specifically includes:
the first computer equipment sends the audio and video stream to be transmitted to the second computer equipment;
and when the second computer equipment receives the audio and video stream, generating a MediaStream object for identifying the audio and video stream through the additive stream call interface of the PeerConnection object, and rendering the MediaStream object on the second computer equipment.
Preferably, the candidate information comprises an IP address of the computer device;
the method for acquiring candidate information by the computer equipment comprises the following steps:
the computer device accessing a first server;
and the first server sends the candidate information of the computer equipment to the computer equipment through the candidate client interface of the PeerConnection object.
Preferably, the first server is a stun server.
According to the technical scheme, the audio and video communication system and the method based on the browser can realize video call or voice chat through the browser or a webpage, and are convenient for users to use.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby. It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
The first embodiment is as follows:
a browser-based audio-video communication system, see fig. 1, includes a first server, a second server, and a plurality of computer devices supporting WebRTC standard; wherein the computer equipment is provided with a browser;
the SDP information and the candidate information are transmitted between the computer devices through the second server to establish an audio and video transmission channel between the computer devices; audio and video communication is carried out between the computer devices through the established audio and video transmission channel;
the first server is used for being accessed by the computer equipment and obtaining the candidate information of the computer equipment.
Specifically, the WebRTC standard contains three APIs (i.e., application programming interfaces): GetUserMedia (camera and microphone access), PeerConnection (sending and receiving media), and DataChannels (sending non-media directly between browsers).
The audio and video communication system can realize video call or voice chat through a browser or a webpage, and is mainly used for establishing audio and video communication between a client supporting the WebRTC standard and a server or performing audio and video communication between other devices supporting the WebRTC standard. The system shortens the session connection time, accelerates the delivery of other OpenTok events, reduces audio/video delay, realizes more natural and relaxed conversation, and is convenient for users to use.
The audio and video communication system is suitable for the requirements of different scenes, and can provide diversified audio and video solutions. The audio and video communication system can realize functions of interactive wheat live broadcasting, multi-person audio and video conference, P2P audio and video calling, real-time live broadcasting, intelligent scheduling, interactive whiteboard, online education and the like, and can meet the requirements on audio and video services in the market. The audio and video communication system reduces the use threshold of users on the audio and video technology.
Preferably, the number of the computer devices is two, and the two computer devices comprise a first computer device and a second computer device;
referring to fig. 2, the method for establishing the audio/video transmission channel between the first computer device and the second computer device includes:
a first computer device (client a in fig. 2) creates a PeerConnection object (i.e., a local connection object) based on the WebRTC standard, then opens a local audio/video device, encapsulates audio/video data into a MediaStream, and adds the MediaStream to the PeerConnection object;
the method comprises the steps that a first computer device calls a request method (namely a CreateOffer method) of a PeerConnection object to create a request SDP object (namely a supply object) for a request, wherein relevant parameters of audio and video data are saved in the SDP object, the first computer device saves the request SDP object through a SetLocalDescription method of the PeerConnection object, and the request SDP object is sent to a second computer device (a client B in a figure 2) through a second server (a server B in a figure 2);
when the second computer device receives the request SDP object, saving the request SDP object through a SetRemoteDescription method of the PeerConnection object, calling a response method (namely a CreateAnswer method) of the PeerConnection object to create a response SDP object for response, saving the response SDP object through a SetLocalDescription of the PeerConnection object, and sending the response SDP object to the first computer device through the second server;
the first computer device saves the received response SDP object, and saves the response SDP object through the SetRemoteDescription of the PeerConnection object;
specifically, through the above steps, in the request/response flow of the SDP information, the first computer device and the second computer device have already created corresponding audio channels and video channels according to the SDP information, and start collecting candidate information.
The first computer equipment collects candidate information thereof through a first server (server A in figure 2) and sends the candidate information to second computer equipment through a second server;
the second computer equipment acquires candidate information thereof through the first server and sends the candidate information to the first computer equipment through the second server;
and establishing the audio and video transmission channel.
Specifically, the computer device sends the candidate information to the computer device of the opposite side through the second server, and when receiving the candidate information of the computer device of the opposite side, the computer device can store the candidate information by an addicCandidate method of the PeerConnection object, so that the audio and video transmission channel is established between the two computer devices, and the audio and video transmission channel is a P2P channel.
Preferably, the method for audio and video communication between computer devices through the established audio and video transmission channel includes:
the first computer equipment sends the audio and video stream to be transmitted to the second computer equipment;
when the second computer device receives the audio and video stream, a MediaStream object for identifying the audio and video stream is generated through an added stream callback interface (namely an OnAddStream callback interface) of the PeerConnection object, and the audio and video stream is rendered on the second computer device. Similarly, the second computer device sends the audio and video stream to the first computer device by adopting the method.
Preferably, the candidate information includes an IP address of the computer device, including but not limited to a local IP address, a public network IP address, a Relay server assigned address, and the like;
the method for acquiring candidate information by the computer equipment comprises the following steps:
the computer device accessing a first server;
the first server sends the candidate information of the computer equipment to the computer equipment through a candidate client interface (namely an OnIcAndidate interface) of the PeerConnection object, and informs the computer equipment.
Preferably, the first server is a stun server, for example, a test server provided by google and implementing the stun protocol (stun: stun.l.google.com: 19302). The second server is a Signal server.
Example two:
an audio and video communication method based on a browser comprises the following steps:
the SDP information and the candidate information are transmitted between the computer devices through the second server to establish an audio and video transmission channel between the computer devices;
audio and video communication is carried out between the computer devices through the established audio and video transmission channel;
the computer equipment supports the WebRTC standard, and a browser is arranged on the computer equipment; the candidate information is obtained by the computer device accessing the first server.
Preferably, the number of the computer devices is two, and the two computer devices comprise a first computer device and a second computer device;
the transmitting the SDP information and the candidate information between the computer devices through the second server to establish the audio/video transmission channel between the computer devices specifically includes:
the method comprises the steps that a first computer device creates a Peerconnection object based on a WebRTC standard, packages audio and video data and adds the packaged audio and video data into the Peerconnection object;
the first computer equipment calls a request method of the PeerConnection object to create a request SDP object for the request, and the request SDP object is sent to the second computer equipment through the second server;
when the second computer equipment receives the request SDP object, calling a response method of the PeerConnection object to create a response SDP object for response, and sending the response SDP object to the first computer equipment through the second server;
the first computer device saves the received response SDP object;
the first computer equipment acquires candidate information thereof through the first server and sends the candidate information to the second computer equipment through the second server;
the second computer equipment acquires candidate information thereof through the first server and sends the candidate information to the first computer equipment through the second server;
and establishing the audio and video transmission channel.
Preferably, the audio and video communication between the computer devices through the established audio and video transmission channel specifically includes:
the first computer equipment sends the audio and video stream to be transmitted to the second computer equipment;
and when the second computer equipment receives the audio and video stream, generating a MediaStream object for identifying the audio and video stream through the additive stream call interface of the PeerConnection object, and rendering the MediaStream object on the second computer equipment.
Preferably, the candidate information comprises an IP address of the computer device;
the method for acquiring candidate information by the computer equipment comprises the following steps:
the computer device accessing a first server;
and the first server sends the candidate information of the computer equipment to the computer equipment through the candidate client interface of the PeerConnection object.
Preferably, the first server is a stun server.
For the sake of brief description, the system provided by the embodiment of the present invention may refer to the corresponding content in the foregoing method embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.