CN112738140B - Video stream transmission method, device, storage medium and equipment based on WebRTC - Google Patents
Video stream transmission method, device, storage medium and equipment based on WebRTC Download PDFInfo
- Publication number
- CN112738140B CN112738140B CN202110353203.9A CN202110353203A CN112738140B CN 112738140 B CN112738140 B CN 112738140B CN 202110353203 A CN202110353203 A CN 202110353203A CN 112738140 B CN112738140 B CN 112738140B
- Authority
- CN
- China
- Prior art keywords
- browser
- streaming
- sdp
- channel
- webrtc
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1101—Session protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
Abstract
The invention discloses a video stream transmission method, a device, a storage medium and equipment based on WebRTC, which comprises the following steps: the browser establishes a session between the browser and the streaming media server through a first Session Description Protocol (SDP) negotiation between the browser and a webpage instant messaging (WebRTC) gateway, and establishes a first streaming channel; the browser acquires a subscription list of video streams, and if the subscription list indicates that one video stream is subscribed, the browser receives the video stream sent by the streaming media server through the first streaming channel; if the subscription list indicates that a plurality of video streams are subscribed, the browser creates a second stream channel based on the session through negotiation with a second SDP between the browser and the WebRTC gateway; and the browser receives the plurality of video streams sent by the streaming media server through the second streaming channel, so that the browser can simultaneously preview the plurality of streams based on the WebRTC.
Description
Technical Field
The present invention relates to streaming media transmission technologies, and in particular, to a video streaming method, apparatus, storage medium, and device based on WebRTC.
Background
At present, the demand for viewing videos through a browser without plug-ins is increasing, and the conventional mainstream plug-in-free video playing method generally adopts an HLS (HTTP Live Streaming, HTTP-based adaptive bitrate Streaming media transport protocol) technology or a WebRTC (Web Real-Time Communication, Web instant messaging) technology.
However, video playback using HLS technology is poor in real-time performance, and video screen delay is large (about 10 seconds), which is unacceptable for real-time video preview. While the video playing scheme using WebRTC has low delay, no solution is given to playing multiple video streams simultaneously.
Disclosure of Invention
The present disclosure provides a method, an apparatus, a storage medium, and a device for transmitting a WebRTC-based video stream, so as to at least solve the above technical problems in the prior art.
The first aspect of the present invention provides a video streaming method based on WebRTC, which is applied to a streaming media system, where the streaming media system includes: browser, webpage instant messaging WebRTC gateway and streaming media server, the method includes:
after the initialization operation of the browser is completed, establishing a session between the browser and the streaming media server through a first Session Description Protocol (SDP) negotiation between the browser and the WebRTC gateway, and establishing a first streaming channel;
the browser acquires a subscription list of video streams and sends the subscription list to the streaming media server through the WebRTC gateway;
if the subscription list indicates to subscribe to a video stream, the browser receives the video stream sent by the streaming media server through the first streaming channel;
if the subscription list indicates that a plurality of video streams are subscribed, the browser creates a second stream channel based on the session through negotiation with a second SDP between the browser and the WebRTC gateway;
and the browser receives the plurality of video streams sent by the streaming media server through the second streaming channel.
Wherein, the creating a session between the browser and the streaming media server through a first session description protocol SDP negotiation with the WebRTC gateway and establishing a first streaming channel includes:
the browser receives a first SDP notification offer message sent by the WebRTC gateway, wherein the first SDP offer message carries the connection information of the streaming media server and a first synchronous source identifier SSRC; the connection information of the streaming media server is acquired by the WebRTC gateway from the streaming media server, and the first SSRC is determined by the WebRTC gateway;
the browser generates a first SDP answer message according to the first SDP offer message and sends the first SDP answer message to the WebRTC gateway to complete the first SDP negotiation;
and the browser establishes a session with the streaming media server according to the connection information, calls an Application Program Interface (API) to establish a first streaming channel, and successfully establishes the first streaming channel when the session is established.
Wherein the first SDP answer message carries the first SSRC, and after the first SDP negotiation is completed, the method further includes:
and sending a first SDP negotiation result to the streaming media server through the WebRTC gateway, wherein the first SDP negotiation result comprises the first SSRC, so that when the streaming media server sends the video stream to the browser through the first streaming channel, the first SSRC is used as the SSRC of the video stream.
Wherein the browser creates a second streaming channel based on the session through a second SDP negotiation with the WebRTC gateway, including:
the browser receives a second SDP offer message sent by the WebRTC gateway;
the browser generates a second SDP answer message according to the second SDP offer message and sends the second SDP answer message to the WebRTC gateway to complete the second SDP negotiation;
and the browser calls an API according to the second SDP offer message to create a second flow channel based on the session.
After the browser acquires the subscription list of the video stream, the method further comprises the following steps:
if the subscription list indicates that a plurality of video streams are subscribed, the browser creates a playing channel corresponding to each video stream, and each playing channel corresponds to a second SSRC;
the subscription list sent by the browser to the streaming media server through the gateway includes a second SSRC corresponding to each video stream, so that when the streaming media server sends the plurality of video streams to the browser through the second streaming channel, a corresponding second SSRC is determined for each video stream, and the second SSRC is used as the SSRC of the video stream.
When the browser acquires the subscription list of the video stream again, the method further comprises the following steps:
if the number of the video streams subscribed at this time is larger than that of the video streams subscribed at the last time, the browser reestablishes a second stream channel through negotiation with a second SDP between the browser and the WebRTC gateway;
if the number of the video streams subscribed at this time is less than or equal to the number of the video streams subscribed at the last time, the browser receives a plurality of video streams subscribed at this time sent by the streaming media server through the established stream channel;
the created flow channel is a first flow channel or a second flow channel.
The initialization operation of the browser comprises the following steps:
establishing a WebSocket connection between the browser and the WebRTC gateway;
and when the WebSocket connection is established, the initialization operation of the browser is completed.
Another aspect of the present invention provides a WebRTC-based video streaming apparatus, which is applied to a browser, and includes:
the initialization module is used for executing initialization operation of the browser;
the SDP module is used for establishing a session between the browser and the streaming media server through a first SDP negotiation with the WebRTC gateway after the initialization operation is completed, and establishing a first streaming channel;
the subscription module is used for acquiring a subscription list of the video stream and sending the subscription list to the streaming media server through the WebRTC gateway;
a data monitoring module, configured to receive, through the first streaming channel, a video stream sent by the streaming media server when the subscription list indicates to subscribe to the video stream;
the SDP module is further configured to create a second streaming channel based on the session through a second SDP negotiation with the WebRTC gateway when the subscription list indicates that a plurality of video streams are subscribed;
the data monitoring module is further configured to receive the plurality of video streams sent by the streaming media server through the second streaming channel.
Yet another aspect of the present invention provides a computer-readable storage medium storing a computer program for executing the WebRTC-based video streaming method.
In another aspect, the present invention provides an electronic device, including:
a processor, a memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory and execute the instructions to implement the WebRTC-based video streaming method.
The session connection and the first flow channel between the browser and the streaming media server are established through the first SDP negotiation once, if the single flow is subscribed, the session connection and the first flow channel are directly transmitted through the first flow channel, the time of the SDP negotiation is saved, and the time delay in the streaming transmission process is reduced; if the subscription is multi-stream, a second stream channel needs to be established through negotiation of a second SDP, the second stream channel is established based on the established session, the session connection is maintained all the time, the process of session establishment between the browser and the streaming media server during each subscription is omitted, the time delay in the stream transmission process is reduced, meanwhile, based on the SSRC negotiation mechanism between the browser and the streaming media server, the browser can be ensured to accurately identify and receive a plurality of streams from one stream channel, and multi-stream simultaneous playing is realized. In addition, the first flow channel and the second flow channel are both established between the browser and the streaming media server, and transfer is not needed, so that the time delay of video stream transmission is further reduced.
Drawings
FIG. 1 illustrates a flowchart of a WebRTC based video streaming method, according to an embodiment;
FIG. 2 illustrates a schematic diagram of a WebRTC based video streaming appliance, according to an embodiment;
fig. 3 is a schematic diagram illustrating a WebRTC-based video streaming process according to an embodiment.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to enable a browser to support low-latency multi-stream playing based on WebRTC technology, as shown in fig. 1, an example of the present disclosure provides a method for transmitting a WebRTC-based video stream, where the method is applicable to a streaming media system, and the streaming media system includes, based on WebRTC technology: browser, WebRTC gateway and streaming media server, this method includes:
step 101, after the initialization operation of the browser is completed, a Session between the browser and the streaming media server is established through negotiation with a first SDP (Session Description Protocol) between the browser and the WebRTC gateway, and a first streaming channel is established.
Firstly, the browser needs to perform initialization operations, which mainly include: and establishing connection between the browser and the WebRTC gateway. When the connection is established, the initialization operation of the browser is completed. In one example, the connection created between the browser and the WebRTC gateway is a Websocket (web socket) connection, and based on the Websocket connection, the WebRTC gateway may actively push a message to the browser, and the browser may also actively send a message to the WebRTC gateway.
Based on the Websocket connection, the browser and the WebRTC gateway conduct a first SDP negotiation. In this example, the creation of a first streaming channel, which can be used for the transmission of one video stream, can be performed by a first SDP negotiation.
The procedure of the first SDP negotiation is as follows:
the browser receives a first SDP offer message sent by the WebRTC gateway, wherein the first SDP offer message carries connection information of the streaming media server and a first SSRC (synchronous source identifier); the connection information of the streaming media server is acquired by the WebRTC gateway from the streaming media server, and the first SSRC is determined by the WebRTC gateway;
the browser generates a first SDP answer message according to the first SDP offer message and sends the first SDP answer message to the WebRTC gateway to complete the first SDP negotiation;
and the browser establishes a session with the streaming media server according to the connection information, calls an API (Application Programming Interface) to establish a first streaming channel, and successfully establishes the first streaming channel when the session is established.
In the first SDP offer message: the connection information of the streaming media server includes: and the intranet outer IP and the intranet outer port number are used for establishing a session with the streaming media server based on the browser. After the first SSRC is determined by the WebRTC gateway, it is configured in the first SDP offer message to perform the first SDP negotiation with the browser.
And 102, the browser acquires a subscription list of the video stream and sends the subscription list to the streaming media server through the WebRTC gateway.
After the first streaming channel is created, the browser may send a subscription request of the video stream to the streaming media server at any time, where the subscription request carries a subscription list, which may indicate identification information of the video stream that the browser needs to preview, for example, in a video conference, a total of A, B, C participants is present, and then the identification information of the video stream may be an identity A, B, C of the participant; each participant has a video conference client, and the identification information of the video stream may also be the identification of each video conference client.
Multiple video streams may be indicated in a subscription list, and one participant (video conference client) corresponds to one video stream in a video conference, and thus one participant (video conference client) is the video source that generates one video stream.
Step 103, if the subscription list indicates to subscribe to a video stream, the browser receives the video stream sent by the streaming media server through the first streaming channel.
In step 101, the first SDP answer message generated by the browser carries the first SSRC, and accordingly, after the first SDP negotiation, the browser may send the first SDP negotiation result to the streaming media server through the WebRTC gateway, where the first SDP negotiation result includes the first SSRC, and thus, after the streaming media server receives the subscription list, the streaming media server may directly send the video stream to the browser through the first stream channel, and use the first SSRC as the SSRC of the video stream.
It should be noted that, after the first channel is created, if a single stream is subscribed immediately, the created first stream channel may be directly multiplexed, which may reduce the delay.
And step 104, if the subscription list indicates that a plurality of video streams are subscribed, the browser creates a second stream channel based on the session through a second SDP negotiation with the WebRTC gateway.
The second SDP negotiation procedure includes:
the browser receives a second SDP offer message sent by the WebRTC gateway;
the browser generates a second SDP answer message according to the second SDP offer message and sends the second SDP answer message to the WebRTC gateway to complete the second SDP negotiation;
and the browser calls an API to create a second flow channel according to the second SDP offer message.
It should be noted that, since the session between the browser and the streaming server is already created at the first SDP negotiation, the second streaming channel is already created when the API is called here to create the second streaming channel. After the creation of the second flow path is completed, the first flow path is then released.
The description mode of the description information of the video stream in the second SDP offer message is a description mode indicated by a multi-stream playing scheme or a single-stream playing scheme supported by the browser, and the multiple video stream playing mode supported by the browser is determined by the WebRTC gateway according to the description information of the browser. The description information of the browser can be acquired by the WebRTC gateway in the establishing process of the Websocket connection of the initialization operation.
In one example, after the browser obtains the subscription list of the video stream:
and if the subscription list indicates that a plurality of video streams are subscribed, the browser creates a playing channel corresponding to each video stream, and allocates a corresponding second SSRC for each playing channel, and correspondingly, the subscription list sent to the streaming media server by the browser through the gateway comprises the second SSRC corresponding to each video stream, so that the streaming media server uses the corresponding second SSRC as the SSRC of the video stream when sending the plurality of video streams to the browser through the second streaming channel.
And 105, the browser receives the plurality of video streams sent by the streaming media server through the second streaming channel.
After the second streaming channel is created, the browser can directly receive the plurality of video streams sent by the streaming media server through the established second streaming channel.
The browser can analyze the second SSRC of each stream, determine which video stream is based on the second SSRC, and play each stream through the corresponding play channel.
The second streaming channel is directly established between the browser and the streaming media server, transfer is not needed, the time delay of data transmission is reduced, the negotiation mechanism of the second SSRC can ensure that the browser can accurately identify each stream based on the data transmission of one channel, and multi-stream simultaneous playing is realized.
In one example, the first SSRC and one of the plurality of second SSRCs are the same.
In one example:
if the browser acquires the subscription list of the video stream again:
if the number of the video streams subscribed at the current time is larger than that of the video streams subscribed at the last time, the browser reestablishes a second stream channel through negotiation with a second SDP between the browser and the WebRTC gateway;
and if the number of the video streams subscribed at this time is less than or equal to the number of the video streams subscribed at the last time, the browser receives the subscribed multiple video streams sent by the streaming media server through the created stream channel. Here, the created stream channel is the first stream channel or the second stream channel, and the multiplexing of the stream channel saves the procedure of re-SDP negotiation, and reduces the time delay.
To implement the foregoing method, an example of the present disclosure provides a WebRTC-based video streaming apparatus, which is applied to a browser, and includes:
an initialization module 10, configured to perform an initialization operation of the browser;
an SDP module 20, configured to establish a session between the browser and the streaming server through negotiation with a first SDP between the WebRTC gateway after the initialization operation is completed, and establish a first streaming channel;
the subscription module 30 is configured to obtain a subscription list of a video stream, and send the subscription list to the streaming media server through the WebRTC gateway;
a data monitoring module 40, configured to receive, through the first streaming channel, one video stream sent by the streaming media server when the subscription list indicates to subscribe to the video stream;
the SDP module 20 is further configured to, when the subscription list indicates that a plurality of video streams are subscribed, create a second streaming channel based on the session through a second SDP negotiation with the WebRTC gateway;
the data monitoring module 40 is further configured to receive the plurality of video streams sent by the streaming media server through the second streaming channel.
In one example, when the session and the first streaming channel are established through a first SDP negotiation with the WebRTC gateway:
an SDP module 20, configured to receive a first SDP notification offer message sent by the WebRTC gateway, where the first SDP offer message carries connection information of the streaming media server and a first SSRC; the connection information of the streaming media server is acquired by the WebRTC gateway from the streaming media server, and the first SSRC is determined by the WebRTC gateway;
the SDP module 20 is further configured to generate a first SDP answer message according to the first SDP offer message, and send the first SDP answer message to the WebRTC gateway, so as to complete the first SDP negotiation;
the SDP module 20 is further configured to establish a session with the streaming media server according to the connection information, and call an API to create a first streaming channel, where the first streaming channel is successfully created after the session is established.
In an example, the first SSRC is carried in the first SDP answer message, and after the first SDP negotiation is completed:
the SDP module 20 is further configured to send a first SDP negotiation result to the streaming media server through the WebRTC gateway, where the first SDP negotiation result includes the first SSRC, so that when the streaming media server sends the video stream to the browser through the first streaming channel, the streaming media server uses the first SSRC as the SSRC for the video stream.
When the second flow channel is created through a second SDP negotiation with the WebRTC gateway:
the SDP module 20 is configured to receive a second SDP offer message sent by the WebRTC gateway; the second SDP answer message is generated according to the second SDP offer message and is sent to the WebRTC gateway, so that the second SDP negotiation is completed;
the SDP module 20 is further configured to call the API to create a second streaming channel according to the second SDP offer message.
In one example, the browser can also include a configuration module (not shown in FIG. 2), then
After a subscription list of the video stream is acquired:
if the subscription list indicates that a plurality of video streams are subscribed, a configuration module is used for creating a playing channel corresponding to each video stream and distributing a corresponding second SSRC for each playing channel;
correspondingly, the subscription list sent by the subscription module 30 to the streaming media server through the gateway includes the second SSRC corresponding to each video stream, so that when the streaming media server sends the plurality of video streams to the browser through the second streaming channel, the corresponding second SSRC is used as the SSRC of the video stream.
In an example, when the subscription module 30 acquires the subscription list of the video streams again, if the number of the video streams subscribed at this time is greater than the number of the video streams subscribed at the last time, the SDP module 20 re-creates the second stream channel through negotiation with the second SDP between the WebRTC gateway and the WebRTC gateway; if the number of the video streams subscribed this time is less than or equal to the number of the video streams subscribed last time, the data monitoring module 40 receives the multiple video streams subscribed this time sent by the streaming media server through the created stream channel.
The following describes the video streaming process of the present disclosure in detail by way of an example, as shown in fig. 3, including:
301. a user logs in a browser, the browser executes initialization operation, firstly, the browser accesses a web server, and acquires a video list capable of being previewed from the web server, for example, the video list contains all participants of a video conference; meanwhile, the browser sends a websocket connection request to the gateway, the websocket connection request is used for requesting to establish websocket connection with the gateway, and the connection request is an http request.
302. After receiving the Websocket connection request, the scheduler allocates a gateway to the browser according to a load balancing strategy, and then sends the Websocket connection request to the gateway.
303. After receiving the websocket connection request, the gateway directly replies a websocket connection response to the browser, so that the web browser and the gateway are successfully connected. The gateway may also parse the browser's description information (e.g., browser's version information) from the websocket connection request.
By this point, the initialization operation of the browser is completed.
304. And after the gateway and the browser establish connection successfully, the gateway sends a streaming media service connection request.
305. And after receiving the streaming media service connection request, the scheduler allocates a streaming media server for the browser according to the load balancing strategy and sends the streaming media service connection request to the streaming media server.
306. After receiving the streaming media service connection request, the streaming media server returns a connection success response to the gateway, wherein the response carries the connection information of the streaming media server: the method comprises an internal network IP, an internal network port number, an external network IP and an external network port number, wherein the internal network port number and the external network port number can be the same.
307. After receiving a connection success response returned by the streaming media server, the gateway initiates a first SDP negotiation with the browser to create a first streaming channel.
First, the gateway creates a first SDP offer message, where the first SDP offer message at least carries connection information of the streaming media server and the first SSRC, and the gateway sends the first SDP offer message to the browser through the websocket connection. The first SSRC is determined by the gateway.
308. After the browser receives the first SDP offer message:
the browser generates a first SDP answer message (carrying a first SSRC) and sends the message to the gateway, and the first SDP negotiation is completed.
Meanwhile, the browser may call the API to create a first stream channel, where the first stream channel is used to transmit a single H264 video stream, and the single H264 video stream transmitted on the first stream channel all uses the first SSRC.
309. After the first SDP negotiation is completed, the gateway informs the streaming media server of the first SDP negotiation result, wherein the first SSRC is carried, so that the streaming media server uses the first SSRC when sending a single H264 video stream.
310. When the browser analyzes the connection information of the streaming media server from the first SDP offer message, session establishment between the browser and the streaming media server can be initiated, and if the browser and the streaming media server are both in an intranet, the browser can establish session connection between the browser and the streaming media server based on an intranet IP and an intranet port number of the streaming media server; if the browser is in the extranet, the browser can establish a session connection with the streaming server based on the extranet IP and extranet port number of the streaming server.
After the session is established, the first flow channel is also established.
Then, the browser obtains a first subscription list, and if the first subscription is a single stream, 311 is executed; if the first subscription is a multi-stream, then execution 315:
311. and after the browser acquires the first subscription list of the user, the browser sends the subscription list to the gateway through the websocket connection. One video source, i.e. the video conference client, is indicated in the subscription list.
312. And after receiving the subscription list, the gateway sends the subscription list to the streaming media server. And the streaming media server determines that the current subscription is a single stream, and then uses the first SSRC as the SSRC of the stream when the streaming media server sends the single stream.
313. And the streaming media server acquires the video stream of the video source according to the subscription list and sends a message of successful subscription to the gateway.
314. And the streaming media server can send the video stream (using the first SSRC) to the browser through the established first stream channel while sending a subscription success message to the gateway.
315. And after the browser acquires the first subscription list of the user, the browser sends the subscription list to the gateway through the websocket connection. A plurality of video sources, i.e. video conference clients, are indicated in the subscription list.
After the browser acquires the subscription list, a corresponding playing channel is created for each video stream subscribed this time, and each playing channel has one SSRC and is marked as a second SSRC.
Correspondingly, the browser sends the subscription list of the gateway to carry the mapping relationship between the video stream subscribed at this time and the second SSRC in addition to the information of the subscribed video stream.
316. And after receiving the subscription list, the gateway sends the subscription list to the streaming media server.
317. And the streaming media server acquires the video stream of the video source according to the subscription list and sends a message of successful subscription to the gateway. Since the subscription list includes the mapping relationship between the video stream and the second SSRC, the streaming media server can determine the SSRC of each video stream according to the mapping relationship.
318. Since the subscription list indicates a plurality of video streams, after receiving the message that the subscription is successful, the gateway initiates a second SDP negotiation with the browser to recreate the stream channel, which is denoted as a second stream channel. It should be noted that when the second flow channel is successfully created, the first flow channel is closed.
And the gateway generates a second SDP offer message and sends the second SDP offer message to the browser. The second SDP offer message carries description information of multiple video streams, and since the second SSRC of each video stream is already obtained by the streaming media server through the subscription list in the subscription process, that is, in step 316, the second SSRC of the video stream is carried in the description information of the video stream here.
The description mode of the multiple video streams in the second SDP offer message depends on the multi-stream playing scheme supported by the browser, such as the planB and the Unified Plan, if the browser supports the planB, the description mode specified by the planB is used for the description information of the multiple H264 video streams in the second SDP offer message, and if the browser supports the Unified Plan, the description mode specified by the Unified Plan is used for the description information of the multiple H264 video streams in the SDP offer message.
The gateway has already acquired the description information of the browser in step 3, for example, the version information of the browser, and the gateway can determine the multi-stream playing scheme supported by the browser through the version information, so that the gateway adopts a proper description mode to carry the description information of the H264 video stream when constructing the second SDP offer message.
319. And after receiving the second SDP offer message, the browser generates a second SDP answer message and sends the second SDP answer message to the gateway to complete the second SDP negotiation.
Meanwhile, the browser calls the API to create the second streaming channel (the API that creates the first streaming channel call and the API that creates the second streaming channel call are the same API).
Since the session between the browser and the streaming server is already created at step 310, the creation of the second streaming channel is completed when the browser calls the API to create the second streaming channel. When the creation of the second flow path is completed, the first flow path is released.
320. After the second SDP negotiation is completed, the gateway informs the streaming media server that the second SDP negotiation is successful.
321. After the second SDP negotiation is successful, the streaming media server may directly send the subscribed multiple video streams to the browser through the second stream channel.
Correspondingly, when the browser receives the plurality of video streams, the video streams are distinguished through the second SSRC, and each video stream is sent to the corresponding playing channel according to the second SSRC.
It should be noted that when the browser retrieves the subscription list again:
if the number of video streams subscribed at this time is greater than the number of video streams subscribed at the last time, step 315 and step 321 are executed again: after the browser acquires the subscription list, a corresponding playing channel is created for each video stream subscribed this time again, that is, a mapping relation between the video stream and the second SSRC is established, and the mapping relation is provided to the streaming media server through the gateway. Correspondingly, after receiving the message of successful subscription, the gateway re-executes the second SDP negotiation, and after the negotiation is completed, sends the subscribed video stream through the re-created second stream channel.
If the number of the video streams subscribed at this time is less than or equal to the number of the video streams requested at the last time, the browser firstly creates a mapping relation between the video streams subscribed at this time and the second SSRC, and provides the mapping relation to the streaming media server through the gateway. After receiving the message of successful subscription, the gateway does not initiate the second SDP negotiation, and directly multiplexes the stream channel created last time, where there are two cases: if the stream channel created last time is the first stream channel, it indicates that each subscription before the current subscription is a single stream, and the current subscription is a single stream or a single stream after the first stream channel is created, and the video stream is transmitted through the first stream channel (using the first SSRC); if the last created stream channel is the second stream channel, the number of the video streams of the current subscription is less than or equal to the number of the video streams of the last subscription (one may be the case of less than the last number), and the video streams of the current subscription are transmitted through the second stream channel created by the last subscription (when the browser establishes the mapping relationship between the video streams and the second SSRC, the browser may allocate a plurality of second SSRCs of the last subscription to the video streams of the current subscription).
In the video streaming transmission process, session connection and a first streaming channel between the browser and the streaming media server are established through a first SDP negotiation, if a single stream is subscribed for the first time, the session connection and the first streaming channel are directly transmitted through the first streaming channel, the time of the SDP negotiation is saved, and the time delay in the streaming transmission process is reduced; if the first subscription is multi-stream, a second stream channel needs to be established through negotiation of a second SDP, the second stream channel is established based on the established session, the session connection is maintained all the time, the process of session establishment between the browser and the streaming media server during each subscription is omitted, the time delay in the stream transmission process is reduced, meanwhile, based on the SSRC negotiation mechanism between the browser and the streaming media server, the browser can be ensured to accurately identify and receive a plurality of streams from one stream channel, and multi-stream simultaneous playing is realized. In addition, the first flow channel and the second flow channel are both established between the browser and the streaming media server, and transfer is not needed, so that the time delay of video stream transmission is further reduced.
The disclosed examples also provide a computer-readable storage medium storing a computer program for the WebRTC-based video streaming method described above.
The disclosed example also provides an electronic device comprising:
a processor, a memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory and execute the instructions to implement the WebRTC-based video streaming method.
In addition to the methods and systems described above, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to the various embodiments of the present application described in the "exemplary methods" section above of this specification.
The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in a method according to various embodiments of the present application described in the "exemplary methods" section above of this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.
The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (10)
1. A WebRTC-based video streaming method applied to a streaming media system, the streaming media system comprising: browser, webpage instant messaging WebRTC gateway and streaming media server, the method includes:
after the initialization operation of the browser is completed, establishing a session between the browser and the streaming media server through a first Session Description Protocol (SDP) negotiation between the browser and the WebRTC gateway, and establishing a first streaming channel;
the browser acquires a subscription list of video streams and sends the subscription list to the streaming media server through the WebRTC gateway;
if the subscription list indicates to subscribe to a video stream, the browser receives the video stream sent by the streaming media server through the first streaming channel;
if the subscription list indicates that a plurality of video streams are subscribed, the browser creates a second stream channel based on the session through negotiation with a second SDP between the browser and the WebRTC gateway;
and the browser receives the plurality of video streams sent by the streaming media server through the second streaming channel.
2. The WebRTC-based video streaming method of claim 1, wherein the creating a session between the browser and the streaming server through a first session description protocol SDP negotiation with the WebRTC gateway and establishing a first streaming channel comprises:
the browser receives a first SDP notification offer message sent by the WebRTC gateway, wherein the first SDP offer message carries the connection information of the streaming media server and a first synchronous source identifier SSRC; the connection information of the streaming media server is acquired by the WebRTC gateway from the streaming media server, and the first SSRC is determined by the WebRTC gateway;
the browser generates a first SDP answer message according to the first SDP offer message and sends the first SDP answer message to the WebRTC gateway to complete the first SDP negotiation;
and the browser establishes a session with the streaming media server according to the connection information, calls an Application Program Interface (API) to establish a first streaming channel, and successfully establishes the first streaming channel when the session is established.
3. The WebRTC-based video streaming method of claim 2, wherein the first SSRC is carried in the first SDP answer message, and after the first SDP negotiation is completed, the method further comprises:
and sending a first SDP negotiation result to the streaming media server through the WebRTC gateway, wherein the first SDP negotiation result comprises the first SSRC, so that when the streaming media server sends the video stream to the browser through the first streaming channel, the first SSRC is used as the SSRC of the video stream.
4. The WebRTC-based video streaming method of claim 1, wherein the browser creates a second streaming channel based on the session through a second SDP negotiation with the WebRTC gateway, comprising:
the browser receives a second SDP offer message sent by the WebRTC gateway;
the browser generates a second SDP answer message according to the second SDP offer message and sends the second SDP answer message to the WebRTC gateway to complete the second SDP negotiation;
and the browser calls an API according to the second SDP offer message to create a second flow channel based on the session.
5. The WebRTC-based video streaming method according to claim 1, wherein after the browser obtains the subscription list of the video stream, the method further comprises:
if the subscription list indicates that a plurality of video streams are subscribed, the browser creates a playing channel corresponding to each video stream, and each playing channel corresponds to a second SSRC;
the subscription list sent by the browser to the streaming media server through the gateway includes a second SSRC corresponding to each video stream, so that when the streaming media server sends the plurality of video streams to the browser through the second streaming channel, the corresponding second SSRC is used as the SSRC of the video stream.
6. The WebRTC-based video streaming method according to claim 1, wherein when the browser acquires the subscription list of the video stream again, the method further comprises:
if the number of the video streams subscribed at this time is larger than that of the video streams subscribed at the last time, the browser reestablishes a second stream channel through negotiation with a second SDP between the browser and the WebRTC gateway;
and if the number of the video streams subscribed at this time is less than or equal to the number of the video streams subscribed at the last time, the browser receives the plurality of video streams subscribed at this time sent by the streaming media server through the created stream channel, wherein the created stream channel is a first stream channel or a second stream channel.
7. The WebRTC-based video streaming method of claim 1, wherein the initialization operation of the browser comprises:
establishing a Websocket connection between the browser and the WebRTC gateway;
and when the Websocket connection is established, the initialization operation of the browser is completed.
8. A WebRTC-based video streaming apparatus, applied to a browser, the apparatus comprising:
the initialization module is used for executing initialization operation of the browser;
the SDP module is used for establishing a session between the browser and the streaming media server through a first SDP negotiation with the WebRTC gateway after the initialization operation is completed, and establishing a first streaming channel;
the subscription module is used for acquiring a subscription list of the video stream and sending the subscription list to the streaming media server through the WebRTC gateway;
a data monitoring module, configured to receive, through the first streaming channel, a video stream sent by the streaming media server when the subscription list indicates to subscribe to the video stream;
the SDP module is further configured to create a second streaming channel based on the session through a second SDP negotiation with the WebRTC gateway when the subscription list indicates that a plurality of video streams are subscribed;
the data monitoring module is further configured to receive the plurality of video streams sent by the streaming media server through the second streaming channel.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, implements the WebRTC-based video streaming method of any of claims 1-7.
10. An electronic device, comprising:
a processor, a memory for storing the processor-executable instructions;
the processor configured to read the executable instructions from the memory and execute the instructions to implement the WebRTC-based video streaming method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110353203.9A CN112738140B (en) | 2021-04-01 | 2021-04-01 | Video stream transmission method, device, storage medium and equipment based on WebRTC |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110353203.9A CN112738140B (en) | 2021-04-01 | 2021-04-01 | Video stream transmission method, device, storage medium and equipment based on WebRTC |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112738140A CN112738140A (en) | 2021-04-30 |
| CN112738140B true CN112738140B (en) | 2021-07-09 |
Family
ID=75596268
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110353203.9A Active CN112738140B (en) | 2021-04-01 | 2021-04-01 | Video stream transmission method, device, storage medium and equipment based on WebRTC |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112738140B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113347391B (en) * | 2021-05-31 | 2022-12-06 | 北京字跳网络技术有限公司 | Data transmission method, data transmission interruption method and device |
| CN113194335B (en) * | 2021-06-30 | 2021-09-17 | 阿里云计算有限公司 | Streaming media transmission method, transmission equipment and playing equipment |
| CN113691526A (en) * | 2021-08-23 | 2021-11-23 | 深圳市商汤科技有限公司 | Pushing method, device, system, equipment and medium based on WebRTC |
| CN114363281A (en) * | 2021-12-31 | 2022-04-15 | 阿里巴巴（中国）有限公司 | Message transmission method, system, device, storage medium and program product |
| CN114338625B (en) * | 2022-01-11 | 2023-09-15 | 平安科技（深圳）有限公司 | Real-time communication method, device, equipment and storage medium |
| CN114844870B (en) * | 2022-03-25 | 2024-03-26 | 阿里巴巴（中国）有限公司 | Media stream acquisition method and device, electronic equipment and storage medium |
| CN115174539B (en) * | 2022-06-28 | 2023-06-02 | 上海网达软件股份有限公司 | Security video streaming transmission method, system, equipment and storage medium |
| CN115361364B (en) * | 2022-10-08 | 2022-12-20 | 成都华栖云科技有限公司 | Data transmission method of communication protocol based on WebRTC |
| CN116489307B (en) * | 2023-04-04 | 2023-11-28 | 上海缓存命中科技有限公司 | Network monitoring system, network monitoring method, network monitoring device and related equipment |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101155298B (en) * | 2006-09-25 | 2012-07-04 | 华为技术有限公司 | Method and system for implementing fast switching television channel |
| CN102761530A (en) * | 2011-04-29 | 2012-10-31 | 华为终端有限公司 | Method, device and system for media code stream transmission |
| US10320676B2 (en) * | 2014-02-28 | 2019-06-11 | Cisco Technology, Inc. | Smarter policy decisions based on metadata in data flows |
| CN107707868B (en) * | 2016-08-08 | 2020-09-25 | 中国电信股份有限公司 | Video conference joining method, multi-access conference server and video conference system |
| CN112584194A (en) * | 2020-11-13 | 2021-03-30 | 深圳市捷视飞通科技股份有限公司 | Video code stream pushing method and device, computer equipment and storage medium |
-
2021
- 2021-04-01 CN CN202110353203.9A patent/CN112738140B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112738140A (en) | 2021-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112738140B (en) | Video stream transmission method, device, storage medium and equipment based on WebRTC | |
| CN112738644B (en) | Video stream transmission method and device based on WebRTC | |
| US8566395B2 (en) | Method and apparatus for transmitting hypertext transfer protocol media | |
| WO2017088384A1 (en) | Method, apparatus and system for uploading live video | |
| US9154822B2 (en) | Method, apparatus, and terminal device for sharing internet protocol television content | |
| KR102387161B1 (en) | Video screen projection method and apparatus, computer equipment, and storage medium | |
| RU2647654C2 (en) | System and method of delivering audio-visual content to client device | |
| US9282448B2 (en) | Method, system and apparatus for providing streaming media service | |
| CN109995741B (en) | Method and system for realizing wheat connection in network live broadcast | |
| US9756107B2 (en) | Method and systems for optimizing bandwidth utilization in a multi-participant full mesh peer-to-peer video session | |
| US20130282820A1 (en) | Method and System for an Optimized Multimedia Communications System | |
| EP3059945A1 (en) | Method and system for video surveillance content adaptation, and central server and device | |
| WO2019128692A1 (en) | Media processing architecture and method | |
| CN107547517B (en) | Audio and video program recording method, network equipment and computer device | |
| CN105049931A (en) | Method and system for converting video with non-supporting format in mobile terminal | |
| US9686334B2 (en) | Transmission method for media data stream and thin client | |
| CN110022286B (en) | Method and device for requesting multimedia program | |
| CN110113298A (en) | Data transmission method, device, signal server and computer-readable medium | |
| WO2019193991A1 (en) | Distribution device, distribution method and program | |
| CN108632681B (en) | Method, server and terminal for playing media stream | |
| CN112532719B (en) | Information stream pushing method, device, equipment and computer readable storage medium | |
| JP2009118361A (en) | Communication control device, and communication control method | |
| CN112995573B (en) | Video conference live broadcasting system and method | |
| KR101528268B1 (en) | System and method for streaming content to remote locations | |
| CN117596231A (en) | Communication method, terminal device, system and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |