CN112235238B - MCU system and method based on WebRTC - Google Patents
MCU system and method based on WebRTC Download PDFInfo
- Publication number
- CN112235238B CN112235238B CN202010909199.5A CN202010909199A CN112235238B CN 112235238 B CN112235238 B CN 112235238B CN 202010909199 A CN202010909199 A CN 202010909199A CN 112235238 B CN112235238 B CN 112235238B
- Authority
- CN
- China
- Prior art keywords
- room
- audio
- server
- stream
- mixed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/083—Network architectures or network communication protocols for network security for authentication of entities using passwords
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/0876—Network architectures or network communication protocols for network security for authentication of entities based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/485—End-user interface for client configuration
- H04N21/4858—End-user interface for client configuration for modifying screen layout parameters, e.g. fonts, size of the windows
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Power Engineering (AREA)
- Human Computer Interaction (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Information Transfer Between Computers (AREA)
Abstract
An MCU system and method based on WebRTC, the MCU system includes WebRTC server, mixer server and STUN/TURN server; the WebRTC server is used for receiving a room adding request from a terminal user, adding the terminal user into a room, receiving audio and video streams issued by each participant in the room, and forwarding the audio and video streams issued by each participant to the Mixer server, wherein the Mixer server is realized based on a GStreamer frame and is used for carrying out mixed flow processing on the multi-path audio and video streams acquired from the WebRTC server, mixing the multi-path audio and video streams into one-path audio and video stream to form a mixed stream, re-issuing the mixed stream into the WebRTC server, and managing the position of each mixed flow window; the STUN/TURN server is built based on a TURN open source server and is used for ICE to achieve an intranet penetration function. The invention can provide richer communication modes for the traditional P2P mode system.
Description
Technical Field
The invention relates to the field of information communication and the like, in particular to a WebRTC-based MCU system and a WebRTC-based MCU method.
Background
With the rapid development of the current mobile internet technology, the application of the industries such as public security, transportation and the like to intelligent mobile terminals becomes mature day by day, and although the traditional systems such as digital talkback, analog talkback and the like still occupy a place in the command and scheduling work, the single voice information scheduling function provided by the traditional systems and the diversity requirements of the command work on information cannot be suitable. After google opens source WebRTC in 2010, the video call technology threshold is greatly reduced, so that more people are willing to participate in the development of the function, but new problems also arise, and although WebRTC is a real-time communication technology mainly using P2P and should not have centralized nodes, in some large-scale multi-person communication scenes, if end-to-end direct connection is used, the problems of bandwidth and performance can be met on the end.
Disclosure of Invention
In view of technical defects and technical drawbacks in the prior art, embodiments of the present invention provide an MCU system and method based on WebRTC, which overcome or at least partially solve the above problems, and the specific scheme is as follows:
as a first aspect of the present invention, there is provided a WebRTC-based MCU system, the MCU system including a WebRTC server, a Mixer server, and a STUN/TURN server;
the WebRTC server is used for receiving a room adding request from a terminal user, adding the terminal user into a room, receiving audio and video streams issued by each participant in the room, and forwarding the audio and video streams issued by each participant to the Mixer server;
the Mixer server is realized based on a GStreamer frame and is used for carrying out mixed flow processing on the multi-path audio and video streams acquired from the WebRTC server, mixing the multi-path audio and video streams into one path of audio and video stream to form a mixed stream, releasing the mixed stream into the WebRTC server again and managing the positions of mixed flow windows at the same time;
the STUN/TURN server is built based on a TURN open source server and is used for ICE to achieve an intranet penetration function.
Further, the WebRTC server is specifically configured to:
receiving a room joining request from a terminal user, wherein the room joining request comprises authentication information, and the authentication information comprises a room ID, a user name password and an STUN/TURN address;
judging whether the room ID and the user name password are matched, if not, prompting no authority, ending the process, if the match is successful, carrying out ICE interaction operation, if the ICE interaction is failed, ending the process, otherwise, starting to receive audio and video streams from participants, forwarding the audio and video streams to a Mixer server, and starting a stream receiving port for receiving mixed streams from the Mixer server;
and receiving a mixed stream from the Mixed server, and informing the participants to subscribe the mixed stream, wherein the mixed stream contains picture information and sound information of all members in the room.
Further, the Mixer server is specifically configured to:
receiving a mixed flow request from a WebRTC server, wherein the mixed flow request comprises a room ID and a flow receiving port number used for receiving mixed flow in the WebRTC server;
confirming whether a mixed flow process named by a corresponding room ID exists or not, if so, adding the received audio and video stream into the existing mixed flow process, otherwise, creating a new mixed flow process named by the room ID, and adding the received audio and video stream into the new mixed flow process;
distributing a mixed flow window for newly added audio and video streams to display the position of the audio and video streams and the position of the audio and video streams, creating a mixed flow device based on a GStreamer frame, executing a mixed flow algorithm, mixing multi-path audio and video streams into a single-path mixed flow, and after the mixed flow algorithm is executed, forwarding the mixed audio and video streams to a stream collecting port of the WebRTC server.
Furthermore, the number of the Mixer servers is multiple, and the WebRTC server comprises a Room module, a Streaming module and a WebRTC protocol layer;
the Room module is used for receiving a Room creating request of a manager, creating a Room for the manager, receiving a Room adding request from a terminal user, authenticating each added terminal user, and adding the authenticated terminal user into the Room;
the Streaming module is used for acquiring the audio and video stream when participants publish the audio and video stream in a room, forwarding the audio and video stream to an available or low-load Mixer server, performing mixed flow processing by the corresponding Mixer server, receiving the mixed audio and video stream from the Mixer server, and notifying each participant to subscribe;
the WebRTC protocol layer is used for realizing the underlying WebRTC protocol.
Further, the Room module is also used for notifying other participants when a participant leaves the Room and notifying the Streaming module, and the Streaming module is also used for notifying the Mixer server to delete the corresponding mixed flow window when the participant leaves the Room.
As a second aspect of the present invention, there is provided a WebRTC-based MCU method, the method comprising:
step 1, a WebRTC server receives a room adding request from a terminal user, adds the terminal user into a room, receives audio and video streams issued by each participant in the room, and forwards the audio and video streams issued by each participant to a Mixer server;
and 2, the mixer server performs mixed flow processing on the multi-path audio and video stream acquired from the WebRTC server, mixes the multi-path audio and video stream into one path of audio and video stream to form mixed stream, and releases the mixed stream to the WebRTC server again, and manages the positions of mixed flow windows at the same time.
Further, in step 1, the WebRTC server receives a room joining request from a terminal user, joins the terminal user into a room, receives audio and video streams issued from each participant in the room, and forwards the audio and video streams issued by each participant to the Mixer server specifically includes:
step 101, receiving a room joining request from a terminal user, wherein the room joining request comprises authentication information, and the authentication information comprises a room ID, a user name and a password and an STUN/TURN address;
step 102, judging whether the room ID is matched with the user name and the password, and if not, entering step 303; if the matching is successful, go to step 304;
And 311, after the mixed stream is obtained, notifying the participants to subscribe the mixed stream.
Further, step 306 further comprises:
Further, in step 2, the Mixer server performs mixing processing on the multiple audio and video streams acquired from the WebRTC server, mixes the multiple audio and video streams into one audio and video stream to form a mixed stream, and reissues the mixed stream to the WebRTC server specifically includes:
and 406, creating a mixer based on the GStreamer frame, executing a mixing algorithm, and mixing the multi-path audio and video stream into a single-path mixed stream.
And step 407, after the mixed flow algorithm is executed, forwarding the mixed audio and video stream to a stream receiving port of the WebRTC server.
Further, step 407 is followed by:
The invention has the following beneficial effects:
the embodiment of the invention provides a WebRTC-based MCU system and a method, wherein the system comprises: webRTC server, mixer server and STUN/TURN server; the WebRTC server is used for room management and stream management and comprises a bottom-layer WebRTC protocol implementation; the Mixer server is realized based on a GStreamer frame, and is used for mixing multiple paths of audio and video streams acquired from WebRTC into one path. Republishing the mixed flow window to a WebRTC server, and managing the positions of the mixed flow windows simultaneously, wherein the mixed flow window comprises but is not limited to single window amplification and reduction and the like; the STUN/TURN server is built based on a TURN open source server and is used for realizing an intranet penetration function by ICE. The embodiment of the invention provides a comprehensive solution in terms of programs and architecture, if 5 users are connected to a room, each video acquisition terminal is connected with a streaming module, a mixing process is responsible for complex logics of all video coding, transcoding, decoding, mixing and the like, each terminal only needs 1 connection, the whole application only consumes 5 connections, the bandwidth occupation (including uplink and downlink) is 10m, the pressure of a browser end is much smaller, more people can be supported for simultaneous audio and video communication, and the method is more suitable for multi-person video conferences. When the number of people in a room is too large and the mixed flow pressure of the Mixer server is high, the SFU mode can be automatically switched to, namely the MCU method is closed. The Mixer server and the STUN/TURN server can support cluster deployment and support the concurrency capability of a horizontal expansion system when the number of users is too large.
Drawings
Fig. 1 is a network topology diagram of an MCU system based on WebRTC according to an embodiment of the present invention;
fig. 2 is a block diagram of a WebRTC-based MCU system structure provided in an embodiment of the present invention;
fig. 3 is a flowchart illustrating an operation of a WebRTC server in an MCU method based on WebRTC according to an embodiment of the present invention;
fig. 4 is a flowchart of a Mixer server in the MCU method based on WebRTC according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1-2, as a first embodiment of the present invention, a WebRTC-based MCU system is provided, which includes a WebRTC server, a Mixer server and STUN/TURN servers, where the Mixer server and STUN/TURN servers may include multiple servers, multiple Mixer servers form a mixed-flow server cluster, multiple STUN/TURN servers form an ICE server cluster, and the mixed-flow server cluster is illustrated in fig. 2 as including two talk servers, and the system communicates with an end user through HTTP/JSON-RPC messages;
the WebRTC server is used for room management and stream management and comprises a bottom layer WebRTC protocol implementation, and comprises the steps of receiving a room adding request from a terminal user, adding the terminal user into a room, receiving audio and video streams issued by participants in the room, and forwarding the audio and video streams issued by the participants to a Mixer server, wherein the room is created by a manager in advance, each room has a room ID and a user name password thereof, and the room adding request comprises the room ID to be added and the corresponding user name password;
the Mixer server is realized based on a GStreamer frame and is used for carrying out mixed flow processing on the multi-path audio and video streams acquired from the WebRTC server, mixing the multi-path audio and video streams into one path of audio and video stream to form a mixed stream, and re-releasing the mixed stream into the WebRTC server; the name of the mixed stream is named with the room ID; and meanwhile, the Mixer server also manages the mixed flow windows, the mixed flow is started by default to open the 6 mixed flow windows, the participants are arranged in sequence, when one user leaves or stops the audio and video distribution, the window occupied by the user is released, and the released window becomes black. When a user double-clicks and amplifies a certain path of window, the Mixer server cancels the mixed flow method and only distributes the path of video flow.
The STUN/TURN server is built based on a TURN open source server and is used for ICE to realize an intranet penetration function.
Preferably, the WebRTC server is specifically configured to:
receiving a room joining request from a terminal user, wherein the room joining request comprises authentication information, and the authentication information comprises a room ID, a user name password and an STUN/TURN address;
judging whether the room ID and the user name password of the room joining request are matched with the room ID and the user name password of the room needing to be joined, if not, prompting no permission, ending the process, if successful matching, carrying out ICE interaction operation, if ICE interaction fails, ending the process, otherwise, starting to receive audio and video streams from participants, forwarding the audio and video streams to a Mixer server, and starting a stream receiving port for receiving mixed streams from the Mixer server;
and receiving a mixed stream from the Mixed server, and informing the participants to subscribe the mixed stream, wherein the mixed stream contains picture information and sound information of all members in the room.
Preferably, the Mixer server is specifically configured to:
receiving a mixed flow request from a WebRTC server, wherein the mixed flow request comprises a room ID and a flow receiving port number used for receiving mixed flow in the WebRTC server;
confirming whether a mixed flow process named by a corresponding room ID exists or not, if so, adding the received audio and video stream into the existing mixed flow process, otherwise, creating a new mixed flow process named by the room ID, and adding the received audio and video stream into the new mixed flow process;
distributing a mixed flow window for newly added audio and video streams to display the position of the audio and video streams and the position of the audio and video streams, creating a mixed flow device based on a GStreamer frame, executing a mixed flow algorithm, mixing multi-path audio and video streams into a single-path mixed flow, and after the mixed flow algorithm is executed, forwarding the mixed audio and video streams to a stream collecting port of the WebRTC server.
Preferably, the WebRTC server includes a Room module, a Streaming module, and a WebRTC protocol layer;
the Room module is used for receiving a Room creating request of a manager, creating a Room for the manager, receiving a Room adding request from a terminal user, authenticating each added terminal user, and adding the authenticated terminal user into the corresponding Room; in addition, the Room module also receives the audio and video streams published by each participant and informs other participants in the Room to subscribe the video streams; the Streaming module is used for acquiring the audio and video stream when participants publish the audio and video stream in a Room, forwarding the audio and video stream to an available or low-load Mixer server, performing mixed flow processing by the corresponding Mixer server, receiving the mixed audio and video stream from the Mixer server, and informing each participant to subscribe, when a user leaves the Room, the Room module can get through other participants and simultaneously inform the Streaming module, and the Streaming module can also inform the Mixer server to delete a mixed flow window; the WebRTC protocol layer is used for realizing a bottom WebRTC protocol.
The embodiment of the invention provides a comprehensive solution in terms of programs and architecture, if 5 users are connected to a room, each video acquisition terminal is connected with a streaming module, a mixing process is responsible for complex logics of all video coding, transcoding, decoding, mixing and the like, each terminal only needs 1 connection, the whole application only consumes 5 connections, the bandwidth occupation (including uplink and downlink) is 10m, the pressure of a browser end is much smaller, more people can be supported for simultaneous audio and video communication, and the method is more suitable for multi-person video conferences. When the number of people in a room is too large and the mixed flow pressure of the Mixer server is high, the SFU mode can be automatically switched to, namely the MCU method is closed. The Mixer server and the STUN/TURN server can support cluster deployment and support the concurrency capability of a horizontal expansion system when the number of users is too large.
As a second embodiment of the present invention, there is provided a WebRTC-based MCU method, including:
step 1, a WebRTC server receives a room adding request from a terminal user, adds the terminal user into a room, receives audio and video streams issued by each participant in the room, and forwards the audio and video streams issued by each participant to a Mixer server;
and 2, the mixer server performs mixed flow processing on the multi-path audio and video stream acquired from the WebRTC server, mixes the multi-path audio and video stream into one path of audio and video stream to form mixed stream, and releases the mixed stream to the WebRTC server again, and manages the positions of mixed flow windows at the same time.
Preferably, as shown in fig. 3, in step 1, the WebRTC server receives a room joining request from an end user, joins the end user into a room, receives audio and video streams issued from each participant in the room, and forwards the audio and video streams issued by each participant to the Mixer server specifically includes:
step 101, receiving a room joining request from a terminal user, wherein the room joining request comprises authentication information, and the authentication information comprises a room ID, a user name and a password and an STUN/TURN address;
step 102, judging whether the room ID is matched with the user name and the password, and if not, entering step 303; if the matching is successful, go to step 304;
309, forwarding the audio and video stream to a Mixer server, starting a stream receiving port for receiving a mixed stream from the Mixer server, and entering step 310;
In step 311, after the mixed stream is obtained, the participants are notified to subscribe to the mixed stream, and it is noted that the whole flow mixed stream is subscribed only once, which is different from the case that in step 308, the participants are notified to subscribe for multiple times.
Preferably, as shown in fig. 4, in step 2, the step of mixing the multiple audio and video streams acquired from the WebRTC server by the Mixer server, mixing the multiple audio and video streams into one audio and video stream to form a mixed stream, and reissuing the mixed stream to the WebRTC server specifically includes:
and 406, creating a mixer based on the GStreamer frame, executing a mixing algorithm, and mixing the multi-path audio and video stream into a single-path mixed stream.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (2)
1. A WebRTC-based MCU system, comprising a WebRTC server, a Mixer server and a STUN/TURN server;
the WebRTC server is used for receiving a room adding request from a terminal user, adding the terminal user into a room, receiving audio and video streams issued by each participant in the room, and forwarding the audio and video streams issued by each participant to the Mixer server;
the Mixer server is used for mixing the multi-path audio and video stream acquired from the WebRTC server, mixing the multi-path audio and video stream into one path of audio and video stream to form a mixed stream, re-issuing the mixed stream into the WebRTC server, and managing the positions of mixed stream windows;
the STUN/TURN server is built based on a coturn open source server and is used for ICE to realize an intranet penetration function;
wherein the Mixer server is specifically configured to:
receiving a mixed flow request from a WebRTC server, wherein the mixed flow request comprises a room ID and a flow receiving port number used for receiving mixed flow in the WebRTC server;
confirming whether a mixed flow process named by a corresponding room ID exists or not, if so, adding the received audio and video stream into the existing mixed flow process, otherwise, creating a new mixed flow process named by the room ID, and adding the received audio and video stream into the new mixed flow process;
distributing a mixed flow window for newly added audio and video streams to display the position of the audio and video streams and the position of the audio and video streams, creating a mixed flow device based on a GSTreamer frame, mixing multi-path audio and video streams into a single-path mixed flow, and forwarding the mixed audio and video streams to a flow receiving port of a WebRTC server;
wherein the WebRTC server is specifically configured to:
receiving a room joining request from a terminal user, wherein the room joining request comprises authentication information, and the authentication information comprises a room ID, a user name password and an STUN/TURN address;
judging whether the room ID is matched with the user name and the password, if not, prompting no authority, ending the process, if successfully matched, carrying out ICE interaction operation, if the ICE interaction is failed, ending the process, otherwise, starting to receive the audio and video stream from the participant, forwarding the audio and video stream to a Mixer server, and starting a stream receiving port for receiving a mixed stream from the Mixer server;
receiving a mixed stream from a Mixer server, and informing a participant to subscribe the mixed stream, wherein the mixed stream comprises picture information and sound information of all members in a room;
the number of the Mixer servers is multiple, and the WebRTC server comprises a Room module, a Streaming module and a WebRTC protocol layer;
the Room module is used for receiving a Room creating request of a manager, creating a Room for the manager, receiving a Room adding request from a terminal user, authenticating each added terminal user, and adding the authenticated terminal user into the Room;
the Streaming module is used for acquiring the audio and video stream when participants publish the audio and video stream in a room, forwarding the audio and video stream to an available or low-load Mixer server, performing mixed flow processing by the corresponding Mixer server, receiving the mixed audio and video stream from the Mixer server, and notifying each participant to subscribe;
the WebRTC protocol layer is used for realizing a bottom WebRTC protocol;
the Room module is used for informing other participants when a participant leaves the Room and informing the Streaming module at the same time, and the Streaming module is also used for informing the Mixer server to delete the corresponding mixed flow window when the participant leaves the Room.
2. An MCU method based on WebRTC, the method comprising:
step 1, a WebRTC server receives a room adding request from a terminal user, adds the terminal user into a room, receives audio and video streams issued by each participant in the room, and forwards the audio and video streams issued by each participant to a Mixer server;
step 2, the mixer server performs mixed flow processing on the multi-path audio and video stream acquired from the WebRTC server, mixes the multi-path audio and video stream into one path of audio and video stream to form a mixed stream, and re-issues the mixed stream into the WebRTC server;
the WebRTC server receives a room joining request from a terminal user, joins the terminal user into a room, receives audio and video streams issued by each participant in the room, and forwards the audio and video streams issued by each participant to the Mixer server specifically includes:
step 301, receiving a room joining request from a terminal user, wherein the room joining request comprises authentication information, and the authentication information comprises a room ID, a username and password and an STUN/TURN address;
step 302, judging whether the room ID is matched with the user name and the password, and if not, entering step 303; if the matching is successful, go to step 304;
step 303, prompting no authority, and ending the process;
step 304, performing ICE interaction operation, if the ICE interaction fails, entering step 305, otherwise, entering step 306;
step 305, ICE interaction fails, and the flow ends;
step 306, starting to receive audio and video stream data from a participant;
step 307, judging the number of the participants in front of the anterior chamber, if the number is less than 6, entering step 309, and if not, entering step 308;
step 308, notifying other participants in the room to subscribe the video stream;
309, forwarding the audio and video stream to a Mixer server, starting a stream receiving port for receiving a mixed stream from the Mixer server, and entering step 310;
step 310, receiving a mixed stream from a Mixer server, wherein the mixed stream comprises picture information and sound information of all members in a room;
step 311, after the mixed stream is obtained, notifying the participants to subscribe the mixed stream;
in step 2, the step of mixing the multiple audio and video streams acquired from the WebRTC server by the Mixer server, mixing the multiple audio and video streams into one audio and video stream to form a mixed stream, and re-issuing the mixed stream to the WebRTC server specifically includes:
step 401, receiving a mixed flow request from a WebRTC server, where the mixed flow request includes a room ID and a stream receiving port number used for receiving a mixed flow in the WebRTC server;
step 402, confirming whether a mixed flow process named by a corresponding room ID exists, if so, entering step 403, otherwise, entering step 404;
step 403, adding the received audio and video stream into the existing mixed flow process, so that the audio and video stream in the same room ID performs the same mixed flow process, and entering step 405;
step 404, a new mixed flow process named by the room ID is created, the same mixed flow process is executed on the audio and video flows in the same room, and after the new mixed flow process is added, the step 405 is executed;
step 405, allocating a mixed flow window for the newly added audio/video stream to display the position of the audio/video stream, and entering step 406;
step 406, creating a mixer based on the GStreamer frame, and mixing the multi-path audio and video stream into a single-path mixed stream;
step 407, forwarding the mixed audio and video stream to a stream receiving port of the WebRTC server;
step 408, after receiving a window amplification request of a terminal user, forwarding the one-way flow;
step 409, after receiving the window shrinking request of the terminal, re-executing step 406 until the whole process is finished.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010909199.5A CN112235238B (en) | 2020-09-02 | 2020-09-02 | MCU system and method based on WebRTC |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010909199.5A CN112235238B (en) | 2020-09-02 | 2020-09-02 | MCU system and method based on WebRTC |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112235238A CN112235238A (en) | 2021-01-15 |
| CN112235238B true CN112235238B (en) | 2022-11-01 |
Family
ID=74115898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010909199.5A Active CN112235238B (en) | 2020-09-02 | 2020-09-02 | MCU system and method based on WebRTC |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112235238B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112995567A (en) * | 2021-02-03 | 2021-06-18 | 游密科技(深圳)有限公司 | Video call method, device, server and storage medium |
| CN113612956B (en) * | 2021-06-28 | 2024-04-05 | 武汉众智数字技术有限公司 | Video storage and playback system and method based on WebRTC |
| CN113382182B (en) * | 2021-08-12 | 2021-11-09 | 江西省天轴通讯有限公司 | Live broadcast picture control method and device, storage medium and equipment |
| CN114666306B (en) * | 2022-02-18 | 2024-06-11 | 阿里巴巴（中国）有限公司 | WebRTC network connection establishment method, server, electronic device and computer readable storage medium |
| CN115883501A (en) * | 2022-12-08 | 2023-03-31 | 武汉斗鱼鱼乐网络科技有限公司 | Multi-user instant messaging method, system, medium and equipment |
| CN117560358B (en) * | 2023-11-30 | 2024-08-06 | 北京流金岁月科技有限公司 | Method, device and system for universal live broadcast voice intercom |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3070876A1 (en) * | 2015-03-17 | 2016-09-21 | Telefonica Digital España, S.L.U. | Method and system for improving teleconference services |
| WO2019075581A1 (en) * | 2017-10-19 | 2019-04-25 | Lazar Entertainment Inc. | Systems and methods for broadcasting live media streams |
| CN110740161A (en) * | 2019-08-30 | 2020-01-31 | 武汉烽火众智数字技术有限责任公司 | system and method for adapting converged communication |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014100384A1 (en) * | 2012-12-19 | 2014-06-26 | Rabbit, Inc. | Audio video streaming system and method |
| US20180063556A1 (en) * | 2016-08-29 | 2018-03-01 | YouNow, Inc. | Systems and methods for providing guest broadcasting on a live stream video platform |
-
2020
- 2020-09-02 CN CN202010909199.5A patent/CN112235238B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3070876A1 (en) * | 2015-03-17 | 2016-09-21 | Telefonica Digital España, S.L.U. | Method and system for improving teleconference services |
| WO2019075581A1 (en) * | 2017-10-19 | 2019-04-25 | Lazar Entertainment Inc. | Systems and methods for broadcasting live media streams |
| CN111386708A (en) * | 2017-10-19 | 2020-07-07 | 拉扎尔娱乐公司 | System and method for broadcasting live media streams |
| CN110740161A (en) * | 2019-08-30 | 2020-01-31 | 武汉烽火众智数字技术有限责任公司 | system and method for adapting converged communication |
Non-Patent Citations (1)
| Title |
|---|
| 《WebRTC 技术在语音平台中的应用研究》;赵宇;《通信技术》;20180228;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112235238A (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112235238B (en) | MCU system and method based on WebRTC | |
| CN107707868A (en) | Video conference Adding Way, multiple access Conference server and video conferencing system | |
| CN112565234B (en) | Cross-network transmission system and method based on WebRTC | |
| CN103475499A (en) | Voice intercom method and system based on internet teleconference | |
| CN110740161B (en) | System and method for adapting converged communication | |
| CN101510990A (en) | Method and system for processing remote presentation conference user signal | |
| CN105323536A (en) | Attendee private chat method and device in television conference | |
| CN110971863A (en) | Multi-point control unit cross-area conference operation method, device, equipment and system | |
| WO2015131750A1 (en) | Method, device and system for establishing multi-party call based on web rtc | |
| CN111541860B (en) | Real-time audio transmission system and using method thereof | |
| CN102185856A (en) | Team organizing video method, device and system used in team organizing game | |
| CN111131759B (en) | Real-time multimedia transmission system and use method thereof | |
| CN106130747A (en) | A kind of method of Conference control, system and mobile terminal | |
| CN104580231A (en) | Fragmentation asynchronous conference system | |
| CN113949596B (en) | Device connection method, device and storage medium | |
| CN104469254B (en) | Meeting roll-call processing method, device and conference system | |
| CN109963108B (en) | One-to-many talkback method and device | |
| CN111641800A (en) | Method and device for realizing conference | |
| CN109587433B (en) | Point adjusting method and point adjusting device | |
| CN105208317B (en) | The method, apparatus and system for holding subtended conference based on reservation | |
| CN108156413B (en) | Video conference transmission method and device and MCU | |
| CN112019488B (en) | Voice processing method, device, equipment and storage medium | |
| CN102316048A (en) | Method for realizing multi-speaker identity switching | |
| CN116193062A (en) | Conference interconnection method and device, electronic equipment and storage medium | |
| WO2012155732A1 (en) | Mobile terminal and method for realizing remote conference based on mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |