CN113746808B

CN113746808B - Converged communication method, gateway, electronic equipment and storage medium for online conference

Info

Publication number: CN113746808B
Application number: CN202110922251.5A
Authority: CN
Inventors: 官仕富; 阮良; 陈功; 朱振昊; 陈丽
Original assignee: Hangzhou Netease Zhiqi Technology Co Ltd
Current assignee: Hangzhou Netease Zhiqi Technology Co Ltd
Priority date: 2021-08-12
Filing date: 2021-08-12
Publication date: 2023-09-08
Anticipated expiration: 2041-08-12
Also published as: CN113746808A

Abstract

The disclosure provides a converged communication method, a gateway, electronic equipment and a storage medium for online conferences, wherein the method comprises the following steps: the convergence communication gateway receives a call request carrying a call identifier of an SIP terminal sent by an SIP server, and acquires an RTC conference identifier of an RTC conference bound with the call identifier; sending a request for joining an RTC conference of the SIP terminal to the RTC server, and after determining that the RTC server joins the SIP terminal into the RTC conference, establishing a first media transmission channel with the RTC server, wherein the request for joining the RTC conference comprises an RTC conference identifier; and the first media stream information of the SIP terminal sent by the SIP server is sent to the RTC server through a first media transmission channel, so that the RTC server sends the first media stream information to the corresponding RTC terminal in the RTC conference. The communication between the SIP terminal and the RTC terminal in the RTC audio-video conference can be realized.

Description

Converged communication method, gateway, electronic equipment and storage medium for online conference

Technical Field

The disclosure relates to the technical field of communication, and in particular relates to a converged communication method, a gateway, electronic equipment and a storage medium for online conferences.

Background

Currently, audio-video telephony systems typically include Real-time communication (Real-Time Communication, RTC) audio-video conferencing systems and session initiation protocol (Session Initiation Protocol, SIP) audio-video conferencing systems. In the RTC audio and video conference system, a signaling control protocol adopted by the access of the RTC terminal to the RTC audio and video conference is a private protocol, namely, the RTC terminal needs to integrate software development kits (Software Development Kit, SDK) realized by all manufacturers to access the RTC audio and video conference system; in the SIP audio-video conference system, the signaling control protocol adopted by the SIP terminal for accessing the SIP audio-video conference is a standard protocol RFC 3261, and the SIP terminal can access the SIP audio-video conference system only by conforming to the standard protocol.

Because the SIP terminal in the SIP audio-video conference is different from the signaling control protocol adopted by the RTC terminal in the RTC audio-video conference, the SIP terminal and the RTC terminal cannot be communicated with each other; however, in some application scenarios, the SIP terminal may need to communicate with the RTC terminal in the RTC audio-video conference, so how to make the SIP terminal communicate with the RTC terminal in the RTC audio-video conference is a problem to be solved.

Disclosure of Invention

The embodiment of the disclosure provides a converged communication method, gateway, electronic equipment and storage medium for an online conference, which are used for enabling an SIP terminal to join in an RTC audio-video conference and realizing communication between the SIP terminal and an RTC terminal in the RTC audio-video conference.

In a first aspect, an embodiment of the present disclosure provides a converged communication method for an online conference, which is applied to a converged communication gateway, including:

receiving a call request carrying a call identifier of an SIP terminal sent by a session initiation protocol SIP server, and acquiring an RTC conference identifier of a real-time communication RTC conference bound with the call identifier;

sending a request for joining an RTC conference of the SIP terminal to an RTC server, and establishing a first media transmission channel with the RTC server after determining that the RTC server joins the SIP terminal into the RTC conference; wherein, the RTC conference request comprises the RTC conference identifier;

and sending the first media stream information of the SIP terminal sent by the SIP server to the RTC server through the first media transmission channel, so that the RTC server sends the first media stream information to a corresponding RTC terminal in the RTC conference.

In one possible implementation, the call request is a request to join a SIP conference, and the call identifier is a SIP conference identifier;

the receiving the call request carrying the call identifier sent by the session initiation protocol SIP terminal, and obtaining the RTC conference identifier of the real-time communication RTC conference bound to the call identifier, includes:

Responding to a SIP conference joining request carrying an SIP conference identifier sent by the SIP terminal, and joining the SIP terminal into an SIP conference corresponding to the SIP conference identifier;

and acquiring an RTC conference identifier of the RTC conference bound with the SIP conference identifier.

In one possible embodiment, the method further comprises:

each RTC terminal in the RTC conference is added into the SIP conference, and a second media transmission channel with the RTC server is established;

receiving second media stream information of each RTC terminal in the RTC conference, which is sent by the RTC server, through the second media transmission channel;

the second media stream information of each RTC terminal and the third media stream information sent by each SIP terminal in the SIP conference are subjected to fusion processing to obtain target media stream information;

and respectively sending the target media stream information to each SIP terminal in the SIP conference.

In a possible implementation, the second media stream information and the third media stream information each comprise audio stream information;

the merging processing is performed on the second media stream information of each RTC terminal and the third media stream information sent by each SIP terminal in the SIP conference, so as to obtain target media stream information, which comprises the following steps:

And mixing audio stream information in the second media stream information of each RTC terminal and audio stream information in the third media stream information sent by each SIP terminal in the SIP conference to obtain first target media stream information.

In a possible implementation manner, the second media stream information and the third media stream information each further comprise video stream information;

the merging processing is performed on the second media stream information of each RTC terminal and the third media stream information sent by each SIP terminal in the SIP conference, so as to obtain target media stream information, and the method further comprises the following steps:

and carrying out screen mixing processing on the video stream information in the second media stream information of each RTC terminal and the video stream information in the third media stream information sent by each SIP terminal to obtain second target media stream information.

In one possible implementation, the call request is a call RTC terminal request;

the method further comprises the steps of:

establishing a third media transmission channel with the RTC server, and receiving fourth media stream information of one RTC terminal in the RTC conference, which is sent by the RTC server, through the third media transmission channel;

And transmitting the fourth media stream information of the RTC terminal to the SIP terminal.

In one possible implementation manner, the sending, by the first media transmission channel, the first media flow information of the SIP terminal sent by the SIP server to the RTC server includes:

transcoding the first media stream information of the SIP terminal sent by the SIP server into a preset coding format;

and sending the transcoded first media stream information to the RTC server through the first media transmission channel.

In a possible implementation manner, the obtaining the RTC conference identifier of the real-time communication RTC conference bound to the call identifier includes:

and sending a verification request carrying the call identifier to the RTC server to obtain an RTC conference identifier of the RTC conference bound with the call identifier, wherein the RTC conference identifier is returned by the RTC server after verification is passed.

In a second aspect, an embodiment of the present disclosure further provides a converged communication gateway, including a session initiation protocol SIP signaling proxy module, at least one SIP media service module, and at least one protocol conversion module;

the SIP signaling proxy module is used for receiving a call request carrying a call identifier of an SIP terminal sent by a Session Initiation Protocol (SIP) server and distributing the call request to a corresponding SIP media service module;

The SIP media service module is used for establishing a call event carrying a call identifier according to the call request and sending the call event to the corresponding protocol conversion module;

the protocol conversion module is used for acquiring an RTC conference identifier of a real-time communication RTC conference bound with the call identifier, sending an RTC conference joining request carrying the RTC conference identifier of the SIP terminal to an RTC server, establishing a first media transmission channel between the SIP service module and the RTC server after determining that the RTC server joins the SIP terminal to the RTC conference, and sending a media forwarding instruction to the SIP media service module;

the SIP media service module is further configured to respond to the media forwarding instruction, and send first media stream information of the SIP terminal sent by the SIP server to the RTC server through the first media transmission channel, so that the RTC server sends the first media stream information to a corresponding RTC terminal in the RTC conference.

the SIP media service module further includes:

The SIP conference management sub-module is used for responding to a SIP conference joining request carrying an SIP conference identifier sent by the SIP terminal and joining the SIP terminal into an SIP conference corresponding to the SIP conference identifier;

the protocol conversion module further includes:

and the acquisition sub-module is used for acquiring the RTC conference identifier of the RTC conference bound with the SIP conference identifier.

In one possible implementation manner, the protocol conversion module further includes:

a first channel establishing sub-module, configured to join each RTC terminal in the RTC conference to the SIP conference, and establish a second media transmission channel with the RTC server;

the SIP media service module further includes:

a receiving sub-module, configured to receive, through the second media transmission channel, second media stream information of each RTC terminal in the RTC conference, where the second media stream information is sent by the RTC server;

the fusion sub-module is used for carrying out fusion processing on the second media stream information of each RTC terminal and the third media stream information sent by each SIP terminal in the SIP conference to obtain target media stream information;

and the first sending sub-module is used for respectively sending the target media stream information to each SIP terminal in the SIP conference.

the fusion sub-module is further configured to:

the protocol conversion module further includes:

a second channel establishing sub-module, configured to establish a third media transmission channel with the RTC server, and receive fourth media stream information of one RTC terminal in the RTC conference, which is sent by the RTC server, through the third media transmission channel;

And the second sending sub-module is used for sending the fourth media stream information of the RTC terminal to the SIP terminal.

In one possible implementation manner, the SIP media service module further includes:

a conversion sub-module, configured to transcode the first media stream information of the SIP terminal sent by the SIP server into a preset encoding format;

and the third sending sub-module is used for sending the transcoded first media stream information to the RTC server through the first media transmission channel.

and the request verification sub-module is used for sending a verification request carrying the call identifier to the RTC server so as to obtain the RTC conference identifier of the RTC conference bound with the call identifier, wherein the RTC conference identifier is returned by the RTC server after verification is passed.

In a third aspect, the present disclosure also provides an electronic device comprising a memory and a processor, the memory having stored thereon a computer program executable on the processor, which when executed by the processor causes the processor to implement the steps of the converged communication method of any one of the online conferences of the first aspect.

In a fourth aspect, the present disclosure also provides a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of the converged communication method of any one of the first aspects.

The converged communication method for online conferences provided by the embodiment of the disclosure has at least the following beneficial effects:

according to the scheme provided by the embodiment of the disclosure, when receiving a call request of an SIP terminal sent by an SIP server, a convergence communication gateway acquires an RTC conference identifier of an RTC conference corresponding to the call request, sends a request of joining the RTC conference of the SIP terminal to the RTC server, and establishes a first media transmission channel with the RTC server after determining that the RTC server joins the SIP terminal to the RTC conference; and then the first media stream information of the SIP terminal sent by the SIP server is sent to the RTC server through a first media transmission channel, so that the RTC server sends the first media stream information to the corresponding RTC terminal in the RTC conference. Therefore, the converged communication gateway is used as a communication link between the SIP server and the RTC server, on one hand, the communication gateway can perform signaling interaction with the SIP server, on the other hand, the communication gateway can perform signaling interaction with the RTC server, so that conversion between the SIP protocol and the RTC protocol is realized, further, the SIP terminal can join in the RTC audio-video conference, and communication between the SIP terminal and the RTC terminal in the RTC audio-video conference is realized.

Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the disclosure. The objectives and other advantages of the disclosure will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art.

Fig. 1 is an application scenario schematic diagram of a converged communication method for online conferences according to an embodiment of the present disclosure;

fig. 2 is a flowchart of a converged communication method for online conferences according to an embodiment of the present disclosure;

fig. 3 is a flowchart of another converged communication method for online conferencing provided by an embodiment of the present disclosure;

fig. 4A is a schematic view of a conference interface of a SIP terminal according to an embodiment of the present disclosure;

Fig. 4B is a schematic view of a conference interface of an RTC terminal provided in an embodiment of the present disclosure;

fig. 5 is a flowchart of another converged communication method for online conferencing provided by an embodiment of the present disclosure;

fig. 6 is a schematic diagram of a converged communication system for online conferencing provided in an embodiment of the present disclosure;

fig. 7 is a block diagram of a converged communication gateway according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

For the purpose of promoting an understanding of the principles and advantages of the disclosure, reference will now be made in detail to the drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the disclosure. Based on the embodiments in this disclosure, all other embodiments that a person of ordinary skill in the art would obtain without making any inventive effort are within the scope of protection of this disclosure.

It should be noted that the terms "first," "second," and the like in the description and in the claims of the present disclosure are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein.

Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For ease of understanding, some of the concepts involved in the embodiments of the present disclosure are explained below.

Session initiation protocol (Session Initiation Protocol, SIP): belonging to the standard protocol RFC (Request For CommentsRFC, a series of numbered files) 3261, is a signaling control protocol for the application layer. SIP is used to create new, manage or terminate existing one or more party sessions as defined by RFC 3261.

Network telephone exchange (Internet Protocol Private Branch Exchange, ip pbx): a corporate SIP telephone system based on the Internet protocol IP.

Real-time communication (Real-Time Communication, RTC): common video telephony, voice telephony, are all techniques. The method is characterized by being capable of bidirectional low-delay communication, and the delay time is generally 300-500 milliseconds.

Real-time transport protocol (Real-Time Transport Protocol, RTP): an application layer transport protocol most commonly used in real-time communication technology, which typically runs on top of the user datagram protocol (User Datagram Protocol, UDP) or the transmission control protocol (Transmission Control Protocol, TCP), mainly carries the transmission of media data.

-a selective forwarding unit (Selective Forwarding Unit, SFU): a real-time audio and video server architecture. The basic principle is as follows: and forwarding the media streams of the sending ends to the receiving ends by the central server.

Multipoint control unit (MultiPoint Control Unit, MCU): an audio-video server architecture for audio-video mixing and screen-mixing. The basic principle is as follows: the server receives the uplink media streams of each end, decodes the audio and video streams in each media stream, mixes the audio and video streams, recodes the mixed audio and video streams, and forwards the recoded audio and video streams to the downlink users of each end.

The following is a summary of the design concepts of the present disclosure.

Currently, in an RTC audio-video conference system, a signaling control protocol adopted by an RTC terminal accessed to an RTC audio-video conference is a private protocol; in the SIP audio-video conference system, the signaling control protocol adopted by the SIP terminal for accessing the SIP audio-video conference is the standard protocol RFC 3261. Because the SIP terminal in the SIP audio-video conference is different from the signaling control protocol adopted by the RTC terminal in the RTC audio-video conference, the SIP terminal and the RTC terminal cannot be communicated with each other; however, in some application scenarios, the SIP terminal may need to communicate with the RTC terminal in the RTC audio-video conference, so how to make the SIP terminal communicate with the RTC terminal in the RTC audio-video conference is a problem to be solved.

In view of this, the embodiments of the present disclosure provide a converged communication method, gateway, electronic device, and storage medium for online conferences, where the converged communication gateway is used as a communication link between an SIP server and an RTC server, so that on one hand, signaling interaction can be performed with the SIP server, on the other hand, signaling interaction can be performed with the RTC server, thereby implementing conversion between an SIP protocol and an RTC protocol, and further enabling an SIP terminal to join in an RTC audio-video conference, so as to implement communication between the SIP terminal and the RTC terminal in the RTC audio-video conference.

The application scenario of the embodiments of the present disclosure is described below with reference to the accompanying drawings.

Referring to fig. 1, an application scenario diagram of a converged communication method for online conferences according to an embodiment of the present disclosure is shown. The application scenario includes SIP terminal 100, SIP server 200, converged communication gateway 300, RTC server 400, and RTC terminal 500; the SIP terminal 100 and the SIP server 200 may be connected through a communication network to form a SIP audio/video conference system; the RTC server 400 and the RTC terminal 500 may be connected through a communication network to constitute an RTC audio-video conference system; the converged communication gateway 300 may be connected to the SIP server 200 and the RTC server 400 via communication networks, respectively. Alternatively, the communication network may be a wired network or a wireless network, and embodiments of the present disclosure are not particularly limited herein.

The SIP terminal 100 may include, but is not limited to, mobile phones, landline phones, desktop computers, mobile computers, tablet computers, media players, smart wearable devices, smart televisions, vehicle-mounted devices, personal digital assistants (personal digital assistant, PDA), and other electronic devices; RTC terminal 500 may include, but is not limited to, desktop computers, mobile phones, mobile computers, tablet computers, media players, smart wearable devices, smart televisions, car-mounted devices, PDAs, and like electronic devices.

The converged communication gateway can be deployed on a server, and the converged communication gateway, the SIP server 200 and the RTC server 400 can be independent physical servers, can be a server cluster or a distributed system formed by a plurality of physical servers, and can also be cloud servers for providing cloud services, cloud databases, cloud computing, cloud storage, cloud functions, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), basic cloud computing services such as big data and artificial intelligent platforms, and the like.

The SIP terminal 100 and the SIP server 200 perform signaling interaction through a SIP protocol, the RTC server 400 and the RTC terminal 500 perform signaling interaction through a RTC protocol, and the converged communication gateway can be used as a communication link between the SIP server and the RTC server, so that the SIP protocol and the SIP server can be adopted to perform signaling interaction on the one hand, and the RTC protocol and the RTC server can be adopted to perform signaling interaction on the other hand, thereby realizing conversion between the SIP protocol and the RTC protocol, further enabling the SIP terminal to join in the RTC audio-video conference created by the RTC server, and realizing communication between the SIP terminal and the RTC terminal in the RTC audio-video conference.

The converged communication method of the online conference according to the exemplary embodiment of the present disclosure is described below in conjunction with the application scenario of fig. 1. The above application scenarios are only shown for facilitating understanding of the spirit and principles of the present disclosure, and embodiments of the present disclosure are not limited in any way in this respect. Rather, embodiments of the present disclosure may be applied to any scenario where applicable.

Referring to fig. 2, an embodiment of the present disclosure provides a converged communication method for online conferences, which is applicable to a converged communication gateway, such as the converged communication gateway 300 shown in fig. 1; the converged communication method of the online conference can comprise the following steps:

step S201, receiving a call request carrying a call identifier of a SIP terminal sent by a SIP server, and acquiring an RTC conference identifier of an RTC conference bound with the call identifier.

The SIP terminal may be a call device supporting SIP protocol, such as a SIP hardware phone, an electronic device with a SIP softphone installed, where the SIP hardware phone includes a PSTN (Public Switched Telephone Network ) mobile phone, a PSTN landline phone, etc., and the electronic device may include, but is not limited to, a desktop computer, a mobile computer, a tablet computer, a media player, a smart wearable device, a smart television, a vehicle-mounted device, a PDA, etc. The SIP terminal and the SIP server may constitute a SIP audio video conference system, for example: the SIP system of the enterprise includes a SIP terminal and an ip pbx system, which may be understood as a SIP server, for managing access of the SIP terminal.

The SIP terminal can send a call request carrying a call identifier to the SIP server, wherein the call identifier can be an SIP conference identifier, an SIP call number and the like and can be determined according to an actual application scene; the SIP server may then send the call request of the SIP terminal to the converged communication gateway.

Further, the converged communication gateway obtains the RTC conference identifier of the RTC conference bound to the SIP conference identifier, which specifically may include the following steps:

and sending a verification request carrying the SIP meeting identifier to the RTC server to obtain the RTC meeting identifier of the RTC meeting which is returned by the RTC server after verification is passed and is bound with the SIP meeting identifier.

In the embodiment of the disclosure, after receiving a call request of an SIP terminal, the converged communication gateway may send a verification request carrying a call identifier to the RTC server, and if the RTC server matches the call identifier from a binding relationship between a pre-created call identifier and an RTC conference identifier, the verification is passed, and sends the RTC conference identifier bound to the call identifier to the converged communication gateway.

Step S202, a request for joining an RTC conference of an SIP terminal is sent to an RTC server, and after the RTC server is determined to join the SIP terminal into the RTC conference, a first media transmission channel with the RTC server is established; the RTC conference request comprises an RTC conference identifier.

In a specific implementation, after the converged communication gateway obtains the RTC conference identifier of the RTC conference, the user of the SIP terminal is simulated to be the RTC user, the RTC server sends a request for joining the RTC conference of the SIP terminal, and after the RTC server joins the SIP terminal into the RTC conference, a successful message for joining the RTC conference is returned to the converged communication gateway, at this time, the converged communication gateway may establish a first media transmission channel with the RTC server, where the first media transmission channel is used to transmit media stream information of the SIP terminal to the RTC server, and then the RTC server sends the media stream information to the corresponding RTC terminal.

In addition, the convergence communication gateway can negotiate a preset coding format of the media stream information transmitted by the first media transmission channel with the RTC server while establishing the first media transmission channel with the RTC server so as to ensure that the media stream information transmitted by the convergence communication gateway can be decoded by the RTC server.

In step S203, the first media stream information of the SIP terminal sent by the SIP server is sent to the RTC server through the first media transmission channel, so that the RTC server sends the first media stream information to the corresponding RTC terminal in the RTC conference.

In this step, the first media stream information of the SIP terminal may include one or both of video stream information and audio stream information. If the coding format of the first media stream information does not accord with the corresponding preset coding format, the first media stream information needs to be transcoded into the preset coding format; if the encoding format of the first media stream information accords with the corresponding preset encoding format, transcoding is not needed.

Therefore, in one possible implementation manner, the first media stream information of the SIP terminal sent by the SIP server in step S202 is sent to the RTC server through the first media transmission channel, and the method may further include the following steps:

b1, transcoding first media stream information of an SIP terminal sent by an SIP server into a preset coding format;

and B2, transmitting the transcoded first media stream information to the RTC server through a first media transmission channel.

Alternatively, considering that the video coding formats supported by the SIP terminal and the RTC terminal may be the same and the performance overhead of video transcoding is relatively large, the corresponding preset coding format for the video stream information may be set to the coding format supported by both the SIP terminal and the RTC terminal, for example, the H264 coding format, so that the video stream information does not need to be transcoded, and additional performance overhead may be avoided.

In addition, for the audio stream information, since the audio coding capability of different terminals is different, for example, the PSTN mobile phone uses coding formats such as G711, G729, etc., and the coding format supported by the RTC terminal is an OPUS coding format, at this time, the preset coding format is set to be an OPUS coding format, so for the audio stream information of the SIP terminal, if the coding format does not conform to the preset coding format, the audio stream information of the SIP terminal needs to be transcoded to the preset coding format, so as to ensure that the transmitted audio stream information can be decoded by the RTC server.

In the embodiment of the disclosure, the converged communication gateway can perform signaling interaction with the SIP server on one hand, and can perform signaling interaction with the RTC server on the other hand, so that conversion between the SIP protocol and the RTC protocol is realized, further, the SIP terminal can join in the RTC conference, and media stream information of the SIP terminal is transmitted to the corresponding RTC terminal, so that communication between the SIP terminal and the RTC terminal in the RTC conference is realized.

The method of the embodiment of the disclosure can be applied to various converged communication scenarios, such as: SIP terminals join RTC conference scenes, SIP terminals call RTC terminals (i.e., point-to-point calls) scenes, etc. These two scenarios are described below as examples.

In some embodiments, for the SIP terminal to join the RTC conference scenario, the call request in step S201 may be a SIP conference joining request, and the call identifier is a SIP conference identifier; in the step S201, the step of receiving a call request carrying a call identifier sent by a session initiation protocol SIP terminal and obtaining an RTC conference identifier of a real-time communication RTC conference bound to the call identifier may include the following steps:

a1, responding to a SIP conference joining request carrying a SIP conference identifier sent by a SIP terminal, and joining the SIP terminal into a SIP conference corresponding to the SIP conference identifier;

A2, acquiring an RTC conference identifier of the RTC conference bound with the SIP conference identifier.

For example, the SIP conference identification may be a SIP conference number, which may be obtained by: the RTC terminal requests to create the RTC conference from the RTC server, when the RTC conference is created, the RTC server generates an RTC conference identifier and a corresponding SIP conference identifier, the RTC conference identifier is bound with the SIP conference identifier and then returned to the RTC terminal, and then the RTC terminal can send the SIP conference identifier to the SIP terminal needing to join the RTC conference in a set mode (such as mail, short message and the like), wherein the RTC conference can be an RTC audio conference or an RTC video conference, and the RTC conference identifier can comprise RTC conference room information and the like.

After the SIP terminal acquires the SIP conference identifier, the SIP terminal can send a call request carrying the SIP conference identifier to the SIP server, the call request can also carry the identifier of the SIP terminal and the like, then the SIP server sends the call request to the converged communication gateway, and the converged communication gateway can join the SIP terminal into the corresponding SIP conference according to the SIP conference identifier and acquire the RTC conference identifier of the RTC conference bound with the SIP conference identifier.

In some possible embodiments, the converged communication gateway obtains the RTC conference identifier of the RTC conference bound to the SIP conference identifier, and may include the steps of:

In this embodiment, after receiving a call request from a SIP terminal, the converged communication gateway may send a verification request carrying a call identifier to the RTC server, and if the RTC server matches the call identifier from the binding relationship between the pre-created call identifier and the RTC conference identifier, the verification is passed, and sends the RTC conference identifier bound to the call identifier to the converged communication gateway.

Further, the steps S202 to S203 may be further performed, which will not be described herein.

In a specific implementation, the SIP terminal may join an RTC conference in a scenario that multiple SIP terminals may join an RTC conference at the same time, and the converged communication gateway may forward media stream information of the SIP terminal to the RTC terminal based on an SFU architecture, where the RTC terminal may select to subscribe to media stream information of the SIP terminal and other RTC terminals, and the RTC server may forward, after receiving media stream information of a certain SIP terminal, media stream information of the SIP terminal to the RTC terminal subscribed to the SIP terminal, and may also forward media stream information of other RTC terminals subscribed to by each RTC terminal to the corresponding RTC terminal. It will be appreciated that the RTC terminal receives multiple media stream information.

In the embodiment of the disclosure, under the condition that the SIP terminal joins the RTC conference, the converged communication gateway can perform signaling interaction with the SIP server on one hand, and can perform signaling interaction with the RTC server on the other hand, so that conversion between the SIP protocol and the RTC protocol is realized, further the SIP terminal can join the RTC conference, media stream information of the SIP terminal is transmitted to the corresponding RTC terminal, and communication between the SIP terminal and each RTC terminal in the RTC conference is realized.

In other embodiments, for the SIP terminal call RTC terminal scenario, the call request in step S201 may be a call RTC terminal request, and the call identifier may be a SIP call number.

For example, in the case that the SIP terminal calls the RTC terminal, the SIP server may be a call center server, and after obtaining the SIP call number, the call center server may send a call request carrying the SIP call number to the converged communication gateway, where the SIP call number may be obtained by:

when the SIP terminal needs to call the RTC terminal, the SIP terminal firstly sends a call request to a call center server, the call center server designates a corresponding RTC terminal according to the call request and sends the called request to the corresponding RTC terminal, so that the RTC terminal creates an RTC conference through the RTC server and generates an RTC conference identifier and a corresponding SIP call number, the RTC conference only comprises one RTC terminal, and the RTC server can send the SIP call number to the call center server. The call center server may construct a call command, send a call request carrying the SIP call number to the converged communication gateway, and the call request may also carry an identification of the SIP terminal, etc.

In some possible embodiments, after the convergence communication gateway receives the call request carrying the SIP call number, the convergence communication gateway may acquire the RTC conference identifier of the RTC conference bound to the SIP call number, and specifically may perform the following steps:

and sending a verification request carrying the SIP calling number to the RTC server to obtain an RTC conference identifier of the RTC conference bound with the SIP calling number, which is returned by the RTC server after verification is passed.

In the embodiment of the disclosure, under the condition that the SIP terminal calls the RTC terminal, the converged communication gateway can perform signaling interaction with the SIP server on one hand and can perform signaling interaction with the RTC server on the other hand, so that conversion of the SIP protocol and the RTC protocol is realized, and further media stream information of the SIP terminal can be transmitted to the corresponding RTC terminal, and point-to-point communication between the SIP terminal and the RTC terminal is realized.

Based on the above embodiments of the present disclosure, the converged communication gateway may not only transmit media stream information of the SIP terminal to the corresponding RTC terminal, but also acquire media stream information of the RTC terminal and transmit media stream information of the RTC terminal to the SIP terminal, and two scenes of joining the RTC conference scene for the SIP terminal and calling the RTC terminal by the SIP terminal are described below respectively.

In some embodiments, for the SIP terminal to join the RTC conference scenario, as shown in fig. 3, the converged communication gateway may further perform the following steps:

step S301, each RTC terminal in the RTC conference is added into the SIP conference, and a second media transmission channel with the RTC server is established.

The converged communication gateway can simulate a fake RTC terminal (i.e. a virtual terminal) to join the RTC conference, then obtain list information of all RTC terminals in the RTC conference returned by the RTC server, then join each RTC terminal to the SIP conference, and establish a second media transmission channel with the RTC server, where the second media transmission channel is used to obtain media stream information of the RTC terminal transmitted by the RTC server.

In addition, the convergence communication gateway establishes a second media transmission channel with the RTC server and simultaneously negotiates a preset coding format of media stream information transmitted by the second media transmission channel with the RTC server so as to ensure that the media stream information transmitted by the RTC server can be decoded by the convergence communication gateway; the preset encoding format of the media stream information transmitted by the second media transmission channel may be the same as or different from the preset encoding format of the media stream information transmitted by the first media transmission channel, and the embodiment of the disclosure is not limited.

Step S302, receiving, through a second media transmission channel, second media stream information of each RTC terminal in the RTC conference, where the second media stream information is sent by the RTC server.

Wherein the second media stream information may include one or both of video stream information and audio stream information. After obtaining the respective second media stream information of each RTC terminal, if the coding format of each second media stream information does not accord with the corresponding preset coding format, the RTC server needs to transcode the second media stream information into the preset coding format; if the coding format of each second media stream information accords with the corresponding preset coding format, transcoding is not needed.

Step S303, the second media stream information of each RTC terminal and the third media stream information sent by each SIP terminal in the SIP conference are integrated to obtain the target media stream information.

Wherein the third media stream information includes one or both of audio stream information and video stream information.

In the embodiment of the disclosure, the converged communication gateway may perform audio mixing processing and/or screen mixing processing on the respective second media stream information of each RTC terminal and the respective third media stream information of each SIP terminal based on the MCU architecture, and then encode the mixed media stream information to obtain the target media stream information.

In one possible implementation, the second media stream information and the third media stream information each include audio stream information; in step S303, the merging process is performed on the second media stream information of each RTC terminal and the third media stream information sent by each SIP terminal in the SIP conference, to obtain target media stream information, which may include the following steps:

The audio mixing process may include three processes of decoding, mixing and encoding, where after each audio stream information is decoded, each decoded audio stream information is mixed, and then the audio stream information after mixing is encoded, so as to obtain first target media stream information.

In another possible implementation, the second media stream information and the third media stream information each further include video stream information; in step S303, the second media stream information of each RTC terminal and the third media stream information sent by each SIP terminal in the SIP conference are integrated to obtain target media stream information, which may further include the following steps:

The screen mixing processing may include three processes of decoding, screen mixing and encoding, where after each video stream information is decoded separately, each decoded video stream information is mixed, and then the video stream information after the screen mixing is encoded, so as to obtain second target media stream information.

Step S304, the target media stream information is respectively sent to each SIP terminal in the SIP conference.

It can be understood that each SIP terminal receives one path of media stream information after mixing the audio and the screen, and the RTC terminal receives multiple paths of media stream information.

For example, the terminals joining the RTC conference include the RTC terminal 1, the RTC terminal 2, the SIP terminal 1, and the SIP terminal 2, and the RTC terminal 1 can receive the multiplexed media stream information of the RTC terminal 2, the SIP terminal 1, and the SIP terminal 2, and in this case, the display screen is a local display screen of the RTC terminal 1, and the RTC terminal 2, the SIP1, and the SIP2 are the media stream information of the RTC terminal 2, the media stream information of the SIP terminal 1, and the media stream information of the SIP terminal 2, respectively, as shown in fig. 4A. The SIP terminal 1 may receive the media stream information of the RTC terminal 1, the RTC terminal 2, and the mixed media stream information of the SIP terminal 1 and the SIP terminal 2, and at this time, the display screen is shown in fig. 4B.

In the embodiment of the disclosure, the SIP terminal can join the RTC conference through the converged communication gateway, so that not only can the media stream information of the SIP terminal be sent to the corresponding RTC terminal, but also the media stream information of the RTC terminal can be obtained, and the media stream information of the RTC terminal is sent to the SIP terminal, thereby realizing the intercommunication between the SIP terminal and the RTC terminal.

In addition, when a plurality of SIP terminals join in an RTC conference at the same time, as for video stream information, because conference pictures displayed by each SIP terminal are the same, the video stream information of each terminal in the RTC conference is fused by the communication gateway, and only one screen mixing process is needed, so that the performance cost can be greatly reduced; for the audio stream information, because each SIP terminal needs to exclude the collected sound, the audio stream information of each terminal in the RTC conference is required to be mixed for N-1 times by the convergence communication gateway, wherein N is the number of each terminal, and compared with the mixing screen processing, the mixing processing has small performance cost, so that large performance cost is not caused.

In other embodiments, for the SIP terminal calling RTC terminal scenario, as shown in fig. 5, the converged communication gateway may further perform the following steps:

Step S401, a third media transmission channel with the RTC server is established, and fourth media stream information of one RTC terminal in the RTC conference, which is sent by the RTC server, is received through the third media transmission channel.

In the step, the converged communication gateway establishes a third media transmission channel with the RTC server and simultaneously negotiates a preset coding format of media stream information transmitted by the third media transmission channel with the RTC server so as to ensure that the media stream information transmitted by the RTC server can be decoded by the converged communication gateway; the preset encoding format of the media stream information transmitted by the third media transmission channel may be the same as or different from the preset encoding format of the media stream information transmitted by the first media transmission channel, and the embodiment of the disclosure is not limited.

Wherein the fourth media stream information may include one or both of video stream information and audio stream information. After obtaining the fourth media stream information of the RTC terminal, if the coding format of the fourth media stream information does not accord with the corresponding preset coding format, the RTC server needs to transcode the fourth media stream information into the preset coding format; if the coding format of the fourth media stream information accords with the corresponding preset coding format, transcoding is not needed.

Step S402, the fourth media stream information of one RTC terminal is sent to the SIP terminal.

According to the embodiment of the disclosure, the point-to-point call between the SIP terminal and the RTC terminal can be realized, the media stream information of the SIP terminal can be sent to the RTC terminal, the media stream information of the RTC terminal can be obtained, and the media stream information of the RTC terminal can be sent to the SIP terminal, so that the intercommunication between the SIP terminal and the RTC terminal is realized.

A converged communication system for online conferencing in accordance with an embodiment of the present disclosure is described below in conjunction with fig. 6.

As shown in fig. 6, the converged communication system for an online conference of an embodiment of the present disclosure includes a SIP docking system, a converged communication gateway, and an RTC service.

The SIP interfacing system includes a SIP server, a SIP terminal (including a SIP hardware terminal, a PSTN mobile phone, a PSTN landline phone, etc.), and the SIP server may be an enterprise SIP system, such as an enterprise telephone switch ip pbx system, where the ip pbx is used to manage access of the SIP terminal. The converged communication gateway, namely the converged communication gateway of the embodiment of the disclosure, can realize RTC/SIP protocol conversion, so that the SIP terminal and the RTC terminal are converged and communicated. The RTC service system and the RTC edge media service in the RTC service may be understood as the RTC server in the foregoing embodiments of the disclosure, where the RTC service is an RTC terminal joining an RTC conference service. The converged communication gateway is described in detail below.

The converged communication gateway includes 3 services: SIP signaling proxy service, SIP media service, thrasher service (SIP/RTC protocol conversion service); wherein the SIP media services may include a plurality of SIP media services, and each SIP media service is bound with one router service; the SIP signaling proxy service may support high availability of virtual IP addresses (Virtual IP Address, VIP) enabling master-slave switching of the SIP signaling proxy service.

In a specific implementation, the SIP signaling proxy service, the plurality of SIP media services and the plurality of protocol conversion services may be deployed on the same server or may be deployed on different servers. For example, when deployed on different servers, the SIP signaling proxy service may be deployed on one server, and one SIP media service and one protocol conversion service of the binding may be deployed on one server.

1) The SIP signaling proxy service mainly plays a role of an SIP signaling route, interfaces with the SIP interfacing system, realizes the incoming and outgoing interfacing of the SIP terminal, dispatches and distributes the SIP media service, and intelligently routes the call request of the SIP terminal to the corresponding SIP media service.

When the SIP signaling proxy service is in butt joint with the SIP butt joint system, the SIP standard butt joint modes of SIP registration and IP butt joint are supported, and incoming and outgoing calls are supported, so that the incoming calls of the SIP terminal and the PSTN mobile fixed phone can be accepted to join in the RTC conference, and the RTC terminal can also actively call the SIP terminal and the PSTN mobile fixed phone.

When the SIP signaling proxy service intelligently routes the SIP media service, a load balancing (Load blance serve, LBS) policy may be adopted, after the SIP signaling proxy service receives a call request of the SIP terminal, the SIP signaling proxy service may allocate an available SIP media service according to the load of the SIP media service and the IP proximity principle, and ensure low latency of the edge node, but if a SIP conference added by the SIP terminal already exists, the SIP signaling proxy service may be allocated to a SIP media service corresponding to the SIP conference.

2) The SIP media service may be based on an open source switch, where the switch is a SIP soft switch system, and integrates Sofia, conference, codec, RTP, SFU, verto, ESL processing modules.

And Sofia is a SIP signaling media module used for receiving or initiating the call processing of the SIP terminal.

Ccmonreference: the conference management module is used for SIP conference room management, audio mixing and screen mixing processing and is based on an MCU architecture.

And the CCCODEC is used for performing audio encoding and decoding processing and video encoding and decoding processing.

RTRTP, RTP engine module, which is used to receive and send RTP media stream, RTP is a standard output.

SFU, SFU media transfer module, transfer the media stream of SIP terminal to RTC edge media service, at RTC service side, RTC terminal can subscribe the media stream of SIP terminal selectively.

Verto: the Verto signaling media module provides a websocket+ rpc (Remote Procedure Call ) mode to carry out signaling media negotiation, and is different from the SIP signaling in that the SIP signaling adopts a standard protocol, the SIP signaling carries the sdp (Session Description Protoco, protocol describing session) media stream information, the websocket+ rpc in Verto supports websocket connection, and the websocket transmits the json (JavaScript Object Notation, JS object numbered musical notation) format of the sdp media stream information.

ESL: the ESL event processing module is connected to the SIP media service through the ESL event processing module, and the generated event can be notified to the SIP/RTC conversion service in the process of processing the incoming and outgoing call of the SIP user by the SIP media service, and meanwhile, the event processing module also receives and processes a control command (call control and SIP conference management) sent by the SIP/RTC conversion service.

3) The router service is used for converting the SIP and RTC protocols, and is a key for fusion and intercommunication of the SIP terminal and the RTC terminal.

Based on the converged communication system, the SIP terminal is added into the RTC conference scene and the SIP terminal and the RTC terminal point-to-point call scene are respectively introduced.

Scene 1: SIP terminal joining RTC conference

Firstly, an RTC terminal requests an RTC server to create an RTC conference in advance, an RTC conference identifier and an SIP conference short number are generated, and the RTC conference identifier and the SIP conference short number have a binding relation.

The SIP terminal dials the SIP conference short number, calls into the SIP signaling proxy service in the converged communication gateway through the SIP system of the enterprise, the SIP signaling proxy intelligently routes the call request to the corresponding SIP media service, the SIP media service receives the call request, performs dialing matching, joins the SIP terminal in the SIP conference, and notifies the Threster service of the joining of the SIP terminal in the SIP conference event.

The server service receives the notification of the SIP terminal joining the SIP conference event, firstly logs in and authenticates to the RTC service system, after the authentication is passed, distributes a nearby RTC edge media service through the dispatching service of the RTC service system, and responds to the server service with the IP address of the RTC edge media service and the RTC conference identifier (such as RTC conference room information).

When the Threslurr service pushes the media stream of the SIP terminal to the RTC edge media service, the SIP terminal is simulated to be the RTC terminal, the RTC conference is requested to be added to the RTC edge media service, if the RTC conference is successful, the Threslurr service and the RTC edge media service create a media transmission channel to perform signaling interaction such as media negotiation, and then an SFU push control command is sent to the SIP media service through the ESL event processing module, the SFU media forward forwards the media stream of the SIP terminal to the RTC edge media service, and if the media stream coding format of the SIP terminal is different from the coding format negotiated by the RTC edge media service, the SFU media forward module firstly carries out transcoding processing on the media stream of the SIP terminal and then transmits the media stream to the RTC edge media service through RTP. The RTC edge media service is based on an SFU architecture, and the RTC terminal can selectively subscribe the audio streams or video streams of the SIP terminal and the RTC terminal, so that the RTC terminal can receive the media streams of the SIP terminal through the scheme.

Similarly, when the media stream of the RTC terminal is pulled to the SIP media service, the server simulates a fake RTC terminal to join the RTC conference, responds to obtain list information of all the RTC terminals in the RTC conference, simulates each RTC terminal to be registered with the SIP media service connection, after each verio terminal is successfully registered, the server creates a media transmission channel with the RTC edge media service and performs media negotiation, and then calls the SIP media service through the verio signaling, so that the SIP media service joins each verio terminal into the SIP conference, carries media stream information of the RTC terminal during calling, and after each verio terminal joins the SIP conference successfully, the SIP media service sends media address information of the SIP media service to the server, and then the server sends the media stream of the pulled RTC terminal to the SIP conference. The SIP conference of the SIP media service is an MCU architecture, the media streams of all terminals in the SIP conference are subjected to audio mixing and screen mixing processing, and then the media streams after audio mixing and screen mixing are sent to the SIP terminals, so that the SIP terminals display a screen mixing picture of the SIP conference, the SIP terminals can receive the media streams of the RTC terminals, and the screen mixing picture can be used for adjusting layout through conference control management.

In the above scenario, when a plurality of SIP terminals join in an RTC conference at the same time, for the video stream information, because the conference picture displayed by each SIP terminal is the same, the video stream information of each terminal in the RTC conference is converged by the communication gateway, and only one screen mixing process is required, so that the performance overhead can be greatly reduced; for the audio stream information, because each SIP terminal needs to exclude the collected sound, the audio stream information of each terminal in the RTC conference is required to be mixed for N-1 times by the convergence communication gateway, wherein N is the number of each terminal, and compared with the mixing screen processing, the mixing processing has small performance cost, so that large performance cost is not caused.

Scene 2: SIP terminal and RTC terminal point-to-point call

Unlike the above SIP terminal joining RTC conference scenario, SIP media services do not need to perform mixing and screen mixing processing due to the 1-to-1 call. When the SIP terminal calls into the converged communication gateway, the Threslurr service can simulate the RTC terminal as the Verto terminal, register to the SIP media service connection, finally convert to the SIP terminal to call the Verto terminal through the SIP media service call processing, the Threslurr service receives the signaling of the SIP terminal to call the Verto terminal, the signaling carries the media stream of the SIP terminal, then the Threslurr service carries out signaling interaction with the RTC edge media service, creates an SIP uplink media channel and an RTC downlink media channel, and responds the Verto signaling carrying the RTC media stream to the SIP media service, thereby realizing the point-to-point intercommunication of the SIP terminal and the RTC terminal. Compared with the situation that a plurality of SIP terminals join in the RTC conference, the method has the greatest advantage that the SIP media service is not required to be subjected to audio mixing and screen mixing processing, and the performance cost of the SIP media service is greatly reduced.

It should be noted that, in the point-to-point call scenario of the SIP terminal and the RTC terminal, the convergence communication gateway supports two modes of signaling proxy and media forwarding. The integrated communication gateway does not receive the media stream of the SIP terminal, namely the media stream of the SIP terminal is directly communicated with the RTC edge media service by the SIP system of the enterprise, and the integrated communication gateway is only responsible for RTC/SIP signaling conversion, so that the performance cost is almost negligible; the media forwarding mode is that the media stream of the SIP terminal is firstly subjected to the SIP media service of the converged communication gateway and then is communicated with the RTC edge media service by the SIP media service, so that the SIP system of an enterprise can be compatible.

The embodiment of the disclosure gives consideration to two call scenes of intercommunication between the SIP terminal and the RTC terminal, namely that the SIP terminal joins the RTC conference and the SIP terminal and the RTC terminal are in point-to-point call. In the scene that the SIP terminals join in the RTC conference, a plurality of SIP terminals can join in the RTC conference, and through RTC/SIP signaling protocol conversion, two architecture modes of MCU and SFU are compatible, so that the SIP terminals and the RTC terminals can be interconnected and intercommunicated. In the SIP and RTC point-to-point calling scene, the RTC/SIP signaling protocol conversion is optimized, the audio mixing and screen mixing processing of SIP media service is not needed, and the performance cost is greatly reduced.

Based on the same inventive concept, the embodiments of the present disclosure further provide a converged communication gateway, and the principle of solving the problem of the gateway is similar to that of the method of the foregoing embodiments, so that the implementation of the gateway may refer to the implementation of the method, and the repetition is omitted.

Referring to fig. 7, a converged communication gateway provided in an embodiment of the present disclosure includes a SIP signaling proxy module 71, at least one SIP media service module 72, and at least one protocol conversion module 73;

the SIP signaling proxy module 71 is configured to receive a call request carrying a call identifier of a SIP terminal sent by a session initiation protocol SIP server, and distribute the call request to the corresponding SIP media service module 72;

the SIP media service module 72 is configured to establish a call event carrying a call identifier according to the call request, and send the call event to the corresponding protocol conversion module 73;

the protocol conversion module 73 is configured to obtain an RTC conference identifier of a real-time communication RTC conference that is bound to the call identifier, send an RTC conference joining request carrying the RTC conference identifier of the SIP terminal to the RTC server, and after determining that the RTC server joins the SIP terminal to the RTC conference, establish a first media transmission channel between the SIP service module and the RTC server, and send a media forwarding instruction to the SIP media service module 72;

The SIP media service module 72 is further configured to respond to the media forwarding instruction, and send the first media stream information of the SIP terminal sent by the SIP server to the RTC server through the first media transmission channel, so that the RTC server sends the first media stream information to the corresponding RTC terminal in the RTC conference.

The SIP signaling proxy module 71, the at least one SIP media service module 72, and the at least one protocol conversion module 73 may be disposed on the same server, or may be disposed on different servers. For example, when deployed on different servers, the SIP signaling proxy module 71 may be deployed on one server, and one SIP media service module 72 and one protocol conversion module 73 bound may be deployed on one server.

The SIP signaling proxy module 71, the SIP media service module 72, and the protocol conversion module 73 may be understood as the SIP signaling proxy service, the SIP media service, and the router service in fig. 5 in the above-described embodiment.

In one possible implementation, the call request is a join SIP conference request and the call identifier is a SIP conference identifier;

the SIP media service module 72 further includes:

the SIP conference management sub-module is used for responding to a SIP conference joining request carrying a SIP conference identifier sent by the SIP terminal and joining the SIP terminal into the SIP conference corresponding to the SIP conference identifier;

The protocol conversion module 73 further includes:

and the acquisition sub-module is used for acquiring the RTC conference identifier of the RTC conference which is bound with the SIP conference identifier.

In one possible implementation, the protocol conversion module 73 further includes:

the first channel establishing sub-module is used for joining each RTC terminal in the RTC conference into the SIP conference and establishing a second media transmission channel with the RTC server;

the SIP media service module 72 further includes:

the receiving sub-module is used for receiving second media stream information of each RTC terminal in the RTC conference, which is sent by the RTC server, through a second media transmission channel;

the fusion sub-module is used for carrying out fusion processing on the respective second media stream information of each RTC terminal and the respective third media stream information sent by each SIP terminal in the SIP conference to obtain target media stream information;

In one possible implementation, the second media stream information and the third media stream information each include audio stream information;

the fusion sub-module is also used for:

and mixing the second media stream information of each RTC terminal and the third media stream information sent by each SIP terminal in the SIP conference to obtain the first target media stream information.

the fusion sub-module is also used for:

the protocol conversion module 73 further includes:

the second channel establishing sub-module is used for establishing a third media transmission channel with the RTC server and receiving fourth media stream information of one RTC terminal in the RTC conference, which is sent by the RTC server, through the third media transmission channel;

and the second sending sub-module is used for sending the fourth media stream information of one RTC terminal to the SIP terminal.

In one possible implementation, the SIP media service module 72 further includes:

the conversion sub-module is used for converting the first media stream information of the SIP terminal sent by the SIP server into a preset coding format;

and the request verification sub-module is used for sending a verification request carrying the call identifier to the RTC server so as to obtain the RTC conference identifier of the RTC conference bound with the call identifier, which is returned by the RTC server after verification is passed.

Based on the same inventive concept, the embodiments of the present disclosure further provide an electronic device, which has a similar principle of solving the problem as the method of the foregoing embodiments, so that the implementation of the electronic device may refer to the implementation of the method, and the repetition is omitted. Fig. 8 shows a schematic structural diagram of an electronic device according to an embodiment of the disclosure.

Referring to fig. 8, an electronic device may include a processor 802 and a memory 801. The memory 801 provides program instructions and data stored in the memory 801 to the processor 802. In the disclosed embodiments, the memory 801 may be used to store programs for converged communication of on-line conferences in the disclosed embodiments.

The processor 802 is configured to execute the method of any of the above-described method embodiments, such as the fusion communication method for online conferencing provided by the embodiment shown in fig. 2, by invoking program instructions stored in the memory 801.

The particular connection medium between the memory 801 and the processor 802 described above is not limited in the presently disclosed embodiments. The embodiment of the present disclosure is illustrated in fig. 8 by a bus 803 connected between a memory 801 and a processor 802, where the bus 803 is illustrated in fig. 8 by a bold line, and the connection between other components is merely illustrative and not limiting. The bus 803 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 8, but not only one bus or one type of bus.

The Memory may include Read-Only Memory (ROM) and random access Memory (Random Access Memory, RAM), and may also include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; but also digital instruction processors (Digital Signal Processing, DSP), application specific integrated circuits, field programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

The disclosed embodiments also provide a computer storage medium, in which a computer program is stored, and from which a processor of a computer device reads the computer program, and the processor executes the computer program, so that the computer device executes the converged communication method of the online conference in any of the above method embodiments.

In a specific implementation, the computer storage medium may include: a universal serial bus flash disk (USB, universal Serial Bus Flash Drive), a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, or the like, which can store program codes.

In some possible embodiments, aspects of the converged communication method for online conferences provided by the present disclosure may also be implemented in the form of a program product comprising program code for causing a computer device to perform the steps of the converged communication method for online conferences according to the various exemplary embodiments of the present disclosure described above when the program product is run on the computer device, e.g. the computer device may perform the converged communication flow for online conferences as in steps S201-S203 shown in fig. 2.

It will be apparent to those skilled in the art that embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to the disclosure. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present disclosure without departing from the spirit or scope of the disclosure. Thus, the present disclosure is intended to include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. A converged communication method for an online conference, which is applied to a converged communication gateway, comprising:

simulating the SIP terminal into an RTC terminal, sending an RTC conference joining request of the SIP terminal to an RTC server, and establishing a first media transmission channel with the RTC server after determining that the RTC server joins the SIP terminal into the RTC conference; wherein, the RTC conference request comprises the RTC conference identifier;

the first media stream information of the SIP terminal sent by the SIP server is sent to the RTC server through the first media transmission channel, so that the RTC server sends the first media stream information to a corresponding RTC terminal in the RTC conference;

if the call request is a request for joining an SIP conference, the call identifier is an SIP conference identifier;

receiving a call request carrying a call identifier of an SIP terminal, and acquiring an RTC conference identifier of a real-time communication RTC conference bound with the call identifier, wherein the method comprises the following steps:

Responding to a SIP conference joining request of the SIP terminal carrying a SIP conference identifier, and joining the SIP terminal into a SIP conference corresponding to the SIP conference identifier;

acquiring an RTC conference identifier of an RTC conference bound with the SIP conference identifier;

the method further comprises the steps of:

simulating an RTC virtual terminal to join an RTC conference, obtaining list information of each RTC terminal in the RTC conference returned by the RTC server, joining each RTC terminal in the RTC conference into the SIP conference, and establishing a second media transmission channel with the RTC server;

based on a multipoint control unit MCU architecture, carrying out fusion processing on the respective second media stream information of each RTC terminal and the respective third media stream information sent by each SIP terminal in the SIP conference to obtain target media stream information;

2. The method of claim 1, wherein the second media stream information and the third media stream information each comprise audio stream information;

3. The method of claim 2, wherein the second media stream information and the third media stream information each further comprise video stream information;

4. The method of claim 1, wherein if the call request is a call RTC terminal request;

The method further comprises the steps of:

5. The method according to any one of claims 1 to 4, wherein the sending the first media stream information of the SIP terminal sent by the SIP server to the RTC server through the first media transmission channel includes:

6. The method according to any one of claims 1 to 4, wherein the obtaining the RTC conference identity of the real-time communication RTC conference bound to the call identity comprises:

7. The converged communication gateway is characterized by comprising a Session Initiation Protocol (SIP) signaling proxy module, at least one SIP media service module and at least one protocol conversion module;

the protocol conversion module is used for acquiring an RTC conference identifier of a real-time communication RTC conference bound with the call identifier, simulating the SIP terminal into an RTC terminal, sending an RTC conference joining request carrying the RTC conference identifier of the SIP terminal to an RTC server, establishing a first media transmission channel of the SIP media service module and the RTC server after determining that the RTC server joins the SIP terminal into the RTC conference, and sending a media forwarding instruction to the SIP media service module;

the SIP media service module is further configured to respond to the media forwarding instruction, and send first media stream information of the SIP terminal sent by the SIP server to the RTC server through the first media transmission channel, so that the RTC server sends the first media stream information to a corresponding RTC terminal in the RTC conference;

The call request is a request for joining an SIP conference, and the call identifier is an SIP conference identifier;

the SIP media service module further includes:

the protocol conversion module further includes:

an acquisition sub-module, configured to acquire an RTC conference identifier of an RTC conference that is bound to the SIP conference identifier;

the protocol conversion module further includes:

a first channel establishing sub-module, configured to simulate an RTC virtual terminal to join an RTC conference, obtain list information of each RTC terminal in the RTC conference returned by the RTC server, join each RTC terminal in the RTC conference to the SIP conference, and establish a second media transmission channel with the RTC server;

the SIP media service module further includes:

the fusion sub-module is used for carrying out fusion processing on the respective second media stream information of each RTC terminal and the respective third media stream information sent by each SIP terminal in the SIP conference based on the MCU architecture of the multipoint control unit to obtain target media stream information;

8. The gateway of claim 7, wherein the second media stream information and the third media stream information each comprise audio stream information;

the fusion sub-module is further configured to:

9. The gateway of claim 8, wherein the second media stream information and the third media stream information each further comprise video stream information;

the fusion sub-module is further configured to:

10. The gateway of claim 7, wherein the call request is a call RTC terminal request;

The protocol conversion module further includes:

11. The gateway according to any of claims 7 to 10, wherein the SIP media service module further comprises:

12. The gateway of any of claims 7 to 10, wherein the protocol conversion module further comprises:

13. An electronic device comprising a processor and a memory, wherein the memory stores program code that, when executed by the processor, causes the processor to perform the steps of the method of any of claims 1-6.

14. A computer readable storage medium, characterized in that it comprises a program code for causing an electronic device to perform the steps of the method according to any one of claims 1-6, when said program code is run on the electronic device.