WO2023197593A1 - Procédé et appareil de commande de conférence multimédia, et système de communication - Google Patents

Procédé et appareil de commande de conférence multimédia, et système de communication Download PDF

Info

Publication number
WO2023197593A1
WO2023197593A1 PCT/CN2022/131215 CN2022131215W WO2023197593A1 WO 2023197593 A1 WO2023197593 A1 WO 2023197593A1 CN 2022131215 W CN2022131215 W CN 2022131215W WO 2023197593 A1 WO2023197593 A1 WO 2023197593A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal
audio stream
audio
gateway
called
Prior art date
Application number
PCT/CN2022/131215
Other languages
English (en)
Chinese (zh)
Inventor
廖涛
Original Assignee
华为云计算技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202210666906.1A external-priority patent/CN116962364A/zh
Application filed by 华为云计算技术有限公司 filed Critical 华为云计算技术有限公司
Publication of WO2023197593A1 publication Critical patent/WO2023197593A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/10—Architectures or entities
    • H04L65/102—Gateways
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40—Support for services or applications
    • H04L65/403—Arrangements for multi-party communication, e.g. for conferences
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems

Definitions

  • the present application relates to the field of communication technology, and in particular to a multimedia conference control method and device, and a communication system.
  • Multimedia conferences refer to virtual conferences realized through communication technology, which can allow geographically dispersed individuals or groups to gather together and exchange information through graphics, sound and other methods.
  • telephone terminals When conducting multimedia conferences, there are often some mobile phones, fixed-line terminals, etc. (hereinafter referred to as telephone terminals) that access the multimedia conference system through the telephone switching network.
  • the telephone switching network is the public switched telephone network (public switched telephone network). network (PSTN) or a private network based on a private branch exchange (PBX).
  • PSTN public switched telephone network
  • PBX private branch exchange
  • a call connection (for example, called call connection 1) is established between the telephone terminal and other terminals (for example, a conference terminal) in the multimedia conference system.
  • the call connection 1 includes the telephone terminal and the telephone.
  • the call connection between the switching networks and the call connection between the telephone switching network and the other terminal, and the media stream (eg audio stream) transmitted between the telephone terminal and the other terminal is forwarded through the telephone switching network.
  • the telephone terminal connected to the multimedia conference system carries out other telephone services (such as answering new phone calls, making new calls)
  • the telephone switching network will connect the call between the telephone terminal and other terminals in the multimedia conference system.
  • call hold that is, controls the call connection 1 to be in the call hold state
  • called prompt tone or call hold prompt tone
  • the other terminal will play the called prompt tone to alert the user of the other terminal.
  • the present application provides a multimedia conference control method and device, and a communication system, which helps to avoid the noise of the telephone terminal being called (such as the called prompt tone) from affecting the development of the multimedia conference.
  • the technical solutions of this application are as follows:
  • a multimedia conference control method includes: the target gateway receives an audio stream sent to the first terminal through the telephone switching system; the target gateway performs denoising processing on the audio stream, and the denoising processing is: Remove the noise of the second terminal being called in the audio stream.
  • the noise when the second terminal is called may be a called prompt sound when the second terminal is called by a terminal other than the first terminal (for example, a third terminal).
  • the called prompt sound is noise for the first terminal.
  • the target gateway after the target gateway receives the audio stream sent to the first terminal through the telephone switching system, the target gateway removes the noise of the second terminal being called in the audio stream, so it can avoid the noise of the second terminal being called.
  • the noise interferes with the first terminal, thereby preventing the second terminal from being called by noise from affecting the development of the multimedia conference.
  • the method before the target gateway performs denoising processing on the audio stream, the method also includes: the target gateway determines that the second terminal is in a called state based on the characteristic parameters of the audio stream. For example, the target gateway determines that the second terminal is in a state of being called by a terminal other than the first terminal (for example, a third terminal) according to the characteristic parameters of the audio stream.
  • the target gateway can determine that the second terminal is in the called state based on the characteristic parameters of the audio stream, that is, the target gateway can sense that the second terminal is in the called state based on the characteristic parameters of the audio stream, so the target gateway The audio stream may be denoised to remove the noise of the second terminal being called in the audio stream.
  • the characteristic parameters include at least one of the following: audio characteristics of the audio stream, and data characteristics of the data packets of the audio stream.
  • the characteristic parameters include audio characteristics of the audio stream, and the audio characteristics include voice segments contained in the audio stream.
  • the target gateway determines that the second terminal is in the called state based on the characteristic parameters of the audio stream, including: target gateway comparison
  • the audio stream contains a voice segment and a designated voice segment, which determines that the second terminal is in a called state, and the designated voice segment is used to describe that the second terminal is in a called state.
  • the target gateway compares the voice segment contained in the audio stream with the specified voice segment, and determines that the voice segment included in the audio stream includes the specified voice segment. Therefore, the target gateway determines that the second terminal is in a called state.
  • the target gateway compares the voice segment contained in the audio stream with the specified voice segment, and determines that the similarity between the voice segment contained in the audio stream and the specified voice segment is greater than the similarity threshold. Therefore, the target gateway determines that the second terminal is in the called state. .
  • the method also includes: the target gateway sends the audio stream to the audio recognition device, and the audio recognition device is used to perform audio recognition on the audio stream to obtain the audio characteristics of the audio stream; the target gateway receives the audio stream sent by the audio recognition device. The audio characteristics of this audio stream.
  • the target gateway sends an audio stream to the audio recognition device, which can facilitate the audio recognition device to perform audio recognition on the audio stream and obtain the audio characteristics of the audio stream.
  • the audio recognition device sends the audio of the audio stream to the target gateway.
  • Features which can facilitate the target gateway to obtain the audio features of the audio stream.
  • the characteristic parameter includes the data characteristic of the data packet of the audio stream.
  • the target gateway determines that the second terminal is in the called state according to the characteristic parameter of the audio stream, including: the target gateway determines that the second terminal is in the called state according to the data packet of the audio stream.
  • the identifier determines that the second terminal is in the called state, and the first identifier is used to indicate that the second terminal is in the called state.
  • the method before the target gateway performs denoising processing on the audio stream, the method also includes: the target gateway receives the target signaling message; the target gateway determines that the second terminal is in a state based on the target signaling message containing specified signaling information. Call state, this designated signaling information is used to indicate that the second terminal is in the called state.
  • the target gateway determines that the second terminal is in the called state based on the target signaling message containing designated signaling information, that is, the target gateway can sense that the second terminal is in the called state based on the target signaling message containing designated signaling information.
  • the target gateway determines that the audio stream sent to the first terminal may contain the noise of the second terminal being called.
  • the target gateway performs denoising processing on the audio stream sent to the first terminal to remove the second noise in the audio stream. Terminal being called noise.
  • the denoising process includes: intercepting the audio stream. That is, the target gateway does not forward the audio stream to the first terminal.
  • the target gateway intercepts the audio stream, which can prevent the audio stream from reaching the first terminal, thereby preventing the first terminal from playing the audio stream, and thereby preventing the noise of the second terminal being called in the audio stream from affecting the third terminal.
  • One terminal causes interference.
  • the denoising process includes: replacing data packets of the audio stream with silence packets, and sending the silence packets to the first terminal.
  • the silent package meets any of the following conditions: does not include audio data, includes audio data, and the audio data cannot trigger physical sound perception.
  • a silence packet is a data packet encapsulated according to the audio protocol and format, and the payload of the silence packet is empty.
  • the payload of the silent package is empty. This means that the silent package does not include the payload, or the silent package includes the payload but the data in the payload is 0.
  • the silent package plays without any sound and cannot induce physical sound perception.
  • the target gateway since the target gateway replaces the data packet of the audio stream with a silence packet and sends the silence packet to the first terminal, that is, the target gateway does not send the audio stream to the first terminal, therefore, the audio stream can be avoided.
  • the stream reaches the first terminal, thereby preventing the first terminal from playing the audio stream, and thereby preventing the noise of the second terminal being called in the audio stream from interfering with the first terminal.
  • the denoising process includes: adding a second identifier to the data packet of the audio stream and sending the data packet of the audio stream to the first terminal, where the second identifier is used to instruct the first terminal not to play the audio stream.
  • the target gateway since the target gateway adds the second identifier to the data packet of the audio stream and sends the data packet of the audio stream including the second identifier to the first terminal, after the first terminal receives the audio stream , the first terminal does not play the audio stream, which can avoid the noise of the second terminal being called in the audio stream from interfering with the first terminal.
  • a second aspect provides a multimedia conference control device, including various modules for executing the method provided in the above-mentioned first aspect or any optional manner of the first aspect.
  • the modules can be implemented based on software, hardware, or a combination of software and hardware, and the modules can be arbitrarily combined or divided based on specific implementations.
  • a multimedia conference control device including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory so that the control device performs the above-mentioned first aspect or the third aspect. Methods provided by either alternative on the one hand.
  • a communication system in a fourth aspect, includes a target gateway, a first terminal and a second terminal.
  • the first terminal is communicatively connected to the target gateway.
  • the second terminal is communicatively connected to the target gateway through a telephone switching system.
  • the target gateway It includes a multimedia conference control device as provided in the above second or third aspect.
  • a computer-readable storage medium is provided.
  • a computer program is stored in the computer-readable storage medium.
  • the implementation is as provided in the above-mentioned first aspect or any optional manner of the first aspect. Methods.
  • a computer program product in a sixth aspect, includes a program or code.
  • the program or code When the program or code is executed, the method provided by the above-mentioned first aspect or any optional manner of the first aspect is implemented.
  • a chip in a seventh aspect, includes programmable logic circuits and/or program instructions. When the chip is run, it is used to implement the method provided by the above-mentioned first aspect or any optional manner of the first aspect.
  • the communication system includes a target gateway, a first terminal, and a second terminal.
  • the first terminal is communicatively connected to the target gateway, and the second terminal communicates with the target gateway through a telephone switching system.
  • the target gateway performs denoising processing on the audio stream to remove the noise of the second terminal being called in the audio stream, so that the second terminal can be avoided
  • the called noise interferes with the first terminal, thereby preventing the called noise of the second terminal from affecting the development of the multimedia conference.
  • a call connection (for example, called call connection 1) is established between the second terminal and the first terminal.
  • the second terminal On the basis that the call connection 1 is established between the second terminal and the first terminal, if the second terminal is based on There are various possible reasons for establishing a call connection 2 with the third terminal (or the second terminal needs to establish a call connection 2 with the third terminal, where the third terminal can be any terminal other than the first terminal), for example, the second terminal needs to establish a call connection 2 with the third terminal.
  • the terminal calls the third terminal, or the second terminal answers the third terminal's phone call, or the second terminal communicates with the third terminal, etc., which will cause the second terminal to establish a call connection 2 with the third terminal, and the telephone switching system will The call connection 1 is on hold, and the telephone switching system will send the called prompt tone to the first terminal.
  • the called prompt tone is likely to cause interference to the first terminal, and the called prompt tone is due to the second terminal establishing a new call.
  • the called prompt sound is generated by the call connection 2, so the called prompt sound is noise for the first terminal, and the called prompt sound can be called the noise of the second terminal being called. Since the target gateway removes the noise of the second terminal being called from the audio stream sent by the telephone switching system to the first terminal, it can avoid the noise of the second terminal being called from interfering with the first terminal.
  • Figure 1 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • Figure 2 is a schematic diagram of another application scenario provided by the embodiment of the present application.
  • Figure 3 is a schematic diagram of yet another application scenario provided by the embodiment of the present application.
  • Figure 4 is a flow chart of a multimedia conference control method provided by an embodiment of the present application.
  • Figure 5 is a flow chart of another multimedia conference control method provided by an embodiment of the present application.
  • Figure 6 is a flow chart of a second terminal accessing a multimedia conference system provided by an embodiment of the present application.
  • Figure 7 is a flow chart of a second terminal in a called state provided by an embodiment of the present application.
  • Figure 8 is a flow chart of another second terminal in a called state provided by an embodiment of the present application.
  • Figure 9 is a flow chart of a second terminal canceling the called state provided by an embodiment of the present application.
  • Figure 10 is a schematic structural diagram of a multimedia conference control device provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of another multimedia conference control device provided by an embodiment of the present application.
  • the telephone switching network is PSTN or a private network based on PBX.
  • PSTN is a telecommunications network that provides telephone services to public users.
  • PSTN includes access systems, telephone switches, relays, etc.
  • PSTN is also called plain old telephone service (POTS).
  • POTS plain old telephone service
  • PBX is a computer-based digital telephone exchange that can be connected to the public switched telephone network and is usually used by businesses.
  • the telephone switching network is also called a telephone switching system, a telephone access system, etc.
  • the telephone terminal communicates with the media gateway in the multimedia conference system through the Internet protocol multimedia subsystem (IMS) and the access gateway in turn.
  • IMS Internet protocol multimedia subsystem
  • the media gateway can process the audio stream from the telephone terminal or the IMS. Code and decode, and send the audio stream to other terminals (such as conference terminals) in the multimedia conference system, so that other terminals in the multimedia conference system can play the audio stream from the phone terminal or from the IMS.
  • other terminals such as conference terminals
  • a call connection (for example, called call connection 1) is established between the telephone terminal and other terminals (for example, a conference terminal) in the multimedia conference system.
  • the call connection 1 includes the telephone terminal and the telephone.
  • the call connection between the switching networks and the call connection between the telephone switching network and the other terminal, and the media stream (eg audio stream) transmitted between the telephone terminal and the other terminal is forwarded through the telephone switching network.
  • 4G 4th generation mobile communication technology
  • 5G 5th generation mobile communication technology
  • a telephone terminal connected to the multimedia conference system carries out other telephone services, such as answering phone calls from terminals outside the multimedia conference system, making calls to terminals outside the multimedia conference system, etc.
  • the telephone switching network will The call connection 1 between the telephone terminal and other terminals in the multimedia conference system performs call hold (that is, controls the call connection 1 to be in the call hold state), and sends the called prompt tone to the other terminal, for example, the called The prompt tone is "The user you dialed is currently on a call" etc.
  • the other terminal will play the called prompt tone to alert the user of the other terminal.
  • the called prompt tone can easily cause interference to other terminals, affecting the development of multimedia conferences.
  • call hold is a type of service that allows the established call connection (such as the aforementioned call connection 1) to be maintained, that is, the transmission of the media stream (such as audio stream) between the calling terminal and the called terminal is stopped.
  • the session resources are not released and the call connection is not removed.
  • the call connection can be restored (or reactivated) when the call hold ends or based on other needs.
  • the user of the telephone terminal can operate the mute key of the telephone terminal or control the local muting of the telephone terminal based on other muting methods provided by the multimedia conferencing system.
  • the telephone terminal After the telephone terminal is muted locally, the telephone terminal does not send media streams to the multimedia conference system, but the call connection (for example, call connection 1) between the telephone terminal and other terminals in the multimedia conference system still exists.
  • the call connection for example, call connection 1
  • the user of the telephone terminal first reacts that the telephone terminal is muted in the multimedia conference system, and may answer the new telephone call.
  • the telephone terminal After the telephone terminal answers the new telephone call, the telephone terminal establishes a call connection (for example, call connection 2) with the calling party of the new telephone call.
  • a call connection for example, call connection 2
  • the telephone switching network defaults to the existence of two call connections for the telephone terminal. Since the telephone terminal answers a new telephone call (that is, call connection 2 is established), the telephone switching network will put the call on hold for call connection 1, and The called prompt tone (or call holding prompt tone) is sent cyclically to other terminals in the multimedia conference system, and other terminals in the multimedia conference system will play the called prompt tone in a loop, seriously interfering with the development of the multimedia conference.
  • Embodiments of the present application provide a multimedia conference control method, device and communication system.
  • the communication system includes a target gateway, a first terminal and a second terminal.
  • the first terminal is communicatively connected to the target gateway
  • the second terminal is communicatively connected to the target gateway through a telephone switching system.
  • the target gateway is a media gateway in a multimedia conference system.
  • One terminal is a conference terminal in the multimedia conference system
  • the second terminal is a telephone terminal connected to the multimedia conference system.
  • the target gateway After the target gateway receives the audio stream sent to the first terminal through the telephone switching system, the target gateway performs denoising processing on the audio stream to remove the noise of the second terminal being called in the audio stream, such as the noise of the second terminal being called. It is the called prompt tone of the second terminal.
  • the target gateway Since the target gateway removes the noise of the second terminal being called in the audio stream sent to the first terminal, it can avoid the noise of the second terminal being called from interfering with the first terminal, thereby preventing the noise of the second terminal being called from affecting the multimedia conference. carry out.
  • FIG. 1 shows a schematic diagram of an application scenario provided by an embodiment of the present application.
  • the application scenario provides a communication system, which includes a target gateway, a first terminal and a second terminal.
  • the first terminal is communicatively connected with the target gateway
  • the second terminal is communicatively connected with the target gateway through the telephone switching system.
  • the communication system includes a multimedia conference system and a telephone switching system, and both the target gateway and the first terminal can be located in the multimedia conference system.
  • the target gateway may receive the audio stream sent to the first terminal through the telephone switching system, and perform denoising processing on the audio stream to remove the noise of the second terminal being called in the audio stream.
  • the target gateway first determines that the second terminal is in a called state, and then removes the noise of the second terminal being called from the audio stream sent to the first terminal.
  • the "second terminal is in the called state" mentioned in this application refers to the state in which the second terminal establishes a call connection with a terminal other than the first terminal and the call connection is activated. For example, "the second terminal is in the called state".
  • “Status” may refer to a state in which the second terminal is called by a terminal other than the first terminal (for example, a third terminal, and the third terminal may be any terminal other than the first terminal), or the second terminal is in a state calling the first terminal.
  • the status of a terminal other than the terminal for example, a third terminal, which may be any terminal other than the first terminal.
  • a call connection for example, called call connection 1
  • call connection 1 is established between the second terminal and the first terminal in the multimedia conference system.
  • the target gateway can determine whether the second terminal is in the called state by determining whether the call connection 1 between the second terminal and the first terminal is in the call hold state (for example, determine whether the second terminal is in the called state).
  • the state of the third terminal call for example, if the target gateway determines that the call connection 1 between the second terminal and the first terminal is in the call hold state, the target gateway determines that the second terminal is in the called state; if the target gateway determines that the second terminal is in the called state; If the call connection 1 between the terminal and the first terminal is not in the call hold state, the target gateway determines that the second terminal is not in the called state.
  • the audio stream sent to the first terminal may be an audio stream sent from the second terminal to the first terminal, or may be an audio stream sent from the telephone switching system to the first terminal.
  • the telephone switching system can send the called prompt tone to the first terminal, and then the call sent to the first terminal
  • the audio stream may be the called prompt tone sent by the telephone switching system to the first terminal.
  • the communication system may include at least one conference terminal and at least one telephone terminal; the at least one conference terminal is communicatively connected with the target gateway, and the at least one conference terminal and the target gateway are both located in the multimedia conference system; At least one telephone terminal is communicatively connected to the target gateway through the telephone switching system, and the at least one telephone terminal can access the multimedia conference system through the telephone switching system.
  • the at least one telephone terminal is a plurality of telephone terminals
  • the plurality of telephone terminals may be communicatively connected to the target gateway through one telephone switching system, or may be communicatively connected to the target gateway through at least two telephone switching systems.
  • the conference terminal refers to a terminal that accesses the multimedia conference system through a conference application (or conference client).
  • the conference terminal can be a mobile phone, a netbook, a laptop, a tablet, etc.
  • the telephone terminal refers to the terminal that accesses the multimedia conference system through the telephone switching system.
  • the telephone terminal can be a mobile phone, a fixed-line terminal, etc.
  • the telephone switching system can be PSTN or a private network based on PBX.
  • the telephone switching network is also called telephone switching system, telephone access system, etc.
  • the second terminal may be any one of the at least one telephone terminal, and the number of the first terminals may be one or more.
  • the at least one conference terminal may be the first terminal.
  • Multimedia conference systems usually include media gateways, signaling gateways, media servers, conference terminals and other equipment.
  • the media server can be a selective forwarding unit (SFU).
  • SFU selective forwarding unit
  • the target gateway described in this application may be a media gateway, or a gateway that integrates the functions of a media gateway, a signaling gateway, and a media server.
  • the target gateway integrates both a signaling gateway and a media server. functions of the media gateway.
  • the telephone switching system is communicatively connected to the target gateway through an access gateway, and the access gateway is used for the telephone switching system to access the target gateway, thereby allowing the telephone switching system to access the multimedia conference system.
  • the access gateway may be a PSTN access gateway or a PBX, which is not limited in this embodiment of the present application.
  • Figure 2 shows a schematic diagram of another application scenario provided by the embodiment of this application.
  • This application scenario is illustrated by taking a media gateway that integrates the functions of a signaling gateway and a media server (that is, the target gateway is a media gateway that integrates the functions of both a signaling gateway and a media server).
  • the communication system provided by this application scenario includes a multimedia conference system, a telephone switching system 101, an access gateway 102 and at least one telephone terminal 103 ( Figure 2 takes a telephone terminal 103 as an example).
  • the multimedia conference system includes a media gateway 104 and at least one conference terminal 105 ( Figure 2 takes two conference terminals 105 as an example).
  • the at least one conference terminal 105 is communicatively connected with the media gateway 104.
  • the telephone terminal 103, the telephone switching system 101, and the access gateway 102 are connected in sequence.
  • the access gateway 102 is communicatively connected with the media gateway 104.
  • the access gateway 102 is used for the telephone switching system 101 to access the media gateway 104, thereby enabling communication with the telephone switching system.
  • the telephone terminal 103 connected through 101 communication accesses the multimedia conference system, and the access gateway 102 is used to forward media streams (eg audio streams) between the telephone switching system 101 and the media gateway 104 .
  • the first terminal may be the conference terminal 105, and the second terminal may be the telephone terminal 103.
  • the communication system shown in FIG. 2 includes two first terminals.
  • the media gateway 104 can receive the audio stream sent to the conference terminal 105 through the telephone switching system 101 and the access gateway 102, and perform denoising processing on the audio stream to remove the noise of the telephone terminal 103 being called in the audio stream.
  • the media gateway 104 first determines that the telephone terminal 103 is in a called state (for example, determines that the telephone terminal 103 is in a state of being called by a terminal other than the conference terminal 105), and then the media gateway 104 processes the audio stream sent to the conference terminal 105. Denoising processing is performed to remove the noise of the telephone terminal 103 being called in the audio stream.
  • the multimedia conference system also includes a conference management device 106.
  • the conference management device 106 is used to manage media resources (such as conference numbers, etc.) used in multimedia conferences, control telephone terminals, conference terminals, etc. to access the multimedia conference, and control Routing of media streams, scheduling instructions for media streams, etc.
  • the conference management device 106 is communicatively connected to the media gateway 104, and the conference management device 106 can send detection indication information to the media gateway 104 (for example, the detection indication information includes the phone number and detection identification of the telephone terminal 103) to instruct the media gateway 104 to
  • the media stream eg audio stream
  • the media stream is subjected to denoising processing to remove the noise of the telephone terminal 103 being called in the audio stream.
  • the call connection between the telephone terminal 103 and the conference terminal 105 is in the call hold state.
  • the call connection between the telephone terminal 103 and the conference terminal 105 is in a call hold state, possibly because the telephone terminal 103 carries out other telephone services.
  • the telephone terminal 103 receives a telephone call from another terminal, and the telephone terminal 103 is in a call hold state. The status of being called by this other terminal.
  • Figure 2 only shows the connection relationship between the conference management device 106 and the media gateway 104.
  • the conference management device 106 can also be connected to the conference terminal 105, the telephone terminal 103, etc. Which specifically the conference management device 106 is connected to.
  • the device or terminal connection can be set according to actual needs, which is not limited in the embodiments of this application.
  • the telephone switching system 101 can trigger the access gateway 102 to send a notification message to the media gateway 104.
  • the media gateway 104 determines that the call connection between the telephone terminal 103 and the conference terminal is in the call hold state based on the notification message, thereby determining that the telephone terminal 103 is in the call hold state. Called status.
  • the notification message may be session initiation protocol (SIP) signaling, private protocol signaling, or a media data packet carrying a special identifier or special information.
  • SIP session initiation protocol
  • the media gateway 104 determines that the telephone terminal 103 is in a called state based on characteristic parameters of the audio stream related to the telephone terminal 103 sent to the conference terminal 105 . For example, when the telephone terminal 103 is in a called state (for example, the telephone terminal 103 is in a state of being called by a terminal other than the conference terminal 105), and the call connection between the telephone terminal 103 and the conference terminal 105 is in a call hold state, the telephone switching system 101 can send the called prompt tone of the telephone terminal 103 (or the call holding prompt tone for holding the call connection between the telephone terminal 103 and the conference terminal 105) to the multimedia conference system, that is, the telephone switching system 101 can Send the audio stream to the multimedia conference system; the media gateway 104 can determine that the call connection between the telephone terminal 103 and the conference terminal 105 is in the call hold state according to the characteristic parameters of the audio stream, thereby determining that the telephone terminal 103 is in the called state.
  • a called state for example, the telephone terminal 103 is in a state of being
  • the characteristic parameters of the audio stream include at least one of the following: audio characteristics of the audio stream, and data characteristics of the data packets of the audio stream.
  • the media gateway 104 determines that the call connection between the telephone terminal 103 and the conference terminal 105 is in the call hold state according to the data characteristics of the data packet of the audio stream, thereby determining that the telephone terminal 103 is in the called state.
  • the media gateway 104 determines that the call connection between the telephone terminal 103 and the conference terminal 105 is in the call hold state based on the first identifier contained in the data packet of the audio stream, thereby determining that the telephone terminal 103 is in the called state.
  • the media gateway 104 determines that the call connection between the telephone terminal 103 and the conference terminal 105 is in the call hold state according to the audio characteristics of the audio stream, thereby determining that the telephone terminal 103 is in the called state. For example, the media gateway 104 determines that the call connection between the telephone terminal 103 and the conference terminal 105 is in the call hold state according to the audio stream including the specified voice segment, thereby determining that the telephone terminal 103 is in the called state; or, the media gateway 104 determines that the audio stream includes the designated voice segment.
  • the similarity between the included voice segment and the specified voice segment is greater than the similarity threshold, it is determined that the call connection between the telephone terminal 103 and the conference terminal 105 is in the call hold state, thereby determining that the telephone terminal 103 is in the called state.
  • the communication system also includes an audio recognition device 107.
  • the audio recognition device 107 is communicatively connected to the media gateway 104.
  • the media gateway 104 can send the audio stream related to the phone terminal 103 to the conference terminal 105.
  • the audio recognition device 107 can perform audio recognition on the audio stream sent by the media gateway 104 to obtain the audio characteristics of the audio stream, and send the audio characteristics of the audio stream to the media gateway 104, so that the media gateway 104 can identify the audio stream based on the audio stream.
  • the audio characteristics of the stream determine that the telephone terminal 103 is in a called state.
  • the media gateway 104 decodes the audio stream into an audio naked code stream file, usually a pulse code modulation (PCM) file, and sends the PCM file of the audio stream to the audio recognition device 107, and the audio recognition device 107 Perform audio identification on the audio stream based on its PCM file.
  • the audio recognition device 107 can be an automatic speech recognition (automatic speech recognition, ASR) device or other audio recognition devices.
  • the media gateway 104 integrates the functions of the signaling gateway and the media server (that is, the media gateway 104 includes the functions of the media gateway, the signaling gateway and the media server).
  • the media gateway 104 includes a signaling module, a media module, an audio processing module, a storage module, etc.
  • the signaling module is used for signaling interaction between the media gateway 104 and the conference management device 106, the access gateway 102, etc., for example,
  • the signaling module is used for the media gateway 104 to receive the scheduling signaling sent by the conference management device 106, and report the access status of the conference terminal 105, the telephone terminal 103, etc.
  • the access gateway 102 interacts with SIP signaling or private protocol signaling; the media module is used for the media gateway 104 to interact with the conference terminal 105, the access gateway 102, etc., for example, the media module is used for the media gateway 104 based on the real-time transmission protocol (real-time transport protocol, RTP) receives the audio stream sent by the access gateway 102, and sends the audio stream to the conference terminal 105 based on RTP, and receives the media stream sent by any conference terminal 105 based on RTP, and sends the audio stream to the conference terminal 105 based on RTP.
  • real-time transport protocol real-time transport protocol
  • the access gateway 102 and other conference terminals 105 send the media stream;
  • the audio processing module is used by the media gateway 104 to decode the audio stream sent to the conference terminal 105 into a PCM file, and send the PCM file of the audio stream to the audio recognition device 107.
  • receive the audio characteristics sent by the audio recognition device 107 and determine that the call connection between the telephone terminal 103 and the conference terminal 105 is in the call hold state according to the audio characteristics sent by the audio recognition device 107;
  • the storage module is used to store the specified audio characteristics (such as specifying Speech segment or similarity threshold), the specified audio feature can be stored in the storage module in the form of text.
  • the audio recognition device 107 may include an audio transceiver module and an audio recognition module.
  • the audio transceiver module is used for the audio recognition device 107 to receive the audio stream sent by the media gateway 104 and provide the audio stream to the audio recognition module for analysis and identification.
  • the audio recognition module The module is used to analyze and identify the received audio stream and obtain the audio characteristics of the audio stream.
  • the access gateway 102 may include a signaling module and a media module.
  • the signaling module is used for signaling interaction between the access gateway 102 and the telephone switching system 101, the media gateway 104, etc.
  • the media module is used for the access gateway 102 to interact with the telephone switching system. System 101, media gateway 104, etc. perform media interaction.
  • the media module is used for media interaction between access gateway 102 and telephone switching system 101, media gateway 104, etc. based on RTP.
  • the division of the functional modules of the media gateway 104, audio recognition device 107, access gateway 102, etc. in this application is only exemplary.
  • the media gateway 104, audio recognition device 107, and access gateway 102 may also include other functional modules. This application The embodiment does not limit this.
  • Figure 2 takes the media gateway integrating the functions of the signaling gateway and the media server as an example.
  • at least two of the signaling gateway, the media gateway, and the media server are deployed independently.
  • FIG. 3 shows a schematic diagram of yet another application scenario provided by the embodiment of the present application.
  • This application scenario takes the signaling gateway, media gateway, and media server as three independent devices as an example.
  • the multimedia conference system also includes a signaling gateway 108 and a media server 109.
  • the signaling gateway 108, the access gateway 102, and the conference management device 106 are respectively connection, the signaling gateway 108 is used for signaling interaction between the access gateway 102 and the conference management device 106, the media server 109 is connected to the media gateway 104 and the conference terminal 105 respectively, and the media server 109 is used to communicate with the conference between the media gateway 104 and the conference terminal 105. Media interaction is performed between terminals 105.
  • the media gateway 104 may include a media module and an audio processing module, but not a signaling module. Signaling-related processing functions may be implemented by the signaling gateway 108, which is not done in this embodiment of the application. limited.
  • the application scenarios shown in Figures 1 to 3 are only examples and are not intended to limit the technical solution of the present application.
  • the number of conference terminals and telephone terminals can be configured as needed, and the application scenario may also include other devices, or the application scenario may include fewer devices than those shown in Figures 1 to 3.
  • Embodiments of the present application There is no restriction on this.
  • FIG. 4 shows a flow chart of a multimedia conference control method provided by an embodiment of the present application.
  • the multimedia conference control method is applied to the target gateway.
  • the target gateway is the media gateway in Figure 2 or Figure 3.
  • the multimedia conference control method includes the following S401 to S402.
  • the target gateway receives the audio stream A sent to the first terminal through the telephone switching system.
  • the target gateway is communicatively connected to the first terminal, and the target gateway is communicatively connected to the second terminal of the telephone switching system.
  • the target gateway can receive audio stream A sent to the first terminal through the telephone switching system.
  • Audio stream A is an audio stream related to the second terminal. Audio stream A may carry the identifier of the second terminal.
  • the audio stream A carries the stream number of the second terminal.
  • the stream number of the second terminal is the stream number assigned to the second terminal, for example, the stream number assigned by the conference management device to the second terminal, or the stream number of the second terminal is the link between the target gateway and the second terminal, the telephone switching system, The stream number of the second terminal determined by the access gateway and others through signaling negotiation.
  • the target gateway is the media gateway 104
  • the first terminal is the conference terminal 105
  • the second terminal is the telephone terminal 103
  • the target gateway ie, the media gateway 104
  • the gateway 102 receives the audio stream A sent to the conference terminal 105.
  • the audio stream A is related to the telephone terminal 103.
  • the audio stream A reaches the target gateway (ie, the media gateway 104) through at least forwarding by the access gateway 102.
  • audio stream A may be an audio stream sent by the second terminal to the first terminal, or may be an audio stream sent by the telephone switching system to the first terminal. That is, the audio stream A may come from the second terminal or the telephone switching system. In other words, the audio stream A may be generated by the second terminal or the telephone switching system.
  • a call connection 1 is established between the second terminal and the first terminal.
  • the second terminal can collect the audio stream (for example, collect the second The voice of the user of the terminal speaking, collecting the sounds in the environment of the second terminal, etc.), and sending the audio stream to the first terminal through the call connection 1.
  • the audio stream A can be sent by the second terminal to the third terminal.
  • a call connection 1 is established between the second terminal and the first terminal.
  • the terminal for example, a third terminal
  • the second terminal is in a called state (for example, the second terminal is called by a third terminal).
  • the telephone switching system will hold the call connection 1 (that is, control the call connection 1).
  • the call connection 1 is in the call hold state (the call hold state can also be called the deactivated state), and the telephone switching system can send the called prompt tone of the second terminal to the first terminal (or the call hold prompt tone of the call connection 1). , that is, the telephone switching system can send the audio stream to the first terminal.
  • the audio stream A can be the audio stream sent by the telephone switching system to the first terminal.
  • audio stream A is the audio stream sent by the telephone switching system to the first terminal when the second terminal is in the called state and call connection 1 is in the call hold state, and the data of audio stream A received by the target gateway.
  • the packet may carry a first identifier, and the first identifier is used to indicate that the second terminal is in a called state.
  • the first identifier is used to indicate that the call connection 1 is in the call hold state, thereby indicating that the second terminal is in the called state.
  • the first identifier may be carried by the telephone switching system in the data packet of audio stream A, or may be added by the access gateway in the received data packet of audio stream A.
  • the called prompt tone of the second terminal may be "the user you dialed is currently on a call”
  • the first identifier may be "holdflag”
  • holdflag is To indicate that the call connection 1 is in the call hold state, thereby indicating that the second terminal is in the called state.
  • the target gateway performs denoising processing on the audio stream A.
  • the denoising processing is to remove the noise of the second terminal being called in the audio stream A.
  • the audio stream A includes the called prompt tone of the second terminal.
  • the call connection 1 between the second terminal and the first terminal is in the call hold state.
  • the second terminal does not conduct a multimedia conference in the multimedia conference system (that is, there is no media transmission between the second terminal and the first terminal in the multimedia conference system), but the first terminal may still conduct a multimedia conference in the multimedia conference system.
  • Audio stream A reaches the first terminal and is played by the first terminal.
  • the called prompt sound of the second terminal in audio stream A is likely to interfere with the first terminal. Therefore, the called prompt sound can be called noise.
  • audio stream A is the called prompt sound of the second terminal, and audio stream A may be called noise for the first terminal.
  • the target gateway can perform denoising processing on the audio stream A to remove the noise of the second terminal being called in the audio stream A (that is, the called prompt tone of the second terminal, or called call connection 1 call hold prompt tone), which can avoid the noise of the second terminal being called in audio stream A from causing interference to the first terminal.
  • the target gateway performs denoising on audio stream A, including the following three possible implementation methods.
  • the first implementation method the target gateway intercepts audio stream A.
  • the target gateway does not forward audio stream A to the first terminal. For example, the target gateway discards the packets of audio stream A.
  • the target gateway intercepts audio stream A to prevent audio stream A from reaching the first terminal, thereby preventing the first terminal from playing audio stream A, and thereby preventing the noise of the second terminal being called in audio stream A from interfering with the first terminal.
  • the second implementation method the target gateway replaces the data packet of audio stream A with a silence packet and sends the silence packet to the first terminal.
  • the target gateway is the media gateway 104
  • the first terminal is the conference terminal 105
  • the target gateway ie, the media gateway 104
  • sends a mute message to the first terminal ie, the conference terminal 105) through the media server 109. Bag.
  • the silent package meets any of the following conditions: does not include audio data, includes audio data, and the audio data cannot trigger physical sound perception.
  • a silence packet is a data packet encapsulated according to the audio protocol and format, and the payload of the silence packet is empty.
  • the payload of the silent package is empty. This means that the silent package does not include the payload, or the silent package includes the payload but the data in the payload is 0.
  • the silent package plays without any sound and cannot induce physical sound perception.
  • the target gateway Since the target gateway replaces the data packet of audio stream A with a silence packet and sends the silence packet to the first terminal, the target gateway does not send audio stream A to the first terminal, which can prevent audio stream A from reaching the first terminal, thereby avoiding the One terminal plays the audio stream A, thereby preventing the noise of the second terminal being called in the audio stream A from interfering with the first terminal.
  • the third implementation method the target gateway adds a second identifier to the data packet of audio stream A, and sends the data packet of audio stream A containing the second identifier to the first terminal.
  • the second identifier is used to instruct the first terminal not to play. Audio stream A.
  • the second identifier is used to instruct the first terminal not to play audio stream A. Therefore, the first terminal does not play audio stream A, and can avoid audio stream A being interrupted. The noise of the second terminal being called interferes with the first terminal.
  • the target gateway after the target gateway receives the audio stream sent to the first terminal through the telephone switching system, the target gateway performs denoising processing on the audio stream to remove the audio stream. Therefore, the noise of the second terminal being called can be prevented from interfering with the first terminal, thereby preventing the noise of the second terminal being called from affecting the development of the multimedia conference.
  • the target gateway may determine that the second terminal is in a called state.
  • the target gateway determines that the second terminal is in the called state, which may include the following two optional embodiments.
  • step S403a is also included.
  • the target gateway determines that the second terminal is in the called state based on the characteristic parameters of audio stream A.
  • the characteristic parameters of the audio stream A include at least one of the following: audio characteristics of the audio stream A, and data characteristics of the data packets of the audio stream A.
  • the characteristic parameter of audio stream A includes the data characteristic of the data packet of audio stream A
  • the target gateway determines that the second terminal is in the called state according to the data characteristic of the data packet of audio stream A.
  • the target gateway determines that the second terminal is in the called state based on the data packet of the audio stream A containing the first identifier, and the first identifier is used to indicate that the second terminal is in the called state.
  • the first identifier is used to indicate that the call connection 1 between the second terminal and the first terminal is in the call hold state, thereby indicating that the second terminal is in the called state.
  • the target gateway can determine whether the data packet of the audio stream A contains The first identifier.
  • the target gateway determines that the call connection 1 is in the call hold state, thereby determining that the second terminal is in the called state; if the data packet of audio stream A does not contain the first identifier, The target gateway determines that the call connection 1 is not in the call hold state, thereby determining that the second terminal is not in the called state.
  • the characteristic parameters of audio stream A include audio characteristics of audio stream A, which audio characteristics include voice segments included in audio stream A.
  • the target gateway determines that the second terminal is in a called state based on the voice segments included in audio stream A.
  • the target gateway compares the voice segment contained in the audio stream A with the specified voice segment, and determines that the second terminal is in the called state based on the comparison result.
  • the target gateway compares the voice segment contained in the audio stream A with the specified voice segment to determine whether the voice segment included in the audio stream A includes the specified voice segment. If the voice segment included in the audio stream A includes the specified voice segment, the target gateway determines whether the second voice segment included in the audio stream A includes the specified voice segment. The terminal is in the called state.
  • the target gateway determines that the second terminal is not in the called state. Alternatively, the target gateway compares the voice segment contained in audio stream A with the specified voice segment to determine whether the similarity between the voice segment contained in audio stream A and the specified voice segment is greater than the similarity threshold. If the voice segment contained in audio stream A is similar to the specified voice segment, If the similarity of the segment is greater than the similarity threshold, the target gateway determines that the second terminal is in the called state. If the similarity between the voice segment contained in the audio stream A and the specified voice segment is not greater than the similarity threshold, the target gateway determines that the second terminal is not in the called state. Call status.
  • the specified voice segment is used to describe that the second terminal is in the called state.
  • the specified voice segment is used to describe that the call connection 1 between the second terminal and the first terminal is in the call hold state, thereby describing the second terminal.
  • the specified voice segment is "The user you are calling is currently on the call.”
  • the target gateway before the target gateway determines that the second terminal is in the called state based on the audio characteristics of audio stream A, the target gateway obtains the audio characteristics of audio stream A. For example, the target gateway sends audio stream A to the audio recognition device, and receives the audio characteristics of audio stream A sent by the audio recognition device. The audio recognition device is used to perform audio recognition on audio stream A to obtain the audio characteristics of audio stream A.
  • the target gateway decodes audio stream A to obtain the audio bare code stream file of audio stream A, such as a PCM file, and then the target gateway sends the PCM file of audio stream A to the audio recognition device, and the audio recognition device Perform audio recognition on the PCM file to obtain the audio characteristics of audio stream A.
  • the audio recognition device performs audio recognition based on the PCM file of audio stream A through the audio recognition model. That is, the audio recognition device can input the PCM file of audio stream A into the audio recognition model, and the audio recognition model can perform audio recognition on the audio stream A. Calculate the PCM file of A to obtain the audio characteristics of audio stream A, and output the audio characteristics of audio stream A.
  • the target gateway receives the target signaling message.
  • the target signaling message is a signaling message related to the second terminal, and the target signaling message may carry the identity of the second terminal.
  • the target signaling message carries the phone number of the second terminal.
  • the target gateway, the access gateway, the telephone switching system, and the second terminal are connected in sequence, and the target gateway receives the target signaling message sent by the access gateway.
  • the target gateway is the media gateway 104
  • the second terminal is the phone terminal 103.
  • the media gateway 104 receives the target signaling message sent by the access gateway 102, and the target signaling message carries the phone number of the phone terminal 103. Number.
  • the target signaling message may be a SIP message or a signaling message based on a private protocol, which is not limited in this embodiment of the present application.
  • the target signaling message may be a negotiation message.
  • a call connection 1 is established between the second terminal and the first terminal in the multimedia conference system.
  • the second terminal is engaged in other activities, if the second terminal is engaged in other activities, The second terminal is in the called state (that is, the second terminal is in the state of being called by the other terminal), and the second terminal is in the state of being called by the other terminal. It is necessary to hold the call on the call connection 1, so the second terminal sends a first negotiation message to the telephone switching system to negotiate with the telephone switching system to put the call on the call connection 1 on hold.
  • the telephone switching system may send a second negotiation message to the access gateway according to the first negotiation message to negotiate with the access gateway to hold the call connection 1.
  • the access gateway may send a target signaling message to the target gateway according to the second negotiation message to negotiate with the access gateway to hold the call connection 1.
  • the target gateway can receive the target signaling message sent by the access gateway.
  • the first negotiation message, the second negotiation message and the target signaling message all carry designated signaling information and the identifier of the second terminal to indicate that the second terminal is in a called state.
  • the calling party still has media to send, for example, indicating that when the call connection between the calling party and the called party is on call hold, the calling party is prompted by voice that the called party is in the called state (or in other words, the calling party and the called party
  • the call connection between the second terminal and the first terminal is in the call hold state).
  • the second terminal is in the called state.
  • the first negotiation message, the second negotiation message, and the target signaling message may be the same signaling message, or they may be three different signaling messages. It can be understood that if the first negotiation message, the second negotiation message, and the target signaling message are the same signaling message, and the signaling message comes from the second terminal, the telephone switching system and the access gateway will receive the signaling message. , relevant processing can be performed according to the signaling message, and the signaling message can be forwarded.
  • the target gateway determines that the second terminal is in the called state according to the target signaling message containing designated signaling information, and the designated signaling information is used to indicate that the second terminal is in the called state.
  • the target gateway determines whether the target signaling message contains the specified signaling information; if the target signaling message contains the specified signaling information, the target gateway determines that the second terminal is in the called state; if the called state does not contain the specified signaling information , the target gateway determines that the second terminal is not in the called state.
  • the designated signaling information is used to indicate that the second terminal is in a called state.
  • the embodiment shown in Figure 5 takes the access gateway notifying the target gateway that the second terminal is in the called state through the target signaling message as an example.
  • the access gateway may also notify the target gateway that the second terminal is in the called state through other methods.
  • the access gateway uses an interface callback or a publish-subscribe method to notify the target gateway that the second terminal is in the called state. That is, the access gateway can call the interface that communicates with the target gateway to notify the target gateway that the second terminal is in the called state, or, in the case where the target gateway subscribes to relevant notifications from the access gateway, the access gateway notifies the target gateway
  • the second terminal is in a called state, which is not limited in the embodiment of the present application.
  • the second terminal when the call connection 1 is established between the second terminal and the first terminal in the multimedia conference system, and the call connection 1 is in the active state, if the second terminal communicates with the first terminal based on various possible reasons, If an external terminal (such as a third terminal) establishes call connection 2, then the second terminal is in the called state (for example, the second terminal is in the state of being called by the third terminal), and the call connection between the second terminal and the first terminal 1 is called on hold (or deactivated). When the second terminal and the third terminal disconnect the call connection 2 or the call connection 2 is held by a call, the second terminal can cancel the called state (for example, the second terminal cancels the state of being called by the third terminal), and can be reactivated at this time.
  • the call connection 1 enables the second terminal and the first terminal to perform media transmission through the call connection 1.
  • the second terminal when the second terminal cancels the called state, the second terminal sends a third negotiation message to the telephone switching system to negotiate with the telephone switching system to cancel the call connection 1 between the second terminal and the first terminal.
  • the call is on hold; after the telephone switching system receives the third negotiation message, the telephone switching system sends a fourth negotiation message to the access gateway according to the third negotiation message to negotiate with the access gateway to cancel the negotiation between the second terminal and the first terminal.
  • the call connection 1 is in the call hold state; after the access gateway receives the fourth negotiation message, the access gateway sends a fifth negotiation message to the target gateway according to the fourth negotiation message to negotiate with the target gateway to cancel the connection between the second terminal and the first terminal.
  • the target gateway determines according to the fifth negotiation message that the second terminal cancels the call hold state of the call connection 1 between the second terminal and the first terminal, thereby determining that the second terminal cancels the called state.
  • the third negotiation message, the fourth negotiation message and the fifth negotiation message all carry the identifier of the second terminal and do not carry designated signaling information to instruct the second terminal to cancel the call connection between the second terminal and the first terminal. 1's call hold status, thereby instructing the second terminal to cancel the called status.
  • the second terminal can collect the audio stream (for example, called audio stream B) and send it to the first through call connection 1.
  • the terminal sends audio stream B.
  • the target gateway can determine that the second terminal cancels the called state (or determines that the second terminal is not in the called state) based on the characteristic parameters of audio stream B.
  • the target gateway can Forward audio stream B to the first terminal.
  • the characteristic parameters of audio stream B include at least one of the following: audio characteristics of audio stream B and data characteristics of the data packets of audio stream B.
  • the target gateway may determine the first identifier based on the fact that the data packets of audio stream B do not contain the first identifier.
  • the second terminal cancels the called state; alternatively, the target gateway can determine that the second terminal cancels the called state based on the fact that the voice fragments contained in audio stream B do not include the specified voice fragment; or, the target gateway can determine that the second terminal cancels the called state based on the fact that the voice fragments contained in audio stream B do not include the specified voice fragment. If the similarity of the voice fragment is not greater than the similarity threshold, it is determined that the second terminal cancels the called state.
  • the multimedia conference system also includes a conference management device.
  • the conference management device is communicatively connected with the target gateway (such as a media gateway).
  • the conference management device can send detection indication information to the target gateway. , to instruct the target gateway to detect the audio stream sent to the first terminal and related to the second terminal.
  • the target gateway can determine whether the second terminal is in a called state based on the characteristic parameters of the audio stream related to the second terminal.
  • the detection indication information includes the identification of the second terminal and the detection identification to instruct the target gateway to detect the audio stream related to the second terminal.
  • the detection indication information can also instruct the target gateway to detect the audio stream related to the second terminal in other ways. The audio stream is detected, and there is no limit here.
  • the target gateway as a media gateway as an example, and describes the technical solution of this application in conjunction with the interaction between different devices in Figure 2.
  • a call connection 1 is established between the second terminal and the first terminal (such as the conference terminal 105) in the multimedia conference system.
  • connection 1 When connection 1 is active, the second terminal and the first terminal perform media transmission through call connection 1.
  • the call connection 1 When the call connection 1 is in the active state, if the second terminal carries out other telephone services (such as answering a phone call from a third terminal), the second terminal is in the called state (that is, in the state of being called by the third terminal), and the second terminal is called.
  • the second terminal establishes a call connection 2 with the third terminal, and the telephone switching system puts the call connection 1 on hold so that the call connection 1 is in a call hold state.
  • the target gateway ie, the media gateway 104
  • the target gateway can perform denoising processing on the media stream sent to the first terminal to remove the noise in the media stream that the second terminal is called.
  • the call connection 2 between the second terminal and the third terminal is disconnected or the call is held.
  • the second terminal cancels the called state and can reactivate the call between the second terminal and the first terminal.
  • Connection 1 enables the second terminal and the first terminal to conduct media transmission through call connection 1. Therefore, the technical solution of the embodiment of the present application involves the stage when the second terminal accesses the multimedia conference system and the stage when the second terminal is in the called state (or in other words, the call connection 1 between the second terminal and the first terminal is in the call hold state. stage) and the stage when the second terminal cancels the called state (or the stage when the call connection 1 between the second terminal and the first terminal is in the active state).
  • FIG. 6 shows a flow chart of a second terminal accessing a multimedia conference system according to an embodiment of the present application.
  • the process of the second terminal accessing the multimedia conference system includes the following steps S601 to S614.
  • the conference management device instructs the target gateway to connect the second terminal to the multimedia conference system.
  • the conference management device sends access instruction information to the target gateway to instruct the target gateway to access the second terminal to the multimedia conference system.
  • the access indication information may include an identity of the second terminal.
  • the access indication information may include a phone number of the second terminal or other identification information used to indicate the second terminal.
  • the conference management device sends access indication information to the target gateway through SIP signaling or private protocol signaling.
  • the target gateway sends call request 1 to the access gateway according to the instruction of the conference management device.
  • the target gateway determines to access the second terminal to the multimedia conference system according to the instruction of the conference management device. Therefore, the target gateway sends call request 1 to the access gateway to request the access gateway to call the second terminal.
  • the call request 1 includes the identity of the second terminal, and the call request 1 may be SIP signaling or private protocol signaling.
  • the access gateway sends call ring response 1 corresponding to call request 1 to the target gateway.
  • the access gateway After the access gateway receives the call request 1, the access gateway determines to call the second terminal according to the call request 1, and the access gateway sends the call ring response 1 corresponding to the call request 1 to the target gateway to inform the target gateway of the access.
  • the gateway is about to call the second terminal, please wait for the subsequent response from the target gateway.
  • the call ring response 1 may include the identification of the second terminal, for example, include the phone number of the second terminal.
  • the call ring response 1 may also include the signaling content of the call ring. For example, the signaling content of the call ring is "180".
  • Call Ring Response 1 can be SIP signaling or proprietary protocol signaling.
  • the access gateway sends call request 2 to the telephone switching system according to call request 1.
  • the access gateway After the access gateway determines to call the second terminal, the access gateway sends a call request 2 to the telephone switching system according to the call request 1 to request the telephone switching system to call the second terminal.
  • the call request 2 includes the identification of the second terminal, for example, the phone number of the second terminal.
  • Call request 2 may be SIP signaling or signaling of a private protocol.
  • the telephone switching system sends the call ring response 2 corresponding to the call request 2 to the access gateway.
  • the telephone switching system After the telephone switching system receives the call request 2, the telephone switching system determines to call the second terminal according to the call request 2, and the telephone switching system sends the call ring response 2 corresponding to the call request 2 to the access gateway to inform the access gateway of the call The switching system is about to call the second terminal. Please access the gateway and wait for the subsequent response.
  • the call ring response 2 may include the identification of the second terminal.
  • the call ring response 2 may also include the signaling content of the call ring. For example, the signaling content of the call ring is "180".
  • Call Ring Response 2 may be SIP signaling or proprietary protocol signaling.
  • the telephone switching system sends call request 3 to the telephone switching system according to call request 2.
  • the telephone switching system After the telephone switching system determines to call the second terminal, the telephone switching system sends a call request 3 to the second terminal according to the call request 2 to call the second terminal.
  • the call request 3 includes the identification of the second terminal, for example, the phone number of the second terminal.
  • Call request 3 may be SIP signaling or proprietary protocol signaling.
  • the second terminal sends the call ring response 3 corresponding to the call request 3 to the telephone switching system.
  • the second terminal After the second terminal receives the call request 3, the second terminal sends the call ring response 3 corresponding to the call request 3 to the telephone switching system to inform the telephone switching system to wait for a subsequent response.
  • the second terminal may also ring according to the call request 3 to prompt the user of the second terminal to answer the phone call.
  • the call ring response 3 includes the identification of the second terminal, for example, the phone number of the second terminal.
  • the call ringing response 3 may also include the signaling content of the call ringing, for example, the signaling content of the call ringing is "180".
  • the call ring response 3 may be SIP signaling or proprietary protocol signaling.
  • the second terminal sends the call connection response 3 corresponding to the call request 3 to the telephone switching system.
  • the second terminal may send a call connection response 3 corresponding to the call request 3 to the telephone switching system to inform the telephone switching system that the second terminal has connected the telephone call to the telephone switching system.
  • the call connection response 3 may include the identification of the second terminal, and may also include the signaling content of the call connection. For example, the signaling content of the call connection is "100".
  • the call connection response 3 may be SIP signaling or signaling of a proprietary protocol.
  • the telephone switching system sends the connection confirmation response 3 corresponding to the call connection response 3 to the second terminal.
  • connection confirmation response 3 After the telephone switching system receives the call connection response 3, the telephone switching system sends the connection confirmation response 3 corresponding to the call connection response 3 to the second terminal to inform the second terminal that the telephone switching system has received the call connection response 3. .
  • the connection confirmation response 3 may be SIP signaling or signaling of a private protocol.
  • the telephone switching system sends the call connection response 2 corresponding to the call request 2 to the access gateway.
  • the telephone switching system After the telephone switching system receives the call connection response 3, the telephone switching system sends the call connection response 2 corresponding to the call request 2 to the access gateway according to the call connection response 3 to inform the access gateway that the telephone switching system is connected. Telephone calls to the gateway.
  • the call connection response 2 may be SIP signaling or signaling of a proprietary protocol.
  • the access gateway sends the connection confirmation response 2 corresponding to the call connection response 2 to the telephone switching system.
  • connection confirmation response 2 corresponding to the call connection response 2 to the telephone switching system to inform the telephone switching system that the access gateway has received the call connection response 2.
  • the connection confirmation response 2 may be SIP signaling or signaling of a private protocol.
  • the access gateway sends the call connection response 1 corresponding to the call request 1 to the target gateway.
  • the telephone switching system After the access gateway receives the call connection response 2, the telephone switching system sends the call connection response 1 corresponding to the call request 1 to the target gateway according to the call connection response 2, so as to inform the target gateway that the telephone access gateway has connected the target. Gateway phone call.
  • the call connection response 1 may be SIP signaling or signaling of a proprietary protocol.
  • the target gateway sends the connection confirmation response 1 corresponding to the call connection response 1 to the access gateway.
  • the target gateway After the target gateway receives the call connection response 1, the target gateway sends the connection confirmation response 1 corresponding to the call connection response 1 to the access gateway to inform the access gateway that the target gateway has received the call connection response 1.
  • the connection confirmation response 1 may be SIP signaling or signaling of a private protocol.
  • the call connection 1 is successfully established between the second terminal and the first terminal, and the second terminal successfully accesses the multimedia conference system.
  • the call connection 1 includes a call connection 11 between the telephone switching system and the second terminal, a call connection 12 between the access gateway and the telephone switching system, a call connection 13 between the target gateway and the access gateway, and, A call connection 10 between a terminal and the target gateway.
  • the call connection 10 between the first terminal and the target gateway is established between the first terminal and the target gateway when the first terminal accesses the multimedia conference system.
  • the target gateway notifies the conference management device that the second terminal has successfully accessed the multimedia conference system.
  • the target gateway sends the access result of the second terminal to the conference management device, so as to notify the conference management device that the second terminal has successfully accessed the multimedia conference system.
  • the access result of the second terminal may be "access successful”.
  • the second terminal and the first terminal in the multimedia conference system can transmit the audio stream through the call connection 1. That is, the second terminal can transmit the audio stream to the first terminal through the call connection 1, and the first terminal can also transmit the audio stream to the second terminal through the call connection 1.
  • FIG. 7 shows a flow chart in which the second terminal is in a called state according to an embodiment of the present application.
  • Figure 7 mainly introduces the process of the second terminal entering the called state and the processing process of the audio stream related to the second terminal after the second terminal enters the called state.
  • the target gateway determines that the second terminal is called based on the signaling message. The status is explained as an example.
  • the process in which the second terminal is in the called state includes the following steps S701 to S715.
  • the second terminal sends a renegotiation request 1 to the telephone switching system.
  • a call connection 1 is established between the second terminal and the first terminal in the multimedia conference system.
  • the call connection 1 is activated, if the second terminal communicates with the first terminal due to other telephone services, Other terminals establish call connection 2.
  • the second terminal answers a phone call from a third terminal.
  • the second terminal is in the called state, and the second terminal can put call connection 1 on hold.
  • the second terminal sends a renegotiation request 1 to the telephone switching system to negotiate with the telephone switching system to hold the call connection 1 on hold.
  • the renegotiation request 1 may be SIP signaling or private protocol signaling.
  • the renegotiation request 1 may include the identity of the second terminal and may also include designated signaling information.
  • the telephone switching system sends a renegotiation request 2 to the access gateway based on the renegotiation request 1.
  • the telephone switching system After the telephone switching system receives the renegotiation request 1, the telephone switching system determines to hold the call connection 1 between the second terminal and the first terminal according to the renegotiation request 1, and the telephone switching system sends a request to the access gateway according to the renegotiation request 1.
  • Send renegotiation request 2 to negotiate with the access gateway to put call connection 1 on hold.
  • renegotiation request 2 is SIP signaling or private protocol signaling, and renegotiation request 2 and renegotiation request 1 are the same signaling.
  • the access gateway sends a renegotiation request 3 to the target gateway based on the renegotiation request 2.
  • the access gateway After the access gateway receives the renegotiation request 2, the access gateway determines to hold the call connection 1 between the second terminal and the first terminal based on the renegotiation request 2, and the access gateway sends a message to the target gateway based on the renegotiation request 2. Renegotiation request 3 to negotiate with the target gateway to put call connection 1 on hold.
  • renegotiation request 3 is SIP signaling or private protocol signaling, and renegotiation request 3 and renegotiation request 2 are the same signaling.
  • the target gateway sends the renegotiation response 3 corresponding to the renegotiation request 3 to the access gateway.
  • the target gateway After the target gateway receives the renegotiation request 3, the target gateway determines to hold the call connection 1 between the second terminal and the first terminal according to the renegotiation request 3, and the target gateway sends a message corresponding to the renegotiation request 3 to the access gateway. Renegotiation response 3.
  • the first terminal When the call connection 1 with the first terminal is in the call hold state (when the call connection 1 between the second terminal and the first terminal is in the call hold state, and the second terminal is in the called state), the first terminal only receives Media streams without sending media streams.
  • the renegotiation response 3 may be SIP signaling or private protocol signaling.
  • the access gateway sends the renegotiation response 2 corresponding to the renegotiation request 2 to the telephone switching system.
  • the access gateway After the access gateway receives the renegotiation response 3, the access gateway sends the renegotiation response 2 corresponding to the renegotiation request 2 to the telephone switching system according to the renegotiation response 3.
  • renegotiation response 2 is SIP signaling or private protocol signaling, and renegotiation response 2 and renegotiation response 3 may be the same signaling.
  • the telephone switching system sends the renegotiation response 1 corresponding to the renegotiation request 1 to the second terminal.
  • the telephone switching system After the telephone switching system receives the renegotiation response 2, the telephone switching system sends the renegotiation response 1 corresponding to the renegotiation request 1 to the second terminal according to the renegotiation response 2.
  • renegotiation response 1 is SIP signaling or private protocol signaling, and renegotiation response 1 and renegotiation response 2 may be the same response.
  • the second terminal sends the renegotiation confirmation 1 corresponding to the renegotiation response 1 to the telephone switching system.
  • the second terminal may send the renegotiation confirmation 1 corresponding to the renegotiation response 1 to the telephone switching system to inform the telephone switching system that the second terminal has received the renegotiation response 1.
  • the two-way call connection 11 between the second terminal and the telephone switching system is adjusted to a one-way call connection 11 from the second terminal to the telephone switching system.
  • the audio stream can be sent to the telephone switching system through the one-way call connection 11 , but the telephone switching system does not send the audio stream to the second terminal through the one-way call connection 11 .
  • the renegotiation confirmation 1 may be SIP signaling or private protocol signaling.
  • the telephone switching system sends the renegotiation confirmation 2 corresponding to the renegotiation response 2 to the access gateway.
  • the telephone switching system may send the renegotiation confirmation 2 corresponding to the renegotiation response 2 to the access gateway according to the renegotiation confirmation 1 to inform the access gateway that the telephone switching system has received the renegotiation. Response 2.
  • the telephone switching system sends the renegotiation confirmation 2 to the access gateway, the two-way call connection 12 between the telephone switching system and the access gateway is adjusted to a one-way call connection 12 from the telephone switching system to the access gateway.
  • the telephone switching system The audio stream can be sent to the access gateway through the one-way call connection 12, but the access gateway does not send the audio stream to the telephone switching system through the one-way call connection 12.
  • renegotiation confirmation 2 is SIP signaling or private protocol signaling
  • renegotiation confirmation 2 and renegotiation confirmation 1 may be the same signaling.
  • the access gateway sends the renegotiation confirmation 3 corresponding to the renegotiation response 3 to the target gateway.
  • the access gateway may send the renegotiation confirmation 3 corresponding to the renegotiation response 3 to the target gateway according to the renegotiation confirmation 2 to inform the target gateway that the access gateway has received the renegotiation response 3. .
  • the two-way call connection 13 between the access gateway and the target gateway is adjusted to a one-way call connection 13 from the access gateway to the target gateway.
  • the access gateway can use this
  • the one-way call connection 13 sends the audio stream to the target gateway, but the target gateway does not send the audio stream to the access gateway through the one-way call connection 13 .
  • renegotiation confirmation 3 is SIP signaling or private protocol signaling, and renegotiation confirmation 3 and renegotiation confirmation 2 may be the same signaling.
  • the target gateway determines that the second terminal is in the called state according to the renegotiation request 3.
  • the target gateway may determine that the second terminal is in the called state according to the specified signaling information carried in the renegotiation request 3.
  • the designated signaling information indicates that the first terminal is prompted by voice when the call connection between the second terminal and the first terminal is in the call hold state.
  • the designated signaling information is used to indicate that the call connection between the second terminal and the first terminal is in the call hold state, thereby indicating that the second terminal is in the called state.
  • the target gateway determines that the second terminal is in the called state based on the designated signaling information. state.
  • the target gateway sends status information 1 to the conference management device.
  • the status information 1 indicates that the second terminal is in the called state.
  • the target gateway can send status information 1 to the conference management device through SIP signaling or private protocol signaling.
  • the status information 1 may include the identity of the second terminal and the called identity to indicate that the second terminal is in the called identity.
  • the conference management device controls the first terminal to display that the second terminal is in a called state.
  • the conference management device may determine that the second terminal is in the called state according to the status information 1, and then the conference management device may send control indication information to the first terminal to indicate that the second terminal is in the called state.
  • the first terminal may display information or an identification indicating that the second terminal is in a called state on the conference interface of the first terminal according to the control instruction information.
  • the telephone switching system sends audio stream 1 to the access gateway.
  • the telephone switching system When the second terminal is in the called state, the telephone switching system generates the called prompt tone and sends the audio stream 1 to the access gateway based on the called prompt tone.
  • the called prompt sound is used to remind the second terminal that it is in a called state.
  • the access gateway forwards audio stream 1 to the target gateway.
  • the target gateway performs denoising processing on the audio stream 1 to remove the noise of the second terminal being called in the audio stream 1.
  • the target gateway intercepts audio stream 1, or the target gateway replaces the data packet of audio stream 1 with a silence packet and sends the silence packet to the first terminal, or the target gateway adds a second packet to the data packet of audio stream 1.
  • the data packet of audio stream 1 is sent to the first terminal, where the second identification is used to instruct the first terminal not to play audio stream 1.
  • the target gateway can prevent audio stream 1 from reaching the first terminal, or even if audio stream 1 reaches the first terminal, it can prevent the first terminal from playing audio stream 1, so it can avoid audio stream 1 from interfering with the first terminal.
  • S701 to S709 describe the process of the second terminal entering the called state
  • S713 to S715 describe the processing process of the audio stream related to the second terminal after the second terminal enters the call hold state.
  • the renegotiation signaling used for the call hold negotiation of the telephone terminal (for example, the second terminal) is terminated at the access gateway. That is, the access gateway receives the renegotiation signaling used for the call hold negotiation of the telephone terminal. After the command, the access gateway does not send renegotiation signaling to the media gateway (such as the target gateway). Therefore, the called status of the phone terminal will not be passed to the media gateway, causing the media gateway to be unable to sense the called status of the phone terminal. When the telephone terminal is in the called state, the media gateway still forwards the called prompt tone of the telephone terminal, causing interference to other terminals in the multimedia conference system.
  • the access gateway after the access gateway receives the renegotiation signaling used for the telephone terminal call hold negotiation, the access gateway sends the renegotiation signaling to the media gateway to negotiate with the media gateway, so that the media gateway can sense the telephone terminal.
  • the called state so that when the phone terminal is in the called state, the media gateway performs denoising processing on the audio streams related to the phone terminal sent to other terminals to remove the called prompt tone of the phone terminal in the audio stream. , to prevent the called prompt tone of the telephone terminal from interfering with other terminals in the multimedia conference system, and to achieve the effect of accurately suppressing unnecessary interfering audio.
  • FIG. 8 shows another flow chart in which the second terminal is in a called state according to an embodiment of the present application.
  • Figure 8 mainly introduces the process of the second terminal entering the called state and the processing process of the audio stream related to the second terminal after the second terminal enters the called state.
  • the target gateway determines that the second terminal is in a state based on the characteristic parameters of the audio stream. Take the called status as an example.
  • the process in which the second terminal is in the called state includes the following steps S801 to S817.
  • the second terminal sends a renegotiation request 1 to the telephone switching system.
  • the telephone switching system sends a renegotiation request 2 to the access gateway based on the renegotiation request 1.
  • the implementation process of S801 to S802 can refer to the implementation process of S701 to 702, which will not be described again here.
  • the access gateway sends the renegotiation response 2 corresponding to the renegotiation request 2 to the telephone switching system.
  • the telephone switching system sends the renegotiation response 1 corresponding to the renegotiation request 1 to the second terminal.
  • the second terminal sends the renegotiation confirmation 1 corresponding to the renegotiation response 1 to the telephone switching system.
  • the two-way call connection 11 between the second terminal and the telephone switching system is adjusted to a one-way call connection 11 from the second terminal to the telephone switching system.
  • the telephone switching system sends the renegotiation confirmation 2 corresponding to the renegotiation response 2 to the access gateway.
  • the two-way call connection 12 between the telephone switching system and the access gateway is adjusted to a one-way call connection 12 from the telephone switching system to the access gateway.
  • the implementation process of S803 to S806 can refer to the implementation process of S705 to 708, and will not be described again here.
  • the conference management device sends detection indication information to the target gateway.
  • the detection indication information is used to instruct detection of the audio stream related to the second terminal.
  • the detection indication information includes the identification of the second terminal and the detection identification to instruct the target gateway to detect the audio stream related to the second terminal.
  • the identifier of the second terminal may be a phone number of the second terminal or other identification information used to indicate the second terminal, which is not limited in this embodiment of the present application.
  • the conference management device sends detection indication information to the target gateway through SIP signaling or private protocol signaling.
  • the telephone switching system sends audio stream 1 to the access gateway.
  • the telephone switching system When the second terminal is in the called state, the telephone switching system generates the called prompt tone and sends the audio stream 1 to the access gateway based on the called prompt tone.
  • the called prompt sound is used to remind the second terminal that it is in a called state.
  • the access gateway forwards audio stream 1 to the target gateway.
  • the target gateway decodes audio stream 1 into a PCM file.
  • the target gateway After the target gateway receives audio stream 1, the target gateway determines that audio stream 1 is related to the second terminal. Since in S807, the conference management device instructs the target gateway to detect the audio stream related to the second terminal, the target gateway determines that it is necessary to detect the audio stream 1 related to the second terminal, and the target gateway decodes the audio stream 1 into a PCM file. .
  • the target gateway sends the PCM file of audio stream 1 to the audio recognition device.
  • the audio recognition device performs audio recognition based on the PCM file of audio stream 1, and obtains the audio characteristics of audio stream 1.
  • the audio recognition device sends the audio characteristics of audio stream 1 to the target gateway.
  • the target gateway determines that the second terminal is in the called state based on the audio characteristics of audio stream 1.
  • the target gateway compares the voice segment contained in the audio stream 1 with the specified voice segment, and determines that the second terminal is in the called state based on the comparison result. For example, the target gateway compares the voice segment included in audio stream 1 with the specified voice segment to determine whether the voice segment included in audio stream 1 includes the specified voice segment. If the voice segment included in audio stream 1 includes the specified voice segment, the target gateway determines whether the second voice segment included in audio stream 1 includes the specified voice segment. The terminal is in the called state. If the voice segment contained in the audio stream 1 does not include the specified voice segment, the target gateway determines that the second terminal is not in the called state.
  • the target gateway compares the voice segment contained in audio stream 1 with the specified voice segment to determine whether the similarity between the voice segment contained in audio stream 1 and the specified voice segment is greater than the similarity threshold. If the voice segment contained in audio stream 1 is similar to the specified voice segment, If the similarity of the segment is greater than the similarity threshold, the target gateway determines that the second terminal is in the called state. If the similarity between the voice segment contained in audio stream 1 and the specified voice segment is not greater than the similarity threshold, the target gateway determines that the second terminal is not in the called state. Call status.
  • the target gateway sends status information 1 to the conference management device.
  • the status information 1 indicates that the second terminal is in the call hold state.
  • the conference management device controls the first terminal to display that the second terminal is in the call hold state.
  • the implementation process of S815 to S816 can refer to the implementation process of S711 to S712, which will not be described again here.
  • the target gateway performs denoising processing on the audio stream 1 to remove the noise of the second terminal being called in the audio stream 1.
  • the implementation process of S817 can refer to the implementation process of S715, and will not be repeated here.
  • S801 to S806 describe the process of the second terminal entering the called state
  • S808 to S817 describe the processing process of the audio stream related to the second terminal after the second terminal enters the called state.
  • the media gateway (such as the target gateway) cannot sense the called status of the phone terminal. Therefore, when the phone terminal is in the called state, the media gateway still forwards the called prompt tone of the phone terminal, resulting in the multimedia conference Other terminals in the system create interference.
  • the media gateway can detect the audio stream related to the telephone terminal to determine that the telephone terminal is in the called state. When the telephone terminal is in the called state, the media gateway detects the audio stream related to the telephone terminal and is sent to other terminals.
  • the audio stream related to the terminal is denoised to remove the called prompt tone of the telephone terminal in the audio stream to prevent the called prompt tone of the telephone terminal from interfering with other terminals in the multimedia conference system to achieve precise suppression of unnecessary The effect of interfering with the audio.
  • FIG. 9 shows a flow chart for a second terminal to cancel the called state provided by an embodiment of the present application.
  • Figure 9 mainly introduces the process of the second terminal canceling the called state and the processing process of the audio stream related to the second terminal after the second terminal cancels the called state.
  • the process for the second terminal to cancel the called state includes the following steps S901 to S917.
  • the second terminal sends a renegotiation request 4 to the telephone switching system.
  • a call connection 1 is established between the second terminal and the first terminal in the multimedia conference system.
  • the call connection 1 is activated, if the second terminal communicates with the first terminal due to other telephone services, When another terminal establishes call connection 2, the second terminal is in the called state, and the call connection 1 between the second terminal and the first terminal is held by the call. If the second terminal ends other telephone services, the call connection 2 between the second terminal and the third terminal is disconnected or the call is on hold, the second terminal can cancel the called state, and at this time, the call between the second terminal and the first terminal can be reactivated. call connection 1 between the second terminal and the first terminal (or cancel the call hold state of call connection 1 between the second terminal and the first terminal).
  • the second terminal when the second terminal cancels the called state, the second terminal sends a renegotiation request 4 to the telephone switching system to negotiate with the telephone switching system to cancel the call hold state of the call connection 1 between the second terminal and the first terminal.
  • the renegotiation request 4 may include the identity of the second terminal, and the renegotiation request 4 does not include designated signaling information.
  • the telephone switching system sends a renegotiation request 5 to the access gateway based on the renegotiation request 4.
  • the telephone switching system After the telephone switching system receives the renegotiation request 4, the telephone switching system determines to cancel the call hold state of the call connection 1 between the second terminal and the first terminal according to the renegotiation request 4, and the telephone switching system accesses the call according to the renegotiation request 4.
  • the gateway sends a renegotiation request 5 to negotiate with the access gateway to cancel the call hold state of the call connection 1 between the second terminal and the first terminal.
  • renegotiation request 5 is SIP signaling or private protocol signaling, and renegotiation request 5 and renegotiation request 4 may be the same signaling.
  • the access gateway sends a renegotiation request 6 to the target gateway based on the renegotiation request 5.
  • the access gateway After the access gateway receives the renegotiation request 5, the access gateway determines to cancel the call hold state of the call connection 1 between the second terminal and the first terminal according to the renegotiation request 5, and the access gateway sends a request to the target gateway according to the renegotiation request 5.
  • a renegotiation request 6 is sent to negotiate with the target gateway to cancel the call hold state of the call connection 1 between the second terminal and the first terminal.
  • renegotiation request 6 is SIP signaling or private protocol signaling, and renegotiation request 6 and renegotiation request 5 may be the same signaling.
  • the target gateway sends the renegotiation response 6 corresponding to the renegotiation request 6 to the access gateway.
  • the target gateway After the target gateway receives the renegotiation request 6, the target gateway determines to cancel the call hold state of the call connection 1 between the second terminal and the first terminal according to the renegotiation request 6, thereby determining that the second terminal cancels the called state, and the target gateway
  • the access gateway sends a renegotiation response 6 corresponding to the renegotiation request 6.
  • the access gateway sends the renegotiation response 5 corresponding to the renegotiation request 5 to the telephone switching system.
  • the access gateway After the access gateway receives the renegotiation response 6, the access gateway sends the renegotiation response 5 corresponding to the renegotiation request 5 to the telephone switching system according to the renegotiation response 6.
  • renegotiation response 5 is SIP signaling or private protocol signaling, and renegotiation response 6 and renegotiation response 5 may be the same signaling.
  • the telephone switching system sends the renegotiation response 4 corresponding to the renegotiation request 4 to the second terminal.
  • the telephone switching system After the telephone switching system receives the renegotiation response 5, the telephone switching system sends the renegotiation response 4 corresponding to the renegotiation request 4 to the second terminal according to the renegotiation response 5.
  • renegotiation response 4 is SIP signaling or private protocol signaling, and renegotiation response 5 and renegotiation response 4 may be the same response.
  • the second terminal sends the renegotiation confirmation 4 corresponding to the renegotiation response 4 to the telephone switching system.
  • the second terminal may send the renegotiation confirmation 4 corresponding to the renegotiation response 4 to the telephone switching system to inform the telephone switching system that the second terminal has received the renegotiation response 4.
  • the renegotiation confirmation 4 may be SIP signaling or signaling of a private protocol.
  • the telephone switching system sends the renegotiation confirmation 5 corresponding to the renegotiation response 5 to the access gateway.
  • the telephone switching system may send the renegotiation confirmation 5 corresponding to the renegotiation response 5 to the access gateway according to the renegotiation confirmation 4 to inform the access gateway that the telephone switching system has received the renegotiation. Response 5.
  • the telephone switching system sends the renegotiation confirmation 5 to the access gateway, the one-way call connection 12 between the telephone switching system and the access gateway is adjusted to a two-way call connection 12 .
  • renegotiation confirmation 5 is SIP signaling or private protocol signaling, and renegotiation confirmation 5 and renegotiation confirmation 4 may be the same signaling.
  • the access gateway sends the renegotiation confirmation 6 corresponding to the renegotiation response 6 to the target gateway.
  • the access gateway can send the renegotiation confirmation 6 corresponding to the renegotiation response 6 to the target gateway according to the renegotiation confirmation 5 to inform the target gateway that the access gateway has received the renegotiation response 6. .
  • the one-way call connection 13 between the access gateway and the target gateway is adjusted to a two-way call connection 13 .
  • the renegotiation confirmation 6 is SIP signaling or private protocol signaling, and the renegotiation confirmation 6 and the renegotiation confirmation 6 may be the same signaling.
  • the target gateway determines that the second terminal cancels the called state according to the renegotiation request 6.
  • the target gateway sends status information 2 to the conference management device.
  • the status information 2 instructs the second terminal to cancel the called state.
  • the target gateway can send status information 2 to the conference management device through SIP signaling or private protocol signaling.
  • the state information 2 includes the identifier of the second terminal and does not include the called identifier, so as to instruct the second terminal to cancel the called state.
  • the conference management device controls the first terminal to display that the second terminal is not in a called state.
  • the conference management device may determine that the second terminal cancels the called state according to status information 2, and then the conference management device sends control indication information to the first terminal to indicate that the second terminal is not in the called state.
  • the first terminal may display an identification or information indicating that the second terminal is not in a called state in the conference interface of the first terminal according to the control instruction information.
  • the second terminal sends audio stream 2 to the telephone switching system.
  • the second terminal can collect the audio stream (for example, called audio stream 2) and send the audio stream 2 to the telephone switching system.
  • the audio stream for example, called audio stream 2
  • the telephone switching system sends audio stream 2 to the access gateway.
  • the access gateway forwards audio stream 2 to the target gateway.
  • the target gateway forwards audio stream 2 to the first terminal.
  • the audio stream 2 does not include the noise that the second terminal is in the called state.
  • the audio stream 2 will not interfere with the first terminal, so the target gateway forwards the audio stream 2 to the first terminal.
  • the first terminal plays audio stream 2.
  • audio stream 1 and the aforementioned audio stream A may be the same audio stream
  • audio stream 2 and the aforementioned audio stream B may be the same audio stream.
  • audio stream 1 and audio stream A may not be the same audio stream.
  • the audio stream, audio stream 2 and the aforementioned audio stream B may not be the same audio stream, and this is not limited in the embodiment of the present application.
  • FIG. 10 shows a schematic structural diagram of a multimedia conference control device 1000 provided by an embodiment of the present application.
  • the control device 1000 may be a target gateway or a functional component in the target gateway, and the target gateway may be a media gateway.
  • the control device 1000 includes: a receiving module 1010 and a processing module 1020.
  • the receiving module 1010 is used to receive the audio stream sent to the first terminal through the telephone switching system; the processing module 1020 is used to denoise the audio stream, and the denoising process is to remove the second terminal being called in the audio stream. noise.
  • the functional implementation of the receiving module 1010 may refer to the above-mentioned implementation process of S401, and the functional implementation of the processing module 1020 may refer to the above-mentioned implementation process of S402.
  • the processing module 1020 is also configured to determine that the second terminal is in a called state according to the characteristic parameters of the audio stream.
  • the function implementation of the processing module 1020 may also refer to the implementation process of S403a mentioned above.
  • the characteristic parameters include at least one of the following: audio characteristics of the audio stream, and data characteristics of the data packets of the audio stream.
  • the characteristic parameters include audio characteristics of the audio stream, and the audio characteristics include voice segments contained in the audio stream.
  • the processing module 1020 is configured to compare the voice segments contained in the audio stream with specified voice segments, and determine that the second terminal is in the state of being called. Call state, this specified voice segment is used to describe that the second terminal is in a called state.
  • the control device 1000 also includes: a sending module 1030, used to send the audio stream to the audio recognition device, and the audio recognition device is used to perform audio recognition on the audio stream to obtain the audio characteristics of the audio stream. ;
  • the receiving module 1010 is also used to receive the audio characteristics of the audio stream sent by the audio recognition device.
  • the sending module 1030 and the functional implementation of the receiving module 1010 please refer to the relevant description in S403a above.
  • the characteristic parameters include data characteristics of the data packet of the audio stream.
  • the processing module 1020 is configured to determine that the second terminal is in the called state according to the first identifier contained in the data packet of the audio stream. The first identifier is used to indicate that the second terminal is in a called state. The second terminal is in the called state.
  • the receiving module 1010 is also configured to receive the target signaling message; the processing module 1020 is also configured to determine that the second terminal is in the called state according to the target signaling message containing designated signaling information.
  • the designated signaling information is used To indicate that the second terminal is in the called state.
  • the functional implementation of the receiving module 1010 may also refer to the relevant description in the above S403b, and the functional implementation of the processing module 1020 may also refer to the relevant description in the above S404b.
  • the denoising process includes: intercepting the audio stream. That is, the target gateway does not forward the audio stream.
  • the target gateway intercepts the audio stream to denoise the audio stream, preventing the audio stream from reaching the first terminal, thereby preventing the first terminal from playing the audio stream, and thereby preventing the second terminal from being called in the audio stream.
  • the noise affects the first terminal.
  • the denoising process includes: replacing data packets of the audio stream with silence packets, and sending the silence packets to the first terminal.
  • the target gateway replaces the data packet of the audio stream with a silence packet and sends the silence packet to the first terminal, which can denoise the audio stream and prevent the audio stream from reaching the first terminal, thereby preventing the first terminal from playing the audio stream. audio stream, thereby preventing the noise of the second terminal being called in the audio stream from affecting the first terminal.
  • the denoising process includes: adding a second identifier to the data packet of the audio stream and sending the data packet of the audio stream to the first terminal, where the second identifier is used to instruct the first terminal not to play the audio stream.
  • the target gateway adds the second identifier to the data packet of the audio stream and sends the data packet of the audio stream with the second identifier added to the first terminal. After the first terminal receives the audio stream, the first terminal does not play the audio. stream, therefore it is possible to avoid the noise caused by the second terminal being called in the audio stream from affecting the first terminal, thereby realizing the denoising process of the audio stream.
  • the processing module performs denoising processing on the audio stream to remove the audio stream. Therefore, the noise of the second terminal being called can be prevented from interfering with the first terminal, thereby preventing the noise of the second terminal being called from affecting the development of the multimedia conference.
  • the multimedia conference control device provided by the embodiment of the present application can also be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD).
  • ASIC application-specific integrated circuit
  • PLD programmable logic device
  • the above-mentioned PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a general array logic (generi carray logic, GAL) or any combination thereof.
  • CPLD complex programmable logical device
  • FPGA field-programmable gate array
  • GAL general array logic
  • the multimedia conference control method provided in the above method embodiment can also be implemented through software.
  • each module in the multimedia conference control device can also be a software module.
  • FIG 11 shows a schematic structural diagram of another multimedia conference control device 1100 provided by an embodiment of the present application.
  • the control device 1100 may be a target gateway or a functional component in the target gateway.
  • the target gateway may be a media gateway.
  • the control device 1100 includes a processor 1102, a memory 1104, a communication interface 1106 and a bus 1108.
  • the processor 1102, the memory 1104 and the communication interface 1106 are communicatively connected to each other through the bus 1108.
  • the connection between the processor 1102, the memory 1104 and the communication interface 1106 shown in Figure 11 is only exemplary.
  • the processor 1102, the memory 1104 and the communication interface 1106 can also communicate with each other using other connection methods besides the bus 1108. connect.
  • the memory 1104 can be used to store a computer program 11042, and the computer program 11042 can include instructions and data.
  • the memory 1104 can be various types of storage media, such as random access memory (random access memory, RAM), read-only memory (read-only memory, ROM), non-volatile RAM (non-volatile RAM), etc. -volatile RAM, NVRAM), programmable ROM (programmable ROM, PROM), erasable PROM (erasable PROM, EPROM), electrically erasable PROM (electrically erasable PROM, EEPROM), flash memory, optical memory and registers, etc.
  • storage 1104 may include a hard disk and/or memory.
  • the processor 1102 may be a general-purpose processor, and the general-purpose processor may be a processor that performs specific steps and/or operations by reading and executing a computer program (eg, computer program 11042) stored in a memory (eg, memory 1104), A general-purpose processor may use data stored in a memory (eg, memory 1104) in performing the above steps and/or operations.
  • the stored computer program can, for example, be executed to implement the related functions of the aforementioned processing module 1020 .
  • a general-purpose processor may be, for example but not limited to, a central processing unit (CPU).
  • the processor 1102 may also be a special-purpose processor.
  • the special-purpose processor may be a processor specially designed to perform specific steps and/or operations.
  • the special-purpose processor may be, for example, but not limited to, a digital signal processor. signal processor (DSP), ASIC, FPGA, etc.
  • the processor 1102 may also be a combination of multiple processors, such as a multi-core processor.
  • the processor 1102 may include at least one circuit to execute all or part of the steps of the multimedia conference control method provided in the above embodiments.
  • the communication interface 1106 may include an input/output (I/O) interface, a physical interface, a logical interface, and other interfaces for interconnecting devices within the control device 1100, and for realizing the interconnection between the control device 1100 and other devices. Interfaces for interconnection of devices (such as terminal devices, servers, gateways, etc.).
  • the physical interface may be a gigabit Ethernet (GE) interface, which may be used to interconnect the control device 1100 with other devices.
  • the logical interface may be an internal interface of the control device 1100 , which may be used to implement the internal interface of the control device 1100 . device interconnection. It is easy to understand that the communication interface 1106 can be used to control the communication between the device 1100 and other devices. For example, the communication interface 1106 is used to send and receive signaling between the control device 1100 and other devices, send and receive audio streams, etc., the communication interface 1106
  • the related functions of the aforementioned receiving module 1010 and sending module 1030 can be implemented.
  • the bus 1108 may be any type of communication bus used to interconnect the processor 1102, the memory 1104, and the communication interface 1106, such as a system bus.
  • the above-mentioned devices may be arranged on separate chips, or at least part or all of them may be arranged on the same chip. Whether each device is independently installed on different chips or integrated on one or more chips often depends on the needs of product design.
  • the embodiments of this application do not limit the specific implementation forms of the above devices.
  • the control device 1100 shown in FIG. 11 is only exemplary. During the implementation process, the control device 1100 may also include other components, which are not listed here.
  • the control device 1100 shown in FIG. 11 can control a multimedia conference by executing all or part of the steps of the multimedia conference control method provided in the above embodiments.
  • the embodiment of the present application provides a communication system.
  • the communication system includes a target gateway, a first terminal and a second terminal.
  • the first terminal is communicatively connected to the target gateway.
  • the second terminal is communicatively connected to the target gateway through a telephone switching system.
  • the target gateway It may include a multimedia conference control device as shown in Figure 10 or Figure 11 .
  • the communication system is as shown in any one of Figures 1 to 3.
  • the target gateway may be a media gateway.
  • Embodiments of the present application provide a computer-readable storage medium.
  • a computer program is stored in the computer-readable storage medium.
  • the computer program is executed (for example, executed by a target gateway, one or more processors, etc.), All or part of the steps of the method provided by the above method embodiments.
  • Embodiments of the present application provide a computer program product.
  • the computer program product includes a program or code.
  • the program or code is executed (for example, executed by a target gateway, one or more processors, etc.), the above method is implemented. Examples provide all or part of the steps of the method.
  • Embodiments of the present application provide a chip that includes programmable logic circuits and/or program instructions. When the chip is run, it is used to implement all or part of the steps of the method provided in the above method embodiments.
  • the above embodiments it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it may be implemented in whole or in part in the form of a computer program product including one or more computer instructions.
  • the computer program instructions When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part.
  • the computer may be a general purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions may be transferred from a website, computer, server, or data
  • the center transmits to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server, data center, or the like that includes one or more available media integrated therein.
  • the available media may be magnetic media (eg, floppy disks, hard disks, tapes), optical media, or semiconductor media (eg, solid state drives), etc.
  • the term “at least one” in this application refers to one or more
  • the term “plurality” refers to two or more
  • the term “at least two” refers to two or more.
  • the symbol “/” means or, for example, A/B means A or B.
  • the term “and/or” in this application is only an association relationship describing related objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist simultaneously, alone There are three situations B.
  • words such as “first”, “second” and “third” are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as “first”, “second” and “third” do not limit the number and execution order.
  • the disclosed devices can be implemented in other configurations.
  • the device embodiments described above are only illustrative.
  • the division of modules is only a logical function division. In actual implementation, there may be other division methods.
  • multiple modules or components may be combined or integrated into Another system, or some features can be ignored, or not implemented.
  • Modules described as separate components may or may not be physically separate.
  • Components described as modules may or may not be physical modules, which may be located in one place or distributed to multiple devices (such as terminal devices, gateways). )superior. Some or all modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Telephonic Communication Services (AREA)

Abstract

La présente invention se rapporte au domaine technique des communications, et concerne un procédé et un appareil de commande de conférence multimédia, ainsi qu'un système de communication. Le procédé comprend les étapes suivantes : une passerelle cible reçoit, au moyen d'un système de central téléphonique, un flux audio envoyé à un premier terminal; et la passerelle cible effectue un traitement de débruitage sur le flux audio, le traitement de débruitage étant destiné à éliminer le bruit, dans le flux audio, avec lequel un second terminal est appelé. Puisque la passerelle cible élimine le bruit, dans le flux audio envoyé au premier terminal, avec lequel le second terminal est appelé, il est possible d'empêcher le premier terminal d'être brouillé par ledit bruit, permettant ainsi d'empêcher ledit bruit d'affecter une conférence multimédia.
PCT/CN2022/131215 2022-04-14 2022-11-10 Procédé et appareil de commande de conférence multimédia, et système de communication WO2023197593A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202210395237.9 2022-04-14
CN202210395237 2022-04-14
CN202210666906.1 2022-06-13
CN202210666906.1A CN116962364A (zh) 2022-04-14 2022-06-13 多媒体会议的控制方法及装置、通信系统

Publications (1)

Publication Number Publication Date
WO2023197593A1 true WO2023197593A1 (fr) 2023-10-19

Family

ID=88328858

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/131215 WO2023197593A1 (fr) 2022-04-14 2022-11-10 Procédé et appareil de commande de conférence multimédia, et système de communication

Country Status (1)

Country Link
WO (1) WO2023197593A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1250302A (zh) * 1998-08-21 2000-04-12 朗迅科技公司 解决电信会议中的持续音乐问题的方法
US20030128830A1 (en) * 2002-01-09 2003-07-10 Coffman James E. Selectable muting on conference calls
CN1611059A (zh) * 2001-12-31 2005-04-27 思科技术公司 用于在多方通信会话期间控制音频内容的方法和系统
CN101076108A (zh) * 2007-06-19 2007-11-21 中兴通讯股份有限公司 视频会议终端

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1250302A (zh) * 1998-08-21 2000-04-12 朗迅科技公司 解决电信会议中的持续音乐问题的方法
CN1611059A (zh) * 2001-12-31 2005-04-27 思科技术公司 用于在多方通信会话期间控制音频内容的方法和系统
US20030128830A1 (en) * 2002-01-09 2003-07-10 Coffman James E. Selectable muting on conference calls
CN101076108A (zh) * 2007-06-19 2007-11-21 中兴通讯股份有限公司 视频会议终端

Similar Documents

Publication Publication Date Title
US11778091B2 (en) Utilizing sip messages to determine the status of a remote terminal in VOIP communication systems
EP1704709B1 (fr) Procede et systeme pour la fourniture d'un service de reponses aux appels entre un telephone d'origine et un telephone destinataire
US8861510B1 (en) Dynamic assignment of media proxy
US6909776B2 (en) Systems and methods for monitoring network-based voice messaging systems
US6031896A (en) Real-time voicemail monitoring and call control over the internet
US7702792B2 (en) Method and system for managing communication sessions between a text-based and a voice-based client
US11546741B2 (en) Call routing using call forwarding options in telephony networks
US9071684B2 (en) Media forking
US11588933B2 (en) Methods and apparatus for identification and optimization of artificial intelligence calls
CN102148775B (zh) 网页呼叫服务网关、呼叫服务系统和方法
US20040156493A1 (en) Method and apparatus for providing a central telephony service for a calling party at the called party telephone
CN114401252B (zh) 话务系统的呼叫方法以及话务系统
US8290138B2 (en) Systems, methods, apparatus and computer program products for sharing resources between turret systems and PBXS using SIP
US9148306B2 (en) System and method for classification of media in VoIP sessions with RTP source profiling/tagging
WO2023197593A1 (fr) Procédé et appareil de commande de conférence multimédia, et système de communication
JP2013046116A (ja) コールセンタシステムにおける通話録音システム及び方法
GB2480552A (en) Providing call disposition information for outgoing calls
CN109479071B (zh) 一种网络电话的处理方法及相关网络设备
US11082557B1 (en) Announcement or advertisement in text or video format for real time text or video calls
CN116962364A (zh) 多媒体会议的控制方法及装置、通信系统
US8837459B2 (en) Method and apparatus for providing asynchronous audio messaging
US8116299B2 (en) Techniques for listening to a caller leaving a voicemail message in real-time and real-time pick up of a call
WO2020076344A1 (fr) Routage d'appel à l'aide d'options de transfert d'appel dans des réseaux de téléphonie
KR20170087941A (ko) 클라이언트 애플리케이션을 통한 pbx 전화 호의 제어
KR20040016952A (ko) 교환기에서 호 보류 서비스를 제공하기 위한 방법 및 그교환기 시스템

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22937225

Country of ref document: EP

Kind code of ref document: A1