CN117880422A - Audio telephone video capability expanding method, service system, equipment and storage medium - Google Patents
Audio telephone video capability expanding method, service system, equipment and storage medium Download PDFInfo
- Publication number
- CN117880422A CN117880422A CN202410058067.4A CN202410058067A CN117880422A CN 117880422 A CN117880422 A CN 117880422A CN 202410058067 A CN202410058067 A CN 202410058067A CN 117880422 A CN117880422 A CN 117880422A
- Authority
- CN
- China
- Prior art keywords
- audio
- service module
- phone
- terminal
- dialogue message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 92
- 230000006854 communication Effects 0.000 claims abstract description 55
- 238000004891 communication Methods 0.000 claims abstract description 55
- 230000011664 signaling Effects 0.000 claims description 84
- 230000003993 interaction Effects 0.000 claims description 17
- 238000011161 development Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 5
- 238000005096 rolling process Methods 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 49
- 230000008569 process Effects 0.000 abstract description 36
- 238000012545 processing Methods 0.000 abstract description 11
- 238000005516 engineering process Methods 0.000 abstract description 10
- 230000006870 function Effects 0.000 description 15
- 238000013461 design Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 239000003999 initiator Substances 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 230000007175 bidirectional communication Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Abstract
The embodiment of the invention relates to the field of electronic communication, and discloses an audio phone video capability expanding method, a service system, equipment and a storage medium. In the invention, the video demonstration capability of the audio telephone is expanded in the process of executing the audio call by the audio telephone by combining the real-time communication technology of the browser, the browser page establishes a communication foundation for the call by the audio telephone to exchange SDP and constructs PeerConnection to realize point-to-point communication with other telephone terminals, the transmission of audio, video and data among the browser pages is carried out by the PeerConnection, and a newly added server is not needed to carry out message processing, so that the audio telephone can also send video demonstration media streams to other telephone terminals through the browser page in the process of executing the audio call, the video demonstration picture is transmitted in the process of executing the audio call, and the video demonstration function of the VoIP audio telephone is expanded.
Description
Technical Field
The embodiment of the invention relates to the field of electronic communication, in particular to an audio phone video capability expanding method, a service system, equipment and a storage medium.
Background
With the rapid development of audio and video technologies and applications and the increasing popularity of office automation, audio and video communication becomes a normal state in office automation, and application development based on webRTC audio and video in real-time communication is also receiving more and more attention. The absolute application capability concept of the VoIP audio phone and the video-free sharing capability managed by the PBX server are not suitable for the current practical network development and the practical application requirements of users, so that the call capability with video demonstration needs to be established for the VoIP audio phone. Real-time communication has been shifted from past research aimed at quality and stable conversations to the need to build rich media platforms with collaborative capabilities such as video presentations. Therefore, it has been a focus of attention to construct communication with video presentation capabilities for VoIP audio handsets.
Common applications that have become interconnected communications when voice, video, or video presentation is communicated synchronously, such as various audio-video communication applications such as video conferencing, online education, and the like. At present, all audio and video communication uses a telephone terminal with video demonstration picture capability, or the real-time transmission and synchronous display of sound and pictures can be realized by a proper coding and decoding algorithm and a network transmission optimization technology; in addition, the existing video transmission function also depends on a newly established server for transmitting video data, and if the video demonstration function expansion of the VoIP audio phone is to be realized, a large amount of cost is required to be consumed to construct the new server. Therefore, a scheme for expanding the picture demonstration capability of the VoIP audio phone equipment capable of realizing the voice call by means of the existing server is needed, so as to improve the remote collaboration capability of the VoIP audio phone, and improve the effectiveness and the use experience of user communication.
Disclosure of Invention
The embodiment of the invention aims to provide an audio phone video capability expanding method, a service system, equipment and a storage medium, so that the picture demonstration capability of VoIP audio phone equipment can be expanded by means of an original phone server under the condition that a video data transmission server is not additionally arranged, thereby improving the remote collaboration capability of the VoIP audio phone, and improving the communication effectiveness and the use experience of users.
In order to solve the above technical problems, an embodiment of the present invention provides an audio phone video capability expanding method, which is applied to a first phone terminal, and includes the following steps: starting an audio call, starting a browser and creating a PeerConnection in a first page; the first Web service module of the first telephone terminal receives the first dialogue message of the PeerConnection and sends the first dialogue message to an opposite terminal through the first signaling service module of the first telephone terminal; the first dialogue message comprises a first SDP object which is created and stored on the first page, the first SDP object stores first media information data of the first phone terminal, and the first media information data comprises first video stream data of the first phone terminal; and receiving a second dialogue message forwarded by the first signaling service module through the first page, wherein the first signaling service module sends the second dialogue message to the first page through the first Web service module.
The embodiment of the invention also provides an audio phone video capability expanding method which is applied to a second phone terminal and comprises the following steps: enabling a browser, and receiving a first dialogue message forwarded by a second signaling service module of the second phone terminal through a second page, wherein the first dialogue message comprises a first SDP object created and stored in a first page by the first phone terminal, the first SDP object stores first media information data of the first phone terminal, and the first media information data comprises first video stream data of the first phone terminal; the second signaling service module sends the first dialogue message to the second page through a second Web service module of the second phone terminal, so that the browser correspondingly generates a second dialogue message according to the first dialogue message; the second Web service module receives and stores the second dialogue message in a local PeerConnection, and sends the second dialogue message to an opposite terminal through the second signaling service module; and the reply content of the second dialogue message is refused to share or accepted to open share.
The embodiment of the invention also provides an audio telephone service system, which comprises: the first telephone terminal, the second telephone terminal and the signaling server are respectively connected with the first telephone terminal and the second telephone terminal; the first telephone terminal and the second telephone terminal are both in communication connection with the signaling server; the signaling server is configured to perform SDP interaction between the first phone terminal and the second phone terminal.
The embodiment of the invention also provides audio telephone equipment, which comprises: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the audio phone video capability extension method described above.
Embodiments of the present invention also provide a computer readable storage medium storing a computer program which, when executed by a processor, implements the audio phone video capability extension method of the claims.
In the embodiment of the invention, the video demonstration capability of the audio telephone is expanded in the process of executing the audio call by the audio telephone through a Real-time communication technology (Web Real-Time Communications, webRTC) of combining the audio telephone with a browser, the browser page establishes a communication foundation exchange SDP of the call by the audio telephone and constructs a PeerConnection to realize point-to-point communication with other telephone terminals, the PeerConnection is used for transmitting audio, video and data among the browser pages, and a newly added video transmission server is not required for carrying out message processing, so that the audio telephone can also transmit the video demonstration media stream content to other telephone terminals through the browser page in the process of executing the voice call, thereby realizing the transmission of video demonstration pictures in the process of executing the audio call and expanding the video demonstration function of the VoIP audio telephone; and the existing signaling server is used for realizing the transmission of video data, the original signaling server service is not changed, and a new video stream transmission server is not needed.
Drawings
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.
Fig. 1 is a flowchart of an audio phone video capability development method applied to a first phone terminal according to an embodiment of the present application;
fig. 2 is a flowchart of an audio phone video capability development method applied to a second phone terminal according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for expanding video capabilities of an audio phone according to an embodiment of the present application;
fig. 4 is a flowchart of interaction of a browser, a phone, and a signaling server according to an embodiment of the present application;
fig. 5 is a second interaction flow chart of a browser, a phone, and a signaling server according to an embodiment of the present application;
fig. 6 is a third interaction flow chart of a browser, a phone, and a signaling server according to an embodiment of the present application;
fig. 7 is a fourth interaction flow chart of a browser, a phone, and a signaling server according to an embodiment of the present application;
fig. 8 is a fifth interaction flow chart of a browser, a phone, and a signaling server according to an embodiment of the present application;
Fig. 9 is a signaling interaction diagram of a video capability development method of an audio phone according to an embodiment of the present application;
fig. 10 is an interaction diagram of SDP processing provided by an embodiment of the present application;
fig. 11 is a schematic structural diagram of an audio phone service system according to an embodiment of the present application;
fig. 12 is a schematic structural diagram of an audio phone device according to an embodiment of the present application.
Detailed Description
Real-time communication has been changed from the past research to the requirement of establishing a rich media platform with collaborative capabilities such as video presentation and the like aiming at pursuing quality and stable conversation, namely, the video-free sharing state of the VoIP audio phone is not suitable for the current actual network development and the actual application requirement of users, and a conversation with new capability and video presentation needs to be established. Thus, building communications with video presentation capabilities has become a focus of attention. However, the real-time communication not only focuses on the quality, real-time performance and stability of the call itself, but also focuses on the effectiveness and convenience of communication and communication, and the construction based on the user requirements is one of important contents of the study. The effectiveness of communication is ensured, the use experience is provided, and the method is also an important premise of demonstration function.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art will understand that in various embodiments of the present invention, numerous technical details have been set forth in order to provide a better understanding of the present application. However, the technical solutions claimed in the present application can be implemented without these technical details and with various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not be construed as limiting the specific implementation of the present invention, and the embodiments can be mutually combined and referred to without contradiction.
An embodiment of the invention relates to an audio phone video capability expanding method which can be applied to VoIP audio phone terminal equipment or embedded phone terminal equipment. The method of the invention comprises the steps of: starting an audio call, starting a browser and creating a PeerConnection in a first page; the first Web service module of the first telephone terminal receives the first dialogue message of the PeerConnection and sends the first dialogue message to the opposite terminal through the first signaling service module of the first telephone terminal; the first dialogue message comprises a first SDP object which is created and stored on the first page, the first SDP object stores first media information data of the first phone terminal, and the first media information data comprises first video stream data of the first phone terminal; and receiving a second dialogue message forwarded by the first signaling service module through the first page, wherein the first signaling service module sends the second dialogue message to the first page through the first Web service module. The video demonstration capability of the audio telephone is expanded in the process of executing the audio call by the audio telephone in combination with the real-time communication technology of the browser, the browser page establishes communication foundation exchange SDP of the call by the audio telephone and constructs PeerConnection to realize point-to-point communication with other telephone terminals, and the audio, video and data transmission among the browser pages is carried out, so that the audio telephone can also transmit video demonstration media stream content to other telephone terminals through the browser page in the process of executing the voice call, thereby realizing video demonstration picture transmission in the process of executing the audio call and expanding the video demonstration function of the VoIP audio telephone; and the existing signaling server is used for realizing the transmission of video data, the original signaling server service is not changed, and a new video stream transmission server is not needed. In addition, the original audio function of the telephone equipment can be reserved and the sound effect is good because the communication characteristic of the VoIP audio telephone terminal equipment or the embedded telephone terminal equipment is kept; because the PeerConnection is a mode of establishing communication connection in the WebRTC, the invention realizes real-time communication of the browser page through the WebRTC, including transmission of audio, video and data, so that a user can directly carry out video conference, voice call, file sharing and the like in the browser, and point-to-point communication is supported.
Implementation details of the audio phone video capability development method according to the embodiment of the present invention are specifically described below, and the following details are provided only for easy understanding, and are not necessary for implementing the present embodiment.
In some embodiments, each VoIP audio phone terminal has a physical location address (Internet Protocol, IP), and the browser page logs in using the phone IP, so that the Web terminal and the phone terminal establish a one-to-one correspondence, and the two terminals are ensured to maintain a connection relationship. After successful login, the browser page of each VoIP audio phone terminal establishes a communication protocol connection through open source Web server software such as lighttpd or webbgi. Optionally, the communication protocol may be a webSocket communication protocol, and related information content established by a call is obtained through webSocket, such as number information, line information, opposite terminal account number and the like of the terminal to which the webSocket belongs. Alternatively, the browser may be a Chrome, fireFox, edge or Safari browser.
As shown in fig. 1, in step 101, the first phone terminal, which is the session initiator, initiates an audio call, enables a browser and creates a PeerConnection in a first page.
In step 102, the first Web service module of the first phone terminal receives the first session message of the PeerConnection, and sends the first session message to the peer through the first signaling service module of the first phone terminal. That is, the browser and the first Web service module of the first phone terminal perform message transfer, and then send the message to the first signaling service module of the first phone terminal through the first Web service module, and finally the first signaling service module performs SDP exchange through Indialog SIP Message. The first dialogue message includes a first SDP object created and stored on the first page, where the first SDP object stores first media information data of the first phone terminal, and the first media information data includes first video stream data of the first phone terminal.
In step 103, the first phone terminal receives, through the first page, the second dialogue message forwarded by the first signaling service module. Wherein the first signaling service module sends the second dialog message to the first page through the first Web service module.
As shown in fig. 2, in step 201, the second phone terminal as a session answering party starts a browser, and receives, through a second page, a first session message forwarded by a second signaling service module of the second phone terminal; the first dialogue message comprises a first SDP object created and stored in a first page by a first phone terminal, wherein the first SDP object stores first media information data of the first phone terminal, and the first media information data comprises first video stream data of the first phone terminal. The second signaling service module sends the first dialogue message to the second page through the second Web service module of the second phone terminal, and the browser correspondingly generates the second dialogue message according to the first dialogue message, wherein whether corresponding second video stream data need to be generated can be determined according to scene requirements.
In step 202, a second Web service module receives and stores the second session message in a local PeerConnection, and sends the second session message to an opposite terminal through the second signaling service module; and the reply content of the second dialogue message is refused to share or accepted to open share.
Taking the example that the first page and the second page are WebUI pages, the method for expanding the video capability of the audio phone according to the embodiment is summarized as follows:
(1) And establishing a binding relation with the IP of the phone terminal through the WebUI of the access terminal, and finally establishing webSocket connection to enable the WebUI and the phone terminal to maintain a connection state.
(2) And starting the audio call through SIP establishment, wherein the audio is local to the terminal.
(3) Acquiring SDP objects in the WebRTC in a WebUI page (namely a first page or a second page written by an agent);
(3) SDP exchange is carried out on the signaling service module through SIP, so that the signaling service module completes the JSEP flow.
(4) The two terminals directly establish a media channel through WebRTC to perform video sharing or data channel (data transmission channel) to transmit other shared data.
For the browser page, after the WebUI page logs in, a binding relation is established with the phone terminal in a bidirectional transmission protocol mode, and the connection state is guaranteed, so that the WebUI page end is in a bidirectional communication state, namely the connection state of the WebUI page is a basic condition for starting sharing. The bidirectional transmission protocol can adopt WebSocket and HTTPS protocols. In addition, in some scenarios, to ensure the security of data transmission, a security protocol in which an encryption algorithm exists, such as WSS in WebSocket, or HTTPS, may be selected.
Aiming at the condition that the media video stream falls on the Web end, namely, processing the audio stream at the telephone terminal; the browser processes the video stream through WebRTC, the two media are separated, and in the case of processing at the two positions of the terminal and the browser, the following two modes exist in the SDP exchanging mode in the media connection process:
(1) The audio frequency is exchanged by the telephone set by adopting SIP; the videos are subjected to SDP exchange by the WebUI page, and the interaction relationship among the browser, the phone and the server is shown in fig. 4. In the process of talking of the phone terminal, a browser is started, a corresponding relation between the phone and the browser is established through a transmission protocol such as a WebSocket protocol, and SDP objects comprising videos are transmitted through the browser; in the conversation process, only the telephone and the server interact and transmit data or information, and the browser does not directly interact with the server to transmit data or information. The existing signaling server is used for realizing the transmission of video data, the original signaling server is not changed, and a new server is not required to be built when the video demonstration function of the VoIP audio phone is expanded.
(2) SDP exchange is carried out by means of the SIP call transmission channel established by the telephone. For the audio-video media expressed in the SDP, the receiver decides the processing scheme according to its own capabilities.
It should be noted that three different implementations of the interaction relationship among the browser, phone and server are shown in fig. 4, 5 and 6. In the process of talking of the phone terminal, a browser is started, and the corresponding relation between the phone and the browser is established through a transmission protocol such as WebSocket protocol, however, the video capability of the phone is specifically required to be expanded according to the capability of the phone. As shown in fig. 4, the video streaming media is transmitted through the phone by SIP, so as to realize the exchange of SDP objects; unlike fig. 4, in fig. 5, the browser and the phone do not interact, and the two ends directly exchange SDP objects through WebUI; in fig. 6, a relationship is established between the browser and the phone, and the phone acquires the video media stream of the browser and then sends the video media stream to the opposite terminal.
Fig. 6, fig. 7 and fig. 8 show different schemes for expanding video capability of the phone, and provide schemes for the receiving end to display the video stream at different terminals according to the capability of the phone itself. Aiming at the situation that the media video stream falls on the telephone terminal, the video stream is sent to the telephone terminal by the browser; the JSEP process is established between the WebUI terminal and the telephone terminal, so that the telephone terminal obtains a video stream; and then the SDP exchange is carried out between the telephone terminal and other telephone terminals through the established SIP call transmission channel. As a receiving party, the processing flow of the other phone end determines a processing scheme according to the own capability, for example, if the receiving end phone has video capability, the video directly falls on the phone end, as shown in fig. 6; if the receiver phone does not have video capability, the video falls to the Web side as shown in fig. 7.
As shown in fig. 3 and 9, in step 301, the first phone terminal initiates an audio call, enables a browser and creates a PeerConnection in a first page.
In some embodiments, the first page of the first handset terminal, when launching the audio call enabled browser, opens a doOffer flow that shares and handles JavaScript session setup protocol (JavaScript Session Establishment Protocol, JSEP).
In some embodiments, after the first phone terminal creates the PeerConnection, a data transmission channel (DataChannel) is enabled by calling a method of createDataChannel, which is used for message transmission, file transmission, and the like.
In some embodiments, the message is json format data.
In step 302, the first Web service module of the first phone terminal receives the first session message of the PeerConnection, and sends the first session message to the peer through the first signaling service module of the first phone terminal. The first dialogue message includes a first SDP object created and stored on the first page, where the first SDP object stores first media information data of the first phone terminal, and the first media information data includes first video stream data of the first phone terminal.
In some embodiments, the first phone terminal creates the first SDP object in the first page by a createOffer method of PeerConnection, and saves the first SDP object by a setLocalDescription method of PeerConnection.
In some embodiments, the first phone terminal sends a first dialogue message to the first signaling service module, wherein the first dialogue message includes number information, line information, transaction ID, first SDP object, etc. of the terminal to which the first dialogue message belongs; the SDP object stores media information for two WebRTC terminals to establish PeerConnection, and after the PeerConnection is established, the SDP object can be used for transmitting information such as audio, video, data and the like.
In some embodiments, the first dialogue message is an open share request message; the first Web service module receives a first dialogue message from the PeerConnection of the browser, and the first signaling service module sends the first dialogue message to the second signaling service module of the second phone terminal through the signaling server.
Specifically, in some embodiments, the first Web service module receives the first session message of the PeerConnection from the browser through webSocket protocol or HTTPS protocol, and the signaling service module performs SDP exchange through Indialog SIP Message. The protocol or method by which the dialogue message is sent may be set and adjusted according to the actual design and requirements, and is not limited herein.
In some embodiments, the second phone terminal enables the browser to receive the first dialogue message forwarded by the second signaling service module through the second page; the first dialogue message comprises a first SDP object created and stored in a first page by a first phone terminal, wherein the first SDP object stores first media information data of the first phone terminal, and the first media information data comprises first video stream data of the first phone terminal. And the second signaling service module sends the first dialogue message to a second page through a second Web service module of a second telephone terminal, so that the browser correspondingly generates a second dialogue message according to the first dialogue message.
In step 303, after the second phone terminal receives the first dialogue message initiated by the first phone terminal on the second page, the second phone terminal displays an interaction option for receiving or rejecting the call on the second page, and generates a second dialogue message according to the selection of the user of the second phone terminal.
Specifically, in some embodiments, the presentation of the interaction options may be performed through an interaction button. It should be noted that how the presentation of the interaction options is performed may be set and adjusted according to the actual design and requirements, and is not limited herein.
In some embodiments, the reply content of the second conversation message is to refuse the sharing or accept the open sharing.
In some embodiments, when the reply content of the second session message is refusal to share, after receiving the second session message forwarded by the first signaling service module through the first page, the method further includes: resetting the line content and state, and rolling back the PeerConnection state.
In some embodiments, the second phone terminal generates a second dialogue message through the second page, saves the second dialogue message to the local PeerConnection, and sends the second dialogue message to the opposite terminal through the second signaling service module.
In step 304, when the user of the second phone terminal selects to refuse to open the sharing, the second phone terminal generates a second dialogue message through the second page, and sends the second dialogue message to the first phone terminal through the second signaling service module. Specifically, in some embodiments, the second dialogue message includes number information of the belonging terminal, line information, a corresponding transaction ID, and a reject shared message body.
In some embodiments, the first phone terminal receives the second dialogue message forwarded by the first signaling service module through the first page.
In step 305, after receiving the second session message from the second phone terminal, the first phone terminal processes the content of the corresponding line and rolls back the PeerConnection.
In some embodiments, when the reply content of the second session message is that the open sharing is accepted, the second session terminal saves a second SDP object in the second session message to the local PeerConnection, where the second SDP object is used to save the content of the second session terminal local presentation media stream. If step 306 is shown, when the user of the second phone terminal selects to answer and start sharing, the second phone terminal saves the first SDP object by the setrequest description method of PeerConnection; and calling a createananswer method of Peerconnection to create a second SDP object, and storing the second SDP object through a setLocalDescription method of Peerconnection.
In step 307, the second phone terminal generates a second dialogue message through the second page, and sends the second dialogue message to the first signaling service module of the first phone terminal through the second signaling service module, and then sends the second dialogue message to the first page through the first Web service module of the first phone terminal. Specifically, in some embodiments, the reply content of the second session message is to accept the open sharing, including the number information of the terminal to which the reply content belongs, the line information, the corresponding transaction ID, the received shared message body, and the second SDP object.
In step 308, the first phone terminal receives the second session message, saves the second SDP object by the setrequest description method of the PeerConnection, and releases the corresponding transaction ID.
In some embodiments, the SDP object includes Candidate information, and after receiving the second dialog message forwarded by the first signaling service module via the first page, further includes: and starting collection of Candidate data through the first page.
In step 309, in the offer/answer flow of the SDP information, the Web page creates a corresponding video channel according to the SDP information and starts collection of Candidate data.
In step 310, after the first page of the first phone terminal collects the Candidate information, a notification is sent to the second page of the second phone terminal through the OnIceCandidate interface.
In step 311, the second page adds and saves the corresponding Candidate information content in the first page by the AddIceCandidate method of PeerConnection.
In step 312, the second page sends the received Candidate information to the first page, and the first page adds and stores the corresponding Candidate information content in the second page by the AddIceCandidate method of PeerConnection, thereby completing the establishment of the video transmission P2P channels of the first phone terminal and the second phone terminal.
In step 313, the page receives a video presentation stream and then renders a picture in the page. Specifically, in some embodiments, after receiving a video presentation stream sent by a first phone terminal, a second page returns a first identifier of an audio/video stream MediaStream object of the first phone terminal through an ondadstream callback interface of PeerConnection, and renders the first identifier in the second page; after the first page receives the video demonstration stream sent by the second phone terminal, the second identification of the audio/video stream media stream object of the second phone terminal is returned through the OnAddStream callback interface of PeerConnection, and rendering is carried out in the second page. It is worth to say that, in the conversation process, after receiving the audio, video and demonstration content of the opposite terminal, the phone terminals in communication with each other can render and display the audio, video and demonstration content of the opposite terminal at the same time; the audio, video and demonstration content of the picture rendering are obtained through an OnAddStream interface of monitoring PeerConnection; and the data information is transmitted through dataChannel transmission. When the audio call ends, the page presentation ends with the audio call.
In some embodiments, when the page displays the demonstration picture, in order to make the communication between the users of the first phone terminal and the second phone terminal faster and more convenient, interactive options such as setting forth, drawing, annotating, marking, sending a file, full screen, exiting full screen and the like are provided for the users in the page, so that the communication efficiency is improved.
In one example, as shown in fig. 10, webRTC creates a connection using a PeerConnection class, which contains two methods of generating SDP objects, createOffer and createonner, respectively, where createOffer method is called by a session initiator to generate offerSDP and send to a signaling server; the createananswer method is invoked to generate an answer sdp after the responder receives the signaling server message, and then also sends back to the signaling server.
The PeerConnection also includes two methods for setting SDP objects, setLocalDescription and setsemotedescription, respectively, where the setLocalDescription method is used to set a local SDP object and the setsemotedescription is used to set a received remote remoteSDP object. It should be noted that, for the session initiator, localSDP is offerSDP, remoteSDP, i.e. answerSDP; for the responder, localSDP is the answerSDP, and remoteSDP is the received offerSDP.
Specifically, when storing the SDP object, the first phone terminal needs to store the generated first SDP object locally, and also stores the second SDP object of the second phone terminal; similarly, the second phone terminal also needs to store the generated second SDP object locally, and also needs to store the first SDP object of the first phone terminal. After receiving the SDP object of the opposite terminal, the first phone terminal and the second phone terminal store the SDP object of the opposite terminal, generate own SDP object and store the SDP object to the local, and further process setRemoteDescription.
In the embodiment of the invention, the video demonstration capability of the audio telephone is expanded in the process of executing the audio call by the audio telephone through the real-time communication technology of combining the audio telephone with the browser, the browser page establishes a communication foundation for the call by the audio telephone to exchange SDP and constructs PeerConnection to realize point-to-point communication with other telephone terminals, the PeerConnection is used for transmitting audio, video and data among the browser pages, and a newly added video transmission server is not needed for message transfer, so that the audio telephone can also transmit video demonstration media stream content to other telephone terminals through the browser page in the process of executing the voice call, thereby realizing the transmission of video demonstration pictures in the process of executing the audio call and expanding the video demonstration function of the VoIP audio telephone; and the existing signaling server is used for realizing the transmission of video data, the original signaling server service is not changed, and a new video stream transmission server is not needed.
The above method is divided into steps, which are only for clarity of description, and may be combined into one step or split into multiple steps when implemented, so long as they include the same logic relationship, and they are all within the protection scope of this patent; it is within the scope of this patent to add insignificant modifications to the algorithm or flow or introduce insignificant designs, but not to alter the core design of its algorithm and flow.
In addition, the examples mentioned in the above embodiments can be freely combined, and any combination can be understood as an embodiment. The appearances of the "embodiment" or "examples" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those skilled in the art will appreciate that the embodiments described herein may be combined with other embodiments.
In summary, particular embodiments of the present subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results.
In the description of the embodiments of the present application, the technical terms "first," "second," etc. are used merely to distinguish between different objects and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated, a particular order or a primary or secondary relationship. In the description of the embodiments of the present application, the meaning of "plurality" is two or more unless explicitly defined otherwise.
Another embodiment of the present invention relates to a method for expanding video capability of an audio phone, which is applied to a second phone terminal, as shown in fig. 2, and includes: enabling a browser, and receiving a first dialogue message forwarded by a second signaling service module of a second phone terminal through a second page; the first dialogue message comprises a first SDP object created and stored in a first page by a first phone terminal, wherein the first SDP object stores first media information data of the first phone terminal, and the first media information data comprises first video stream data of the first phone terminal. The second signaling service module sends the first dialogue message to the second page through the second Web service module of the second phone terminal, so that the browser can correspondingly generate the second dialogue message according to the first dialogue message. The second Web service module receives and stores the second dialogue message in the local PeerConnection, and sends the second dialogue message to the opposite terminal through the second signaling service module; and the reply content of the second dialogue message is refused to share or accepted to open share.
In the embodiment of the invention, the video demonstration capability of the audio telephone is expanded in the process of executing the audio call by the audio telephone through the real-time communication technology of combining the audio telephone with the browser, the browser page establishes a communication foundation for the call by the audio telephone to exchange SDP and constructs PeerConnection to realize point-to-point communication with other telephone terminals, the PeerConnection is used for transmitting audio, video and data among the browser pages, and a newly added video transmission server is not needed for message transfer, so that the audio telephone can also transmit video demonstration media stream content to other telephone terminals through the browser page in the process of executing the voice call, thereby realizing the transmission of video demonstration pictures in the process of executing the audio call and expanding the video demonstration function of the VoIP audio telephone; and the transmission of video data is realized by using the existing signaling service module, the service of the original signaling service module is not changed, and a new video stream transmission server is not needed.
Another embodiment of the present invention relates to an audio phone service system, as shown in fig. 11, including: a first handset terminal 401, a second handset terminal 402, and a signaling server 403. The first phone terminal 401 is the first phone terminal in the above embodiment, the second phone terminal 402 is the second phone terminal in the above embodiment, and both the first phone terminal 401 and the second phone terminal 402 are in communication connection with the signaling server 403; the signaling server 403 is configured to perform SDP interaction between the first handset terminal 401 and the second handset terminal 402.
The operation flow of the first phone terminal 401 and the second phone terminal 402 is described in detail in the above embodiments, which is not described herein again, and the signaling server 403 performs message transmission in a transparent manner, so as to implement SDP interaction between the first phone terminal 401 and the second phone terminal 402. The video demonstration media stream can be sent to other telephone terminals through the browser page in the voice call process of the audio telephone, so that the video demonstration picture is transmitted in the audio call process, and the video demonstration function of the VoIP audio telephone is expanded.
It is to be noted that this embodiment is an embodiment of the apparatus corresponding to the above-described method embodiment, and this embodiment may be implemented in cooperation with the above-described method embodiment. The related technical details mentioned in the above method embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the related technical details mentioned in the present embodiment can also be applied in the above-described method embodiments.
It should be noted that, each module involved in this embodiment is a logic module, and in practical application, one logic unit may be one physical unit, or may be a part of one physical unit, or may be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, units less closely related to solving the technical problem presented by the present invention are not introduced in the present embodiment, but it does not indicate that other units are not present in the present embodiment.
In the embodiment of the invention, the video demonstration capability of the audio telephone is expanded in the process of executing the audio call by the audio telephone through the real-time communication technology of combining the audio telephone with the browser, the browser page establishes a communication foundation for the call by the audio telephone to exchange SDP and constructs PeerConnection to realize point-to-point communication with other telephone terminals, the PeerConnection is used for transmitting audio, video and data among the browser pages, and a newly added video transmission server is not needed for message transfer, so that the audio telephone can also transmit video demonstration media stream content to other telephone terminals through the browser page in the process of executing the voice call, thereby realizing the transmission of video demonstration pictures in the process of executing the audio call and expanding the video demonstration function of the VoIP audio telephone; and the existing phone signaling server is used for realizing the transmission of video data, the original phone signaling server service is not changed, and a new video streaming server is not needed.
Another embodiment of the invention is directed to an audio handset device, as shown in fig. 12, comprising at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform an audio phone video capability extension method as described above.
Where the memory and the processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting the various circuits of the one or more processors and the memory together. The bus may also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or may be a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over the wireless medium via the antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory may be used to store data used by the processor in performing operations.
In the embodiment of the invention, the video demonstration capability of the audio telephone is expanded in the process of executing the audio call by the audio telephone through the real-time communication technology of combining the audio telephone with the browser, the browser page establishes a communication foundation for the call by the audio telephone to exchange SDP and constructs PeerConnection to realize point-to-point communication with other telephone terminals, the PeerConnection is used for transmitting audio, video and data among the browser pages, and a newly added video transmission server is not needed for carrying out message processing, so that the audio telephone can also transmit video demonstration media stream content to other telephone terminals through the browser page in the process of executing the voice call, thereby realizing the transmission of video demonstration pictures in the process of executing the audio call and expanding the video demonstration function of the VoIP audio telephone; and the existing signaling server is used for realizing the transmission of video data, the original signaling server service is not changed, and a new video stream transmission server is not needed.
Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The computer program, when executed by the processor, implements the audio phone video capability extension method embodiments described above.
That is, it will be understood by those skilled in the art that all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program stored in a storage medium, where the program includes several instructions for causing a device (which may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps in the methods of the embodiments described herein. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In the embodiment of the invention, the video demonstration capability of the audio telephone is expanded in the process of executing the audio call by the audio telephone through the real-time communication technology of combining the audio telephone with the browser, the browser page establishes a communication foundation for the call by the audio telephone to exchange SDP and constructs PeerConnection to realize point-to-point communication with other telephone terminals, the PeerConnection is used for transmitting audio, video and data among the browser pages, and a newly added video transmission server is not needed for carrying out message processing, so that the audio telephone can also transmit video demonstration media stream content to other telephone terminals through the browser page in the process of executing the voice call, thereby realizing the transmission of video demonstration pictures in the process of executing the audio call and expanding the video demonstration function of the VoIP audio telephone; and the existing signaling server is used for realizing the transmission of video data, the original signaling server service is not changed, and a new video stream transmission server is not needed.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples of carrying out the invention and that various changes in form and details may be made therein without departing from the spirit and scope of the invention.
Claims (10)
1. An audio phone video capability extension method, applied to a first phone terminal, comprising the steps of:
starting an audio call, starting a browser and creating a PeerConnection in a first page;
the first Web service module of the first telephone terminal receives the first dialogue message of the PeerConnection and sends the first dialogue message to an opposite terminal through the first signaling service module of the first telephone terminal; the first dialogue message comprises a first SDP object which is created and stored on the first page, the first SDP object stores first media information data of the first phone terminal, and the first media information data comprises first video stream data of the first phone terminal;
and receiving a second dialogue message forwarded by the first signaling service module through the first page, wherein the first signaling service module sends the second dialogue message to the first page through the first Web service module.
2. The audio phone video capability extension method according to claim 1, wherein the first dialogue message is an open sharing request message;
the first Web service module of the first telephone terminal receives the first dialogue message of the PeerConnection and sends the first dialogue message to an opposite terminal through the first signaling service module of the first telephone terminal; comprising the following steps:
the first Web service module receives a first dialogue message of the PeerConnection from the browser;
the first signaling service module sends the first dialogue message to a second signaling service module of a second phone terminal through a signaling server.
3. The audio phone video capability extension method according to claim 1, wherein the reply content of the second dialogue message is refusal of sharing or acceptance of open sharing.
4. The audio phone video capability extension method according to claim 3, wherein when the reply content of the second dialogue message is refusal of sharing, the method further comprises:
and after the second dialogue message forwarded by the first signaling service module is received through the first page, resetting line content and state, and rolling back the PeerConnection state.
5. The audio telephony video capability extension method as set forth in claim 3, wherein when the reply content of the second session message is to accept open sharing, after receiving the second session message forwarded by the first signaling service module via the first page, the method further comprises:
and storing the second SDP object in the second dialogue message into a local PeerConnection, wherein the second SDP object is used for storing second media information data of a second phone terminal, and the second media information data comprises second video stream data of the second phone terminal.
6. The audio phone video capability extension method of claim 2, wherein the first Web service module receiving a first dialogue message from the PeerConnection of the browser comprises:
and the first Web service module receives a first dialogue message of the PeerConnection from the browser through webSocket protocol or HTTPS protocol.
7. An audio phone video capability development method, which is applied to a second phone terminal, comprising the following steps:
enabling a browser, and receiving a first dialogue message forwarded by a second signaling service module of the second phone terminal through a second page, wherein the first dialogue message comprises a first SDP object created and stored in a first page by the first phone terminal, the first SDP object stores first media information data of the first phone terminal, and the first media information data comprises first video stream data of the first phone terminal; the second signaling service module sends the first dialogue message to the second page through a second Web service module of the second phone terminal, so that the browser correspondingly generates a second dialogue message according to the first dialogue message;
The second Web service module receives and stores the second dialogue message in a local PeerConnection, and sends the second dialogue message to an opposite terminal through the second signaling service module; and the reply content of the second dialogue message is refused to share or accepted to open share.
8. An audio phone service system, comprising: a first handset terminal as claimed in any one of claims 1 to 6, a second handset terminal as claimed in claim 7 and a signalling server;
the first telephone terminal and the second telephone terminal are both in communication connection with the signaling server; the signaling server is configured to perform SDP interaction between the first phone terminal and the second phone terminal.
9. An audio phone device, comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the audio phone video capability extension method of any one of claims 1 to 7.
10. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the audio phone video capability extension method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410058067.4A CN117880422A (en) | 2024-01-12 | 2024-01-12 | Audio telephone video capability expanding method, service system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410058067.4A CN117880422A (en) | 2024-01-12 | 2024-01-12 | Audio telephone video capability expanding method, service system, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117880422A true CN117880422A (en) | 2024-04-12 |
Family
ID=90592945
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410058067.4A Pending CN117880422A (en) | 2024-01-12 | 2024-01-12 | Audio telephone video capability expanding method, service system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117880422A (en) |
-
2024
- 2024-01-12 CN CN202410058067.4A patent/CN117880422A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9894111B2 (en) | System and method for data transfer between terminals in voice communication under voice over internet protocol (VoIP) | |
| US10798138B2 (en) | Instant calling method, apparatus and system | |
| JP4299320B2 (en) | Group communication server | |
| US20170034149A1 (en) | Intelligent Communications Method, Terminal, and System | |
| CN101924772B (en) | Communication system and method supporting cross-network and cross-terminal realization of multimedia session merging | |
| JP2016517642A (en) | Video communication method, home terminal and home server | |
| CN101317363A (en) | Media stream replacement method and system in terminal conversation course | |
| CN113746808A (en) | Converged communication method for online conference, gateway, electronic device, and storage medium | |
| CN104902111B (en) | A kind of method, apparatus and system established based on Web RTC multi-party call | |
| CN101540870A (en) | Realization method of video call service | |
| KR101589195B1 (en) | METHOD AND APPARATUS FOR SEAMLESSlY IMPLEMENTING TRNASFERRING DUAL-PARTY CALL INTO CONFERENCE | |
| CN101317434B (en) | Method, system and terminal unit for acquiring media characteristic information | |
| CN101562735A (en) | Small video terminal and method for realizing video sharing at same | |
| US20120089680A1 (en) | Communication apparatus, communication system and session control method | |
| CN107666396B (en) | Multi-terminal conference processing method and device | |
| CN110650254B (en) | Information transmission method, information reception method, terminal, and storage medium | |
| CN102055961A (en) | Method for monitoring visible terminal of called party and video monitoring system | |
| EP3002943B1 (en) | Method for call processing and gateway | |
| JP2008067083A (en) | System and method for controlling group call, and mobile communication terminal | |
| CN117880422A (en) | Audio telephone video capability expanding method, service system, equipment and storage medium | |
| CN108616496A (en) | Implementation method, device, mostly logical terminal and the network side equipment of multi-party call | |
| JP5095120B2 (en) | Group call system, mobile phone terminal and server | |
| JP4644813B2 (en) | Multi-party call system, call terminal and call server in multi-party call system, multi-party call method | |
| CN113573006B (en) | Special line video conference communication method, device, equipment and storage medium | |
| WO2023011476A1 (en) | Communication method, data channel establishment method, device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |