CN107168856B - Method for displaying terminal application program by Chrome webrtc-interpnals tool - Google Patents
Method for displaying terminal application program by Chrome webrtc-interpnals tool Download PDFInfo
- Publication number
- CN107168856B CN107168856B CN201710350201.8A CN201710350201A CN107168856B CN 107168856 B CN107168856 B CN 107168856B CN 201710350201 A CN201710350201 A CN 201710350201A CN 107168856 B CN107168856 B CN 107168856B
- Authority
- CN
- China
- Prior art keywords
- module
- webrtc
- data
- statistical data
- chrome
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/323—Visualisation of programs or trace data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
Abstract
The invention relates to a method for displaying a terminal application program by a Chrome webrtc-interpnals tool, which comprises an instant messaging application program module and a Chrome browser module; the instant communication application program module comprises; the system comprises a webrtc library and a statistical data module, wherein the statistical data module is provided with an interface module for acquiring statistical data; the chrome browser module comprises a webrtc-interpnals module and a webpage rendering and displaying module, and the webrtc library comprises a data interaction module which is communicated with the webrtc-interpnals module. According to the method for displaying the terminal application program by the Chrome webrtc-interpnals tool, the webrtc-interpnals tool not only can display webrtc statistical data based on a webpage version, but also can display webrtc statistical data integrated with the terminal application program.
Description
Technical Field
The invention relates to a communication data processing method, in particular to a method for displaying a terminal application program by a Chrome webrtc-interpnals tool.
Background
Webrtc is used as a three-party open source library, has a very powerful and perfect statistical data collection mechanism, and when a developer develops an instant messaging application based on Webrtc, the problem can be tracked and positioned based on the existing statistical data of Webrtc.
The Chrome webrtc-intermels tool is mainly used for displaying statistical data in real time during the operation of web-based instant messaging so as to facilitate tracking and positioning of web problems, and the main process is as shown in fig. 1, (1) a tag (tag 1) is opened in a Chrome browser to operate the web-based instant messaging. This is the module for production and collection of webrtc statistics. (2) In the chrome browser, a tab (tab 2) is opened to enter chrome:// webrtc-interlals/run webrtc-interlals tool in the web site bar. In the webpage of the tag 2, statistical data of the Webrtc in the tag 1 during instant messaging is displayed in real time, which is a module for displaying the statistical data of the Webrtc-interpnals, and is mainly obtained by inter-process communication (IPC) with a module in a Webrtc library.
Webrtc is taken as an open-source three-party library, and not only can be called by a webpage-version JavaScript program, but also can be called by a terminal application program in an integrated manner. While the webrtc-interlals tool provided by the chrome can only display webrtc statistical data based on a webpage version, and cannot display webrtc statistical data integrated by a terminal application program.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a method for displaying the terminal application program by using a Chrome webrtc-interlals tool, which can display webrtc statistical data integrated with the terminal application program in webrtc-interlals in real time.
The technical scheme adopted by the invention is as follows:
a method for displaying terminal application programs by a Chrome webrtc-interpnals tool comprises an instant messaging application program module and a Chrome browser module; the instant communication application program module comprises; the system comprises a webrtc library and a statistical data module, wherein the statistical data module is provided with an interface module for acquiring statistical data; the chrome browser module comprises a webrtc-interpnals module and a webpage rendering and displaying module, and the webrtc library comprises a data interaction module which is communicated with the webrtc-interpnals module.
The data interaction module comprises: named pipe A and named pipe B; the named pipeline A is communicated with the statistical data interface module; named pipe B communicates with the webrtc-interpnals module.
The instant messaging application program module calls a webrtc library to perform data processing; the statistical data module in the webrtc can count the statistical data of the webrtc;
after the thread processing modules finish respective work, corresponding statistical data are updated to the objects of the statistical information temporary storage module;
the main thread module periodically updates the statistical information agent module object, and the statistical information agent module object pulls data from the statistical information temporary storage module object and stores the data in the member variable of the statistical information of the main thread module;
the statistical information agent module object acquires a statistical data interface thread calling process to reach the statistical information agent module object, and acquires and returns statistical data; returning to the named pipeline A, communicating the pipeline A with the pipeline B, and transmitting the statistical data to the pipeline B;
the webrtc-internnals tool calls the interface of the named pipeline B regularly to request the statistical data of the webrtc; and after receiving the request, the named pipeline B can communicate with the named pipeline A in the instant messaging application program, and the named pipeline A can call an interface for acquiring statistical data in the webrtc library to acquire the statistical data of the webrtc, return the statistical data to the named pipeline B and finally transmit the statistical data to the chrome browser for rendering and displaying.
And the instant messaging application program module calls the webrtc library to process data, namely audio or video data.
The thread processing module comprises: the device comprises a network module, a decoding module and a rendering display module.
And the main thread module of the periodically updated statistical information agent module object is a video processing main thread module.
And the main thread module of the periodically updated statistical information agent module object is an audio processing main thread module.
Compared with the prior art, the invention has the beneficial effects that:
according to the method for displaying the terminal application program by the Chrome webrtc-interpnals tool, the webrtc-interpnals tool not only can display webrtc statistical data based on a webpage version, but also can display webrtc statistical data integrated with the terminal application program.
Drawings
FIG. 1 is a flow chart of a prior art Chrome webrtc-intermels tool displaying statistics in real time during web-based instant messaging operations;
FIG. 2 is a flow chart of the Chrome webrtc-interlals tool of the present invention displaying statistical data in real time during the operation of instant messaging based on the webrtc web page version;
FIG. 3 is a flow chart of video data update and acquisition according to the present invention;
FIG. 4 is an ebRTC architecture diagram of the method of the Chrome webrtc-interpnals tool display terminal application of the present invention;
FIG. 5 is a frame diagram of the instant messaging between web pages implemented by webrtc of the method for displaying terminal applications by the webrtc-intermenals tool of Chrome in accordance with the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and examples:
fig. 1-5 show that a method for displaying terminal application programs by Chrome webrtc-interpnals tool includes an instant messaging application program module and a Chrome browser module; the instant communication application program module comprises; the system comprises a webrtc library and a statistical data module, wherein the statistical data module is provided with an interface module for acquiring statistical data; the chrome browser module comprises a webrtc-interpnals module and a webpage rendering and displaying module, and the webrtc library comprises a data interaction module which is communicated with the webrtc-interpnals module.
The data interaction module comprises: named pipe A and named pipe B; the named pipeline A is communicated with the statistical data interface module; named pipe B communicates with the webrtc-interpnals module.
The communication modules of named pipelines are respectively integrated into a personal redundancy _ client test program of the webrtc and a webrtc-internals tool of the chrome, and when the webrtc test program is operated, the webrtc-internals test tool is opened through the chrome to display the content of the designated character strings.
The instant messaging application program module calls a webrtc library to perform data processing; the statistical data module in the webrtc can count the statistical data of the webrtc;
the following describes a collecting process of WebRTC collecting video related information, taking an example when an instant messaging application program is in a video call with a remote end.
After the three thread processing modules finish respective work, corresponding statistical data are updated to the object of the statistical information temporary storage module;
the main thread module periodically updates the statistical information agent module object, and the statistical information agent module object pulls data from the statistical information temporary storage module object and stores the data in the member variable of the statistical information of the main thread module;
the statistical information agent module object acquires a statistical data interface thread calling process to reach the statistical information agent module object, and acquires and returns statistical data; returning to the named pipeline A, communicating the pipeline A with the pipeline B, and transmitting the statistical data to the pipeline B;
the working thread, the updating thread and the pulling thread work together in a cooperative mode to complete generation, updating and pulling of the statistical data.
Inputting chrome:// webrtc-interlals/, on the chrome browser;
the webrtc-internnals tool calls the interface of the named pipeline B regularly to request the statistical data of the webrtc; and after receiving the request, the named pipeline B can communicate with the named pipeline A in the instant messaging application program, and the named pipeline A can call an interface for acquiring statistical data in the webrtc library to acquire the statistical data of the webrtc, return the statistical data to the named pipeline B and finally transmit the statistical data to the chrome browser for rendering and displaying.
And the instant messaging application program module calls the webrtc library to process data, namely audio or video data.
The thread processing module comprises: the device comprises a network module, a decoding module and a rendering display module.
And the main thread module of the periodically updated statistical information agent module object is a video processing main thread module.
The main thread module of the periodically updated statistical information agent module object is an audio processing main thread module
According to the technical scheme, the data interaction module with the webrtc-interpnals tool is additionally arranged in the webrtc library, so that the webrtc statistical data integrated by the terminal application program can be displayed in real time in the webrtc-interpnals.
WebRTC, the name of which is derived from the abbreviation of Web Real-Time Communication, is an API that supports Web browsers for Real-Time voice conversations or video conversations.
(1) Purpose of WebRTC
The final purpose of the WebRTC project is mainly to enable a Web developer to easily and quickly develop rich real-time multimedia application based on a browser (Chrome, FireFox and the like), without downloading and installing any plug-in, and the Web developer does not need to pay attention to the digital signal processing process of multimedia and can realize the purpose only by writing a simple Javascript program.
(2) WebRTC architecture diagram
As illustrated in the architectural diagram of fig. 4:
a. a network interface: interface layer for web page developer
Webrtc C + + interface: is an interface layer for browser manufacturers
c. Audio capture/play, video capture, network input/output: is an interface which can be defined by each browser manufacturer
(3) The WebRTC architecture component includes:
a. web page application
The web page developer can develop a real-time communication application based on video and audio based on a web page interface provided by a browser integrated with WebRTC.
b. Network interface
The WebRTC standard API (Javascript) facing to the third-party developer enables the developer to easily develop a web application similar to web video chat.
WebRTC C + + interface
And the local C + + interface layer enables a browser manufacturer to easily realize a WebRTC standard network interface and abstractly process a digital signal process.
(d) Transport/Session (Transport/Session layer)
The session layer component is implemented by adopting a partial component of a libjingle library without using an xmpp/jingle protocol.
● RTP Stack protocol Stack
Real Time Protocol
●P2P STUN/TURN/ICE
P2P is end-to-end, and can establish call connection between different types of networks Through STUN (Simple Traversal of UDP Through NATs), TURN (Traversal of relay Through NAT, i.e., Traversal of relay Through NAT) and ICE (Interactive Connectivity Establishment) components.
● Session Management (Session Management)
An abstract session layer provides session establishment and management functions. The layer protocol is left to the application developer to implement by self.
(5) Audio engine (VoiceEngine)
The audio engine is a framework containing a series of audio multimedia processing, including the whole solution from video capture card to network transmission end.
PS: VoiceEngine is one of the extremely valuable technologies of WebRTC and is open source after Google purchases GIPS. The technical industry is leading in VoIP, and the latter article will be understood in detail.
a.iSAC
Internet Speech Audio Codec
Wideband and ultra-wideband audio codecs for VoIP and audio streams, the default codec for WebRTC audio engine
Sampling frequency: 16khz, 24khz, 32 khz; (default is 16khz)
The self-adaptive rate is 10 kbit/s-52 kbit/;
adaptive packet size: 30-60 ms;
algorithm delay: frame +3ms
b.iLBC
Internet Low Bitrate Codec
Narrowband speech codec for VoIP audio streams
Sampling frequency: 8 khz;
the 20ms frame bit rate is 15.2kbps
The 30ms frame bit rate is 13.33kbps
The standards are defined by IETF RFC3951 and RFC3952
(Audio network equalization) NetEQ for Voice
Voice signal processing element for audio software implementation
NetEQ algorithm: an adaptive jitter control algorithm and a speech packet loss concealment algorithm. The method can quickly adapt to the continuously changing network environment with high resolution, and ensure beautiful tone quality and minimum buffer delay.
PS: the NetEQ is also a valuable technology in WebRTC, has obvious effect on improving the VoIP quality, and is better in effect when being integrated and used by modules such as AEC \ NR \ AGC and the like.
d.Acoustic Echo Canceler(AEC)
The echo canceller is a signal processing element based on software, and can remove echoes collected by mic in real time.
e.Noise Reduction(NR)
Noise suppression is also a software-based signal processing element that eliminates some type of background noise (hiss, fan noise, etc.) associated with the associated VoIP.
(6) Video engine (VideoEngine)
WebRTC video processing engine
The VideoEngine is an integral framework containing a series of video processing, and is a solution for the whole integral process from video acquisition by a camera, video information network transmission and video display.
a.VP8
And the video image codec is the default codec of the WebRTC video engine.
VP8 is suitable for real-time communication application scenarios because it is mainly a codec designed for low latency.
VPx codec was an open source after Google procurement ON2, VPx is now part of the WebM project, one of the HTM L5 standards that Google pursued
b. Video Jitter (Video Jitter Buffer)
The video jitter buffer can reduce adverse effects caused by video jitter and video information packet loss.
c. Image enhancement (Image enhancements)
Image quality enhancement module
The method comprises the steps of processing images collected by the network camera, wherein the functions of shading detection, color enhancement, noise reduction processing and the like are included, and the video quality is improved.
(7) Runtime information for WebRTC
a. Collecting and counting at a sending end: corresponding to the generation of media data, including frame rate, frame size, clock frequency of the media data source, codec name, etc.
b. RTP statistics of a sending end: corresponding to the transmission of media data, including the number of transmitted packets, the number of transmitted bytes, the round trip time RTT, etc.
c. RTP statistics of a receiving end: corresponding to the reception of media data, including the number of received packets, the number of bytes received, the number of discarded packets, the number of lost packets, the network jitter, etc.
d. Rendering statistics at a receiving end: rendering corresponding to the media data includes a drop frame number, a render delay, and so forth.
There are also miscellaneous statistics such as DataChannel metrics, network interface metrics, certificate statistics, and so on. Among many pieces of information, there are some core metrics reflecting the operating state of WebRTC, including round trip time RTT, packet loss rate, and receiving end delay, which are respectively expressed as follows:
round trip time RTT: which represents the time taken for the data to travel over the network, is typically calculated by correlation fields in the SR/RR packets of RTCP. This metric directly reflects how good the network conditions are.
The packet loss rate affects the audio and video quality of a receiving end, and sound jump or video mosaic can be caused under severe conditions, so that the quality of the network condition is reflected from the side.
After the audio/video data arrives at the receiving end, the processes of packet receiving, decoding, rendering and the like are required, and the process brings delay. The receiver-side delay is an important component of the one-way delay of data from acquisition to rendering.
From the above analysis, the WebRTC return information contains statistical information of each stage of the data pipeline, from data acquisition, encoding to transmission, to data reception, decoding, and rendering. This provides first-hand data for monitoring the running state of the WebRTC application.
(8) The frame diagram of instant messaging between web pages is realized by webrtc.
Before starting a video call, the initiator and the receiver need some interaction, such as notifying the other party to start a video, receiving the video, negotiating video parameters (SDP information), and exchanging NAT addresses, which is called signaling interaction, Webrtc does not define a standard signaling format, and can use either sip (session Initiation protocol), xmpp (extensible Messaging presentation protocol), or a customized signaling format, and the simplest way is to use websocket or XM L HttpRequest, and the customized format completes the signaling interaction process.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the structure of the present invention in any way. Any simple modification, equivalent change and modification of the above embodiments according to the technical spirit of the present invention are within the technical scope of the present invention.
Claims (5)
1. A method for displaying terminal application programs by a Chrome webrtc-interpnals tool comprises an instant messaging application program module and a Chrome browser module; the instant communication application program module comprises; the system comprises a webrtc library and a statistical data module, wherein the statistical data module is provided with an interface module for acquiring statistical data; the chrome browser module comprises a webrtc-interlals module and a webpage rendering and displaying module, and is characterized in that the webrtc library comprises a data interaction module which is communicated with the webrtc-interlals module;
the data interaction module comprises: named pipe A and named pipe B; the named pipeline A is communicated with the statistical data interface module; the named pipeline B is communicated with the webrtc-interpnals module;
the instant messaging application program module calls a webrtc library to perform data processing; the statistical data module in the webrtc can count the statistical data of the webrtc;
after the thread processing modules finish respective work, corresponding statistical data are updated to the objects of the statistical information temporary storage module;
the main thread module periodically updates the statistical information agent module object, and the statistical information agent module object pulls data from the statistical information temporary storage module object and stores the data in the member variable of the statistical information of the main thread module;
the statistical information agent module object acquires a statistical data interface thread calling process to reach the statistical information agent module object, and acquires and returns statistical data; returning to the named pipeline A, communicating the pipeline A with the pipeline B, and transmitting the statistical data to the pipeline B;
the webrtc-internnals tool calls the interface of the named pipeline B regularly to request the statistical data of the webrtc; and after receiving the request, the named pipeline B can communicate with the named pipeline A in the instant messaging application program, and the named pipeline A can call an interface for acquiring statistical data in the webrtc library to acquire the statistical data of the webrtc, return the statistical data to the named pipeline B and finally transmit the statistical data to the chrome browser for rendering and displaying.
2. The Chrome webrtc-interpnals tool of claim 1, wherein: and the instant messaging application program module calls the webrtc library to process data, namely audio or video data.
3. The Chrome webrtc-interpnals tool of claim 1, wherein: the thread processing module comprises: the device comprises a network module, a decoding module and a rendering display module.
4. The Chrome webrtc-interpnals tool of claim 1, wherein: and the main thread module of the periodically updated statistical information agent module object is a video processing main thread module.
5. The Chrome webrtc-interpnals tool of claim 1, wherein: and the main thread module of the periodically updated statistical information agent module object is an audio processing main thread module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710350201.8A CN107168856B (en) | 2017-05-18 | 2017-05-18 | Method for displaying terminal application program by Chrome webrtc-interpnals tool |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710350201.8A CN107168856B (en) | 2017-05-18 | 2017-05-18 | Method for displaying terminal application program by Chrome webrtc-interpnals tool |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107168856A CN107168856A (en) | 2017-09-15 |
| CN107168856B true CN107168856B (en) | 2020-07-10 |
Family
ID=59815256
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710350201.8A Active CN107168856B (en) | 2017-05-18 | 2017-05-18 | Method for displaying terminal application program by Chrome webrtc-interpnals tool |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107168856B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108173863B (en) * | 2017-12-29 | 2021-08-17 | 深圳市泛海三江科技发展有限公司 | Method and system for establishing lightweight WebRTC system suitable for Internet of things equipment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104284324A (en) * | 2013-07-05 | 2015-01-14 | 思科技术公司 | Integrated signaling between mobile data networks and enterprise networks |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9552338B2 (en) * | 2013-03-11 | 2017-01-24 | Futurewei Technologies, Inc. | Mechanisms to compose, execute, save, and retrieve hyperlink pipelines in web browsers |
| CN105099897B (en) * | 2014-05-13 | 2020-01-21 | 中兴通讯股份有限公司 | Method and gateway for communication between browser and telecommunication network |
-
2017
- 2017-05-18 CN CN201710350201.8A patent/CN107168856B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104284324A (en) * | 2013-07-05 | 2015-01-14 | 思科技术公司 | Integrated signaling between mobile data networks and enterprise networks |
Non-Patent Citations (1)
| Title |
|---|
| "Kurento 之 WebRTC 统计";网友;《http://ju.outofmemory.cn/entry/301421》;20170217;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107168856A (en) | 2017-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Singh et al. | VoIP: State of art for global connectivity—A critical review | |
| US8311038B2 (en) | Instant internet browser based VoIP system | |
| US20070192427A1 (en) | Ease of use feature for audio communications within chat conferences | |
| Milner et al. | Robust speech recognition over IP networks | |
| US20100058409A1 (en) | Audio and video testing methodology | |
| US9246545B1 (en) | Adaptive estimation of delay in audio systems | |
| KR102580418B1 (en) | Acoustic echo cancelling apparatus and method | |
| US7408884B2 (en) | Methods and devices for correlating audio sample comparisons and network performance statistics | |
| JP2015532019A (en) | User interaction monitoring for adaptive real-time communication | |
| WO2016096339A1 (en) | Delay estimation for echo cancellation using ultrasonic markers | |
| US9014175B2 (en) | Routing of voice internet protocol packets to a selected processor | |
| US9219456B1 (en) | Correcting clock drift via embedded sin waves | |
| WO2012079510A1 (en) | Mute indication method and device applied to video conferencing | |
| US10204634B2 (en) | Distributed suppression or enhancement of audio features | |
| CN107168856B (en) | Method for displaying terminal application program by Chrome webrtc-interpnals tool | |
| CN113709581A (en) | System, method and device for audio and video real-time transmission and electronic equipment | |
| CN113727185A (en) | Video frame playing method and system | |
| WO2019100475A1 (en) | Data transmission method, apparatus and device, and storage medium | |
| US20080037514A1 (en) | Method, system, and computer program product for controlling a voice over internet protocol (voip) communication session | |
| CN103325385B (en) | Voice communication method and equipment, the method and apparatus of operation wobble buffer | |
| US11100930B1 (en) | Avoiding false trigger of wake word from remote device during call | |
| TWI435589B (en) | Voip integrating system and method thereof | |
| Valin et al. | Requirements for an Internet Audio Codec | |
| US9059860B2 (en) | Techniques for announcing conference attendance changes in multiple languages | |
| CN113572898B (en) | Method and corresponding device for detecting silent abnormality in voice call |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |