CN115987951A - WebRTC performance and stability test method and system - Google Patents

Abstract

The invention provides a method and a system for testing WebRTC performance and stability, which comprise the following steps: a background management module comprising a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule is established at a server side, and a client side provided with a terminal scheduling module and a media execution module based on the WebRTC technology is recorded and deployed through the terminal management submodule; the task issuing sub-module creates a test task and issues the test task to a specified client, and a media execution module is called to execute the test task after a terminal scheduling module of the client configures relevant parameters; the exception reporting sub-module receives the exception execution condition fed back by the media execution module in the process of executing the test task and monitors the exception service condition fed back by the service end in the process of running the service end in real time. The invention can realize the simulation of a large-format conference and the performance and stability test of the client and the server in the conference process.

Description

WebRTC performance and stability test method and system

Technical Field

The invention relates to the technical field of real-time communication, in particular to a method and a system for testing WebRTC performance and stability.

Background

With the rapid development and scientific and technological progress of the internet, the popularization of the video conference system has slowly permeated into various industries, so that the development of the video conference system reaches a new height and is generally applied to the industries of the internet, education and the like. For testers, testing of video conferences will become a key research direction in the current testing field. The core index for measuring the depth of the audio and video technical ability is how many users can be supported by the audio and video conference. WebRTC (Web Real-Time Communications, web page Real-Time communication) is used as a Real-Time communication technology, provides a core technology of a video conference, is a mainstream implementation scheme in the industry, but at present, an efficient performance verification scheme is lacked for service performance evaluation, most of the methods are still mainly manual, and part of the methods may be called to cloud equipment for batch testing, and the following defects generally exist:

the first disadvantage is that a large number of conference simulation is carried out manually, and more equipment resources are required to be occupied. For example, if a 100-person conference is simulated, 100 persons need to be added, and the conference is performed through real devices, the conference needs to be supported by 100 devices, and generally, the test resources allocated to each tester are limited, and thus, the conference scene verification of this magnitude cannot be supported.

And secondly, even if enough equipment resources are provided, different combination scenes need to be simulated in a large-format conference, and if the combination scenes are verified in a manual mode, the process needs to be implemented with high labor input.

The third disadvantage is that for streaming media related services, it is inevitable that multiple services are required to perform cascade cooperation, and the reasonability of service performance distribution in the process needs to be concerned, but no specific scheme about elastic service performance pressure measurement is provided in the industry.

The disadvantage is that in some conferences with a large number of parties, some extreme critical values are required to be simulated, for example, in a few seconds, a plurality of users are added to the conference instantly, even hundreds of users are added to the conference, and in such a scene, one or two testers can not complete the simulation easily.

Disclosure of Invention

The technical problem to be solved by the invention is as follows: the WebRTC performance and stability testing method and system are provided, and performance and stability testing of a large-number-of-parties conference is effectively achieved.

In order to solve the technical problems, the invention adopts the technical scheme that:

a method for testing performance and stability of WebRTC comprises the following steps:

s1, a background management module comprising a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule is established at a server side, and a client side provided with a terminal scheduling module and a media execution module based on a WebRTC technology is recorded and deployed through the terminal management submodule;

s2, the task issuing submodule creates a test task and issues the test task to the specified client, and the media execution module is called to execute the test task after the terminal scheduling module of the client configures relevant parameters;

and S3, the abnormal reporting submodule receives the abnormal execution condition fed back by the media execution module in the process of executing the test task and the abnormal service condition fed back by the performance detection submodule in the process of monitoring the operation of the server in real time.

In order to solve the technical problem, the invention adopts another technical scheme as follows:

a test system for WebRTC performance and stability comprises a background management module, a terminal scheduling module and a media execution module;

the background management module is deployed at a server side, and the terminal scheduling module and the media execution module are deployed at a client side;

the background management module comprises a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule;

the terminal management submodule is used for recording a client which is provided with a terminal scheduling module and a media execution module based on the WebRTC technology;

the task issuing submodule is used for creating a test task and issuing the test task to the specified client;

the terminal scheduling module is used for receiving the test task, configuring relevant parameters and then calling the media execution module to execute the test task;

the media execution module is used for executing the test task and feeding back an abnormal execution condition to the abnormal reporting submodule;

the performance detection submodule is used for monitoring the operation process of the server in real time and feeding back an abnormal service condition to the abnormal reporting submodule;

the exception reporting sub-module is configured to receive an exception execution condition fed back by the media execution module and an exception service condition fed back by the performance detection sub-module.

The invention has the beneficial effects that: the invention provides a method and a system for testing WebRTC performance and stability, wherein a background management module comprising a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule is established at a server, a terminal scheduling module and a WebRTC technology-based media execution module are deployed at a client, a test task is issued to a specified client by the task issuing submodule, and then the media execution module executes after parameters are configured by the terminal scheduling module of the client to realize the situation that a single end simulates a multi-user to participate in a conference, and simultaneously, exceptions appearing in the execution process are fed back and are matched with the performance detection module of the server to monitor the operation performance of the server in real time, so that the simulation of a large-square-number conference and the performance and stability tests of the client and the server in the conference process are integrally realized.

Drawings

Fig. 1 is an overall flowchart of a WebRTC performance and stability testing method according to an embodiment of the present invention;

fig. 2 is a flowchart of a task issuing sub-module in a test method for WebRTC performance and stability according to an embodiment of the present invention;

fig. 3 is a flowchart of a terminal scheduling module in a method for testing WebRTC performance and stability according to an embodiment of the present invention;

fig. 4 is a flowchart illustrating a media execution module in a method for testing WebRTC performance and stability according to an embodiment of the present invention;

fig. 5 is a flowchart of the operation of a performance detection sub-module in the test method for WebRTC performance and stability according to the embodiment of the present invention;

fig. 6 is an overall architecture diagram of a test system for WebRTC performance and stability according to an embodiment of the present invention.

Description of reference numerals:

1. a test system for WebRTC performance and stability;

2. a background management module; 21. a terminal management submodule; 22. a task issuing submodule; 23. a performance detection submodule; 24. an exception reporting submodule;

31. a terminal scheduling module; 32. and a media execution module.

Detailed Description

In order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.

Referring to fig. 1 to 5, a method for testing performance and stability of WebRTC includes the steps of:

s1, a background management module comprising a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule is established at a server side, and a client side provided with a terminal scheduling module and a media execution module based on a WebRTC technology is recorded and deployed through the terminal management submodule;

s2, the task issuing submodule creates a test task and issues the test task to the specified client, and the media execution module is called to execute the test task after the terminal scheduling module of the client configures relevant parameters;

and S3, the abnormal reporting submodule receives the abnormal execution condition fed back by the media execution module in the process of executing the test task and monitors the abnormal service condition fed back by the service terminal in the process of operating the service terminal in real time.

As can be seen from the above description, the beneficial effects of the present invention are: the method comprises the steps that a background management module comprising a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule is established at a server, a terminal scheduling module and a media execution module based on a WebRTC technology are deployed at a client, a test task is issued to a specified client by the task issuing submodule, the media execution module executes parameters after the terminal scheduling module of the client configures the parameters to realize single-end simulation of the situation that multiple users participate in a conference, exceptions occurring in the execution process are fed back at the same time, and the performance detection module of the server is matched to monitor the operation performance of the server in real time, so that the simulation of a large-square-number conference and the performance and stability test of the client and the server in the conference process are integrally realized.

Further, in step S1, the client deployed with the terminal scheduling module is recorded by the terminal management submodule, and specifically includes:

the terminal management submodule creates a terminal list in advance, receives a terminal registration request initiated by the client for deploying the terminal scheduling module for the first time in real time, and adds the client into the terminal list, wherein the terminal registration request comprises an equipment name and an equipment IP.

According to the description, when each client deploys the terminal scheduling module, a terminal registration request is sent to the server to send the device name and the device IP of each client to the terminal management submodule of the server for storage, and terminal registration is completed, so that the terminal management submodule can uniformly manage the clients with the terminal scheduling modules according to the device names and the device IPs, and the clients for executing test tasks can be conveniently and quickly selected subsequently.

Further, the step S1 further includes:

and the terminal management submodule monitors the state of the client regularly and marks the online state and the offline state of each client in the terminal list according to the online state and the offline state of the client.

As can be seen from the above description, the terminal management submodule can quickly query the currently available client according to the client state.

Further, the step S2 specifically includes:

s21, the task issuing sub-module creates a test task and selects configuration parameters to be input, wherein the configuration parameters comprise: the conference system comprises a conference environment, the number of people in the conference, the joining type of the conference, the number of rooms in the conference, whether to subscribe the audio and video, whether to push the video, the setting of stream pushing code rate and the setting of the upper limit of the number of people subscribed;

s22, the task issuing sub-module selects one or more online clients from the terminal list, issues a test task to the terminal scheduling module, and the terminal scheduling module analyzes the test task, acquires the selected configuration parameters needing to be input, and configures related parameter values for the configuration parameters;

s23, the terminal scheduling module splices the related parameter values into URL parameter links which can be identified by the media execution module, and calls a Chrome process to open a program site of the media execution module;

and S24, the media execution module enters the program site according to the URL parameter link, the program site analyzes the URL parameter link correspondingly, corresponding behaviors of the analyzed corresponding parameters are executed correspondingly, and meanwhile, the terminal scheduling module records the PID process of the Chrome process in real time.

As can be seen from the above description, the task issuing sub-module supports single selection and batch selection of available clients to improve the simulation effect of a large-format conference, that is, the media execution module of one client can simulate multiple users, and the media execution modules of multiple clients can simulate more users; meanwhile, when the task issuing sub-module issues the test task, configuration parameters needing to be input are configured for the test task, so that a terminal scheduling module of the client can further configure related parameter values according to the selected configuration parameters, and the simulation effect of a large-square-number conference is improved; in addition, the terminal scheduling module starts the media execution module to perform corresponding conference behavior simulation by calling the Chrome process, namely, the engine mode, so that the execution efficiency and the stability of the simulated conference are effectively improved.

Further, the step S3 is followed by the step of:

and S4, after the media execution module finishes executing the test task, the task issuing sub-module issues a task release instruction to the terminal scheduling module, and the terminal scheduling module releases the corresponding Chrome process according to the recorded PID process and closes the media execution module.

As can be seen from the above description, the terminal scheduling module closes the media execution module in a process release manner, thereby ensuring complete release of the test task and reducing memory occupation of the client.

Further, the abnormal execution condition comprises a code execution error, error type information, equipment information and time of the client;

the abnormal service condition comprises performance abnormity of the server, service instance abnormity and an abnormal time point.

It can be known from the above description that the media execution module feeds back the performance of the client, and the performance detection sub-module feeds back the performance of the server, so as to ensure the comprehensive performance and stability test of a large number of simulated conferences.

Further, the step S3 further includes:

if the abnormal execution condition or the abnormal service condition is newly added content, newly adding an error type in the abnormal reporting submodule to record the abnormal execution condition or the abnormal service condition, and counting the number of the error types to be 1;

and if the abnormal execution condition or the abnormal service condition is the content which exists in the history in the abnormal reporting submodule, the count of the corresponding condition record is +1.

As can be seen from the above description, if the abnormal execution condition or the abnormal service condition is the same as the abnormal condition in the history, the abnormal execution condition or the abnormal service condition does not need to be recorded again, and only the count of the abnormal condition needs to be increased, so as to reduce the memory usage of the server.

Further, the performance detection submodule in step S3 monitors the operation process of the server in real time, specifically:

the performance detection submodule monitors whether the CPU performance, the memory size, the disk size and the bandwidth size of the server side, the task number and the service instance number of the task issuing submodule exceed preset thresholds or not in real time.

According to the description, the performance detection module monitors whether various performance parameters of the server exceed the threshold value in real time, and ensures the operation stability of the server.

Further, the step S3 further includes:

and the performance detection submodule exports the monitored performance data of the server in the operation process into a performance report.

As can be seen from the above description, the performance detection module further supports exporting reports of the monitored performance data of the server, so as to further provide for specific performance analysis.

Referring to fig. 6, a WebRTC performance and stability testing system includes a background management module, a terminal scheduling module, and a media execution module;

the background management module is deployed at a server side, and the terminal scheduling module and the media execution module are deployed at a client side;

the background management module comprises a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule;

the terminal management submodule is used for recording a client which is provided with a terminal scheduling module and a media execution module based on the WebRTC technology;

the task issuing submodule is used for creating a test task and issuing the test task to the specified client;

the terminal scheduling module is used for receiving the test task, configuring relevant parameters and calling the media execution module to execute the test task;

the media execution module is used for executing the test task and feeding back an abnormal execution condition to the abnormal reporting submodule;

the performance detection submodule is used for monitoring the operation process of the server in real time and feeding back an abnormal service condition to the abnormal reporting submodule;

the exception reporting sub-module is configured to receive an exception execution condition fed back by the media execution module and an exception service condition fed back by the performance detection sub-module.

As can be seen from the above description, the beneficial effects of the present invention are: based on the same technical concept, a WebRTC performance and stability test system is provided, a background management module comprising a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule is established at a server, a terminal scheduling module and a WebRTC technology-based media execution module are deployed at a client, a test task is issued to a specified client by the task issuing submodule, and then parameters are configured by the terminal scheduling module of the client and then executed by the media execution module to realize the situation that a single end simulates a multi-user to participate in a conference, and simultaneously, the exception occurring in the execution process is fed back and is matched with the performance detection module of the server to monitor the operation performance of the server in real time, so that the simulation of a large-square-number conference and the performance and stability test of the client and the server in the conference process are integrally realized.

The method and the system for testing the performance and the stability of the WebRTC provided by the invention realize the test of the operation performance and the stability of the server and the client by simulating a real-time video conference scene with a large number of parties, and are specifically described in combination with the embodiment below.

Referring to fig. 1, a first embodiment of the present invention is:

a method for testing performance and stability of WebRTC (web real time communication) is shown in figure 1 and comprises the following steps:

s1, a background management module comprising a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule is established at a server side, and a client side provided with a terminal scheduling module and a media execution module based on a WebRTC technology is recorded and deployed through the terminal management submodule.

The terminal management submodule is mainly responsible for client side, namely registration, keep-alive, query and the like of the terminal; the task issuing sub-module is mainly responsible for creating test tasks and executing batch and increment issuing of the test tasks to the client; the terminal scheduling module is responsible for receiving a test task issued by the server, configuring corresponding parameter values, opening a media execution module tool to perform test execution on the test task, releasing an execution process of the media execution module when the test task is finished, closing the media execution module, wherein in the whole execution process of the test task, the media execution module can detect the abnormity of the execution process and report the abnormal execution condition to the abnormity reporting sub-module of the server for statistics, and meanwhile, the performance detection module of the server can monitor the performance change data of the server in real time, mark the abnormal data and report the abnormal data to the abnormity reporting sub-module for statistics. In this embodiment, the test task may be understood as a simulation task of a real-time communication conference.

In this embodiment, the step S1 of recording, by the terminal management submodule, the client to which the terminal scheduling module is deployed includes:

the terminal management sub-module creates a terminal list in advance, receives a terminal registration request initiated by a client for deploying the terminal scheduling module for the first time in real time, and adds the client to the terminal list, wherein the terminal registration request comprises an equipment name and an equipment IP.

That is, in this embodiment, when each client deploys the terminal scheduling module, a terminal registration request is initiated to the server to send the device name and the device IP of each client to the terminal management sub-module of the server for storage, and the terminal registration is completed, where the device IP is used as a unique identifier, thereby ensuring the uniqueness of each client; therefore, the terminal management submodule can uniformly manage the client terminal with the terminal scheduling module according to the equipment name and the equipment IP, and is convenient for quickly selecting the client terminal for executing the test task in the follow-up process.

After the client requests for successful registration, the background of the server can search for the newly added corresponding client by inquiring the terminal list in the terminal management submodule.

Meanwhile, in this embodiment, step S1 further includes:

and the terminal management submodule monitors the state of the client regularly and marks the online state and the offline state of each client in the terminal list according to the online state and the offline state of the client.

Namely, the background of the server accesses the terminal scheduling module of each client regularly, and if the background of the server cannot access the terminal scheduling module, the state of the client is set to be an offline state in a terminal list; when the client is restarted, namely the terminal scheduling module can access, a starting signal can be returned to the background of the server through the terminal scheduling module to inform the terminal management submodule to restore the state of the client from an off-line state to an on-line state, and the currently available, namely on-line client can be quickly inquired by marking the state of each client in the terminal list.

In addition, it should be noted that, in this embodiment, the priority of each client in the terminal list supports customization, that is, the call priority of each client can be adjusted according to actual needs.

The specific test procedure of this embodiment is as follows:

and S2, the task issuing sub-module creates a test task and issues the test task to a specified client, and the terminal scheduling module of the client configures the relevant parameters and then calls the media execution module to execute the test task.

And S3, the abnormal reporting submodule receives the abnormal execution condition fed back by the media execution module in the process of executing the test task and the abnormal service condition fed back by the performance detection submodule in the process of monitoring the operation of the server in real time.

In this embodiment, a background management module including a terminal management sub-module, a task issuing sub-module, an exception reporting sub-module, and a performance detection sub-module is created at a server, a terminal scheduling module and a media execution module based on the WebRTC technology are deployed at a client, a test task is issued to a specified client by the task issuing sub-module, and then the client is executed by the media execution module after parameters are configured by the terminal scheduling module of the client to realize single-ended simulation of a situation that multiple users participate in a conference, and simultaneously, exceptions occurring in the execution process are fed back and the performance detection module of the server is matched to monitor the operation performance of the server in real time, so that simulation of a large-number conference and performance and stability tests of the client and the server in the conference process are integrally realized.

Referring to fig. 2 to 4, a second embodiment of the present invention is:

on the basis of the first embodiment, in this implementation, the step S2 specifically includes:

s21, the task issuing sub-module creates a test task and selects configuration parameters to be input, wherein the configuration parameters comprise: the conference system comprises a conference environment, the number of people in the conference, the joining type of the conference, the number of rooms in the conference, whether to subscribe audio and video, whether to push video, a push stream code rate setting and a number of people subscribing upper limit setting.

S22, the task issuing sub-module selects one or more online clients from the terminal list, issues the test tasks to the terminal scheduling module, and the terminal scheduling module analyzes the test tasks, acquires the selected configuration parameters needing to be input, and configures the relevant parameter values for the configuration parameters.

It should be noted that the task issuing sub-module may first create the test task and perform input configuration parameter selection on the test task, and then select the client to issue the test task, or may first select the client, then create the test task and perform input configuration parameter selection on the test task, and then issue the test task to the client, which is not limited herein. In this embodiment, the task issuing sub-module supports single selection and batch selection of the clients, which can improve the simulation effect of a large-format conference, that is, the media execution module of one client can simulate multiple users, and the media execution modules of multiple clients can simulate more users.

Meanwhile, in this embodiment, after the task issuing sub-module has selected the configuration parameters that need to be input, the interval time of the session may also be configured, that is, after the multiple clients are selected, the interval time of each client receiving the test task may be configured, and the configuration may be performed in units of seconds. And then, the configured test task is issued to the selected client, the selected configuration parameters needing to be input are received and analyzed and obtained by a terminal scheduling module of the client, and relevant parameter values are input in the configuration parameters according to task requirements.

Meanwhile, in the task issuing sub-module, after the test task is executed on the selected client, clicking operation can be executed on a background page of the server to finish the test task on the corresponding client, and then the execution program on the corresponding client can be released; and when a plurality of clients are provided, the test can be finished by directly clicking, all the execution programs of the corresponding clients are finished in batch, and the notification of finishing the task can be issued to the corresponding clients by the terminal management submodule. The work flow of the whole task issuing submodule can be shown in fig. 2.

In this embodiment, the terminal scheduling module, in addition to initiating a terminal registration request to the server when deployed to the client, further includes an independently executable interface service, and the mainly provided external interface includes an online state request interface, a task issuing execution interface, a task ending issuing interface and a performance query structure, where these interfaces are standard restful interfaces, where the online state request interface requests the terminal management submodule of the server for an online state again when the client is restarted, and the task issuing execution interface issues a test task instruction to the corresponding media execution module for testing after receiving a test task sent by the task issuing submodule; the task ending issuing interface feeds back a task ending notice to the task issuing submodule after the media execution module finishes testing, and receives a notice of ending the task issued by the terminal management submodule, and the performance query interface receives the performance index change condition of the equipment where the media execution module is located in the process of executing the testing task.

And S23, the terminal scheduling module splices the related parameter values into URL parameter links which can be identified by the media execution module, namely, the parameters are started, and a Chrome process is called to open a program site of the media execution module.

And S24, the media execution module enters a program site according to the URL parameter link, the program site analyzes the corresponding URL parameter link, corresponding behavior execution is carried out on the analyzed corresponding parameter, and meanwhile, the terminal scheduling module records the PID process of the Chrome process in real time.

It is worth to be noted that, the media execution module is a front-end site written by JavaScript, mainly takes charge of a pressure measurement implementation tool of media services, is a simulation program combining an interface of WebRTC and a custom service parameter continuation, supports simulation of interfaces related to WebRTC, supports establishment of a conference, conference joining, subscription of audio/video streams, push of stream code rate setting and subscription of configuration of upper limit of large and small streams; in the aspect of behavior execution, the media execution module supports service configuration including but not limited to environments (such as different environments of test, pre-production, production and the like), tenants (parameters of data isolation); in the aspect of environment selection, the media execution module initializes the API interface capability related to the operation of WebRTC into an object list, and can support the user-defined multiple objects to simultaneously configure the same behavior execution and environment selection.

When the media execution module receives a URL parameter link input by the terminal scheduling module on the Chrome process, the media execution module acquires a value corresponding to the parameter, performs corresponding behavior execution according to the corresponding parameter, and introduces an actual case, such as: a terminal device A is appointed by a background, a pre-production conference room is set to be added, 10 adding personnel are set, the number of the added conference is appointed to be conference ID _ A, a plug flow is set to be True, subscription is True, other fields are not filled (a subscription flow stream is default to subscribe 1 path, other defaults are all subscribed to a small path, namely, when each user adds the conference, 1 path of large video stream, 9 paths of small video streams and audio stream are all subscribed by default), and then the background clicks to issue a task. When the terminal scheduling module on the terminal equipment A receives the instruction issued by the server, the terminal scheduling module processes the instruction into a URL which can be analyzed by the media execution module, and designates a Chrome browser to open a corresponding URL, at this time, 10 users are added into the conference room, and all 10 users open videos and subscribe video streams of other users, and corresponding forwarding pressure is generated on the server in a mode of 1-path major stream and 9-path minor stream. At this time, we can observe the performance impact of a conference of 10 people on each service of the service end, such as: CPU and memory have had corresponding promotion changes.

In this embodiment, the work flow of the media execution module can be as shown in fig. 5.

In this embodiment, the task issuing sub-module supports single selection and batch selection of available clients to improve the simulation effect of a large-square-number conference, that is, the media execution module of one client can simulate multiple users, and the media execution modules of multiple clients can simulate more users; meanwhile, when the task issuing sub-module issues the test task, configuration parameters needing to be input are configured for the test task, so that a terminal scheduling module of the client can further configure related parameter values for the configuration parameters according to the selection, and the simulation effect of a large-square-number conference is improved; in addition, the terminal scheduling module starts the media execution module to perform corresponding conference behavior simulation by calling the Chrome process, namely an engine mode, so that the execution efficiency and the stability of the simulated conference are effectively improved.

In addition, in this embodiment, the method further includes, after step S3:

and S4, after the media execution module finishes executing the test task, the task issuing sub-module issues a task release instruction to the terminal scheduling module, and the terminal scheduling module releases the corresponding Chrome process according to the recorded PID process and closes the media execution module. Namely, the terminal scheduling module closes the media execution module in a process releasing manner, so that the complete release of the test task is ensured, and the memory occupation of the client is reduced. It should be noted that the instruction for releasing the task may be issued by the task issuing sub-module or by the terminal management sub-module, which is not limited herein. Wherein the work flow of the terminal scheduling module is shown in fig. 4.

Meanwhile, in the performance query interface included in the external interface in the middle terminal scheduling module, in this embodiment, the server may also monitor the performance of the system environment where the client is located by calling the performance query interface in the terminal scheduling module, and if the CPU or the memory margin is small, the server may preferentially issue the test task to other clients, so that the test stability may be ensured to the greatest extent.

Referring to fig. 5, a third embodiment of the present invention is:

on the basis of the first embodiment or the second embodiment, in this embodiment, the exception report sub-module is responsible for receiving exception reports of the media execution module and the performance detection sub-module, and performing corresponding data statistics and query.

The media execution module reports abnormal execution conditions including code execution errors, error type information, equipment information and time of the client, namely, the abnormal errors in the execution process of the client program code tend to be statistics of the code errors. The performance detection submodule reports abnormal service conditions including performance abnormality of the server, service instance abnormality and abnormal time points, and is biased to detect operation and maintenance of the server, and pays attention to detecting performance indexes of the server and activity preservation of instances, such as: whether the memory continuously rises, whether a certain water level is broken through, whether the memory overflows and the like; if a certain instance service disappears, the service is very likely to be down, and meanwhile, if the average performance of a plurality of instances of a certain service is not consumed much, and a certain service runs under high load, the load imbalance is very likely to occur.

In other words, in this embodiment, the media execution module mainly feeds back the performance of the client, and the performance detection sub-module mainly feeds back the performance of the server, that is, the anomaly monitoring dimensions of the two modules are different, so that the comprehensive performance and stability test of the large-square analog conference is ensured.

In this embodiment, step S3 further includes:

if the abnormal execution condition or the abnormal service condition is newly increased content, an error type is newly added in the abnormal reporting submodule to record the abnormal execution condition or the abnormal service condition, and the count is 1; and if the abnormal execution condition or the abnormal service condition is the content which exists in the history in the abnormal reporting submodule, the count of the corresponding condition record is +1. That is, if the abnormal execution condition or the abnormal service condition is the same as the abnormal condition in the history record, the abnormal execution condition or the abnormal service condition does not need to be recorded again, and only the count of the abnormal condition needs to be increased, so that the memory occupation of the server can be reduced.

In this embodiment, the performance detection module is a module responsible for detecting the performance condition of the service in real time, and has the capability of querying performance data of related microservices under different cluster environments, and the work flow of the performance detection module is shown in fig. 3;

first, the performance detection module provides environment selection, which can be customized by the user to query different environments, such as: testing, pre-production and production environment;

then, the performance detection module provides two modes of inquiry, wherein one mode is specific single data inquiry and is directly displayed in a data table mode, and the other mode is displayed in a trend graph associated with a time shaft;

next, in the first embodiment, the operation process of the real-time monitoring server by the performance detection submodule in step S3 specifically includes:

the performance detection submodule monitors whether the CPU performance, the memory size, the disk size and the bandwidth size of the server side, the task number and the service instance number of the task issuing submodule exceed a preset threshold value or not in real time.

In this embodiment, the specific performance test may include, but is not limited to, the above-mentioned ones, which not only supports the average calculation of multiple instance indexes of a certain service, but also supports the specific numerical query of a single instance. For example: and setting the Memory of the WebRTC service instance to be 0.85 and the Disk to be 0.85, and reporting to the exception reporting submodule when detecting that the Memory of the WebRTC exceeds 0.85 or the Disk exceeds 0.85.

Namely, the performance detection module monitors whether various performance parameters of the server exceed a threshold value in real time, and ensures the operation stability of the server.

In addition, in this embodiment, step S3 further includes:

and the performance detection submodule exports the monitored performance data of the server in the operation process into a performance report. Namely, the performance detection module also supports the export report of the monitored performance data of the server so as to further provide specific performance analysis.

On the basis, the present embodiment provides the following four application scenarios, which are respectively:

scene one:

the classmate A is responsible for testing one video conference system, and currently, the team only has 3 testers (including the classmate A, and the classmate B and the classmate C for testing). When the initial service belongs to the function realization stage, 3 testers can realize multi-user interactive test of the audio and video conference, along with the improvement of functions, a project party requires a test team to carry out performance verification of a large number of parties, and the program is further optimized by touching the current performance bottleneck. However, the equipment of testers is relatively limited, one mobile phone can only simulate one user, and one PC only has one available camera, so that the verification of 20+ is difficult in view of the current equipment resource condition of a project. The project side requires that at least 100 sides of the simulation can be verified, and the existing resources of the test team obviously cannot support the verification of the scene.

Through the scheme of the embodiment, the student A can easily realize the verification of the user conference effect of the 100 simulating party only by issuing the conference instruction to the computers of the company colleagues for distributed pressure measurement in the background within the spare time period of the PC of the colleagues.

Scene two:

the test classmates a and B are responsible for the testing of a video conference project, and during the functional testing process, many extreme boundary problems occur in the discovery process, such as: (1) Two users join the same conference at the same time, and as a result, the two users cannot see each other; (2) When a plurality of users exit the conference at the same time, other users in the conference are disconnected. Meanwhile, the above problem is extremely difficult to verify when reproduced, because the triggering time is required to be in the order of milliseconds, and for such a problem, even if the test classmates a and B operate simultaneously, it is difficult to trigger the time node. Meanwhile, if a large class meeting is assumed, the simulation is more difficult if the number of people is dozens or even hundreds, but the stability of the limit operation cannot be tested.

Through the scheme of the embodiment, the classmate A only needs to join the specified conference by configuring the N persons, or release the N persons to exit the conference at the same time. By means of automatic program calling, dozens of users can be added in 1S, and even more users can be added when multiple machines are operated. The method can easily reproduce the limit scene which can not be verified by manpower in the test, so that the problem of small probability is changed into the problem of necessity to be verified, and a tester can conveniently verify whether the problem is completely repaired. Meanwhile, the tester can also realize the mixed scenes of frequently entering and exiting a conference, switching on and off videos, switching on and off audios, sending messages and the like by one member through the program, and perform a robustness test for the stability of the system. The stability of the procedure can also be observed during the process, which can contribute to potential problems with adequate excavation systems.

Scene three:

the test classmates A and B are responsible for testing a video conference project, an activity is on-line in project preparation, the user can use the video conference to carry out a large-scale on-line activity, but the project has no bottom on the service performance of the current product. Therefore, the tester can verify that at most a plurality of participants can be supported under the condition of a certain number of people on the platform. In addition, if all people start the video to be on the platform, the number of people who join the conference can be supported at most. When receiving the requirement, a tester is very difficult, the current hand-held test resources are not enough to verify scenes of one hundred persons even if the tester borrows the equipment of other colleagues in a department, even if one hundred pieces of equipment exist, not all persons can cooperate with the tester to verify various scenes at any time, the change condition of a process link cannot be controlled, and the tester cannot know whether the current persons are opened, whether the process is closed or not, and the like.

According to the scheme of the embodiment, the student A only needs to borrow the idle time of the project colleague computer and is provided with the background management module, the terminal scheduling module and the media execution module, each computer can simulate load tests of at least ten users approximately, the student A can issue users to join a conference in a user-defined mode according to the requirements of the student A, and the plug-flow and subscription states of each user when the user joins the conference are defined in the user-defined mode. And the task execution condition in the current meeting and the resource use condition of each current service instance can be concerned according to a performance monitoring module provided by the background, and process data support is provided for the performance test result.

Scene four:

in the third scenario, even if the small test classmates a and the small test classmates B have sufficient device resources to operate, when some devices are abnormal, for example, a short-time black screen occurs and then reconnection is performed, it is easy to ignore the error performance of individual devices in the process because too many test devices can only focus on the performance of the local devices at the same time.

By the scheme of the embodiment, in the process of increasing the number of parties in a conference, if the server side is abnormal, and the terminal is abnormal and has errors such as abnormal black screen and the like, the abnormal reporting module reports to the background, the background performs further statistics, and a tester can know the current operation summary only by inquiring the performance report of the background.

Referring to fig. 6, a fourth embodiment of the present invention is:

a WebRTC performance and stability testing system, as shown in fig. 6, includes a background management module, a terminal scheduling module, and a media execution module.

In this embodiment, the background management module is deployed at the server, and the terminal scheduling module and the media execution module are deployed at the client; the background management module comprises a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule.

In this embodiment, the terminal management submodule is configured to record a client on which a terminal scheduling module and a WebRTC technology-based media execution module are deployed; the task issuing submodule is used for creating a test task and issuing the test task to a specified client; the terminal scheduling module is used for receiving the test task, configuring relevant parameters and calling the media execution module to execute the test task; the media execution module is used for executing the test task and feeding back the abnormal execution condition to the abnormal reporting submodule; the performance detection submodule is used for monitoring the operation process of the server in real time and feeding back the abnormal service condition to the abnormal reporting submodule; and the exception reporting submodule is used for receiving the exception execution condition fed back by the media execution module and the exception service condition fed back by the performance detection submodule.

The method is characterized in that a background management module comprising a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule is established at a server, a terminal scheduling module and a media execution module based on the WebRTC technology are deployed at a client, a test task is issued to a specified client by the task issuing submodule, and then the media execution module executes after parameters are configured by the terminal scheduling module of the client to realize the simulation of a single-end conference as a situation that multiple users participate in the conference, simultaneously feed back exceptions occurring in the execution process and cooperate with the performance detection module of the server to monitor the operation performance of the server in real time, so that the simulation of a large-number conference and the performance and stability test of the client and the server in the conference process are integrally realized.

In summary, the test method and system for WebRTC performance and stability provided by the present invention have the following beneficial effects:

1. the invention can realize the test of the simulation video conference, and the tester can realize that a single person simulates hundreds of users to participate in the conference and observes the performance conditions of the client and the server under the large square conference;

2. by using the invention to perform the simulation video conference test, the tester can realize the real-time service performance monitoring and provide quantitative data for the service performance optimization;

3. the invention can simulate the video conference test, can realize the simultaneous operation of a plurality of conferences and can simulate a scene test of a plurality of conferences and users.

4. Through the Exception reporting function, the related Exception condition Exception thrown by the system in the whole conference process can be observed, and the basis for program optimization is provided conveniently.

5. Through the configuration of the relevant parameters of the test tasks, the scene which is more in line with the application of the actual process can be realized, and more comprehensive simulation evaluation is provided for the conference test.

6. By the method of adding the meetings in batches, high concurrent meetings can be simulated, rapid operation verification is provided for stability of the server and the client, and the embarrassment that manual operation is difficult to simulate is overcome.

7. The invention can be used for carrying out the automatic test of the simulation video conference and carrying out the cases such as playback, batch conference quit, violent conference in and out and the like, thereby being beneficial to the further robustness verification of the service.

The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent changes made by using the contents of the present specification and the drawings, or applied directly or indirectly to the related technical fields, are included in the scope of the present invention.

Claims (10)

1. A method for testing performance and stability of WebRTC is characterized by comprising the following steps:

s1, a background management module comprising a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule is established at a server side, and a client side provided with a terminal scheduling module and a WebRTC technology-based media execution module is recorded and deployed through the terminal management submodule;

s2, the task issuing submodule creates a test task and issues the test task to the specified client, and the media execution module is called to execute the test task after the terminal scheduling module of the client configures relevant parameters;

and S3, the abnormal reporting submodule receives the abnormal execution condition fed back by the media execution module in the process of executing the test task and the abnormal service condition fed back by the performance detection submodule in the process of monitoring the operation of the server in real time.

2. The method for testing the performance and stability of the WebRTC according to claim 1, wherein in the step S1, the client deployed with the terminal scheduling module is recorded by the terminal management sub-module, which specifically includes:

the terminal management submodule creates a terminal list in advance, receives a terminal registration request initiated by the client for deploying the terminal scheduling module for the first time in real time, and adds the client to the terminal list, wherein the terminal registration request comprises an equipment name and an equipment IP.

3. The method for testing the performance and stability of WebRTC according to claim 2, wherein the step S1 further comprises:

and the terminal management submodule monitors the state of the client regularly and marks the online state and the offline state of each client in the terminal list according to the online state and the offline state of the client.

4. The method for testing the performance and stability of the WebRTC according to claim 3, wherein the step S2 specifically comprises:

s21, the task issuing sub-module creates a test task and selects configuration parameters to be input, wherein the configuration parameters comprise: the conference system comprises a conference environment, the number of people in the conference, the joining type of the conference, the number of rooms in the conference, whether to subscribe audio and video, whether to push video, a stream pushing code rate setting and a number of people subscribing upper limit setting;

s22, the task issuing sub-module selects one or more online clients from the terminal list, issues a test task to the terminal scheduling module, and the terminal scheduling module analyzes the test task, acquires the selected configuration parameters needing to be input, and configures related parameter values for the configuration parameters;

s23, the terminal scheduling module splices the related parameter values into URL parameter links which can be identified by the media execution module, and calls a Chrome process to open a program site of the media execution module;

and S24, the media execution module enters the program site according to the URL parameter link, the program site analyzes the URL parameter link correspondingly, corresponding behaviors of the analyzed corresponding parameters are executed correspondingly, and meanwhile, the terminal scheduling module records the PID process of the Chrome process in real time.

5. The method for testing the performance and stability of WebRTC according to claim 4, wherein the step S3 is followed by the steps of:

and S4, after the media execution module finishes executing the test task, the task issuing sub-module issues a task release instruction to the terminal scheduling module, and the terminal scheduling module releases the corresponding Chrome process according to the recorded PID process and closes the media execution module.

6. The method for testing the performance and stability of the WebRTC according to claim 1, wherein the abnormal execution condition includes a code execution error, error type information, device information and time of the client;

the abnormal service condition comprises performance abnormality, service instance abnormality and an abnormal time point of the server.

7. The method for testing the performance and stability of WebRTC according to claim 1, wherein the step S3 further comprises:

if the abnormal execution condition or the abnormal service condition is newly added content, newly adding an error type in the abnormal reporting submodule to record the abnormal execution condition or the abnormal service condition, and counting the number of the error types to be 1;

and if the abnormal execution condition or the abnormal service condition is the content which exists in the history in the abnormal reporting submodule, the count of the corresponding condition record is +1.

8. The method for testing the performance and stability of the WebRTC according to claim 1, wherein the performance detection submodule in step S3 monitors the operation process of the server in real time, specifically:

the performance detection submodule monitors whether the CPU performance, the memory size, the disk size and the bandwidth size of the server side, the task number and the service instance number of the task issuing submodule exceed preset thresholds or not in real time.

9. The method for testing the performance and stability of WebRTC according to claim 1, wherein the step S3 further comprises:

and the performance detection submodule exports the monitored performance data of the server in the operation process into a performance report.

10. A test system for WebRTC performance and stability is characterized by comprising a background management module, a terminal scheduling module and a media execution module;

the background management module is deployed at a server side, and the terminal scheduling module and the media execution module are deployed at a client side;

the background management module comprises a terminal management submodule, a task issuing submodule, an exception reporting submodule and a performance detection submodule;

the terminal management submodule is used for recording a client which is provided with a terminal scheduling module and a media execution module based on the WebRTC technology;

the task issuing submodule is used for creating a test task and issuing the test task to the specified client;

the terminal scheduling module is used for receiving the test task, configuring relevant parameters and calling the media execution module to execute the test task;

the media execution module is used for executing the test task and feeding back an abnormal execution condition to the abnormal reporting submodule;

the performance detection submodule is used for monitoring the operation process of the server in real time and feeding back an abnormal service condition to the abnormal reporting submodule;

the exception reporting sub-module is configured to receive an exception execution condition fed back by the media execution module and an exception service condition fed back by the performance detection sub-module.

Images