CN114816968B

CN114816968B - Application test method, device, electronic device and storage medium

Info

Publication number: CN114816968B
Application number: CN202110069107.1A
Authority: CN
Inventors: 李书赫; 许景禧
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-01-19
Filing date: 2021-01-19
Publication date: 2025-07-15
Anticipated expiration: 2041-01-19
Also published as: CN114816968A

Abstract

The present application provides a method, device, electronic device and storage medium for application testing. The method for application testing includes: when a target application runs on a first test network, obtaining to-be-processed network status parameters generated by the first test network; using a sliding window to sample and process the to-be-processed network status parameters to obtain M groups of network status parameters, wherein each group of network status parameters in the M groups of network status parameters includes a characteristic value corresponding to a network attribute, and M is an integer greater than 1; obtaining N groups of network status parameters from the M groups of network status parameters, wherein the N groups of network status parameters are parameters associated with a network abnormality state of the first test network, and N is an integer greater than or equal to 1 and less than M; generating a second test network according to the characteristic values included in the N groups of network status parameters; and controlling the target application to run in the second test network to obtain a test result of the target application.

Description

Application testing method, device, electronic equipment and storage medium

Technical Field

The present application relates to the field of computers and communications technologies, and in particular, to an application testing method, an application testing device, an electronic device, and a storage medium.

Background

In the internet, network environments in different areas and time periods are usually different, and in order to be suitable for the network environments in different areas and time periods, an application program needs to be tested and optimized in different network environments, so that the application program in different network environments can obtain good user experience.

In order to test the performance of an application in different network environments, it is often necessary to collect the network environment of a real network and simulate it with a simulated network, and then run the application in the simulated network in order to determine its performance and user experience.

However, the network environment of the real network is complex and the change speed is high, and for the network state of sudden change, the simulation network often has the problem that the dynamic change information is difficult to process, so that the change situation of the real network cannot be accurately simulated, and the test result of the application program is inaccurate.

Disclosure of Invention

Based on the technical problems, the application provides an application testing method to reproduce various complex situations and sudden changes occurring in a real network in a testing network, so that targeted tests can be carried out on target applications aiming at the network situations in the testing network, and the accuracy of application program testing is improved.

Other features and advantages of the application will be apparent from the following detailed description, or may be learned by the practice of the application.

According to an aspect of an embodiment of the present application, there is provided a method for application testing, including:

When a target application runs on a first test network, acquiring a network state parameter to be processed generated by the first test network;

sampling the network state parameters to be processed by adopting a sliding window to obtain M groups of network state parameters, wherein each group of network state parameters in the M groups of network state parameters comprises a characteristic value corresponding to a network attribute, and M is an integer greater than 1;

Acquiring N groups of network state parameters from the M groups of network state parameters, wherein the N groups of network state parameters are parameters related to the network abnormal state of the first test network, and N is an integer which is more than or equal to 1 and less than M;

Generating a second test network according to the characteristic values contained in the N groups of network state parameters;

and controlling the target application to run in the second test network so as to acquire a test result of the target application.

According to an aspect of an embodiment of the present application, there is provided an application testing apparatus including:

the first acquisition module is used for acquiring network state parameters to be processed generated by a first test network when a target application runs on the first test network;

The sampling module is used for sampling the network state parameters to be processed by adopting a sliding window to obtain M groups of network state parameters, wherein the N groups of network state parameters indicate the network abnormal state of the first test network, each group of network state parameters in the M groups of network state parameters comprise characteristic values corresponding to network attributes, and M is an integer greater than 1;

The second acquisition module is used for acquiring N groups of network state parameters from the M groups of network state parameters, wherein N is an integer which is more than or equal to 1 and less than M;

the generation module is used for generating a second test network according to the characteristic values contained in the N groups of network state parameters;

and the control module is used for controlling the target application to run in the second test network so as to acquire a test result of the target application.

In an embodiment of the present application, based on the above technical solution, the second obtaining module may include:

The acquisition unit is further used for acquiring first characteristic values corresponding to the network attributes from the first parameter set and acquiring second characteristic values corresponding to the network attributes from the second parameter set;

The determining unit is further used for determining a characteristic average value corresponding to each network attribute according to the first characteristic value and the second characteristic value;

And the determining unit is further used for determining the characteristic average value corresponding to the network attribute as the N groups of network state parameters.

The clustering unit is further used for clustering the first characteristic value and the second characteristic value to obtain a clustering group and a clustering center value corresponding to the clustering group, wherein the characteristic value in the clustering group is from at least one of the first characteristic value and the second characteristic value;

the determining unit is further used for determining abnormal clustering groups in the clustering groups according to the clustering center value and the abnormal threshold value;

And the determining unit is further used for determining the N groups of network state parameters according to the abnormal clustering group.

In one embodiment of the present application, based on the above technical solution, the generating module may include:

the updating unit is used for obtaining a second test network at the appointed moment according to the characteristic value of a third network state parameter in the N groups of network state parameters when the appointed moment is reached;

and the updating unit is also used for updating the second test network obtained by updating at the previous moment according to the characteristic value of the fourth network state parameter in the N groups of network state parameters at each moment after the appointed moment to obtain an updated second test network.

In one embodiment of the present application, based on the above technical solution, the control module 1005 may include:

The receiving unit is used for receiving the data packet sent by the target application;

The statistics unit is used for counting the data packets to obtain network response data, wherein the network response data comprises at least one of packet loss rate, delay and bandwidth;

And the result generating unit is used for generating a test result corresponding to the target application according to the network response data.

According to an aspect of an embodiment of the present application, there is provided an electronic device including a processor, and a memory for storing executable instructions of the processor, wherein the processor is configured to perform a method of application testing as in the above technical solution via execution of the executable instructions.

According to an aspect of the embodiments of the present application, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of application testing as in the above technical solutions.

According to an aspect of embodiments of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the method of application testing provided in the various alternative implementations described above.

In the embodiment of the application, the acquisition of network state parameters is carried out aiming at the existing network where the target application is located, the specific network state parameters are selected from the network state parameters, the corresponding network state is reappeared in the test network according to the specific network state parameters, and then the test of the target application is carried out in the test network. By the method, various complex conditions and sudden changes which occur in the real network can be reproduced in the test network, so that targeted tests can be carried out on target applications aiming at the network conditions in the test network, and the accuracy of application program tests can be improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is evident that the drawings in the following description are only some embodiments of the present application and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art. In the drawings:

FIG. 1 is a schematic diagram of a composition architecture of an application scenario to which the present application is applied;

FIG. 2 is a flow chart of a method of applying testing in an embodiment of the application;

FIG. 3 is a flow chart of a method of applying testing in an embodiment of the application;

FIG. 4 is a flow chart of a method of applying testing in an embodiment of the application;

FIG. 5 is a flow chart of a method of applying testing in an embodiment of the application;

FIG. 6 is a flow chart of a method of applying testing in an embodiment of the application;

FIG. 7 is a flow chart of a method of applying testing in an embodiment of the application;

FIG. 8 is a flow chart of a method of applying testing in an embodiment of the application;

FIG. 9 is a flow chart of a method of applying testing in an embodiment of the application;

FIG. 10 schematically shows a block diagram of an application test apparatus in an embodiment of the application;

fig. 11 shows a schematic diagram of a computer system suitable for use in implementing an embodiment of the application.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be embodied in many different forms and should not be construed as limited to the examples set forth herein, but rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.

The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.

The scheme of the application is suitable for testing the performance of the application program under abnormal or abrupt specific network conditions so as to improve the accuracy of the application program test. The network state parameter mentioned in the application can be the network state data of the software products such as application programs and the like when running in the network, and the state in the real network can be reproduced in the test network by collecting, analyzing and simulating the network state data, so that the software products are tested, and the problems of poor performance, frequent faults and the like in the software products can be found technically.

The scheme of the application can be applied to a personal computer, a server or a server system formed by a plurality of servers, and in order to improve the efficiency of determining the application program test, the scheme of the application can also be applied to a cloud platform or other computing systems.

For easy understanding, the scenario that the scheme of the application is applied to the cloud platform is taken as an example for explanation. Referring to fig. 1, fig. 1 is a schematic diagram of a composition architecture of an application scenario to which the present application is applicable.

As can be seen from fig. 1, the scenario includes an application client 110, a first test network 120, a cloud platform 130, a second test network 140, and a test client 150.

The application client 110 specifically includes a plurality of clients in which a target application program to be tested can be run. Accordingly, the test client 150 may also include a plurality of clients, and the same target application as the application client 110 is executed in the clients. The target application program in the application client 110 runs in the network environment of the first test network 120, and in general, the first test network 120 may be an actual application environment in which the target application program is located, such as the internet.

The cloud platform 130 comprises the application testing device provided by the application. In the running process of the target application program on the application client 110, the cloud platform 130 may collect various network parameters generated by the first test network by using the application testing device, and analyze and filter the collected network parameters to obtain the network parameters in the specific state of the first test network, for example, in an abnormal state or in a state with a larger load. The cloud platform 130 may then reproduce the network state of the first test network 120 using the second test network 140 and run in the second test network using the test client 150 to test the performance of the target application in that particular network state.

The cloud platform can also be called as a cloud computing platform, and is a network platform constructed based on a cloud technology. Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data.

Cloud technology (Cloud technology) is based on the general terms of network technology, information technology, integration technology, management platform technology, application technology and the like applied by Cloud computing business models, and can form a resource pool, so that the Cloud computing business model is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical network systems require a large amount of computing, storage resources, such as data collection and analysis, and so forth. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.

Cloud computing (clouding) is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to acquire computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the cloud are infinitely expandable in the sense of users, and can be acquired at any time, used as needed, expanded at any time and paid for use as needed.

As a basic capability provider of cloud computing, a cloud computing resource pool (abbreviated as a cloud platform, generally referred to as IaaS (Infrastructure AS A SERVICE) platform) is established, in which multiple types of virtual resources are deployed for external clients to select for use.

In the scenario of fig. 1, the data collection, filtering and analysis of the first test network may be completed by the cloud server on the cloud platform, and the processing of simulating the state of the first test network in the second test network according to the analysis result, and so on.

It can be appreciated that, in fig. 1, taking the example that the cloud platform directly obtains the network state parameters from the first test network, in practical application, the cloud platform may also obtain the network state parameters through other detection devices or through other network paths, which is not limited.

It should be noted that, in fig. 1, for convenience of understanding, an application of the present application to a cloud platform is illustrated by taking an example, and when an application test is performed by using a single computing device, a server or a server cluster, only the cloud platform in the scenario shown in fig. 1 needs to be replaced by a corresponding device or server cluster, and the scenario is similar and will not be repeated here.

The following describes the technical scheme provided by the application in detail by combining the specific embodiments.

As shown in fig. 2, fig. 2 is a flow chart illustrating a method for applying testing in an embodiment of the present application, where the method of the embodiment may be applied to the aforementioned computer device, server, or cloud platform, etc. The method of the embodiment can comprise the following steps:

Step S201, when a target application runs on a first test network, acquiring a to-be-processed network state parameter generated by the first test network.

The network state parameters to be processed can be obtained by means of streaming data, and specifically can be obtained directly by directly detecting the first test network by the server, or collected by a client running the target application program and actively reported to the server, or detected by a detection device of a third party and sent to the server. The network state parameters to be processed may be parameter information representing network communication performance, and each of the network state parameters may represent a network state faced by the target application in the running process.

It will be appreciated that for an application or test network, different network states and network state parameters may occur at different times, and thus the network state parameters to be processed may actually comprise a plurality of network state parameters at a plurality of different times, respectively. For example, the network status parameter to be processed may be a network status report, which contains a plurality of pieces of network status data, which can be regarded as a sequence of network status data generated in chronological order.

It should be noted that, in practical applications, multiple network state parameters may be obtained from multiple different sources at the same time, but the filtering and analysis process for each network state parameter is the same, and may be used for testing the application program in the subsequent steps of the present application.

Step S202, sampling the network state parameters to be processed by adopting a sliding window to obtain M groups of network state parameters, wherein each group of network state parameters in the M groups of network state parameters comprises a characteristic value corresponding to a network attribute, and M is an integer greater than 1.

Specifically, for the network state parameters to be processed in the form of stream data, data sampling is performed on each stream data in a sliding window mode. The specific size of the sliding window may depend on the sampling frequency of the network state parameter to be processed. For example, if the network state parameter to be processed is sampled once per second, the size of the sliding window may be in seconds, for example 5 seconds. The network state parameters obtained by using the sliding window at least comprise two groups of network state parameters, that is, if the network state parameters to be processed are sampled once every second, the size of the sliding window is at least 2 seconds.

Each set of network state parameters comprises at least one network attribute, and the state value of each network attribute, namely the corresponding characteristic value, of the first test network. The network attributes may include at least dynamic delay, dynamic packet loss rate, dynamic bandwidth, static queue size, etc.

It will be appreciated that since a software product will typically have a plurality of different functional modes, for example, the software product comprises different functional modules, which implement different functional modes. For example, a gaming application may include a gaming communication module and a text messaging module. The network attributes contained in the network state parameters to be processed may be different for games when running different functional modules. In practical applications, the network state parameters to be processed for different functional modules can be classified into different packets and processed separately.

Accordingly, for functional modules of the same or nearly the same specific form, similar data may be classified into the same packet for processing together based on the data composition characteristics of the network state parameters to be processed. For example, for video class applications, the network properties involved in the video comment module and the in-station communication module may be the same or nearly the same, and the network state parameters to be processed for the two modules may then be processed together.

It should be understood that the network state parameter is a parameter generated according to the state of the first test network, and the change of the network state parameter is not necessarily dependent on the use state of the target application program, and the network state parameter for different functional modules may be understood as a parameter generated by the first test network when the target application program performs the function of a specific module.

Step S203, acquiring N sets of network state parameters from the M sets of network state parameters, where the N sets of network state parameters are parameters associated with the network abnormal state of the first test network, and N is an integer greater than or equal to 1 and less than M.

The manner of acquiring the N groups of network state parameters may be different, for example, a statistical data manner may be adopted, and the N groups of network state parameters may be obtained by screening according to specific user feedback information, performing iterative evaluation by using big data, and the like. The obtained N groups of network state parameters have an association relationship with the network abnormal state of the first test network. Specifically, the N sets of network state parameters may be parameters generated when the first test network is in an abnormal state, or statistical parameters indicating that the first test network is in an abnormal state. The statistical parameters may be, for example, a percentage of points whose delay is greater than a threshold, a percentage of points whose packet loss rate is greater than a threshold, a delay average, a delay standard deviation, a packet loss rate average, a packet loss rate standard deviation, and the like. N is an integer greater than or equal to 1 and less than M, i.e., when there are 10 sets of network state parameters, the value range of N may be 1 to 9.

When acquiring the N groups of network state parameters, the server can judge whether each group of network state parameters are matched with at least one abnormality detection rule. The anomaly detection rules include conditions that need to be met by network attributes in each set of network parameter states. For example, the network state parameters may include a delay and a queue length, and the anomaly detection rules may include a delay greater than a predetermined delay and a queue length greater than a predetermined length, and so on.

It can be understood that the above-mentioned anomaly detection rule can be set according to the network attribute in the network status parameter and the factors of different functional modules in the related target application program, etc. in combination with actual needs, which is not limited.

Step S204, a second test network is generated according to the characteristic values contained in the N groups of network state parameters.

Specifically, the server may control the network state simulator in the second test network to simulate the state of the first test network according to the characteristic values of each of the N sets of network state parameters, so as to achieve the purpose of reproducing the abnormal state of the first test network.

Specifically, the second test network may include a network state simulator and a test terminal to be connected. The test equipment uses the network state simulator as a gateway and is connected to the network through the network state simulator. The network state simulator is directly connected to the network and is responsible for receiving and forwarding network communications for the test terminals.

The server configures control parameters of the network state simulator according to the characteristic values of the network attributes in the respective network state parameters. The network state simulator forwards the communication of the test terminal according to the control parameters. For example, if the server adjusts the real-time delay in the control parameters of the simulator to 500ms, the simulator will delay the communication of the test terminal by 500ms and then forward the communication.

Step S205, controlling the target application to run in the second test network, so as to obtain a test result of the target application.

The server may send control instructions to the test devices in the second test network. The test equipment runs the target application according to the control instruction so as to obtain the performance of the target application in the second test network as a test result. For example, for a live broadcast application, the packet loss rate of the server in the second test network is set to a higher value, then the test equipment in the second test network is controlled to perform live broadcast operation or viewing operation, and the performance of the live broadcast application in terms of live broadcast and viewing is collected as a test result, which is used as a basis for optimizing the performance of the live broadcast application when the packet loss rate is high.

In the embodiment of the application, the acquisition of network state parameters is carried out aiming at the existing network where the target application is located, the specific network state parameters are selected from the network state parameters, the corresponding network state is reappeared in the test network according to the specific network state parameters, and then the test of the target application is carried out in the test network. By the method, various complex conditions and sudden changes in the real network can be repeated in the test network, so that targeted tests can be carried out on target applications aiming at the network conditions in the test network, and the accuracy of application program tests is improved.

In one embodiment of the present application, the M sets of network status parameters originate from the same sliding window, and in order to obtain the N sets of network status parameters from the M sets of network status parameters, as shown in fig. 3, the step S203 may be the following steps S301 to S303, which are described in detail below:

in step S301, feature values corresponding to the respective network attributes included in the M sets of network state parameters are acquired.

And for each group of network state parameters in the M groups of network state parameters, reading the characteristic values corresponding to the network attributes from the network state parameters.

In step S302, a feature average value corresponding to each network attribute is calculated according to the feature value corresponding to the network attribute.

In particular, for M sets of network state parameters, the same network attributes may be integrated together and the characteristic average calculated. Wherein the calculation of the feature average may employ a subset of the M sets of network state parameters. For example, for 6 sets of network state parameters in one sliding window, the feature average may be an average of 6 sets of network state parameters for each of the network attributes, i.e. there is only one feature average for each network attribute. The feature average may be an average of every 2 or 3 sets of network state parameters, i.e. there are 3 or 2 feature averages for each network attribute. In another embodiment, before calculating the feature average, the feature values may be filtered first, for example, for 10 sets of network state parameters in a sliding window, for one of the network attributes, the feature values exceeding a predetermined threshold may be filtered first or sorted by feature value size and the feature values of the first number of rows are obtained, and then the feature average is calculated based on these filtered feature values.

In step S303, N sets of network state parameters are generated according to the feature average values corresponding to the respective network attributes.

Specifically, each network attribute obtains a characteristic average value, which can be combined into N sets of network state parameters. For different feature average value acquisition modes, the mode of generating N groups of network state parameters is different. The N sets of network state parameters may include only 1 set of feature averages derived based on all network state parameters within the sliding window, or may include multiple sets of feature averages, each set of feature averages derived based on a subset of the M sets of network state parameters.

In this embodiment, the acquired network state parameters are processed by calculating the average value, so that the transient network abnormality does not have an excessive influence on the subsequent test process, and thus, the transient or occasional transient fluctuation of the network cannot be excessively tested, the test resources are saved, and the test efficiency is improved.

In one embodiment of the present application, the M sets of network status parameters originate from the same sliding window, and in order to obtain the N sets of network status parameters from the M sets of network status parameters, as shown in fig. 4, the step S203 may be the following steps S401 to S404, which are described in detail below:

in step S401, feature values corresponding to the network attributes included in the M sets of network state parameters are acquired.

The acquisition method in this step is the same as that in step S301, and will not be described here again.

In step S402, the feature values corresponding to the network attributes are clustered to obtain a cluster group and a cluster center value corresponding to the cluster group.

Specifically, depending on the form of the network state parameters, the clustering algorithm may be calculated in various suitable ways, such as StreamKM ++ or Kmeans. Next, a method using streamKM ++ for stream data will be described as an example.

For a network attribute, there will be M eigenvalues in M network state parameters of a sliding window, and they will be entered into StreamKM ++ clustering algorithm in sequence according to the time of acquisition.

The StreamKM ++ clustering algorithm calculates the clustering center value as follows:

n_t+1＝n_t+m_t

Wherein c _t represents a cluster center value obtained by the last clustering calculation and is a vector, n _t represents the number of characteristic values adopted by the last clustering calculation, x _t represents a center value of the characteristic values input at this time and is a vector, m _t represents the number of newly added characteristic values calculated by the current clustering calculation, a is a forgetting factor, and the value range is a epsilon (0, 1).

The clustering process is as follows:

Assuming that each network state parameter includes two network attributes, such as number of connections and queue length, for the first calculation, two cluster center values will be randomly generated. Assume that the randomly generated cluster center c _t is [ [0,0], [10,10] ]. Assuming that M is 4, the number of feature values input in the present clustering process is 4, and assuming that the number of feature values is [ [0,1], [1,0], [2,1], [9,9] ], the forgetting factor a is 0.5. For each input feature value, the cluster center value closest to itself will be calculated, and then n _t、x_t、m_t will be calculated. Taking cluster a with cluster center value of 0,0 as an example, n _t＝0、x_t＝[1,2/3]、m_t =3, for the present cluster calculation,

At this time, assuming that the characteristic values inputted are [ [1,0], [0,1], [11,9], [10,10] ] for the second calculation, then for cluster a, c _t+1＝[1,2/3]、n_t+1＝3、x_t+1＝[0.5,0.5]、m_t+1 =2, then for the second clustering calculation,

For subsequent inputs, subsequent iterations may continue to loop to obtain a cluster center value.

In step S403, it is determined that an abnormal cluster group is determined from among the cluster groups, based on the cluster center value and the abnormality threshold.

Specifically, the abnormal cluster group is judged according to whether the cluster center value and the abnormal threshold value meet at least one abnormal detection rule. The anomaly detection rules include conditions that the cluster center value needs to satisfy as compared to the anomaly threshold value. For example, the network state parameter may include a time delay and the anomaly detection rule may include a cluster center value of the time delay being greater than an anomaly threshold value.

In step S404, N sets of network state parameters are determined from the outlier cluster group.

For all outlier cluster groupings, statistics may be made of the condition of the eigenvalues within each group. For example, the ratio between the number of abnormal values and the number of normal values within each group or the degree to which the abnormal values themselves exceed the normal values is counted, each abnormal cluster group is sorted, and the cluster group with the highest sorting is regarded as the N-group network state parameter. Specifically, for example, the abnormal cluster group includes a delay cluster and a packet loss rate cluster, each cluster includes 20 eigenvalues, where the delay cluster includes 15 outliers, and the packet loss rate cluster includes 18 outliers, and N sets of network state parameters may be generated according to the outlier eigenvalues in the packet loss rate cluster.

It may be understood that, in the N sets of network state parameters, only the feature values in the abnormal cluster packet may be included, or each network state parameter corresponding to the feature values may be added to the N sets of network state parameters. For example, in the above example, the N sets of network state parameters may include only 18 outliers in the packet loss rate cluster, or may include a vector formed by 18 outliers and the corresponding delay feature values.

In one embodiment, the class determination may be made for the currently entered M network state parameters using the resulting cluster center value of the previous cycle. If the input M network state parameters comprise a cluster center value with the characteristic value larger than that of the previous period, the network state parameters can be determined to be abnormal values, at the moment, an abnormal state report can be generated based on the network state parameters, and the abnormal state report is reported to a user so as to monitor in real time.

In this embodiment, the abnormal network state parameters are screened by the clustering mode, and because the clustering mode can more fully embody the variation trend of each network attribute in the network, when the network condition is worse, the selected network state parameters can more fully embody the abnormal network state, thereby more accurately simulating the abnormal condition and improving the accuracy of network simulation.

In one embodiment of the application, the M sets of network state parameters originate from at least two sliding windows, the M sets of network state parameters comprising a first parameter set and a second parameter set, the first parameter set and the second parameter set comprising the same number of sets of network state parameters. In order to obtain N sets of network state parameters from M sets of network state parameters, as shown in fig. 5, step S203 may be the following steps S501 to S503, which are described in detail below:

in step S501, a first feature value corresponding to each network attribute is obtained from the first parameter set, and a second feature value corresponding to each network attribute is obtained from the second parameter set.

Specifically, the number of network state parameters included in the first parameter set and the second parameter set is the same, and the types of network attributes that are generally included are also the same. Depending on the number of network attributes, the first characteristic value may comprise a plurality of characteristic values specifically including a plurality of network attributes respectively, and similarly, the second characteristic value may also comprise a plurality of characteristic values corresponding to a plurality of network attributes respectively. And for each network attribute, reading the characteristic value corresponding to each network attribute from each group of network state parameters in the first parameter group and the second parameter group.

In step S502, a feature average value corresponding to each network attribute is determined according to the first feature value and the second feature value.

Specifically, for each network attribute of the first parameter set and the second parameter set, the first characteristic value and the second characteristic value corresponding to the same network attribute are integrated together, and the characteristic average value thereof is calculated. The network properties comprised by the first parameter set and the second parameter set may be different. For network properties commonly included in the first parameter set and the second parameter set, the feature average value is calculated across the sliding window, and for different network properties, the feature average value is calculated only within the window.

The calculation of the feature average may employ a subset of the first feature value and the second feature value. For example, for a total of 6 sets of network state parameters from two different windows, where the first and second sets of parameters respectively comprise 3 sets of network state parameters, the feature average may be an average of 6 sets of network state parameters for each of the network attributes, i.e. there is only one feature average for each network attribute. The characteristic average value may also be an average value of a plurality of sets of network state parameters from the first and second parameter sets, respectively, i.e. for each network attribute, one set of network state parameters in the first parameter set is averaged with one set of network state parameters in the second parameter set, so that for each network attribute in the 6 sets of network state parameters there will be 3 sets of average values.

In step S503, the feature average value corresponding to the network attribute is determined as N sets of network state parameters.

Specifically, each network attribute obtains a characteristic average value, which can be combined into N sets of network state parameters. For different feature average value acquisition modes, the mode of generating N groups of network state parameters is different. The N sets of network state parameters may comprise only 1 set of feature averages, which are obtained based on all network state parameters within at least two sliding windows, or may comprise a plurality of sets of feature averages, each set of feature averages being obtained based on feature values from the first parameter set and the second parameter set, respectively.

It can be understood that, although only two sliding windows are taken as examples in the above embodiment, the manner of calculating the feature average value for more than two sliding windows is analogized according to the above process, and will not be described herein.

In this embodiment, the characteristic average value is calculated based on network state parameters from different sliding windows as a calculation result, and since the sampling result of the sliding window can only aim at a transient state in one window, calculating the average value across the sliding window makes the calculation result based on a sample space with sufficient time span, thereby weakening the influence of an individual special case on the calculation result and being capable of more accurately simulating the network state according to the calculation result.

In one embodiment of the application, the M sets of network state parameters originate from at least two sliding windows, the M sets of network state parameters comprising a first parameter set and a second parameter set, the first parameter set and the second parameter set comprising the same number of sets of network state parameters. In order to obtain N sets of network state parameters from M sets of network state parameters, as shown in fig. 6, the above step S203 may be the following steps S601 to S604, which are described in detail as follows:

in step S601, first feature values corresponding to the respective network attributes are acquired from the first parameter set, and second feature values corresponding to the respective network attributes are acquired from the second parameter set.

The acquisition method in this step is the same as that in step S501, and will not be described here again.

In step S602, the first feature value and the second feature value are clustered to obtain a cluster group and a cluster center value corresponding to the cluster group, where the feature value in the cluster group is at least one of the first feature value and the second feature value.

The first characteristic value and the second characteristic value may in particular comprise a plurality of characteristic values, depending on the number of network attributes included in the network state parameter.

For one network attribute, among the M network state parameters derived from at least two sliding windows, there will be M eigenvalues, which may specifically be from at least one of the first eigenvalue and the second eigenvalue, and they will be entered into the StreamKM ++ clustering algorithm in a sequence according to the acquired time.

The specific calculation process of StreamKM ++ clustering algorithm is referred to the description in step S402, and is not described herein.

In step S603, an abnormal cluster group among the cluster groups is determined according to the cluster center value and the abnormality threshold value.

In step S604, N sets of network state parameters are determined from the outlier cluster group.

In this embodiment, when the abnormal network state parameters are screened by using the network state parameters from at least two sliding windows, the number of the input values of the clustering algorithm is increased, so that the clustering center value can be close to the real condition in the network, thereby more accurately simulating the abnormal condition and improving the accuracy of network simulation.

In one embodiment of the present application, based on the above technical solution, as shown in fig. 7, step S204 may specifically be described above, where the second test network is generated according to the feature values in the N sets of network state parameters, and step S701 and step S702 may be as follows:

in step S701, when the specified time is reached, obtaining a second test network at the specified time according to the feature value of the third network state parameter in the N sets of network state parameters;

In step S702, at each time after the designated time, the second test network updated at the previous time is updated according to the feature value of the fourth network state parameter in the N sets of network state parameters, so as to obtain an updated second test network.

Specifically, the server includes a network state simulator in the second test network, and generates the second test network using the network state simulator. Referring to fig. 8, fig. 8 is a schematic diagram of a functional module of the network status simulator. As shown in fig. 8, the network status simulator 800 includes a remote control interface 801, a parameter controller 802, an upstream queue 803, a downstream queue 804, and a data visualization device 805. The remote control interface 801 may be used to send and receive control signals to and from a server and to and from test equipment in the second test network. The parameter controller 802 is configured to control forwarding of the data packet according to a control signal of the server. The up queue 803 and the down queue 804 are used for transmitting and receiving data packets from a target application on the test device. The data visualization device 805 counts and displays the communication information as a chart for the user to check according to the records of the uplink queue and the downlink queue.

Specifically, in this embodiment, the server starts a timer in the parameter controller for each set of network state parameters based on the N sets of network state parameters. The timing length of the timer is mainly determined according to the sequence of the network state parameters and the degree of abnormality, for example, the timing length of the timer of the network state parameter with earlier occurrence time in the first test network may be shorter, and the network state parameter with higher degree of abnormality may also be shorter. It will be appreciated that the timing length of each timer should be varied to ensure that each set of network state parameters has the appropriate test duration.

When any timer expires, the current time reaches the designated time, and at this time, the parameter controller updates the control parameter of the network state simulator according to the characteristic value of the third network state parameter corresponding to the timer. Then, the network state simulator provides communication service for the target application of the test equipment in the second test network according to the control parameters, and for the communication data needing abnormal simulation, the uplink queue or the downlink queue can be used for performing operations such as delay, packet loss, bandwidth reduction and the like, for example, according to the delay parameters, the communication request of the target application in the test equipment is added into the uplink queue or the downlink queue for a certain time to delay and then is forwarded, so as to simulate the high-delay network.

When the network state simulator controls communication according to the network state parameters, filtering judgment can be carried out according to an internet protocol address, a port, a protocol type and the like of the test equipment for communication, so that only communication data related to a target application are added into a queue for communication control.

After the specified time is reached, the other timers will continue to count. At each time after the specified time, when any timer expires, the second test network updated at the previous time is updated according to the characteristic value of the fourth network state parameter in the N sets of network state parameters, that is, the parameter controller in the network state simulator updates its control parameters according to the characteristic value of the fourth network state parameter, so as to simulate and test the new abnormal network state.

After the network state simulator updates the control parameters, an instruction can be sent to the test equipment in the second test network through the remote control interface to start the target application to execute the test operation. The launching of the target application to perform the test operation may also be performed in other ways, such as actively triggered by a third party, without limitation.

It will be appreciated that the network state simulator described above may be executed by a stand-alone device or may be acted upon by the server itself executing the method of application testing. In embodiments served by the server itself, the server may send control instructions between its own functional modules to generate the second test network based on the obtained network state parameters associated with the abnormal network state.

In the application, the second test network is updated when the appointed time and each subsequent time are reached, so that the switching between the network states can be automatically carried out, the manual access configuration process is avoided, and the efficiency of testing the application can be improved.

In one embodiment of the present application, based on the above technical solution, as shown in fig. 9, step S205 may specifically be performed to control the target application to run in the second test network, so as to obtain a test result corresponding to the target application, and step S901 and step S903 may be performed as follows:

in step S901, a data packet sent by a target application is received;

in step S902, the data packets are counted to obtain network response data, where the network response data includes at least one of packet loss rate, delay and bandwidth;

In step S903, a test result corresponding to the target application is generated according to the network response data.

Specifically, the server may receive, through the network state simulator, a data packet sent by the target application. The data packet may include information that the target application monitors and counts its own running state, and the server parses and counts the data packet to obtain network response data. The data packet may also be a data packet for performing service processing, where the server may perform statistics on the service performance of the target application, so as to obtain network response data. The network response data typically includes at least one of packet loss rate, delay, bandwidth of the target application in the communication. The network response data may also include the performance of the target application itself on the network status, where such performance varies depending on the class and purpose of the target application itself, for example, for gaming applications, the smoothness of the game at high latency may be counted, and for transaction applications, the transaction success rate of the transaction at high packet loss rate may be counted. The test content related to the specific performance of the target application is not limited herein.

After the network response data is obtained, statistics can be carried out according to the content in the network response data to serve as a test result corresponding to the target application, and meanwhile, the network response data can be combined with the test result obtained before to carry out graphical display so as to be convenient for a user to use.

According to the application, the network response data is obtained according to the data packet sent by the target application, so that the test result of the target application under the second test network can be automatically obtained, and thus, the manual intervention in the test process can be reduced, the cost consumption of the test is saved, and the test efficiency and operability are improved.

It should be noted that although the steps of the methods of the present application are depicted in the accompanying drawings in a particular order, this does not require or imply that the steps must be performed in that particular order, or that all illustrated steps be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.

The following describes a method implemented by the apparatus of the present application, which may be used to perform the application testing in the above-described embodiments of the present application. Fig. 10 schematically shows a block diagram of the application test apparatus in an embodiment of the application. As shown in fig. 10, the application test apparatus 1000 may mainly include:

a first obtaining module 1001, configured to obtain a network state parameter to be processed generated by a first test network when a target application runs on the first test network;

The sampling module 1002 is configured to sample network state parameters to be processed by using a sliding window, so as to obtain M groups of network state parameters, where N groups of network state parameters indicate a network abnormal state of the first test network, each group of network state parameters in the M groups of network state parameters includes a feature value corresponding to a network attribute, and M is an integer greater than 1;

A second obtaining module 1003, configured to obtain N sets of network state parameters from M sets of network state parameters, where N is an integer greater than or equal to 1 and less than M;

a generating module 1004, configured to generate a second test network according to the feature values included in the N sets of network state parameters;

The control module 1005 is configured to control the target application to run in the second test network to obtain a test result of the target application.

In one embodiment of the present application, based on the above technical solution, the second obtaining module 1003 may include:

the acquisition unit is used for acquiring the characteristic values corresponding to the network attributes contained in the M groups of network state parameters;

the computing unit is used for computing the characteristic average value corresponding to each network attribute according to the characteristic value corresponding to the network attribute;

and the generating unit is used for generating the N groups of network state parameters according to the characteristic average value corresponding to each network attribute.

the acquisition unit is also used for acquiring the characteristic values corresponding to the network attributes contained in the M groups of network state parameters;

The clustering unit is used for clustering the characteristic values corresponding to the network attributes to obtain clustering groups and clustering center values corresponding to the clustering groups;

A determining unit, configured to determine an abnormal cluster group from the cluster groups according to the cluster center value and an abnormal threshold value;

In one embodiment of the present application, based on the above technical solution, the generating module 1004 may include:

It should be noted that, the apparatus provided in the foregoing embodiments and the method provided in the foregoing embodiments belong to the same concept, and a specific manner in which each module performs an operation has been described in detail in the method embodiment, which is not described herein again.

It should be noted that, the computer system 1100 of the electronic device shown in fig. 11 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.

As shown in fig. 11, the computer system 1100 includes a central processing unit (Central Processing Unit, CPU) 1101 that can perform various appropriate actions and processes according to a program stored in a Read-Only Memory (ROM) 1102 or a program loaded from a storage section 1108 into a random access Memory (Random Access Memory, RAM) 1103. In the RAM 1103, various programs and data required for system operation are also stored. The CPU 1101, ROM 1102, and RAM 1103 are connected to each other by a bus 1104. An Input/Output (I/O) interface 1105 is also connected to bus 1104.

Connected to the I/O interface 1105 are an input section 1106 including a keyboard, a mouse, and the like, an output section 1107 including a Cathode Ray Tube (CRT), a Liquid crystal display (Liquid CRYSTAL DISPLAY, LCD), and the like, and a speaker, and the like, a storage section 1108 including a hard disk, and the like, and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 1109 performs communication processing via a network such as the internet. The drive 1110 is also connected to the I/O interface 1105 as needed. Removable media 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is installed as needed on drive 1110, so that a computer program read therefrom is installed as needed into storage section 1108.

In particular, the processes described in the various method flowcharts may be implemented as computer software programs according to embodiments of the application. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 1109, and/or installed from the removable media 1111. When executed by a Central Processing Unit (CPU) 1101, performs the various functions defined in the system of the present application.

It should be noted that, the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM), a flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, etc., or any suitable combination of the foregoing.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.

From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the method according to the embodiments of the present application.

Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains.

It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method of application testing, comprising:

2. The method of claim 1, wherein the M sets of network state parameters originate from the same sliding window;

Obtaining N sets of network state parameters from the M sets of network state parameters, including:

acquiring characteristic values corresponding to all network attributes contained in the M groups of network state parameters;

Calculating a characteristic average value corresponding to each network attribute according to the characteristic value corresponding to the network attribute;

and generating the N groups of network state parameters according to the characteristic average value corresponding to each network attribute.

3. The method of claim 1, wherein the M sets of network state parameters originate from the same sliding window;

Clustering the characteristic values corresponding to the network attributes to obtain clustering groups and clustering center values corresponding to the clustering groups;

determining an abnormal cluster group from the cluster groups according to the cluster center value and the abnormal threshold value;

and determining the N groups of network state parameters according to the abnormal clustering group.

4. The method of claim 1, wherein the M sets of network state parameters originate from at least two sliding windows, the M sets of network state parameters comprising a first set of parameters and a second set of parameters, the first set of parameters and the second set of parameters comprising the same number of sets of network state parameters;

acquiring first characteristic values corresponding to all network attributes from the first parameter set, and acquiring second characteristic values corresponding to all network attributes from the second parameter set;

determining a characteristic average value corresponding to each network attribute according to the first characteristic value and the second characteristic value;

And determining the characteristic average value corresponding to the network attribute as the N groups of network state parameters.

5. The method of claim 1, wherein the M sets of network state parameters originate from at least two sliding windows, the M sets of network state parameters comprising a first set of parameters and a second set of parameters, the first set of parameters and the second set of parameters comprising the same number of sets of network state parameters;

Clustering the first characteristic value and the second characteristic value to obtain a cluster group and a cluster center value corresponding to the cluster group, wherein the characteristic value in the cluster group is from at least one of the first characteristic value and the second characteristic value;

Determining an abnormal cluster group in the cluster groups according to the cluster center value and the abnormal threshold value;

6. The method according to any one of claims 1 to 5, wherein said generating a second test network from said eigenvalues of said N sets of network state parameters comprises:

When the appointed time is reached, a second test network at the appointed time is obtained according to the characteristic value of a third network state parameter in the N groups of network state parameters;

And updating the second test network updated at the previous time according to the characteristic value of the fourth network state parameter in the N groups of network state parameters at each time after the appointed time to obtain an updated second test network.

7. The method according to any one of claims 1 to 5, wherein the controlling the target application to run in the second test network to obtain the test result corresponding to the target application includes:

receiving a data packet sent by the target application;

Counting the data packets to obtain network response data, wherein the network response data comprises at least one of packet loss rate, delay and bandwidth;

and generating a test result corresponding to the target application according to the network response data.

8. An application testing apparatus, comprising:

The sampling module is used for sampling the network state parameters to be processed by adopting a sliding window to obtain M groups of network state parameters, wherein each group of network state parameters in the M groups of network state parameters comprises a characteristic value corresponding to a network attribute, and M is an integer greater than 1;

a second obtaining module, configured to obtain N sets of network state parameters from the M sets of network state parameters, where the N sets of network state parameters are parameters associated with a network abnormal state of the first test network, and N is an integer greater than or equal to 1 and less than M;

9. An electronic device, comprising:

A processor;

A memory for storing executable instructions of the processor;

wherein the processor is configured to perform the method of application testing of any of claims 1 to 7 via execution of the executable instructions.

10. A computer readable medium on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements a method of application testing according to any of claims 1 to 7.

11. A computer program product, characterized in that it comprises computer instructions stored in a computer-readable storage medium, from which computer instructions a processor of a computer device reads, which processor executes the computer instructions, so that the computer device performs the method of application testing according to any of claims 1 to 7.