CN104717236A - Equipment performance test method and device - Google Patents

Abstract

The invention discloses a device performance testing method. The maximum value of the number of concurrent threads that can be increased at present is determined by periodically collecting various performance data of the tested device in the simulated test thread, and the maximum value and the preset concurrent thread are single-step Select the smaller one among the increased values to increase the test thread currently used by the device under test for simulation, and judge whether the various performance data of the device under test exceed the preset performance threshold. Therefore, on the premise that the tested equipment does not crash, the tested equipment can be adjusted in real time to achieve the highest performance, and then the bottleneck of the business system can be located in time to optimize the system.

Description

A device performance testing method and device

technical field

The invention relates to the field of communication technology, in particular to a device performance testing method. The present invention also relates to an equipment performance testing device

Background technique

Usually, before a multi-user-oriented web application system is built and launched, it needs to obtain performance indicators such as the traffic it can carry, response speed, and fault tolerance. Generally, stress testing is carried out through performance testing tools to obtain performance indicators such as the running status and response time of the web application system.

The main function of the performance testing tool is to simulate the real business operation in the production environment, implement stress load testing on the tested system, monitor the performance of the tested system under different business and different stress performance, and find out potential performance bottlenecks for analysis ,optimization.

A commonly used system performance testing tool records performance indicators such as the number of concurrent client requests and response delay while simulating client behavior. However, under the existing test conditions, test tools and business systems are distributed on different hosts, and the host performance of the business system is monitored by three parties. Changes in the status of the business host cannot be directly fed back to the test client. Too many, the limited resources of the web application host, such as high cpu usage, too many connections, etc., cause the system to overload and crash, and the host can only be restarted manually. At this time, the three-party monitoring will also be affected, and it is impossible to locate the abnormality when the system crashes in time Environmental information, runtime data, etc., sometimes it is difficult to reproduce abnormal situations.

It can be seen that the commonly used system performance testing tools record performance indicators such as the number of concurrent client requests and response delay while simulating the behavior of the client. However, in normal testing situations, test tools and business systems are distributed on different hosts, and the host performance changes of the business system are monitored by three parties. Changes in the status of the business host cannot be directly fed back to the test client, and the following problems often occur :

(1) Due to too many requests from the client, the limited resources of the web application host, such as cpu, are too high, and the number of connections is too high, so that the web application system is overloaded and crashes, and the host can only be restarted manually.

(2) The overload of the system will also affect the three-party monitoring, and it is impossible to locate the abnormal environmental information and runtime data when the system crashes in time.

(3) It is difficult to reproduce the abnormal situation, and it is more complicated to locate and capture the cause of the abnormality and the bottleneck of the system.

It can be seen that the test equipment in the prior art cannot dynamically adjust the test conditions of the tested equipment according to the pressure situation in real time, and then obtain the test data of the equipment operating at full load.

Contents of the invention

In view of the deficiencies in the prior art, the present invention proposes a method for testing equipment performance, which is characterized in that it includes:

The test equipment regularly collects various performance data of the tested equipment in the simulated test thread;

The test device determines the maximum value of the number of concurrent threads that can be increased currently according to the various performance data;

The test device selects the smaller one between the maximum value and the preset concurrent thread incremental value as the concurrent thread incremental value, and increases the current value of the device under test according to the concurrent thread incremental value. simulated test thread;

The test device collects various performance data of the tested device in the simulated increased test thread, and judges whether the various performance data exceed a preset performance threshold;

If not, the test device sets the number of test threads as a preset initial value, and instructs the device under test to simulate the test threads.

Correspondingly, the present invention also proposes a device performance testing device, which is characterized in that it includes:

The collection module is used to regularly collect various performance data of the tested equipment in the simulated test thread;

A determining module, configured to determine the maximum value of the number of concurrent threads that can be increased currently according to the various performance data;

The selection module is used to select the smaller one between the maximum value and the preset concurrent thread incremental value as the concurrent thread incremental value, and increase the current usage rate of the device under test according to the concurrent thread incremental value. for simulated test threads;

A judging module, configured to collect various performance data of the device under test in the simulated increased test thread, and judge whether the various performance data exceed a preset performance threshold;

If not, the judging module sets the number of test threads as a preset initial value, and instructs the device under test to simulate the test threads.

It can be seen that by applying the above technical solutions, the maximum value of the number of concurrent threads that can be increased is determined by the performance data of the tested device in the simulated test thread collected regularly, and the maximum value and the preset concurrent threads are increased in a single step. Choose the smaller one among the values to increase the test thread currently used by the device under test for simulation, and judge whether the performance data of the device under test exceeds the preset performance threshold. Therefore, on the premise that the tested equipment does not crash, the tested equipment can be adjusted in real time to achieve the highest performance, and then the bottleneck of the business system can be located in time to optimize the system.

Description of drawings

Fig. 1 is the schematic flow sheet of a kind of equipment performance testing method that the present invention proposes;

Fig. 2 is the composition and structural representation of the testing system that the specific embodiment of the present invention proposes;

Fig. 3 is a schematic flow chart of a device performance testing method proposed by a specific embodiment of the present invention;

Fig. 4 is a schematic structural diagram of a device performance testing device proposed by the present invention.

Detailed ways

In order to solve the problem in the prior art that the tested equipment cannot obtain its own test pressure and use it to dynamically adjust the test conditions, this paper proposes a web application system stress test method controlled by remote monitoring. , remotely monitor the performance indicators of various limited resources of the system under test, and collect the results, dynamically adjust the strategy and concurrency of the stress test client, so that the host under test can maintain an effective processing state, and system crashes can also be recorded The number of concurrency before and after and the business system environment data realize the automatic optimization of the business system.

As shown in Figure 1, for a kind of equipment performance testing method that the present invention proposes, comprises the following steps:

S101. The testing device regularly collects various performance data of the tested device in a simulation test thread.

It should be noted that, in this step and in S104, the test device will continuously collect various performance data of the device under test according to the preset collection times and obtain the root mean square average value of the collection results, and convert the mean square The root average value is used as the performance data of each item.

S102. The test device determines the maximum value of the current number of concurrent threads that can be increased according to the various performance data.

Specifically, in this step, the test device first determines the estimated maximum number of threads that can be increased Tadd corresponding to the performance numbers according to Tadd=T*(Pmax-Pnns)/Pnns, where T is the current number of test threads, and Pmax is equal to The maximum value corresponding to the various performance numbers, Pnns is the root mean square average value corresponding to the various performance numbers; then the minimum value in the estimation of the maximum number of threads that can be increased corresponding to the various performance numbers As the maximum number of concurrent threads that can be increased by the device under test.

S103. The test device selects the smaller one between the maximum value and the preset concurrent thread increment value as the concurrent thread increment value, and increases the current value of the device under test according to the concurrent thread increment value. The test thread used for simulation, .

S104. The test device collects various performance data of the tested device in the simulated increased test thread, and judges whether the various performance data exceed a preset performance threshold.

S105. If not, the test device sets the number of test threads as a preset initial value, and instructs the device under test to simulate the test threads.

Corresponding to this step, if the testing device judges that the performance data of the tested device exceeds a preset performance threshold, the testing device sets the number of test threads to a preset maximum value, and instructs the The device under test simulates the test thread.

In addition, after this step, the test device increases the test thread currently used by the device under test for simulation every preset time, and the number of test threads added each time is a single-step increase value for the preset concurrent threads. In this way, the cyclic pressurization test of the equipment under test is realized.

In order to further illustrate the technical idea of the present invention, the technical solution of the present invention will now be described in conjunction with specific application scenarios. As shown in Figure 2, the test system proposed for the specific embodiment of the present invention is composed of a performance test client and a Web application server agent, wherein the modules corresponding to each composition are as follows:

Performance test client:

(1) Test cases: Realization of user behavior simulation use cases of the web application system.

(2) Stress test unit: Simulate the test thread, call the test case to initiate the request.

(3) Concurrency control module: According to the strategy provided by the configuration and monitoring module, initialize the test unit, control the number of test threads, and control the pressure of the test thread;

(4) Remote monitoring module: Initiate a remote performance index monitoring request and implement the pressure strategy algorithm;

(5) Configuration module: configure the test process, duration, number of single-step supercharging threads, etc., and the threshold of monitoring indicators.

(6) Message middleware: realize the interface protocol and communication function between the test client and the business server.

Web application server proxy (system under test):

(1) performance monitoring agent;

(2) Business system software.

The performance test process based on the above settings is shown in Figure 3, including:

Step 1. Initialize and start the stress test unit according to the configuration, initialize the remote monitoring module and the performance monitoring agent;

Step 2. The concurrency control module starts from the initial number of threads and increases gradually. For the interval of each increase, refer to the configuration module;

Step 3. Schedule remote performance collection n times, and calculate the average value of each performance index according to the collection index (the performance index of limited resources is normalized to the proportion, and in order to reflect the pressure of the host performance, the root mean square average is used).

Step 4. Adjust the algorithm according to the stress test strategy, and calculate the number of concurrent threads increased at one time.

Step 5, the concurrency control module increases concurrent threads to increase the pressure on the web application system;

Step 6, remote performance collection and monitoring again, if the performance threshold is exceeded, the test thread that needs to be terminated is calculated according to the strategy;

Step 7. Repeat steps 2-6 to continuously adjust the performance pressure to find the maximum system pressure load. And count the number of concurrency, delay, success rate, and performance indicators of remote hosts.

Step 8. After the test is over, sort out the statistical results of the client stress test, including concurrency, delay, success rate, and remote performance monitoring data.

Among them, the detailed algorithm of stress test strategy supercharging is as follows:

First, configure the initial values of the stress test simulated users:

The initial number of test threads T _init and the maximum value T _max , the current number of test threads T, the single-step concurrent growth configuration T ₀ , the number of cases C _n executed by a single test thread at one time, the cycle times or duration of the test thread,

Then, configure the remote performance monitoring index threshold, and the performance index of limited resources is normalized to the proportion according to the resource limit, for example:

The highest cpu occupancy rate P _cpumax , the highest proportion of business threads (maximum value of web application threads/system user threads * 100%) P _threadmax , the largest proportion of memory usage (used_mem/tatle_mem) P _memmax , the proportion of used connections (web The number of application port connections/the number of user process files) P _linkmax , the proportion of network IO usage P _niomax , and the proportion of disk IO usage P _diomax .

Based on the above configuration values, the specific algorithm is as follows:

The calculation method of the root mean square average value of the n times value of each performance index P is as follows:

${\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; nns</span>}}<span\; class="notranslate">\; =</span>\sqrt{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; no</span>}\underset{\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

The algorithm Rp of the maximum increase in the number of concurrent requests limited by the threshold value of each performance index is:

${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; avg</span>}}}{{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; nns</span>}}}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; nns</span>}}<span\; class="notranslate">\; )</span>$

Estimation of the maximum number of threads that can be increased (the minimum value of the results calculated by multiple indicators):

T _add =R _p ×T/R _avg

The algorithm of the maximum number of test threads T _next for single-step boosting is

T _next = min(T ₀ , T _add ).

In order to achieve the above technical objectives, the present invention also proposes a device performance testing device, as shown in Figure 4, including:

Acquisition module 410, is used for regularly collecting various performance data of the device under test in the simulated test thread;

A determination module 420, configured to determine the maximum value of the current number of concurrent threads that can be increased according to the various performance data;

A selection module 430, configured to select the smaller one between the maximum value and the preset single-step increase value of concurrent threads as the incremental value of concurrent threads, and increase the current value of the device under test according to the incremental value of concurrent threads. Test thread for simulation;

A judging module 440, configured to collect various performance data of the device under test in the simulated increased test thread, and judge whether the various performance data exceed a preset performance threshold;

If not, the judging module 440 sets the number of test threads as a preset initial value, and instructs the device under test to simulate the test threads.

In a specific application scenario, it also includes: an increment module, configured to, after the judging module 440 sets the number of test threads as a preset initial value and instructs the device under test to simulate the test threads, The test thread currently used by the device under test for simulation is increased at preset time intervals, and the number of test threads added each time is a single-step increase value of the preset concurrent thread.

In a specific application scenario, the judging module 440 is further configured to set the number of test threads to a preset maximum value and instruct The device under test simulates the test thread.

In a specific application scenario, the collection module 410 or the judgment module 440 is specifically used for:

Continuously collect various performance data of the device under test according to preset collection times and obtain a root mean square average value of the collection results, and use the root mean square average value as the various performance data.

In a specific application scenario, the determining module 420 is specifically used for:

According to T _add =T*(P _max -P _nns )/P _nns determine the estimate T _add of the maximum number of threads that can be increased corresponding to the various performance numbers, T is the current number of test threads, and P _max is the The maximum value corresponding to the item performance number, P _nns is the root mean square average value corresponding to the various performance numbers, and the minimum value in the estimation of the maximum number of threads that can be increased corresponding to the various performance numbers is used as the The current maximum number of concurrent threads that can be increased by the device under test.

By applying the above technical solutions, the maximum value of the number of concurrent threads that can be increased is determined through the regularly collected performance data of the device under test in the simulated test thread, and the choice is made between the maximum value and the single-step increase value of the preset concurrent threads. The smaller one is to increase the test thread currently used by the device under test for simulation, and judge whether the performance data of the device under test exceeds the preset performance threshold. Therefore, on the premise that the tested equipment does not crash, the tested equipment can be adjusted in real time to achieve the highest performance, and then the bottleneck of the business system can be located in time to optimize the system.

Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be realized by hardware, or by software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present invention can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.), including several The instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in various implementation scenarios of the present invention.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present invention.

Those skilled in the art can understand that the modules in the devices in the implementation scenario can be distributed among the devices in the implementation scenario according to the description of the implementation scenario, or can be located in one or more devices different from the implementation scenario according to corresponding changes. The modules of the above implementation scenarios can be combined into one module, or can be further split into multiple sub-modules.

The above serial numbers of the present invention are for description only, and do not represent the pros and cons of the implementation scenarios.

The above disclosures are only some specific implementation scenarios of the present invention, however, the present invention is not limited thereto, and any changes conceivable by those skilled in the art shall fall within the protection scope of the present invention.

Claims (10)

1. A device performance testing method, characterized in that, comprising: The test equipment regularly collects various performance data of the tested equipment in the simulated test thread; The test device determines the maximum value of the number of concurrent threads that can be increased currently according to the various performance data; The test device selects the smaller one between the maximum value and the preset concurrent thread incremental value as the concurrent thread incremental value, and increases the current value of the device under test according to the concurrent thread incremental value. simulated test thread; The test device collects various performance data of the tested device in the simulated increased test thread, and judges whether the various performance data exceed a preset performance threshold; If not, the test device sets the number of test threads as a preset initial value, and instructs the device under test to simulate the test threads. 2. The method according to claim 1, wherein, after the test device sets the number of test threads as a preset initial value and instructs the device under test to simulate the test threads, further comprising: The test device increases the test thread currently used by the device under test for simulation every preset time, and the number of test threads added each time is the preset concurrent thread single-step increase value. 3. The method of claim 1, wherein, If the testing equipment judges that the performance data of the tested equipment exceeds the preset performance threshold, the testing equipment sets the number of test threads as a preset maximum value, and instructs the tested equipment to perform the test Threads are simulated. 4. The method according to claim 1, characterized in that, the test equipment regularly collects the performance data of the tested equipment in the simulated test thread or the test equipment collects the simulated increased test thread of the tested equipment The performance data in , specifically: The testing device continuously collects various performance data of the device under test according to preset collection times and obtains a root mean square average value of the collection results, and takes the root mean square average value as the various performance data. 5. The method according to claim 4, wherein the test device determines the maximum value of the number of concurrent threads that can be currently increased according to the various performance data, specifically: The test device determines the estimated maximum number of threads T _add corresponding to the performance numbers according to T _add =T*(P _max -P _nns )/P _nns , T is the current number of test threads, and P _max is The maximum value corresponding to the various performance numbers, P _nns is the root mean square average value corresponding to the various performance numbers; The test device takes the minimum value among the estimates of the maximum number of threads that can be increased corresponding to the various performance numbers as the maximum value of the current number of concurrent threads that can be increased by the device under test. 6. A device performance testing device, characterized in that, comprising: The collection module is used to regularly collect various performance data of the tested equipment in the simulated test thread; A determining module, configured to determine the maximum value of the number of concurrent threads that can be increased currently according to the various performance data; The selection module is used to select the smaller one between the maximum value and the preset concurrent thread incremental value as the concurrent thread incremental value, and increase the current usage of the device under test according to the concurrent thread incremental value. for simulated test threads; A judging module, configured to collect various performance data of the device under test in the simulated increased test thread, and judge whether the various performance data exceed a preset performance threshold; If not, the judging module sets the number of test threads as a preset initial value, and instructs the device under test to simulate the test threads. 7. The device of claim 6, further comprising: an increment module, configured to increase the tested device every preset time after the judging module sets the number of test threads as a preset initial value and instructs the tested device to simulate the test threads The number of test threads currently used for simulation, the number of test threads added each time is the value of the preset concurrent thread step-by-step increase. 8. The apparatus of claim 6, wherein The judging module is further configured to set the number of test threads to a preset maximum value when judging that each item of performance data of the device under test exceeds a preset performance threshold, and instruct the device under test to The test thread performs the simulation. 9. The device according to claim 6, wherein the acquisition module or the judgment module is specifically used for: Continuously collect various performance data of the device under test according to preset collection times and obtain a root mean square average value of the collection results, and use the root mean square average value as the various performance data. 10. The device according to claim 9, wherein the determining module is specifically configured to: According to T _add =T*(P _max -P _nns )/P _nns determine the estimate T _add of the maximum number of threads that can be increased corresponding to the various performance numbers, T is the current number of test threads, and P _max is the The maximum value corresponding to the item performance number, P _nns is the root mean square average value corresponding to the various performance numbers, and the minimum value in the estimation of the maximum number of threads that can be increased corresponding to the various performance numbers is used as the The current maximum number of concurrent threads that can be increased by the device under test.