CN111984527A - Software performance testing method, device, equipment and medium - Google Patents
Software performance testing method, device, equipment and medium Download PDFInfo
- Publication number
- CN111984527A CN111984527A CN202010661925.6A CN202010661925A CN111984527A CN 111984527 A CN111984527 A CN 111984527A CN 202010661925 A CN202010661925 A CN 202010661925A CN 111984527 A CN111984527 A CN 111984527A
- Authority
- CN
- China
- Prior art keywords
- performance
- test
- software
- report
- tested
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 242
- 238000011056 performance test Methods 0.000 claims abstract description 197
- 238000000034 method Methods 0.000 claims abstract description 43
- 238000013522 software testing Methods 0.000 claims abstract description 6
- 238000004590 computer program Methods 0.000 claims description 13
- 238000005457 optimization Methods 0.000 claims description 10
- 238000010835 comparative analysis Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000004044 response Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3664—Environments for testing or debugging software
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
Abstract
The application relates to the technical field of software testing, and provides a software performance testing method and device, computer equipment and a storage medium. The method and the device can improve the efficiency of obtaining the software performance test result. The method comprises the following steps: the performance test method comprises the steps of determining the performance to be tested of software to be tested, testing the software to be tested on the basis of a test scene corresponding to the performance to be tested to obtain a performance test report under the test scene, carrying out performance index comparison analysis on the performance test report and a preset baseline report under the test scene, and obtaining a performance test result of the software to be tested according to the comparison analysis result.
Description
Technical Field
The present application relates to the field of software testing, and in particular, to a method and an apparatus for testing software performance, a computer device, and a storage medium.
Background
The software development process involves software update iteration, and generally needs to test the performance of new and old versions of software, compare the performance test results of different software, and evaluate whether the new software meets the performance requirements or is superior to the performance requirements of the old version of software.
In the prior art, the performance test is respectively executed on new and old versions of software by manually creating a performance test scene in the existing test, and after a performance test report is obtained, the performance index data in the report is manually compared, so that the operation mode is complex and the efficiency is low.
Disclosure of Invention
In view of the above, it is necessary to provide a software performance testing method, apparatus, computer device and storage medium for solving the technical problem of low efficiency of software performance testing in the prior art.
A method of performance testing of software, the method comprising:
determining the performance to be tested of the software to be tested;
testing the software to be tested based on a preset test scene corresponding to the performance to be tested;
acquiring a performance test report of the software to be tested in the preset test scene, and triggering the performance test report to perform performance index comparison analysis with a preset baseline report in the preset test scene;
and comparing the analysis result according to the performance index to obtain a performance test result of the software to be tested.
An apparatus for performance testing of software, the apparatus comprising:
the performance determining module is used for determining the performance to be tested of the software to be tested;
the software testing module is used for testing the software to be tested based on a preset testing scene corresponding to the performance to be tested;
the index comparison module is used for acquiring a performance test report of the software to be tested in the preset test scene, and triggering performance index comparison analysis with a preset baseline report in the preset test scene;
and the result acquisition module is used for comparing the analysis result according to the performance index and acquiring the performance test result of the software to be tested.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
determining the performance to be tested of the software to be tested; testing the software to be tested based on a preset test scene corresponding to the performance to be tested; acquiring a performance test report of the software to be tested in the preset test scene, and triggering the performance test report to perform performance index comparison analysis with a preset baseline report in the preset test scene; and comparing the analysis result according to the performance index to obtain a performance test result of the software to be tested.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
determining the performance to be tested of the software to be tested; testing the software to be tested based on a preset test scene corresponding to the performance to be tested; acquiring a performance test report of the software to be tested in the preset test scene, and triggering the performance test report to perform performance index comparison analysis with a preset baseline report in the preset test scene; and comparing the analysis result according to the performance index to obtain a performance test result of the software to be tested.
According to the performance testing method, device, computer equipment and storage medium of the software, the performance to be tested of the software to be tested is determined, the software to be tested is tested on the basis of the testing scene corresponding to the performance to be tested, the performance testing report under the testing scene is obtained, the performance indexes of the performance testing report and the baseline report preset under the testing scene are compared and analyzed, the performance testing result of the software to be tested is obtained according to the comparison and analysis result, the performance testing report of the software to be tested can be automatically compared with the baseline report under the corresponding testing scene to obtain the performance testing result, and the efficiency of obtaining the software performance testing result is improved.
Drawings
FIG. 1 is a diagram illustrating an internal structure of a computer device according to an embodiment;
FIG. 2 is a flowchart illustrating a method for testing the performance of software according to one embodiment;
FIG. 3 is a flowchart illustrating a method for testing the performance of software according to one embodiment;
FIG. 4 is a flow diagram that illustrates a platform for performance testing of software, according to one embodiment;
FIG. 5 is a flow diagram that illustrates a platform for performance testing of software, according to one embodiment;
FIG. 6 is a block diagram showing a performance testing apparatus for software according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The software performance testing method provided by the application can be applied to computer equipment, the computer equipment can be a server, and the internal structure diagram can be shown in fig. 1. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used for storing data such as software performance test scenes, software performance test reports, performance indexes and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of performance testing of software.
Those skilled in the art will appreciate that the architecture shown in fig. 1 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, as shown in fig. 2, a method for testing the performance of software is provided, which is described by taking the application of the method to the server in fig. 1 as an example, and includes the following steps:
step S201, determining the performance to be tested of the software to be tested.
The software performance is an index which is used for describing the speed, efficiency and capacity of the software and can indicate the compliance degree of a software system to the characteristic requirements; software performance may be a characteristic of software that may be measured in terms of time, such as response time, throughput per unit time, and the like. The software performance test is the fastest speed or best performing target software in responding to the corresponding demand under a given benchmark environment. When software performance testing is carried out, a performance testing platform needs to be established according to the testing process and the testing requirement of the performance to be tested, and testing items, performance testing scenes and the performance testing process are managed.
In a specific implementation, the server may determine one or more performances to be tested according to a test requirement of the software to be tested. In some embodiments, the server may establish a performance test project according to the test requirements, including the performance test requirements, the server resource requirements, and information to be tested, so as to deploy service information according to the performance test project. And setting a source of the deployment service package and a deployment destination server according to the deployment service information, determining configuration information needing to be replaced, and starting the service by the server according to the deployment service information. The server creates a performance test scenario according to the performance test project requirements, wherein the performance test scenario may include a performance test script, performance test execution parameters, performance test indexes and the like, the performance test execution parameters may include the number of concurrent users, the pressure test execution time, the concurrent user increment mode and the like, and the performance test indexes may include transaction response time, throughput, transaction success rate, resource utilization rate and the like. Multiple performance testing scenarios may be created according to the performance testing business model. The performance testing business model may include one or more performances to be tested. In some embodiments, when performing a performance test on the software of the previous version, the performance test scenario may be solidified, the solidified performance test scenario is set as a reference performance test scenario of the performance to be tested, and the software of the next version may perform a test in the reference performance test scenario. The reference performance test scenario may also be a comparison of multiple performance test scenarios, and the performance test scenario that best matches the performance test service is selected as the reference performance test scenario.
Step S202, testing the software to be tested based on a preset test scene corresponding to the performance to be tested.
The test scenario is a series of action sets simulating a real user business operation process in a performance test process. The performance test scenario corresponds to a type of performance to be tested of the software to be tested.
In the specific implementation, the server determines a preset test scene corresponding to the performance to be tested according to the type of the performance to be tested, and performs performance testing on the software to be tested according to environments such as a performance test script and performance test parameters of the preset test scene. For example, the server may perform a stress test on the software to be tested, observe and monitor the change of the performance of the software to be tested by gradually increasing the load, and finally determine the load critical condition under which the software to be tested will be in a failure state, so as to obtain the maximum service level that the software to be tested can provide.
Step S203, acquiring a performance test report of the software to be tested in a preset test scene, and triggering to perform performance index comparison analysis with a preset baseline report in the preset test scene.
The preset baseline report in the preset test scene is a performance test report aiming at the condition that each index of the performance test meets the performance test requirement of the performance test scene in the specific performance test scene, and each index is in the most state. Performance indicators may include transaction response time, throughput, transaction success rate, resource utilization, and the like.
In the specific implementation, after the software to be tested performs the performance test in the preset test scene, the server obtains a performance test report of the software to be tested in the test scene, wherein the performance test report includes a performance index result of the performance to be tested. In order to know the performance of the to-be-tested software, the server automatically compares the obtained performance test report with a preset baseline report in the test scene, for example, the server may compare the average value of each performance index in the performance test report with that in the preset baseline report, or may respectively compare each performance index in the performance test report with that in the preset baseline report, so as to obtain a comparative analysis result of the to-be-tested performance of the to-be-tested software.
And step S204, comparing the analysis results according to the performance indexes, and acquiring the performance test result of the software to be tested.
The performance test result of the software to be tested can comprise that the test is passed and the test is not passed, and the server can send out a test early warning instruction or a performance tuning instruction according to the actual comparison analysis result when the test is not passed.
In the specific implementation, the server obtains a comparative analysis result of the performance test report and a preset baseline report, determines whether the performance index of the software to be tested is superior to the performance index in the preset baseline report according to the comparative analysis condition of the indexes, and determines that the software to be tested passes the test, sends a test early warning instruction or sends a performance tuning instruction according to different performances of the analysis result. And when the software to be tested passes the test, ending the test. When the server sends out one or two of a test early warning instruction or a performance tuning instruction, the software to be tested needs to be correspondingly adjusted according to the content of the instruction. In some embodiments, the server may set a performance index threshold in the performance test scenario, compare the performance index in the performance test report of the software to be tested with the performance index threshold by using the performance index comparison analysis result in the preset baseline report, when the performance index comparison result exceeds the corresponding performance threshold range, the test does not pass, and the server may send one or both of a test early warning instruction and a performance tuning instruction.
In the performance test method of the software, the performance test report under the test scene is obtained by determining the performance to be tested of the software to be tested, testing the software to be tested based on the test scene corresponding to the performance to be tested, performing performance index comparison analysis on the performance test report and a baseline report preset under the test scene, and obtaining the performance test result of the software to be tested according to the comparison analysis result, so that the performance test report of the software to be tested can be automatically compared with the baseline report of the corresponding test scene to obtain the performance test result, and the efficiency of obtaining the performance test result of the software is improved.
In an embodiment, the obtaining of the performance test report of the software to be tested in the preset test scenario in step S203, and triggering the performance index comparison analysis with the preset baseline report in the preset test scenario further includes:
when the performance test report does not have a preset baseline report under a preset test scene, taking the performance test report as a first baseline report under the preset test scene; the first baseline report is used for performing performance index comparison analysis with the next performance test report of a preset test scene
In this embodiment, when the test item is a new item that has not been tested, or the test item has been tested but does not satisfy the condition for generating the baseline report, the corresponding test scenario may not include the preset baseline report. When there is no preset baseline report in the preset test scenario, the server may use the performance test report generated after the test of the software to be tested as the first baseline report in the preset performance test scenario. When the next performance test report is generated in the test scenario, the next performance test report can be compared with the baseline report to serve as a basis for judging the performance of the software. In some embodiments, the server may set a condition for generating a baseline report in a preset test slave scenario, for example, a performance index in a performance test report of the software to be tested needs to meet a basic performance index of the preset test scenario and can be used as the baseline report, so as to ensure that the software that finally passes the performance test all meets a basic requirement of the software performance set in the preset test scenario.
According to the scheme of the embodiment, when the preset test scene does not have the preset baseline report, the performance test report of the software to be tested is used as the first baseline report for comparison with the next performance test report of the test scene, and the source of the preset baseline report in the preset test scene is provided, so that the next performance test has a comparison reference, and the accuracy of obtaining the software performance test result is improved.
In an embodiment, before the triggering in step S203 and performing performance index comparison analysis on the preset baseline report in the preset test scenario, the method may further include the following steps:
acquiring basic performance indexes of a preset test scene; comparing the performance index in the performance test report with the basic performance index; and determining whether to trigger performance index comparison analysis of the performance test report of the software to be tested and a preset baseline report under a preset test scene according to the basic performance index comparison result.
In this embodiment, the preset test scenario includes a basic performance index, which is used to indicate a minimum requirement of the preset test scenario on a performance index of the software to be tested. The base performance indicator may be comparable to or not better than the performance indicator in the preset baseline report of the preset test scenario. In order to improve the software testing efficiency, the basic performance index can be used as the minimum requirement for the software performance test, and when the performance index in the performance test report does not meet the requirement of the basic performance index, subsequent tests are not needed. In specific implementation, after the server obtains the performance test report of the software to be tested, the performance index in the performance test report can be compared with the basic performance index, and whether the performance test report needs to be executed or not is determined according to the comparison and analysis result, so that the performance index comparison and analysis is performed on the performance test report and the preset baseline report in the preset scene. For example, the server may compare the performance test report with the average value of each index in the basic performance indexes, or may compare the performance test report with each performance index in the basic performance indexes, respectively, to obtain a comparative analysis result of the basic performance of the software to be tested.
According to the scheme of the embodiment, after the performance indexes of the performance test report are compared with the basic performance indexes of the preset test scene, whether the performance indexes of the performance test report are compared with the performance indexes of the preset baseline report needs to be determined, and the efficiency of obtaining the software performance test result is improved.
In one embodiment, the method may further determine whether to trigger performance index comparison analysis between a performance test report of the software to be tested and a preset baseline report in a preset test scenario, including:
when the basic performance index comparison result is that the performance index in the performance test report does not accord with any one of the basic performance indexes, feeding back a performance tuning instruction; and the performance tuning instruction is used for indicating performance optimization of the performance to be tested of the software to be tested and then performing performance testing.
In this embodiment, the basic performance index may be equivalent to or not better than the performance index in the preset baseline report of the preset test scenario. When the performance index in the performance test report does not accord with any one of the basic performance indexes, the performance of the software to be tested needs to be improved, and the comparison with a preset baseline report is not needed. And the server feeds back a performance tuning instruction, indicates that the performance to be tested of the software to be tested is optimized and then is tested, and the software to be tested does not pass the test.
According to the scheme of the embodiment, after the performance indexes of the performance test report are compared with the basic performance indexes of the preset test scene, when the performance indexes in the performance test report do not accord with any one of the basic performance indexes, the software test fails and the comparison with the preset baseline report is not carried out, so that the efficiency of obtaining the software performance test result is improved.
In one embodiment, the method may further determine whether to trigger performance index comparison analysis between a performance test report of the software to be tested and a preset baseline report in a preset test scenario, including:
and when the basic performance index comparison result indicates that the performance index in the performance test report meets the basic performance index, determining to trigger the performance index comparison analysis of the performance test report of the software to be tested and a preset baseline report under a preset test scene.
In this embodiment, when the performance index in the performance test report meets the basic performance index, which indicates that the performance of the software to be tested meets the basic requirement, the performance index is compared with the performance index of the preset baseline report to determine whether the software to be tested passes the test.
According to the scheme of the embodiment, the performance test report meeting the basic performance index requirement in the preset test scene can be triggered to be compared with the preset baseline report, the frequency of comparing the performance test report with the baseline report can be reduced, and the efficiency of obtaining the software performance test result is improved.
Further, comparing the analysis result according to the performance index in step S204 to obtain a performance test result of the software to be tested, further comprising:
acquiring a performance index threshold value of a preset test scene according to a preset baseline report; and when the performance index comparison analysis result is not contained in the performance index threshold range, sending a test early warning instruction and a performance tuning instruction.
In this embodiment, the test early warning instruction is used to indicate that the performance test result of the software to be tested conforms to the basic performance index of the preset test scenario, but is not better than the preset baseline report; and the performance tuning instruction is used for indicating performance optimization of the performance to be tested of the software to be tested and then performing performance testing. The performance index threshold may be a performance index fluctuation threshold in a preset test scenario, and is used to determine a floatable range of the performance index comparison result.
In a specific implementation, the server may store each performance index included in the preset baseline report as a baseline index, and determine the performance index threshold according to the baseline index, or the server may determine the performance index threshold according to one or more factors such as the preset baseline report, a test purpose, or characteristics of a preset test scenario. And when the performance index comparison analysis result is not contained in the performance index threshold range, the software to be tested fails to be tested, and the server sends out a test early warning instruction to indicate that the software performance accords with the basic performance index but is not superior to the preset baseline report. And simultaneously sending a performance tuning instruction to indicate that performance optimization needs to be carried out on the to-be-tested performance of the to-be-tested software, and then carrying out performance test. In some embodiments, the test pre-warning instruction and the performance tuning instruction may further include a comparison result of each performance index or a comprehensive comparison result of the to-be-tested performance of the to-be-tested software.
According to the scheme of the embodiment, the type of the performance test result of the software to be tested is further determined by setting the performance test index threshold of the preset test scene, and corresponding indication is sent, so that related personnel can quickly identify the performance test result of the software, and the efficiency of obtaining the performance test result of the software is improved.
In an embodiment, after obtaining the performance index threshold of the preset test scenario according to the preset baseline report, the method further includes:
and when the performance index comparative analysis result is contained in the performance index threshold range of the preset test scene, and/or the performance index of the performance test report is superior to that of the preset baseline report, taking the performance test report as a new baseline report in the preset test scene.
In this embodiment, the baseline report in the preset test scenario is a performance test report that meets the basic performance requirements and has the optimal performance indexes among the performance test reports obtained in the preset test scenario, and with multiple executions of the test, when a performance test report with better indexes appears, the better performance test report is replaced by a new baseline report. In some embodiments, the performance analysis result may also include a performance test report within a threshold range of the performance index as a baseline report in the preset test scenario.
According to the scheme of the embodiment, the better performance test report is set as the new baseline report, so that the performance index requirements of the preset test scene are continuously improved, and the efficiency of software performance test and performance optimization is improved.
In one embodiment, as shown in fig. 3, a method for testing the performance of software is provided, the method comprising:
step S301, the server determines the performance to be tested of the software to be tested.
Step S302, the server tests the software to be tested based on the preset test scene corresponding to the performance to be tested.
Step S303, the server obtains a performance test report of the software to be tested in a preset test scene.
Step S304, the server acquires basic performance indexes of a preset test scene; the basic performance index is used for indicating the minimum requirement of a preset test scene on the performance index of the software to be tested; comparing the performance index in the performance test report with the basic performance index; and determining whether to trigger performance index comparison analysis of the performance test report of the software to be tested and a preset baseline report under a preset test scene according to the basic performance index comparison result.
Step S305, when the basic performance index comparison result is that the performance index in the performance test report does not accord with any one of the basic performance indexes, the server feeds back a performance tuning instruction; the performance tuning instruction is used for indicating performance optimization of the performance to be tested of the software to be tested and then performing performance testing; and when the basic performance index comparison result indicates that the performance index in the performance test report meets the basic performance index, determining to trigger the performance index comparison analysis of the performance test report of the software to be tested and a preset baseline report under a preset test scene.
Step S306, the server obtains the performance index threshold value of the preset test scene according to the preset baseline report.
Step S307, when the performance index comparison analysis result is not included in the performance index threshold range, the server sends out a test early warning instruction and a performance tuning instruction; the test early warning instruction is used for indicating that the performance test result of the software to be tested accords with the basic performance index of a preset test scene but is not superior to the preset baseline report; the performance tuning instruction is used for indicating performance optimization of the performance to be tested of the software to be tested and then performing performance testing; and when the performance index comparative analysis result is contained in the performance index threshold range of the preset test scene and/or the performance index of the performance test report is superior to that of the preset baseline report, taking the performance test report as a new baseline report in the preset test scene.
In the scheme of the embodiment, the performance to be tested of the software to be tested is determined, the software to be tested is tested on the basis of the test scene corresponding to the performance to be tested, the performance test report under the test scene is obtained, after the performance test report meets the basic performance index in the preset test scene, the performance test report and the baseline report preset under the test scene are subjected to performance index comparative analysis, the performance test result of the software to be tested is obtained according to the comparative analysis result, the performance test report with the performance test result superior to that of the preset baseline report is taken as the new baseline report of the test scene, the method enables the performance test report of the software to be tested to be automatically compared with the baseline report of the corresponding test scene to obtain the performance test result, the iteration of the test baseline report is realized, and the efficiency of the performance test and the performance tuning of the software is improved, and the efficiency of obtaining the software performance test result is also improved.
It should be understood that although the steps in the flowcharts of fig. 2 to 3 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2 to 3 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least some of the other steps or stages.
In one embodiment, as shown in fig. 6, a performance testing apparatus for software is provided, the apparatus 600 comprising:
a performance determining module 601, configured to determine a performance to be tested of the software to be tested;
a software testing module 602, configured to test software to be tested based on a preset testing scenario corresponding to performance to be tested;
an index comparison module 603, configured to obtain a performance test report of the software to be tested in a preset test scenario, and trigger a performance index comparison analysis with a preset baseline report in the preset test scenario;
the result obtaining module 604 is configured to compare the analysis result with the performance index to obtain a performance test result of the software to be tested.
In one embodiment, the index comparison module 603 is further configured to, when there is no preset baseline report in the preset test scenario, take the performance test report as a first baseline report in the preset test scenario; the first baseline report is used for performing performance index comparison analysis with the next performance test report of the preset test scene.
In one embodiment, the apparatus 600 further comprises: the basic performance index unit is used for acquiring basic performance indexes of a preset test scene; the basic performance index is used for indicating the minimum requirement of a preset test scene on the performance index of the software to be tested; comparing the performance index in the performance test report with the basic performance index; and determining whether to trigger performance index comparison analysis of the performance test report of the software to be tested and a preset baseline report under a preset test scene according to the basic performance index comparison result.
In one embodiment, the basic performance index unit is further configured to feed back the performance tuning instruction when the basic performance index comparison result indicates that the performance index in the performance test report does not conform to any one of the basic performance indexes; and the performance tuning instruction is used for indicating performance optimization of the performance to be tested of the software to be tested and then performing performance testing.
In an embodiment, the basic performance index unit is further configured to determine to trigger performance index comparison analysis between the performance test report of the software to be tested and a preset baseline report in a preset test scenario when the basic performance index comparison result indicates that the performance index in the performance test report satisfies the basic performance index.
In an embodiment, the result obtaining module 604 is further configured to obtain a performance index threshold of a preset test scenario according to a preset baseline report; when the performance index comparison analysis result is not contained in the performance index threshold range, sending a test early warning instruction and a performance tuning instruction; the test early warning instruction is used for indicating that the performance test result of the software to be tested accords with the basic performance index of a preset test scene but is not superior to a preset baseline report; and the performance tuning instruction is used for indicating performance optimization of the performance to be tested of the software to be tested and then performing performance testing.
In one embodiment, the apparatus 600 further comprises: and the baseline report iteration module is used for taking the performance test report as a new baseline report in the preset test scene when the performance index comparison analysis result is contained in the performance index threshold range of the preset test scene and/or the performance index of the performance test report is superior to that of the preset baseline report.
For specific limitations of the software performance testing apparatus, reference may be made to the above limitations of the software performance testing method, which are not described herein again. The modules in the software performance testing device may be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In order to more clearly illustrate the solution provided by the present application, the software performance testing method provided by the present application is applied to a performance testing platform for description, and fig. 4 and 5 are flowcharts corresponding to the software performance testing platform. The main flow of the performance testing platform comprises the following parts:
s1: and (5) managing the project. And establishing a performance test project which comprises performance test requirements, server resource requirements, transaction information needing to be tested and service information deployment.
S2: and (5) deployment management. The deployment service package source, the deployment destination server, and the deployment replacement-required configuration information are set according to the deployment service information in S1. After the platform is set, deployment can be started, and the platform can automatically deploy services and start module services.
S3: and managing a performance test scene. Creating a performance test scene, including a performance test script, performance test execution parameters (including the number of concurrent users, the execution time of pressure test, the incremental mode of concurrent users) and the like; a plurality of performance test scenes can be created according to different performance test service models. And meanwhile, setting a benchmark performance test scene.
S4: and (6) executing the test. When the tested services are deployed and started and a performance test scene is created, a performance test is executed, and the system can automatically collect performance index data through a tool.
S5/S10: and managing the test report. And when the test execution is finished, the system can automatically generate a performance test report and store various performance index data in the test report.
S6: combining the steps of S4 and S5, after the test is executed for many times, setting the performance test report meeting the requirement of the basic performance index and under the condition that each performance index is optimal as a performance test baseline report under the test scene, and using the performance test baseline report as the reference of the subsequent iteration version. Wherein, only one effective baseline report can be set in one performance test scene.
S7: setting and setting a performance index threshold range according to a performance test result, wherein the performance index comprises TPS, transaction response time, throughput, resource utilization rate (CPU, memory, disk, bandwidth) and the like.
Software version iteration is started with the following steps:
s8: and (5) deployment management. When the software version is iterated, project requirements and deployment information are obtained from information maintained by the previous version, and the platform automatically allocates idle server resources and performs module deployment and service starting.
S9: and (6) executing the test. In line with S4, the performance test scenario for test execution comes from the benchmark performance test scenario set in S3.
S11: comparing and early warning the test reports, after the test report is generated in S10, automatically comparing the test report with the baseline report set in S6, calculating the fluctuation value of the average value of each performance index in the two test reports, evaluating whether the comparison result of the performance indexes exceeds the threshold range by combining the threshold range of the performance indexes set in S7, and if the comparison result exceeds the threshold range, giving out a result that the performance is abnormal, giving an early warning to the related personnel of the project, and sending a performance early warning instruction. If the baseline report is not set before, comparison is not carried out, and the test result of the software to be tested is judged to meet the requirement according to the basic performance index condition.
S12: like S6, when the performance test results in the iterative software version are also expected or the performance is improved, the baseline report may be reset, the previous baseline report may be cancelled, and the latest baseline report may be used for subsequent version iterations.
In the scheme of the embodiment, the performance test process is managed by establishing the performance test items and the performance test scenes, when no baseline report exists in the performance test scenes, the performance test report meeting the basic performance index requirements of the performance test scenes is taken as the baseline report, the performance index threshold is set according to the test result, when the subsequent software performance test is required, a tester only needs to select the reference performance test scenes to execute the performance test, the system automatically compares the performance test report generated at the time with the baseline report when the performance test is finished, the platform automatically judges whether the comparison value of each performance index is in the preset threshold range, the platform sends out the performance early warning instruction and the performance tuning instruction according to whether the comparison result of the performance index meets the threshold range, and the efficiency of the software performance test and the performance tuning is improved, and the efficiency of obtaining the software performance test result is also improved.
In one embodiment, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the above method embodiments when the processor executes the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method for testing the performance of software, the method comprising:
determining the performance to be tested of the software to be tested;
testing the software to be tested based on a preset test scene corresponding to the performance to be tested;
acquiring a performance test report of the software to be tested in the preset test scene, and triggering the performance test report to perform performance index comparison analysis with a preset baseline report in the preset test scene;
and comparing the analysis result according to the performance index to obtain a performance test result of the software to be tested.
2. The method according to claim 1, wherein the acquiring a performance test report of the software under test in the preset test scenario and triggering a performance index comparison analysis with a preset baseline report in the preset test scenario comprises:
when the preset baseline report under the preset test scene does not exist, taking the performance test report as a first baseline report under the preset test scene; and the first baseline report is used for performing performance index comparison analysis with the next performance test report of the preset test scene.
3. The method of claim 1, wherein before the triggering performance metric comparison analysis with a predetermined baseline report in the predetermined test scenario, the method further comprises:
acquiring basic performance indexes of the preset test scene; the basic performance index is used for indicating the minimum requirement of the preset test scene on the performance index of the software to be tested;
comparing the performance index in the performance test report with the basic performance index;
and determining whether to trigger performance index comparison analysis of the performance test report of the software to be tested and a preset baseline report under a preset test scene according to a basic performance index comparison result.
4. The method of claim 3, wherein the determining whether to trigger performance index comparison analysis of the performance test report of the software under test and a preset baseline report under the preset test scenario according to a basic performance index comparison result comprises:
when the basic performance index comparison result indicates that the performance index in the performance test report does not accord with any one of the basic performance indexes, feeding back a performance tuning instruction; and the performance tuning instruction is used for indicating performance optimization of the performance to be tested of the software to be tested and then performing performance testing.
5. The method of claim 3, wherein the determining whether to trigger performance index comparison analysis of the performance test report of the software under test and a preset baseline report under the preset test scenario according to a basic performance index comparison result comprises:
and when the basic performance index comparison result shows that the performance indexes in the performance test report meet the basic performance index, determining to trigger performance index comparison analysis of the performance test report of the software to be tested and a preset baseline report under a preset test scene.
6. The method according to claim 5, wherein the obtaining the performance test result of the software to be tested according to the performance index comparison analysis result comprises:
acquiring a performance index threshold value of the preset test scene according to the preset baseline report;
when the performance index comparison analysis result is not contained in the performance index threshold range, sending a test early warning instruction and a performance tuning instruction; the test early warning instruction is used for indicating that the performance test result of the software to be tested accords with the basic performance index of the preset test scene but is not superior to the preset baseline report; and the performance tuning instruction is used for indicating performance optimization of the performance to be tested of the software to be tested and then performing performance testing.
7. The method of claim 6, wherein after obtaining the performance indicator threshold for the predetermined test scenario from the predetermined baseline report, the method further comprises:
and when the performance index comparative analysis result is contained in the performance index threshold range of the preset test scene and/or the performance index of the performance test report is superior to that of the preset baseline report, taking the performance test report as a new baseline report in the preset test scene.
8. An apparatus for testing performance of software, the apparatus comprising:
the performance determining module is used for determining the performance to be tested of the software to be tested;
the software testing module is used for testing the software to be tested based on a preset testing scene corresponding to the performance to be tested;
the index comparison module is used for acquiring a performance test report of the software to be tested in the preset test scene, and triggering performance index comparison analysis with a preset baseline report in the preset test scene;
and the result acquisition module is used for comparing the analysis result according to the performance index and acquiring the performance test result of the software to be tested.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010661925.6A CN111984527A (en) | 2020-07-10 | 2020-07-10 | Software performance testing method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010661925.6A CN111984527A (en) | 2020-07-10 | 2020-07-10 | Software performance testing method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111984527A true CN111984527A (en) | 2020-11-24 |
Family
ID=73438318
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010661925.6A Pending CN111984527A (en) | 2020-07-10 | 2020-07-10 | Software performance testing method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111984527A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114968747A (en) * | 2022-07-12 | 2022-08-30 | 杭州数列网络科技有限责任公司 | Automatic extreme pressure test performance test method and device, electronic equipment and storage medium |
| CN114968741A (en) * | 2022-05-27 | 2022-08-30 | 重庆长安汽车股份有限公司 | Performance test method, system, equipment and medium based on scene platform |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106201856A (en) * | 2015-05-04 | 2016-12-07 | 阿里巴巴集团控股有限公司 | A kind of multi version performance test methods and device |
| CN106855844A (en) * | 2015-12-09 | 2017-06-16 | 北京神州泰岳软件股份有限公司 | A kind of performance test methods and system |
| CN107590069A (en) * | 2017-08-16 | 2018-01-16 | 国家电网公司 | A kind of software definition storage performance evaluating method |
| CN109726100A (en) * | 2018-04-19 | 2019-05-07 | 平安普惠企业管理有限公司 | Application performance test method, apparatus, equipment and computer readable storage medium |
| CN110134574A (en) * | 2019-04-19 | 2019-08-16 | 平安普惠企业管理有限公司 | A kind of software performance testing method, system, medium and electronic equipment |
| CN110572280A (en) * | 2019-08-22 | 2019-12-13 | 北京新宇航星科技有限公司 | network monitoring method and system |
-
2020
- 2020-07-10 CN CN202010661925.6A patent/CN111984527A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106201856A (en) * | 2015-05-04 | 2016-12-07 | 阿里巴巴集团控股有限公司 | A kind of multi version performance test methods and device |
| CN106855844A (en) * | 2015-12-09 | 2017-06-16 | 北京神州泰岳软件股份有限公司 | A kind of performance test methods and system |
| CN107590069A (en) * | 2017-08-16 | 2018-01-16 | 国家电网公司 | A kind of software definition storage performance evaluating method |
| CN109726100A (en) * | 2018-04-19 | 2019-05-07 | 平安普惠企业管理有限公司 | Application performance test method, apparatus, equipment and computer readable storage medium |
| CN110134574A (en) * | 2019-04-19 | 2019-08-16 | 平安普惠企业管理有限公司 | A kind of software performance testing method, system, medium and electronic equipment |
| CN110572280A (en) * | 2019-08-22 | 2019-12-13 | 北京新宇航星科技有限公司 | network monitoring method and system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114968741A (en) * | 2022-05-27 | 2022-08-30 | 重庆长安汽车股份有限公司 | Performance test method, system, equipment and medium based on scene platform |
| CN114968747A (en) * | 2022-07-12 | 2022-08-30 | 杭州数列网络科技有限责任公司 | Automatic extreme pressure test performance test method and device, electronic equipment and storage medium |
| CN114968747B (en) * | 2022-07-12 | 2022-10-28 | 杭州数列网络科技有限责任公司 | Automatic extreme pressure test performance test method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106294120B (en) | Method, apparatus and computer program product for testing code | |
| CN109542712B (en) | Server pressure testing method and device, computer equipment and storage medium | |
| CN110377454B (en) | Data verification method and device, computer equipment and storage medium | |
| CN108492005B (en) | Project data processing method and device, computer equipment and storage medium | |
| CN112559365A (en) | Test case screening method and device, computer equipment and storage medium | |
| CN110083514B (en) | Software test defect evaluation method and device, computer equipment and storage medium | |
| CN109408371A (en) | Software defect analyzes input method, device, computer equipment and storage medium | |
| CN111984527A (en) | Software performance testing method, device, equipment and medium | |
| CN110855477A (en) | Link log monitoring method and device, computer equipment and storage medium | |
| CN109522181B (en) | Performance test method, device and equipment of distributed storage system | |
| CN113111000A (en) | Continuous integration automation test system and method, electronic device and storage medium | |
| CN112559364A (en) | Test case generation method and device, computer equipment and storage medium | |
| CN111124911A (en) | Automatic testing method, device, equipment and readable storage medium | |
| CN110798376A (en) | Interface testing method and device, computer equipment and storage medium | |
| CN112363937A (en) | Method and device for testing differential coverage rate, computer equipment and storage medium | |
| CN114385485A (en) | Interface test method, device, equipment and storage medium | |
| CN108776642B (en) | Test report generation method and device, computer equipment and storage medium | |
| CN110971478B (en) | Pressure measurement method and device for cloud platform service performance and computing equipment | |
| CN111767218A (en) | Automatic testing method, equipment and storage medium for continuous integration | |
| CN112416648A (en) | Data verification method and device | |
| CN115017018A (en) | Method and device for testing performance of server and computer readable storage medium | |
| CN111209197B (en) | Application continuous integration test method, system, equipment and storage medium | |
| CN114385498A (en) | Performance test method, system, computer equipment and readable storage medium | |
| CN113485933A (en) | Automatic testing method and distributed system | |
| CN110162467B (en) | Software testing method, device, computer equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Applicant after: Zhaolian Consumer Finance Co.,Ltd. Address before: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Applicant before: MERCHANTS UNION CONSUMER FINANCE Co.,Ltd. Country or region before: China |
|
| CB02 | Change of applicant information |