WO2023273561A1

WO2023273561A1 - Test method and apparatus, electronic device, and storage medium

Info

Publication number: WO2023273561A1
Application number: PCT/CN2022/088910
Authority: WO
Inventors: 李雄清; 关健; 李永; 任政委; 周东琳
Original assignee: 中国民航信息网络股份有限公司
Priority date: 2021-06-30
Filing date: 2022-04-25
Publication date: 2023-01-05
Also published as: CN113485918A

Abstract

The present invention provides a test method and apparatus, an electronic device, and a storage medium, applied to a distributed performance test platform. The method comprises: in response to a Seed Job selection instruction, determining a Seed Job indicated by the Seed Job selection instruction as a target Seed Job; executing the target Seed Job, such that the target Seed Job generates a test execution Job on the basis of configuration information; executing the test execution Job, such that the test execution Job acquires a corresponding test script according to the configuration information; and executing the test script to direct test traffic to a tested service system indicated by the configuration information to perform a test until the tested service system completes the test, acquiring corresponding test monitoring data by means of the test execution Job, and generating a test report on the basis of the test monitoring data. On the basis of the present invention, the problems in the prior art that the overall performance of the service system cannot be accurately reflected and it is also inconvenient to maintain the service system can be solved.

Description

A test method, device, electronic equipment and storage medium

This application claims the priority of the Chinese patent application with the application number 202110739282.7 and the title of the invention "a test method, device, electronic equipment and storage medium" submitted to the China Patent Office on June 30, 2021, the entire contents of which are incorporated by reference incorporated in this application.

technical field

The present invention relates to the field of computer technology, more specifically, to a test method, device, electronic equipment and storage medium.

Background technique

With the development of the network, people rely more and more on the network, which brings serious system load to the system that provides network services. The performance of the service system will seriously affect the interests of the enterprise. Therefore, the performance test of the service system is more are gaining more and more attention from the industry.

In the existing testing process, testers of each project select performance testing tools for performance testing according to the projects they are responsible for. However, in this existing method, testers of different projects may have different selections of performance testing tools, as well as inconsistencies in performance testing standards, scripts, execution processes, test strategies, and test reports. It accurately reflects the overall performance of the service system, and it is not convenient to maintain the service system.

Contents of the invention

In view of this, the present invention provides a testing method, device, electronic equipment and storage medium to solve the problem that the prior art cannot accurately reflect the overall performance of the service system and is not convenient for maintaining the service system.

The first aspect of the present invention discloses a test method applied to a distributed performance test platform, the method comprising:

In response to the Seed Job selection instruction, the Seed Job indicated by the Seed Job selection instruction is determined as the target Seed Job;

Executing the target Seed Job so that the target Seed Job generates a test execution Job based on configuration information, wherein the configuration information is pre-configured by the user according to project requirements;

Execute the test execution Job, so that the test execution Job obtains a corresponding test script according to the configuration information;

Execute the test script to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, obtain corresponding test monitoring data through the test execution Job, and based on the test Monitor data to generate test reports.

The second aspect of the present invention discloses a test device, which is applied to a distributed performance test platform, and the device includes:

The first determining unit is configured to respond to a Seed Job selection instruction, and determine the Seed Job indicated by the Seed Job selection instruction as a target Seed Job;

A first execution unit, configured to execute the target Seed Job, so that the target Seed Job generates a test execution Job based on configuration information, wherein the configuration information is pre-configured by the user according to project requirements;

The second execution unit is used to execute the test execution job, so that the test execution job obtains a corresponding test script according to the configuration information;

The third execution unit is configured to execute the test script to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, and obtain corresponding test monitoring through the test execution Job data, and generate a test report based on the test monitoring data.

The third aspect of the present invention discloses an electronic device, including: a processor and a memory, the processor and the memory are connected through a communication bus; wherein, the processor is used to call and execute the program stored in the memory; the memory is used to store the program, The program is used to implement the testing method disclosed in the first aspect of the present invention.

The fourth aspect of the present invention discloses a computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium, and the computer-executable instructions are used to execute the testing method disclosed in the above-mentioned first aspect of the present invention.

The present invention provides a test method, device, electronic equipment and storage medium. There are multiple Seed Jobs preset. Users can select corresponding Seed Jobs from the preset multiple Seed Jobs according to project requirements. After the user determines the selected Seed Job After Job, in response to the Seed Job selection instruction, determine the Seed Job indicated by the Seed Job selection instruction as the target Seed Job, and execute the target Seed Job so that the target Seed Job generates a corresponding test execution Job based on the configuration information pre-configured by the user. After the generated test is executed, the test script corresponding to the configuration information can be obtained. Finally, by executing the obtained test script, the test traffic is directed to the service system under test indicated by the configuration information for testing until the service system under test completes the test. Obtain the corresponding test monitoring data through the test execution Job, and generate a test report based on the test monitoring data. The technical solution provided by the present invention supports multiple testing strategies at the same time by pre-setting a variety of Seed Jobs, and the user can select a corresponding Seed Job from a variety of pre-set Seed Jobs according to the project requirements, without having to select different ones according to different projects. Model performance testing tools, thus solving the situation in the prior art that due to the different types of performance testing tools and the inconsistency of data such as performance testing standards, scripts, execution processes, test strategies, and test reports, it cannot accurately reflect The overall performance of the service system and the inconvenient maintenance of the service system.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention, and those skilled in the art can also obtain other drawings according to the provided drawings without creative work.

The above and other features, advantages and aspects of the disclosed embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.

Fig. 1 is a schematic structural diagram of a distributed performance testing platform provided by an embodiment of the present invention;

Fig. 2 is a schematic flow chart of a test method provided by an embodiment of the present invention;

Fig. 3 is a schematic flow chart of another testing method provided by the embodiment of the present invention;

Fig. 4 is a schematic structural view of a test device provided by an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

detailed description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein; It is for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments disclosed in the present invention are only for exemplary purposes, and are not intended to limit the scope of protection disclosed in the present invention.

As used herein, the term "comprise" and its variations are open-ended, ie "including but not limited to". The term "based on" is "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one further embodiment"; the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms will be given in the description below.

It should be noted that concepts such as "first" and "second" mentioned in the disclosure of the present invention are only used to distinguish different devices, modules or units, and are not used to limit the functions performed by these devices, modules or units. sequence or interdependence.

It should be noted that the modifications of "one" and "multiple" mentioned in the disclosure of the present invention are illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "a or more".

Referring to Fig. 1, it shows a schematic structural diagram of a distributed performance testing platform provided by an embodiment of the present invention, the distributed performance testing platform 100 includes a configuration service module 101, a Jenkins control machine 102, a test script management module 103, a report service Module 104, Jenkins node machine cluster 105, multiple tested service systems 106 and monitoring service module 107.

Referring to FIG. 2, it shows a schematic flow chart of a test method provided by an embodiment of the present invention. The test method is applied to the distributed performance test platform shown in FIG. 1, and the test method specifically includes the following steps:

S201: In response to the Seed Job selection instruction, determine the Seed Job indicated by the Seed Job selection instruction as the target Seed Job.

In step S201, test technicians can build templates of multiple test execution jobs corresponding to each project on the Jenkins control machine of the distributed performance testing platform, that is, build multiple seed jobs corresponding to each project.

It should be noted that Seed Job can execute job templates for single-interface tests, single-scenario tests, mixed-scenario tests, mixed-scenario RPS mode tests, load tests, and optimal QPS for single interfaces. Automatic discovery of test execution Job templates, load point automatic discovery of test execution jobs.

During the specific execution of step S201, the user can select the corresponding Seed Job from the various Seed Jobs preset on the Jenkins control machine of the distributed performance testing platform according to the project requirements, and after the user confirms the selected Seed Job, respond to the Seed Job Select an instruction, and determine the Seed Job indicated by the Seed Job selection instruction as the target Seed Job.

It should be noted that the user here can be a tester or a test group.

S202: Executing the target Seed Job, so that the target Seed Job generates a test execution Job based on configuration information, wherein the configuration information is pre-configured by the user according to project requirements.

In step S202, the user can configure corresponding configuration information according to project requirements on the configuration service module of the distributed performance testing platform.

It should be noted that the configuration information includes the relevant information of the test team (user), the git location and branch of the test script, the access entry address of the service system under test, that is, the performance test traffic entry, the name, version and test report of the service system under test Archive address, application name and version information of dependent services, application name and version information of middleware, monitoring service address and monitored service system name under test, IP address, test node machine information (such as IP, login account information).

It should be noted that the information of multiple test node machines can be configured so as to achieve high concurrent processing by triggering multiple test node machines at the same time.

During the specific execution of step S202, after determining the target Seed Job, the user can input configuration address information on the selected Seed Job, and click the Execute button to execute the target Seed Job, so that the target Seed Job can be configured according to the input configuration address information , obtain the corresponding configuration information, analyze the obtained configuration information, determine the name of the test execution job to be created, and judge whether there is a test execution job corresponding to the name of the test execution job to be created. If it exists, update the current one according to the configuration information There is a test execution job corresponding to the name of the test execution job to be created. If it does not exist, create a new test execution job according to the configuration information.

Further, in the embodiment of the present application, while executing the target Seed Job so that the target Seed Job generates a test execution Job based on the configuration information, the engine construction Job can also be updated based on the corresponding parameters of the generated test execution Job, using The updated engine construction job obtains the code of the engine construction job from Git of the test engine, and runs the acquired code of the engine construction job to obtain the test engine Bundle.

S203: Execute the test execution job, so that the test execution job acquires a corresponding test script according to the configuration information.

In the process of specifically executing step S203, after executing the target Seed Job so that the target Seed Job generates a test execution job based on the configuration information, the generated test execution job can be executed so that the test execution job can be executed according to the test script in the configuration information The git location where it is located obtains the corresponding test script from the test script management module.

S204: Execute the test script to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, obtain corresponding test monitoring data through the test execution Job, and generate a test report based on the test monitoring data.

In the process of specifically executing step S204, after executing the test execution Job, the test execution Job is made to obtain the corresponding test script according to the configuration information, and automatically executes the test script based on the test engine Bundle to trigger the test node machine information indication in the configuration information The test node machine, direct the test traffic to the service system under test indicated by the information of the test node machine for testing. When the monitoring service module detects that the service system under test has completed the test, it triggers the test execution Job according to the name and IP of the service system under test being monitored. address, and obtain the corresponding test monitoring data, so that the report service module can integrate various information in the obtained test monitoring data to generate a test report and store it in the database. Wherein, the test monitoring data includes test model data, monitoring information, test scenario data, test strategy, Gatling result information and execution information generated during the execution of the test execution Job.

In the embodiment of the present application, after the test report is generated, the generated test report can be displayed for the user to view, and a comprehensive test report can also be generated according to multiple test reports obtained from multiple tests. Among them, the test node machine is pre-set in the Jenkins node machine cluster.

The present invention provides a testing method, in which a variety of Seed Jobs are pre-set, and the user can select a corresponding Seed Job from the various pre-set Seed Jobs according to project requirements, and respond to the Seed Job selection instruction after the user determines the selected Seed Job , determine the Seed Job indicated by the Seed Job selection command as the target SeedJob, execute the target Seed Job, so that the target Seed Job generates a corresponding test execution Job based on the configuration information pre-configured by the user, and by executing the generated test execution, you can obtain the same The test script corresponding to the configuration information, and finally execute the obtained test script to direct the test traffic to the service system under test indicated by the configuration information for testing, until the service system under test completes the test, and obtain the corresponding test monitoring through the test execution Job data, and generate test reports based on test monitoring data. The technical solution provided by the present invention supports multiple testing strategies at the same time by presetting a variety of Seed Jobs, and users can select the corresponding Seed Job from the various pre-set Seed Jobs according to project requirements, without having to choose according to different projects Different types of performance testing tools, thus solving the problem of different types of performance testing tools in the prior art, as well as the inconsistency of performance testing standards, scripts, execution processes, test strategies, test reports and other data, resulting in inability to accurately It reflects the overall performance of the service system and the inconvenient maintenance of the service system.

In the embodiment of the present application, if the Seed Job indicated by the Seed Job selection instruction is a single-interface optimal QPS automatic discovery test execution Job template, see Figure 3, which shows a schematic flow chart of another test method provided by the embodiment of the present invention , the test method specifically includes the following steps:

S301: In response to the SeedJob selection instruction, determine the Seed Job indicated by the SeedJob selection instruction as the target Seed Job, wherein the Seed Job indicated by the SeedJob selection instruction is a single-interface optimal QPS automatic discovery test execution Job template.

During the specific execution of step S301, the user can select the corresponding Seed Job from the various Seed Jobs preset on the Jenkins control machine of the distributed performance testing platform according to the project requirements, and after the user determines the selected Seed Job, respond to the Seed Job selection Instruction, determine the Seed Job indicated by the Seed Job selection instruction as the target Seed Job. Among them, the Seed Job indicated by the Seed Job selection instruction is a single-interface optimal QPS automatic discovery test execution Job template.

S302: Execute the target Seed Job, so that the target Seed Job generates a single-interface optimal QPS discovery test execution job based on the configuration information.

In the process of specifically executing step S302, after determining the target Seed Job, the user can input configuration address information in the selected target Seed Job, and click the Execute button to execute the target Seed Job, so that the target SeedJob can, according to the input configuration address information, Obtain the corresponding configuration information, analyze the obtained configuration information, determine the name of the test execution job to be created, and determine whether there is currently a single-interface optimal QPS discovery test execution job corresponding to the name of the test execution job to be created. If it exists, then Update the currently existing single-interface optimal QPS discovery test execution job corresponding to the name of the test execution job to be created according to the configuration information. If it does not exist, create a new single-interface optimal QPS discovery test execution job according to the configuration information.

Further, in the embodiment of the present application, while executing the target Seed Job so that the target Seed Job generates a single-interface optimal QPS discovery test execution job based on the configuration information, it can also be based on the generated single-interface optimal QPS discovery test execution Update the engine construction job with the corresponding parameters of the job, use the updated engine construction job to obtain the code of the engine construction job from Git of the test engine, and run the acquired code of the engine construction job to obtain the test engine Bundle.

S303: Execute the single-interface optimal QPS discovery test execution job, so that the single-interface optimal QPS discovery test execution job responds to the user's interface selection instruction, determines the interface indicated by the interface selection instruction as the target interface, and obtains the target interface according to the configuration information. For the corresponding test script, if the target interface needs to prepare data scripts, obtain the data preparation scripts returned by other interfaces associated with the target interface, and execute the data preparation scripts to obtain data files.

In step S303, the target Seed Job is executed, so that the target Seed Job can generate the single-interface optimal QPS discovery test execution job based on the configuration information, and then display the generated single-interface optimal QPS discovery test execution job, and the user can use the displayed single interface Select the corresponding interface in the optimal QPS discovery test execution job, and configure the corresponding configuration parameters. Wherein, the configured configuration parameters include at least a preset response time threshold and a preset error rate.

In the process of specifically executing step S303, after executing the target Seed Job so that the target Seed Job generates a single-interface optimal QPS discovery test execution job based on the configuration information, execute the single-interface optimal QPS discovery test execution job to make the single-interface optimal QPS discovery test execution job. The optimal QPS finds that the test execution job responds to the user's interface selection instruction, determines the interface indicated by the interface selection instruction as the target interface, and obtains the test script corresponding to the target interface from the test script management module at the git location of the test script in the configuration information, if The target interface needs to prepare data scripts, obtain the data preparation scripts returned by other interfaces associated with the target interface, and execute the data preparation scripts to obtain data files.

It should be noted that the single-interface optimal QPS discovery test execution job is executed so that the single-interface optimal QPS discovery test execution job responds to the user's interface selection command, determines the interface indicated by the interface selection command as the target interface, and obtains the target interface Corresponding test scripts, if the target interface does not need to prepare data scripts, automatically execute the obtained test scripts to direct the test traffic to the service system under test indicated by the configuration information for testing, when the monitoring service module detects that the service system under test completes the test , trigger the test execution Job to obtain the corresponding test monitoring data according to the monitored service system name and IP address, and generate a test report based on the test monitoring data.

S304: Execute the test script based on the data file to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, obtain the corresponding test monitoring data through the test execution Job, and generate based on the test monitoring data testing report.

In the process of specifically executing step S304, the test script is executed based on the data file to trigger the test node machine indicated by the test node machine information in the configuration information, and point the test flow to the tested service system indicated by the test node machine information for testing. The monitoring service module detects that the service system under test has completed the test, and triggers the test execution Job to obtain the corresponding test monitoring data according to the name and IP address of the service system under test being monitored, so that the reporting service module can obtain the corresponding test monitoring data according to the obtained test monitoring data. Class information is integrated to generate test reports.

It should be noted that the generated test report includes at least the interface QPS, the response time and error rate corresponding to the interface QPS.

S305: Execute the job through the single-interface optimal QPS discovery test to determine whether the interface optimal QPS is found according to the test report and configuration parameters; if found, execute step S306; if not found, execute step S308.

During the specific execution of step S305, after the test report is generated, the interface QPS in the test report, the response time of the interface QPS and the error rate of the interface QPS are obtained through the single interface optimal QPS discovery test execution job, and the interface is judged. Whether the QPS response time is less than the preset response time in the configuration parameters, whether the error rate of the interface QPS is less than the preset error rate in the configuration parameters, and whether the difference between the interface QPS and the last historical interface QPS exceeds the preset deviation threshold.

If the response time of the interface QPS is less than the preset response time in the configuration parameters, the error rate of the interface QPS is less than the preset error rate in the configuration parameters, and the difference between the interface QPS and the last historical interface QPS does not exceed the preset deviation threshold, the interface QPS in this test report may be determined as the interface optimal QPS, and step S306 is executed.

If the response time of the interface QPS is not less than the preset response time in the configuration parameters, the error rate of the interface QPS is not less than the preset error rate in the configuration parameters, or the difference between the interface QPS and the latest interface QPS in history exceeds the preset If the deviation threshold is set, it can be determined that the interface QPS in this test report is not the interface optimal QPS, and step S308 is executed.

S306: Execute the job through a single interface optimal QPS discovery test, and determine whether the difference between the current interface optimal QPS and the last historical interface optimal QPS, and the ratio of the most recent historical interface optimal QPS exceeds a preset threshold. If it exceeds, execute step S307.

During the specific execution of step S306, if the optimal QPS of the interface is found in this test report, the optimal QPS of the single interface is found to execute the job, and the difference between the optimal QPS of the current interface and the most recent optimal QPS of the interface in history is calculated. Difference value, whether the ratio of the most recent interface optimal QPS in history exceeds the preset threshold, if it exceeds, it can be determined that the performance of the service system under test is poor, and the corresponding alarm information will be output; if it is not too high, it can be determined that the service system under test The performance is good, end this test.

In the embodiment of this application, in the process of discovering the optimal QPS of a single interface and executing the job according to the test report and configuration parameters, the optimal QPS of the interface is discovered, the optimal QPS of the interface to be discovered, and the process of discovering the optimal QPS of the interface Related data are stored together in the database.

S307: Execute a job by discovering and testing the optimal QPS of a single interface, and output corresponding alarm information.

During the specific execution of step S307, when it is determined that the difference between the current optimal QPS of the interface and the optimal QPS of the most recent interface in history, and the ratio of the optimal QPS of the most recent interface in history exceeds a preset threshold, it can be determined that the service under test The performance of the system is poor. The optimal QPS of the single interface is used to find the test execution job and output the corresponding alarm information in the form of email.

S308: Execute the job through the discovery test of the optimal QPS of a single interface, and judge whether the error rate corresponding to the current interface QPS exceeds the preset error rate; if the error rate corresponding to the current interface QPS is less than the preset error rate, perform step S309; if the current interface QPS The corresponding error rate is not less than the preset error rate, and step S310 is executed.

During the specific execution of step S308, if the optimal QPS of the interface is not found in this test report, the optimal QPS of the single interface is found to test and execute the job to determine whether the error rate corresponding to the current interface QPS exceeds the preset error rate , if the error rate corresponding to the current interface QPS is less than the preset error rate, perform step S309; if the error rate corresponding to the current interface QPS is not less than the preset error rate, perform step S310.

S309: Increase the access data of the service system under test.

During the specific execution of step S309, when it is determined that the error rate corresponding to the current interface QPS is less than the preset error rate, increase the access data of the service system under test, and return to execute the interface optimal QPS discovery test execution job until it is found Up to the optimal QPS of the interface.

In this embodiment of the application, after the optimal QPS of the interface is discovered, the discovered optimal QPS of the interface and the data generated during the process of discovering the optimal QPS of the interface can be stored in the database, and the optimal QPS of the interface can also be found In the process of optimizing QPS, the QPS of each interface is generated to generate the optimal QPS trend graph, and the generated optimal QPS trend graph is displayed.

S310: Reduce the access data of the service system under test.

During the specific execution of step S310, when it is determined that the error rate corresponding to the current interface QPS is not less than the preset error rate, reduce the access data of the service system under test, and return to execute the interface optimal QPS discovery test execution job, Until the optimal QPS of the interface is found.

In the embodiment of the present application, when the Seed Job indicated by the Seed Job selection instruction is a single-interface optimal QPS automatic discovery test execution Job template, that is, when the target Seed Job is a single-interface optimal QPS automatic discovery test execution Job template, execute Target Seed Job, so that the target Seed Job generates a single-interface optimal QPS discovery test execution job based on the configuration information, and executes the single-interface optimal QPS discovery test execution job, so that the single-interface optimal QPS discovery test execution job obtains according to the configuration information After the corresponding test script is executed, the test script is automatically executed to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, and obtains the corresponding test monitoring data through the test execution Job, and based on the test monitoring The test report is generated from the data, and then the test execution job is found through the optimal QPS of the single interface. According to the test report and configuration parameters, it is judged whether the optimal QPS of the interface is found, so as to judge the performance of the service system under test based on the optimal QPS of the interface found. When it is determined that the performance of the service system under test is poor, corresponding alarm information is output, so that the operation and maintenance personnel can maintain the service system under test according to the output alarm information.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

In some embodiments, the client and the server can communicate using any currently known or future network protocols such as HTTP (HyperText Transfer Protocol, Hypertext Transfer Protocol), and can communicate with digital data in any form or medium The communication (eg, communication network) interconnections. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network of.

The names of messages or information exchanged between multiple devices in the embodiments disclosed in the present invention are for illustrative purposes only, and are not used to limit the scope of these messages or information.

Although operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or to be performed in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous.

It should be understood that the various steps described in the method implementations disclosed in the present invention may be executed in different orders, and/or executed in parallel. Additionally, method embodiments may include additional steps and/or omit performing illustrated steps. The scope of the present disclosure is not limited in this respect.

Computer program code for performing the operations disclosed in the present invention may be written in one or more programming languages, or combinations thereof, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, Also included are conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).

Corresponding to a test method disclosed in the above-mentioned embodiment of the present invention, referring to FIG. 4 , the embodiment of the present invention also provides a schematic structural diagram of a test device, which includes:

The first determination unit 41 is used to respond to the Seed Job selection instruction, and determine the SeedJob indicated by the Seed Job selection instruction as the target Seed Job;

The first execution unit 42 is used to execute the target Seed Job, so that the target Seed Job generates a test execution Job based on configuration information, wherein the configuration information is pre-configured by the user according to project requirements;

The second execution unit 43 is configured to execute the test execution Job, so that the test execution Job obtains a corresponding test script according to the configuration information;

The third execution unit 44 is used to execute the test script to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, obtain the corresponding test monitoring data through the test execution Job, and based on the test Monitor data to generate test reports.

The specific principle and execution process of each unit in the test device disclosed in the above-mentioned embodiments of the present invention are the same as the test methods disclosed in the above-mentioned embodiments of the present invention. Please refer to the corresponding parts in the test methods disclosed in the above-mentioned embodiments of the present invention, which will not be repeated here to repeat.

The present invention provides a test device, which is preset with multiple Seed Jobs, and the user can select the corresponding Seed Job from the preset multiple Seed Jobs according to the project requirements, and respond to the Seed Job selection instruction after the user determines the selected Seed Job , determine the Seed Job indicated by the Seed Job selection command as the target SeedJob, execute the target Seed Job, so that the target Seed Job generates a corresponding test execution Job based on the configuration information pre-configured by the user, and by executing the generated test execution, you can obtain the same The test script corresponding to the configuration information, and finally execute the obtained test script to direct the test traffic to the service system under test indicated by the configuration information for testing, until the service system under test completes the test, and obtain the corresponding test monitoring through the test execution Job data, and generate test reports based on test monitoring data. The technical solution provided by the present invention supports multiple testing strategies at the same time by presetting a variety of Seed Jobs, and users can select the corresponding Seed Job from the various pre-set Seed Jobs according to project requirements, without having to choose according to different projects Different types of performance testing tools, thus solving the problem of different types of performance testing tools in the prior art, as well as the inconsistency of performance testing standards, scripts, execution processes, test strategies, test reports and other data, resulting in inability to accurately It reflects the overall performance of the service system and the inconvenient maintenance of the service system.

Further, the test device provided by the present invention also includes:

The update unit is used to update the engine construction job based on the test execution job.

Optionally, the third execution unit is specifically used to execute the test script based on the test engine Bundle, to trigger the test node machine indicated by the configuration information, and direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test Complete the test, obtain the corresponding test monitoring data through the test execution Job, and generate a test report based on the test monitoring data;

Among them, the test engine Bundle is obtained by using the updated engine construction job to obtain and run the code of the engine construction job from Git of the test engine.

Optionally, the first execution unit is specifically used to execute the target Seed Job, so that the target Seed Job obtains and parses the configuration information according to the input configuration address information, and determines the name of the test execution Job to be created. If it does not exist and to be created Create the test execution Job corresponding to the test execution Job name of the test execution job according to the configuration information;

Wherein, the configuration information is pre-configured by the user, and the configuration address information is input by the user.

Optionally, if the test execution job is a single-interface optimal QPS discovery test execution job, the second execution unit is specifically used to execute the single-interface optimal QPS discovery test execution job, so that the single-interface optimal QPS discovery test execution job responds The user's interface selection command determines the interface indicated by the interface selection command as the target interface, and obtains the test script corresponding to the target interface according to the configuration information. If the target interface needs to prepare data scripts, obtain the preparation returned by other interfaces associated with the target interface Data script, and execute the prepare data script to get the data file;

Correspondingly, the third execution unit is also used to execute the test script based on the data file.

Further, the test device provided by the present invention also includes:

The output unit is used to find the optimal QPS of the interface through the single interface optimal QPS discovery test execution job according to the test report and configuration parameters, and the difference between the current optimal QPS of the interface and the last optimal QPS of the interface in history is the same as the latest historical one When the ratio of the optimal QPS of the interface exceeds the preset threshold, the corresponding alarm information is output;

The access data adjustment unit is used for when the optimal QPS of a single interface is found for the test execution job. According to the test report and configuration parameters, the optimal QPS of the interface is not found, and the error rate corresponding to the current interface QPS is less than the preset error rate threshold, and the added value is added. After testing the access data of the service system, return to execute the single-interface optimal QPS discovery test execution job until the interface optimal QPS is found; or, through the single-interface optimal QPS discovery test execution job according to the test report and configuration parameters, no interface optimal QPS is found Excellent QPS, and the error rate corresponding to the current interface QPS is not less than the preset error rate, after reducing the access data of the service system under test, return to execute the single interface optimal QPS discovery test execution job until the interface optimal QPS is found;

Among them, the configuration parameters are uploaded by the user based on the single-interface optimal QPS discovery test to execute the job upload, the configuration parameters include at least the preset error rate, and the test report includes the error rate corresponding to the current interface QPS.

The units involved in the disclosed embodiments of the present invention may be implemented by means of software or hardware. Wherein, the name of the unit does not constitute a limitation of the unit itself under certain circumstances, for example, the first obtaining unit may also be described as "a unit for obtaining at least two Internet Protocol addresses".

The functions described above in this application may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), System on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

The embodiment of the present application also provides an electronic device, the electronic device includes: a processor and a memory, the processor and the memory are connected through a communication bus; wherein, the processor is used to call and execute the program stored in the memory; the memory is used to Stores the program used to implement the test method.

Referring now to FIG. 5 , it shows a schematic structural diagram of an electronic device suitable for implementing the disclosed embodiments of the present invention. Electronic devices in the disclosed embodiments of the present invention may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals ( Mobile terminals such as car navigation terminals) and stationary terminals such as digital TVs, desktop computers and the like. The electronic device shown in FIG. 5 is only an example, and should not limit the functions and scope of use of the disclosed embodiments of the present invention.

As shown in FIG. 5 , an electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc. 501), which may be loaded into a random access memory ( The program in RAM) 503 executes various appropriate actions and processes. In RAM 503, various programs and data required for the operation of electronic equipment are also stored. Processing device 501, ROM 502 and RAM 503 are connected to each other by bus 504 An input/output (I/O) interface 505 is also connected to the bus 504 .

Typically, the following devices can be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration an output device 507 such as a computer; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication means 509 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. While FIG. 5 shows an electronic device having various means, it should be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.

In particular, according to the disclosed embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the disclosed embodiments of the present invention include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 509, or from storage means 508, or from ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the methods of the disclosed embodiments of the present invention are performed.

Furthermore, the embodiment of the present invention also provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are used to execute the testing method.

The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: responds to the Seed Job selection instruction, and determines the Seed Job indicated by the Seed Job selection instruction as the target Seed Job; Execute the target Seed Job, so that the target Seed Job generates a test execution job based on the configuration information, where the configuration information is pre-configured by the user according to the project requirements; Execute the test execution job, so that the test execution job obtains the corresponding Test script: Execute the test script to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, obtain corresponding test monitoring data, and generate a test report based on the test monitoring data.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

It should be noted that the above-mentioned computer-readable medium disclosed in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.

Claims

A testing method is characterized in that being applied to a distributed performance testing platform, the method comprises:

In response to the Seed Job selection instruction, the Seed Job indicated by the Seed Job selection instruction is determined as the target Seed Job;

Executing the target Seed Job so that the target Seed Job generates a test execution Job based on configuration information, wherein the configuration information is pre-configured by the user according to project requirements;

Execute the test execution Job, so that the test execution Job obtains a corresponding test script according to the configuration information;

Execute the test script to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, obtain corresponding test monitoring data through the test execution Job, and based on the test Monitor data to generate test reports.
The method according to claim 1, wherein, after executing the target Seed Job so that the target Seed Job generates a test based on configuration information and executes the Job, it also includes:

The engine construction job is updated based on the test execution job.
The method according to claim 2, characterized in that the test script is executed to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, and the test execution Job obtains Corresponding test monitoring data, and generating a test report based on the test monitoring data, including:

Execute the test script based on the test engine Bundle to trigger the test node machine indicated by the configuration information, point the test traffic to the service system under test indicated by the configuration information for testing, until the service system under test completes the test, and execute through the test Job obtains corresponding test monitoring data, and generates a test report based on the test monitoring data;

Wherein, the test engine Bundle is obtained by using the updated engine construction Job to obtain and run the code of the engine construction job from Git of the test engine.
The method according to claim 1, wherein said executing the target Seed Job, so that said target Seed Job generates a test execution Job based on configuration information, comprising:

Execute the target Seed Job so that the target Seed Job obtains and parses the configuration information according to the input configuration address information, and determines the name of the test execution Job to be created, if there is no corresponding to the name of the test execution Job to be created The test executes the Job, and creates the test execution Job according to the configuration information;

Wherein, the configuration information is pre-configured by the user, and the configuration address information is input by the user.
The method according to claim 1, characterized in that, if the test execution job is a single-interface optimal QPS discovery test execution job, execute the test execution job, so that the test execution job obtains the corresponding test script according to the configuration information ,include:

Execute the single-interface optimal QPS discovery test execution job, so that the single-interface optimal QPS discovery test execution job responds to the user's interface selection instruction, and the interface indicated by the interface selection instruction is determined as the target interface, and according to the selected The configuration information obtains the test script corresponding to the target interface, if the target interface needs to prepare a data script, obtains the prepared data script returned by other interfaces associated with the target interface, and executes the prepared data script, get the data file;

Correspondingly, the execution of the test script includes:

A test script is executed based on the data file.
The method according to claim 5, characterized in that, after said generating said test report, the method further comprises:

When the optimal QPS of the single interface is found to be tested and executed according to the test report and configuration parameters, the optimal QPS of the interface is found, and the difference between the optimal QPS of the current interface and the optimal QPS of the last interface in history is the same as that of the When the ratio of the optimal QPS of the interface in the last time in history exceeds the preset threshold, the corresponding alarm information will be output;

When the optimal QPS of the single interface is found through the test execution job according to the test report and the configuration parameters, the optimal QPS of the interface is not found, and whether the error rate corresponding to the current interface QPS is less than the preset error rate threshold, increase After the access data of the service system under test, return to execute the single interface optimal QPS discovery test execution job until the optimal QPS of the interface is found;

Or, according to the test report and the configuration parameters through the single interface optimal QPS discovery test execution job, the optimal QPS of the interface is not found, and the error rate corresponding to the current interface QPS is not less than the preset When the error rate is lower, after reducing the access data of the service system under test, return to execute the single interface optimal QPS discovery test execution job until the optimal QPS of the interface is found;

Wherein, the configuration parameters are uploaded by the user based on the single interface optimal QPS discovery test execution job, the configuration parameters include at least the preset error rate, and the test report includes at least the error rate corresponding to the current interface QPS Rate.
A test device, characterized in that it is applied to a distributed performance test platform, said device comprising:

The first determination unit is configured to respond to a Seed Job selection instruction, and determine the SeedJob indicated by the Seed Job selection instruction as a target SeedJob;

A first execution unit, configured to execute the target SeedJob, so that the target Seed Job generates a test execution Job based on configuration information, wherein the configuration information is pre-configured by the user according to project requirements;

The second execution unit is used to execute the test execution job, so that the test execution job obtains a corresponding test script according to the configuration information;

The third execution unit is configured to execute the test script to direct the test traffic to the service system under test indicated by the configuration information for testing until the service system under test completes the test, and obtain corresponding test monitoring through the test execution Job data, and generate a test report based on the test monitoring data.
The device according to claim 7, characterized in that the device further comprises:

An update unit, configured to update the engine construction job based on the test execution job.
An electronic device, characterized in that it includes: a processor and a memory, the processor and the memory are connected through a communication bus; wherein, the processor is used to call and execute the program stored in the memory; the memory is used to store the program, and the program uses To realize the testing method described in any one of claims 1-6.
A computer-readable storage medium, characterized in that computer-executable instructions are stored in the computer-readable storage medium, and the computer-executable instructions are used to execute the testing method described in any one of claims 1-6.