CN114154169A

CN114154169A - Jenkins and JMeter-based automatic test method and device

Info

Publication number: CN114154169A
Application number: CN202111477163.5A
Authority: CN
Inventors: 赵铭; 林圳杰; 陈骋逵
Original assignee: Shenzhen Digital Power Grid Research Institute of China Southern Power Grid Co Ltd
Current assignee: Shenzhen Digital Power Grid Research Institute of China Southern Power Grid Co Ltd
Priority date: 2021-12-06
Filing date: 2021-12-06
Publication date: 2022-03-08

Abstract

The invention discloses an automatic testing method and device based on Jenkins and JMeter, wherein the method comprises the following steps: receiving a JMeter test script, creating a first project on Jenkins, configuring parameters in the first project to obtain a target project, calling a JMeter test tool through a preset execution script, executing the JMeter test script to test each test case in the target project to obtain a test result set, sending the test result set to a folder corresponding to a preset first target path, generating a first target file corresponding to the test result set in the folder, and generating a target test report according to the test result set and the first target file. Therefore, the method and the device can improve the efficiency of detecting the security vulnerabilities existing in the software functions, and are beneficial to improving the accuracy of detecting the security vulnerabilities existing in the software functions.

Description

Jenkins and JMeter-based automatic test method and device

Technical Field

The invention relates to the technical field of computers, in particular to an automatic testing method and device based on Jenkins and JMeter.

Background

The DevOps platform is a collective term for a group of processes, methods and systems, and is used to promote development, technical operation and communication, cooperation and integration among quality assurance departments, and the automated testing refers to running a system or an application under a preset condition, and evaluating a running result of the system or the application according to a preset condition, so as to detect vulnerabilities existing in software.

At present, the security vulnerabilities of the codes are usually checked through specific rules by using code scanning, and the code scanning is mostly based on syntax and coding rules, so that although the security vulnerabilities existing in the codes can be found, the continuous and efficient detection of the security vulnerabilities existing in the software functions cannot be realized. Therefore, it is very important to provide a new method for testing bugs existing in software functions.

Disclosure of Invention

The technical problem to be solved by the invention is to provide an automatic testing method and device based on Jenkins and JMeter, which can continuously integrate and continuously release the functional vulnerabilities existing in software, and is beneficial to improving the efficiency of detecting the security vulnerabilities existing in software functions and further beneficial to improving the accuracy of detecting the security vulnerabilities existing in software functions.

In order to solve the technical problem, the first aspect of the invention discloses an automatic testing method based on Jenkins and JMeter, which comprises the following steps:

receiving a JMeter test script;

creating a first project on Jenkins, configuring parameters in the first project to obtain a target project, wherein the target project comprises a plurality of test cases to be tested;

calling a JMeter test tool through a preset execution script, executing the JMeter test script to test each test case in the target project to obtain a test result set, wherein the test result set comprises a test result corresponding to each test case;

sending the test result set to a folder corresponding to a preset first target path, and generating a first target file corresponding to the test result set in the folder;

and generating a target test report according to the test result set and the first target file.

As an alternative implementation, in the first aspect of the present invention, the method further includes:

judging whether each test result in the test result set meets preset result conditions or not;

when the test result is judged to meet the preset result condition, determining the test result as a test success result;

and when the test result is judged not to meet the preset result condition, determining the test result as a test failure result.

judging whether a preset abnormal keyword exists in each test result in the test result set;

and modifying the test result into a test abnormal result when the test result is judged to have the preset abnormal keyword.

determining a test case with a test result corresponding to the test failure result and a test case with a test result corresponding to the test abnormal result as first test cases, and judging whether a security vulnerability exists in the software function of each first test case;

when the software function of the first test case is judged to have the security vulnerability, analyzing the security vulnerability to obtain a security vulnerability analysis result, wherein the security vulnerability analysis result comprises a target reason for the software function of the first test case to have the security vulnerability;

for each target reason, judging whether the target reason meets the preset reason condition;

and when the target reason is judged not to meet the preset reason condition, updating the JMeter test script to test the first test case corresponding to the target reason again to obtain the target reason meeting the preset reason condition.

detecting the influence degree of the security vulnerability according to the security vulnerability analysis result;

judging whether the influence degree is greater than a preset influence degree threshold value or not;

when the influence degree is judged to be larger than the preset influence degree threshold value, outputting first warning information to an administrator terminal, wherein the first warning information is used for informing the administrator that the influence degree of the security vulnerability is larger than the preset influence degree threshold value and reminding the administrator to repair the security vulnerability;

and when the influence degree is judged to be less than or equal to the preset influence degree threshold value, automatically repairing the security vulnerability in the first test case.

As an optional implementation manner, in the first aspect of the present invention, the configuring parameters in the first item to obtain a target item includes:

and adding character string parameters to the first project to obtain a target project, wherein the character string parameters comprise a test case identifier and an Execute shell construction item.

As an optional implementation manner, in the first aspect of the present invention, before creating a first item on Jenkins, configuring parameters in the first item, obtaining an object item, calling a JMeter test tool through a preset execution script, and executing the JMeter test script to test each test case in the object item, and obtaining a test result set, the method further includes:

and uploading the JMeter test script to a specified directory corresponding to the test case identifier through the Execute shell construction item.

The second aspect of the invention discloses an automatic testing method based on Jenkins and JMeter, which comprises the following steps:

the receiving module is used for receiving the JMeter test script;

the system comprises a configuration module, a test module and a test module, wherein the configuration module is used for creating a first project on Jenkins, configuring parameters in the first project and obtaining a target project, and the target project comprises a plurality of test cases to be tested;

the test module is used for calling a JMeter test tool through a preset execution script, executing the JMeter test script to test each test case in the target project to obtain a test result set, and the test result set comprises a test result corresponding to each test case;

the generating module is used for sending the test result set to a folder corresponding to a preset first target path and generating a first target file corresponding to the test result set in the folder;

the generating module is further configured to generate a target test report according to the test result set and the first target file.

As an alternative embodiment, in the second aspect of the present invention, the apparatus further comprises:

the judging module is used for judging whether each test result in the test result set meets preset result conditions or not;

the determining module is used for determining the test result as a test success result when the judging module judges that the test result meets the preset result condition;

the determining module is further configured to determine the test result as a test failure result when the judging module judges that the test result does not satisfy the preset result condition.

As an optional implementation manner, in the second aspect of the present invention, the determining module is further configured to determine, for each test result in the test result set, whether the test result has a preset abnormal keyword;

the determining module is further configured to modify the test result into a test abnormal result when the judging module judges that the test result has the preset abnormal keyword.

As an optional implementation manner, in the second aspect of the present invention, the determining module is further configured to determine, as the first test case, the test case whose test result is the test failure result and the test case whose test result is the test exception result;

the judging module is further used for judging whether the software function in each first test case has a security vulnerability or not;

the device further comprises:

the analysis module is used for analyzing the security vulnerability to obtain a security vulnerability analysis result when the judgment module judges that the software function of the first test case has the security vulnerability, wherein the security vulnerability analysis result comprises a target reason for the existence of the security vulnerability in the software function of the first test case;

the judging module is further configured to judge, for each target reason, whether the target reason meets a preset reason condition;

and the updating module is used for updating the JMeter test script when the judging module judges that the target reason does not meet the preset reason condition so as to test the first test case corresponding to the target reason again to obtain the target reason meeting the preset reason condition.

the detection module is used for detecting the influence degree of the security vulnerability according to the security vulnerability analysis result;

the judging module is also used for judging whether the influence degree is greater than a preset influence degree threshold value;

the output module is used for outputting first warning information to an administrator terminal when the judging module judges that the influence degree is greater than the preset influence degree threshold value, wherein the first warning information is used for informing the administrator that the influence degree of the security vulnerability is greater than the preset influence degree threshold value and reminding the administrator of repairing the security vulnerability;

and the repairing module is used for automatically repairing the security vulnerability in the first test case when the judging module judges that the influence degree is less than or equal to the preset influence degree threshold value.

As an optional implementation manner, in the second aspect of the present invention, the manner in which the configuration module configures parameters in the first item to obtain the target item specifically is:

the uploading module is used for creating a first project on Jenkins through the configuration module, configuring parameters in the first project, calling a JMeter testing tool through a preset execution script after a target project is obtained, executing the JMeter testing script to test each test case in the target project, and uploading the JMeter testing script to a specified directory corresponding to the test case identification through the Execute shell construction item before a test result set is obtained.

The third aspect of the invention discloses another automatic testing device based on Jenkins and JMeter, which comprises:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to execute the Jenkins and JMeter-based automatic test method disclosed by the first aspect of the invention.

The fourth aspect of the invention discloses a computer storage medium, which stores computer instructions, and when the computer instructions are called, the computer instructions are used for executing the Jenkins and JMeter-based automatic testing method disclosed by the first aspect of the invention.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

in the embodiment of the invention, a JMeter test script is received, a first project is created on Jenkins, parameters are configured in the first project to obtain a target project, a JMeter test tool is called through a preset execution script, the JMeter test script is executed to test each test case in the target project to obtain a test result set, the test result set is sent to a folder corresponding to a preset first target path, a first target file corresponding to the test result set is generated in the folder, and a target test report is generated according to the test result set and the first target file. Therefore, by implementing the method and the device, each test case can be tested by executing the JMeter test script and constructing the task through Jenkins, the automatic test of continuous integration and continuous deployment can be realized, and the method and the device are favorable for improving the efficiency of detecting the security vulnerabilities existing in the software functions, so that the accuracy of detecting the security vulnerabilities existing in the software functions is improved, and the intelligence of detecting the security vulnerabilities existing in the software functions is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic flow chart of an automated testing method based on Jenkins and JMeter according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of another Jenkins and JMeter-based automated testing method disclosed in the embodiments of the present invention;

FIG. 3 is a schematic structural diagram of an automated testing apparatus based on Jenkins and JMeter according to an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of another Jenkins and JMeter-based automatic test device disclosed in the embodiment of the present invention;

fig. 5 is a schematic structural diagram of another automation testing apparatus based on Jenkins and JMeter according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The terms "first," "second," and the like in the description and claims of the present invention and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, article, or article that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or article.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

The invention discloses an automatic testing method and device based on Jenkins and JMeter, which can test each test case by executing a JMeter testing script and constructing a task through Jenkins, can realize continuous integration and continuous deployment automatic testing, and is beneficial to improving the efficiency of detecting security holes in software functions, thus being beneficial to improving the accuracy of detecting the security holes in the software functions and further being beneficial to improving the intelligence of detecting the security holes in the software functions. The following are detailed below.

Example one

Referring to fig. 1, fig. 1 is a schematic flow chart of an automated testing method based on Jenkins and JMeter according to an embodiment of the present invention. The method for automatically testing based on Jenkins and JMeter described in fig. 1 may be applied to an automatic testing apparatus based on Jenkins and JMeter, and the embodiment of the present invention is not limited thereto. As shown in fig. 1, the Jenkins and JMeter based automated testing method may include the following operations:

101. a JMeter test script is received.

In the embodiment of the invention, the JMeter test script is a script which is written by a tester and used for testing the application interface and the software performance. Optionally, the writing of the JMeter test script can be written according to the test requirements, and the writing of the JMeter test script can be written in a visual form. Further alternatively, the format of the JMeter test script may be jmx format. Further, the JMeter assertion can be configured as one of JMeter test scripts, wherein the configured JMeter assertion can include one or more of a response assertion, an XPath assertion, a JSON assertion, and a size assertion. Therefore, the comprehensiveness of the subsequent test of each test case to be tested can be improved by receiving the JMeter test script, and the accuracy of the subsequent test result corresponding to each test case is improved.

In the embodiment of the invention, the JMeter test script can be applied to a DevOps platform. DevOps is a collective term for a set of processes, methods and systems for facilitating communication, collaboration and integration between development, technical operations and quality assurance departments. Through the DevOps platform, the work of software development members can be integrated, verification can be performed through automatic construction, each integrated work can be verified through the JMeter test script, and errors can be timely found in batches. Further, the time of the integration work in the DevOps platform may be specific, for example, the integration work is performed every half hour, or may be random, or may be triggered according to an instruction, which is not limited in the embodiment of the present invention. Therefore, the work of development members of different departments is integrated through the DevOps platform, and the JMeter test script is used for testing, so that the efficiency of testing each test case is improved, and the efficiency of obtaining the result corresponding to each test case is improved.

102. Creating a first project on Jenkins, configuring parameters in the first project, and obtaining a target project, wherein the target project comprises a plurality of test cases to be tested.

In the embodiment of the invention, Jenkins is a continuous integration tool developed based on a computer programming language Java, and is a continuous integration tool which is open-source and can provide a friendly operation interface. Jenkins can be used to monitor the continual repetitive work and is intended to provide an open and easy-to-use software platform on which software projects can be continually integrated. Therefore, the test cases are continuously integrated through Jenkins, all the test cases to be tested can be continuously integrated, the efficiency of subsequently testing each test case to be tested is favorably improved, and the efficiency of subsequently obtaining the test result corresponding to each test case is favorably improved.

In an optional embodiment, configuring the parameter in the first item to obtain the target item includes:

and adding character string parameters in the first project to obtain a target project, wherein the character string parameters comprise a test case identifier and an Execute shell construction item.

In this optional embodiment, optionally, the test case identifier may be a fliename. It should be noted that the filename is a parameter variable, and corresponds to the test case name created in the target project, and the filename has a function of allowing a user to customize the test case or allowing the user to customize the test case set. By adding the test case identification in the first project, different test cases or different test case sets can be distinguished, test results corresponding to different test cases or different test case sets can be generated, so that the situation of visually knowing the test results by testers is facilitated, different test cases or different test case sets can be respectively tested by adding the test case identification, the efficiency of testing the test cases is facilitated to be improved, and the accuracy of testing each test case is facilitated to be improved.

In this optional embodiment, optionally, the Execute shell building item may be used to run an external program, and may also be used to open a registered file, open a directory, print a file, and so on. Further optionally, the Execute shell build item may be used to open any file in the computer, and may also be used to open files in url format.

Therefore, in the optional embodiment, the target project is obtained by adding the character string parameters to the first project, wherein the character string parameters include the test case identifier and the Execute shell construction item, so that a user or a tester can customize the test case or the test case set, and can classify the test case or the test case set, thereby being beneficial to improving the efficiency of testing each test case and further being beneficial to improving the accuracy of testing each test case.

In another optional embodiment, a first project is created on Jenkins, parameters are configured in the first project, after a target project is obtained, a JMeter test tool is called through a preset execution script, the JMeter test script is executed to test each test case in the target project, and before a test result set is obtained, the method further includes:

and uploading the JMeter test script to a specified directory corresponding to the test case identification through the Execute shell construction item.

In this optional embodiment, optionally, the specified directory corresponding to the test case identifier may be determined according to the test case identifier. Further optionally, when there are multiple test case identifiers, there may be a specified directory corresponding to each test case identifier. Therefore, the appointed directories corresponding to the test case identifications are distinguished according to the test case identifications, the JMeter test script can be uploaded to the directories corresponding to the JMeter test script, the efficiency of calling the JMeter test tool to execute the JMeter test script through the preset execution script is improved, the efficiency of subsequently testing each test case is improved, and the efficiency of obtaining the test result corresponding to each test case is improved.

Therefore, in the optional embodiment, the JMeter test script is uploaded to the specified directory corresponding to the test case identification through the Execute shell construction item, so that the efficiency of subsequently executing the JMeter test script can be improved, the efficiency of testing each test case in the target project can be improved, and the efficiency of obtaining the test result corresponding to each test case can be improved.

103. And calling a JMeter test tool through a preset execution script, executing the JMeter test script to test each test case in the target project to obtain a test result set, wherein the test result set comprises a test result corresponding to each test case.

In the embodiment of the present invention, optionally, the preset execution script may be a shell script. It should be noted that, the shell script is an execution script, and various commands can be put into a file in advance, so as to facilitate a program file for executing various commands at one time. Therefore, all kinds of commands can be gathered into one file through the shell script, batch processing of all kinds of commands is facilitated, efficiency of processing all kinds of commands is improved, and efficiency of testing each test case in a target project is improved.

In the embodiment of the invention, optionally, the JMeter test tool is a test tool based on computer programming language Java, and the JMeter test tool is mainly used for performing pressure test on software. The JMeter test tool can be used to test static resources as well as dynamic resources, such as one or more of static files, Java applets, CGI scripts, Java objects, databases, FTP servers. As a further alternative, the JMeter test tool may be used to simulate a huge load on a server or network, test under different stress categories, test the strength of the server or the network to analyze its overall performance. Further optionally, the JMeter test tool can be used to perform functional or regression testing on the application, and the JMeter test tool can verify the results returned by the software function or application by creating a script with an assertion (e.g., a JMeter test script), thereby completing the detection of the software function or application. Therefore, by calling the JMeter testing tool, the comprehensiveness of testing each test case in the target project can be improved, the accuracy of obtaining the test result corresponding to each test case in the target project is improved, and the efficiency of testing each test case in the target project can be improved according to the characteristics of small size, complete functions and convenience in use of the JMeter testing tool.

104. And sending the test result set to a folder corresponding to a preset first target path, and generating a first target file corresponding to the test result set in the folder.

In this embodiment of the present invention, optionally, the preset first target path may be a JMeter test script, or may be a path corresponding to the test result set, and further optionally, the preset first target path may be a JMeter result file path.

In this embodiment of the present invention, optionally, the format of the first target file may be an xml format. It should be noted that the xml-formatted file is an extensible markup language, is a subset of a standard general markup language, and can be used for marking an electronic file to make the electronic file have a structural markup language, and further, the xml-formatted file can be used for marking data, defining data types, and allowing a user to define a source language of his own markup language. Further optionally, the xml formatted file can also be used to perform one or more of separating data, exchanging data, sharing data, creating new languages. Therefore, the xml file corresponding to the test result set is generated in the folder corresponding to the preset first target path, so that the transmission and storage of the test results corresponding to all the test cases included in the test result set are facilitated, and the efficiency of generating the target test report is further facilitated.

105. And generating a target test report according to the test result set and the first target file.

In the embodiment of the present invention, optionally, the test result set includes a test result corresponding to each test case, and the first target file stores the test result corresponding to each test case.

In the embodiment of the present invention, further optionally, the target test report may include one or more of an execution duration of each target test case, a task type of each target test case, a test mode of each target test case, an operation state of each target test case, an execution time of each target test case, an execution sequence of each target test case, a success rate of each target test case, a test case execution result of each target test case, a test priority of each target test case, and an execution number of each target test case. Therefore, the contents included in the target test report are more comprehensive and more detailed, the accuracy of generating the target test report is improved, the reading impression of the target test report viewed by a tester is improved, and the tester can visually know the detailed test condition of each test case.

It can be seen that, by implementing the method for automated testing based on Jenkins and JMeter described in fig. 1, a JMeter test script can be received, a first project is created on Jenkins, parameters are configured in the first project to obtain a target project, a JMeter test tool is called through a preset execution script, the JMeter test script is executed to test each test case in the target project to obtain a test result set, the test result set is sent to a folder corresponding to a preset first target path, a first target file corresponding to the test result set is generated in the folder, and a target test report is generated according to the test result set and the first target file. Therefore, the method and the device can be used for improving the efficiency of detecting the security vulnerabilities existing in the software functions and improving the accuracy of detecting the security vulnerabilities existing in the software functions.

Example two

Referring to fig. 2, fig. 2 is a schematic flow chart of an automated testing method based on Jenkins and JMeter according to an embodiment of the present invention. The method for automatically testing based on Jenkins and JMeter described in fig. 2 may be applied to an automatic testing apparatus based on Jenkins and JMeter, and the embodiment of the present invention is not limited thereto. As shown in fig. 2, the Jenkins and JMeter based automated testing method may include the following operations:

201. a JMeter test script is received.

202. Creating a first project on Jenkins, configuring parameters in the first project, and obtaining a target project, wherein the target project comprises a plurality of test cases to be tested.

203. And calling a JMeter test tool through a preset execution script, executing the JMeter test script to test each test case in the target project to obtain a test result set, wherein the test result set comprises a test result corresponding to each test case.

204. And sending the test result set to a folder corresponding to a preset first target path, and generating a first target file corresponding to the test result set in the folder.

205. And generating a target test report according to the test result set and the first target file.

In the embodiment of the present invention, for other descriptions of steps 201 to 205, please refer to the detailed description of steps 101 to 105 in the first embodiment, which is not repeated herein.

206. And judging whether the test result meets a preset result condition or not for each test result in the test result set.

In the embodiment of the present invention, when it is determined in step 206 that the test result meets the predetermined result condition, step 207 is triggered to be executed; when the step 206 determines that the test result does not satisfy the predetermined result condition, the step 208 is triggered.

In the embodiment of the present invention, optionally, the determining whether the test result meets the preset result condition may include:

and judging whether the test result meets the preset result condition or not through the response assertion of the JMeter configured in the JMeter.

In the embodiment of the present invention, optionally, the response assertion of JMeter can be used to check whether the response content obtained from the server is in accordance with the expectation. Optionally, the determining whether the test result meets the preset result condition may be determining whether the test result is consistent with a preset response result. Therefore, whether the test result meets the preset result condition or not is judged by each test result in the test result set, and the accuracy of determining the test result corresponding to each test case can be improved.

207. And determining the test result as a test success result.

208. The test result is determined as a test failure result.

In the embodiment of the present invention, when it is determined in step 206 that the test result meets the predetermined result condition, the test result is determined as a successful test result; when the step 206 determines that the test result does not satisfy the predetermined result condition, the test result is determined as a test failure result. Therefore, according to the judgment of whether the test result meets the preset result condition, if so, the test result is determined as the test success result, and if not, the test result is determined as the test failure result, so that the accuracy of determining the test result corresponding to each test case can be improved, and the accuracy of subsequently determining the test abnormal result is improved.

It can be seen that, by implementing the method for automatically testing based on Jenkins and JMeter described in fig. 2, a JMeter test script can be received, a first project is created on Jenkins, parameters are configured in the first project to obtain a target project, a JMeter test tool is called through a preset execution script, the JMeter test script is executed to test each test case in the target project to obtain a test result set, the test result set is sent to a folder corresponding to a preset first target path, a first target file corresponding to the test result set is generated in the folder, a target test report is generated according to the test result set and the first target file, and whether the test result meets a preset result condition or not is judged for each test result in the test result set, and if yes, the test result is determined as a test success result; if not, determining the test result as a test failure result. Therefore, the method and the device can improve the efficiency of detecting the security vulnerabilities existing in the software functions, thereby being beneficial to improving the accuracy of detecting the security vulnerabilities existing in the software functions and further being beneficial to improving the intelligence of detecting the security vulnerabilities existing in the software functions.

In an optional embodiment, the method further comprises:

and when the test result is judged to have the preset abnormal keyword, modifying the test result into a test abnormal result.

In this alternative embodiment, there may be one or more preset exception keywords, which is not limited in this embodiment of the present invention. Optionally, the abnormal keyword may be an abnormal keyword such as "abnormal test mode", "abnormal call mode", "too long test duration", and the like. For example, when the test time for testing the test case is too long, the test result is modified into a test exception result, and the exception reason corresponding to the test exception result is marked as "too long test time". Furthermore, the abnormality cause corresponding to the test abnormality result can be input into the target test report. Therefore, whether the preset abnormal keywords exist in the test result or not is judged, and the abnormal keywords existing in the test result are further analyzed, so that the tester can know the reason that the test result is the test abnormal result.

Therefore, in the optional embodiment, whether a preset abnormal keyword exists in the test result is judged, if yes, the test result is modified into the test abnormal result, and the reason of the test result abnormality is determined according to the abnormal keyword, so that a tester can perform subsequent solving operation on the test abnormal result, wherein the solving operation is matched with the abnormal reason corresponding to the test abnormal result, and the use safety of the software function corresponding to the target test case is improved.

In another optional embodiment, the method further comprises:

determining a test case corresponding to a test result as a test failure result and a test case corresponding to a test result as a test abnormal result as first test cases, and judging whether a security vulnerability exists in the software function of each first test case;

when the software function of the first test case is judged to have a security vulnerability, analyzing the security vulnerability to obtain a security vulnerability analysis result, wherein the security vulnerability analysis result comprises a target reason for the software function of the first test case to have the security vulnerability;

In this optional embodiment, optionally, the first test case may include one or more test cases, which are not limited in this embodiment of the present invention. Further optionally, for each test case in the first test case, it is determined whether a security vulnerability exists in the software function in the first test case, if so, the security vulnerability is analyzed, and if not, the operation may be ended.

In this optional embodiment, optionally, the preset reason condition may be an operation of determining whether the target reason is specific, ending the process if the target reason is determined to be specific, and triggering execution of updating the JMeter test script if the target reason is determined to be not specific, so as to test the first test case corresponding to the target reason again, and obtain the target reason meeting the preset reason condition. For example, when the target cause is data missing, the target cause is determined not to satisfy the predetermined cause condition because the target cause does not detect what kind of data the missing data is or the amount of the missing data corresponds to, the JMeter test script is updated, the first test case is tested again according to the updated JMeter test script, the target cause obtained after the test again is the missing communication data, the predetermined cause condition is met, and the process can be ended. Therefore, whether each target reason meets the preset reason condition or not can be judged, the specific reason with the security vulnerability can be found, the efficiency of analyzing the influence degree of the security vulnerability subsequently is facilitated, the accuracy of analyzing the security vulnerability subsequently is facilitated, and therefore an administrator can directly know the reason with the security vulnerability and execute the solution operation matched with the target reason, the efficiency and the accuracy of repairing the security vulnerability are improved, and the safety and the protection of software functions are improved.

It can be seen that, in the optional embodiment, by determining whether there is a security vulnerability in each first test case, if so, analyzing the security vulnerability to obtain a security vulnerability analysis result and a target reason for the security vulnerability in the software function of the first test case, further determining whether the target reason meets a preset reason condition, if not, updating the JMeter test script to test the first test case again, so as to find out the specific reason for the security vulnerability, facilitate the subsequent analysis of the efficiency of the impact degree of the security vulnerability, facilitate the subsequent analysis of the accuracy of the security vulnerability, facilitate the administrator to directly know the reason for the security vulnerability in the first test case and execute the solution operation matched with the target reason, and further facilitate the improvement of the efficiency and accuracy of repairing the security vulnerability, the method is favorable for improving the safety and the protection of software functions.

In yet another optional embodiment, the method further comprises:

In this optional embodiment, optionally, the influence degree is used to represent the influence degree of the security vulnerability on the software function corresponding to the test case. Further alternatively, the degree of influence may be used to represent the influence from various aspects, such as the influence from communication aspects, function aspects, interface aspects, operation aspects, and the like.

In this optional embodiment, the first warning information is used to notify the administrator that the degree of influence of the security vulnerability is large, and to remind the administrator that the security vulnerability needs to be repaired in time, for example, the first warning information may be "the degree of influence of the detected security vulnerability on the software performance is large, the user is seriously influenced to use the software, and the administrator is requested to repair the security vulnerability as soon as possible". Further optionally, the first warning information may include a reason for the existence of the security vulnerability in the software function of the test case, and thus, by sending the degree of influence of the security vulnerability and the target reason for the existence of the security vulnerability to the administrator terminal, the administrator can be timely reminded to repair the security vulnerability, which is beneficial to improving the efficiency of repairing the security vulnerability, and by sending the target reason for the existence of the security vulnerability to the administrator terminal, the administrator is beneficial to timely knowing the reason for the existence of the vulnerability and making a solution operation matched with the target reason, which is beneficial to improving the efficiency of repairing the security vulnerability, and further beneficial to improving the security and the protection performance of the software function.

In this optional embodiment, when it is determined that the influence degree of the security vulnerability is smaller than the preset influence degree threshold, the security vulnerability can be automatically repaired. Further optionally, after completing the vulnerability repair of the security vulnerability, obtaining a repair result, judging whether the repair result meets a preset repair condition, and determining that the security vulnerability has been repaired when the repair result meets the preset repair condition; and when the repairing result is judged not to meet the preset repairing condition, sending second reminding information to the administrator terminal, wherein the second reminding information is used for reminding the administrator that the automatic repairing of the security vulnerability fails at this time, and asking the administrator to timely inspect and repair the security vulnerability. Whether preset repair conditions are met or not is judged after automatic repair, if yes, repair is completed, if not, an administrator is reminded to repair the bug manually, the efficiency of repairing the security bug is improved, the accuracy of repairing the security bug is improved, the intelligence of repairing the security bug is improved, and the safety and the protection of the software function are improved.

It can be seen that, in this optional embodiment, the influence degree of the security vulnerability is detected according to the security vulnerability analysis result, whether the influence degree of the security vulnerability is greater than a preset influence degree threshold value is judged, if so, a prompt message is sent to the administrator terminal, and if not, the security vulnerability is automatically repaired, so that the intelligence of repairing the security vulnerability is improved, the efficiency of repairing the security vulnerability is improved, the accuracy of repairing the security vulnerability is improved, and the security and the protection of the software function are improved.

EXAMPLE III

Referring to fig. 3, fig. 3 is a schematic structural diagram of an automated testing apparatus based on Jenkins and JMeter according to an embodiment of the present invention. As shown in fig. 3, the Jenkins and JMeter based automatic testing apparatus includes:

a receiving module 301, configured to receive a JMeter test script.

The configuration module 302 is configured to create a first project on Jenkins, configure parameters in the first project, and obtain a target project, where the target project includes multiple test cases to be tested.

The test module 303 is configured to call a JMeter test tool through a preset execution script, and execute the JMeter test script to test each test case in the target project to obtain a test result set, where the test result set includes a test result corresponding to each test case.

The generating module 304 is configured to send the test result set to a folder corresponding to a preset first target path, and generate a first target file corresponding to the test result set in the folder.

The generating module 304 is further configured to generate a target test report according to the test result set and the first target file.

It can be seen that, the Jenkins and JMeter-based automated testing apparatus described in fig. 3 can receive a JMeter test script, create a first project on Jenkins, configure parameters in the first project to obtain a target project, call a JMeter testing tool through a preset execution script, execute the JMeter test script to test each test case in the target project to obtain a test result set, send the test result set to a folder corresponding to a preset first target path, generate a first target file corresponding to the test result set in the folder, and generate a target test report according to the test result set and the first target file. Therefore, the method and the device can improve the efficiency of detecting the security vulnerabilities existing in the software functions, and are beneficial to improving the accuracy of detecting the security vulnerabilities existing in the software functions.

In an alternative embodiment, as shown in fig. 4, the Jenkins and JMeter based automatic test device further includes:

the determining module 305 is configured to determine, for each test result in the test result set, whether the test result satisfies a preset result condition.

A determining module 306, configured to determine the test result as a test success result when the determining module 305 determines that the test result satisfies the preset result condition.

The determining module 306 is further configured to determine the test result as a test failure result when the determining module 305 determines that the test result does not satisfy the predetermined result condition.

It can be seen that, the Jenkins and JMeter-based automatic testing apparatus described in fig. 4 is implemented to determine, for each test result in the test result set, whether the test result meets a preset result condition, and if yes, determine the test result as a successful test result; if not, determining the test result as a test failure result. Therefore, the method and the device can improve the efficiency of detecting the security vulnerabilities existing in the software functions, and are beneficial to improving the intelligence of detecting the security vulnerabilities existing in the software functions, so that the accuracy of detecting the security vulnerabilities existing in the software functions is improved.

In another alternative embodiment, as shown in fig. 4, the determining module 305 is further configured to determine, for each test result in the test result set, whether the test result has a preset abnormal keyword.

The determining module 306 is further configured to modify the test result into a test exception result when the determining module 305 determines that the test result has the preset exception keyword.

It can be seen that, by implementing the Jenkins and JMeter-based automatic testing device described in fig. 4, whether a preset abnormal keyword exists in the test result can be judged, and if the preset abnormal keyword exists, the test result is modified into the test abnormal result, which is beneficial for a tester to perform subsequent solving operation on the test abnormal result, wherein the solving operation is matched with the abnormal reason corresponding to the test abnormal result, so that the accuracy of detecting the security holes existing in the software functions is improved, and the security of the software functions corresponding to the target test case is improved.

In yet another alternative embodiment, as shown in fig. 4, the determining module 306 is further configured to determine, as the first test case, the test case corresponding to the test result that is the test failure result and the test case corresponding to the test result that is the test exception result.

The determining module 305 is further configured to determine whether a security vulnerability exists in the software function in each first test case.

The device also includes:

and the analyzing module 307 is configured to, when the determining module 305 determines that the software function of the first test case has a security vulnerability, analyze the security vulnerability to obtain a security vulnerability analysis result, where the security vulnerability analysis result includes a target reason for the software function of the first test case having the security vulnerability.

The determining module 305 is further configured to determine, for each target reason, whether the target reason meets a preset reason condition.

And an updating module 308, configured to update the JMeter test script to perform a test again on the first test case corresponding to the target reason to obtain the target reason meeting the preset reason condition when the determining module 305 determines that the target reason does not meet the preset reason condition.

It can be seen that the implementation of the Jenkins and JMeter-based automatic testing device described in fig. 4 can find the specific reason of the existence of the security hole, thereby facilitating the subsequent analysis of the efficiency of the degree of influence of the security hole and the subsequent analysis of the accuracy of the security hole, facilitating the administrator to directly know the reason of the existence of the security hole and execute the solution operation matched with the target reason, further facilitating the improvement of the efficiency and the accuracy of repairing the security hole, and facilitating the improvement of the security and the protection of software functions.

In yet another alternative embodiment, as shown in fig. 4, the apparatus further comprises:

and the detection module 309 is configured to detect the degree of influence of the security vulnerability according to the security vulnerability analysis result.

The determining module 305 is further configured to determine whether the influence degree is greater than a preset influence degree threshold.

The output module 310 outputs first warning information to the administrator terminal when the determining module 305 determines that the influence degree is greater than the preset influence degree threshold, where the first warning information is used to inform the administrator that the influence degree of the security vulnerability is greater than the preset influence degree threshold, and remind the administrator to repair the security vulnerability.

And a repairing module 311, configured to automatically repair the security vulnerability in the first test case when the determining module 305 determines that the influence degree is less than or equal to the preset influence degree threshold.

It can be seen that implementing the automation testing arrangement based on Jenkins and JMeter described in fig. 4 can detect the degree of influence of the security hole according to the security hole analysis result, and judge whether the degree of influence of the security hole is greater than a preset degree of influence threshold, if so, send a prompt message to the administrator terminal, if not, automatically repair the security hole, and is favorable for improving the efficiency of repairing the security hole, and is favorable for improving the intelligence of repairing the security hole, thereby being favorable for improving the accuracy of repairing the security hole, and further being favorable for improving the security and the protection of the software function.

In yet another alternative embodiment, as shown in fig. 4, the configuration module 302 configures parameters in the first item, and the manner of obtaining the target item specifically is:

It can be seen that, by implementing the Jenkins and JMeter-based automatic testing device described in fig. 4, a target project can be obtained by adding a string parameter to a first project, where the string parameter includes a test case identifier and an Execute shell construction item, a user or a tester can customize a test case or a test case set, and can classify the test case or the test case set, so as to improve the efficiency of testing each test case, and further improve the efficiency of obtaining a test result corresponding to each test case.

an uploading module 312, configured to create a first project on Jenkins by the configuration module 302, configure parameters in the first project, obtain a target project, call a JMeter test tool through a preset execution script by the test module 303, Execute the JMeter test script to test each test case in the target project, and upload the JMeter test script to an assigned directory corresponding to the test case identifier by an Execute shell construction item before obtaining a test result set.

It can be seen that, by implementing the Jenkins and JMeter-based automation testing device described in fig. 4, the JMeter test script can be uploaded to the specified directory corresponding to the test case identifier through the Execute shell construction item, so that the efficiency of subsequently executing the JMeter test script can be improved, and the efficiency of testing each test case in the target project can be improved.

Example four

Referring to fig. 5, fig. 5 is a schematic structural diagram of another automatic testing apparatus based on Jenkins and JMeter according to an embodiment of the present invention. As shown in fig. 5, the Jenkins and JMeter based automated test equipment may include:

a memory 401 storing executable program code;

a processor 402 coupled with the memory 401;

the processor 402 calls the executable program code stored in the memory 401 to execute the steps of the Jenkins and JMeter-based automated testing method described in the first embodiment of the present invention or the second embodiment of the present invention.

EXAMPLE five

The embodiment of the invention discloses a computer storage medium, which stores computer instructions, and the computer instructions are used for executing steps in an automation testing method based on Jenkins and JMeter described in the first embodiment or the second embodiment of the invention when being called.

EXAMPLE six

The embodiment of the invention discloses a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program, wherein the computer program is operable to make a computer execute the steps of the Jenkins and JMeter-based automatic testing method described in the first embodiment or the second embodiment.

The above-described embodiments of the apparatus are merely illustrative, and the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above detailed description of the embodiments, those skilled in the art will clearly understand that the embodiments may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. Based on such understanding, the above technical solutions may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, where the storage medium includes a Read-Only Memory (ROM), a Random Access Memory (RAM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), a One-time Programmable Read-Only Memory (OTPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc-Read-Only Memory (CD-ROM), or other disk memories, CD-ROMs, or other magnetic disks, A tape memory, or any other medium readable by a computer that can be used to carry or store data.

Finally, it should be noted that: the automation test method and apparatus based on Jenkins and JMeter disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used for illustrating the technical solution of the present invention, not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art; the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic testing method based on Jenkins and JMeter is characterized by comprising the following steps:

receiving a JMeter test script;

2. The Jenkins and JMeter-based automated testing method as claimed in claim 1, further comprising:

3. The Jenkins and JMeter-based automated testing method as claimed in claim 2, wherein the method further comprises:

4. The Jenkins and JMeter-based automated testing method as claimed in claim 3, further comprising:

5. The Jenkins and JMeter-based automated testing method as claimed in claim 4, wherein the method further comprises:

6. The Jenkins and JMeter-based automation testing method of claim 5, wherein the configuring parameters in the first project to obtain a target project comprises:

7. The method as claimed in claim 6, wherein the method for automated testing based on Jenkins and JMeter comprises the steps of creating a first project on Jenkins, configuring parameters in the first project, calling a JMeter testing tool through a preset execution script after obtaining a target project, executing the JMeter testing script to test each test case in the target project, and before obtaining a test result set, the method further comprising:

8. An automated testing device based on Jenkins and JMeter, the device comprising:

the receiving module is used for receiving the JMeter test script;

9. An automated testing device based on Jenkins and JMeter, the device comprising:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to execute the Jenkins and JMeter based automated testing method of any of claims 1-7.

10. A computer storage medium storing computer instructions for performing the Jenkins and JMeter based automated testing method of any of claims 1-7 when invoked.