CN113392000A - Test case execution result analysis method, device, equipment and storage medium - Google Patents
Test case execution result analysis method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113392000A CN113392000A CN202110650500.XA CN202110650500A CN113392000A CN 113392000 A CN113392000 A CN 113392000A CN 202110650500 A CN202110650500 A CN 202110650500A CN 113392000 A CN113392000 A CN 113392000A
- Authority
- CN
- China
- Prior art keywords
- test case
- target test
- failure
- target
- case
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 455
- 238000004458 analytical method Methods 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 claims abstract description 54
- 238000004140 cleaning Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 3
- 238000011835 investigation Methods 0.000 abstract description 3
- 230000004048 modification Effects 0.000 abstract description 3
- 238000012986 modification Methods 0.000 abstract description 3
- 238000013522 software testing Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 13
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000008878 coupling Effects 0.000 description 5
- 238000010168 coupling process Methods 0.000 description 5
- 238000005859 coupling reaction Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 238000013024 troubleshooting Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010224 classification analysis Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Test And Diagnosis Of Digital Computers (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application provides a method, a device, equipment and a storage medium for analyzing an execution result of a test case, and relates to the technical field of software testing. The method comprises the following steps: acquiring a target test report of a target test case with execution failure; analyzing the target test report to obtain failure error reporting information of the target test case; and determining the failure analysis result of the test case with failed execution according to the failure error reporting information of the target test case. By the aid of the method and the device, the failure analysis result of the rapid positioning test case can be accurately analyzed, the automatic analysis efficiency of failure reasons of the test case which fails to be executed is improved, the investigation and modification efficiency of the target test case is improved, and the automatic test efficiency is improved.
Description
Technical Field
The invention relates to the technical field of software testing, in particular to a method, a device, equipment and a storage medium for analyzing an execution result of a test case.
Background
In the software version development process, several rounds of automated tests are generally required to be performed on a pre-programmed test case. However, the test cases often generate exceptions during execution, thereby causing test failures. Therefore, in order to improve the efficiency and quality of the automated testing, the test case that fails to be executed needs to be analyzed in each test, so as to find out the root cause of the test case that fails to be executed, and thus, the tester can repair the failed test case in time.
At present, when test cases with execution failures are analyzed, mainly by a tester, all the test cases with execution failures in a test report are analyzed and checked one by one, and information states, feedback results and the like in the execution process of each node in all the test cases with execution failures are obtained, so that nodes causing error reporting in the whole test case are searched, and the tester repairs the searched error reporting nodes in time.
However, with the prior art method, a lot of time and effort are consumed by the tester to analyze and investigate all the failed test cases one by one in the test report, which results in low analysis efficiency of the execution result of the failed test case.
Disclosure of Invention
The present invention is directed to provide a method, an apparatus, a device and a storage medium for analyzing test case execution results, so as to improve the efficiency of analyzing the failed test case execution results.
In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:
in a first aspect, an embodiment of the present application provides a method for analyzing an execution result of a test case, including:
acquiring a target test report of a target test case with execution failure;
analyzing the target test report to obtain failure error reporting information of the target test case;
and determining a failure analysis result of the target test case according to the failure error reporting information of the target test case.
Optionally, the analyzing the target test report to obtain failure error reporting information of the target test case includes:
analyzing the case file of the target test case to obtain the hierarchical structure of the target test case;
extracting parameter information of the target test case from the hierarchical structure of the target test case;
and analyzing the target test report by using the parameter information of the target case to obtain the failure error reporting information of the target test case.
Optionally, the extracting parameter information of the target test case from the hierarchical structure of the target test case includes:
extracting parameter information of a target test case according to a frame structure of an information file of the target test case;
and writing the parameter information of the target test case into an information file of the target test case.
Optionally, the analyzing the target test report by using the parameter information of the target use case to obtain failure error reporting information of the target test case includes:
and analyzing the target test report by using the parameter information in the information file of the target test case to obtain the execution result and reason information of each parameter in the target test case.
Optionally, after determining the failure analysis result of the target test case according to the failure error reporting information of the target test case, the method further includes:
classifying the target test cases according to the failure analysis result of each target test case;
and obtaining a plurality of test case sets according to the classification result, wherein the similarity of the execution failure reasons of the test cases in each test case set meets the preset condition.
Optionally, the classifying the target test cases according to the failure analysis result of each target test case includes:
calculating the similarity of the failure reasons of the target test cases according to the preset weight of each parameter in the failure reasons of the target test cases;
and classifying the target test cases according to the similarity of the failure reasons of the target test cases.
Optionally, after determining the failure analysis result of the test case with failed execution according to the failure error reporting information of the target test case, the method further includes:
and carrying out data cleaning on the failure error reporting information to obtain the cleaned failure error reporting information.
In a second aspect, an embodiment of the present application further provides an apparatus for analyzing a test case execution result, where the apparatus includes:
the acquisition module is used for acquiring a target test report of a target test case with execution failure;
the analysis module is used for analyzing the target test report to obtain failure error reporting information of the target test case;
and the determining module is used for determining the failure analysis result of the target test case according to the failure error reporting information of the target test case.
Optionally, the parsing module is further configured to:
analyzing the case file of the target test case to obtain the hierarchical structure of the target test case;
extracting parameter information of the target test case from the hierarchical structure of the target test case;
and analyzing the target test report by using the parameter information of the target case to obtain the failure error reporting information of the target test case.
Optionally, the parsing module is further configured to:
extracting parameter information of a target test case according to a frame structure of an information file of the target test case;
and writing the parameter information of the target test case into an information file of the target test case.
Optionally, the parsing module is further configured to:
and analyzing the target test report by using the parameter information in the information file of the target test case to obtain the execution result and reason information of each parameter in the target test case.
Optionally, the apparatus further comprises:
the classification module is used for classifying the target test cases according to the failure analysis result of the target test cases; and obtaining a plurality of test case sets according to the classification result, wherein the similarity of the execution failure reasons of the test cases in each test case set meets the preset condition.
Optionally, the classifying module is further configured to:
calculating the similarity of the failure reasons of the target test cases according to the preset weight of each parameter in the failure reasons of the target test cases;
and classifying the target test cases according to the similarity of the failure reasons of the target test cases.
Optionally, the apparatus further comprises:
and the cleaning module is used for carrying out data cleaning on the failure error reporting information to obtain the cleaned failure error reporting information.
In a third aspect, an embodiment of the present application further provides a processing device, including: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when a processing device is running, the processor executing the machine-readable instructions to perform the steps of the method provided by the first aspect.
In a fourth aspect, this embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the method provided in the first aspect.
The beneficial effect of this application is:
the embodiment of the application provides a method, a device, equipment and a storage medium for analyzing the execution result of a test case, wherein the method comprises the following steps: acquiring a target test report of a target test case with execution failure; analyzing the target test report to obtain failure error reporting information of the target test case; and determining the failure analysis result of the test case with failed execution according to the failure error reporting information of the target test case. In the scheme, the test report of the test case which fails to be executed in the test is obtained firstly, then the obtained test report is analyzed to obtain the failure error reporting information of the test case which fails to be executed, so that the fact that the test case fails to be executed due to the fact that the information in the test case which fails to be executed is abnormal is further determined, the failure analysis result of the test case can be accurately and quickly located, the automatic analysis efficiency of the failure reason of the test case which fails to be executed is improved, the checking and modifying efficiency of the target test case is improved, and the automatic test efficiency is further improved.
In addition, the failure analysis results of the test cases with execution failures are classified and counted to obtain a set containing a plurality of test cases, and the plurality of test cases in the same set have the same failure reasons or the failure reasons are very similar, so that a tester can quickly and accurately analyze the failure reasons of all the test cases with execution failures in an automatic test task according to the plurality of test case sets in the classification results, and further can quickly locate the node with the test case with the failure in the execution process without repeatedly checking and analyzing the same error reasons, thereby improving the efficiency of checking and modifying the target test case and further improving the efficiency of automatic testing.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of a method for analyzing an execution result of a test case according to an embodiment of the present application;
fig. 3 is a schematic flowchart of another test case execution result analysis method according to an embodiment of the present application;
fig. 4 is a schematic flowchart of another test case execution result analysis method according to an embodiment of the present application;
fig. 5 is a schematic flowchart of another test case execution result analysis method according to an embodiment of the present application;
fig. 6 is a schematic flowchart of another test case execution result analysis method according to an embodiment of the present application;
fig. 7 is an overall schematic diagram of a test case execution result analysis method according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a test case execution result analysis apparatus according to an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.
Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure; the electronic device may be a computer or a processing device such as a server, for example, to implement the test case execution result analysis method of the present application. As shown in fig. 1, the electronic apparatus includes: a processor 101 and a memory 102.
The processor 101 and the memory 102 are electrically connected directly or indirectly to realize data transmission or interaction. For example, electrical connections may be made through one or more communication buses or signal lines.
The processor 101 may be an integrated circuit chip having signal processing capability. The Processor 101 may be a general-purpose Processor including a Central Processing Unit (CPU), a Network Processor (NP), and the like. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The Memory 102 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
It will be appreciated that the configuration depicted in FIG. 1 is merely illustrative and that electronic device 100 may include more or fewer components than shown in FIG. 1 or may have a different configuration than shown in FIG. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.
The memory 102 is used for storing a program, and the processor 101 calls the program stored in the memory 102 to execute the test case execution result analysis method provided in the following embodiment.
Fig. 2 is a schematic flowchart of a method for analyzing an execution result of a test case according to an embodiment of the present application, and optionally, an execution subject of the method may be a server, a computer, or a computer, and has a data processing function. As shown in fig. 2, the method includes:
s201, acquiring a target test report of the target test case with execution failure.
In this embodiment, the target test with execution failure refers to a test case with execution failure in one automated test, and may refer to one failed test case or multiple failed test cases.
Generally, after an automated test task is executed, test reports of all test cases are obtained, and the execution result of one test case (or all test cases) can be recorded in one test report. The execution result of the test case comprises the following steps: fail or pass.
At this time, the target test case a with failed execution may be retrieved according to the execution result recorded in the test report, and the test report after the target test case a is executed may be obtained.
S202, analyzing the target test report to obtain failure error reporting information of the target test case.
The target test report refers to a test report of a test case that fails to be executed, that is, failure error reporting information of the test case that fails to be executed is recorded in the target test report.
The "failure error reporting information" of the target test case may include the following contents:
1. executing failed use case names and failure error reporting information of use case levels;
2. the name of a keyword which fails to execute under the test case and failure error reporting information of a keyword level;
3. executing failed operation function under the key word, and failure error reporting information of the operation function level.
In this embodiment, since most test reports are Extensible Markup Language (XML) files in HyperText Markup Language (HTML) format, the execution result recorded in the target test report may be analyzed by using an open source library of tree controls (ElementTree), so as to obtain failure error reporting information of the target test case, such as error reporting information of a keyword name of a certain failed execution of the test case a, or error reporting information of an operation function of a certain failed execution of the test case a.
In addition, the target test report can also be analyzed by using an XML analysis technology to obtain the failure error reporting information of the target test case.
S203, determining a failure analysis result of the target test case according to the failure error reporting information of the target test case.
On the basis of the above embodiment, for example, after the failure error reporting information of the test case a is obtained through analysis, classification analysis may be performed on the failure error reporting information of the test case a according to specific contents included in the failure error reporting information of the test case a, so as to further determine that the test case a fails to be executed due to which exception of the case name, the keyword, or the operation function in the test case a occurs, so that the failure analysis result of the test case can be accurately and quickly analyzed, the automatic analysis efficiency of the failure cause of the test case that fails to be executed is improved, the investigation and modification efficiency of the target test case is improved, and the automatic test efficiency is improved.
To sum up, an embodiment of the present application provides a method for analyzing an execution result of a test case, where the method includes: acquiring a target test report of a target test case with execution failure; analyzing the target test report to obtain failure error reporting information of the target test case; and determining the failure analysis result of the test case with failed execution according to the failure error reporting information of the target test case. In the scheme, the test report of the 'test case with failed execution' in the test is obtained firstly, and then the obtained test report is analyzed to obtain the failure error reporting information of the test case with failed execution, so that the condition that the target test case has failed execution due to abnormal information in the 'test case with failed execution' is further determined, the failure analysis result of the test case can be accurately and quickly positioned, the automatic analysis efficiency of the failure reason of the test case with failed execution is improved, the checking and modifying efficiency of the target test case is improved, and the automatic test efficiency is further improved.
Specifically explaining how to analyze the target test report through the following embodiments, failure error reporting information of the target test case is obtained.
Fig. 3 is a flowchart illustrating another method for analyzing an execution result of a test case according to an embodiment of the present application, and optionally, as shown in fig. 3, the step S202: analyzing the target test report to obtain failure error reporting information of the target test case, wherein the failure error reporting information comprises:
s301, analyzing the case file of the target test case to obtain the hierarchical structure of the target test case.
The hierarchical structure of the test case is a tree structure, and each end node corresponds to one piece of test case description information. For example, the hierarchy of test cases may be: each test case may contain one or more keywords, and each keyword may have one or more operation functions.
Illustratively, the use case file of the target test case a is in a text file (txt) format, for example, and may also be in other formats.
In this embodiment, the case file of the target test case may be obtained in the following three ways:
firstly, a case name of a target test case is obtained, and a test case file directory is traversed according to the case name of the target test case to obtain a case file of the target test case.
And secondly, directly acquiring a storage path of the case file of the target test case, and inquiring the file under the storage path to obtain the case file of the target test case.
And thirdly, directly using a case file of a target test case through a case data reading interface provided in the processing equipment.
After the case file of the target test case is obtained, the case file of the target test case can be analyzed by using an XML analysis technology to obtain the hierarchical structure of the target test case.
S302, parameter information of the target test case is extracted from the hierarchical structure of the target test case.
Optionally, the following manner may be adopted to extract the parameter information of the target test case from the hierarchical structure of the target test case.
Fig. 4 is a flowchart illustrating a further method for analyzing an execution result of a test case according to an embodiment of the present application, optionally, as shown in fig. 4, in step S302: extracting parameter information of the target test case from the hierarchical structure of the target test case, wherein the parameter information comprises the following steps:
s401, extracting parameter information of the target test case according to the frame structure of the target test case information file.
The frame structure of the target use case information file can be defined in advance, and the frame structure of the target use case information file can be flexibly defined according to actual conditions and requirements. This frame construction mainly includes: information of each hierarchy structure constituting the test case, execution environment information of the test case, and the like. For example, the parameter information of the frame structure includes, but is not limited to: the module name, the case name, the keywords, the operation function and the storage position of the test case.
Therefore, in this embodiment, a plurality of parameter information of the target test case may be obtained according to the frame structure of the target test case information file. For example, the acquired parameter information of the test case a includes: use case names, keywords, operation functions, storage locations, execution environments, and the like.
S402, writing the parameter information of the target test case into an information file of the target test case.
On the basis of the above embodiment, after the parameter information of the target test case is extracted, the parameter information of the target test case can be written into the information file of the target test case to generate the information file of the target test case.
In addition, common parameters and variables of the universality in the target test case can be extracted, and the parameter information, the common parameters and the variables of the universality of the target test case are written into an information file of the target test case together to generate an information file of the target test case, so that key data support is provided for the follow-up analysis of failure error reporting information in a test report and the analysis of the root cause of the execution failure of the test case.
S303, analyzing the target test report by using the parameter information of the target case to obtain the failure error reporting information of the target test case.
Alternatively, the target test report may be parsed as follows.
Optionally, the parameter information in the information file of the target test case is used to analyze the target test report, so as to obtain the execution result and the reason information of each parameter in the target test case.
The execution result of each parameter refers to whether each parameter in the target test case fails to be executed or passes being executed. For example, the execution result of the key N in the "target test case" is execution pass, and the execution result of the operation function P in the key M is execution failure.
The reason information is failure error reporting detail information of the parameter that failed to execute in the "target test case". For example, the failure error-reporting detailed information of the execution failure of the operation function P in the keyword M in the "target test case" is, for example, that one variable in the operation function P exceeds a set value.
Specifically, in this embodiment, for example, the parameter information in the information file of the target test case generated as described above, such as parameter information of a case name, a keyword, an operation function, a storage location, an execution environment, and the like, may be used to analyze the test report of the target test case, and the "execution result" and the "cause information" may be obtained after the execution of each parameter in the "target test case" in which the execution fails.
According to the method and the device, the execution result and the reason information can be obtained after the obtained parameters are executed, the keywords which fail to be executed in the target test case and the positions of the operation functions which are wrongly reported in the keywords can be timely located, so that the test case which fails to be executed can be conveniently subjected to fault recovery, the workload of manually checking and correcting errors can be greatly reduced, the checking and modifying efficiency of the test case which fails to be executed is improved, and the testing efficiency is further ensured.
In the embodiment provided by the application, it is considered that in an actual testing process, because a plurality of test cases have a certain degree of coupling, some operation steps may repeatedly occur in different test cases, and finally, the same failure cause may cause the execution failure of all the test cases. In the process of analysis and check by adopting a manual mode, because whether the failure cause of each test case is the same or not cannot be predicted, all the failed cases can be checked one by one, and the failure analysis efficiency of the target test case with execution failure is low.
Therefore, it will be specifically explained by the following embodiments how to perform classification statistics on the "failure analysis results of the target test cases" with multiple execution failures after determining the failure analysis results of the target test cases according to the failure error reporting information of the target test cases, so as to find the commonality of the failure reasons of the "target test cases" with multiple execution failures.
Fig. 5 is a flowchart illustrating another method for analyzing an execution result of a test case according to an embodiment of the present application, and optionally, as shown in fig. 5, in step S203: after determining the failure analysis result of the target test case according to the failure error reporting information of the target test case, the method further comprises the following steps:
s501, classifying the target test cases according to the failure analysis results of the target test cases.
Optionally, before classifying each target test case, data cleaning may be performed on the failure error reporting information to obtain the cleaned failure error reporting information.
Specifically, after the failure error reporting information is obtained, the failure error reporting information can be subjected to data cleaning according to the effectiveness of the failure error reporting information and the coupling degree of different levels of information under the same test case, useless information and repeated information in the failure error reporting information are removed, the cleaned failure error reporting information is obtained, and the cleaned failure error reporting information is stored, so that the cleaned failure error reporting information can be subjected to subsequent processing.
For example, after the failure analysis results of a plurality of target test cases are obtained, all the test cases which fail to be executed may be classified and counted according to the failure analysis results of each target test case.
S502, obtaining a plurality of test case sets according to the classification result, wherein the similarity of the execution failure reasons of the test cases in each test case set meets the preset condition.
In this embodiment, the obtained classification result may be counted to obtain a set including a plurality of test cases, and the plurality of test cases in the same set have the same failure reason or the failure reasons are very similar, so that a tester may quickly and accurately implement fault analysis on the failure reasons of all test cases that fail to be executed in an automated test task according to the plurality of test case sets in the classification result, and further quickly locate a node where a test case has an error in the execution process, and need not repeatedly check and analyze the same error reason, thereby improving the efficiency of troubleshooting and modifying a target test case, and further improving the efficiency of automated testing.
The following embodiments specifically explain how to classify each target test case according to the failure analysis result of each target test case.
Fig. 6 is a flowchart illustrating a further method for analyzing an execution result of a test case according to an embodiment of the present application, and optionally, as shown in fig. 6, the step S501: classifying the target test cases according to the failure analysis result of each target test case, wherein the classification comprises the following steps:
s601, calculating the similarity of the failure reasons of the target test cases according to the preset weight of each parameter in the failure reasons of the target test cases.
It can be understood that, in the test case execution process, the same error in the hierarchical structure of the same test case is not necessarily identical in the generated failure error reporting information under different test cases and different test environments. Therefore, all parameters in failure reasons of a plurality of test cases which fail to be executed may not be completely the same, but all parameters are caused by the same error to fail to be executed by the 'plurality of target test cases', that is, failure reasons of the 'plurality of target test cases' are the same.
In order to solve this problem, in this example, the similarity of the failure reasons of each target test case may be calculated according to the preset weight of each parameter in the failure reasons of each target test case and the similarity of each parameter.
For example, the failure reasons in the test case include four parameters, i.e., a, b, c, and d, and the preset weight ratios are 10%, 20%, 30%, and 40%, respectively. The parameters of the failure cause of the test case a are a1, B1, c1 and d1, and the parameters of the failure root cause of the test case B are a2, B2, c2 and d2, so that: the similarity between the failure reasons of test case a and test case B is 10%. a12+ 20%. B12+ 30%. c12+ 40%. d12, where a12 is the similarity between parameters a1 and a2, B12 is the similarity between parameters B1 and B2, and so on, c12 and d 12.
In another realizable manner, the similarity of the failure reasons of each target test case may not be calculated according to the preset weight of each parameter in the failure reasons of each target test case. Specifically, the failure reason of each target test case may be obtained first, and the influence of each parameter in each target test case on the final execution error report result may be statistically analyzed. For example, if changing one of the parameters in the "target tests" causes a change in the execution result or the cause information of the "target tests", the weight ratio of each parameter to the failure cause in the "target tests" is calculated, and then the similarity of the failure causes of the target test cases is further calculated based on the weight ratio of each parameter to the failure cause in the "target tests".
S602, classifying the target test cases according to the similarity of the failure reasons of the target test cases.
On the basis of the above embodiment, when the similarity of the failure reasons of the "multiple test cases" falls within the same defined value range, the failure reasons of the "multiple test cases" of this class may be classified into the same class.
For example, the defined value of the similarity may be calculated and obtained by decrementing the setting multiple times. For example, after setting the range of the bounded values to 95%, 90%, 80%, 70%, 60%, respectively, each range of the bounded values corresponds to one classification set.
For example, if the similarity of the failure reasons of the test case a and the test case B is 97%, the test case a and the test case B may be classified into the first classification set, and so on, each target test case may be classified to obtain a classification result.
In the example, statistics can be further performed on the number of the test cases in each obtained test case set, and sorts and arranges the test report, the information file of the test case and the failure reason of each test case which fails to be executed, and converts the test report of each test case with failed execution, the information file of the test case, the failure reason into a file including but not limited to XLS form, and storing XLS form file as statistical result, the statistical result can support printing, dumping, writing and other forms and output to external processing equipment, the external processing equipment includes data base, text file, storage medium and so on, the method helps testers to quickly investigate and solve the reason of test case execution failure in one test task, can greatly reduce labor cost, and improves the investigation and modification efficiency of target test.
It can be understood that, when the number of the test cases with execution failures in the same class is larger (that is, the number of times of occurrence of the same failure reason is larger), the efficiency of troubleshooting the failure reason in the test cases with execution failures is improved more by the test case execution result analysis method provided by the embodiment.
The following embodiments will specifically explain the overall procedures of the test case execution result analysis method provided in the present application.
Fig. 7 is a schematic overall flow diagram of a method for analyzing an execution result of a test case according to an embodiment of the present application, and optionally, as shown in fig. 7, the method includes:
s701, acquiring a target test report of the target test case with execution failure.
S702, analyzing the case file of the target test case to obtain the hierarchical structure of the target test case.
S703, extracting parameter information of the target test case from the hierarchical structure of the target test case.
S704, extracting parameter information of the target test case according to the frame structure of the information file of the target test case; and writing the parameter information of the target test case into the target case information file.
S705, analyzing the target test report by using the parameter information in the information file of the target test case to obtain the execution result and the reason information of each parameter in the target test case.
S706, calculating the similarity of the failure reasons of the target test cases according to the preset weight of each parameter in the failure reasons of the target test cases.
And S707, classifying the target test cases according to the similarity of the failure reasons of the target test cases.
S708, obtaining a plurality of test case sets according to the classification result, wherein the similarity of the execution failure reasons of the test cases in each test case set meets the preset condition.
Optionally, the overall implementation steps and the generated beneficial effects of the test case execution result analysis method provided in the embodiment of the present application have been described in detail in the foregoing specific embodiments, and are not described in detail here.
The following describes a device and a storage medium for analyzing the execution result of the test case provided by the present application, and the specific implementation process and technical effects thereof are referred to above, and will not be described in detail below.
Fig. 8 is a schematic structural diagram of an apparatus for analyzing test case execution results according to an embodiment of the present disclosure; as shown in fig. 8, the apparatus includes: an acquisition module 801, an analysis module 802, and a determination module 803.
An obtaining module 801, configured to obtain a target test report of a target test case that fails to be executed;
the analysis module 802 is configured to analyze the target test report to obtain failure error reporting information of the target test case;
the determining module 803 is configured to determine a failure analysis result of the target test case according to the failure error reporting information of the target test case.
Optionally, the parsing module 802 is further configured to:
analyzing a case file of the target test case to obtain a hierarchical structure of the target test case;
extracting parameter information of the target test case from the hierarchical structure of the target test case;
and analyzing the target test report by using the parameter information of the target case to obtain the failure error reporting information of the target test case.
Optionally, the parsing module 802 is further configured to:
extracting parameter information of the target test case according to a frame structure of an information file of the target test case;
and writing the parameter information of the target test case into an information file of the target test case.
Optionally, the parsing module 802 is further configured to:
and analyzing the target test report by using the parameter information in the information file of the target test case to obtain the execution result and the reason information of each parameter in the target test case.
Optionally, the apparatus further comprises:
the classification module is used for classifying the target test cases according to the failure analysis result of each target test case; and obtaining a plurality of test case sets according to the classification result, wherein the similarity of the execution failure reasons of the test cases in each test case set meets the preset condition.
Optionally, the classification module is further configured to:
calculating the similarity of the failure reasons of the target test cases according to the preset weight of each parameter in the failure reasons of the target test cases;
and classifying the target test cases according to the similarity of the failure reasons of the target test cases.
Optionally, the apparatus further comprises:
and the cleaning module is used for carrying out data cleaning on the failure error reporting information to obtain the cleaned failure error reporting information.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Optionally, the invention also provides a program product, for example a computer-readable storage medium, comprising a program which, when being executed by a processor, is adapted to carry out the above-mentioned method embodiments.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Claims (10)
1. A method for analyzing test case execution results is characterized by comprising the following steps:
acquiring a target test report of a target test case with execution failure;
analyzing the target test report to obtain failure error reporting information of the target test case;
and determining a failure analysis result of the target test case according to the failure error reporting information of the target test case.
2. The method of claim 1, wherein the analyzing the target test report to obtain failure error reporting information of the target test case comprises:
analyzing the case file of the target test case to obtain the hierarchical structure of the target test case;
extracting parameter information of the target test case from the hierarchical structure of the target test case;
and analyzing the target test report by using the parameter information of the target case to obtain the failure error reporting information of the target test case.
3. The method according to claim 2, wherein the extracting the parameter information of the target test case from the hierarchical structure of the target test case comprises:
extracting parameter information of a target test case according to a frame structure of an information file of the target test case;
and writing the parameter information of the target test case into an information file of the target test case.
4. The method of claim 3, wherein the analyzing the target test report using the parameter information of the target use case to obtain failure error reporting information of the target test case comprises:
and analyzing the target test report by using the parameter information in the information file of the target test case to obtain the execution result and reason information of each parameter in the target test case.
5. The method according to any one of claims 1 to 3, wherein after determining the result of the failure analysis of the target test case according to the failure error reporting information of the target test case, the method further comprises:
classifying the target test cases according to the failure analysis result of each target test case;
and obtaining a plurality of test case sets according to the classification result, wherein the similarity of the execution failure reasons of the test cases in each test case set meets the preset condition.
6. The method of claim 5, wherein the classifying each of the target test cases according to the result of the failure analysis of each of the target test cases comprises:
calculating the similarity of the failure reasons of the target test cases according to the preset weight of each parameter in the failure reasons of the target test cases;
and classifying the target test cases according to the similarity of the failure reasons of the target test cases.
7. The method according to any one of claims 1 to 3, wherein after determining the result of the failure analysis of the test case that fails to execute according to the failure error reporting information of the target test case, the method further comprises:
and carrying out data cleaning on the failure error reporting information to obtain the cleaned failure error reporting information.
8. An apparatus for analyzing a test case execution result, the apparatus comprising:
the acquisition module is used for acquiring a target test report of a target test case with execution failure;
the analysis module is used for analyzing the target test report to obtain failure error reporting information of the target test case;
and the determining module is used for determining the failure analysis result of the target test case according to the failure error reporting information of the target test case.
9. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the method according to any one of claims 1 to 7.
10. A storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110650500.XA CN113392000B (en) | 2021-06-10 | 2021-06-10 | Test case execution result analysis method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110650500.XA CN113392000B (en) | 2021-06-10 | 2021-06-10 | Test case execution result analysis method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113392000A true CN113392000A (en) | 2021-09-14 |
| CN113392000B CN113392000B (en) | 2024-01-30 |
Family
ID=77620429
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110650500.XA Active CN113392000B (en) | 2021-06-10 | 2021-06-10 | Test case execution result analysis method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113392000B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114721936A (en) * | 2022-02-28 | 2022-07-08 | 阿里巴巴（中国）有限公司 | Data processing method, electronic device, medium, and program product |
| CN115048315A (en) * | 2022-07-06 | 2022-09-13 | 东软睿驰汽车技术（沈阳）有限公司 | Analysis method and device for automatic driving simulation test result and electronic equipment |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521124A (en) * | 2011-11-29 | 2012-06-27 | 大唐移动通信设备有限公司 | Method and system for outputting failure reasons |
| US20130151906A1 (en) * | 2011-12-08 | 2013-06-13 | International Business Machines Corporation | Analysis of Tests of Software Programs Based on Classification of Failed Test Cases |
| CN107145445A (en) * | 2017-05-05 | 2017-09-08 | 携程旅游信息技术（上海）有限公司 | The automatic analysis method and system of the daily record that reports an error of software automated testing |
| WO2018000607A1 (en) * | 2016-06-30 | 2018-01-04 | 乐视控股（北京）有限公司 | Method and electronic apparatus for identifying test case failure causes |
| CN109144882A (en) * | 2018-09-19 | 2019-01-04 | 哈尔滨工业大学 | A kind of software fault positioning method and device based on program invariants |
| CN109344060A (en) * | 2018-09-20 | 2019-02-15 | 迈普通信技术股份有限公司 | A kind of analysis method and device of automatic test |
| CN109491891A (en) * | 2018-10-17 | 2019-03-19 | 深圳壹账通智能科技有限公司 | Full-automatic interface pressure surveys method, apparatus, computer equipment and storage medium |
| CN110287115A (en) * | 2019-06-26 | 2019-09-27 | 北京金山云网络技术有限公司 | Generation method, device and the server of test report |
| CN110990575A (en) * | 2019-12-18 | 2020-04-10 | 斑马网络技术有限公司 | Test case failure reason analysis method and device and electronic equipment |
| CN111831556A (en) * | 2020-06-18 | 2020-10-27 | 中国科学院空间应用工程与技术中心 | Software multi-fault decoupling and parallel positioning method and device |
| US20200394127A1 (en) * | 2019-06-13 | 2020-12-17 | International Business Machines Corporation | Fault detection using breakpoint value-based fingerprints of failing regression test cases |
| AU2020103310A4 (en) * | 2020-11-09 | 2021-01-14 | A. Bhuvaneswari | Method for minimizing the time consumption in software testing process |
| CN112363920A (en) * | 2020-11-06 | 2021-02-12 | 广州品唯软件有限公司 | Test case repairing method and device, computer equipment and storage medium |
| US20210064515A1 (en) * | 2019-08-27 | 2021-03-04 | Nec Laboratories America, Inc. | Deep q-network reinforcement learning for testing case selection and prioritization |
-
2021
- 2021-06-10 CN CN202110650500.XA patent/CN113392000B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521124A (en) * | 2011-11-29 | 2012-06-27 | 大唐移动通信设备有限公司 | Method and system for outputting failure reasons |
| US20130151906A1 (en) * | 2011-12-08 | 2013-06-13 | International Business Machines Corporation | Analysis of Tests of Software Programs Based on Classification of Failed Test Cases |
| WO2018000607A1 (en) * | 2016-06-30 | 2018-01-04 | 乐视控股（北京）有限公司 | Method and electronic apparatus for identifying test case failure causes |
| CN107145445A (en) * | 2017-05-05 | 2017-09-08 | 携程旅游信息技术（上海）有限公司 | The automatic analysis method and system of the daily record that reports an error of software automated testing |
| CN109144882A (en) * | 2018-09-19 | 2019-01-04 | 哈尔滨工业大学 | A kind of software fault positioning method and device based on program invariants |
| CN109344060A (en) * | 2018-09-20 | 2019-02-15 | 迈普通信技术股份有限公司 | A kind of analysis method and device of automatic test |
| CN109491891A (en) * | 2018-10-17 | 2019-03-19 | 深圳壹账通智能科技有限公司 | Full-automatic interface pressure surveys method, apparatus, computer equipment and storage medium |
| US20200394127A1 (en) * | 2019-06-13 | 2020-12-17 | International Business Machines Corporation | Fault detection using breakpoint value-based fingerprints of failing regression test cases |
| CN110287115A (en) * | 2019-06-26 | 2019-09-27 | 北京金山云网络技术有限公司 | Generation method, device and the server of test report |
| US20210064515A1 (en) * | 2019-08-27 | 2021-03-04 | Nec Laboratories America, Inc. | Deep q-network reinforcement learning for testing case selection and prioritization |
| CN110990575A (en) * | 2019-12-18 | 2020-04-10 | 斑马网络技术有限公司 | Test case failure reason analysis method and device and electronic equipment |
| CN111831556A (en) * | 2020-06-18 | 2020-10-27 | 中国科学院空间应用工程与技术中心 | Software multi-fault decoupling and parallel positioning method and device |
| CN112363920A (en) * | 2020-11-06 | 2021-02-12 | 广州品唯软件有限公司 | Test case repairing method and device, computer equipment and storage medium |
| AU2020103310A4 (en) * | 2020-11-09 | 2021-01-14 | A. Bhuvaneswari | Method for minimizing the time consumption in software testing process |
Non-Patent Citations (4)
| Title |
|---|
| BRUNO AUDONE: "A novel method to characterize immunity pass/no pass test results", 《2013 INTERNATIONAL SYMPOSIUM ON ELECTROMAGNETIC COMPATIBILITY》, pages 1 - 9 * |
| ROOT_123: "自动化用例执行失败问题定位分析", pages 1 - 6, Retrieved from the Internet <URL:《 https://www.jianshu.com/p/85017e483522 》> * |
| 姜文: "自动化测试用例测试失败类型分析", 《计算机技术发展》, pages 1 - 7 * |
| 王兴亚: "基于模糊C均值聚类的软件多缺陷定位方法", 《计算机学报》, pages 206 - 232 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114721936A (en) * | 2022-02-28 | 2022-07-08 | 阿里巴巴（中国）有限公司 | Data processing method, electronic device, medium, and program product |
| CN114721936B (en) * | 2022-02-28 | 2024-09-17 | 阿里巴巴（中国）有限公司 | Data processing method, electronic device, medium, and program product |
| CN115048315A (en) * | 2022-07-06 | 2022-09-13 | 东软睿驰汽车技术（沈阳）有限公司 | Analysis method and device for automatic driving simulation test result and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113392000B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11386154B2 (en) | Method for generating a graph model for monitoring machinery health | |
| Abreu et al. | On the accuracy of spectrum-based fault localization | |
| US20080313616A1 (en) | Methods and systems for testing tool with comparative testing | |
| US8607198B2 (en) | Cross-concern code coverage assessment | |
| CN112131830B (en) | Parasitic parameter verification method and device, electronic equipment and storage medium | |
| CN112308126A (en) | Fault recognition model training method, fault recognition device and electronic equipment | |
| CN105912413B (en) | Method and device for evaluating the availability of a system, in particular a safety-critical system | |
| US11429574B2 (en) | Computer system diagnostic log chain | |
| CN113392000B (en) | Test case execution result analysis method, device, equipment and storage medium | |
| CN113672456A (en) | Modular self-monitoring method, system, terminal and storage medium of application platform | |
| US8327189B1 (en) | Diagnosing an incident on a computer system using a diagnostics analyzer database | |
| CN112486841B (en) | Method and device for verifying buried point acquisition data | |
| CN112769615B (en) | Anomaly analysis method and device | |
| CN114625554A (en) | Fault repairing method and device, electronic equipment and storage medium | |
| US11790249B1 (en) | Automatically evaluating application architecture through architecture-as-code | |
| CN111767213A (en) | Method and device for testing database check points, electronic equipment and storage medium | |
| CN113282496B (en) | Automatic interface testing method, device, equipment and storage medium | |
| CN114491044A (en) | Log processing method and device | |
| CN114020645A (en) | Test method, device, equipment, readable storage medium and computer program product | |
| CN113722229A (en) | Software testing method and device, electronic equipment and storage medium | |
| CN114253846B (en) | Automatic test abnormality positioning method, device, equipment and readable storage medium | |
| CN112069749B (en) | Power supply connection verification method and device, electronic equipment and storage medium | |
| CN116846741B (en) | Alarm convergence method, device, equipment and storage medium | |
| US11907628B2 (en) | Message signoffs | |
| CN118331843B (en) | Hierarchical data automatic test method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |