CN113032254B - Test coverage condition evaluation method and device - Google Patents
Test coverage condition evaluation method and device Download PDFInfo
- Publication number
- CN113032254B CN113032254B CN202110294920.9A CN202110294920A CN113032254B CN 113032254 B CN113032254 B CN 113032254B CN 202110294920 A CN202110294920 A CN 202110294920A CN 113032254 B CN113032254 B CN 113032254B
- Authority
- CN
- China
- Prior art keywords
- log
- calling
- relation
- standardized
- record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 166
- 238000011156 evaluation Methods 0.000 title claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 191
- 238000004458 analytical method Methods 0.000 claims abstract description 74
- 238000005065 mining Methods 0.000 claims abstract description 38
- 238000007639 printing Methods 0.000 claims abstract description 34
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 15
- 230000007812 deficiency Effects 0.000 abstract description 9
- 238000004445 quantitative analysis Methods 0.000 abstract description 6
- 230000009191 jumping Effects 0.000 description 20
- 238000010586 diagram Methods 0.000 description 18
- 230000006870 function Effects 0.000 description 6
- 239000003795 chemical substances by application Substances 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 239000000243 solution Substances 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 230000008676 import Effects 0.000 description 2
- 238000002347 injection Methods 0.000 description 2
- 239000007924 injection Substances 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012946 outsourcing Methods 0.000 description 1
- 238000005498 polishing Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 230000000246 remedial effect Effects 0.000 description 1
- 238000012502 risk assessment Methods 0.000 description 1
- 238000000528 statistical test Methods 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a test coverage condition evaluation method and a test coverage condition evaluation device, which can be used in the financial field or other fields, wherein the method comprises the following steps: determining a method call relation between programs according to source codes of the tested application; performing log instrumentation according to a method calling relation between programs, and determining a printing log format; executing the tested application according to the printing log format, outputting a standardized log, performing mining analysis on the standardized log, and determining the actual executing calling relation; and determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs. The invention solves the problem that the actual test progress situation is difficult to acquire, by using the invention, the situation that the program calling relation is actually covered can be monitored, the test coverage situation of the tested application can be estimated, the test risk can be predicted, the test deficiency can be found out, the supplementary test can be made, and the quantitative analysis and the timely risk early warning can be carried out on the actual test progress.
Description
Technical Field
The present invention relates to the field of computer testing technologies, and in particular, to a method and an apparatus for evaluating coverage of a test.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
At present, in the system testing stage, it is difficult to accurately acquire the actual system testing coverage condition in time.
The existing solution for acquiring the test coverage rate is based on the execution condition use list management of test cases, or reflects the test progress according to the test conditions reported by each tester; the test case is set to be the executed or reported progress which cannot represent the situation that the program is actually covered, so that the statistical test progress report is distorted, has deviation from the actual coverage, cannot reflect the actual program coverage, and the program coverage missing problem is difficult to find, so that the corresponding project risk is difficult to find in time in the project process, and corresponding remedial measures are made; especially in the case of test outsourcing, the situation of test progress distortion is particularly severe, often the situation that the final stage is checked for test coverage by acceptance testing.
Therefore, how to provide a new solution to the above technical problem is a technical problem to be solved in the art.
Disclosure of Invention
The embodiment of the invention provides a test coverage condition assessment method, which solves the problem that the actual test progress condition is difficult to acquire, by using the method, the condition that the program calling relationship is actually covered can be monitored, the test coverage condition of the tested application is assessed, the test risk is predicted, the test deficiency is found out, the supplementary test is made, and the actual test progress is quantitatively analyzed and timely risk early warning is carried out, and the method comprises the following steps:
determining a method call relation between programs according to source codes of the tested application;
Performing log instrumentation according to a method calling relation between programs, and determining a printing log format;
executing the tested application according to the printing log format, outputting a standardized log, performing mining analysis on the standardized log, and determining the actual executing calling relation;
and determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs.
The embodiment of the invention also provides an evaluation device for testing coverage condition, which comprises:
The program method call relation analysis module is used for determining a method call relation between programs according to the source code of the tested application;
The program instrumentation module is used for performing log instrumentation according to the method call relation among the programs and determining a printing log format;
the log mining analysis module is used for executing the tested application according to the printing log format, outputting a standardized log, carrying out mining analysis on the standardized log and determining the actual executing calling relation;
And the coverage rate evaluation module is used for determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the evaluation method of the test coverage condition when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for executing the evaluation method for testing coverage condition.
The method and the device for evaluating the test coverage condition provided by the embodiment of the invention comprise the following steps: firstly, determining a method call relation between programs according to source codes of tested applications; then log instrumentation is carried out according to the method calling relation among programs, and a printing log format is determined; then executing the tested application according to the printing log format, outputting a standardized log, carrying out mining analysis on the standardized log, and determining the actual executing calling relation; and finally, determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs. The embodiment of the invention obtains the source code from the tested application, analyzes and obtains the method call relation between the whole programs, obtains the actual execution call relation through log instrumentation and log mining analysis, and can obtain the test coverage condition of the tested application through mutual comparison; the invention solves the problem that the actual test progress situation is difficult to acquire, by using the invention, the situation that the program calling relation is actually covered can be monitored, the test coverage situation of the tested application can be estimated, the test risk can be predicted, the test deficiency can be found out, the supplementary test can be made, and the quantitative analysis and the timely risk early warning can be carried out on the actual test progress.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
fig. 1 is a schematic diagram of an evaluation method for testing coverage according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of call relation xml file of an evaluation method for testing coverage condition according to an embodiment of the present invention.
Fig. 3 is a flowchart illustrating a method call relationship determination between programs of an evaluation method for testing coverage according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of a method call relationship between procedures of an evaluation method for testing coverage according to an embodiment of the present invention.
Fig. 5 is a flowchart illustrating a pile driver processing procedure of an evaluation method for testing coverage according to an embodiment of the present invention.
FIG. 6 is a flow chart of a data analysis of a call relationship matching analysis tool for an evaluation method for testing coverage conditions according to an embodiment of the present invention.
FIG. 7 is a schematic diagram of a computer device running a test coverage assessment method embodying the present invention.
Fig. 8 is a schematic diagram of an evaluation apparatus for testing coverage according to an embodiment of the present invention.
Fig. 9 is a schematic diagram of another embodiment of an evaluation device for testing coverage according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention. It should be noted that the method and the device for evaluating the test coverage condition of the present invention can be used in the financial field and any field other than the financial field, and the application field of the method and the device for evaluating the test coverage condition of the present invention is not limited.
Fig. 1 is a schematic diagram of a test coverage condition evaluation method according to an embodiment of the present invention, as shown in fig. 1, where the problem that an actual test progress condition is difficult to obtain is solved, by using the present invention, a condition that a program call relationship is actually covered can be monitored, a test coverage condition of a tested application is evaluated, a test risk is predicted, a test deficiency is found, a supplemental test is made, and an actual test progress is quantitatively analyzed and timely risk early warning is performed, where the method includes:
step A01: determining a method call relation between programs according to source codes of the tested application;
step A02: performing log instrumentation according to a method calling relation between programs, and determining a printing log format;
Step A03: executing the tested application according to the printing log format, outputting a standardized log, performing mining analysis on the standardized log, and determining the actual executing calling relation;
step A04: and determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs.
The method and the device for evaluating the test coverage condition provided by the embodiment of the invention comprise the following steps: firstly, determining a method call relation between programs according to source codes of tested applications; then log instrumentation is carried out according to the method calling relation among programs, and a printing log format is determined; then executing the tested application according to the printing log format, outputting a standardized log, carrying out mining analysis on the standardized log, and determining the actual executing calling relation; and finally, determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs. The embodiment of the invention obtains the source code from the tested application, analyzes and obtains the method call relation between the whole programs, obtains the actual execution call relation through log instrumentation and log mining analysis, and can obtain the test coverage condition of the tested application through mutual comparison; the invention solves the problem that the actual test progress situation is difficult to acquire, by using the invention, the situation that the program calling relation is actually covered can be monitored, the test coverage situation of the tested application can be estimated, the test risk can be predicted, the test deficiency can be found out, the supplementary test can be made, and the quantitative analysis and the timely risk early warning can be carried out on the actual test progress.
The embodiment of the invention provides an evaluation method for testing coverage conditions based on program method call, which is characterized in that the method call relationship between the whole classes is obtained from the source code analysis program call relationship of the tested application engineering package, the actual execution conditions of the call relationship between the classes are obtained through log instrumentation and mining, and the testing coverage conditions of the whole project are evaluated according to the summarized execution conditions and the call relationship of the whole class method. By using the invention, the condition that the program calling relation is actually covered and the test times thereof can be monitored, the test progress is evaluated, the test risk is predicted, the test deficiency is found out, and the supplementary test is made. When the method for evaluating the coverage test provided by the embodiment of the invention is implemented, the method can comprise the following steps:
determining a method call relation between programs according to source codes of the tested application;
Performing log instrumentation according to a method calling relation between programs, and determining a printing log format;
executing the tested application according to the printing log format, outputting a standardized log, performing mining analysis on the standardized log, and determining the actual executing calling relation;
and determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs.
When the method for evaluating the coverage test provided by the embodiment of the invention is implemented, in one embodiment, the method call relationship between the programs is determined according to the source code of the tested application, and the method includes:
Acquiring a source code of an application to be tested from an engineering package of a code warehouse; wherein, the source code may include: java code;
analyzing the source code into an abstract syntax tree by using a syntax analysis tool; the grammar analysis tool can use a development tool carried by an Eclipse development platform;
traversing all abstract grammar trees to extract calling relations among classes and generating calling relation xml files which store the calling relations among class methods;
obtaining an xml record from a calling relation xml file, and analyzing which methods in other classes are specifically called by a certain method of the class according to < Import >, < VariableDeclarationStatement > and < ExpressionStatement > tags in the calling relation xml file;
judging whether the calling party and the called party belong to the calling between the class internal methods;
If the calling party and the called party do not belong to the call between the class internal methods, judging whether the called party is an interface or an abstract class;
If the called party is an interface or an abstract class, acquiring a corresponding realization class name through a configuration file containing dependency injection;
saving the calling relation between the calling class and the called class;
Judging whether the analysis of the call relation xml file is completed or not, and outputting the call relation between the realization class name and the call class and the called class as the method call relation between the programs if the analysis of the call relation xml file is completed;
If the calling party and the called party belong to the calling among the methods in the class, directly jumping to judge whether the analysis of the calling relation xml file is completed or not;
if the called party is not an interface or an abstract class, directly jumping to save the calling relation between the calling class and the called class;
If the analysis of the call relation xml file is not completed, directly jumping to obtain a new xml record from the call relation xml file.
Fig. 3 is a flowchart for determining a method call relationship between programs of an evaluation method for testing coverage conditions according to an embodiment of the present invention, as shown in fig. 3, in the embodiment, the determining a method call relationship between programs according to source codes of an application to be tested may include the following steps:
S101: acquiring source codes of the tested application from a code warehouse;
s102: the grammar analysis tool can use the development tool of the Eclipse development platform to analyze, and can analyze Java codes into abstract grammar trees by using the grammar analysis tool;
S103: traversing all abstract grammar trees to extract calling relations among classes, and finally generating a calling relation xml file (shown as a schematic diagram of the calling relation xml file of an evaluation method for testing coverage conditions in the embodiment of the invention in fig. 2) which stores the calling relations among class methods;
S104: obtaining an xml record (shown in fig. 2), and analyzing which methods in other classes are specifically called by a certain method of the class according to the < Import >, < VariableDeclarationStatement > and < ExpressionStatement > tags in the calling relation xml file;
S105: judging whether the calling party and the called party belong to the calling between the class internal methods, if so, jumping to S109; if not, jumping to S106;
S106: judging whether the called party is an interface or an abstract class, if yes, jumping to S107, and if not, jumping to S108;
S107: the method comprises the steps that a corresponding realization class name is obtained through configuration files such as dependency injection and the like when the interface or abstract class belongs to;
s108: saving the calling relation between the calling class and the called class;
s109: and judging whether the xml analysis is completed or not, if not, jumping to S104, and if so, outputting the realization class name and the calling relation between the calling class and the called class as the method calling relation between the programs.
Fig. 4 is a schematic diagram of a method call relationship between programs of an evaluation method for testing coverage conditions according to an embodiment of the present invention, as shown in fig. 4, the method call relationship between programs finally output includes: source, target, isinterface, interfacename four classes; wherein source represents a method path of the calling party class; target represents the method path of the called party instance class, which contains parameter types of the parameter; isinterface is 1 to indicate that an interface or abstract class is called; INTERFACENAME denotes the called party interface or abstract class name, method, and its type of entry.
When the method for evaluating the coverage test provided by the embodiment of the present invention is implemented, in one embodiment, before log instrumentation according to the method call relationship between programs, the method further includes:
and eliminating call relations with business context relations smaller than a preset threshold value in the method call relations among the programs and call relation paths which cannot be covered by actual business.
In the embodiment, because the calling relations under the whole engineering package are more, before log instrumentation is carried out according to the method calling relations among programs, the calling relations with small business context relations can be subjected to mark elimination processing, for example, the simple data transmission object calling, the database object calling and the like can be manually eliminated; if part of the paths are impossible to cover by the actual business, the mark elimination processing can be carried out from a database which stores the method call relation among the programs.
When the method for evaluating the coverage test provided by the embodiment of the invention is implemented, in one embodiment, log instrumentation is performed according to the method call relationship between programs, and a print log format is determined, which includes:
summarizing a called party method set according to a method calling relation among programs;
Summarizing a calling party method set according to a method calling relation among programs;
acquiring records from a called party method set to obtain a called party method;
Log instrumentation in a called party method, comprising: a unified log is inserted into each called method body, the log shows information such as time, thread id, class name, method name and parameter value transmitted by a calling party when the program method is called, a hash value is used for defaulting the parameter value when the log is written, and an original value is used if the parameter value is of a basic data type such as integer, character type and byte type;
judging whether the called party method set has records or not, and if the called party method set has records, acquiring the records in the called party method set again;
if the called party method set does not have the record, acquiring the record from the calling party method set to obtain a calling party method;
Log instrumentation is performed before the calling party method, including: locating the place of the calling program according to the class name, the method and the parameter type of the called party, and inserting a unified log, wherein the log reflects the information such as time, thread id, the class name of the calling party, the class name of the called party, the method name of the called party, the parameter value and the like when the calling party calls other program methods, and the parameter value is replaced by a hash value by default when the log is written, and if the parameter value is of a basic data type such as integer, character type, byte type and the like, the original value is used;
judging whether the calling party method set has records or not, and if the calling party method set has records, acquiring the records in the calling party method set again;
If the calling party method set does not have records, packaging the instrumentation program into a jar packet and deploying the jar packet into a test environment, adding the starting parameters of the javaagent jar packet on a starting statement of the program of the tested application, enabling the instrumentation program to run before the tested application, and outputting a printing log format.
Fig. 5 is a flowchart of a instrumentation program processing of a test coverage evaluation method according to an embodiment of the present invention, as shown in fig. 5, in the embodiment, log instrumentation is processed based on a method call relationship between the obtained programs, and a java agent instrumentation program may be written by using a dynamic agent tool, such as a javassit tool, and the specific process may include:
S201: summarizing a called party method set according to a method calling relation among programs, wherein the method comprises the following steps: summarizing a called party method set according to class names, methods and dimensions of the called party, and de-duplicating the set;
s202: summarizing a calling party method set according to a method calling relation among programs, wherein the summarizing calling party method set comprises the following steps: summarizing a calling party method set according to class names, methods and dimensions of the entering type of the calling party, and de-duplicating the set;
S203: acquiring records from a called party method set to obtain a called party method;
S204: log instrumentation is carried out in the called party method, a unified log is inserted in each called party method body, the log reflects information such as time, thread id, class name, method name and parameter value transmitted by the calling party when the program method is called, a hash value is used for defaulting the parameter value when the log is written, and an original value is used if the parameter value is of a basic data type such as integer, character type and byte type;
S205: judging whether the called party method set has records, if so, jumping to S203, and if not, jumping to S206;
S206: acquiring records from a calling party method set to obtain a calling party method;
S207: log instrumentation is carried out before a calling party method, the logging is positioned to a calling program place according to the class name, the method and the parameter type of a called party, and a unified log is inserted, wherein the log shows information such as time, thread id, calling party class name, calling party method name, called party class name, called party method name, parameter value and the like when the calling party calls other program methods, and the value of the parameter value is the same as that of S204, and the log comprises the following steps: the hash value is used for default replacement of the parameter value when writing the log, and the original value is used if the parameter value is of a basic data type such as integer, character type, byte type and the like;
S208: and judging whether the calling party method set has records, if yes, jumping to S206, if not, ending, packaging the instrumentation program into jar packages, deploying the jar packages into a test environment, adding the starting parameters of javaagent jar packages on the starting statement of the program of the tested application, enabling the instrumentation program to run before the tested application, and outputting a printing log format.
In one example of an embodiment of the present invention, a print log format is defined as follows:
The format of the caller is as follows:
ProgramCaller-2021.01.12_14:04:20:0543-ID234-com.icbc.xxxxx.method1()-com.icbc.yyyyy.method2()-arg1_arg2_arg3.
the format of the called party is as follows:
ProgramCallee-2021.01.12_14:04:20:0546-ID234-com.icbc.xxxxx.method2()-arg1_arg2_arg3.
Description:
ProgramCaller: the calling party log keyword is used for retrieving the log segment through ELASTICSEARCH;
ProgramCallee: the key word of the called party log is searched for the log through ELASTICSEARCH;
2021.01.12_14:04:20:0543: program run time stamp, accurate to millisecond;
com.icbc.xxxxx.method1 (): class name and method name of calling program;
com.icbc.yyyyy.method2 (): class name and method name of the called program;
arg1_arg2_arg3: parameter values entered when the program method is invoked.
When the method for evaluating the coverage condition of the test provided by the embodiment of the invention is implemented, in one embodiment, the executing the tested application according to the printing log format, outputting the standardized log, performing mining analysis on the standardized log, and determining the actual executing call relationship, includes:
executing the tested application according to the printing log format, and outputting a standardized log;
Deploying a log collection engine, configuring calling party log keywords and called party log keywords on the log collection engine, acquiring standardized logs output on each server in real time, transmitting the standardized logs to a search server, and establishing indexes on the search server;
The deployment analysis tool is connected with the search server, the standardized log is retrieved through the keywords and the index, and the caller standardized log and the callee standardized log are distinguished according to the keywords after analysis;
registering the caller standardized log into a calling equation sequence execution record table;
Registering the standardized log of the called party to a called equation sequence execution record table;
And deploying a calling relation matching analysis tool, performing mining analysis on the calling party program execution record table and the called equation sequence execution record table, and determining the actual execution calling relation.
When the evaluation method for testing coverage conditions provided by the embodiment of the present invention is implemented, in one embodiment, the foregoing deployment calling relationship matching analysis tool performs mining analysis on the calling party program execution record table and the called equation sequence execution record table, and determines an actual execution calling relationship, including:
acquiring a set A of calling equation sequence execution records in a set time period according to the equation sequence execution record table;
judging whether the set A has a next record, and if the set A has no next record, directly ending the mining analysis;
If the next record exists in the set A, acquiring a record a from the set A, and taking out the name and the method of the called party class in the record a;
inquiring a set B of called equation sequence execution records in a called equation sequence execution record table according to the class name and the method of the called party;
Judging whether the next record exists in the set B, if the next record does not exist in the set B, judging whether the next record exists in the set A again, and acquiring the next record of the set A when the next record exists in the set A;
if the next record exists in the set B, the record B is taken out of the set B;
Judging whether the thread ID of the record a and the thread ID of the record b are the same; if the thread ID of the record a is different from the thread ID of the record B, judging whether the next record exists in the set B again, and acquiring the next record in the set B to continue processing when the next record exists in the set B;
If the thread ID of the record a and the thread ID of the record b are the same, judging whether the parameter values of the record a and the record b are the same or not; if the parameter values of the record a and the record B are different, judging whether the next record exists in the set B again, and acquiring the next record in the set B for continuous processing when the next record exists in the set B;
If the parameter values of the record a and the record b are the same, judging whether the time difference of code execution in the record a and the record b is within a set range or not; if the time difference of code execution in the record a and the record B is not in the set range, judging whether the next record exists in the set B again, and acquiring the next record in the set B for continuous processing when the next record exists in the set B;
If the time difference of code execution in the record a and the record B is in the set range, calling the called party, accumulating the calling times, and obtaining the next note in the set B for continuous processing; after the data in the set A and the set B are processed completely, analyzing the calling times from all calling parties to called parties to obtain the coverage times, and determining the actual executing calling relation.
In the embodiment, on the basis of the unified and standard printing log format obtained by log instrumentation, a large number of logs are output when the tested application is tested and executed, and the output logs are required to be mined and analyzed; the executing the tested application according to the print log format, outputting the standardized log, performing mining analysis on the standardized log, and determining the actual executing call relationship may include:
executing the tested application according to the printing log format, and outputting a standardized log;
Deploying a log collection engine such as logstash, configuring a calling party log keyword and a called party log keyword on the log collection engine, acquiring standardized log files on each server in real time when the log collection engine is started, transmitting the collected standardized logs to a search server such as ELASTICSEARCH by a tool, and establishing an index on the search server;
An analysis tool is deployed, the analysis tool is connected with a search server, the output standardized logs are retrieved through keywords and indexes, after analysis, the caller standardized logs and the callee standardized logs are respectively registered into a calling equation sequence execution record table and a callee equation sequence execution record table according to the keywords; as shown below, table 1 is a calling equation order execution record table, and table 2 is a called equation order execution record table. Other forms of self-polishing tools with the analysis tool function are within the scope of the embodiments of the present invention.
TABLE 1
| Thread ID | Program_1 (calling party) | Program_2 (called party) | Parameter value transmission | timetmp |
| ID234 | com.icbc.A.method1() | com.icbc.B.method2() | arg1_arg2_arg3 | 2021.01.12 14:04:0243 |
| ID234 | com.icbc.A.method1() | com.icbc.C.method2() | arg1_arg2_arg3 | 2021.01.12 14:05:0643 |
| ID235 | com.icbc.B.method2() | com.icbc.D.method3() | arg1_arg2_arg3 | 2021.01.12 14:07:0544 |
TABLE 2
| Thread ID | Program_2 (called party) | Parameter value transmission | timetmp |
| ID234 | com.icbc.B.method2() | arg1_arg2_arg3 | 2021.01.12 14:04:0343 |
| ID234 | com.icbc.C.method2() | arg1_arg2_arg3 | 2021.01.12 14:05:0743 |
| ID235 | com.icbc.D.method3() | arg1_arg2_arg3 | 2021.01.12 14:07:0644 |
Deploying a calling relation matching analysis tool, performing mining analysis on the calling party program execution record table and the called equation sequence execution record table, and determining an actual execution calling relation; other forms of self-grinding tools with the calling relation matching analysis tool function are all within the protection scope of the embodiment of the invention.
FIG. 6 is a data analysis flow chart of a call relation matching analysis tool of an evaluation method for testing coverage conditions according to an embodiment of the present invention, as shown in FIG. 6, a process analysis flow for determining an actually executed call relation by using the call relation matching analysis tool mainly includes:
s301: acquiring a set A of calling party program execution records in a period of time according to an execution record table of an equation sequence;
S302: judging whether the next record exists in the set A, if so, jumping to S303, otherwise, ending the mining analysis;
S303: acquiring a record a from the set A, and taking out the class name and the method of the called party in the record a;
S304: inquiring a set B of called equation sequence execution records in a called equation sequence execution record table according to the class name and the method of the called party;
S305: judging whether the next record exists in the set B, if so, jumping to S306, otherwise, jumping to S302, and obtaining the next record of the set A;
S306: fetching record B from collection B;
S307: judging whether the thread IDs of the record a and the record B are the same, if so, jumping to S308, otherwise jumping to S305, and acquiring the next record in the set B;
S308: judging whether the parameter values of the record a and the record B are the same, if so, jumping to S309, otherwise jumping to S305, and obtaining the next record in the set B;
S309: judging whether the time difference of code execution in the record a and the record B is in a specified range, if so, jumping to S310, otherwise jumping to S305, and acquiring the next record in the set B;
S310: recognizing that the calling party calls the called party and accumulates the calling times, continuing to jump to S305, and acquiring and processing the next note in the set B; after the data in the set A and the set B are processed completely, analyzing the calling times from all calling parties to called parties to obtain the coverage times, and determining the actual executing calling relation.
When the method for evaluating the test coverage condition provided by the embodiment of the invention is implemented, in one embodiment, the determining the test coverage condition of the tested application according to the method call relationship and the actual execution call relationship between the programs includes:
counting the coverage times of the same calling relation;
And the calling relation comparison tool compares the calling relation of the method among the programs with the actual executing calling relation, and determines the test coverage condition of the tested application by combining the coverage times of the same calling relation.
In an embodiment, determining a test coverage condition of an application under test according to a method call relationship and an actual execution call relationship between programs may include:
according to the obtained coverage times of all calling parties to called parties, counting the coverage times with the same calling relation;
the calling relation comparison tool compares the calling relation of the method between the programs with the actually executed calling relation, and determines the test coverage condition of the tested application by combining the coverage times of the same calling relation;
And generating a test report according to the test coverage condition of the tested application, reporting the test report to quality control personnel and project managers, and organizing and judging which calling relations need to be further subjected to supplementary test to perform risk analysis and compensation.
The invention provides a test coverage rate evaluation method and device based on a method call relation, and aims to evaluate test progress and detect missing test conditions by timely acquiring the coverage condition of an actual program call relation.
By using the method and the device, the condition that the program calling relation is actually covered and the test times thereof can be monitored, the test progress is evaluated, the test risk is predicted, the test deficiency is found out, and the supplementary test is made. And obtaining the method call relation between classes in the whole program from the program call relation of the engineering package of the tested application, obtaining the execution condition of the actual execution call relation between the classes through log instrumentation and mining analysis, and evaluating the test coverage condition of the whole project according to the summarized execution condition and the method call relation between the whole program. The invention solves the problem that the actual test progress situation is difficult to obtain, obtains the calling relation link of the program through analyzing the code logic grammar tree, analyzes the actual covered program path through the test output log, and compares the actual test progress with the actual program path to obtain the actual test progress; the device can be used by functional testers to obtain uncovered program paths and review test scenes and codes; project management personnel can utilize the device to carry out quantitative analysis and timely risk early warning to the actual test progress of the project.
Fig. 7 is a schematic diagram of a computer device for running a test coverage assessment method implemented by the present invention, and as shown in fig. 7, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the test coverage assessment method when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for executing the evaluation method for realizing the test coverage condition.
The embodiment of the invention also provides an evaluation device for testing coverage conditions, as described in the following embodiment. Because the principle of the device for solving the problem is similar to that of a test coverage condition evaluation method, the implementation of the device can refer to the implementation of a test coverage condition evaluation method, and the repetition is omitted.
Fig. 8 is a schematic diagram of an evaluation device for testing coverage according to an embodiment of the present invention, and as shown in fig. 8, an embodiment of the present invention further provides an evaluation device for testing coverage, which may include:
The program method call relation analysis module B01 is used for determining a method call relation between programs according to source codes of the tested application;
The program instrumentation module B02 is used for performing log instrumentation according to the method call relation among programs and determining a printing log format;
The log mining analysis module B03 is used for executing the tested application according to the printing log format, outputting a standardized log, carrying out mining analysis on the standardized log and determining the actual executing calling relation;
And the coverage rate evaluation module B04 is used for determining the test coverage condition of the tested application according to the method call relationship and the actual execution call relationship between the programs.
When the evaluation device for testing coverage provided by the embodiment of the present invention is implemented, in one embodiment, the foregoing program method call relationship analysis module is further configured to:
and eliminating call relations with business context relations smaller than a preset threshold value in the method call relations among the programs and call relation paths which cannot be covered by actual business.
When the evaluation device for testing coverage conditions provided by the embodiment of the present invention is implemented, in one embodiment, the log mining analysis module is specifically configured to:
executing the tested application according to the printing log format, and outputting a standardized log;
Deploying a log collection engine, configuring calling party log keywords and called party log keywords on the log collection engine, acquiring standardized logs output on each server in real time, transmitting the standardized logs to a search server, and establishing indexes on the search server;
The deployment analysis tool is connected with the search server, the standardized log is retrieved through the keywords and the index, and the caller standardized log and the callee standardized log are distinguished according to the keywords after analysis;
registering the caller standardized log into a calling equation sequence execution record table;
Registering the standardized log of the called party to a called equation sequence execution record table;
And deploying a calling relation matching analysis tool, performing mining analysis on the calling party program execution record table and the called equation sequence execution record table, and determining the actual execution calling relation.
When the device for evaluating the coverage of the test provided by the embodiment of the invention is implemented, in one embodiment, the coverage evaluation module is specifically configured to:
counting the coverage times of the same calling relation;
And the calling relation comparison tool compares the calling relation of the method among the programs with the actual executing calling relation, and determines the test coverage condition of the tested application by combining the coverage times of the same calling relation.
Fig. 9 is a schematic diagram of another embodiment of an apparatus for evaluating coverage of a test according to an embodiment of the present invention, as shown in fig. 9, and another embodiment of an apparatus for evaluating coverage of a test according to an embodiment of the present invention includes: program method call relation analysis unit 1, program instrumentation unit 2, log mining analysis unit 3, coverage rate evaluation unit 4.
Program calling relation calculating unit 1, according to the source code of the tested application, determining the method calling relation between the programs;
The program instrumentation unit 2 is used for processing based on the method call relations among different programs obtained by the program call relation calculation unit 1, and a dynamic agent tool such as javassit tool is used for writing a java agent instrumentation program to determine a printing log format;
the log mining analysis unit 3 outputs a large number of logs when testing is executed on the basis that the previous program pile inserting unit obtains a unified and standardized printing log format, and carries out mining analysis on the standardized logs to determine the actual execution calling relationship;
And the coverage rate evaluation unit 4 determines the test coverage condition of the tested application according to the method calling relationship and the actual execution calling relationship between the programs.
In summary, the method and the device for evaluating the coverage condition of the test provided by the embodiment of the invention comprise the following steps: firstly, determining a method call relation between programs according to source codes of tested applications; then log instrumentation is carried out according to the method calling relation among programs, and a printing log format is determined; then executing the tested application according to the printing log format, outputting a standardized log, carrying out mining analysis on the standardized log, and determining the actual executing calling relation; and finally, determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs. The embodiment of the invention obtains the source code from the tested application, analyzes and obtains the method call relation between the whole programs, obtains the actual execution call relation through log instrumentation and log mining analysis, and can obtain the test coverage condition of the tested application through mutual comparison; the invention solves the problem that the actual test progress situation is difficult to acquire, by using the invention, the situation that the program calling relation is actually covered can be monitored, the test coverage situation of the tested application can be estimated, the test risk can be predicted, the test deficiency can be found out, the supplementary test can be made, and the quantitative analysis and the timely risk early warning can be carried out on the actual test progress.
By using the embodiment of the invention, the condition that the program calling relation is actually covered and the test times thereof can be monitored, the test progress is evaluated, the test risk is predicted, the test deficiency is found out, and the supplementary test is made. And obtaining the method call relation between classes in the whole program from the program call relation of the engineering package of the tested application, obtaining the execution condition of the actual execution call relation between the classes through log instrumentation and mining analysis, and evaluating the test coverage condition of the whole project according to the summarized execution condition and the method call relation between the whole program. The invention solves the problem that the actual test progress situation is difficult to obtain, obtains the calling relation link of the program through analyzing the code logic grammar tree, analyzes the actual covered program path through the test output log, and compares the actual test progress with the actual program path to obtain the actual test progress; the device can be used by functional testers to obtain uncovered program paths and review test scenes and codes; project management personnel can utilize the device to carry out quantitative analysis and timely risk early warning to the actual test progress of the project.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (10)
1. A method of evaluating test coverage, comprising:
determining a method call relation between programs according to source codes of the tested application;
Performing log instrumentation according to a method calling relation between programs, and determining a printing log format;
executing the tested application according to the printing log format, outputting a standardized log, performing mining analysis on the standardized log, and determining the actual executing calling relation;
determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs;
performing log instrumentation according to a method call relationship between programs, determining a printing log format, including:
Inserting a first log into each called method body, wherein the first log reflects the time, thread id, class name, method name and parameter value information transmitted by a calling party when a program method is called; locating the place of the calling program according to the class name, the method and the parameter type of the called party, and inserting a second log, wherein the second log reflects the time, the thread id, the calling party class name, the calling party method name, the called party class name, the called party method name and the parameter value information when the calling party calls other program methods;
Mining and analyzing the standardized log to determine the actual execution calling relationship, including:
Analyzing the standardized log, and distinguishing a caller standardized log and a callee standardized log according to the keywords after analyzing; registering the caller standardized log into a calling equation sequence execution record table; registering the standardized log of the called party to a called equation sequence execution record table;
Acquiring a first set of calling equation sequence execution records within a set time period; acquiring a first record from a first set, and taking out the class name and the method of the called party in the first record;
inquiring a second set of called equation sequence execution records in a called equation sequence execution record table according to the class name and the method of the called party in the first record; retrieving a second record from the second set;
If the thread id of the first record is the same as that of the second record, the parameter values of the first record are the same as that of the second record, and the time difference of code execution in the first record and the second record is within a set range, the calling party calls the called party and accumulates the calling times to acquire the next note record in the second set to continue processing; after the data in the first set and the second set are processed completely, analyzing the calling times from all calling parties to called parties to obtain the coverage times, and determining the actual executing calling relation.
2. The method of claim 1, further comprising, prior to logging based on the method call relationships between the programs:
and eliminating call relations with business context relations smaller than a preset threshold value in the method call relations among the programs and call relation paths which cannot be covered by actual business.
3. The method of claim 1, wherein executing the application under test according to the print log format, outputting a standardized log, performing mining analysis on the standardized log, and determining the actual execution call relationship, comprises:
executing the tested application according to the printing log format, and outputting a standardized log;
Deploying a log collection engine, configuring calling party log keywords and called party log keywords on the log collection engine, acquiring standardized logs output on each server in real time, transmitting the standardized logs to a search server, and establishing indexes on the search server;
The deployment analysis tool is connected with the search server, the standardized log is retrieved through the keywords and the index, and the caller standardized log and the callee standardized log are distinguished according to the keywords after analysis;
registering the caller standardized log into a calling equation sequence execution record table;
Registering the standardized log of the called party to a called equation sequence execution record table;
And deploying a calling relation matching analysis tool, performing mining analysis on the calling party program execution record table and the called equation sequence execution record table, and determining the actual execution calling relation.
4. The method of claim 1, wherein determining the test coverage of the application under test based on the method call relationship and the actual execution call relationship between the programs comprises:
counting the coverage times of the same calling relation;
And the calling relation comparison tool compares the calling relation of the method among the programs with the actual executing calling relation, and determines the test coverage condition of the tested application by combining the coverage times of the same calling relation.
5. An evaluation device for testing coverage, comprising:
The program method call relation analysis module is used for determining a method call relation between programs according to the source code of the tested application;
The program instrumentation module is used for performing log instrumentation according to the method call relation among the programs and determining a printing log format;
the log mining analysis module is used for executing the tested application according to the printing log format, outputting a standardized log, carrying out mining analysis on the standardized log and determining the actual executing calling relation;
The coverage rate evaluation module is used for determining the test coverage condition of the tested application according to the method calling relation and the actual execution calling relation between the programs;
The program instrumentation module is specifically configured to: inserting a first log into each called method body, wherein the first log reflects the time, thread id, class name, method name and parameter value information transmitted by a calling party when a program method is called; locating the place of the calling program according to the class name, the method and the parameter type of the called party, and inserting a second log, wherein the second log reflects the time, the thread id, the calling party class name, the calling party method name, the called party class name, the called party method name and the parameter value information when the calling party calls other program methods;
the log mining analysis module is specifically used for: analyzing the standardized log, and distinguishing a caller standardized log and a callee standardized log according to the keywords after analyzing; registering the caller standardized log into a calling equation sequence execution record table; registering the standardized log of the called party to a called equation sequence execution record table;
Acquiring a first set of calling equation sequence execution records within a set time period; acquiring a first record from a first set, and taking out the class name and the method of the called party in the first record;
inquiring a second set of called equation sequence execution records in a called equation sequence execution record table according to the class name and the method of the called party in the first record; retrieving a second record from the second set;
If the thread id of the first record is the same as that of the second record, the parameter values of the first record are the same as that of the second record, and the time difference of code execution in the first record and the second record is within a set range, the calling party calls the called party and accumulates the calling times to acquire the next note record in the second set to continue processing; after the data in the first set and the second set are processed completely, analyzing the calling times from all calling parties to called parties to obtain the coverage times, and determining the actual executing calling relation.
6. The apparatus of claim 5, wherein the program method call relationship analysis module is further to:
and eliminating call relations with business context relations smaller than a preset threshold value in the method call relations among the programs and call relation paths which cannot be covered by actual business.
7. The apparatus of claim 5, wherein the log mining analysis module is specifically configured to:
executing the tested application according to the printing log format, and outputting a standardized log;
Deploying a log collection engine, configuring calling party log keywords and called party log keywords on the log collection engine, acquiring standardized logs output on each server in real time, transmitting the standardized logs to a search server, and establishing indexes on the search server;
The deployment analysis tool is connected with the search server, the standardized log is retrieved through the keywords and the index, and the caller standardized log and the callee standardized log are distinguished according to the keywords after analysis;
registering the caller standardized log into a calling equation sequence execution record table;
Registering the standardized log of the called party to a called equation sequence execution record table;
And deploying a calling relation matching analysis tool, performing mining analysis on the calling party program execution record table and the called equation sequence execution record table, and determining the actual execution calling relation.
8. The apparatus of claim 5, wherein the coverage assessment module is configured to:
counting the coverage times of the same calling relation;
And the calling relation comparison tool compares the calling relation of the method among the programs with the actual executing calling relation, and determines the test coverage condition of the tested application by combining the coverage times of the same calling relation.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 4 when executing the computer program.
10. A computer readable storage medium, characterized in that it stores a computer program which, when executed by a processor, implements the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110294920.9A CN113032254B (en) | 2021-03-19 | 2021-03-19 | Test coverage condition evaluation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110294920.9A CN113032254B (en) | 2021-03-19 | 2021-03-19 | Test coverage condition evaluation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113032254A CN113032254A (en) | 2021-06-25 |
| CN113032254B true CN113032254B (en) | 2024-05-31 |
Family
ID=76471962
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110294920.9A Active CN113032254B (en) | 2021-03-19 | 2021-03-19 | Test coverage condition evaluation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113032254B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103365780A (en) * | 2013-07-22 | 2013-10-23 | 百度在线网络技术（北京）有限公司 | Method and device for calculating coverage rate of abnormal test |
| CN107797923A (en) * | 2017-10-10 | 2018-03-13 | 平安科技（深圳）有限公司 | Code coverage rate analysis method and application server |
| CN111831573A (en) * | 2020-07-27 | 2020-10-27 | 中国工商银行股份有限公司 | Method, device, computer system and medium for determining code branch coverage condition |
| CN111857681A (en) * | 2020-06-08 | 2020-10-30 | 北京大学 | Software-defined key function positioning and extracting method of C + + system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060294503A1 (en) * | 2005-06-24 | 2006-12-28 | Microsoft Corporation | Code coverage analysis |
| CN103246602B (en) * | 2012-02-14 | 2017-03-01 | 阿里巴巴集团控股有限公司 | Code coverage determines method and system, code coverage detection method and system |
-
2021
- 2021-03-19 CN CN202110294920.9A patent/CN113032254B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103365780A (en) * | 2013-07-22 | 2013-10-23 | 百度在线网络技术（北京）有限公司 | Method and device for calculating coverage rate of abnormal test |
| CN107797923A (en) * | 2017-10-10 | 2018-03-13 | 平安科技（深圳）有限公司 | Code coverage rate analysis method and application server |
| CN111857681A (en) * | 2020-06-08 | 2020-10-30 | 北京大学 | Software-defined key function positioning and extracting method of C + + system |
| CN111831573A (en) * | 2020-07-27 | 2020-10-27 | 中国工商银行股份有限公司 | Method, device, computer system and medium for determining code branch coverage condition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113032254A (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106656536B (en) | Method and equipment for processing service calling information | |
| CN107665171B (en) | Automatic regression testing method and device | |
| US8312440B2 (en) | Method, computer program product, and hardware product for providing program individuality analysis for source code programs | |
| CN108804326B (en) | Automatic software code detection method | |
| CN110990362A (en) | Log query processing method and device, computer equipment and storage medium | |
| CN109857637B (en) | Java language method coverage rate and method input and output statistical method based on annotation | |
| CN112579437A (en) | Program operation process conformance verification method | |
| CN110543422A (en) | software package code defect data processing method, system and medium for FPR | |
| CN108255700A (en) | Test result generation method and device | |
| CN110147313B (en) | Log output method and device | |
| CN113032254B (en) | Test coverage condition evaluation method and device | |
| CN112631944A (en) | Source code detection method and device based on abstract syntax tree and computer storage medium | |
| CN112486841A (en) | Method and device for checking data collected by buried point | |
| CN111767213A (en) | Method and device for testing database check points, electronic equipment and storage medium | |
| CN116627804A (en) | Test method, system, electronic equipment and storage medium based on artificial intelligence | |
| CN111258562A (en) | Java code quality inspection method, device, equipment and storage medium | |
| CN115576831A (en) | Test case recommendation method, device, equipment and storage medium | |
| US11526775B2 (en) | Automatically evaluating application architecture through architecture-as-code | |
| CN113791980A (en) | Test case conversion analysis method, device, equipment and storage medium | |
| CN114490413A (en) | Test data preparation method and device, storage medium and electronic equipment | |
| CN114492324A (en) | Component data statistical method and device | |
| CN114676061A (en) | Knowledge graph-based automatic firmware detection method | |
| CN114020645A (en) | Test method, device, equipment, readable storage medium and computer program product | |
| KR20080043204A (en) | File format analysis system and method for unknown file format to use software security testing | |
| CN110717032A (en) | Method for generating and displaying multi-system method call link diagram |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |