CN112817865A - Coverage precision test method and system based on componentized distributed system - Google Patents
Coverage precision test method and system based on componentized distributed system Download PDFInfo
- Publication number
- CN112817865A CN112817865A CN202110207015.5A CN202110207015A CN112817865A CN 112817865 A CN112817865 A CN 112817865A CN 202110207015 A CN202110207015 A CN 202110207015A CN 112817865 A CN112817865 A CN 112817865A
- Authority
- CN
- China
- Prior art keywords
- case
- code
- use case
- coverage
- test
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a coverage precision test method based on a modularized distributed system, which comprises the following steps of S1: testing the modular distributed system, performing differential analysis, and analyzing effective code change; step S2, test analysis and case recommendation: analyzing the range of the influence function according to the change code, and recommending an effective case according to the code and the case management relation; step S3, use case execution: the recommended use case is automatically scheduled to be executed if the recommended use case is an automatic use case; if the function is a case, manual execution is needed; step S4, quality evaluation: performing differential coverage rate analysis, and if missing codes which are not covered are analyzed, performing a supplementary case and performing execution; if the analysis of the code coverage rate is completed, a test report is generated, so that the positioning problem is clearer and more accurate; the test case can be revised quickly, and the test coverage rate is maximized.
Description
Technical Field
The invention relates to the technical field of automatic testing, in particular to a coverage precision testing method and a coverage precision testing system based on a componentized distributed system.
Background
In the prior art, the test of code change comprises a code coverage rate test, an admission test and a smoking test; the disadvantages of these tests are as follows: 1. agile iteration is updated fast, and manpower is limited; 2. merging multi-branch codes into a main branch, modifying which line of which file, and testing uncontrollable; 3. code updates affect which functions are not aware; 4. the method completely depends on the capability, experience and business familiarity of testers, and has high online risk; 5. the quality of the code cannot be monitored more accurately; 6. the method cannot achieve high efficiency, accuracy and inequity.
Disclosure of Invention
In order to overcome the problems, the invention aims to provide a coverage accurate test method based on a componentized distributed system, which effectively improves the test coverage rate, obtains accurate quantitative data, uses the quantitative data to perform quality evaluation, greatly reduces the blindness and workload of regression test, quickly revises test cases, and maximizes the test coverage rate.
The invention is realized by adopting the following scheme: a coverage accurate test method based on a componentized distributed system comprises the following steps:
step S1, differential analysis: testing the modular distributed system, performing differential analysis, and analyzing effective code change;
step S2, test analysis and case recommendation: analyzing the range of the influence function according to the change code, and recommending an effective case according to the code and the case management relation;
step S3, use case execution: the recommended use case is automatically scheduled to be executed if the recommended use case is an automatic use case; if the function is a case, manual execution is needed;
step S4, quality evaluation: performing differential coverage rate analysis, and if missing codes which are not covered are analyzed, performing a supplementary case and performing execution; and if the analysis of the code coverage rate is completed, generating a test report, so that the positioning problem is clearer and more accurate.
Further, the step S1 is further specifically: acquiring a change method list and a coverage rate report generated by a newly added automatic case from a code hosting platform, and inquiring invalid changes in a code analyzer, wherein the invalid changes comprise adding spaces or adding comments; a one-time yield change code delta coverage report that records details of changes, the details comprising: adding, modifying and deleting code lines, hierarchical relations, classes and methods, marking uncovered methods and branch codes, providing measurement data for testing coverage and obtaining uncovered codes aiming at design cases.
Further, the step S2 is further specifically: obtaining a change code from differential analysis, then finding a case corresponding to the code and the method of the corresponding change in a case code relation library, and outputting a corresponding case or recommending a test case; the use case code relational database is a basic use case and coverage code mapping information database, and the use case and coverage code mapping relation is as follows: a single use case and a list mapping relation of all code methods which can be covered by the single use case are obtained; firstly, collecting a use case, namely an execution use case, and collecting the use case to a use case detailed table, wherein the use case detailed table comprises a use case id, an item id, a catalogue id, creation time, a creator, a use case name, a premise, detailed steps, priority, an expected result and an actual result; secondly, acquiring the method call relation executed by the use case, namely dynamically acquiring the method call chain in a pile inserting mode, wherein the pile inserting mode tool comprises the following steps: 1. source code instrumentation: a tool GCT; 2. inserting a middle code: code coverage tool JACOCO or EMMA; 3. and (3) binary code instrumentation: the XDebug tool collects a use case correlation function call relation information table, and the relation information table comprises a call method, a called method and a call index; then managing the use case and the method, namely acquiring a use case association method original information table, wherein the use case association method original information table comprises a method id, a method name, a class, a source file path, whether to associate or not and updating time; and finally, storing data to obtain a use case and function information table relation general table, wherein the use case and function information table relation general table comprises a use case id, a use case name, a use case owner, an affiliated item, a use case associated method original information table name, a use case associated method calling relation information table name, modification time and creation time.
Further, the effective recommending case in step S2 is specifically: after mapping a basic information base by using a case and code method list, screening use cases; firstly, a version comparison function is provided through a code hosting platform, a code change file between commit is directly obtained after 2 times of code submission, and the code change file is returned in a json format; and secondly, acquiring a code change file, directly acquiring a change method list based on different language method structures, developing an API for planning dynamic update in an automation framework, enabling the planning and case binding to be dynamically updated, and configuring environment, variable and permission elements simultaneously, so that the screened effective cases can be executed by taking the plan as a unit.
Further, the step S3 is further specifically: according to the recommended use cases, firstly classifying the use cases into automatic use cases and functional use cases, and secondly, carrying out different scheduling modes and execution modes according to the classification, wherein the automatic use cases automatically schedule the use cases and automatically execute the scheduling use cases; and in addition, after the main process scene of a core interface of a module is covered by the automatic use case, a change method list of bottom business logic is searched, the mapping library relation is inquired to obtain an influence use case list, and then the use case request URIs or interface names are subjected to deduplication and are aggregated to be displayed in a report form.
The invention also provides a coverage precision test system based on the componentized distributed system, wherein the test system comprises a differentiation analysis module, a test analysis and recommended case module, a case execution module and a quality evaluation module;
the differential analysis module is used for starting testing of the modular distributed system, performing differential analysis and analyzing effective code change;
the test analysis and case recommendation module is used for analyzing the range of the influence function according to the change code and recommending an effective case according to the code and case management relation;
the case execution module automatically schedules and executes the recommended case if the recommended case is an automatic case; if the function is a case, manual execution is needed;
the quality evaluation module is used for analyzing the difference coverage rate, and if missing codes which are not covered are analyzed, a supplementary case is carried out for execution; and if the analysis of the code coverage rate is completed, generating a test report, so that the positioning problem is clearer and more accurate.
Further, the implementation manner of the differentiation analysis module is further specifically: acquiring a change method list and a coverage rate report generated by a newly added automatic case from a code hosting platform, and inquiring invalid changes in a code analyzer, wherein the invalid changes comprise adding spaces or adding comments; a one-time yield change code delta coverage report that records details of changes, the details comprising: adding, modifying and deleting code lines, hierarchical relations, classes and methods, marking uncovered methods and branch codes, providing measurement data for testing coverage and obtaining uncovered codes aiming at design cases.
Further, the implementation manner of the test analysis and recommended case module is further specifically: obtaining a change code from differential analysis, then finding a case corresponding to the code and the method of the corresponding change in a case code relation library, and outputting a corresponding case or recommending a test case; the use case code relational database is a basic use case and coverage code mapping information database, and the use case and coverage code mapping relation is as follows: a single use case and a list mapping relation of all code methods which can be covered by the single use case are obtained; firstly, collecting a use case, namely an execution use case, and collecting the use case to a use case detailed table, wherein the use case detailed table comprises a use case id, an item id, a catalogue id, creation time, a creator, a use case name, a premise, detailed steps, priority, an expected result and an actual result; secondly, acquiring the method call relation executed by the use case, namely dynamically acquiring the method call chain in a pile inserting mode, wherein the pile inserting mode tool comprises the following steps: 1. source code instrumentation: a tool GCT; 2. inserting a middle code: code coverage tool JACOCO or EMMA; 3. and (3) binary code instrumentation: the XDebug tool collects a use case correlation function call relation information table, and the relation information table comprises a call method, a called method and a call index; then managing the use case and the method, namely acquiring a use case association method original information table, wherein the use case association method original information table comprises a method id, a method name, a class, a source file path, whether to associate or not and updating time; and finally, storing data to obtain a use case and function information table relation general table, wherein the use case and function information table relation general table comprises a use case id, a use case name, a use case owner, an affiliated item, a use case associated method original information table name, a use case associated method calling relation information table name, modification time and creation time.
Further, the effective case recommendation in the test analysis and case recommendation module specifically comprises: after mapping a basic information base by using a case and code method list, screening use cases; firstly, a version comparison function is provided through a code hosting platform, a code change file between commit is directly obtained after 2 times of code submission, and the code change file is returned in a json format; and secondly, acquiring a code change file, directly acquiring a change method list based on different language method structures, developing an API for planning dynamic update in an automation framework, enabling the planning and case binding to be dynamically updated, and configuring environment, variable and permission elements simultaneously, so that the screened effective cases can be executed by taking the plan as a unit.
Further, the implementation manner of the use case execution module is further specifically: according to the recommended use cases, firstly classifying the use cases into automatic use cases and functional use cases, and secondly, carrying out different scheduling modes and execution modes according to the classification, wherein the automatic use cases automatically schedule the use cases and automatically execute the scheduling use cases; and in addition, after the main process scene of a core interface of a module is covered by the automatic use case, a change method list of bottom business logic is searched, the mapping library relation is inquired to obtain an influence use case list, and then the use case request URIs or interface names are subjected to deduplication and are aggregated to be displayed in a report form.
The invention has the beneficial effects that: 1. help regression, save manpower; 2. scenes and data preparation which cannot be built by manual testing are built, or some test cases which cannot be made by manual testing are executed, so that the test coverage rate is effectively improved; 3. the pre-test makes the test and development possible to be parallel, improves the project agility and reduces the test exclusive period; 4. on the basis of fully understanding the requirements and corresponding system architecture and implementation details, determining a test range from a code level by paying attention to development implementation; 5. the final value of the software quality is the improvement of the user experience. The realization process of the automatic coverage precision test method is actually a representation of software test process digitization. The automatic coverage precision test method improves the user experience by improving the software quality, thereby endowing enterprises with more reliable software capability of digital transformation.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
FIG. 2 is a logic flow diagram of a method of an embodiment of the present invention.
FIG. 3 is a schematic diagram of a differential analysis flow according to an embodiment of the present invention.
Fig. 4 is a business logic diagram of a list mapping relationship generation manner of a specific use case and an overlay code method according to an embodiment of the present invention.
Fig. 5 is a flowchart illustrating a process of recommending a valid use case according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating a use case execution according to an embodiment of the present invention.
FIG. 7 is a diagram of quality assessment and test report generation according to an embodiment of the present invention.
Fig. 8 is a schematic block diagram of the system of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
The invention discloses a coverage accurate test method based on a componentized distributed system, which monitors the activity of a test process, comprises the incidence relation between a test case and a development code in a code and case management relation library, analyzes the collected monitoring data after the code is changed, recommends to change the relevant test case, strives for where to test, obtains accurate quantitative data, uses the quantitative data to evaluate the quality, and can promote the continuous improvement of the test process by utilizing the analysis data to form a measurement and analysis closed loop.
Referring to fig. 1, a coverage precision testing method based on a componentized distributed system includes the following steps:
step S1, differential analysis: testing the modular distributed system, performing differential analysis, and analyzing effective code change;
step S2, test analysis and case recommendation: analyzing the range of the influence function according to the change code, and recommending an effective case according to the code and the case management relation;
step S3, use case execution: the recommended use case is automatically scheduled to be executed if the recommended use case is an automatic use case; if the function is a case, manual execution is needed;
step S4, quality evaluation: performing differential coverage rate analysis, and if missing codes which are not covered are analyzed, performing a supplementary case and performing execution; and if the analysis of the code coverage rate is completed, generating a test report, so that the positioning problem is clearer and more accurate.
The invention is further illustrated below with reference to a specific embodiment:
referring to fig. 2, a coverage precision test method based on a componentized distributed system is implemented by the following steps:
1. differentiation analysis: testing the modular distributed system, performing differential analysis, and analyzing effective code change;
2. test analysis and case recommendation: analyzing an influence range (namely the range of influencing functions after the codes are changed) according to the changed codes, and recommending effective use cases according to the code and use case management relation;
3. case execution: the recommended use case is automatically scheduled to be executed if the recommended use case is an automatic use case; if the function is a case, manual execution is needed;
4. and (3) quality evaluation: performing differential coverage rate analysis, and if missing codes which are not covered are analyzed, performing a supplementary case and performing execution; and if the code coverage rate is analyzed to be basically completed, generating a test report, so that the positioning problem is clearer and more accurate.
First, differentiation analysis
The modularization distributed system is based on the aim of reusability, a large software system is split into a plurality of independent components according to the form of a separation focus, the main aim is to reduce coupling, split the system into a plurality of components, separate component boundaries and responsibilities, and facilitate independent upgrading and maintenance.
Distributed is a collection of independent computers that collectively provide services to the outside, but to users of the system, as if one computer were providing services. Distributed means that more common computers can be used to form a distributed cluster to provide services to the outside. The more computers, the more CPUs, memories, storage resources, etc., the greater the amount of concurrent access that can be handled.
In summary, a large software system is split into a plurality of independent components in a form of separated points of interest, then different components are deployed on different machines, and the components communicate with each other through remote service calls and the like. Providing services to the outside by a distributed system to form a componentized distributed system;
then, a change method list and a coverage report generated by a newly added automatic case are obtained from a code hosting platform (such as code cloud gite, quick, free and stable online code hosting platform based on Git), combined calculation is carried out in an analyzer (code analyzer, such as Understand, multi-language support, multi-platform, code syntax highlight, code check, dependency analysis, instant search, basic reading functions capable of generating various figures for the code and the like), invalid change is inquired through an invalid change algorithm (such as adding a plurality of spaces or adding comments and the like), an incremental coverage report of the change code is produced at one time (recording the details of the change, such as adding, modifying, deleting code lines, hierarchical relations, classes, methods and the like), meanwhile, uncovered methods and branch codes are marked, data are provided for testing coverage, and codes which can be moved to be uncovered aiming at design cases, for a specific implementation of the service logic, see fig. 3.
Second, test analysis and recommended case
And obtaining a change code from differentiation analysis, then finding a case corresponding to the code and the method of the corresponding change in a case code relation library, and outputting the corresponding case or recommending a test case.
1. And the use case code relational library is a basic use case and coverage code mapping information library. As the name implies, the mapping relationship between the use case and the code is as follows: a single use case maps with a list of code methods that it can cover (not classes, branches, or lines). Firstly, collecting a use case, namely an execution use case, and collecting the use case to a use case detailed table (comprising use case id, project id, catalog id, creation time, creator, use case name, premise, detailed steps, priority, expected result, actual result and the like); secondly, acquiring a method call relation executed by the use case, namely dynamically acquiring a method call chain in a instrumentation mode (instrumentation mode tool: 1, source code instrumentation: GCT, and the like; 2, intermediate code instrumentation: JACOCO \ EMMA, and the like; 3, binary code instrumentation: XDebug, and the like), and acquiring a use case associated function call relation information table (including a call method, a called method, a call index, and the like); then managing the use case and the method, namely acquiring a use case association method original information table (method id, method name, class, source file path, association or not, update time); and finally, storing data, namely a general relation table of the use case and the function information table (comprising a use case id, a use case name, a use case owner, an affiliated item, a use case associated method original information table name, a use case associated method calling relation information table name, modification time and creation time). For a specific example and an overlay code method list mapping relationship generation scheme, please refer to fig. 4.
2. Automatic case screening (recommended case)
After mapping the basic information base by using the use case and code method list, use case screening is needed. Firstly, a version comparison function is provided based on a company general code hosting platform, a code change file between commit can be directly obtained after 2 times of code submission, and the code change file is returned in a json format, so that the processing is more convenient. And secondly, acquiring a code change file, directly acquiring a change method list based on different language method structures, developing a dynamically updated API of the plan in an automation framework, enabling the plan and case binding to be dynamically updated, and configuring elements such as environment, variables, authority and the like, so that the screened automation cases can be executed by taking the plan as a unit. For a specific implementation of the service logic, see fig. 5.
Third, case execution
According to the recommended use cases, firstly classifying the use cases into automatic use cases, functional use cases and the like, and then carrying out different scheduling modes and execution modes according to the classification, wherein the automatic use cases automatically schedule the use cases and automatically execute the scheduling use cases; and pushing the function use case to a designated person, and allowing the designated person to manually execute the function use case. Certainly, after the main flow scene of the core interface of a certain module is covered by the automatic use case, we can consider that the change method list of the bottom service logic also queries the mapping library relation to obtain the influence on the use case list, and then the use case request URIs or interface names are deduplicated, aggregated and displayed in the form of a report; please refer to fig. 6.
Fourthly, quality evaluation and test report generation
Referring to fig. 7, performing differential coverage analysis, and if missing uncovered codes are analyzed, performing a supplementary case for execution; if the analysis shows that the code coverage rate is basically completed, a test report is generated, QA can do manual regression test with more pertinence according to the report, the phenomenon of missing evaluation is prevented, and meanwhile, the regression range can be reduced, so that the regression is more pertinence.
Referring to fig. 8, the present invention further provides a coverage precision test system based on a componentized distributed system, wherein the test system comprises a differentiation analysis module, a test analysis and recommended case module, a case execution module, and a quality evaluation module;
the differential analysis module is used for starting testing of the modular distributed system, performing differential analysis and analyzing effective code change;
the test analysis and case recommendation module is used for analyzing the range of the influence function according to the change code and recommending an effective case according to the code and case management relation;
the case execution module automatically schedules and executes the recommended case if the recommended case is an automatic case; if the function is a case, manual execution is needed;
the quality evaluation module is used for analyzing the difference coverage rate, and if missing codes which are not covered are analyzed, a supplementary case is carried out for execution; and if the analysis of the code coverage rate is completed, generating a test report, so that the positioning problem is clearer and more accurate.
The realization mode of the differentiation analysis module is further embodied as follows: the modularization distributed system is based on the aim of reusability, a large software system is split into a plurality of independent components according to the form of a separation focus, the main aim is to reduce coupling, split the system into a plurality of components, separate component boundaries and responsibilities, and facilitate independent upgrading and maintenance.
Distributed is a collection of independent computers that collectively provide services to the outside, but to users of the system, as if one computer were providing services. Distributed means that more common computers can be used to form a distributed cluster to provide services to the outside. The more computers, the more CPUs, memories, storage resources, etc., the greater the amount of concurrent access that can be handled.
In summary, a large software system is split into a plurality of independent components in a form of separated points of interest, then different components are deployed on different machines, and the components communicate with each other through remote service calls and the like. Providing services to the outside by a distributed system to form a componentized distributed system;
then, by acquiring a change method list and a coverage rate report generated by the newly added automation use case from a code hosting platform (such as code cloud gite, quick, free and stable online code hosting platform based on Git), the method comprises the steps of performing combined calculation in an analyzer (code analyzer, such as basic reading functions of supporting multiple languages, multiple platforms, code syntax highlighting, code checking, dependency analysis, instant search, and the like, and capable of generating multiple graphs on codes), inquiring invalid change through an invalid change algorithm (such as adding a blank space or adding a comment and the like), and generating a change code increment coverage rate report (recording the details of change, such as adding, modifying, deleting code lines, hierarchical relations, classes, methods and the like) at one time, and meanwhile, marking the uncovered method and the branch code, providing measurement data for testing the coverage and walking to the uncovered code aiming at the design case.
The implementation manner of the test analysis and recommended case module is further specifically as follows: obtaining a change code from differential analysis, then finding a case corresponding to the code and the method of the corresponding change in a case code relation library, and outputting a corresponding case or recommending a test case; the use case code relational database is a basic use case and coverage code mapping information database, and the use case and coverage code mapping relation is as follows: a single use case and a list mapping relation of all code methods which can be covered by the single use case are obtained; firstly, collecting a use case, namely an execution use case, and collecting the use case to a use case detailed table, wherein the use case detailed table comprises a use case id, an item id, a catalogue id, creation time, a creator, a use case name, a premise, detailed steps, priority, an expected result and an actual result; secondly, acquiring the method call relation executed by the use case, namely dynamically acquiring the method call chain in a pile inserting mode, wherein the pile inserting mode tool comprises the following steps: 1. source code instrumentation: a tool GCT; 2. inserting a middle code: code coverage tool JACOCO or EMMA; 3. and (3) binary code instrumentation: the XDebug tool collects a use case correlation function call relation information table, and the relation information table comprises a call method, a called method and a call index; then managing the use case and the method, namely acquiring a use case association method original information table, wherein the use case association method original information table comprises a method id, a method name, a class, a source file path, whether to associate or not and updating time; and finally, storing data to obtain a use case and function information table relation general table, wherein the use case and function information table relation general table comprises a use case id, a use case name, a use case owner, an affiliated item, a use case associated method original information table name, a use case associated method calling relation information table name, modification time and creation time.
The effective case recommendation in the test analysis and case recommendation module is specifically as follows: after mapping a basic information base by using a case and code method list, screening use cases; firstly, providing a version comparison function based on a company universal code hosting platform, directly obtaining a code change file between 2 times of code submission commit, and returning the code change file in a json format; and secondly, acquiring a code change file, directly acquiring a change method list based on different language method structures, developing an API for planning dynamic update in an automation framework, enabling the planning and case binding to be dynamically updated, and configuring environment, variable and permission elements simultaneously, so that the screened effective cases can be executed by taking the plan as a unit.
The implementation manner of the use case execution module is further specifically as follows: according to the recommended use cases, firstly classifying the use cases into automatic use cases and functional use cases, and secondly, carrying out different scheduling modes and execution modes according to the classification, wherein the automatic use cases automatically schedule the use cases and automatically execute the scheduling use cases; and in addition, after the main process scene of a core interface of a module is covered by the automatic use case, a change method list of bottom business logic is searched, the mapping library relation is inquired to obtain an influence use case list, and then the use case request URIs or interface names are subjected to deduplication and are aggregated to be displayed in a report form.
The application scenarios of the invention are as follows: with the development of the testing industry, software testing is more and more standardized, but most tests are still based on understanding of services, and have a gap with real service data, accuracy is difficult to guarantee, testing results cannot accurately define and judge software quality, and after a system is on line, problems begin to be exposed, use experience is poor, and even more, huge economic loss is caused.
With traditional manual testing, a black box testing method based on manual evaluation is adopted, and a large amount of labor cost is required to be invested in creating a software product with high reliability. Because the test execution cannot be accurately and quantitatively controlled, the subjective qualitative evaluation result is taken as the main point, the dependence on the human experience is large, the personnel change condition is large, the quality jitter is severe, and the clear test gap and the quantitative target cannot be seen. Although the test is performed continuously, the defect finding rate is not high, and the ineffective test consumes a great deal of test cost.
The core idea of the automatic coverage precision test method is to use very precise and intelligent software to solve the problem of software test, and fundamentally lead the transformation from an empirical method to a technical method. The quality assessment is no longer empirical but is determined by accurate data.
The automatic coverage precision test method does not change the traditional software test method, and only has the difference that the software collects the code logic and test data executed in the test process, and automatically establishes the logic relationship between the test case and the program code. The forward and reverse tracing can be formed by adding the software acquisition process in the test process.
Through forward tracing, developers can see the code details of the test case executed by the testers, so that the defects can be repaired conveniently, test data can directly provide basis for development and debugging, and the defects can be quickly positioned and repaired.
Through reverse tracing, a tester quickly determines the range of the test case through the modified source code, so that the blindness and the workload of regression testing are greatly reduced, the test case is quickly revised, and the test coverage is maximized.
1. Software testing dashboards
Some data indexes of program operation are automatically analyzed in the function test process, and real-time output is carried out in the form of waveforms. The software test dashboard is a real-time monitor, and calculates and displays test process data in real time.
2. Bidirectional tracing of use cases and codes
After a test case is executed, the accurate test automatically records and displays the code executed by the test case through a program. If the tester is interested in some line of code, it can trace back which use cases run through the line of code during execution.
3. Intelligent screening regression case set
The case needing regression is directly and accurately positioned according to the variation range of the code, so that the time required by regression testing is shorter, and the regression range is more accurate.
4. Test coverage accuracy analysis
Accurate test coverage is of a variety of forms, with the highest support for 100% coverage requirements of the standard MC/DC (revised conditions/decision coverage).
5. Fast positioning of software defects
And quickly finding out the code line corresponding to the execution use case according to the corresponding relation between the defects and the use cases.
The software quality requirement can be met regardless of challenges brought by project acceptance fair-letter means, test efficiency management and control, cooperation of test and developers and a complex distributed architecture. And the final value of the software quality is the improvement of the user experience. The realization process of the automatic coverage precision test method is actually a representation of software test process digitization. The automatic coverage precision test method improves the user experience by improving the software quality, thereby endowing enterprises with more reliable software capability of digital transformation.
In short, the invention collects the code logic and test data process executed by the test process by software, and automatically establishes the logic relation between the test case and the program code. The forward and reverse tracing can be formed by adding the software acquisition process in the test process. Through forward tracing, developers can see the code details of the test case executed by the testers, so that the defects can be repaired conveniently, test data can directly provide basis for development and debugging, and the defects can be quickly positioned and repaired. Through reverse tracing, a tester quickly determines the range of the test case through the modified source code, so that the blindness and the workload of regression testing are greatly reduced, the test case is quickly revised, and the test coverage is maximized.
The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.
Claims (10)
1. A coverage accurate test method based on a componentized distributed system is characterized by comprising the following steps: the method comprises the following steps:
step S1, differential analysis: testing the modular distributed system, performing differential analysis, and analyzing effective code change;
step S2, test analysis and case recommendation: analyzing the range of the influence function according to the change code, and recommending an effective case according to the code and the case management relation;
step S3, use case execution: the recommended use case is automatically scheduled to be executed if the recommended use case is an automatic use case; if the function is a case, manual execution is needed;
step S4, quality evaluation: performing differential coverage rate analysis, and if missing codes which are not covered are analyzed, performing a supplementary case and performing execution; and if the analysis of the code coverage rate is completed, generating a test report, so that the positioning problem is clearer and more accurate.
2. The coverage precision test method based on the componentized distributed system according to claim 1, characterized in that: the step S1 further includes: acquiring a change method list and a coverage rate report generated by a newly added automatic case from a code hosting platform, and inquiring invalid changes in a code analyzer, wherein the invalid changes comprise adding spaces or adding comments; a one-time yield change code delta coverage report that records details of changes, the details comprising: adding, modifying and deleting code lines, hierarchical relations, classes and methods, marking uncovered methods and branch codes, providing measurement data for testing coverage and obtaining uncovered codes aiming at design cases.
3. The coverage precision test method based on the componentized distributed system according to claim 1, characterized in that: the step S2 further includes: obtaining a change code from differential analysis, then finding a case corresponding to the code and the method of the corresponding change in a case code relation library, and outputting a corresponding case or recommending a test case; the use case code relational database is a basic use case and coverage code mapping information database, and the use case and coverage code mapping relation is as follows: a single use case and a list mapping relation of all code methods which can be covered by the single use case are obtained; firstly, collecting a use case, namely an execution use case, and collecting the use case to a use case detailed table, wherein the use case detailed table comprises a use case id, an item id, a catalogue id, creation time, a creator, a use case name, a premise, detailed steps, priority, an expected result and an actual result; secondly, acquiring the method call relation executed by the use case, namely dynamically acquiring the method call chain in a pile inserting mode, wherein the pile inserting mode tool comprises the following steps: 1. source code instrumentation: a tool GCT; 2. inserting a middle code: code coverage tool JACOCO or EMMA; 3. and (3) binary code instrumentation: the XDebug tool collects a use case correlation function call relation information table, and the relation information table comprises a call method, a called method and a call index; then managing the use case and the method, namely acquiring a use case association method original information table, wherein the use case association method original information table comprises a method id, a method name, a class, a source file path, whether to associate or not and updating time; and finally, storing data to obtain a use case and function information table relation general table, wherein the use case and function information table relation general table comprises a use case id, a use case name, a use case owner, an affiliated item, a use case associated method original information table name, a use case associated method calling relation information table name, modification time and creation time.
4. The coverage precision test method based on the componentized distributed system according to claim 3, characterized in that: the effective recommended case in step S2 is specifically: after mapping a basic information base by using a case and code method list, screening use cases; firstly, a version comparison function is provided through a code hosting platform, a code change file between commit is directly obtained after 2 times of code submission, and the code change file is returned in a json format; and secondly, acquiring a code change file, directly acquiring a change method list based on different language method structures, developing an API for planning dynamic update in an automation framework, enabling the planning and case binding to be dynamically updated, and configuring environment, variable and permission elements simultaneously, so that the screened effective cases can be executed by taking the plan as a unit.
5. The coverage precision test method based on the componentized distributed system according to claim 1, characterized in that: the step S3 further includes: according to the recommended use cases, firstly classifying the use cases into automatic use cases and functional use cases, and secondly, carrying out different scheduling modes and execution modes according to the classification, wherein the automatic use cases automatically schedule the use cases and automatically execute the scheduling use cases; and in addition, after the main process scene of a core interface of a module is covered by the automatic use case, a change method list of bottom business logic is searched, the mapping library relation is inquired to obtain an influence use case list, and then the use case request URIs or interface names are subjected to deduplication and are aggregated to be displayed in a report form.
6. The utility model provides an accurate test system of cover based on componentization distributed system which characterized in that: the test system comprises a differentiation analysis module, a test analysis and recommended case module, a case execution module and a quality evaluation module;
the differential analysis module is used for starting testing of the modular distributed system, performing differential analysis and analyzing effective code change;
the test analysis and case recommendation module is used for analyzing the range of the influence function according to the change code and recommending an effective case according to the code and case management relation;
the case execution module automatically schedules and executes the recommended case if the recommended case is an automatic case; if the function is a case, manual execution is needed;
the quality evaluation module is used for analyzing the difference coverage rate, and if missing codes which are not covered are analyzed, a supplementary case is carried out for execution; and if the analysis of the code coverage rate is completed, generating a test report, so that the positioning problem is clearer and more accurate.
7. The coverage precision test system based on the componentized distributed system according to claim 6, wherein: the realization mode of the differentiation analysis module is further embodied as follows: acquiring a change method list and a coverage rate report generated by a newly added automatic case from a code hosting platform, and inquiring invalid changes in a code analyzer, wherein the invalid changes comprise adding spaces or adding comments; a one-time yield change code delta coverage report that records details of changes, the details comprising: adding, modifying and deleting code lines, hierarchical relations, classes and methods, marking uncovered methods and branch codes, providing measurement data for testing coverage and obtaining uncovered codes aiming at design cases.
8. The coverage precision test system based on the componentized distributed system according to claim 6, wherein: the implementation manner of the test analysis and recommended case module is further specifically as follows: obtaining a change code from differential analysis, then finding a case corresponding to the code and the method of the corresponding change in a case code relation library, and outputting a corresponding case or recommending a test case; the use case code relational database is a basic use case and coverage code mapping information database, and the use case and coverage code mapping relation is as follows: a single use case and a list mapping relation of all code methods which can be covered by the single use case are obtained; firstly, collecting a use case, namely an execution use case, and collecting the use case to a use case detailed table, wherein the use case detailed table comprises a use case id, an item id, a catalogue id, creation time, a creator, a use case name, a premise, detailed steps, priority, an expected result and an actual result; secondly, acquiring the method call relation executed by the use case, namely dynamically acquiring the method call chain in a pile inserting mode, wherein the pile inserting mode tool comprises the following steps: 1. source code instrumentation: a tool GCT; 2. inserting a middle code: code coverage tool JACOCO or EMMA; 3. and (3) binary code instrumentation: the XDebug tool collects a use case correlation function call relation information table, and the relation information table comprises a call method, a called method and a call index; then managing the use case and the method, namely acquiring a use case association method original information table, wherein the use case association method original information table comprises a method id, a method name, a class, a source file path, whether to associate or not and updating time; and finally, storing data to obtain a use case and function information table relation general table, wherein the use case and function information table relation general table comprises a use case id, a use case name, a use case owner, an affiliated item, a use case associated method original information table name, a use case associated method calling relation information table name, modification time and creation time.
9. The coverage precision test system based on the componentized distributed system according to claim 8, wherein: the effective recommended case in the test analysis and recommended case module is specifically as follows: after mapping a basic information base by using a case and code method list, screening use cases; firstly, a version comparison function is provided through a code hosting platform, a code change file between commit is directly obtained after 2 times of code submission, and the code change file is returned in a json format; and secondly, acquiring a code change file, directly acquiring a change method list based on different language method structures, developing an API for planning dynamic update in an automation framework, enabling the planning and case binding to be dynamically updated, and configuring environment, variable and permission elements simultaneously, so that the screened effective cases can be executed by taking the plan as a unit.
10. The coverage precision test system based on the componentized distributed system according to claim 1, wherein: the implementation manner of the use case execution module is further specifically as follows: according to the recommended use cases, firstly classifying the use cases into automatic use cases and functional use cases, and secondly, carrying out different scheduling modes and execution modes according to the classification, wherein the automatic use cases automatically schedule the use cases and automatically execute the scheduling use cases; and in addition, after the main process scene of a core interface of a module is covered by the automatic use case, a change method list of bottom business logic is searched, the mapping library relation is inquired to obtain an influence use case list, and then the use case request URIs or interface names are subjected to deduplication and are aggregated to be displayed in a report form.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110207015.5A CN112817865A (en) | 2021-02-24 | 2021-02-24 | Coverage precision test method and system based on componentized distributed system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110207015.5A CN112817865A (en) | 2021-02-24 | 2021-02-24 | Coverage precision test method and system based on componentized distributed system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112817865A true CN112817865A (en) | 2021-05-18 |
Family
ID=75865371
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110207015.5A Pending CN112817865A (en) | 2021-02-24 | 2021-02-24 | Coverage precision test method and system based on componentized distributed system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112817865A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113626325A (en) * | 2021-08-05 | 2021-11-09 | 杭州遥望网络科技有限公司 | Interface testing method, device and equipment in code iterative updating |
| CN113704112A (en) * | 2021-08-30 | 2021-11-26 | 浙江网商银行股份有限公司 | Data processing method and system |
| CN113704094A (en) * | 2021-08-06 | 2021-11-26 | 北京城市网邻信息技术有限公司 | Test case knowledge base construction method and device, electronic equipment and storage medium |
| CN114490424A (en) * | 2022-02-24 | 2022-05-13 | 北京启明星辰信息安全技术有限公司 | Automatic testing method and device |
| CN115391233A (en) * | 2022-10-31 | 2022-11-25 | 深圳联友科技有限公司 | Accurate test method based on coverage rate analysis and link tracking |
| CN115756553A (en) * | 2023-01-09 | 2023-03-07 | 傲网信息科技（厦门）有限公司 | Software server updating method |
| CN115858354A (en) * | 2022-11-29 | 2023-03-28 | 广发银行股份有限公司 | Test coverage rate calculation method, system, terminal and storage medium |
| CN115905031A (en) * | 2023-03-14 | 2023-04-04 | 云筑信息科技(成都)有限公司 | Test case recommendation method based on accurate quality assurance system |
| CN116578498A (en) * | 2023-07-12 | 2023-08-11 | 西南交通大学 | Automatic generation method and system for unit test cases |
| CN116701164A (en) * | 2022-11-08 | 2023-09-05 | 荣耀终端有限公司 | Test method and electronic equipment |
| CN117331849A (en) * | 2023-12-01 | 2024-01-02 | 杭银消费金融股份有限公司 | Testing method and system based on accurate analysis of codes |
| CN118394455A (en) * | 2024-07-01 | 2024-07-26 | 北京科杰科技有限公司 | Big data assembly cluster management system based on java language |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170262361A1 (en) * | 2016-03-11 | 2017-09-14 | Intuit Inc. | Dynamic testing based on automated impact analysis |
| CN107315689A (en) * | 2017-07-04 | 2017-11-03 | 上海爱数信息技术股份有限公司 | The Automation regression testing method of granularity is retrieved based on Git code files |
| CN110554962A (en) * | 2019-08-14 | 2019-12-10 | 平安普惠企业管理有限公司 | Regression testing process covering method, server and computer readable storage medium |
| CN111831564A (en) * | 2020-07-09 | 2020-10-27 | 北京齐尔布莱特科技有限公司 | Regression testing method and device and computing equipment |
-
2021
- 2021-02-24 CN CN202110207015.5A patent/CN112817865A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170262361A1 (en) * | 2016-03-11 | 2017-09-14 | Intuit Inc. | Dynamic testing based on automated impact analysis |
| CN107315689A (en) * | 2017-07-04 | 2017-11-03 | 上海爱数信息技术股份有限公司 | The Automation regression testing method of granularity is retrieved based on Git code files |
| CN110554962A (en) * | 2019-08-14 | 2019-12-10 | 平安普惠企业管理有限公司 | Regression testing process covering method, server and computer readable storage medium |
| CN111831564A (en) * | 2020-07-09 | 2020-10-27 | 北京齐尔布莱特科技有限公司 | Regression testing method and device and computing equipment |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113626325A (en) * | 2021-08-05 | 2021-11-09 | 杭州遥望网络科技有限公司 | Interface testing method, device and equipment in code iterative updating |
| CN113626325B (en) * | 2021-08-05 | 2024-06-21 | 杭州遥望网络科技有限公司 | Interface testing method, device and equipment in code iteration updating |
| CN113704094A (en) * | 2021-08-06 | 2021-11-26 | 北京城市网邻信息技术有限公司 | Test case knowledge base construction method and device, electronic equipment and storage medium |
| CN113704112A (en) * | 2021-08-30 | 2021-11-26 | 浙江网商银行股份有限公司 | Data processing method and system |
| CN114490424A (en) * | 2022-02-24 | 2022-05-13 | 北京启明星辰信息安全技术有限公司 | Automatic testing method and device |
| CN115391233A (en) * | 2022-10-31 | 2022-11-25 | 深圳联友科技有限公司 | Accurate test method based on coverage rate analysis and link tracking |
| CN116701164A (en) * | 2022-11-08 | 2023-09-05 | 荣耀终端有限公司 | Test method and electronic equipment |
| CN116701164B (en) * | 2022-11-08 | 2024-05-03 | 荣耀终端有限公司 | Test method and electronic equipment |
| CN115858354B (en) * | 2022-11-29 | 2023-08-18 | 广发银行股份有限公司 | Test coverage rate calculation method, system, terminal and storage medium |
| CN115858354A (en) * | 2022-11-29 | 2023-03-28 | 广发银行股份有限公司 | Test coverage rate calculation method, system, terminal and storage medium |
| CN115756553B (en) * | 2023-01-09 | 2023-04-28 | 傲网信息科技（厦门）有限公司 | Software server updating method |
| CN115756553A (en) * | 2023-01-09 | 2023-03-07 | 傲网信息科技（厦门）有限公司 | Software server updating method |
| CN115905031A (en) * | 2023-03-14 | 2023-04-04 | 云筑信息科技(成都)有限公司 | Test case recommendation method based on accurate quality assurance system |
| CN115905031B (en) * | 2023-03-14 | 2023-06-23 | 云筑信息科技(成都)有限公司 | Test case recommendation method based on accurate quality assurance system |
| CN116578498A (en) * | 2023-07-12 | 2023-08-11 | 西南交通大学 | Automatic generation method and system for unit test cases |
| CN116578498B (en) * | 2023-07-12 | 2023-09-29 | 西南交通大学 | Automatic generation method and system for unit test cases |
| CN117331849A (en) * | 2023-12-01 | 2024-01-02 | 杭银消费金融股份有限公司 | Testing method and system based on accurate analysis of codes |
| CN117331849B (en) * | 2023-12-01 | 2024-02-27 | 杭银消费金融股份有限公司 | Testing method and system based on accurate analysis of codes |
| CN118394455A (en) * | 2024-07-01 | 2024-07-26 | 北京科杰科技有限公司 | Big data assembly cluster management system based on java language |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112817865A (en) | Coverage precision test method and system based on componentized distributed system | |
| CN107622014B (en) | Test report generation method and device, readable storage medium and computer equipment | |
| Rausch et al. | An empirical analysis of build failures in the continuous integration workflows of java-based open-source software | |
| Da Costa et al. | A framework for evaluating the results of the szz approach for identifying bug-introducing changes | |
| CN110232024B (en) | Software automation test framework and test method | |
| Sawant et al. | fine-GRAPE: fine-grained APi usage extractor–an approach and dataset to investigate API usage | |
| CN106227654B (en) | A kind of test platform | |
| Wen et al. | Blimp tracer: Integrating build impact analysis with code review | |
| US20160154830A1 (en) | Systems and methods for data integration | |
| Bratman et al. | The software factory | |
| CN110728422A (en) | Building information model, method, device and settlement system for construction project | |
| CN114546868A (en) | Code coverage rate testing method and device and electronic equipment | |
| Söylemez et al. | Challenges of software process and product quality improvement: catalyzing defect root-cause investigation by process enactment data analysis | |
| Petrulio et al. | SZZ in the time of pull requests | |
| CN117743154A (en) | Quantitative analysis method and system for software code iteration change influence domain | |
| Tanaka et al. | Selenium based testing systems for analytical data generation of website user behavior | |
| Valenzuela-Toledo et al. | EGAD: A moldable tool for GitHub Action analysis | |
| CN115525575A (en) | Data automation testing method and system based on Dataworks platform | |
| Gavriliev et al. | Model and Procedure for Assessing the Qualification of a Software Developer | |
| JP2006059276A (en) | Source code evaluating system | |
| CN114331165A (en) | Automatic generation and analysis method and system for urban safety performance evaluation report | |
| Stjepandić et al. | Data quality management for interoperability | |
| Ghazarian | A case study of defect introduction mechanisms | |
| CN112597057B (en) | Method and device for differentially processing blueprint data | |
| CN114064157B (en) | Automatic flow implementation method, system, equipment and medium based on page element identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |