CN116541308B - Regression testing method and system - Google Patents
Regression testing method and system Download PDFInfo
- Publication number
- CN116541308B CN116541308B CN202310801534.3A CN202310801534A CN116541308B CN 116541308 B CN116541308 B CN 116541308B CN 202310801534 A CN202310801534 A CN 202310801534A CN 116541308 B CN116541308 B CN 116541308B
- Authority
- CN
- China
- Prior art keywords
- information
- test
- case
- regression testing
- tested
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 486
- 230000001419 dependent effect Effects 0.000 claims description 42
- 230000006870 function Effects 0.000 claims description 39
- 238000000034 method Methods 0.000 claims description 20
- 238000004088 simulation Methods 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims 1
- 230000004048 modification Effects 0.000 description 8
- 238000012986 modification Methods 0.000 description 8
- 238000011161 development Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 238000007373 indentation Methods 0.000 description 3
- 238000013515 script Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000003252 repetitive effect Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013142 basic testing Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013102 re-test Methods 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 238000013522 software testing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application provides a regression testing method and a regression testing system, comprising the following steps: acquiring basic use case information corresponding to an item to be tested from a database, wherein the basic use case information comprises basic characteristic information, basic execution information and basic configuration information; obtaining a template file, wherein the template file is customized by a user based on the category of an item to be detected, the template file comprises upper-level use case information, and the upper-level use case information comprises upper-level characteristic information, upper-level execution information, upper-level configuration information and judgment information; generating a test case for carrying out regression testing on the item to be tested based on the basic case information and the upper-level case information; running test cases and establishing a working catalog for the test cases, wherein the working catalog comprises the test cases and working logs of the test cases in running; and determining regression test results of the items to be tested based on the working catalog. The regression testing method and system provided by the application have better universality, flexibility and high efficiency.
Description
Technical Field
The application relates to the technical field of software testing, in particular to a regression testing method and system.
Background
Regression testing refers to the re-testing after an old code has been modified to confirm that the modification did not introduce new errors or caused other code to produce errors. Regression testing is often a cumbersome, repetitive tool, requires a lot of manpower and time, and is prone to omission or error, and regression testing tools have been developed in order to increase the efficiency of regression testing and reduce the cost of manpower. However, conventional regression testing tools often can only perform regression testing on codes of a certain category according to specific work and business requirements, cannot meet the requirements of various testing scenes, and lacks versatility and flexibility.
Therefore, how to improve the versatility and flexibility of regression testing tools is a current problem that needs to be addressed.
Disclosure of Invention
One embodiment of the present application provides a regression testing method, including: acquiring basic use case information corresponding to an item to be detected from a database, wherein the basic use case information comprises basic characteristic information, basic execution information and basic configuration information; obtaining a template file, wherein the template file is customized by a user based on the category of the item to be detected, the template file comprises upper-level use case information, and the upper-level use case information comprises upper-level characteristic information, upper-level execution information, upper-level configuration information and judgment information; generating a test case for carrying out regression testing on the item to be tested based on the basic case information and the upper case information; running the test cases and establishing a working catalog for the test cases, wherein the working catalog comprises the test cases and a working log of the test cases when the test cases run; and determining regression test results of the items to be tested based on the working catalogue.
In some embodiments, the test case includes header information, middle information, and trailer information combined in order of operation logic, the header information including the upper level characteristic information, the middle information including the base case information, the trailer information including the upper level execution information, the upper level configuration information, and the judgment information; the judging information comprises keyword information for judging success and/or failure of the test case operation.
In some embodiments, the determining regression testing results for the item under test based on the working catalog comprises: scanning the keyword information appearing in the work log; and judging whether the test case is successfully operated or not based on the keyword information.
In some embodiments, the method further comprises: obtaining the dependent case information of the basic case information from a database; generating a dependent case of the test case based on the dependent case information and the upper-level case information; running the dependent use case; and determining whether to run the test case or not based on the running result of the dependent case.
In some embodiments, the running the test case includes: generating a test program based on the test case and the item to be tested; and calling a simulation tool to run the test program.
In some embodiments, the generating a test program based on the test case and the item under test includes: acquiring a plug-in file customized by a user; extracting a first entry function from the plug-in file; compiling the test case and the item to be tested into a test program through the first entry function.
In some embodiments, when the test case run ends, the method further comprises: extracting a second entry function from the add-in file; acquiring related information when the test case runs successfully or fails from the work log through the second entry function; when the test case fails to run, acquiring error reporting information which causes the test case to run failure from the work log through the second entry function; and when the test case is successfully operated, acquiring the test success point of the test case on the item to be tested from the work log through the second entry function and summarizing the test success point.
In some embodiments, the method further comprises: and outputting a regression report based on at least the working catalog and the related information when the test case runs successfully or fails.
One embodiment of the present application provides a regression testing system, including: the first acquisition module is used for acquiring basic use case information corresponding to the item to be detected from the database, wherein the basic use case information comprises basic characteristic information, basic execution information and basic configuration information; the second acquisition module is used for acquiring a template file, the template file is customized by a user based on the category of the item to be detected, the template file comprises upper-level use case information, and the upper-level use case information comprises upper-level characteristic information, upper-level execution information, upper-level configuration information and judgment information; the generation module is used for generating a test case for carrying out regression testing on the item to be tested based on the basic case information and the upper-level case information; the operation module is used for operating the test cases and establishing a working catalog for the test cases, wherein the working catalog comprises the test cases and a working log of the test cases when the test cases are operated; and the determining module is used for determining regression test results of the items to be tested based on the working catalogue.
In some embodiments, the running module is further configured to run multiple test cases simultaneously and build a working catalog for each test case.
In some embodiments, the system further comprises a management module for managing the time, number, and resource duty cycle of the test case run.
In some embodiments, the system further includes a preview module, where the preview module is configured to display related information of a test case to be used when performing the regression test on the item to be tested under a condition that the regression test system is not started.
According to the regression testing method and system, the user generates the test cases according to the template file and the basic case information customized by the types of the items to be tested, so that the information about the types of the items to be tested in the test cases provided by the regression testing tool is not from the interior of the regression testing tool any more, but from the template file customized by the user, the completeness of the test cases can be ensured to ensure the accuracy of the regression testing, and the restriction that the regression testing tool can only carry out the regression testing on the items to be tested in a certain type is relieved, so that the regression testing tool can be applied to more test scenes, can be suitable for the items to be tested in various different types, and has good universality and flexibility.
Drawings
The following drawings describe in detail exemplary embodiments disclosed in the present application. Wherein like reference numerals refer to like structure throughout the several views of the drawings. Those of ordinary skill in the art will understand that these embodiments are non-limiting, exemplary embodiments, and that the drawings are for illustration and description only and are not intended to limit the scope of the application, as other embodiments may equally well accomplish the inventive intent in this disclosure. It should be understood that the drawings are not to scale. Wherein:
FIG. 1 is a flow chart of a regression testing method according to some embodiments of the application;
FIG. 2 is a flow chart of a method of generating a test program according to some embodiments of the application;
FIG. 3 is a flow chart of a method of determining whether to run a test case according to some embodiments of the application;
FIG. 4 is a block diagram of a regression testing system according to some embodiments of the application.
Detailed Description
The following description provides specific applications and requirements of the application to enable any person skilled in the art to make and use the application. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the application. Thus, the present application is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the claims.
In the software development process, once the software code is modified, new problems may be introduced, and at this time, regression testing needs to be performed on the modified software code by using test cases to determine whether the modified code references other errors or causes the other software code to generate new errors. Regression testing is often a tedious, repetitive task, requires a lot of manpower and time, and is prone to omission or error. For this, regression testing tools have been developed to implement automated regression testing of software code to improve the efficiency and quality of the regression testing.
Currently, many regression testing tools are available on the market, but they can only perform regression testing on a certain class of software codes for specific work and business needs, for example, these regression testing tools can only be used for performing regression testing on software codes of a specified company; as another example, these regression testing tools can only be used to perform regression testing on software code written in a specified programming language or specified programming software; as another example, these regression testing tools may only be used to perform regression testing on software code developed based on a certain requirement. Further, the information about the categories of software codes in the test cases is typically from within the regression testing tool, and each regression testing tool typically has only information about one category of software codes within it, which can result in the regression testing tool having a requirement for the category of software codes. Therefore, the test scenes of the regression testing tools are often single, and the universality and the flexibility are lacking. Meanwhile, the regression testing tools are poor in framing, robustness and development capability, and cannot meet the requirements of long-term development and demand evolution in work. In addition, these regression testing tools often cannot be put into enough manpower to develop more powerful task processes and functions for result management based on the consideration of the use value and development cost, so that the management capability and analysis capability for the regression testing task are also weak.
The embodiment of the application provides a regression testing method, which comprises the following steps: acquiring basic use case information corresponding to an item to be tested from a database, wherein the basic use case information at least comprises basic characteristic information, basic execution information and basic configuration information; obtaining a template file, wherein the template file is customized by a user based on the category of the item to be detected; generating a test case for carrying out regression testing on the item to be tested based on the basic case information and the template file; running test cases and establishing a working catalog for the test cases; and outputting regression test results of the items to be tested based on the working catalog. According to the regression testing method provided by the embodiment of the application, the user generates the test case according to the template file and the basic case information customized by the category of the item to be tested, so that the information about the category of the item to be tested in the test case provided by the regression testing tool is not from the interior of the regression testing tool any more, but from the template file customized by the user, the integrity of the test case can be ensured so as to ensure the accuracy of the regression test, and the limitation that the regression testing tool can only carry out the regression test on the item to be tested in a certain category is relieved, so that the regression testing tool can be applied to more test scenes, can be suitable for various items to be tested in different categories, and has good universality and flexibility.
The technical scheme of the application is described in detail below with reference to the examples and the accompanying drawings.
FIG. 1 is a flow chart of a regression testing method according to some embodiments of the application. Wherein the regression testing method shown in FIG. 1 may be implemented by the regression testing system 400 shown in FIG. 4.
As shown in fig. 1, the regression testing method 100 provided by the embodiment of the present application may include the following steps:
step S110, basic use case information corresponding to the item to be tested is obtained from the database. Specifically, step S110 may be performed by the first acquisition module 410 in the regression testing system 400 illustrated in fig. 4.
In some embodiments, the item under test refers to software code that requires regression testing. Specifically, when software development is performed, once the software code is modified, regression testing is performed on the modified software code to confirm that the modification does not introduce new errors or cause new errors to be generated by other software codes (i.e., the software code before modification), wherein the modified software code is the item to be tested, and the software code before modification may be referred to as a previous version of the item to be tested in this specification.
In some embodiments, when the previous version of the item under test is the original software code originally developed, the base case information corresponding to the item under test may include information in a test case for performing a verification test on the previous version of the item under test. The verification test may refer to performing a verification test on the software code during or at the end of the development of the software code to confirm whether the software code meets the expected business requirement. In some embodiments, when the previous version of the item under test is not the original software code originally developed, the base case information corresponding to the item under test may include information in a test case for regression testing of the previous version of the item under test. Further, the basic use case information corresponding to the item to be tested may include basic characteristic information, basic execution information, and basic configuration information.
In some embodiments, the basic property information may include property information of a test case (or referred to as a prior test case) that tests a prior version of the item under test (e.g., a verification test or a regression test), and the property information of the prior test case may include a name, a number, an author, a date of generation, a description, and the like of the prior test case. The description of the previous test case may include a description of the purpose, etc. of the previous test case, for example, related information of the software code tested by the previous test case, which functions of the software code can be verified by the previous test case, etc. In some embodiments, the basic execution information may include execution information of a prior test case, e.g., the execution information of the prior test case may include specific execution steps when the prior test case tests a prior version of the item to be tested. In some embodiments, the basic configuration information may include configuration information of prior test cases, which may include configuration parameters employed by the prior test cases when testing prior versions of the item under test, such as environmental configuration parameters for configuring a regression testing environment. In some embodiments, the previous test cases used by the previous version of the item under test may be one or more. For example, multiple prior test cases may be used to verify multiple functions of the prior version of the item under test, respectively.
In some embodiments, the database may be a pre-established library for storing basic test case information, where a plurality of case information directories corresponding to a plurality of items to be tested are stored. In some embodiments, the case information directory may include information of one or more previous test cases corresponding to the item under test. In some embodiments, the base case information catalog may be a collection of information of one or more prior test cases used by a prior version of the item under test, information of one or more test cases (which may also be referred to as prior test cases) used by a version preceding the prior version of the item under test. In some embodiments, when the information of a plurality of previous test cases is included in the basic case information directory, the information of the appropriate previous test case may be selected as the basic case information according to actual needs (for example, a function that actually needs to be verified).
In some embodiments, the database may be located at the cloud. As an exemplary illustration, in step S110, the first obtaining module 410 may obtain basic use case information corresponding to the item to be tested from the cloud through the network. In some embodiments, the first obtaining module 410 may find the basic use case information directory corresponding to the to-be-tested item from the database based on the keywords (e.g., keywords related to name, number, use, description, etc.) in the to-be-tested item, and then may screen out the most suitable basic use case information from the basic use case information directory according to some parameter values or characters customized by the user.
In some embodiments, the base case information stored in the data may be Comma-Separated Values (CSV, sometimes referred to as character separation Values, because separation characters may or may not be Comma) formatted files. By storing the basic use case information corresponding to the plurality of items to be tested in the database by adopting the CSV file storage format, the basic use case information can be edited or checked in a text (e.g. gvim) and table (e.g. openoffice. Org) mode, thereby being beneficial to analyzing, maintaining, classifying and the like of the basic use case information in the database.
Step S120, a template file is acquired. Specifically, step S120 may be performed by the second acquisition module 420 in the regression testing system 400 illustrated in fig. 4.
In step S120, the template file may be customized by the user based on the category of the item to be tested. Specifically, the user may create a template file according to the category of the item to be tested, and then input the template file into the regression testing system 400, and acquired by the second acquisition module 420 in the regression testing system 400. In some embodiments, the categories of the project under test may include programming languages in which the project under test is written, programming software in which the project under test is written, companies in which the project under test is written, development requirements of the project under test, and the like, or combinations thereof. In some embodiments, template files corresponding to a plurality of different items under test under the same category may be uniform.
In some embodiments, the template file may include upper-level case information, where the upper-level case information may refer to information that is common (or the same) in a test case in which multiple items to be tested under the same category perform regression testing, that is, information related to the category of the items to be tested in the test case. Further, the upper level use case information may include upper level characteristic information, upper level execution information, upper level configuration information, and judgment information.
In some embodiments, the upper-level characteristic information may include characteristic information that is common in a test case in which regression testing is performed on a plurality of items to be tested in the same category, and the common characteristic information may include a name, a number, a creator, a creation date, a description, and the like of a template file, where the description of the template file may be a category description about a test case applicable to the items to be tested. In some embodiments, the upper-level execution information may include general execution information in a test case in which multiple items to be tested under the same category perform regression testing, for example, the general execution information may include general execution steps in a test case in which multiple items to be tested under the same category perform regression testing. In some embodiments, the upper configuration information may include configuration parameters that are common in test cases in which multiple items under test in the same category perform regression testing.
The user-defined template file and the second acquisition module 420 are used for acquiring information about the category of the item to be tested in the test case for carrying out regression test on the item to be tested, so that the regression test system 400 does not have requirements on the category of the item to be tested, the regression test system 400 can be applied to more test scenes, the method and the device are suitable for a plurality of different categories of items to be tested, and good universality and flexibility are achieved. Further description of template files may be found elsewhere in this specification (e.g., fig. 3 and its associated description), and will not be described in detail herein.
It should be noted that, there is no strict execution sequence between the step S120 and the step S110, and the step S110 may be executed before the step S120, or may be executed simultaneously with the step S120, or may be executed after the step S120.
Step S130, a test case for performing regression testing on the item to be tested is generated based on the basic case information and the upper case information. Specifically, step S130 may be performed by the generation module 430 in the regression testing system 400 shown in fig. 4.
In step S130, the generating module 430 may generate a test case for performing a regression test on the item under test based on the basic case information and the upper case information. The test case can refer to the description of regression testing tasks for the items to be tested, and is embodied as a testing scheme, a method, a technology, a strategy and the like, and the content of the test case can comprise a testing target, a testing environment, input data, testing steps, expected results, testing scripts and the like, so that a document or an automatically executable script code converted by the document is finally formed. It should be noted that, unless otherwise specified, the test cases in the present application may be regarded as script code that can be automatically executed. In some embodiments, the scripting language for the test cases may be a perl, python, php, tcl, guile, ruby, or like programming language.
Further, the test case may include header information, middle information, and trailer information combined in the order of the run logic. The header information may include upper level property information, the middle information may include basic use case information (e.g., basic property information, basic execution information, and basic configuration information), and the trailer information may include upper level execution information, upper level configuration information, and judgment information. The judgment information may include keyword information for judging whether the test case is successful or not.
Step S140, running the test cases and establishing a working catalog for the test cases. Specifically, step S140 may be performed by the run module 440 in the regression testing system 400 shown in fig. 4.
In step S140, the operation module 440 may configure parameters of the item to be tested (i.e. the test case is operated) during the regression test according to the basic configuration information and the upper configuration information in the test case, for example, configure a test environment for performing the regression test on the item to be tested according to the basic configuration information and the environment configuration parameters in the upper configuration information; the execution module 440 may also execute the test case according to the execution steps in the basic execution information and the upper execution information in the test case. In some embodiments, the priority of the upper level configuration information may be higher than the basic configuration information, that is, when configuring the parameters of the test case during runtime, the execution module 440 may configure according to the upper level configuration information first and then configure according to the basic configuration information. In some embodiments, the upper level execution information may be higher priority than the base execution information, that is, the execution steps in the base execution information may be sub-steps (or actions) in the execution steps in the upper level execution information when the test case is run.
In some embodiments, the worksheet directory may include test cases and a worksheet log of the test case runtime. Further, in step S140, the operation module 440 may operate the test case and generate a work log of the test case when the test case is operated, and then establish a work catalog according to the test case and the corresponding work log.
In some embodiments, the test cases and the corresponding working catalogs adopted by each regression test of the item to be tested can be uploaded into the database so as to realize maintenance and update of the database.
In some embodiments, the run module 440 may run the test cases directly. In some embodiments, the run module 440 may also run test cases based on simulation tools. Further, the run module 440 may generate a test program based on the test cases and the items under test, and then call the simulation tool to run the test program. Whether the test program runs successfully or not can be used for indicating whether the test case runs successfully or not. In some embodiments, the simulation tool may be selected according to the category of the item under test (e.g., programming language used by the item under test, etc.). For example, when the programming language used for the project under test is perl, python, etc., the simulation tool may employ a compiled Verilog simulator. The accuracy of the regression test result can be improved by running the test case through the simulation tool.
And step S150, determining regression test results of the items to be tested based on the working catalogue. Specifically, step S150 may be performed by the determination module 450 in the regression testing system 400 shown in fig. 4.
In step S150, since the tail information of the test case includes the judgment information, when the test case is run, the generated work log may include the corresponding keyword information in the judgment information, and the determining module 450 may scan the keyword information in the work log and determine whether the test case is run successfully based on the keyword information. As an exemplary illustration, when keyword information for judging success of the test case appears in the judgment information in the work log, it may be determined that the test case is successfully operated, so that it is determined that no error occurs in the regression test result of the item to be tested; when the keyword information used for judging the test case failure appears in the judging information in the work log, the test case operation failure can be determined, so that the regression test result of the item to be tested is determined to be in error.
FIG. 2 is a flow chart of a method of generating a test program according to some embodiments of the application.
As shown in fig. 2, a method 200 of generating a test program may include the steps of:
Step S210, obtaining a plug-in file customized by a user.
In step S210, a user may customize a plug-in file, and then input the plug-in file into the regression testing system 400, where the plug-in file may be used to configure a pre-operation before the test case starts to run, for example, to generate a test program that can be run by a corresponding simulation tool based on the test case and the item to be tested.
Step S220, extracting the first entry function from the add-in file.
In step S220, when the add-in file is input into the regression testing system 400, a first entry function may be extracted from the add-in file. In some embodiments, the first entry function may be a compiler (or compiler), which may translate the test cases and items under test into equivalent machine language format target programs (i.e., test programs) so as to be capable of being run by the simulation tool. In some embodiments, the first entry function may include compilers such as MyEclipse, pyCharm, visual Studio Code, android Studio, VCS, and XRUN. In some embodiments, a corresponding compiler may be selected as the first entry function based on the programming language type of the test case and the item under test. For example, when the test case and the item to be tested are in the Python language, pyCharm can be selected as the first entry function, and PyCharm is a compiling program mainly aiming at the Python language, so that the quick construction of the Python framework can be supported, and the compiling efficiency of the test case and the item to be tested can be improved.
Step S230, compiling the test case and the item to be tested into a test program through a first entry function.
In step S230, the regression testing system 400 may compile the test cases and the items to be tested into a test program through the first entry function.
In some embodiments, the plug-in file may also be used to configure post operations after test case execution is completed, e.g., when test case execution is completed, find relevant information about test case execution success or failure. Further, the regression testing system 400 may extract a second entry function from the plug-in file, and then obtain relevant information when the test case runs successfully or fails from the work log through the second entry function. Further, when the test case fails to run, error reporting information which causes the test case to run failure can be obtained from the work log through the second entry function, wherein the error reporting information can include the position, the type, the specific description and the like of the error in the test case running. The types of errors may include, among others, variable name errors (e.g., undefined variables appear in the test case), grammar errors (e.g., errors on code statements or characters unrecognizable by a computer in the test case), indentation errors (incorrect indentation of code in the test case), etc. In some embodiments, the second entry function may include a hash function, such as a cyclic redundancy check, that enables code error detection. And when the test case is successfully operated, the test success point of the test case to the item to be tested can be obtained from the work log through the second entry function and can be summarized. The test success point of the test case to-be-tested item may refer to a function that is tested successfully in the test case to-be-tested item when the test case is running.
In some embodiments, the regression testing method 100 may further include outputting a regression report based at least on the working catalog, the relevant information when the test case is run successful or failed (e.g., the test success point of the item to be tested when the test case is run successful and the summary result of the test success point, the error information that causes the test case to run failure when the test case is run failed). Further, the regression testing system 400 may output a regression report, such as in html format, for visual display to the user. The regression report may include, among other things, the process of performing regression testing on the item under test and the summary of the results. In some embodiments, the regression report may include information regarding the point in time at which the test case was run, the working catalog of the test case, the number of test case runs, and the like. In some embodiments, the regression report may include a classification of the regression test results of the test cases, e.g., the number and percentage of test cases that run successfully or fail. In some embodiments, the regression report may also include classification of the error information that caused the test case to fail, e.g., the number and percentage of test cases that fail due to various types of error information (e.g., variable name errors, grammar errors, and indentation errors), etc.
In some embodiments, the number of prior test cases used by the prior version of the item under test may be multiple, and each prior test case may be used to test a different function of the prior version of the item under test, where there may be dependencies between the prior test cases. For example, if the previous test case a is a search function and the previous test case B is a search function (e.g., a filtering function, etc.), then the previous test case a is a dependent case of the previous test case B, and if the previous test case a fails to run when the regression test is performed on the previous version of the item to be tested, the previous test case B must fail to run. Further, in the application, if a previous test case including basic case information corresponding to a to-be-tested item has a dependency case, a test case for performing a regression test on the to-be-tested item also has a corresponding dependency case, and when the test case is used for performing the regression test on the to-be-tested item, the dependency case may be used for performing the regression test on the to-be-tested item first, and whether the test case needs to be run is determined according to the running result of the dependency case.
FIG. 3 is a flowchart of a method of determining whether to run a test case according to some embodiments of the application.
As shown in FIG. 3, a method 300 of determining whether to run a test case may include the steps of:
step S310, obtaining the dependent case information of the basic case information from the database.
In step S310, the dependent case information of the base case information may refer to information of a dependent case of a previous test case corresponding to the base case information. Wherein the basic use case information and the dependent use case information thereof are stored together in the database under the same use case information catalog, the first acquisition module 410 may acquire the basic use case information together with the dependent use case information of the basic use case information when acquiring the basic use case information from the database.
Step S320, generating the dependent case of the test case based on the dependent case information and the upper-level case information.
In step S320, since the dependent case information and the base case information are both information of previous test cases in nature, an example in which regression tests are performed on items to be tested can be generated based on the dependent case information and the upper case information as well, and since the previous test case corresponding to the dependent case information is a dependent example of a previous test case corresponding to the base case information, an example generated based on the dependent case information and the upper case information can be used as a dependent example of a test case generated based on the base case information and the upper case information.
Step S330, running dependent use cases.
In step S330, since the method of running the dependent case is similar to the method of running the test case, for example, the running module 440 may run the dependent case and build a working catalog for the dependent case. The relevant description of the run-dependent case may refer to the description of the run-test case in FIG. 1.
Step S340, determining whether to run the test case or not based on the running result of the dependent case.
In step S340, the running result of the dependent use case, that is, the success or failure of the running of the dependent use case, may be determined according to the working catalog of the dependent use case. If the dependent case is successfully operated, the test case is rerun, if the dependent case is failed to be operated, the test case is not operated any more, and the reason that the test case is not operated is output.
When regression testing is carried out on the item to be tested, unnecessary test cases can be prevented from running through the dependency relationship between the test cases and the dependent cases, invalid tests are reduced, the overall time of the regression tests is saved, and the efficiency of the regression tests is improved.
FIG. 4 is a block diagram of a regression testing system according to some embodiments of the application.
As shown in fig. 4, the embodiment of the present application further provides a regression testing system 400, where the regression testing system 400 may include a first obtaining module 410, a second obtaining module 420, a generating module 430, a running module 440, and a determining module 450.
The first obtaining module 410 may be configured to obtain, from a database, basic use case information that obtains basic characteristic information, basic execution information, and basic configuration information corresponding to an item to be tested from the database.
The second obtaining module 420 may be configured to obtain a template file, where the template file is customized by a user based on a category of the item to be tested, and the template file includes upper level use case information, where the upper level use case information includes upper level characteristic information, upper level execution information, upper level configuration information, and judgment information.
The generating module 430 may be configured to generate a test case for performing a regression test on the item under test based on the basic case information and the upper case information. The test case can comprise head information, middle information and tail information which are combined according to an operation logic sequence, wherein the head information comprises the upper-level characteristic information, the middle information comprises the basic case information, and the tail information comprises the upper-level execution information, the upper-level configuration information and the judgment information; the judging information comprises keyword information for judging success and/or failure of the test case operation.
The run module 440 may be configured to run test cases and create a working catalog for the test cases, where the working catalog includes the test cases and a working log of the test case run time, where the working log includes judgment information. In some embodiments, the run module 440 may be configured to run multiple test cases simultaneously and build a working catalog for each test case, thereby facilitating an increase in the test efficiency of the regression testing system 400.
The determination module 450 may be configured to determine regression testing results for the item under test based on the working catalog.
In some embodiments, the determination module 450 may also include a scanning sub-module and a determination sub-module. The scanning sub-module can be used for scanning keyword information appearing in the work log; the judging sub-module can be used for judging whether the test case runs successfully or not based on the keyword information.
In some embodiments, the first obtaining module 410 may be further configured to obtain the dependent case information of the basic case information from the database; the generating module 430 may be configured to generate a dependent case of the test case based on the dependent case information and the upper level case information; the run module 440 may be used to run dependent use cases; the determination module 450 may also be used to determine whether to run the test case based on the run results of the dependent case.
In some embodiments, the run module 440 may also include a generate sub-module and a call sub-module. The generating submodule can be used for generating a test program based on the test case and the item to be tested; the calling sub-module may be used to call a simulation tool to run a test program.
In some embodiments, the generation sub-module may include an acquisition unit, an extraction unit, and a compilation unit. The acquisition unit can be used for acquiring the plug-in file customized by a user; the extraction unit may be configured to extract a first entry function from the add-in file; the compiling unit may be configured to compile the test case and the item to be tested into a test program through the first entry function.
In some embodiments, the extraction unit may be further configured to extract the second entry function from the plug-in file. In some embodiments, the regression testing system 400 may further include a third obtaining module that may be configured to obtain, from the work log, error reporting information that may result in the test case failing to run through the second entry function when the test case fails to run.
In some embodiments, the regression testing system 400 may also include an output module that may be used to output a regression report based at least on the work catalog, the regression test results, and the error reporting information.
In some embodiments, the regression testing system 400 may also include a management module that focuses on managing the time, number, and resource duty cycle of test case runs so that the regression testing system 400 has a strong management capability.
In some embodiments, the regression testing system 400 may further include a preview module, where the preview module may be configured to display relevant information, such as a name, a description, etc., of a test case to be used when the regression testing system 400 performs the regression test on the item to be tested, before the regression testing system 400 performs the regression test on the item to be tested, i.e., in a case where the regression testing system is not started, so that a user may reject some unnecessary test cases according to a display result of the preview module before the regression test, thereby improving efficiency of performing the regression test on the item to be tested by the regression testing system 400.
The possible beneficial effects of the embodiment of the application include but are not limited to: (1) According to the regression testing method and the regression testing system for executing the regression testing method, through the custom template file, the restriction that the regression testing system can only carry out regression testing on a certain type of items to be tested can be relieved, so that the regression testing system can be applied to more testing scenes, can be suitable for various different types of items to be tested, and has good universality and flexibility; (2) The regression testing method provided by the application can run the test cases through the simulation tool, so that the accuracy of regression testing results can be improved; (3) In the regression testing method provided by the application, the regression report can be output so as to be convenient for a user to check; (4) In the regression testing method provided by the application, through the dependency relationship between the test cases and the dependent cases, unnecessary test cases can be avoided to run, invalid tests are reduced, the overall time of the regression test is saved, and the efficiency of the regression test is improved; (5) The regression testing system provided by the application can realize intelligent management of the running time and the process number of the test cases; (6) The regression testing system provided by the application is provided with the preview module, so that a user can remove some unnecessary test cases according to the display result of the preview module before regression testing, and the efficiency of the regression testing system for carrying out regression testing on the items to be tested is improved.
It should be noted that, the advantages that may be generated by different embodiments may be different, and in different embodiments, the advantages that may be generated may be any one or a combination of several of the above, or any other possible advantages that may be obtained.
While the basic concepts have been described above, it will be apparent to those skilled in the art that the foregoing detailed disclosure is by way of example only and is not intended to be limiting. Although not explicitly described herein, various modifications, improvements and adaptations of the application may occur to one skilled in the art. Such modifications, improvements, and modifications are intended to be suggested within this specification, and are therefore within the spirit and scope of the exemplary embodiments of this application.
It should be noted that, in the description of the present application, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "fixed" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; the device can be rotationally connected or slidingly connected; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art in combination with specific cases.
In addition, when terms such as "first", "second", "third", etc. are used in the present specification to describe various features, these terms are only used to distinguish between the features, and are not to be construed as indicating or implying any association, relative importance, or implicitly indicating the number of features indicated.
In addition, the present description describes example embodiments with reference to idealized example cross-sectional and/or plan and/or perspective views. Thus, differences from the illustrated shapes, due to, for example, manufacturing techniques and/or tolerances, are to be expected. Thus, the exemplary embodiments should not be construed as limited to the shapes of regions illustrated herein but are to include deviations in shapes that result, for example, from manufacturing. Thus, the regions illustrated in the figures are schematic in nature and their shapes are not intended to illustrate the actual shape of a region of a device and are not intended to limit the scope of the exemplary embodiments.
Meanwhile, the present application uses specific words to describe the embodiments of the present specification. Reference to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic is associated with at least one embodiment of the application. Thus, it is emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various positions are not necessarily referring to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of the application may be combined as suitable.
Similarly, it should be noted that in order to simplify the description of the present disclosure and thereby aid in understanding one or more inventive embodiments, various features are sometimes grouped together in a single embodiment, figure, or description thereof. This method of disclosure, however, is not intended to imply that more features than are required by the subject application. Indeed, less than all of the features of a single embodiment disclosed above.
Finally, it should be understood that the embodiments described herein are merely illustrative of the principles of the embodiments of the present application. Other variations are also possible within the scope of the application. Thus, by way of example, and not limitation, alternative configurations of embodiments of the application may be considered in keeping with the teachings of the application. Accordingly, the embodiments of the present application are not limited to the embodiments explicitly described and depicted herein.
Claims (10)
1. A regression testing method, comprising:
acquiring basic use case information corresponding to an item to be detected from a database, wherein the basic use case information comprises basic characteristic information, basic execution information and basic configuration information;
the method comprises the steps that a template file is obtained, the template file is customized by a user based on the category of a to-be-tested item, the category of the to-be-tested item comprises programming language, the template file comprises upper-level use case information, the upper-level use case information comprises upper-level characteristic information, upper-level execution information, upper-level configuration information and judgment information, wherein the upper-level characteristic information comprises characteristic information which is common in test cases for carrying out regression testing on a plurality of to-be-tested items in the same category, the upper-level execution information comprises execution information which is common in test cases for carrying out regression testing on a plurality of to-be-tested items in the same category, and the upper-level configuration information comprises configuration parameters which are common in test cases for carrying out regression testing on a plurality of to-be-tested items in the same category;
Generating a test case for carrying out regression testing on the item to be tested based on the basic case information and the upper-level case information, wherein the test case comprises head information, middle information and tail information which are combined according to an operation logic sequence, the head information comprises the upper-level characteristic information, the middle information comprises the basic case information, the tail information comprises the upper-level execution information, the upper-level configuration information and the judgment information, and the judgment information comprises keyword information for judging success and/or failure of operation of the test case;
running the test case and establishing a working catalog for the test case, wherein the working catalog comprises the test case and a working log of the test case in running, and the running the test case comprises the following steps: generating a test program based on the test case and the item to be tested; calling a simulation tool to run the test program;
and determining regression test results of the items to be tested based on the working catalogue.
2. The regression testing method of claim 1 wherein said determining the regression testing result for the item under test based on the working catalog comprises:
Scanning the keyword information appearing in the work log;
and judging whether the test case is successfully operated or not based on the keyword information.
3. The regression testing method of claim 2, further comprising:
obtaining the dependent case information of the basic case information from a database;
generating a dependent case of the test case based on the dependent case information and the upper-level case information;
running the dependent use case;
and determining whether to run the test case or not based on the running result of the dependent case.
4. The regression testing method of claim 1 wherein the generating a test program based upon the test case and the item under test comprises:
acquiring a plug-in file customized by a user;
extracting a first entry function from the plug-in file;
compiling the test case and the item to be tested into a test program through the first entry function.
5. The regression testing method of claim 4, further comprising, when the test case run ends:
extracting a second entry function from the add-in file;
acquiring related information when the test case runs successfully or fails from the work log through the second entry function; when the test case fails to run, acquiring error reporting information which causes the test case to run failure from the work log through the second entry function; and when the test case is successfully operated, acquiring the test success point of the test case on the item to be tested from the work log through the second entry function and summarizing the test success point.
6. The regression testing method of claim 5, further comprising:
and outputting a regression report based on at least the working catalog and the related information when the test case runs successfully or fails.
7. A regression testing system, comprising:
the first acquisition module is used for acquiring basic use case information corresponding to the item to be detected from the database, wherein the basic use case information comprises basic characteristic information, basic execution information and basic configuration information;
the second acquisition module is used for acquiring a template file, the template file is customized by a user based on the category of the item to be tested, the category of the item to be tested comprises programming language, the template file comprises upper-level use case information, the upper-level use case information comprises upper-level characteristic information, upper-level execution information, upper-level configuration information and judgment information, wherein the upper-level characteristic information comprises characteristic information which is common in test cases for carrying out regression testing on a plurality of items to be tested in the same category, the upper-level execution information comprises execution information which is common in test cases for carrying out regression testing on a plurality of items to be tested in the same category, and the upper-level configuration information comprises configuration parameters which are common in test cases for carrying out regression testing on a plurality of items to be tested in the same category;
The generating module is used for generating a test case for carrying out regression testing on the item to be tested based on the basic case information and the upper-level case information, the test case comprises head information, middle information and tail information which are combined according to an operation logic sequence, the head information comprises the upper-level characteristic information, the middle information comprises the basic case information, the tail information comprises the upper-level execution information, the upper-level configuration information and the judging information, and the judging information comprises keyword information for judging the success and/or failure of the operation of the test case;
the operation module is used for operating the test cases and establishing a working catalog for the test cases, wherein the working catalog comprises the test cases and working logs of the test cases when the test cases are operated, and the operation of the test cases comprises the following steps: generating a test program based on the test case and the item to be tested; calling a simulation tool to run the test program;
and the output module is used for determining regression test results of the items to be tested based on the working catalogue.
8. The regression testing system of claim 7 wherein the run module is further configured to run a plurality of test cases simultaneously and build a working catalog for each test case.
9. The regression testing system of claim 7 further comprising a management module for managing the time, number and resource duty cycle of the test case run.
10. The regression testing system of claim 7, further comprising a preview module for displaying information about the test case to be used in regression testing of the item under test without activation of the regression testing system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310801534.3A CN116541308B (en) | 2023-07-03 | 2023-07-03 | Regression testing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310801534.3A CN116541308B (en) | 2023-07-03 | 2023-07-03 | Regression testing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116541308A CN116541308A (en) | 2023-08-04 |
| CN116541308B true CN116541308B (en) | 2023-10-24 |
Family
ID=87452801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310801534.3A Active CN116541308B (en) | 2023-07-03 | 2023-07-03 | Regression testing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116541308B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117077592B (en) * | 2023-10-12 | 2024-03-08 | 芯耀辉科技有限公司 | Regression data monitoring method, monitoring device and monitoring system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104978191A (en) * | 2015-06-24 | 2015-10-14 | 小米科技有限责任公司 | Application deployment method and server |
| CN109885476A (en) * | 2018-12-14 | 2019-06-14 | 平安万家医疗投资管理有限责任公司 | Automated testing method and device, terminal and computer readable storage medium |
| CN111832236A (en) * | 2020-06-29 | 2020-10-27 | 山东云海国创云计算装备产业创新中心有限公司 | Chip regression testing method and system, electronic equipment and storage medium |
| CN114201408A (en) * | 2021-12-17 | 2022-03-18 | 平安壹钱包电子商务有限公司 | Regression testing method, device, computer equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9032361B2 (en) * | 2011-12-15 | 2015-05-12 | Tata Consultancy Services Limited | Agile unit and regression testing framework for domain specific languages |
| US11663113B2 (en) * | 2020-02-20 | 2023-05-30 | International Business Machines Corporation | Real time fault localization using combinatorial test design techniques and test case priority selection |
-
2023
- 2023-07-03 CN CN202310801534.3A patent/CN116541308B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104978191A (en) * | 2015-06-24 | 2015-10-14 | 小米科技有限责任公司 | Application deployment method and server |
| CN109885476A (en) * | 2018-12-14 | 2019-06-14 | 平安万家医疗投资管理有限责任公司 | Automated testing method and device, terminal and computer readable storage medium |
| CN111832236A (en) * | 2020-06-29 | 2020-10-27 | 山东云海国创云计算装备产业创新中心有限公司 | Chip regression testing method and system, electronic equipment and storage medium |
| CN114201408A (en) * | 2021-12-17 | 2022-03-18 | 平安壹钱包电子商务有限公司 | Regression testing method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116541308A (en) | 2023-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109254907B (en) | Java-based interface test report generation method and system | |
| CN109189479B (en) | Parallel automatic verification method for processor instruction set | |
| CN110716870B (en) | Automatic service testing method and device | |
| US7895575B2 (en) | Apparatus and method for generating test driver | |
| CN116541308B (en) | Regression testing method and system | |
| CN111209185B (en) | Keyword-based automated testing method and computer-readable storage medium | |
| US20020116153A1 (en) | Test automation framework | |
| CN111382070A (en) | Compatibility testing method and device, storage medium and computer equipment | |
| CN112241360A (en) | Test case generation method, device, equipment and storage medium | |
| CN107329889B (en) | Method for automatically testing C compiler | |
| CN111797006B (en) | Method, device and equipment for testing thread and storage medium | |
| CN110543427A (en) | Test case storage method and device, electronic equipment and storage medium | |
| US10733075B2 (en) | Data-driven synthesis of fix patterns | |
| CN114416547A (en) | Test case based test method | |
| CN110928760A (en) | Unit testing method and device in embedded system | |
| CN112084108A (en) | Test script generation method and device and related components | |
| CN110413518B (en) | Control object library generation method and device and computer readable storage medium | |
| KR100777103B1 (en) | Apparatus and method for generation of test driver | |
| US11740895B2 (en) | Generation of software program repair explanations | |
| CN115629956A (en) | Software defect management method and system based on interface automatic test | |
| CN114661615A (en) | FPGA software testing method and device | |
| CN113672509A (en) | Automatic testing method, device, testing platform and storage medium | |
| Halabi et al. | Mutation testing tools for Java programs–a survey | |
| JP2021111389A (en) | Improvement in parsability of code snippet | |
| CN112328503A (en) | Task acceptance method for software development working platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |