CN116166534A

CN116166534A - Regression testing method, device and equipment

Info

Publication number: CN116166534A
Application number: CN202211651020.6A
Authority: CN
Inventors: 李福庆; 蒋皓; 胡涛; 沈斌
Original assignee: Hangzhou Netease Cloud Music Technology Co Ltd
Current assignee: Hangzhou Netease Cloud Music Technology Co Ltd
Priority date: 2022-12-21
Filing date: 2022-12-21
Publication date: 2023-05-26

Abstract

The application discloses a regression testing method, device and equipment, wherein the method comprises the following steps: acquiring an execution set, wherein the execution set comprises online operation data; based on the modified code, performing playback operation according to the online operation data to obtain a playback execution result; and generating a test report according to the playback execution result and the online operation data to complete the regression test of the modified code. According to the method and the device, the use case execution set which is manually maintained is replaced by the operation data on the drainage line, so that the labor cost in the test process can be reduced. And the regression test is carried out according to the online operation data, so that the real scene of the online service can be highly simulated, and the simulation test of the service in regression iteration is realized.

Description

Regression testing method, device and equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a regression testing method, apparatus, and device.

Background

Regression testing is a component of the software lifecycle, and takes a significant weight percentage of the overall software testing process.

In the related art, when performing regression testing, the new and old functions are usually returned by using a test regression execution set in a regression environment. However, since the test regression execution set requires manual maintenance, an increase in labor cost is caused.

Disclosure of Invention

The embodiment of the application provides a regression testing method, device and equipment, which can reduce labor cost and can also highly simulate the real scene of on-line service, thereby realizing the simulation test of the service in regression iteration.

In a first aspect, there is provided a regression testing method, comprising:

acquiring an execution set, wherein the execution set comprises online operation data;

based on the modified code, performing playback operation according to the online operation data to obtain a playback execution result;

and generating a test report according to the playback execution result and the online operation data to complete regression testing of the modified code.

Optionally, the method further comprises:

acquiring a gateway log, wherein the gateway log comprises on-line operation data, the on-line operation data comprises interface information and parameter information of a plurality of interfaces, and each interface information comprises corresponding interface attribute information;

and classifying the interface information and the parameter information of the interfaces according to the attribute information corresponding to the interfaces to obtain a plurality of execution sets.

Optionally, the performing playback operation according to the online operation data based on the modified code to obtain a playback execution result includes:

And based on the modified code and an online shadow library, performing playback operation according to the online operation data to obtain a playback execution result, wherein the online shadow library is used for storing data generated by the playback operation of the regression test.

Optionally, if the on-line operation data is an on-line write operation, the method further includes:

and updating the playback execution result corresponding to the online write operation to the online shadow library.

Optionally, the method further comprises:

and comparing the online main library with the online shadow library to generate a database comparison result, wherein initial data of the online main library is consistent with initial data of the online shadow library, and the online main library is used for storing data generated by online operation.

Optionally, the method further comprises:

and based on the modified codes and the online shadow library, performing playback operation according to the online operation data to obtain playback execution link information, and storing the playback execution link information.

Optionally, the on-line operation data includes an on-line execution result, and the generating a test report according to the execution result and the on-line operation data includes:

comparing the playback execution result with the online execution result to obtain a first comparison result;

And generating a test report according to the first comparison result.

Optionally, the method further comprises:

extracting on-line execution link information according to the on-line operation data;

comparing the playback execution link information with the online execution link information to obtain a second comparison result;

and generating a test report according to the first comparison result and the second comparison result.

Optionally, the generating a test report according to the first comparison result and the second comparison result includes:

and if the first comparison result indicates that the playback execution result is consistent with the online execution result and the second comparison result indicates that the playback execution link information is consistent with the online execution link information, determining that the execution result is a first execution result and generating a test report according to the first execution result.

and if the first comparison result indicates that the playback execution result is inconsistent with the online execution result or the second comparison result indicates that the playback execution link information is inconsistent with the online execution link information, determining that the execution result is a second execution result, and generating a test report according to the second execution result.

and if the first comparison result indicates that the playback execution result is inconsistent with the online execution result and the second comparison result indicates that the playback execution link information is inconsistent with the online execution link information, determining that the execution result is a third execution result, and generating a test report according to the third execution result.

In a second aspect, there is provided a regression testing apparatus, comprising:

the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an execution set, and the execution set comprises online operation data;

the playback module is used for carrying out playback operation according to the online operation data based on the modified code to obtain a playback execution result;

and the generating module is used for generating a test report according to the playback execution result and the online operation data so as to complete regression testing of the modified code.

In a third aspect, there is provided a computer readable storage medium storing a computer program adapted to be loaded by a processor for performing the steps of the regression testing method as described in the first aspect above.

In a fourth aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory having stored therein a computer program, the processor being adapted to perform the steps of the regression testing method as described in the first aspect above by invoking the computer program stored in the memory.

In a fifth aspect, there is provided a computer program product comprising computer instructions which, when executed by a processor, implement the steps in the regression testing method as described in the first aspect above.

According to the embodiment of the application, the execution set is obtained, the execution set comprises on-line operation data, then playback operation is carried out according to the on-line operation data based on the modified code, a playback execution result is obtained, and then a test report is generated according to the playback execution result and the on-line operation data so as to complete regression testing of the modified code. According to the embodiment of the application, the use case execution set which is manually maintained is replaced by the operation data on the drainage line, so that the labor cost in the test process can be reduced. And the regression test is carried out according to the online operation data, so that the real scene of the online service can be highly simulated, and the simulation test of the service in regression iteration is realized.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a first application scenario of a test environment provided in an embodiment of the present application.

Fig. 2 is a schematic diagram of a second application scenario of the test environment provided in the embodiment of the present application.

Fig. 3 is a schematic diagram of a third application scenario of the test environment provided in the embodiment of the present application.

Fig. 4 is an overall flow chart of product on-line in the related art according to the embodiment of the present application.

Fig. 5 is a flow chart of a regression testing method according to an embodiment of the present application.

Fig. 6 is a schematic diagram of a first application scenario of the regression testing method provided in the embodiment of the present application.

Fig. 7 is an overall flow chart of product on-line of the regression testing method provided in the embodiment of the present application.

Fig. 8 is a schematic diagram of an overall process of the regression testing method according to the embodiment of the present application.

Fig. 9 is a schematic structural diagram of a regression testing apparatus according to an embodiment of the present application.

Fig. 10 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

The data related to the application can be data authorized by a user or fully authorized by each party, and the acquisition, the transmission, the use and the like of the data all meet the requirements of national related laws and regulations, and the implementation modes/embodiments of the application can be mutually combined.

The embodiment of the application provides a regression testing method, a regression testing device, computer equipment and a storage medium. Specifically, the regression testing method of the embodiments of the present application may be performed by a computer device.

First, partial terms or terminology appearing in the course of describing the embodiments of the present application are explained as follows:

The local environment (locale) refers to a local environment of a developer, and is mainly used for local code development, debugging, self-testing and the like. Each developer's own computer may act as a local environment.

The development environment (Dev environment) is an environment specially used by developers for development and joint debugging, the service configuration is relatively random, only the local code configuration of the developers is affected, and all error reports are generally opened for development and debugging convenience. This environment is used by developers and is generally less stable.

Test environment (Testing environment), which refers to a description of the software and hardware environment on which the test is run, as well as any other software that interacts with the software under test, including drivers and stub. The test environment refers to the collective term for computer hardware, software, network devices, and historical data necessary to complete the software testing work.

The characteristics of the test environment are as follows:

(1) The whole test team shares a set of test environments. Under the micro-service architecture, when a service is relied upon by multiple services, if the service is unstable, it affects other large numbers of services to fail testing. As shown in fig. 1, service B is dependent on a plurality of services, and when service B is not available, other services depending on service B cannot be used.

(2) A complete set of test environments for each tester, see fig. 2. In this case, although the problem of environmental dependency can be solved, the software and hardware costs are high, the environmental maintenance costs are high, and the server resource utilization rate is low. For example, a business system contains 10 micro-services, 5 people on a test team, and then 50 servers are needed to manage the test environment.

(3) The virtualization technology of service level multiplexing realizes multiplexing of partial services in the cluster based on control of message routing, as shown in fig. 3.

Among other things, the focus of the development environment: the testers will perform new functional tests, regression verification Bug and other contents in test environments of different forms according to different stages and state of the art of the company, which includes layered test strategies (integrated tests, component tests, contract tests, end-to-end tests) under the micro-service architecture, and some non-functional types of tests.

The pre-release environment (starting environment) is a set of environment which is set up for avoiding missing test of defects caused by the difference of the test environment and the online environment, and is the test environment closest to the production environment. The pre-release environment, which can be seen from the name, is used for pre-rendering and verification before formal release. There are some differences between the test environment and the production environment, and in order to avoid missing detection of defects caused by these differences, a pre-release environment has been developed. Although the server performance of the pre-release environment is substantially consistent with the production environment performance, the number of server instances that mainly characterize the pre-release environment is typically only 1 or a few.

Specifically, the pre-release environments of different companies are slightly different, for example, the pre-release environments may use a database backup of the production environment, or the pre-release environments may use the same database as the production environment. However, due to the large number of lines, and multiple libraries and tables. Thus, the pre-release environment is generally consistent with the online environment data.

The pre-release environment is characterized by high simulation, is the last environment before formal release, and is the same as the production environment. For the feature of "database and production environment", special care should be taken that for the same piece of user data, changes to it in both the pre-release environment and the production environment should be avoided. Because the database cache exists in the two sets of environments, the problems of inconsistent data and the like can be generated, and the positioning and the repairing are difficult.

The focus of the pre-release environment mainly comprises the following points:

(1) The release process comprises the following steps: previewing is performed for the operation steps of the release process. If a release requires both updating the database and releasing the service, the order of the operations of the two needs to be clarified. The method of updating the database and then updating the service is common, the method of releasing the service and then updating the database is rare, and errors are easy to report.

(2) Regression testing: regression testing is performed in this environment to avoid dirty data as much as possible. The publishing process requires traffic to verify and recommends end-to-end automated testing at the UI (user interface) level.

(3) Special content test: the test environment may be limited, and some processes or data may not be tested, so that verification may be performed in the pre-release environment, thereby ensuring product online quality.

The production environment (Prod environment) refers to a formal environment for externally distributing services, and is an environment used by end users. It is therefore noted that tests are not to be done at will in this environment, especially tests that may generate dirty data or may result in service outages. Because of the high system complexity, large data volume stored, and large number of server instances of the production environment, a large number of real users can produce a wide variety of behaviors, which can lead to unexpected phenomena, especially in terms of performance or abnormal scenarios. After a problem occurs in a production environment, rights are required for positioning and solving the problem, and special personnel are usually required for operation, so that the working efficiency is affected.

Among these, the focus of the production environment is mainly the following:

(1) On-line testing: service logic test, which can be gray scale test. The effect class verifies the AB test.

(2) On-line monitoring: the production environment is subjected to business and technology monitoring, data, logs and the like of the production environment are analyzed, and the aim of pre-discovering quality risks and exposing problems is achieved.

(3) Continuous integration & continuous delivery (CI/CD)

Continuous integration: in the software production process, the action of submitting codes and integrating the codes into the backbone is carried out by research members at every moment.

Continuous delivery: in the software production process, new requirements are inserted at any moment, and the actions of value delivery are performed in time.

CI/CD is a method of frequently delivering applications to clients by introducing automation during the application development phase. The core concepts of CI/CD are continuous integration, continuous delivery, and continuous deployment. It is a development and operation team oriented solution, mainly directed to the problems raised when integrating new code. CI/CD allows continuous automation and continuous monitoring throughout the life cycle of the application (from integration and testing phases to delivery and deployment). These associated transactions are often collectively referred to as CI/CD pipelines, supported cooperatively by development and operations team in agile fashion.

QA: quality Assurance, is interpreted as quality assurance. The quality assurance is a very important and special component in a quality system, the work of the quality assurance relates to each link of a software research and development flow, each personnel participating in research and development, and meanwhile, the work of the quality assurance does not relate to specific software research and development details.

Referring to fig. 4, the overall flow of the product on-line is shown in fig. 4. First, a need, such as an improvement need or a new function need, is put forth by a planner, and then developed by a developer for the need. After the environment development is completed, the developer submits the test. After the QA completes the new function test in the test environment, executing interface regression execution set to regress the new and old functions in the regression environment, and then regress the new and old functions in the pre-release environment. After the regression, the user can be online to provide service for the user.

However, the labor cost increases due to the need for maintenance by QA personnel for both the current test regression execution set and the pre-sent regression execution set.

Therefore, the embodiment of the application provides a regression testing method, and the operation data on the drainage line is used for replacing a use case execution set maintained manually, so that the labor cost in the testing process can be reduced. And the regression test is carried out according to the online operation data, so that the real scene of the online service can be highly simulated, and the simulation test of the service in regression iteration is realized.

The following will describe in detail. It should be noted that the following description order of embodiments is not a limitation of the priority order of embodiments.

Referring to fig. 5 to fig. 8, an embodiment of the present application provides a regression testing method, and fig. 5 is a schematic flow chart of the regression testing method provided in the embodiment of the present application, where the regression testing method mainly includes steps 101 to 103, which are described as follows:

step 101, an execution set is acquired, wherein the execution set comprises online operation data.

The online operation data is atomic data introduced from an online micro service, and may include interface information, parameter information, and the like. The on-line operation data may include write operation data and read operation data.

In this embodiment, the method may further include: acquiring a gateway log, wherein the gateway log comprises on-line operation data, the on-line operation data comprises interface information and parameter information of a plurality of interfaces, and each interface information comprises corresponding interface attribute information; and classifying the interface information and the parameter information of the interfaces according to the attribute information corresponding to the interfaces to obtain a plurality of execution sets.

The gateway plays a role of forwarding and is the only outlet for providing services to the outside. The log is an important component of the gateway, and the generated data can be used for information retention, security audit, data analysis and other directions. The gateway log may include a request log and a response log, and the gateway log includes a record of each interface request. Each record may include important information such as caller IP, URL of request, response delay, return status code, number of request and response bytes, etc.

Specifically, after the gateway log is acquired, the drainage data, that is, the online operation data, may be acquired from the gateway log. The on-line operation data comprises interface information and parameter information, wherein the parameter information mainly comprises interface types such as GET or POST, and the like, and also comprises interface names, URL type entry parameters, body type entry parameters, header parameters and the like. Referring to fig. 6, the use case management layer may classify according to attribute information corresponding to each interface in the on-line operation data. The attribute information may include a home department, a home service, a home application, a home module, and the like. For example, as shown in fig. 6, the use case management hierarchy classifies interface information and parameter information according to attribute information such as a home department, a home service, a home application, and a home module, and obtains a plurality of execution sets (corresponding to the online drainage data in fig. 6). In particular, the haly management hierarchy can efficiently organize and manage multiple execution sets.

And 102, based on the modified codes, performing playback operation according to the online operation data to obtain a playback execution result.

The modified code may be a code developed by a developer, for example, a code with a new function added, and possibly a code with a modified fault. It is to be understood that the playback operation can be understood as re-execution of the on-line operation data. Specifically, after the playback execution result is obtained, in a subsequent step, the on-line execution result and the playback execution result in the on-line operation data may be compared to evaluate the function of the modified code.

Specifically, step 102 may mainly include: and performing playback operation according to the online operation data based on the modified code and the online shadow library to obtain a playback execution result, wherein the online shadow library is used for storing data generated by the playback operation of the regression test.

It will be readily appreciated that all data will be stored in an online master library for a wide variety of service platforms, such as user information data, basic service data, etc. Taking the internet enterprise as an example, the internet enterprise platform has a wide variety of data, such as user information data, basic service information data, running state data, etc., including each behavior data for each data, which are stored in an online master library. When the user performs an online operation, the online master library performs a read operation and a write operation.

In this embodiment, the on-line shadow library refers to a database identical to the on-line main library, and the data and storage structure are identical. The on-line main library is used for reading data and writing data when a user operates on the line, and the on-line shadow library is used for reading data and writing data when the user operates on the line.

It is easy to understand that the independent on-line shadow library is adopted for playback operation, so that the accuracy of regression test is not affected, and the generation of a piece of real data in the on-line main library during playback of write operation in test can be avoided.

In this embodiment, if the on-line operation data is an on-line write operation, the method may further include: and updating the playback execution result corresponding to the online write operation to the online shadow library.

Specifically, when playback operation is performed for online write operation, the playback execution result corresponding to the write operation is updated to the online shadow library in real time. In the subsequent step, the on-line shadow library and the on-line main library can be compared to determine whether the playback execution result is correct, thereby verifying whether the function of the modified code is normal.

In addition, in the embodiment, the on-line operation and the test operation can be separated by adopting the on-line shadow library mode, so that the normal experience of a user is prevented from being interfered.

For example, taking an online write operation as "issue a message," when an information is issued, there is a real data generation. If the on-line shadow library is not adopted, when playback operation is carried out on the on-line writing operation, the same two pieces of data exist in the on-line main library, and the user can see the information of the two pieces of information which are identical, so that the normal experience of the user is affected.

In this embodiment, the method may further include: and comparing the online main library with the online shadow library to generate a database comparison result, wherein initial data of the online main library is consistent with initial data of the online shadow library, and the online main library is used for storing data generated by online operation.

The on-line shadow library is the same database as the on-line main library, and the on-line shadow library after the on-line operation data is played back and the on-line main library after the on-line operation data is executed should be the same. By comparing the on-line shadow library with the on-line main library, whether the playback execution result after the modified code is played back is correct or not can be determined, so that whether the new and old functions of the modified code can still be executed correctly or not can be determined.

In this embodiment, the method may further include: and based on the modified codes and the on-line shadow library, performing playback operation according to the on-line operation data to obtain playback execution link information, and storing the playback execution link information.

The playback execution link information refers to a behavior trace left when the playback operation is executed. Taking an enterprise service as an example, in general, an enterprise has a plurality of kinds of data, and each behavior is marked when performing read-write operations on the data, for example, when reading user information, modifying the user information, and the like. Specifically, storing the playback execution link information may facilitate comparing the playback execution link information with the on-line operation link information in a subsequent process, thereby evaluating the performance of the modified code.

And step 103, generating a test report according to the playback execution result and the online operation data to complete the regression test of the modified code.

Specifically, the on-line operation data includes interface information and parameter information, the parameter information includes parameter entering information and parameter exiting information, an on-line execution result of the on-line operation can be obtained according to the parameter entering information and the parameter exiting information, and on-line execution link information of the on-line operation can be extracted. Then, a test report can be generated according to the on-line execution result, the on-line execution link information and the playback execution result, and the test report is provided for an ending personnel to further locate the fault of the code.

In this embodiment, the on-line operation data includes an on-line execution result, and step 103 may mainly include: comparing the playback execution result with the online execution result to obtain a first comparison result; and generating a test report according to the first comparison result.

Specifically, the playback execution result is a result obtained after a playback operation is performed based on the modified code in accordance with the online operation data. The on-line execution result is a result of executing on-line operation data based on the pre-modification code. By comparing whether the playback execution result is consistent with the online execution result, whether the new and old functions of the modified code can be normally and accurately executed can be determined.

For example, taking Key-Value pair (Key-Value) as an example, if the Key-Value pair result in the playback execution result is compared with the Key-Value result in the on-line execution result, that is, the Key is identical, and the Value is identical, it can be determined that the playback execution result is compared with the on-line execution result to be identical. If the Key Value pair result in the playback execution result is not completely consistent with the Key Value pair result in the on-line execution result, namely, the Key of the playback execution result and the Key of the on-line execution result, or the Value of the playback execution result and the Value of the on-line execution result are inconsistent, the situation that at least one is inconsistent can be determined, and the playback execution result and the on-line execution result are not completely consistent.

In this embodiment, the method may further include: extracting on-line execution link information according to on-line operation data; comparing the playback execution link information with the online execution link information to obtain a second comparison result; and generating a test report according to the first comparison result and the second comparison result.

Specifically, the on-line operation data includes interface information and parameter information, the parameter information includes parameter in-coming information and parameter out-coming information, and on-line execution link information of the on-line operation can be extracted and obtained according to the parameter in-coming information and the parameter out-coming information. Specifically, when a playback operation is performed on the on-line operation data, playback execution link information of the playback operation is stored. By comparing playback execution link information and online execution link information, it can be determined whether a link is changed when the same operation is performed.

Specifically, at present, the request service is usually a micro-service architecture, one request service usually passes through a plurality of architectures, and by comparing the playback execution link information with the on-line execution link information and generating a test report according to the comparison result, a technician can evaluate the influence range of the code modification.

For example, if it is determined by comparison that the playback execution link information and the on-line execution link information are completely identical, it can be determined that the execution link is not changed when the same operation as on-line is performed based on the modified code. If it is determined by comparison that the playback execution link information and the on-line execution link information are at least partially inconsistent, it can be determined that the execution link has been changed when the same operation as on-line is performed based on the modified code.

In this embodiment, the step of generating the test report according to the first comparison result and the second comparison result may mainly include: if the first comparison result indicates that the playback execution result is consistent with the online execution result and the second comparison result indicates that the playback execution link information is consistent with the online execution link information, determining that the execution result is the first execution result, and generating a test report according to the first execution result.

Specifically, according to the first comparison result and the second comparison result, three execution results, which are completely similar, incompletely similar, and completely dissimilar, respectively, can be obtained.

Specifically, if the first comparison result indicates that the playback execution result and the online execution result are completely consistent, and the second comparison result indicates that the playback execution link information and the online execution link information are completely consistent, it may be determined that the execution results are completely similar, and a test report is generated according to the completely similar execution results.

In this embodiment, the step of generating the test report according to the first comparison result and the second comparison result may further include: if the first comparison result indicates that the playback execution result is inconsistent with the online execution result or the second comparison result indicates that the playback execution link information is inconsistent with the online execution link information, determining that the execution result is a second execution result, and generating a test report according to the second execution result.

Specifically, if the first comparison result indicates that the playback execution result and the online execution result are not completely consistent, or if the second comparison result indicates that the playback execution link information and the online execution link information are at least partially inconsistent, it may be determined that the execution results are not completely similar, and a test report may be generated according to the incompletely similar execution results.

In this embodiment, the step of "generating the test report according to the first comparison result and the second comparison result" may further include: if the first comparison result indicates that the playback execution result is inconsistent with the online execution result and the second comparison result indicates that the playback execution link information is inconsistent with the online execution link information, determining that the execution result is a third execution result, and generating a test report according to the third execution result.

Specifically, if the first comparison result indicates that the playback execution result and the online execution result are not completely consistent, and the second comparison result indicates that the playback execution link information and the online execution link information are at least partially inconsistent, it may be determined that the execution results are completely dissimilar, and a test report is generated according to the completely dissimilar execution results.

Specifically, referring to fig. 7, a product online flow according to the regression testing method provided in the embodiment of the present application will be described herein:

first, a need, such as an improvement need or a new function need, is put forth by a planner, and then developed by a developer for the need. After the environment development is completed, the developer submits the test. After the QA completes the new function test in the test environment, carrying out regression test on the new and old functions in the regression environment by the regression test method provided by the embodiment of the application, and then returning the new and old functions in the pre-release environment. After the regression, the user can be online to provide service for the user.

In order to better explain the regression testing method provided in the embodiment of the present application, please refer to fig. 8, which is an overall description of the regression testing method provided in the embodiment of the present application:

first, the on-line traffic is drained. Specifically, the micro service traffic information, including interface information, parameter information, and the like, is stored by being introduced. And then, according to interface information, classifying interface information and parameter information of the online traffic according to the attribution department, attribution application, attribution service, attribution module and the like to obtain an application dimension execution set. Then, according to the execution set, read link playback and write link playback are performed. And, playback execution link information of playback of the read link and playback execution link information of the write link are stored. And simultaneously, during playback of the write link, writing the generated data into an on-line shadow library. Then, the playback execution results of the read link and the write link can be compared with the on-line execution results through the interface request and the response information, and the stored playback execution link information can be compared with the on-line execution link information. The on-line execution link information can be extracted according to parameter information of on-line traffic information. In addition, the on-line shadow library may also be compared to the on-line master library. And then, integrating the centralized comparison results, carrying out result evaluation, generating a test report and providing the test report for QA.

As can be seen from the foregoing, in this embodiment, the manner of generating the execution set automatically by online drainage replaces manual maintenance of the execution set, so that the labor cost can be reduced. In addition, the embodiment introduces an on-line shadow library, separates the on-line writing operation data from the writing operation data in the regression test, and avoids influencing the user experience. The embodiment can also evaluate the influence scope of the code after the modification by comparing the playback execution link information with the online execution link information. And the real scene of the on-line service can be highly simulated through the on-line drainage, so that the simulation test of the service in the iterative process is realized. By performing link analysis, test scenario similarity can also be fully diagnosed. In summary, in the quality assurance process of the embodiment, the quality assurance process is greatly optimized.

All the above technical solutions may be combined to form an optional embodiment of the present application, which is not described here in detail.

In order to facilitate better implementation of the regression testing method of the embodiment of the application, the embodiment of the application also provides a regression testing device. Referring to fig. 9, fig. 9 is a schematic structural diagram of a regression testing apparatus according to an embodiment of the present application. The regression testing apparatus 10 may include:

an acquisition module 11, configured to acquire an execution set, where the execution set includes online operation data;

a playback module 12, configured to perform playback operation according to the online operation data based on the modified code, to obtain a playback execution result;

and the generating module 13 is used for generating a test report according to the playback execution result and the online operation data so as to complete the regression test of the modified code.

Optionally, the regression testing apparatus 10 may further include a data processing module that may be configured to: acquiring a gateway log, wherein the gateway log comprises on-line operation data, the on-line operation data comprises interface information and parameter information of a plurality of interfaces, and each interface information comprises corresponding interface attribute information; and classifying the interface information and the parameter information of the interfaces according to the attribute information corresponding to the interfaces to obtain a plurality of execution sets.

Optionally, the playback module 12 may be configured to: and performing playback operation according to the online operation data based on the modified code and the online shadow library to obtain a playback execution result, wherein the online shadow library is used for storing data generated by the playback operation of the regression test.

Optionally, if the on-line operation data is an on-line write operation, the regression testing apparatus 10 may further include an update module that may be used to: and updating the playback execution result corresponding to the online write operation to the online shadow library.

Optionally, the regression testing apparatus 10 may further include a first comparison module that may be configured to: and comparing the online main library with the online shadow library to generate a database comparison result, wherein initial data of the online main library is consistent with initial data of the online shadow library, and the online main library is used for storing data generated by online operation.

Optionally, the playback module 12 may also be configured to: and based on the modified codes and the on-line shadow library, performing playback operation according to the on-line operation data to obtain playback execution link information, and storing the playback execution link information.

Optionally, the online operation data includes online execution results, and the generating module 13 may mainly be used to: comparing the playback execution result with the online execution result to obtain a first comparison result; and generating a test report according to the first comparison result.

Optionally, the generating module 13 may be further specifically configured to: extracting on-line execution link information according to on-line operation data; comparing the playback execution link information with the online execution link information to obtain a second comparison result; and generating a test report according to the first comparison result and the second comparison result.

Optionally, the generating module 13 may specifically be configured to: if the first comparison result indicates that the playback execution result is consistent with the online execution result and the second comparison result indicates that the playback execution link information is consistent with the online execution link information, determining that the execution result is the first execution result, and generating a test report according to the first execution result.

Optionally, the generating module 13 may specifically be configured to: if the first comparison result indicates that the playback execution result is inconsistent with the online execution result or the second comparison result indicates that the playback execution link information is inconsistent with the online execution link information, determining that the execution result is a second execution result, and generating a test report according to the second execution result.

Optionally, the generating module 13 may specifically be configured to: if the first comparison result indicates that the playback execution result is inconsistent with the online execution result and the second comparison result indicates that the playback execution link information is inconsistent with the online execution link information, determining that the execution result is a third execution result, and generating a test report according to the third execution result.

The units in the regression testing apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above units may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor invokes and executes operations corresponding to the above units.

The regression testing apparatus 10 may be integrated in a terminal or a server having a memory and a processor mounted therein and having an arithmetic capability, or the regression testing apparatus 10 may be the terminal or the server.

According to the regression testing device 10 provided by the embodiment of the application, the acquisition module 11 acquires an execution set, the execution set comprises on-line operation data, the playback module 12 performs playback operation according to the on-line operation data based on the modified code to obtain a playback execution result, and the generation module 13 generates a test report according to the playback execution result and the on-line operation data to complete the regression test of the modified code. According to the embodiment of the application, the use case execution set which is manually maintained is replaced by the operation data on the drainage line, so that the labor cost in the test process can be reduced. And the regression test is carried out according to the online operation data, so that the real scene of the online service can be highly simulated, and the simulation test of the service in regression iteration is realized.

Optionally, the application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

Fig. 10 is a schematic structural diagram of a computer device provided in an embodiment of the present application, and as shown in fig. 10, the computer device 1000 may include: a communication interface 1001, a memory 1002, a processor 1003, and a communication bus 1004. Communication interface 1001, memory 1002, and processor 1003 realize communication with each other through communication bus 1004. The communication interface 1001 is used for data communication between the computer device 1000 and an external device. The memory 1002 may be used to store software programs and modules, and the processor 1003 may execute the software programs and modules stored in the memory 1002, such as software programs for corresponding operations in the foregoing method embodiments.

Alternatively, the processor 1003 may call a software program and module stored in the memory 1002 to perform the following operations: acquiring an execution set, wherein the execution set comprises online operation data; based on the modified code, performing playback operation according to the online operation data to obtain a playback execution result; and generating a test report according to the playback execution result and the online operation data to complete the regression test of the modified code.

The present application also provides a computer-readable storage medium for storing a computer program. The computer readable storage medium may be applied to a computer device, and the computer program causes the computer device to execute a corresponding flow in the regression testing method in the embodiment of the present application, which is not described herein for brevity.

The present application also provides a computer program product comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the corresponding flow in the regression testing method in the embodiment of the present application, which is not described herein for brevity.

The present application also provides a computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the corresponding flow in the regression testing method in the embodiment of the present application, which is not described herein for brevity.

It should be appreciated that the processor of an embodiment of the present application may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The processor may be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

It will be appreciated that the memory in embodiments of the present application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server) to perform all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A regression testing method, comprising:

2. The regression testing method of claim 1, further comprising:

3. The regression testing method of claim 1, wherein performing a playback operation based on the modified code based on the on-line operation data to obtain a playback execution result comprises:

4. The regression testing method of claim 3, further comprising:

5. The regression testing method of claim 4 wherein the on-line operational data includes an on-line execution result and wherein the generating a test report from the execution result and the on-line operational data comprises:

and generating a test report according to the first comparison result.

6. The regression testing method of claim 5 wherein the method further comprises:

7. A regression testing apparatus, the apparatus comprising:

8. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded by a processor for performing the regression testing method according to any of the claims 1-6.

9. A computer device, characterized in that it comprises a processor and a memory, in which a computer program is stored, the processor being arranged to execute the regression testing method according to any of claims 1-6 by calling the computer program stored in the memory.

10. A computer program product comprising computer instructions which, when executed by a processor, implement the regression testing method of any one of claims 1-6.