JP2015011396A - Regression test support system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a regression test support system capable of inspecting degrade of a non-functional requirement.SOLUTION: The regression test support system for performing a regression test of an application comprises a test execution device 100 including: execution storage means 140 for recording data obtained by executing the regression test; and execution processing means 110 for executing and processing a program included in the application, recording the data related to a non-functional requirement other than a function achieved by the application to the execution storage means 140, and performing processing for recording data of a dump list related to the execution result.

Description

  The present invention relates to a system for supporting a regression test.

  For example, a regression test is performed in the development of a web application that provides a service using a web page as an interface. The regression test is a test for confirming whether an unexpected change has occurred in a function in a program other than the program to be changed when the program is changed by debugging, specification change, or the like. For example, in a regression test of a web application, after changing a web application or a program that configures the web application, whether or not functions such as the transition order of the web page and the display content of the web page match expected data. Compare and verify the validity (see, for example, Patent Document 1).

JP 2003-248597 A

  Here, if an application or a program (hereinafter referred to as an application) constituting the application is changed, the quality may be deteriorated (degraded) due to an unexpected influence. A typical event of degradation is, for example, that a function result is not output as expected even if an application is executed. Degradation may occur even for non-functional requirements such as performance degradation of program processing speed and (test) coverage (function execution coverage rate) decrease. For example, in a mission critical system, performance degradation impairs system availability, so performance requirements are strict for system applications. In addition, an application with low coverage may have an unnecessary program in the application, and such a program hinders the impact investigation and adversely affects the maintainability of the system.

  A regression test is effective in confirming the degradation, and efficiency has been improved by automating the regression test. However, it has been difficult to verify degradation such as performance and coverage.

  This is because it is necessary to write (embed) a description for verification in the program for each viewpoint. Since embedding is performed by changing the program contents, there is a risk of breaking the program. In addition, the efficiency was hindered by the effort involved in embedding.

  Therefore, it has been required to realize a regression test support system that can verify the degradation of non-functional requirements.

  Therefore, the regression test support system according to the present invention is included in the application based on the test storage and execution storage means for recording data obtained by performing the regression test in the regression test support system for performing the regression test of the application. A test execution apparatus comprising: an execution processing unit that executes a program, records data related to non-functional requirements other than functions realized by the application in an execution storage unit, and records data of a dump list related to the execution result It is what has.

  According to the present invention, since the data related to the non-functional requirements is recorded when the regression test is performed, the degradation can be efficiently confirmed. At this time, we created a test application that incorporated the aspect described in the aspect-oriented program into the source code of the application and performed a regression test. Therefore, after creating an application that did not include the functions necessary only for testing With this, it is possible to perform a regression test that allows more advanced verification. In addition, the application can be secured and protected.

It is a figure which shows the structure of the regression test assistance system which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the test execution apparatus 100 which concerns on Embodiment 1 of this invention. It is a figure explaining the application for a test concerning Embodiment 1 of the present invention. It is a figure which shows the structure of the comparison verification apparatus 200 which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the coverage analysis processing apparatus 300 which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the process which the coverage analysis process means 310 which concerns on Embodiment 1 of this invention performs. It is a figure which shows the structure of the performance verification apparatus 400 which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the process which the performance verification process means 410 which concerns on Embodiment 1 of this invention performs.

Embodiment 1 FIG.
FIG. 1 is a diagram showing a configuration of a regression test support system according to Embodiment 1 of the present invention. The system in the present embodiment includes a test execution device 100, a comparison verification device 200, a coverage analysis processing device 300, and a performance verification device 400.

  The test execution device 100 is a device that executes a regression test of an application to be tested. Here, in the test execution apparatus 100 according to the present embodiment, a test application in which an aspect is incorporated in the application is executed. Aspect weaving will be described later. The comparison verification device 200 compares the dump list output as the execution result of the test execution device 100 with the dump list in the regression test executed last time. And verification processing, such as whether the assumed result is obtained, is performed.

  The coverage analysis processing device 300 analyzes the coverage log, which is data output together with the dump list when the test execution device 100 executes the regression test. The performance verification device 400 performs a verification process on a performance log, which is data output together with a dump list when the test execution device 100 executes a regression test. Here, the performance is assumed to be processing time (performance) related to execution of an application in the present embodiment. In the verification process, for example, a performance log related to the previous test execution is compared with a performance log related to the current test execution. As described above, in the present embodiment, analysis, verification, and the like related to coverage and processing time among non-functional requirements will be described.

  FIG. 2 is a diagram showing a configuration of the test execution apparatus 100 according to Embodiment 1 of the present invention. 2, the test execution apparatus 100 includes an execution processing unit 110, an execution input unit 120, an execution display unit 130, and an execution storage unit 140.

  The execution processing unit 110 executes a test application (program) stored in the application storage unit 143. Then, the dump list that is the result of execution processing based on the input data stored in the input data storage unit 141 and the test data stored in the test data storage unit 142 is stored in the dump list storage unit 144. In the execution process, the recorded coverage log and performance log are stored in the coverage log storage unit 145 and the performance log storage unit 146. Further, the execution input unit 120 inputs an instruction of the apparatus user. The execution display unit 130 displays an execution result or the like to the apparatus user.

  The execution storage unit 140 stores data necessary for the execution process performed by the execution processing unit 110, data obtained as a result of the execution process, and the like. The input data storage unit 141 stores input data. Input data is a parameter. The test data storage unit 142 stores test data. Test data is data related to a test when a regression test is performed. The application storage unit 143 stores the test target application. The dump list storage unit 144 stores a dump list as an execution result as data. The coverage log storage unit 145 stores the coverage log when the application is executed as data. The performance log storage unit 146 stores the performance log as data. The coverage log and performance log will be described later.

  FIG. 3 is a diagram illustrating the test application according to Embodiment 1 of the present invention. As described above, in this embodiment, a test application is created from an application and a regression test is executed. In the present embodiment, a test application is created by compiling an aspect into an application source file.

  Here, an aspect is a program module described based on the idea of aspect orientation. In aspect-oriented programming, common processes performed in a distributed manner in an application (program) are collectively used as an aspect and combined (woven) during compilation or execution to construct an application.

  For example, in object-oriented programming, a set of related data and a procedure (method) associated therewith are grouped into a module called a class. Then, an application (program) is configured by combining a plurality of classes. For this reason, even if there is a process common to a plurality of classes, it is necessary to describe in a distributed manner in each class (method).

  On the other hand, in aspect-oriented programming, common processing is described as an aspect separately from a class, and a class (method) that interweaves aspect processing is specified from the aspect processing side. This eliminates the need to describe common processing in multiple classes.

  For example, in this embodiment, by compiling an aspect described as an aspect-oriented program together with a source file of an application, a process that records when a method is called in each class and a process that records a method start time and an end time. Create a test application. Therefore, it is possible to create a test application that incorporates processing related to recording without directly changing the contents of the application source file. In the regression test, processing related to recording can be executed.

  In addition, processing (functions) common to a plurality of classes (required across the classes) can be described in a collective form. Since it is not necessary to describe for each class, it is possible to prevent omission of description, excessive description, erroneous description, application destruction, and the like. Since the process related to the recording is not executed except during the regression test, it is not necessary to describe the process related to the recording in the actual application, so that the program can be shortened.

  Furthermore, it is possible to easily change the recorded contents. Further, for example, in this embodiment, time is recorded, but for example, JAVA (registered trademark) can be recorded in nanosecond order. For example, when detailed time measurement is required, by describing the description in the aspect so as to be recorded in the order of nanoseconds, analysis and verification can be performed more effectively and easily.

  FIG. 4 is a diagram showing a configuration of the comparison verification apparatus 200 according to Embodiment 1 of the present invention. 4, the comparison verification apparatus 200 includes a comparison processing unit 210, a comparison input unit 220, a comparison display unit 230, and a comparison storage unit 240.

  The comparison input unit 220 sends a user instruction or the like as a signal to the comparison processing unit 210. Further, the comparison display means 230 displays the results processed by the comparison processing means 210 to the user. The comparison storage unit 240 temporarily or long-term stores data necessary for the comparison processing unit 210 to perform processing.

  Here, it is assumed that the comparison storage unit 240 of the present embodiment has an assumed difference storage unit 241 in particular. The assumed difference storage unit 241 stores, as excluded data, data (for example, text data) that defines items (difference items) that are assumed that the dump list of the current regression test is different from the dump list of the previous regression test. To do. For example, if the contents of the application (program) are changed from the previous regression test, the result of executing the regression test may be different. Such a difference in results is a difference that occurs within a range that is assumed in advance, and may not be necessary for the validity verification of the test results. Thus, in the difference detection of the dump list, the exclusion data stored in the assumed difference storage unit 241 is excluded without being detected (not displayed as a difference on the comparison display unit 230). For this reason, verification efficiency can be improved, for example, by reducing the amount of difference related to detection.

  The comparison processing unit 210 performs a process of comparing the contents of the dump list in the previous regression test with the contents of the dump list in the current regression test to detect a difference, and causing the comparison display unit 230 to display the difference. Here, in the comparison process, the difference stored as the excluded data in the assumed difference storage unit 241 is not detected.

  FIG. 5 is a diagram showing a configuration of the coverage analysis processing apparatus 300 according to Embodiment 1 of the present invention. In FIG. 5, the coverage analysis processing apparatus 300 includes a coverage analysis processing unit 310, a coverage analysis input unit 320, a coverage analysis display unit 330, and a coverage analysis storage unit 340.

  The coverage analysis input unit 320 sends a user instruction or the like as a signal to the coverage analysis processing unit 310. Further, the coverage analysis display means 330 displays the results processed by the coverage analysis processing means 310 to the user. The coverage analysis storage unit 340 stores data necessary for the coverage analysis processing unit 310 to perform processing temporarily or in a long term.

  FIG. 6 is a diagram for explaining processing performed by the coverage analysis processing unit 310 according to Embodiment 1 of the present invention. The coverage analysis processing unit 310 performs an analysis process based on a coverage log or the like. Here, the coverage analysis processing unit 310 includes a source file analysis unit 311, a coverage log analysis unit 312, and a coverage calculation unit 313. Here, the function of each unit may be provided as a program, and the function performed by each unit may be realized by the processing means being hardware executing the program.

  The source file analysis unit 311 performs a process of counting the number of methods (number of executable methods) of each class based on the application source file. Further, the coverage log analysis unit 312 counts the number of methods related to the call of each class (the number of methods actually called and executed) based on, for example, the coverage log stored in the coverage log storage unit 145 of the test execution apparatus 100. I do.

The coverage calculation unit 313 calculates the coverage of each class in the regression test performed by the test execution apparatus 100 and displays the coverage on the coverage analysis display unit 330 together with the number of methods. Here, in the present embodiment, the coverage calculation unit 313 calculates S1 coverage. As shown in the following equation (1), the S1 coverage is a percentage of the number of methods actually called and executed with respect to the number of executable methods.
S1 coverage =
Number of methods actually called and executed / number of executable methods × 100 (1)

  As described above, the user can obtain coverage in the regression test. For example, if the coverage in the current regression test is significantly lower than the coverage in the previous regression test, the possibility of unnecessary functions being latent or unexpected effects may have been caused by changing the application Can be considered. Here, although not particularly limited, a reference value related to coverage (for example, 98%) is determined in advance, and if the coverage analysis processing unit 310 determines that the reference value is below the reference value, the user is notified. You may make it perform.

  FIG. 7 is a diagram showing a configuration of the performance verification apparatus 400 according to Embodiment 1 of the present invention. 7, the performance verification apparatus 400 includes a performance verification processing unit 410, a performance verification input unit 420, a performance verification display unit 430, and a performance verification storage unit 440.

  The performance verification input unit 420 sends a user instruction or the like as a signal to the performance verification processing unit 410. Further, the performance verification display unit 430 displays the result processed by the performance verification processing unit 410 to the user. The performance verification storage unit 440 temporarily or long-term stores data necessary for the performance verification processing unit 410 to perform processing.

  FIG. 8 is a diagram for explaining processing performed by the performance verification processing unit 410 according to Embodiment 1 of the present invention. The performance verification processing unit 410 includes a time calculation processing unit 411 and a time comparison processing unit 412. Here, the function of each unit may be provided as a program, and the function performed by each unit may be realized by the processing means being hardware executing the program.

  The time calculation processing unit 411 calculates the processing time for each method based on the start time and end time of the method recorded as a performance log in the performance log storage unit 146 of the execution storage unit 140. Here, the time calculation processing unit 411 calculates the processing time for the current regression test execution and the previous regression test execution. Further, as described above, the processing time can be calculated on the order of nanoseconds. Then, the time comparison processing unit 412 calculates the time difference between the processing time of the current test execution and the previous test execution for each method based on the processing time calculated by the time calculation processing unit 411, and performs performance verification. It is displayed on the display means 430.

  As described above, in the regression test support system according to the present embodiment, when the regression test is executed in the test execution device 100, the coverage log and the performance log, which are non-functional requirements, are respectively stored in the coverage log storage unit 145 and the performance log. Since it is recorded in the log storage unit 146, the degradation can be confirmed efficiently. At this time, we created a test application that incorporates the aspect described in the aspect-oriented program into the application source code and executed the regression test, so we created an application that does not include the functions required only for testing. Above, you can do a more advanced regression test.

  In addition, since the coverage analysis processing device 300 analyzes the coverage log and calculates the coverage, for example, compares it with the previous coverage, detection of a program that has become unnecessary due to a program change error or change in the application Etc. can be supported. In the performance verification apparatus 400, the performance log is verified, the processing time is calculated, and, for example, compared with the previous processing time, performance degradation can be supported.

Embodiment 2. FIG.
In the above-described embodiment, each device is divided. However, for example, one or a plurality of devices may be configured by one device.

  In the first embodiment described above, coverage and processing time (performance) have been described as non-functional requirements. However, the present invention is not limited to this. Other non-functional requirements include capacity (memory usage). If a program whose memory is not properly managed is executed, the memory may not be released successfully. For this reason, a memory leak may occur. Furthermore, if the amount of memory required to execute the program is insufficient, the execution of the program may be interrupted. Therefore, for example, when an instruction that is likely to cause a memory leak is executed, a log relating to the use of the memory can be recorded for verification.

100 Test Execution Device 110 Execution Processing Unit 120 Execution Input Unit 130 Execution Display Unit 140 Execution Storage Unit 141 Input Data Storage Unit 142 Test Data Storage Unit 143 Application Storage Unit 144 Dump List Storage Unit 145 Coverage Log Storage Unit 146 Performance Log Storage Unit 200 Comparison verification apparatus 210 Comparison processing means 220 Comparison input means 230 Comparison display means 240 Comparison storage means 241 Assumed difference storage section 300 Coverage analysis processing apparatus 310 Coverage analysis processing means 311 Source file analysis section 312 Coverage log analysis section 313 Coverage calculation section 320 Coverage Analysis input means 330 Coverage analysis display means 340 Coverage analysis storage means 400 Performance verification device 410 Performance verification processing means 411 Time calculation processing unit 412 Comparison processing unit 420 commissioning input means 430 commissioning display unit 440 commissioning storage means

Claims (7)

In a regression test support system that performs application regression tests,
Execution storage means for recording data obtained by performing the regression test;
Based on the test data, the program included in the application is executed, data related to non-functional requirements other than the functions realized by the application is recorded in the execution storage means, and dump list data related to the execution result is recorded A regression test support system comprising: a test execution device including execution processing means for performing processing.   A test application incorporating an aspect describing execution of a function for recording the non-functional requirement by an aspect-oriented program is created based on the application, and the regression test is performed by the test application. The regression test support system according to claim 1.   The regression test support system according to claim 1, wherein the non-functional requirement is the number of programs executed in the application. Coverage analysis processing means for calculating a coverage which is a ratio of the number of executed programs to the number of programs in the application;
The regression test support system according to claim 3, further comprising a coverage analysis processing device including coverage analysis display means for displaying the coverage calculated by the coverage analysis processing means.   The regression test support system according to claim 1, wherein the non-functional requirement is a processing time of the program in the application. Performance verification processing means for calculating a time difference between the processing time of the program in the previous regression test and the processing time of the program in the current regression test;
6. The regression test support system according to claim 5, further comprising a performance verification device including performance verification display means for displaying the time difference calculated by the performance verification processing means. Comparison processing means for detecting a difference between the data of the dump list in the previous regression test and the data of the dump list in the current regression test;
Comparison display means for displaying the difference detected by the comparison processing means;
Comparing storage means for storing in advance the items assumed as the difference as excluded data,
The regression test support system according to claim 1, wherein the comparison processing unit further includes a comparison verification device that performs processing so as not to detect the difference based on the exclusion data.

Images