JP5703195B2 - Differential comparison test system and test method for new and old versions of program - Google Patents

Description

  The present invention relates to a program test method, and more particularly, to a difference comparison test system and a test method by comparing differences in output results for new and old versions of a program.

  Conventionally, in a program test, a test method for finding a defect by comparing an output result of a program under test with a correct output result has been widely performed. However, when a program is revised or a bug is fixed, there are many cases where a bug other than the one that has been corrected occurs, and not only a new function or a test of the correction, but also an existing function is not adversely affected. It is indispensable to perform a so-called degradation test or a regression test to check whether or not.

  For this reason, a number of test methods have been devised for efficiently performing tests at the time of program revision. For example, in the test method described in Patent Document 1, the test operation procedure for the program under test and the output result of the program under test for the operation procedure are recorded, and the recorded test operation procedure is reproduced for the same program under test. Then, the difference between the output result and the recorded output result is taken and set as comparison exclusion data. Then, after revising the program under test, the previously recorded test operation procedure is reproduced for the revised program under test, the difference between the output result and the recorded output result is taken, and the comparison exclusion data is further added. It is characterized by outputting a difference that does not exist as a true difference.

  Further, in the test method described in Patent Document 2, a data input operation on a web page is detected and recorded in an operation content file, and a program code is set according to the operation content file, so that the execution is performed on the web page. A test program for reproducing the series of operations is generated, and the web page after the test program is executed is recorded as test result data. Then, compare the previous test result data after the previous test program execution with the current test result data after the current test program execution, and output a comparison result image that identifies and displays the mismatched part as a result of the comparison. It is characterized by.

JP 2001-282578 A JP 2011-164783 A

  The test method described in Patent Document 1 is a difference comparison test method that compares the program execution results before and after the revision of the program and obtains a difference. The comparison source created in advance for the test program before the revision at the time of operation recording Compare the output results at the time of test execution with the master, and if there is a difference between the two, exclude it as a difference in specifications, and then perform a comparative test with the output result before the revision for the revised program. , Only the true difference excluding this comparison exclusion is output. However, test data is assumed to be input by a tester or the like, and there is a problem that it takes a lot of time to create test data corresponding to various variations. In addition, since the same unmodified program is tested twice and the difference caused by the difference in execution date is excluded from comparison, application programs that depend on the date and time (for example, maturity date and interest calculation) There is a problem that necessary input / output items may be included in the comparison exclusion.

  The test method described in Patent Document 2 is also a difference comparison test method, which compares the program output result with the output result on the screen and outputs the image of the inconsistent portion. It is necessary to store and compare many screen images, and there is a problem that a load is applied to the test system. In addition, there is a problem that it cannot be applied to a program in which an output result of a program is passed as input to another system as data on a database instead of on a screen.

  In testing a program, checking all the variations of various input parameters is an enormous amount of work, especially the regression test and degradation test to confirm the existing functions are both tedious but difficult. In many cases, it is difficult to carry out each version upgrade in consideration of time. If you purchase package software and incorporate it into the system, it is a black box except for input and output, and the relationship between functions and program modules is unclear. It is also necessary to confirm whether or not an unexpected failure has occurred.

  Therefore, in the present invention, in view of the above-described problems, various variations of test data are automatically created from a large number of actual environments where the program has been operated in order to greatly reduce the labor of creating test data. There is also provided a difference comparison test system and a test method capable of easily comparing the difference between the output results of the new and old versions (after revision and before revision) without saving the output result of the program as a screen image. For the purpose.

  In order to solve the above problems, the test system of the present invention provides the following solution.

  A test system that supports a test for a revised version of a program, in an actual environment in which the version before the revision of the program operates, an operation log accumulation DB that accumulates operation logs that record input and output via an operation screen; A test data generation unit that generates test data from the operation log accumulation DB using an old version input / output item table and a new version input / output item table respectively defining input / output items in the pre-revision and post-revision versions of the program The test data generated by the test data generation unit is input to each test environment to which the revised version of the program and the version before the revision are applied, and the test is automatically executed. Get From the test execution unit and the test result, the comparison item is extracted using the old version input / output item table and the new version input / output item table, and the difference between the old version test result and the new version test result is extracted and the difference comparison result And a difference comparison unit that outputs to the table.

  With this configuration, test data can be created at the time of program revision (version upgrade or release upgrade) from many actual environments in which the old version of the program (version before revision) is running. The operation log can be automatically created from the accumulated DB. At this time, input data for testing is created using an input / output item table that defines input / output items of the new version (revised version) and the old version of the program via the operation screen. In addition, the same test data generated is input to each test environment to which the old and new version programs are applied, and the test results are obtained. The test results of the new version are compared with the test results of the old version. The test comparison result is output on the difference comparison result table in which items to be compared are extracted again using the new and old version input / output item tables, and is used for analysis of the test results.

  By doing so, the test data including many variations in the real environment that were difficult to assume at the time of design becomes abundant, and the test coverage can be expanded. In addition, by making it easy to compare the differences between the test results of the old and new versions, the work of the regression test and the degradation test can be greatly eased.

  Further, the above test system may be configured to further include the following features. That is, in addition to the operation log by the operator in the actual environment, the test data generation unit further includes a new function operation log DB for recording an operation log of the new function by the test operator in the test environment, Test data for functions supported only in the new version of the program is generated from the log DB.

  In other words, in addition to the operation log by the operator in the actual environment, a new function operation log DB that records the operation log including the new function by the test operator (tester) in the test environment is provided. It can be generated by the same procedure as that from the operation log in the real environment.

  The difference comparison unit of the test system may be configured to further include the following features. That is, when the difference comparison unit executes test data that can be executed only in the new version of the program, the difference comparison unit is obtained by searching the test result in the new version and the new function comparison master DB created in advance. The corresponding data is compared and output to the difference comparison result table.

  By configuring in this way, when test data of a new function that is supported only in the new version and there is no test result of the old version, a new function comparison master DB created in advance is searched and the corresponding is performed from there. If the correct output result data is obtained, the new version test result is compared with the data, so that it can be output to the difference comparison result table in the same format as in the case of the test of the conventional function. In the test data generated based on the operation log in the real environment, if the test results of the old version and the new version with the same function are compared with the old and new, the new version will find a defect (degrade) However, features that are only supported in the new version cannot be compared with the test results of the previous version. Therefore, the test of a new function usually compares and compares the actual test result with the correct test result (referred to as a comparison master) that humans expect from the specification. In this test system, By creating the comparison master in advance in a format that is output by the program targeted by this test system, the comparison between the comparison master and the test result in the new version can be realized using the same difference comparison result table. .

  Further, the above test system may be configured to further include the following features. That is, the new version input / output item table is substituted by the old version input / output item table for the operation screen in which the input / output items are not changed from the old version of the program.

  By configuring in this way, items that do not change the input / output items in the old and new versions depending on the operation screen, do not create the new version acquisition output item table, but substitute the old version input / output items for the items. The same test operation can be performed with or without changes.

  Further, the test execution unit of the test system may be configured to further include the following features. That is, the test execution unit adjusts the program execution date and time in the test environment to the program execution date and time recorded in the operation log.

  With this configuration, the internal clock of the test environment is temporarily adjusted according to the execution date and time of the program recorded in the operation log, and the program execution date and time at the time of the test are matched to output according to the execution date and time. The same difference comparison method can be applied to a program whose result fluctuates without using a comparison exclusion means.

  The difference comparison unit of the test system may be configured to further include the following features. That is, the difference comparison unit forms one record for each transaction executed by the program via the operation screen, compares the test result of the new version and the test result of the old version in record units, and has a difference. Only the record pair is output to the difference comparison result table.

  By configuring in this way, only the difference between the input and output results of the old and new versions in the transaction unit, which is the processing unit of the program executed via the operation screen, is output to the difference comparison result table for comparison and analysis. Can be easy.

  The difference comparison unit of the test system may be configured to further include the following features. That is, the difference comparison unit compares the test result of the new version of the program with the test result of the old version in units of records, and provides an identification display means for an output item having a difference with respect to records corresponding to the old and new versions. It outputs to a difference comparison result table, It is characterized by the above-mentioned.

  By configuring in this way, the identification display means (flag, color, figure, etc.) that makes the difference noticeable for each output item that has a difference between the old and new is output to the difference comparison result table, so that the difference in the item unit can be found. It can be made easier.

  The test system of the present invention can also be regarded as an invention of the following test method, and has the same effects as the test system of the present invention.

  A test method for a revised version of a program, comprising: accumulating an operation log in which an input / output via an operation screen is recorded in an operation log accumulation DB in an actual environment in which the version before the revision of the program operates; Generating test data from the operation log accumulation DB using the old version input / output item table and the new version input / output item table respectively defining the input / output items in the version before and after the revision of the program; For each test environment to which the version after revision and the version before revision are applied, the test data generated in the step of generating the test data is input and the test is automatically executed, and the test result is obtained from each test environment. Take A comparison item is extracted from the test result using the old version input / output item table and the new version input / output item table, and a difference between the old version test result and the new version test result is extracted. And storing the data in a test method.

  According to the present invention, in order to greatly reduce the labor of creating test data, various variations of test data are automatically created from a large number of actual environments where the program has been used, and the output results of the program are displayed on the screen. It is possible to provide a difference comparison test system and a test method that can easily compare differences between output results of old and new versions of programs without having to store them as images.

It is a figure which shows the functional block of the test system 100 which concerns on embodiment of this invention. It is a figure which shows the outline of the operator operation screen, input / output item table, and operation log which concern on embodiment of this invention. It is a figure which shows the specific example of the operation log which concerns on embodiment of this invention. It is a figure which shows the specific example of the input / output item table which concerns on embodiment of this invention. It is a figure which shows the test execution procedure which concerns on embodiment of this invention. It is a figure which shows the new function test execution procedure which concerns on embodiment of this invention. It is a figure which shows the outline of the difference comparison result table which concerns on embodiment of this invention. It is a figure which shows the specific example of the difference comparison result table which concerns on embodiment of this invention.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention (hereinafter referred to as embodiments) will be described in detail with reference to the accompanying drawings. Note that the same number is assigned to the same element throughout the description of the embodiment.

  FIG. 1 is a diagram showing functional blocks of a test system 100 according to the embodiment of the present invention. As shown in the figure, a test system 100 according to this embodiment is a system that supports performing a program test automatically or semi-automatically. As a function execution unit, a test data generation unit 20 that generates test data from an operation log, a test A test execution unit 50 for inputting data into the program under test and a difference comparison unit 70 for comparing the output results of the old and new version programs and extracting the difference are included.

  In addition, the test system 100 has an operation log accumulation DB 10 (global log) in which operation logs 220 obtained from the real environment 200 of the program under test are accumulated as a data storage unit, and a new function supported only by the new version as a test operator ( New function operation log DB 11 for recording logs operated by the tester), test data DB 40 for storing test data generated from the operation logs, and old and new version test result data DBs 90 and 91 for storing test results of old and new versions of the program under test , And a new function comparison master DB 92. The test system 100 includes old and new version input / output item tables 30 and 31 as other input data, and includes a difference comparison result table 80 as output data of the difference comparison unit 70. Hereinafter, the function of each functional block will be described in order.

  The real environment 200 indicates a system environment in which an old version of a program under test that has already been released is actually operating. The program to be tested is assumed to be an application program in which an operator in the actual environment inputs input data from the operation screen 210 and displays the output result on the screen or outputs it to the DB. Here, the operator includes not only a specialized terminal operator but also an individual customer who operates the terminal in an unmanned store or the like. More specifically, the actual environment 200 exists in a plurality of places such as stores and branches (hereinafter collectively referred to as bases) where the program under test can be operated. Each base has an operation log 220 in which the operator's operation procedure, input data, and output data are sequentially recorded. The contents of each operation log 220 are accumulated in the operation log accumulation DB 10 of the test system 100 at a predetermined timing.

  The test data generation unit 20 analyzes the contents of the operation log from the operation log accumulation DB 10, processes it into test data (test file) that can be executed by the program under test, and stores it in the test data DB 40. In this processing, an old version input / output item table 30 prepared in advance is referred to. The operation log includes input items and output items, but only input items are required as test data. The input / output item table is a table in which input items (input fields) and output items (output fields) are defined for each operation screen of the program under test, and is usually created at the time of screen design.

  In addition, the test data generation unit 20 can generate test data that can be executed only in the new version from the new function operation log DB 11 that records screen operations of functions supported only in the new version. In this case as well, the format (format) of the generated test data is the same. A specific example of the test data will be described later.

  If the input / output item is changed when the program under test is upgraded, a new version input / output item table 31 is prepared. The new version I / O item table 31 adds I / O items and extended parameters corresponding to the function (New Function) newly supported in this version upgrade, and functions that are no longer supported in the new version ( It is created from the old version input / output item table 30 by deleting input / output items corresponding to (Discontinue Function). Details of the input / output item table will be described later.

  The test execution unit 50 performs the function of reading the test data stored in the test data DB 40 and inputting the test data into the execution environments 60 and 61 of the old and new program under test to execute the program under test. The test execution unit 50 also functions to store test results obtained by inputting test data from the new and old version application program execution environment 60 and 61 in the old and new version test result data DBs 90 and 91, respectively. That is, the test execution unit 40 performs a function of automatically executing a test. Such a test execution means is realized by a commercially available tool or the like (for example, QTP of HP). Of course, the format of the test data may be arbitrary as long as it is supported by a commercially available tool. However, since the program under test in the implementation of the present invention is a financial program, OFS (described later) It uses a test data format called an Open Financial Service file.

  The difference comparison unit 70 reads the test result data from the old and new version test result data DBs 90 and 91 in which the test results of the old and new versions are accumulated, and the test result data of the old and new versions are converted into transaction processing units which are units of program processing. In the comparison, only records that are different between the test results of the old and new versions are output to the difference comparison result table 80 together with the input / output data. Here, the input / output item table of the old and new versions is referred to in creating the difference comparison result table. In this case, it goes without saying that not only input items but also output item columns are referred to. As will be described in detail later, the difference comparison result table 80 is output in a general table format so that it can be easily analyzed by the test manager and the person in charge of debugging. Since new functions supported only in the new version do not work in the old version, the new version test result data DB 91 and the new function comparison master DB 92 created in advance by the test manager based on the specifications are compared, and the difference comparison is similarly performed. A result table 80 is output.

The configuration of the functional block described above is merely an example, and one functional unit may be further divided, or a plurality of functional units may be configured as one functional unit. Each functional unit has a CPU (Central Processing Unit) built in the device and a ROM (Read Only
A computer program stored in a storage device such as a memory or a hard disk is read, and the computer program executed by the CPU is stored in a database (DB; Data Base) stored in the storage device or a storage area on the memory. This is realized by reading / writing necessary data and controlling related hardware (for example, input / output device, display device, communication interface device) in some cases. In addition, the database (DB) in the embodiment of the present invention may be a commercial database, but it simply means a collection of tables and files, and the internal structure of the database itself does not matter.

  FIG. 2 is a diagram showing an operator operation screen, an input / output item table, and an operation log according to the embodiment of the present invention. The operator operation screen 210 is one of user interfaces for the program to be tested, and is one or a plurality of screens that display user input and output results from the program for each item. As shown in the figure, the operator operation screen 210 displays a plurality of input items input by the operator, one or more output items as the execution result of the program, and one or more operation buttons operated by the operator. In addition, input items (items that cannot be input) that are automatically displayed without an operator input are also displayed. For example, the customer name displayed when the current date and time, the screen ID, and the customer number are entered are items that cannot be entered.

  The old and new version input / output item tables 30 and 31 have item IDs, value ranges (when possible values are limited), and remarks (description text of input / output items) for each input / output item included in each screen. , Input / output (a distinction between input and output) and display / non-display (a distinction as to whether or not to be displayed on the screen) are summarized in a table format. The distinction between display and non-display is provided because input / output items do not necessarily need to be displayed on the operation screen, but are necessary as input / output data of the program under test. However, items that are displayed on the screen but are not relevant as input / output data of the program under test may be omitted from the input / output item table. For example, when a program other than the program under test is called from the same operation screen for displaying help, displaying the current time, or logging in, the password, data necessary for authentication, and the like. As described above, when there are input / output items added / changed in the new version of the program under test, the new version input / output item table 31 is created by converting the old version input / output item table 30 in advance. deep. If there is a change in the value range (parameter range) even if there is no addition / change in the input / output items, the new version input / output item table 31 is created. When there is no addition / change in the old and new versions, the same old version input / output item table 30 is used instead for the screen ID.

  In the operation log stored in the operation log accumulation DB 10, the program name, version information, program execution date and time, base ID (operation screen input location ID), and screen ID of the used program under test are stored in the control unit 11 for each transaction. (Header part), the data part 12 records an input / output item ID (field ID) and a value for an input or output item for each input / output item. In addition, accompanying information regarding the operation status such as operation start time, operation end time, operator ID and approver ID may be included.

  FIG. 3 is a diagram showing a specific example of the operation log according to the embodiment of the present invention. In this specific example, as shown in the figure, the operation log is a tag-format text file, and the program name, version name, transaction ID, screen ID as data of the control unit 11 starting from the program execution date and time for each transaction. Are recorded in the form specified before and after by tags of <application>, <version>, <transactionID>, and <windowName>, respectively. Each input / output item of the data part 12 is recorded in such a way that the item name is specified before and after by the <fieldName> tag, and the value input or output for the item is specified by the <value> tag. The The order in which the items are recorded is usually the order entered by the operator, but this need not necessarily be the case. Although not shown here, depending on the input / output items, <mutiValueNumer> and <subValueNumber> tags are defined after <fieldName> in preparation for the case where there are multiple values (Value). You can also specify the value for an input / output item and the number of its subvalues. In the case of a single value, <mutiValueNumer> is 1, and there is no <subValueNumber> tag at this time.

  In the following, a specific example of a program that handles loan lending in a customer service at a financial institution will be described as an application program to be tested. In the data section 12 of the operation log when this program is operated, specific input / output data processed by this program, such as loan amount, subject code, currency, customer number, maturity date, applicable interest rate, interest calculation method, etc. It is recorded. These input / output data include not only data displayed on the operation screen but also data used in internal processing.

  Test data is generated by the test data generation unit 20 of the test system 100 from the operation log accumulated in the operation log accumulation DB 10 or the new function operation log DB 11. At this time, only the data necessary for the test is extracted by referring to the old version input / output item table 30. That is, only items (field names and their values) designated as “input” on the screen in this input / output item table are used as test data to be input to the program under test. Specifically, an OFS (Open Financial Service) file is used as the format of the input and test data in the operation log. The OFS file is a text file that represents each input / output item and its value in the form of “field name”: m: s = “value”. Here, “m” and “s” are the values of “multiValueNumber” and “subValueNumber” described above. In the case of a single value, both “m” and “s” are 1. For example, if the value input in the “loan amount” field is 1,000,000, it is described in the OFS file as AMOUNT: 1: 1 = 1000,000. Of course, other file formats can be used for test data. The new function operation log DB 11 is created in the same format.

  The same test data generated in this way is input to the new and old version application program execution environments 60 and 61. However, when the test data of the old version is input to the new version as it is, it may be necessary to perform formal conversion depending on the design of the screen of the new version. In that case, the conversion describing the conversion rule Prepare a table separately.

  FIG. 4 is a diagram showing a specific example of the input / output item table according to the embodiment of the present invention. The input / output item table is a table in which all input items and output items processed on the operation screen are collected. As described above, the input / output items may be changed by the revision of the program. Therefore, the old version I / O item table 30 and the new version I / O item table 31 correspond to the version of the program before and after the revision. Exists.

  Taking a new loan acceptance screen as a specific example of one operation screen of the loan lending business program mentioned above, the old version input / output item table 30 is specifically as shown in the upper part of FIG. When the input / output items are changed due to the revision of the program, a new version input / output item table 31 is created, for example, as shown in the lower part of FIG. In this example, the currency “KHR” (31a) is added as the value (parameter) of the “CURRENCY” field, and the “ACCURUAL.PARAM” field (31b) is newly added to support new functions in the new version. It shows that. In the new version, not only such function expansion but also the input / output items of the old version may be deleted in the new version due to the consolidation of functions.

  Since the fields and parameters added in the new version do not appear on the operation log in the environment where the old version program is actually running, only the old version input / output item table 30 is used to create test data. It is done. However, the creation of the difference comparison result table requires new and old version input / output items.

  5 and 6 are diagrams showing a test execution procedure according to the embodiment of the present invention. FIG. 5 shows a basic test execution procedure, and FIG. 6 shows a new function test procedure. First, in step S10 in FIG. 5, an instruction from the test administrator is received as to whether there is a new function supported only in the new version. If there is a new function, the process proceeds to step S11 to perform a new function test. If this test is a regression test after bug correction that does not include a new function, step S11 is skipped. The new function test is a test for checking whether the function newly added in the new version is in conformity with the specification, which does not exist in the old version. In the new function test, there is no operation log in the real environment, so test data generated from the real environment cannot be created automatically, so it is not possible to compare the difference with the test result of the previous version. This is a test.

  Even in new function tests, test data can be automatically generated using operation logs. However, in this case, not the operation log of the actual environment, but the test operator performs the operation on the operation screen according to the predetermined test scenario in the test environment, and the operation content and the result are displayed in the operation log of the actual environment. Are recorded in the new function operation log DB 11 in the same format, the test data generation unit 20 can generate test data for the new function. The new version input / output item table 31 is used to generate test data. Thus, by making the format of the test data the same, the test execution unit 50 can input the new function test data and the old version test data into the test program execution environment without particularly distinguishing them. However, unlike the regression of existing functions, the results of new function tests in the new version do not exist in the old version, so they are stored in the new function comparison master DB 92 prepared in advance by the test administrator based on the specifications. Will be compared with the correct output.

  FIG. 6 shows the procedure for this new function test, but the basic flow is not much different from the comparison procedure with the previous version of FIG. Omitted.

  Returning to FIG. 5, the test data generation unit 20 of the test system 100 reads the old and new version input / output item tables 30 and 31 in step S12, but if there is no change in the input and output items in the old and new versions, the old version Only the input / output item table 30 is read. Further, in step S13, the test data generation unit 20 generates test data from the operation log accumulation DB 10. Here, as already described, a test data file of a predetermined format that can be automatically executed by a program such as an OFS file is generated.

  Next, in step S14, the test execution unit 50 of the test system 100 sets the program execution date and time in the test environment to match the program execution date and time recorded in the test data. This is because even if the input data is the same, the processing result may differ depending on the date and time when the program is executed. For example, not only the current date and time displayed on the operation screen etc., but also the calculation of loan maturity date and interest, etc., the calculation results will differ depending on the execution date (processing date) of the program. It is necessary to adjust the internal clock so that the execution date / time of the test program matches the execution date / time of the program. Of course, if the program is designed so that all the date and time data itself that affects the output result, such as the execution date and time of such a program, is handled as input data, the processing of step S14 can be skipped.

  Next, in step S15, the test execution unit 50 inputs the same test data generated by the test data generation unit 20 into the new and old version application program execution environments 60 and 61, respectively. In step S16, the output result of the program under test (output for the test data) is recorded in the new and old version test result data DBs 90 and 91.

  As a result, in step S17, the difference comparison unit 70 of the test system 100 reads out the new and old version test result data DBs 90 and 91, and creates the difference comparison result table 80. At this time, the old and new version input / output item tables 30 and 31 are referred to. The above steps S14 to S17 are repeated for all test data (for each screen and for each transaction) (step S18). From the difference comparison result table 80 obtained in this way, the test administrator can check whether there is a defect in the new version of the program.

  FIG. 7 is a diagram showing an outline of the difference comparison result table according to the embodiment of the present invention. As already described, the difference comparison result table 80 is a table in which the same test data is input to the old and new versions of the program under test, and the output result is compared with the input data in a list form.

  As shown in the figure, the input / output results of the old and new versions are displayed as a pair of two rows (86a, 86b, 86c...) In the table. One line (record) is a record created for each transaction (processing unit) of the program under test. The column 81b is called a “Diff flag”. For example, if there is a symbol “*”, it indicates that there is a difference between old and new records. Since the old and new comparisons are performed for all transactions, all the comparison results may be described in the difference comparison result table 80, whether there is a difference or there is no difference. May be described in the difference comparison result table 80.

  The “new and old only flag” in the column 82 indicates that a record corresponding to the transaction exists only in either the new version or the old version. For example, if test data for a new function that is only supported by the new version is accidentally input to the old version of the program, the old version of the program will generate an error and no output result will be obtained, so only the new version of the record exists. It will be. In this case, the “new and old only flag” becomes “NEW ONLY”, and this record is described in the table. Similarly, when test data of a function that was supported in the old version but not supported in the new version (Discontinue Functon) is input to the new version of the program, the “old and new only flag” is set to “OLD ONLY”. Only the records that become are listed in the table.

  The “new and old only flag” searches the new function comparison master DB 92 for a record corresponding to a record whose “new and old only flag” is “NEW ONLY”, and if there is a corresponding record, extracts it as a comparison target. To play a role. In addition, it is useful not only at the site operator but also when verifying new function test data created from the new function operation log DB 11 created by the test. That is, it can be used to check whether the input / output items included in the new function test data are correctly reflected in the difference comparison result table 80.

  A column 83 “transaction ID” is a column in which a transaction ID unique to each processing unit is written. Normally, new and old records exist for each transaction ID. A column 84 “new and old record” is a flag indicating whether the record line is a test result (NEW) of a new version program or an old version test result (OLD). When the comparison master is used as the comparison source, if the “new and old records” are “MASTER”, the record pairs are “NEW” and “MASTER”. The column 85 is all input items and output items defined for each screen, that is, all items defined in the old and new version input / output item tables 30 and 31. However, there is a “Diff flag” column in front of each output item, which indicates that the output result of the corresponding output item is different between old and new (or new and MASTER). For example, in the record of 86a, since the result of the output item 1 is different, the “Diff flag” immediately before the output item 1 is indicated by the symbol “*”. Thus, by providing a “Diff flag” for the entire record and for each output item, it is possible to easily find the difference between the old and new. Of course, instead of the “Diff flag” symbol, as the identification display means, the display color and size of cells and characters may be changed, or a graphic display may be added.

  FIG. 8 is a diagram showing a specific example of the difference comparison result table according to the embodiment of the present invention. The figure shows a part of the difference comparison result table 80 as a result of testing a loan interest calculation program as a specific example of a test target. As described in the previous figure, the record pairs 86a, 86b, 86c, 86d, and 86e, which are new and old comparison results, are shown. 86f indicates a record for which a result is obtained only in the old version, and 86g indicates a record for which a result is obtained only in the new version, that is, a record for which there is no test result for the version to be compared.

  In this figure, instead of the “Diff flag” corresponding to each output item, an output item in which the test results of the old and new versions are different in a thick square frame. That is, in the record pair 86a, the “principal amount”, “interest amount”, and “processing date” are different between the old and new, but due to the difference in the “processing date” of the loan schedule for some reason, The result of “interest amount” may be different. Further, in the record pair 86c, a difference in the calculation result of the interest amount occurs due to the difference in the “interest calculation days”, and in the record pair 86e, the calculation result of the “interest amount” differs depending on the difference in the “applied rate”. It is thought that. In the record pair 86g, the compared records are “NEW” and “MASTER”, which is a comparison result in the case of the new function test (actually, the comparison test with the old version and the new function test are , It ’s done separately, so it ’s not mixed).

  In this way, it is easy to compare the output result of the new version of the program under test with the output result of the proven old version of the program under test, for many variations of input data, so early detection of defects, It can be used for early resolution. In particular, when upgrading the version of a program, it can be used to find an unexpected defect of a conventional function that is difficult to find in a normal function test. It is also particularly useful when building a system by incorporating a black box program inside, such as package software.

  As mentioned above, although this invention was demonstrated using embodiment, it cannot be overemphasized that the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiments. Further, it is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

10 Operation log collection DB
11 New function operation log DB
20 Test Data Generation Unit 30 Old Version I / O Item Table 31 New Version I / O Item Table 40 Test Data DB
50 Test Execution Unit 60 Old Version Application Tested Program Execution Environment 61 New Version Application Tested Program Execution Environment 70 Difference Comparison Unit 80 Difference Comparison Result Table 90 Old Version Test Result Data DB
91 New version test result data DB
92 New Function Comparison Master DB
100 Test system 200 Real environment 210 Operator operation screen 220 Operation log

Claims (8)

A test system that supports testing against a revised version of a program,
In an actual environment where the pre-revision version of the program operates, an operation log accumulation DB that accumulates operation logs recording input / output via the operation screen;
A test data generation unit that generates test data from the operation log accumulation DB using an old version input / output item table and a new version input / output item table respectively defining input / output items in the pre-revision and post-revision versions of the program When,
The test data generated by the test data generation unit is input to each test environment to which the revised version of the program and the version before the revision are applied, and the test is automatically executed, and the test result is obtained from each of the test environments. A test execution unit to perform,
From the test result, the comparison item is extracted using the old version input / output item table and the new version input / output item table, the difference between the old version test result and the new version test result is extracted and output to the difference comparison result table A comparison unit;
A test system comprising: In addition to the operation log by the operator in the actual environment, the system further comprises a new function operation log DB for recording an operation log of the new function by the test operator in the test environment,
The test system according to claim 1, wherein the test data generation unit generates test data of a function supported only by the new version of the program from the new function operation log DB.   When the difference comparison unit executes test data that can be executed only in the new version of the program, the test result in the new version and the corresponding data obtained by searching the previously created new function comparison master DB Are output to the difference comparison result table. The test system according to claim 2.   2. The new version input / output item table is substituted by the old version input / output item table for the operation screen in which the input / output items are not changed from the old version of the program. Testing system.   The test system according to claim 1, wherein the test execution unit adjusts a program execution date and time in a test environment to a program execution date and time recorded in the operation log.   The difference comparison unit forms one record for each transaction executed by the program via the operation screen, compares the test result of the new version and the test result of the old version in record units, and records pairs having a difference The test system according to claim 1, wherein only is output to the difference comparison result table.   The difference comparison unit compares the test result of the new version of the program with the test result of the old version in units of the records, and outputs an identification display means for an output item having a difference with respect to a record corresponding to the old and new versions. The test system according to claim 6, wherein the test system outputs the result table. A test method for the revised version of the program,
A step of accumulating an operation log in which an input / output through an operation screen is recorded in an operation log accumulation DB in an actual environment in which a pre-revision version of the program is operating;
Generating test data from the operation log accumulation DB using an old version input / output item table and a new version input / output item table respectively defining input / output items in the pre-revision and post-revision versions of the program;
For each test environment to which the version after the revision of the program and the version before the revision are applied, the test data generated in the step of generating the test data is input and the test is automatically executed;
Obtaining test results from each of the test environments;
A step of extracting comparison items from the test results using the old version input / output item table and the new version input / output item table, extracting a difference between the old version test result and the new version test result, and storing the difference in the difference comparison result table When,
A test method characterized by comprising:

Images