JP2009163636A - Consistency checking method, apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a consistency checking method, apparatus, and program for determining consistency between codes using a new code coverage instead of an existing code coverage, e.g. MC/DC. <P>SOLUTION: Test data is stored in a storage part. The test data stored in the storage part is executed with respect to new and old codes, respectively. Operator coverage information for calculating the coverage of an operator described in the executed code is measured with respect to the new and old codes, respectively. Constant parameter coverage information for calculating the coverage of a constant described in the executed code is measured with respect to the new and old codes, respectively. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a consistency check method, a consistency check apparatus, and a program, and more particularly to a consistency check method, a consistency check apparatus, and a program for evaluating code coverage of old and new codes before and after code change.

  Conventionally, in the field of software, a considerable amount of time has been spent on debugging tests, and various debugging test techniques have been developed.

  For example, the technique of Patent Document 1 stores code coverage information in a code coverage database, and uses this code coverage database to delete or comment out non-executable code descriptions. The design method and design apparatus of an integrated circuit device for performing the code optimization.

  The technology of Patent Document 2 prepares a patch code corresponding to a breakpoint of the first program instruction in a host having both functions and capacity, and executes it by replacing it with a patch code for identifying the breakpoint at the time of execution. A code coverage measurement method and apparatus for an embedded processor system that determines whether software is good or bad compared with expected results.

  The technique of Patent Document 3 is a method and system for verifying software functions for a control unit, which includes the use of a simulation model and a control unit for simulating software functions.

  The technique of Patent Document 4 is an in-vehicle ECU inspection device that performs a pass / fail determination by inputting a pseudo signal to an ECU to be inspected and evaluating its output, and an ECU corresponding to the ECU type by replacing an optional unit of the inspection device. The inspection is to be performed.

  Moreover, the technique of patent document 5 is a coverage acquisition system provided with the means which calculates the ratio of the number of test passing lines with respect to the total number of lines of a source code, and produces | generates weighted coverage information.

  The technique of Non-Patent Document 1 detects an untested portion in the source code using an industry standard coverage measurement standard called MC / DC (Modified Condition / Decision Coverage).

  In the technique of Patent Document 6, when a system LSI is inspected using test data created at random, how much random data satisfies the inspection items by using a coverage checker based on the MC / DC. Is a system verification apparatus and verification method that uses coverage as an indicator of test integrity.

  Thus, when the software structure is changed, it is necessary to check whether the functions (input / output relationships) before and after the change match. Typically, there is a method of executing prepared test data on the code before and after the change and comparing the results as in Patent Document 6 above, and the sufficiency (accuracy) of the test is It is determined from the code coverage information (coverage rate) (see FIG. 1).

  Here, FIG. 1 is a data flow diagram showing an example of a consistency check method based on a conventional code coverage (for example, MC / DC) as shown in Patent Document 6 above. In FIG. 1, the test data is executed for each new and old code, and as a result, the code output difference is “0”, so it is determined that “new and old code outputs match”. In addition, in order to evaluate the accuracy of this test, coverage measurement is performed for each new and old code using the MC / DC, and as a result, the coverage rate of each code becomes “100%”. It is judged as “high”.

JP 2004-355130 A JP 2007-004793 A JP 2005-504377 A JP-A 63-168837 JP 2006-1550920 A JP 2004-086838 A Kelly J., et al. Hayhurst et al., “A Practical Turbial On Modified Condition / Decision Coverage”, NASA / TM-2001-210876, May 2001.

  However, with conventional code coverage (for example, MC / DC described in Non-Patent Document 1 and Patent Document 6), code changes (difference between new code and old code) may not be detected, and stricter code coverage Needed to be defined.

  Here, the conventional code coverage problem will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of conventional code coverage (for example, MC / DC: Modified Condition / Decision Coverage) and test data that satisfies 100% of the MC / DC. Here, as described above, “MC / DC” is an industry standard coverage metric used to detect an untested portion in the source code, and is used for a software debug test.

  In FIG. 2, in the conventional code coverage (for example, MC / DC), the code of (a) before the change of FIG. 2 and the code of (b) after the change have completely different functions, but the output values (O1 and O2) are different. Since they match, it is determined that these codes match 100%. In other words, when MC / DC is used for a code coverage test, the output may match even though there is a difference in the function in the code depending on the test data, and there is a risk of judging that the mismatch is a match. Has the problem.

  The present invention has been made in view of the above, and it is possible to clearly distinguish between a match and a mismatch between codes by using a new code coverage in place of the conventional code coverage (for example, MC / DC). An object of the present invention is to provide a sex checking method, a consistency checking device, and a program.

  In order to achieve such an object, the consistency check method according to claim 1 is a consistency check including at least a storage unit and a control unit for evaluating code coverage of old code and new code before and after code change. A consistency check method executed in the apparatus, wherein test data executed in the control unit is stored in the storage unit in a test data storage step for storing the test data in the storage unit, and the test data storage step. In addition, an execution step for executing the test data for each new and old code, and operator coverage information for calculating the coverage rate of the operator described in the code executed in the execution step are newly・ Operator coverage measurement step for each old code and the constants described in the above code executed in the execution step The constant parameters coverage information for calculating the ledge rate, characterized in that it comprises a constant parameter coverage measurement step of measuring at another new and old code, the.

  The consistency check method according to claim 2 is the consistency check method according to claim 1, wherein the operator coverage information measured in the operator coverage measurement step is executed in the control unit. And a test evaluation step for evaluating the accuracy of the test data for each new and old code based on the constant parameter coverage information measured in the constant parameter coverage measurement step and storing the accuracy in the storage unit. It is characterized by.

  According to a third aspect of the present invention, there is provided the consistency check method according to the second aspect, wherein the test evaluation step includes the test data that has been newly evaluated for accuracy and the storage unit that has already been evaluated. The stored test data is compared with each other, and the test data having the higher accuracy of both test data is stored in the storage unit by new and old codes.

  According to a fourth aspect of the present invention, there is provided the consistency check method according to the first aspect, wherein the consistency check apparatus further includes a display unit and is executed in the control unit. The output difference between the old code and the new code executed in step 1, the operator coverage information measured in the operator coverage measurement step, and / or the constant measured in the constant parameter coverage measurement step The method further includes a display step of displaying the parameter coverage information on the display unit.

  Further, the consistency check device according to claim 5 is a consistency check device including at least a storage unit and a control unit for evaluating code coverage of the old code and the new code before and after the code change. The unit includes test data storage means for storing test data, and the control section executes the test data stored in the storage section by the test data storage means for each new and old code, and The operator coverage measurement means for measuring the operator coverage information for calculating the coverage rate of the operator described in the code executed by the execution means for each new and old code, and executed by the execution means. Constants that measure constant parameter coverage information for calculating the coverage rate of the constants described in the above code for each new and old code And La meter coverage measurement means, and further comprising a.

  The consistency check apparatus according to claim 6 is the consistency check apparatus according to claim 5, wherein the control unit includes the operator coverage information measured by the operator coverage measurement means and the constant. The apparatus further comprises test evaluation means for evaluating the accuracy of the test data for each new and old code based on the constant parameter coverage information measured by the parameter coverage measurement means and storing the accuracy in the storage unit. .

  According to a seventh aspect of the present invention, there is provided the consistency check apparatus according to the sixth aspect, wherein the test evaluation unit includes the test data that has been newly evaluated for accuracy and the storage unit that has already been evaluated. The stored test data is compared with each other, and the test data having the higher accuracy of both test data is stored in the storage unit by new and old codes.

  The consistency check apparatus according to claim 8 is the consistency check apparatus according to claim 5, wherein the consistency check apparatus further includes a display unit, and the control unit is executed by the execution unit. Output difference between the old code and the new code, the operator coverage information measured by the operator coverage measuring means, and / or the constant parameter coverage information measured by the constant parameter coverage measuring means, The display device further includes display means for displaying on the display unit.

  A program according to claim 9 is a program for evaluating a code coverage of an old code and a new code before and after a code change, and for causing a consistency check apparatus including at least a storage unit and a control unit to execute the program. A test data storage step for storing the test data in the storage unit, which is executed in the control unit, and the test data stored in the storage unit in the test data storage step is executed for each new and old code. And an operator coverage measurement step for measuring operator coverage information for calculating the coverage rate of the operator described in the code, which is executed in the execution step, for each new and old code. The constant parameter for calculating the coverage rate of the constant described in the code executed in the execution step. The meter coverage information, and constant parameters coverage measurement step of measuring at another new and old codes, characterized in that to the execution.

  The consistency check method according to claim 10 is executed in a consistency check apparatus including at least a storage unit and a control unit that evaluates code coverage while checking the match of output values of a plurality of codes. A test data storage step for storing a plurality of test data in the storage unit, and the test stored in the storage unit in the test data storage step. An operator that measures the coverage rate of an operator in the code that has been executed a plurality of predetermined times in the execution step as operator coverage information by causing the plurality of codes to execute data as input values The predetermined plurality of output values output by the code executed in the coverage measurement step and the execution step. Ri, the coverage rate constants constants that described in the operator is uniquely be estimated, characterized in that it comprises a constant parameter coverage measurement step of measuring as a constant parameter coverage information.

  The consistency check method according to claim 11 is the consistency check method according to claim 10, wherein the operator coverage information measured in the operator coverage measurement step is executed in the control unit. Or a test evaluation step for storing the test data for increasing the coverage rate in the storage unit based on the constant parameter coverage information measured in the constant parameter coverage measurement step. To do.

  The consistency check method according to claim 12 is the consistency check method according to claim 11, wherein the operator coverage measurement step includes the code in the operator executed in the execution step. Line information is stored in the operator coverage information, and the constant parameter coverage measurement step includes the line information in the code of the constant estimated from the output value output by the code in the execution step. Stored in the constant parameter coverage information, and the test evaluation step includes the line information corresponding to the test data stored in the storage unit and the line corresponding to the test data newly executed in the execution step. By comparing with the information, it is determined whether or not the test data increases the coverage rate, and increases The test data in association with the line information be stored in the storage unit when it is determined, characterized by.

  According to the present invention, test data is stored in the storage unit, the test data stored in the storage unit is executed for each new and old code, and the coverage rate of the operator described in the executed code is calculated. Operator coverage information for the new and old code is measured, and constant parameter coverage information for calculating the coverage rate of the constant described in the executed code is measured for the new and old code. , Based on “operator coverage information” and “constant parameter coverage information”, the test data satisfying 100% in the conventional code coverage (for example, MC / DC) described above with reference to FIG. 1 cannot be distinguished. There is an effect that the difference between the codes can be detected more accurately.

  Further, according to the present invention, the accuracy of the test data is evaluated for each new and old code based on the measured operator coverage information and constant parameter coverage information and stored in the storage unit. When multiple test data are used for children and constants, redundant tests can be removed and a simple test database for new and old code can be created.

  Further, according to the present invention, the test data newly evaluated for accuracy is compared with the past test data already stored in the storage unit, and the test data with the higher accuracy of both test data is newly updated. -Since the old code is stored in the storage unit, if the operator coverage rate and the constant parameter coverage rate become 100%, the test data that has reached 100% will be used when the next new / old code is tested. Since it is possible to perform a debug test using, after obtaining highly accurate test data, the measurement of operator coverage information and constant parameter coverage information can be omitted, and the entire debug test can be simplified. Play.

  Further, according to the present invention, the consistency check apparatus further includes a display unit, and outputs an output difference between the executed old code and the new code, measured operator coverage information, and / or constant parameter coverage information. Since it is displayed on the display unit, there is an effect that the coverage rates for the operators and constants of the new and old codes can be easily compared visually.

  Hereinafter, embodiments of a consistency checking apparatus and a consistency checking method according to the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

[Outline of the present invention]
Hereinafter, the outline of the present invention will be described, and then the configuration and processing of the present invention will be described in detail. FIG. 3 is a flowchart showing an example of a basic operation process of the consistency checking device according to the present invention.

  The present invention generally has the following basic features. That is, according to the present invention, first, test data is stored in the storage unit.

  Next, the present invention executes the test data stored in the storage unit for each new and old code (SA-1).

  And this invention measures the operator coverage information for calculating the coverage rate of the operator described in the executed code for each new and old code (SA-2). Here, “Operator Coverage” (OC: Operator Coverage) is a new metric for coverage testing. As a metric for operator coverage rate, each input independently affects the output. Used to confirm. Since the number of inputs in the code and the form of calculation are known, they are defined by operators.

  Then, the present invention measures constant parameter coverage information for calculating the coverage ratio of the constant described in the executed code for each new and old code (SA-3). Here, “Constant Parameter Coverage” (CPC: Constant Parameter Coverage) is a new metric for coverage testing, and as a metric for the constant coverage rate, how accurately the test data estimates the constant parameters of the system. It is a criterion that indicates whether it can be done.

  Then, the present invention evaluates the accuracy of the test data for each new and old code based on the measured operator coverage information and constant parameter coverage information and stores it in the storage unit (SA-4).

  Here, the present invention compares the test data newly evaluated for accuracy with the past test data already stored in the storage unit, and the test data with the higher accuracy of both test data is newly You may make it store in a memory | storage part according to an old code | cord | chord.

  Then, the present invention displays the output difference between the executed old code and the new code, measured operator coverage information, and / or constant parameter coverage information on the display unit (SA-5).

  This is the end of the summary of the present invention.

[Configuration of Consistency Checking Device 100]
Next, the configuration of the consistency checking apparatus 100 that evaluates the code coverage of the old code and the new code before and after the code change will be described. FIG. 4 is a block diagram showing an example of a configuration of a consistency check device based on operator coverage and constant parameter coverage to which the present invention is applied, and only a portion related to the present invention is conceptually shown in the configuration. It shows. The consistency checking device 100 is generally configured to include at least a storage unit 106, a control unit 102, and a display unit 114.

  In FIG. 4, the consistency checking apparatus 100 is schematically a communication device (not shown) such as a control unit 102 such as a CPU that controls the entire consistency checking apparatus 100 in an integrated manner, a communication line, and the like. ) Connected to the communication control interface unit (not shown), the input unit (not shown), the input / output control interface unit (not shown) connected to the display unit 114, and various databases and tables. The storage unit 106 is configured to be stored, and these units are communicably connected via an arbitrary communication path. Further, the consistency checking device 100 may be communicably connected to a network via a communication device such as a router and a wired or wireless communication line such as a dedicated line.

  Various databases and tables (test data DB 106a to new code test data DB 106c) stored in the storage unit 106 are storage means such as a fixed disk device, and various programs, tables, files, databases, and the like used for various processes. Stores web pages and the like.

  Among the constituent elements of the storage unit 106, the test data DB 106a is a test data storage unit that stores test data. That is, the test data DB 106a is a database that stores test data input when executing the new / old code and checking the consistency.

  The old code test data DB 106b is an old code test data storage unit that stores old code test data whose accuracy is evaluated based on the measured operator coverage information and constant parameter coverage information. . That is, the test data DB 106b for old code is a database that stores test data in association with each other when a new coverage measurement result is obtained from the test evaluation unit 102e-1, and this database stores the test data and its test data. Information (operator coverage (OC) information / constant parameter coverage (CPC) information) indicating which operator or constant in the code contributed to the coverage is stored in association with the test data.

  The new code test data DB 106c is a new code test data storage unit that stores the new code test data whose accuracy has been evaluated based on the measured operator coverage information and constant parameter coverage information. . That is, the test data DB 106c for new code is a database that stores test data in association with each other when a new coverage measurement result is obtained from the test evaluation unit 102e-2. Information (operator coverage (OC) information, constant parameter coverage (CPC) information, etc.) indicating which operator or constant in the code contributed to coverage is stored in association with the test data.

  The communication control interface unit performs communication control between the consistency checking device 100 and a network (or a communication device such as a router). That is, the communication control interface unit has a function of communicating data with other terminals via a communication line.

  The input / output control interface unit controls the input unit and the display unit 114. Here, as the display unit 114, a speaker can be used in addition to a monitor (including a home television). In the present invention, the display unit 114 displays, as execution results, a log representing the output difference between the new and old codes, and coverage measurement values (operator coverage measurement information and constant parameter coverage information) from the measurement unit 102b. Display means. As the input unit, a keyboard, a mouse, a microphone, and the like can be used.

  4, the control unit 102 has a control program such as an OS (Operating System), a program defining various processing procedures, and an internal memory for storing necessary data. Information processing for executing various processes is performed. The control unit 102 includes an execution unit 102a, a measurement unit 102b, and a test evaluation unit 102e in terms of functional concept.

  Among them, the execution unit 102a is an execution unit that executes test data stored in the test data DB 106a for each new and old code. That is, the execution unit 102a is an execution unit that executes the new and old codes to be compared. Here, as shown in FIG. 4, the execution unit 102 a includes an old code execution unit 102 a-1 and a new code execution unit 102 a-2. In other words, the old code execution unit 102a-1 is an old code execution unit that receives the test data stored in the test data DB 106a as input and executes the old code and outputs an output value. The new code execution unit 102a-2 The new code execution means executes the new code with the test data stored in the test data DB 106a as input, and outputs an output value. The execution unit 102a calculates an output difference between the output value output by the old code execution unit 102a-1 and the output value output by the new code execution unit 102a-2, and stores the log. Here, the execution unit 102a may determine the output consistency before and after the code change (the output difference is “0”) by calculating the output difference between the new code and the old code. The output difference is displayed on the display unit 114.

  In addition, the measurement units 102b-1 and 102b-1 and the operator coverage information for calculating the operator coverage rate described in the code executed by the execution unit 102a for each of the new and old codes, and the constant coverage This is a measurement means for measuring constant parameter coverage information for calculating the rate for each new and old code. Here, as shown in FIG. 4, the measurement unit 102b is configured to include operator coverage measurement units 102c-1 and 102c-1 and constant parameter coverage measurements 102d-1 and 102d. Among these, the operator coverage measurement units 102c-1 and 102-2 measure operator coverage information for calculating the coverage rate of the operator described in the code executed by the execution unit 102a for each new and old code. Operator coverage measurement means. The constant parameter coverage measurements 102d-1 and 102d-2 are constant parameters for measuring constant parameter coverage information for calculating the coverage rate of constants described in the code executed by the execution unit 102a for each new and old code. It is a coverage measurement means.

  The test evaluation units 102e-1 and 102e-1 and 102e-2 are operator coverage information measured by the operator coverage measurement units 102c-1 and 102c-1 and constant parameter coverage information measured by the constant parameter coverage measurement units 102d-1 and 102d-2. The test evaluation means evaluates the accuracy of the test data for each new and old code based on the above and stores them in the old code test DB 106b and the new code test DB 106c of the storage unit 106, respectively. Here, the test evaluation unit 102e includes test data newly evaluated for accuracy by the test evaluation unit 102e, test data already stored in the storage unit 106 (test DB 106b for old code or test DB 106c for new code), And the test data with the higher accuracy of the two test data may be stored in the storage unit 106 (test DB 106b for old code or test DB 106c for new code). That is, the test evaluation units 102e-1 and 102-2 are test data evaluation units based on coverage measurement results, and measurement result information (operator coverage information or constant parameter coverage information) corresponding to newly executed test data. Compare the past measurement result information stored in the database (test DB 106b for old code or test DB 106c for new code), and if the accuracy of the newly obtained measurement information is high, the current test data is added to the database. The measurement information stored in is updated.

  Further, here, the control unit 102 controls the display unit 114 to output difference between the old code and the new code executed by the execution unit 102a, and the operator coverage measured by the operator coverage measurement units 102c-1 and 102-2. The information and / or the constant parameter coverage information measured by the constant parameter coverage measurement units 102d to 1-2 may be displayed on the display unit 114.

  Above, description of the structure of this consistency test | inspection apparatus 100 is finished.

[Processing of Consistency Checking Device 100]
Next, an example of the process of the consistency checking device 100 according to the present embodiment configured as described above will be described in detail with reference to FIG.

[Basic operation processing]
First, details of the basic operation processing will be described with reference to FIG. 3 again. FIG. 3 is a flowchart showing an example of a basic operation process of the consistency checking device according to the present invention.

(Preprocessing)
First, the control unit 102 stores test data in the test data DB 106 a of the storage unit 106 as preprocessing. Thereafter, the process proceeds to the present process (SA-1 to SA-5).

(This processing)
Next, the execution unit 102a receives the test data stored in the test data DB 106a of the storage unit 106 as an input and executes it for each new and old code (SA-1). That is, the old code execution unit 102a-1 executes the old code with the test data as an input, and the new code execution unit 102a-2 executes the new code with the test data as an input.

  The operator coverage measurement units 102c-1 and 102c-1 and 102-2 of the measuring units 102b-1 and 102-2 execute operator coverage information for calculating the coverage rate of the operators described in the code executed by the execution unit 102a. Are measured separately for new and old codes (SA-2). The operator coverage information measurement process in SA-2 will be described in detail in the operator coverage information measurement process of Example 1 described later with reference to FIGS.

  Then, the constant parameter coverage measurement units 102d-1 and 102d-1 and 102d-2 of the measurement units 102b-1 and 102-2 include constant parameter coverage information for calculating the coverage rate of the constants described in the code executed by the execution unit 102a. Measure by new and old codes (SA-3). The constant parameter coverage information measurement process in SA-3 will be described in detail in the constant parameter coverage information measurement process of Example 2 described later with reference to FIGS.

  Then, the test evaluation units 102e-1 and 102e-1 and the operator coverage information measured by the operator coverage measurement units 102c-1 and 102c-1 and 102b-1 and the constant parameters of the measurement units 102b-1 and 102b-2 are measured. Based on the constant parameter coverage information measured by the coverage measuring units 102d-1 and 102-2, the accuracy of the test data is evaluated for each new and old code, the old code data DB 106b in the storage unit 106, and the new code data Store in the DB 106c (SA-4).

  Here, the test evaluation units 102 e-1 and 102-1 have the test data newly evaluated for accuracy and the past test data already stored in the storage unit 106 (the old code data DB 106 b or the new code data DB 106 c). And the test data with higher accuracy of both test data may be stored in the storage unit 106 (old code data DB 106b or new code data DB 106c) for each new and old code. The test evaluation process in SA-4 will be described in detail in the test evaluation process of Example 3 described later with reference to FIG.

  Then, the control unit 102 controls the display unit 114 to output difference between the old code and the new code executed by the execution unit 102a, operator coverage information measured by the operator coverage measurement units 102c-1 and 102-2, And / or the constant parameter coverage information measured by the constant parameter coverage measuring units 102d-1 and 102d-1 and 2 is displayed on the display unit (SA-5). This display process in SA-5 will be described in detail in the display process of Example 4 described later with reference to FIGS.

  This is the end of the description of the basic operation process.

  This concludes the description of the process of the consistency checking apparatus 100.

[Example]
Next, Examples 1 to 4 of the consistency checking apparatus 100 to which the present embodiment is applied will be described in detail with reference to FIGS. 5 to 13.

(Example 1: Operator coverage information measurement processing)
First, the operator coverage information measurement process corresponding to SA-2 in FIG. 3 will be described in detail with reference to FIGS. Here, FIG. 5 is a diagram illustrating an example of operator coverage (OC) in the present embodiment, and FIG. 6 is a diagram illustrating an example of analysis by operator coverage (OC) in the present embodiment. FIG. 7 is a diagram illustrating an example of analysis by operator coverage (OC) in this embodiment in more detail.

  First, the definition of operator coverage (OC: Operator Coverage) in the present invention will be described. Operator coverage (OC) is a new metric for coverage testing. This OC is defined by an operator because the number of inputs in the code and the form of the operation are known. In operator coverage (OC), each input is output independently, for example, by checking whether the output is calculated according to an arithmetic expression while changing only a given input value as a standard for measuring the coverage rate. Used to confirm that it affects Here, operator coverage (OC) can be applied to Boolean variables, integers, and real numbers (boolean, linear, and nonlinear arithmetic expressions). When applied to a Boolean expression using a Boolean variable, the result is the same as the MC / DC coverage measurement result.

  Next, an example of operator coverage (OC) will be described with reference to FIG. In FIG. 5, the upper diagram shows an example of a result obtained by performing an operator coverage (OC) test on a Boolean variable (OC test cases for Boolean AND), and the middle diagram represents an integer multiplication. An example of a result of performing an operator coverage (OC) test (OC test cases for integrity) is shown, and the figure below shows an operator coverage (OC) test for inequality (OC test cases for integrity). ) Shows an example of the result.

  In the upper diagram of FIG. 5, for example, when the input value (aB) is “1” and the input value (bB) is “1”, when the Boolean operation is performed, the output value (oB) becomes “1”. . For example, when the input value (aB) is “0” and the input value (bB) is “1”, when the Boolean operation is performed, the output value (oB) becomes “0” and the input value (aB) Is “1” and the input value (bB) is “0”, the output value (oB) becomes “0” when the Boolean operation is performed. At this time, as described above, when operator coverage (OC) is applied to a Boolean expression using a Boolean variable, the result is the same as the conventional MC / DC coverage measurement result.

  In the middle diagram of FIG. 5, for example, when the input value (a) is “10” and the input value (b) is “200”, the output value (oB) becomes “200” when numerical multiplication is performed. . Further, for example, when the input value (a) is “15” and the input value (b) is “20”, when the numerical multiplication is performed, the output value (oB) becomes “300”, and the input value (a) Is “10” and the input value (b) is “5”, the output value (oB) becomes “50” when the numerical multiplication is performed. At this time, there may be a difference from the conventional coverage measurement result by MC / DC. Details will be described later with reference to FIGS.

  In the lower diagram of FIG. 5, for example, when the input value (a) is “x1” and x1 ≧ T, the numerical value comparison (a ≧ T) results in the output value (oB) being “1”. For example, when the input value (a) is “x2” and x2 <T, the numerical value comparison (a ≧ T) results in the output value (oB) being “0”. At this time, the coverage measurement result based on the conventional MC / DC may be different. Details will be described later with reference to FIGS.

  Next, a result of applying operator coverage (OC) to the above-described test data of FIG. 2 will be described. Here, (a) to (c) in FIG. 6 correspond to (a) to (c) in FIG. 2 representing the above-described conventional coverage (for example, MC / DC).

  As shown in FIG. 6, for example, when the input value (a) is “2” and the input value (b) is “2”, the code before the change proceeds to block 1, so the output value (O1 ) Becomes “4 (Y)”, and the output value (O2) becomes “4”. For example, when the input value (a) is “1” and the input value (b) is “2”, the code before the change proceeds to block 2, and thus the output value (O 1) is “2 ( Y) ”, the output value (O2) is“ 2 ”, the input value (a) is“ 2 ”, and the input value (b) is“ 1 ”, the process proceeds to block 2 in the same manner. The output value (O1) is “2 (Y)” and the output value (O2) is “2”.

  In FIG. 6, code coverage based on conventional MC / DC using test data (data with input values (a, b) = (2, 2), (1, 2), (2, 1)). When the measurement is performed, the output values (O1 and O2) all match, but the MC / DC is 100% satisfied. As a result, it was evaluated that the code before the change and the code after the change were 100% identical despite the difference in the function of the code (the difference between “+” and “*” in the example of FIG. 6). .

  On the other hand, when the operator coverage (OC) in the present invention is applied, even if the same test data as the test data satisfying 100% with the conventional MC / DC is used, the test is performed in the code like MC / DC. Since the condition / determination is not used as a coverage metric and the output value obtained through which operator in the code is evaluated, the difference in the function in the code that has been overlooked in the conventional MC / DC is detected. Can do.

  That is, as shown in FIG. 6C, when the code before the change is executed with the test data of number 1 as an input, the output value (O1) is (the operator “of the block 1 in the code before the change“ As a result of calculation with “+”, “4 (Y)” is obtained, and when the code after change is executed with the test data of number 1 as input, the output value (O2) is (in the code after change) As a result of calculation by the operator “*” of the block 3, the same value “4” is obtained, there is no output difference, MC / DC is 100%, and the code difference cannot be determined. Since the determination is based on “+” or “*” in the example of FIG. 6, the code difference can be evaluated even when there is no output difference before and after the change.

  For example, as shown in FIG. 6C, when the test data of numbers 1 to 3 are used, the consistency check apparatus 100 executes the code before change, and as a result, the output value (O1) of number 1 is a block. Since the output value (O1) of numbers 2 and 3 is calculated by the operator “*” of the block 2, the operator coverage ( OC) is evaluated as 1/3 and 2/3, respectively. On the other hand, as a result of executing the code after the change, the consistency checking apparatus 100 has calculated all the output values (O2) of numbers 1 to 3 by the operator “*” of the block 3. Child coverage (OC) is evaluated as 100% of 3/3. That is, among the numbers 1 to 3 that are test signals, the probability of passing through the block 1 is 1/3, and the probability of passing through the block 2 is 2/3. In other words, in FIG. 6C, “4 (Y)” of the output value (O1) of the number 1 has an operator coverage (OC) of “1/3” for the block 1, and the numbers 2 and 3 "2 (N)" of the output value (O2) of the operator has an operator coverage (OC) of "2/3" for the block 2. In block 3 in the code after the change, any value of the test data is always calculated in block 3, so the operator coverage (OC) for block 3 is 100%.

  As described above, as a result of applying operator coverage (OC) to the test data of FIG. 2 described above, even if test data satisfying 100% with conventional coverage (for example, MC / DC) is used, operator coverage (OC) By applying this, it can be evaluated that this OC is not 100% with respect to all the blocks 1 to 3. Therefore, unlike MC / DC, only the test data shown in FIG. 6C does not satisfy the operator coverage (OC) 100%, and therefore the test data that completely cannot be determined to match or not match completely. I do not judge.

  Next, test data when the operator coverage (OC) is 100% for each of the blocks 1 to 3 will be described with reference to FIG. Here, in FIG. 7, FIG. 7 (c) shows an example of test data in which the operator coverage (OC) of block 1 is 100%, and FIG. 7 (d) shows the operator coverage (OC) of block 2. ) Shows an example of test data with 100%, and FIG. 7E shows an example of test data with an operator coverage (OC) of block 3 of 100%.

  As shown in FIG. 7C, the test data in FIG. 7C has input values (a, b) = (2, 2), (4, 2), (2, 4). When this test data is input to the code before the change, all of the conditional expressions “(a> 1) and (b> 1)?” Are judged to be Yes (Y), and the process proceeds to block 1. Therefore, the test data shown in FIG. 7C is evaluated as test data in which the operator coverage (OC) of block 1 is 100%. As shown in FIG. 7C, the value “6 (Y)” of the output value (O1) in the numbers 2 and 3 may be different from the value “8” of the output value (O2). That is, there may be a difference between the output value (O1) and the output value (O2). Therefore, the above-mentioned conventional MC / DC 100% test data (data with input values (a, b) = (2, 2), (1, 2), (2, 1)) cannot be discriminated. The difference in the output of the old code can be discriminated (FIG. 7c).

  Further, as shown in FIG. 7D, the test data in FIG. 7D has input values (a, b) = (1, 1), (1, 2), (2, 1). When this test data is input to the code before the change, all of the conditional expressions “(a> 1) and (b> 1)?” Are judged as No (N), and the process proceeds to block 2. Therefore, the test data shown in FIG. 7D is evaluated as test data having an operator coverage (OC) of block 2 of 100%.

  As shown in FIG. 7E, the test data in FIG. 7E has input values (a, b) = (2, 2), (1, 2), (2, 1). When this test data is input to the code after the change, the process proceeds to block 3 and is calculated by the operator “*” of block 3, so that the test data shown in FIG. Test data in which (OC) is 100%.

  In this way, by performing code coverage measurement using the operator coverage (OC) of the present embodiment, if test data having an operator coverage (OC) of 100% is used for each block, the conventional code coverage ( For example, it is possible to determine the difference between the new and old codes of the software after the change, which cannot be determined as 100% in MC / DC. Thereby, as a result of performing code coverage measurement using test data with 100% operator coverage (OC), it is possible to accurately determine code mismatch.

  This is the end of the description of the embodiment of the operator coverage information measurement process.

(Example 2: Constant parameter coverage information measurement process)
Next, constant parameter coverage information measurement processing corresponding to SA-3 in FIG. 3 will be described in detail with reference to FIGS. Here, FIG. 8 is a diagram illustrating an example of constant parameter coverage (CPC) in the present embodiment, FIG. 9 is a diagram illustrating an example of estimation by constant parameter coverage (CPC) in the present embodiment, and FIG. 10 is a diagram showing an example of estimation by constant parameter coverage (CPC) in the present embodiment in more detail.

  First, the definition of constant parameter coverage (CPC) in the present invention will be described. Constant parameter coverage (CPC) is a new metric for coverage testing, which is a measure of how accurately test data can estimate the system's constant parameters. By this CPC, a boundary where discontinuity is likely to occur can be detected (hereinafter described in detail).

  Here, an example of constant parameter coverage (CPC) will be described with reference to FIG. FIG. 8 shows an example of constant parameter coverage (CPC) for accurately estimating the input data (x) and the constant 3 included in the function (x / 3). Here, FIG. 8A is a graph of the function (x / 3), and FIG. 8B is a table of test data (x) and CPC.

  As shown in FIG. 8, when an input value x = 0,5, which is test data, is input and a code whose function is y = x / 3 is executed, the output values are y = 0, 1, respectively. Become. Therefore, even if the operators are the same in “x / K”, the same output value (y = 0, 1) is obtained when the constants K = 3, 4, and 5. In other words, when this test data (x = 0, 5) is used, the constant K in the integer division function (y = x / K) can be estimated in three ways: “3, 4, 5”. That is, when (input value x, output value y) = (2, 0), (5, 1), the integer division function (y = x / K) is not only y = x / 3 but also y = Both x / 4 and y = x / 5 are considered.

  Accordingly, when “2, 5” is used as the test data (x) for y = x / K, three constants can be estimated, so the constant parameter coverage (CPC) is “1/3” (33% ).

  Next, an example of test data for increasing the constant parameter coverage (CPC) to 100% in the code shown in FIG. 8 and the constant parameter coverage (CPC) in this case will be described with reference to FIG.

  The upper diagram of FIG. 9 corresponds to FIG. In the upper diagram of FIG. 9, when the test data (x) is “2, 5”, it can be estimated that the constant K is three, K = 3,4,5, so the OC is 1/3 (33% ).

  Next, in the middle diagram of FIG. 9, when test data (x = 2, 4) is input, the output values are y = 0, 1, respectively, so the constant K is two types of K = 3,4. The constant parameter coverage (CPC) is “2/3” (66%) for y = x / 3.

  In the lower diagram of FIG. 9, when test data (x = 2, 3) is input, the output values are y = 0, 1, respectively. Therefore, it can be estimated that the constant K is one K = 3, and y = X / 3, the constant parameter coverage (CPC) is “3/3” (100%). That is, when “2, 3” of the test data (x) is applied, the constant K cannot be other than “3”.

  As described above, when constant parameter coverage (CPC) is applied, it is possible to detect a boundary where discontinuity is likely to occur, and it is possible to determine a difference in constant even when the output values are the same.

  In addition, by changing the value of the test data (x) while measuring the constant parameter coverage (CPC) in this way, test data with a constant parameter coverage (CPC) of 100% can be obtained while suppressing redundancy. Obtainable.

  Subsequently, with reference to FIG. 10, it is shown that the function (x / 3) and the function (x / 5) can be distinguished by using constant parameter coverage (CPC) as an example. In FIG. 10, FIG. 10 (a) is a table showing the calculation result by the input data (x) and the function, and FIG. 10 (b) is an operator coverage (OC) and constant parameter in a certain combination of input values. It is a graph which shows coverage (CPC).

In FIG. 10A, for example, when the input data x = 0, 1, 2, 3, 4, 5, 6, according to the function (x / 3), y ₁ = 0, 0, 0, ₁ , 1, 1 and ₂ are obtained. According to the function (x / 5), output values y ₂ = 0, 0, 0, 0, 0, ₁ , 1 are obtained. FIG. 10B shows a graph plotting these values (input data (x), output value (y ₁ ), output value (y ₂ )).

  Here, as described in the first embodiment, the operator coverage (OC) is calculated as an arithmetic expression while changing only a given input value, for example, as a coverage rate measurement standard. For example, it is confirmed that each input independently affects the output.

  As shown in FIG. 10 (b), when two inputs are executed, each input has an influence on the output independently of the function (x / 3) and the function (x / 5). %, The operator coverage (OC) is 100% for any combination. On the other hand, even in the case of OC 100%, the constant parameter coverage (CPC) is not necessarily 100% because the constant K cannot be uniquely estimated.

  That is, as shown in FIG. 10B, when the test data is (x = 2, 5), the operator coverage (OC) is 100%, but the function (x / 3) is also the function (x / 4). However, since the output value y = 0, 1 even in the function (x / 5) and the two cannot be distinguished, the constant parameter coverage (CPC) is evaluated as “1/3” (33%). On the other hand, when the test data (x = 2, 3) is used, the OC is 100%, and the function (x / 3) can be estimated uniquely, so that the CPC is 100%. The same applies to the test data (x = 4, 5).

  Thus, as shown in FIG. 10B, the difference in constants could not be determined in the test data with 100% operator coverage (OC). However, if constant parameter coverage (CPC) was applied, the difference in constants was not found. It is possible to evaluate clearer code differences including. In other words, by measuring operator coverage (OC) and constant parameter coverage (CPC), the difference between new and old code before and after software change can be determined more accurately than conventional coverage (for example, MC / DC). can do.

  This is the end of the description of the embodiment of the constant parameter coverage information measurement process.

(Example 3: Test evaluation process)
Next, the test evaluation process corresponding to SA-4 in FIG. 3 will be described in detail with reference to FIG. Here, FIG. 11 is a diagram illustrating an example of a test evaluation process in the present embodiment. In FIG. 11, SB-1 and SB-2 in FIG. 11 correspond to SA-1 in FIG. 3, and similarly, SB-3 corresponds to SA-2 and SA-3, and SB- 4 and SB-5 correspond to SA-4.

  Next, the generation of test databases (new code test database 106b and old code test database 106c) by the processing of the test evaluation units 102e-1 and 102-2 will be described with reference to FIG. In FIG. 11, operator coverage (OC) and constant parameter coverage (CPC) are defined for specific operators and constants in the code, respectively. Therefore, information (for example, line information in the code) indicating which line of each test data operator and constant OC and CPC in the code is maximized is stored. Therefore, when a plurality of test data are used for the same operator and constant, a redundant test can be removed and a simple test database can be created. In the following description, the new code side and the old code side may be described without distinction, but the same processing is executed on the new code side and the old code side, respectively.

  As shown in FIG. 11, first, the control unit 102 reads test data from the test data DB 106a of the storage unit 106 in which the test data is stored (SB-1).

  Then, the execution unit 102a executes the test data read in SB-2 for each new and old code (SB-2).

  Then, the operator coverage measurement unit 102c of the measurement unit 102b measures the operator coverage information for calculating the coverage rate of the operator described in the code executed by the execution unit 102a for each new and old code. In addition, the constant parameter coverage measurement unit 102d of the measurement unit 102b obtains constant parameter coverage information for calculating the coverage rate of the constant described in the code executed by the execution unit 102a for each new and old code. (SB-3).

  Then, the test evaluation unit 102e acquires operator coverage (OC) information and constant parameter coverage (CPC) information corresponding to the newly executed test data (SB-4).

  Then, the OC / CPC information (including line information) of the test data acquired in SA-4 is compared with the OC / CPC information stored in the test data DB corresponding to the line information, and a new test is performed. It is determined whether or not the accuracy of the data is higher than that of the test data stored in the test data DB (SB-5).

  When the test evaluation unit 102e determines that the accuracy is high (SB-5: Yes), the test evaluation unit 102e registers the test data in association with the row information in the test data DB. That is, the test data and the test line information, the information indicating which operator or constant of the code the test data and the data contributed to, are expressed as “test data: operator coverage (OC) information” and “test data. : Constant parameter coverage (CPC) information ”and stored in association with each other.

  As a result, after obtaining high-precision test data, the next new / old code test is performed using the test data stored in the test data DB from which redundancy has been removed, to perform a debug test. In addition, measurement of operator coverage information and constant parameter coverage information can be omitted at the next execution, and the entire debug test can be simplified.

  This is the end of the description of the embodiment of the test evaluation process.

(Example 4: Display processing)
Finally, display processing corresponding to SA-5 in FIG. 3 will be described in detail with reference to FIGS. 12 and 13. Here, FIG. 12 and FIG. 13 are diagrams illustrating an example of a display screen by display processing in the present embodiment.

  In FIG. 12, the display unit 114 displays the test result. This test result is displayed with information indicating how much coverage is possible for operators and constants of the new and old codes, and the number of test data executed. For example, the old code browser on the left side of the screen displays the old code before the change, and the new code browser on the right side of the screen displays the new code before the change so that both codes can be visually compared. . The box at the lower left of the screen displays the number of tested data and output difference as common information regarding the test. In addition, the operator coverage (OC) rate (%) in the entire code and the constant parameter coverage (CPC) rate (%) in the entire code are displayed at the lower center of the screen for each new and old code. In addition, there is an OC status bar and a CPC status bar on the right side of each new and old code browser. The OC and CPC status corresponding to the operators and constants in the measured code can be displayed in different colors and shades for each new and old code. Is displayed. The OC status bar and CPC status bar will be described in detail below with reference to FIG.

  Next, the OC status bar and the CPC status bar will be described with reference to FIG. FIG. 13 shows an example of the OC status bar. In FIG. 13, the operator coverage (OC) of the test data executed for the multiplication operator “*” is displayed in color and shading. Here, FIG. 13A shows a case where the test data satisfies 1/3 of the OC, FIG. 13B shows a case where the test data satisfies 2/3 of the OC, and FIG. c) shows a case where the test data satisfies 100% of the OC. As shown in FIGS. 13A and 13B, the OC status bar is displayed darker as the OC value increases. Although not shown, the CPC status bar is displayed in the same manner.

  This makes it possible to easily visually compare the coverage rates for the operators and constants of the new and old codes.

  This is the end of the description of the display processing embodiment.

  This concludes the description of the embodiment of the consistency checking apparatus 100.

[Other embodiments]
Although the embodiments of the present invention have been described so far, the present invention is not limited to the above-described embodiments, but can be applied to various different embodiments within the scope of the technical idea described in the claims. It may be implemented.

  For example, the case where the consistency checking apparatus 100 performs processing in a stand-alone form has been described as an example. However, processing is performed in response to a request from a client terminal configured with a separate housing from the consistency checking apparatus 100, You may comprise so that the process result may be returned to the said client terminal.

  In addition, among the processes described in the embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method.

  In addition, the processing procedures, control procedures, specific names, information including parameters such as registration data and search conditions for each processing, screen examples, and database configurations shown in the literature and drawings, unless otherwise specified It can be changed arbitrarily.

  In addition, regarding the consistency checking apparatus 100, each illustrated component is functionally schematic and does not necessarily need to be physically configured as illustrated.

  For example, the processing functions provided in each device of the consistency checking device 100, in particular, the processing functions performed by the control unit 102, all or any part thereof are interpreted by a CPU (Central Processing Unit) and the CPU. It can be realized by a program to be executed, or can be realized as hardware by wired logic. The program is recorded on a recording medium to be described later, and is mechanically read by the consistency checking device 100 as necessary. In other words, the storage unit 106 such as ROM or HD stores a computer program for performing various processes by giving instructions to the CPU in cooperation with an OS (Operating System). This computer program is executed by being loaded into the RAM, and constitutes the control unit 102 in cooperation with the CPU.

  The computer program may be stored in an application program server connected to the consistency checking device 100 via an arbitrary network, and may be downloaded in whole or in part as necessary. It is.

  The program according to the present invention can also be stored in a computer-readable recording medium. Here, the “recording medium” refers to any “portable physical medium” such as a flexible disk, a magneto-optical disk, a ROM, an EPROM, an EEPROM, a CD-ROM, an MO, and a DVD, or a LAN, WAN, or Internet. It includes a “communication medium” that holds the program in a short period of time, such as a communication line or a carrier wave when the program is transmitted via a network represented by

  The “program” is a data processing method described in an arbitrary language or description method, and may be in any format such as source code or binary code. Note that the “program” is not necessarily limited to a single configuration, but is distributed in the form of a plurality of modules and libraries, or in cooperation with a separate program typified by an OS (Operating System). Including those that achieve the function. Note that a well-known configuration and procedure can be used for a specific configuration for reading a recording medium, a reading procedure, an installation procedure after reading, and the like in each device described in the embodiment.

  Various databases (test data DB 106a to new code test data DB 106c) stored in the storage unit 106 are storage means such as a memory device such as a RAM and a ROM, a fixed disk device such as a hard disk, a flexible disk, and an optical disk. Various programs, tables, databases, web page files, etc. used for various processing and website provision are stored.

  In addition, the consistency checking apparatus 100 is connected to an information processing apparatus such as a known personal computer or workstation, and software (including programs, data, etc.) for realizing the method of the present invention is installed in the information processing apparatus. May be realized.

  Furthermore, the specific form of distribution / integration of the devices is not limited to the one shown in the figure, and all or a part thereof is configured to be functionally or physically distributed / integrated in an arbitrary unit according to various additions. can do.

  As described in detail above, according to the present invention, using the new code coverage (OC and CPC) instead of the conventional code coverage (for example, MC / DC), matching between codes after software change can be achieved. Since it is possible to provide a consistency check method, a consistency check apparatus, and a program that can clearly distinguish a mismatch, it is useful in the software field.

It is a data flow figure which shows an example of the consistency check method based on the conventional code coverage (for example, MC / DC). It is a figure which shows an example of the test data which satisfy | fills 100% of the conventional code coverage (for example, MC / DC) and this MC / DC. It is a flowchart which shows an example of the basic operation | movement process of the consistency inspection apparatus in this invention. It is a block diagram which shows an example of a structure of the consistency inspection apparatus based on operator coverage and constant parameter coverage to which this invention is applied. It is a figure which shows an example of operator coverage (OC) in a present Example. It is a figure which shows an example of the analysis by operator coverage (OC) in a present Example. It is a figure which shows an example of the analysis by operator coverage (OC) in a present Example in detail. It is a figure which shows an example of the constant parameter coverage (CPC) in a present Example. It is a figure which shows an example of estimation by the constant parameter coverage (CPC) in a present Example. It is a figure which shows an example of the estimation by the constant parameter coverage (CPC) in a present Example in detail. It is a figure which shows an example of the test evaluation process in a present Example. It is a figure which shows an example of the display screen by the display process in a present Example. It is a figure which shows an example of the display screen by the display process in a present Example.

Explanation of symbols

100 Consistency inspection device
102 Control unit
102a execution unit
102a-1 Old code execution part
102a-2 New code execution part
102b-1 to 2 measuring unit
102c-1 to 2 operator coverage measurement unit
102d-1 ~ 2 constant parameter coverage measurement unit
102e-1-2 Test evaluation unit
106 Storage unit
106a Test data DB
106b Test data DB for old code
106c Test data DB for new code
114 display

Claims (12)

A consistency check method executed in a consistency check apparatus including at least a storage unit and a control unit for evaluating code coverage of old code and new code before and after code change,
Executed in the control unit,
A test data storage step for storing the test data in the storage unit;
An execution step of executing the test data stored in the storage unit in the test data storage step according to new and old codes;
An operator coverage measurement step for measuring operator coverage information for calculating the coverage rate of the operator described in the code executed in the execution step according to new and old codes;
Constant parameter coverage measurement step for measuring the constant parameter coverage information for calculating the coverage ratio of the constant described in the code executed in the execution step, for each new and old code,
Consistency inspection method characterized by including. The coincidence inspection method according to claim 1,
Executed in the control unit,
Based on the operator coverage information measured in the operator coverage measurement step and the constant parameter coverage information measured in the constant parameter coverage measurement step, the accuracy of the test data is determined for each new and old code. A matching test method, further comprising a test evaluation step of evaluating and storing in the storage unit. In the consistency check method according to claim 2,
The test evaluation step is
Compare the test data newly evaluated for accuracy with the past test data already stored in the storage unit, and store the test data with the higher accuracy of both test data for each new and old code. A consistency check method, characterized in that the method is stored in a unit. The coincidence inspection method according to claim 1,
The coincidence inspection device
A display unit,
Executed in the control unit,
The output difference between the old code and the new code executed in the execution step, the operator coverage information measured in the operator coverage measurement step, and / or the constant parameter coverage measurement step. And a display step for displaying the constant parameter coverage information on the display unit. A consistency checking device comprising at least a storage unit and a control unit for evaluating code coverage of old code and new code before and after code change,
The storage unit
A test data storage means for storing test data;
The control unit
Execution means for executing the test data stored in the storage unit by the test data storage means for each new and old code;
Operator coverage measurement means for measuring operator coverage information for calculating the coverage rate of the operator described in the code executed by the execution means for each new and old code;
Constant parameter coverage measurement means for measuring constant parameter coverage information for calculating the coverage rate of the constant described in the code executed by the execution means for each new and old code,
A coincidence inspection apparatus comprising: In the consistency inspection apparatus according to claim 5,
The control unit
Based on the operator coverage information measured by the operator coverage measuring means and the constant parameter coverage information measured by the constant parameter coverage measuring means, the accuracy of the test data is evaluated for new and old codes. And a test evaluation means for storing in the storage unit. The coincidence inspection apparatus according to claim 6,
The test evaluation means is
Compare the test data newly evaluated for accuracy with the past test data already stored in the storage unit, and store the test data with the higher accuracy of both test data for each new and old code. A coincidence inspection apparatus characterized by being stored in a unit. In the consistency inspection apparatus according to claim 5,
The coincidence inspection device
A display unit,
The control unit
The output difference between the old code and the new code executed by the execution means, the operator coverage information measured by the operator coverage measurement means, and / or the constant measured by the constant parameter coverage measurement means A consistency inspection apparatus, further comprising display means for displaying parameter coverage information on the display unit. A program for evaluating a code coverage of an old code and a new code before and after a code change, and causing a consistency checking device including at least a storage unit and a control unit to execute the code coverage,
Executed in the control unit,
A test data storage step for storing the test data in the storage unit;
An execution step of executing the test data stored in the storage unit in the test data storage step according to new and old codes;
An operator coverage measurement step for measuring operator coverage information for calculating the coverage rate of the operator described in the code executed in the execution step according to new and old codes;
Constant parameter coverage measurement step for measuring the constant parameter coverage information for calculating the coverage ratio of the constant described in the code executed in the execution step according to new and old codes, and
A program for running A consistency check method executed in a consistency check apparatus including at least a storage unit and a control unit, which evaluates code coverage while checking the match of output values by a plurality of codes,
Executed in the control unit,
A test data storage step of storing a plurality of test data in the storage unit;
An execution step of causing the plurality of codes to execute the test data stored in the storage unit in the test data storage step as an input value;
An operator coverage measurement step for measuring operator coverage information for calculating a coverage rate of an operator in the code, which is executed a plurality of times in the execution step, and
A constant parameter for calculating a coverage rate of a constant that is capable of uniquely estimating the constant described in the operator from the predetermined plurality of output values output by the code executed in the execution step. A constant parameter coverage measurement step for measuring coverage information;
A method for checking consistency, comprising: In the consistency check method according to claim 10,
Executed in the control unit,
The test data for increasing the coverage rate based on the operator coverage information measured in the operator coverage measurement step or the constant parameter coverage information measured in the constant parameter coverage measurement step A test evaluation step to be stored in the storage unit;
The consistency check method characterized by further including. In the consistency check method according to claim 11,
The operator coverage measurement step is
The line information in the code of the operator executed in the execution step is stored in the operator coverage information,
The constant parameter coverage measurement step is
The line information in the code of the constant estimated by the output value output by the code in the execution step is stored in the constant parameter coverage information,
The test evaluation step is
By comparing the line information corresponding to the test data stored in the storage unit and the line information corresponding to the test data newly executed in the execution step, the test data is Determining whether to increase the coverage rate, and storing the test data in the storage unit in association with the row information when it is determined to increase the coverage rate;
A method for checking consistency.

Images