WO2014087493A1

WO2014087493A1 - Test pattern creation device and creation method, and software verification device and verification method

Info

Publication number: WO2014087493A1
Application number: PCT/JP2012/081419
Authority: WO
Inventors: 浩晃中谷
Original assignee: 株式会社日立製作所
Priority date: 2012-12-04
Filing date: 2012-12-04
Publication date: 2014-06-12

Abstract

In order to create a test pattern for which a description of the coverage acquisition mechanism is unnecessary and for which there is little verification leakage in the verification of software, a test pattern detection unit (11) divides an input test pattern into prescribed collected units, and forms libraries therefrom. A test coverage measurement unit (13) measures the test coverage by verifying, for each library, the test patterns that have been formed into the library. A library optimization unit (16) uses the measured test coverage and verification priority order information to change the library execution sequence or change a control parameter for the libraries so as to expand the test coverage, thereby creating a new test pattern.

Description

Test pattern creation device and creation method, and software verification device and validation method

The present invention relates to an apparatus and method for creating a test pattern used for software verification, and an apparatus and method for verifying software using the test pattern.

In software development, the performance of the created software is verified (tested) using a test pattern. The software verification range is also called test coverage. The larger the test coverage, the wider the range of software is covered by the test. In order to expand the verification range of software and prevent verification failure, the following methods have been proposed.

The method disclosed in Patent Document 1 expands the test pattern using a software simulator that is a non-real machine. In this case, the coverage acquisition mechanism is described in a test pattern corresponding to a predetermined test item, the test coverage is measured, the coverage result is analyzed, the test constraint condition is controlled, and the software verification range is expanded.

The method disclosed in Patent Document 2 is to expand the verification range by monitoring the variables of the verification target software and directly setting the combination of unverified variables in the software. In this case, instead of expanding the test pattern, software variables are directly set in the actual machine test to expand the verification range.

JP 2006-119922 A JP 2009-157456 A

As embedded system devices such as embedded processors become more sophisticated, multifunctional, and complex, new software will be introduced. For example, in a storage system having the same basic operation (read / write), when new software is introduced as a result of performance measures or function addition, a portion that cannot be verified by a conventional test pattern, a so-called verification failure may occur. As a result, when an unverified command pattern is executed in the actual machine system, there is a possibility that the actual machine operation may be stopped or a hardware failure may occur.

As described above, a method for expanding test patterns or a method for directly setting software variables has been proposed in order to expand the verification range (test coverage) of software. However, in the test pattern expansion method of Patent Document 1, it is necessary to describe a coverage acquisition mechanism in the test pattern itself. For example, when a test pattern passed from a customer in a black box is used to perform verification in the customer's system, it is difficult to describe the coverage acquisition mechanism, and the verification range cannot be expanded.

In addition, in the variable direct setting method of Patent Document 2, it is necessary to provide a dedicated environment for enabling control of all variable states of the target software. However, an actual system close to the product specification has only a limited debug environment (verification debug pins, pads, etc.), and it is difficult to control all variable states, and there is a limit to the expansion of the verification range.

An object of the present invention is to provide a test pattern creation apparatus and creation method and a software verification apparatus and verification method that do not require a description of a coverage acquisition mechanism in software verification and have few verification omissions.

The present invention is a test pattern creation device for creating a test pattern used for software verification, a test pattern detection unit that divides an input test pattern into a predetermined unit and creates a library, and a test that is made into a library A test coverage measurement unit that verifies the pattern for each library and measures test coverage; and a library optimization unit that changes the execution order of the libraries so that the test coverage is expanded based on the measured test coverage. Alternatively, the library optimization unit changes control parameters of the library so that the test coverage is expanded based on the measured test coverage.

The present invention is a software verification apparatus that verifies software using the test pattern created by the test pattern creation apparatus, and that verifies software verification status and extracts combinations of software variables used at the time of verification. From the monitoring unit, the variable information storage unit that stores the extracted variable combination and extracts an unexecuted variable combination at the time of verification, and the external terminal of the verification target device incorporating the software using the unexecuted variable combination A variable control unit for setting software variables.

According to the present invention, verification failure in software verification can be reduced by a new test pattern that optimizes an existing test pattern. Furthermore, when verification is performed using a new test pattern, verification failure can be compensated by forcibly setting variables from the outside. This expands test coverage during verification and contributes to software quality improvement.

1 is a configuration diagram of a test pattern creation device according to Embodiment 1. FIG. 1 is a configuration diagram of a test pattern creation device according to Embodiment 1. FIG. It is a figure which illustrates typically the preparation method of a new test pattern. It is a flowchart which shows a new test pattern creation process. It is a detailed flowchart of a library optimization process. It is a detailed flowchart of a library optimization process. FIG. 6 shows a configuration diagram of a software verification apparatus according to a second embodiment. FIG. 6 shows a configuration diagram of a software verification apparatus according to a second embodiment. It is a figure explaining a software verification method typically. It is a flowchart which shows a software verification process.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Example 1 relates to creating a new test pattern from an existing test pattern. By detecting and analyzing an existing test pattern, creating a library, and automatically optimizing it, the verification range of the test is prevented and the verification range is verified. A new test pattern that can expand (test coverage) is created.

1A and 1B are configuration diagrams of a test pattern creation device according to a first embodiment.
FIG. 1A shows a case where a test pattern creation device 10 is connected to the outside of the actual machine 1 to be tested. The actual machine 1 incorporates software to be verified. Two input units 31 and 32 are provided, an existing test pattern is input to the input unit 31, and verification priority order information indicating the priority order of verification items is input to the input unit 32. The test pattern detection unit 11 divides the input test pattern into a certain unit and creates a library. The library storage unit 12 stores the divided library. The test target real machine 1 is connected to the library storage unit 12, and a test execution library or all test libraries are input from the library storage unit 12 to the test target real machine 1. In the real machine 1 to be tested, a software verification test is performed using the input library. The test coverage measurement unit 13 connected to the test target real machine 1 measures the test coverage (verification range) of each library, and the coverage data storage unit 14 stores the measured test coverage data (coverage data). The test acceptance / rejection determination unit 15 determines whether or not the coverage target value has been achieved for the acquired coverage data, that is, the pass / fail of the test result. If it is determined to be acceptable, a test pattern composed of the library is output from the output unit 33.

If the test acceptance / rejection determination unit 15 determines a failure, the test result is returned to the coverage data storage unit 14 and output to the library optimization unit 16. The library optimizing unit 16 uses the library stored in the library storage unit 12 to expand the test coverage based on the test result obtained from the coverage data storage unit 14 and the verification priority information input from the input unit 32. Perform optimization. Then, the test is performed again on the test target real machine 1 using the optimized library, and coverage measurement and pass / fail determination are performed again. Thus, this process is repeated until the test passes, that is, the coverage target value is achieved, and an optimized new test pattern is output from the output unit 33.

FIG. 1B shows a case where the test pattern creation device 10 is mounted inside the actual machine 1 to be tested. In this case, the test pattern creation device 10 is connected to the test target software storage module 2 in the test target real machine 1, and the test target software storage module 2 stores software to be verified. The internal configuration of the test pattern creation device 10 is the same as that in FIG. 1A, and repeated description is omitted.

FIG. 2 is a diagram schematically illustrating a method for creating a new test pattern. Here, the existing test pattern 101 is divided into a plurality of libraries, and a new test pattern 108 is created by optimizing each library (changing execution order, changing control parameters).

The input existing test pattern 101 is divided into a group of units in the test pattern detection unit 11 and is made into a library. A certain unit is, for example, a unit for each test scenario (video playback, application activation, high-speed calculation, etc.) existing in one test pattern, or for each data size for performing a data read operation in the video playback scenario. is there. As a result, a test pattern 102 composed of a set of a plurality of libraries 1 to n is obtained. Here, a control parameter in each library is indicated by a symbol P. The control parameter P corresponds to, for example, the length of data in the library and the address range.

The verification test is individually executed by each library, and the coverage data 1 to n is measured by the test coverage measuring unit 13. The measured coverage data 103 of each library is stored in the coverage data storage unit 14. In addition to this, the coverage data storage unit 14 also stores coverage data of the entire test pattern (a total value of coverage data of all libraries).

In the test pattern 102, the library optimization unit 16 optimizes the configurations of the libraries 1 to n included therein. For optimization, the verification priority information 104 set at the time of software design or verification and the coverage data 103 stored in the coverage data storage unit 14 are used. That is, by referring to the verification priority information 104, a priority order for expanding the test coverage is determined from the source code having a low test coverage. Further, by referring to the coverage data 103, a source code of software having low test coverage is determined. Based on these, the test pattern 105 (a) by changing the execution order of the library, the test pattern 106 (b) by changing the control parameter in the library, or the test pattern 107 using both (a) + (b). Perform optimization. Replacing the execution order of the library in (a) is effective for coverage expansion because the software state can change in the operation order. Further, changing the control parameter in the library in (b) is effective for expanding the coverage by changing the software variable value, the hardware register value, or the like, for example, and increasing the coverage. By combining these two, the coverage can be further expanded. In addition, the library optimization enables the test coverage to reach the target value at an early stage, and thus has the effect of shortening the verification time. A coverage target value is achieved by these optimization processes, and is output as a new test pattern 108.

FIG. 3 is a flowchart showing a new test pattern creation process. Hereinafter, it demonstrates in order of a process.
In S <b> 301, the existing test pattern 101 is input to the test pattern detection unit 11. In S302, the test pattern detection unit 11 divides the input test pattern into a certain unit and creates a library. The library storage unit 12 stores each divided library 102. In S303, a verification test is performed for each library on the actual machine, and the test coverage measurement unit 13 measures the test coverage for each library. In S304, the coverage data storage unit 14 stores the measured coverage data 103. In S305, the test pass / fail determination unit 15 adds the coverage data for each library and determines whether or not the target coverage has been reached. If the target is reached in the determination, the process ends here.

If the target is not reached in the determination of S305, the process proceeds to S306, and the library optimization unit 16 optimizes the library. The library optimizing unit 16 performs library optimization such as changing the library execution order and changing library control parameters, and creates a new test pattern. Thereafter, the process returns to S303, where the verification test is performed again with the new test pattern, and the test coverage for each library is measured. Coverage data is stored in S304, and it is determined whether or not the target coverage has been reached in S305. If the target has been reached, the process ends. If the target has not been reached, the optimization of S306 is performed again, and the process is repeated until the target is reached. In this way, a new test pattern 108 that satisfies the target coverage is created.

4A and 4B are detailed flowcharts of the library optimization step S306. FIG. 4A shows a case where the library execution order is changed, and FIG. 4B shows a case where the library control parameters are changed.

In FIG. 4A, optimization is performed by changing the library execution order using the verification priority information 104.
In step S401, the verification priority information 104 is analyzed to determine which part of the source code is to be preferentially verified or which test pattern is to be preferentially verified. In S402, optimization is performed by changing the execution order of the libraries based on the analysis result in S401. Thus, for example, a new test pattern as shown by 105 in FIG. 2 is created. In S403, a verification test is performed on the new test pattern, and the test coverage is measured.

In S404, the coverage data of the previous test pattern is compared with the coverage data of the new test pattern. In this comparison, the coverage data of the entire test pattern is compared. However, coverage data for each library may be used to compare whether coverage can be ensured in a short time. If it is determined in S404 that the coverage is improved from the previous time, the process returns to S402 and the library execution order is changed again. The library optimization is repeatedly executed while the coverage is improved in the determination of S404.

If it is determined in S404 that the coverage has not improved (or decreased) compared to the previous time, the process proceeds to S405, where the previous test pattern, that is, the test pattern of the previous execution order to be optimized is returned. In S406, it is determined whether to optimize again by changing the library execution order. In this determination, the number of times that the coverage has not improved as a result of the determination in S404 may be counted, and if this reaches a predetermined number, it may be determined that “not optimized again”.

When optimizing again in the determination of S406, the process returns to S402, the library execution order is changed, and the coverage is improved. If the optimization is not performed again in the determination of S406, the process ends here. Or you may switch to other optimization processes (for example, control parameter change in a library of Drawing 4B).

In FIG. 4B, optimization is performed by changing the control parameter in the library using the verification priority information 104.
In S411, the verification priority order information 104 is analyzed, the coverage results at the time of executing each library are collated, and what part of the source code is preferentially verified or which test pattern is preferentially verified. To decide. In S412, based on the analysis result in S411, optimization is performed by changing the control parameter in the library. Thus, for example, a new test pattern as shown by 106 in FIG. 2 is created. In S413, a verification test is performed on the new test pattern, and the test coverage is measured.

In S414, the coverage data of the previous test pattern is compared with the coverage data of the new test pattern. In this comparison, the coverage data of the entire test pattern is compared. However, coverage data for each library may be used to compare whether coverage can be ensured in a short time. If it is determined in S414 that the coverage has improved compared to the previous time, the process returns to S412 to change the control parameters in the library again. Then, the library optimization is repeatedly executed while the coverage is improved in the determination of S414.

If it is determined in S414 that the coverage has not improved (or decreased) compared to the previous time, the process proceeds to S415, and the test pattern of the previous control parameter to be optimized is returned to the previous test pattern. In S416, it is determined whether to optimize again based on the control parameter in the library.

When optimizing again in the determination in S416, the process returns to S412 to change the control parameter in the library and improve the coverage. If the optimization is not performed again in the determination in S416, the process ends here. Alternatively, it may be switched to another optimization process (for example, change of library execution order in FIG. 4A).

4) The library optimization in FIG. 4A and FIG. 4B is executed until the target coverage is reached. As an optimization method, the library execution order change in FIG. 4A and the in-library control parameter change in FIG. 4B can be performed, and either one may be selected as appropriate. When the goal cannot be achieved by only one of them, it is easy to achieve the coverage of the goal by executing both regardless of the order.

According to this embodiment, it is possible to easily create a new test pattern by optimizing an existing test pattern in correspondence with software in which a new function is added or modified. When optimizing, changing the library execution order and changing library control parameters can improve test coverage and shorten verification time, thereby improving test quality and reducing test man-hours.

More specifically, for example, when executing the existing test pattern, even if the cause of the low test coverage is unknown, analyzing the test coverage data for each test pattern divided into library units can provide more detailed test pattern information. It is possible to determine which part has low verification capability, that is, low test coverage. Based on this, test coverage can be improved by changing the execution order of libraries and changing library parameters. In addition, library execution order changes and library parameter changes are performed automatically, reducing the man-hours for test pattern designers. Furthermore, even if the test coverage is not changed by changing the execution order of the library, the test execution time can be shortened by changing the order in which the test coverage is saturated at an early stage. And, since test pattern analysis is performed by the test pattern detection unit and test pattern optimization is automatically performed, even when using an existing test pattern that is a black box, without newly describing a coverage acquisition mechanism, New test patterns can be created.

In the second embodiment, when the target coverage cannot be achieved even with the new test pattern created in the first embodiment, software variables are set directly from the outside at the time of software verification for further coverage expansion. Is.

FIGS. 5A and 5B are configuration diagrams of the software verification apparatus according to the second embodiment. The same elements as those of the first embodiment (FIGS. 1A and 1B) are denoted by the same reference numerals.

FIG. 5A shows a case where the software verification device 20 is connected to the outside of the actual machine 1 to be tested. The actual machine 1 incorporates software to be verified. Two input units 31 and 32 are provided, an existing test pattern is input to the input unit 31, and verification priority information is input to the input unit 32. The test pattern detection unit 11, the library storage unit 12, the test coverage measurement unit 13, the coverage data storage unit 14, the test pass / fail determination unit 15, and the library optimization unit 16 in the software verification apparatus 20 are the same as those in the first embodiment (FIG. 1A). ) And the description thereof is omitted.

If the test pass / fail determination unit 15 determines that the test is successful (achieving the coverage target value), the test target machine 1 software is verified using the test pattern. If it is determined that the test has failed, the test result is returned to the coverage data storage unit 14 and is output to the library optimization determination unit 21. The library optimization determination unit 21 determines whether further library optimization, that is, further expansion of test coverage is possible. If possible, the library optimization unit 16 is instructed to optimize the library. At that time, the library optimizing unit 16 optimizes the library based on the test result obtained from the coverage data storage unit 14 and the verification priority information input from the input unit 32.

The optimization determining unit 21 sends a variable setting operation start signal to the variable control unit 24 when further library optimization (further expansion of test coverage) is impossible. The verification information monitoring unit 22 is connected to the test target real machine 1, monitors and extracts the test pattern variable combination information during the software verification operation, and stores the variable combination information in the variable information storage unit 23. The variable information storage unit 23 extracts unexecuted variable combination information that has not been tested, and outputs the unexecuted variable combination information to the variable control unit 24. Coverage data is input to the variable control unit 24 from the coverage data storage unit 14 in addition to unimplemented variable combination information. Based on the two pieces of information, the variable control unit 24 makes a one-to-one correspondence between the location where the verification failure has occurred and the unexecuted variable combination at that time, and sets the variable combination information where the verification has failed to the test target actual machine 1. Output. The test coverage measurement unit 13 measures the test coverage again and executes variable control until the coverage target value is achieved. In addition to variable control, the library optimization determination unit 21 may instruct the library optimization unit 16 to perform library optimization when it is determined that library optimization is possible on the way.

FIG. 5B shows a case where the software verification device 20 is installed inside the actual machine 1 to be tested. In this case, the software verification device 20 is connected to the test target software storage module 2 in the test target real machine 1, and the test target software storage module 2 stores software to be verified. The internal configuration of the software verification device 20 is the same as that in FIG. 5A, and repeated description is omitted.

FIG. 6 is a diagram schematically illustrating the software verification method. Here, when the created new test pattern 108 cannot achieve the target coverage value, the variable control information 204 is created for further coverage expansion, and the variables are set directly from the outside using hardware. The process from the input existing test pattern 101 to the creation of a new test pattern 108 is the same as that shown in FIG.

∙ If a verification failure occurs even if the new test pattern 108 is used and the coverage target value cannot be achieved, software variable control is performed during the verification operation. When executing the new test pattern 108, the verification information monitoring unit 22 monitors the verification state, extracts the variable combination used in the verification, and stores it as variable combination information 201 in the variable information storage unit 23. Further, the coverage data 202 when the new test pattern 108 is executed is measured and stored in the coverage data storage unit 14. The variable information storage unit 23 refers to the variable combination information 201, extracts variable combinations that have not been verified (verification is omitted), and generates unexecuted variable combination information 203.

The variable control unit 24 generates variable control information 204 using the unexecuted variable combination information 203 and the coverage data 202. That is, the location of the combination of variables in which verification failure has occurred is identified from the coverage data 202, and unexecuted variable combination information that could not be verified is output as variable control information 204. Using this variable control information 204, for example, the hardware setting is changed directly from an external port (serial, GPIO (General Purpose Input / Output), debug pin, etc.) to forcibly set the variable. Thereby, a verification omission location is complemented and a coverage target value is achieved.

FIG. 7 is a flowchart showing the software verification process. This process consists of a new test pattern creation process in the first half and a variable setting process for verification omission in the second half. The new test pattern creation process (S701 to S705, S707) in the first half is the same as that in FIG. 3, FIG. 4A, and FIG.

If it is determined in S705 that the new test pattern has not reached the target coverage, the process proceeds to S706. In S706, the library optimization determination unit 21 determines whether or not library optimization is possible, that is, whether or not there is a possibility that coverage is improved. If library optimization is possible, the process advances to step S707, and the library optimization unit 16 performs library optimization such as changing the library execution order and changing library control parameters to generate a new test pattern.

If it is determined in step S706 that library optimization is impossible, that is, if the generated target pattern cannot be achieved even with the generated new test pattern, the process proceeds to step S708 and subsequent steps to the variable control unit 24 during software verification. Instructs to set variables from outside.

In S708, a verification test of the actual software is performed with the new test pattern, and the verification information monitoring unit 22 monitors the verification information. In S <b> 709, information 201 such as a variable combination at the time of test execution is extracted from the verification information and stored in the variable information storage unit 23. At this time, coverage data 202 is also measured from the verification information. In S710, the variable information storage unit 23 refers to the variable combination information 201, and extracts information 203 on variable combinations that have not been tested. In S711, the unexecuted variable combination information 203 and the measured coverage data 202 are associated on a one-to-one basis, and variable combination information of a place where a verification failure has occurred is specified.

In S712, the variable control unit 24 generates variable control information 204 based on the unexecuted variable combination information 203 and executes variable control. That is, a verification test is performed by forcibly setting a variable from an external port. In S713, the test coverage measurement unit 13 measures the test coverage after the variable control. In S714, the test pass / fail determination unit 15 determines whether the target coverage has been reached. If the target is reached in the determination, the process ends here. If the target has not been reached, the process proceeds to S715, the variable setting timing is adjusted, and the variable corresponding to this is set at the timing when the verification failure occurs. Returning to S712, the timing is adjusted while monitoring the verification state, and variable control is performed again. In this way, the variable setting is repeated to reach the target coverage.

According to the present embodiment, by using the variable combination information and test coverage data at the time of the verification test, the location where the verification failure occurs is associated with the variable combination that has not been executed at that time, It is forcibly set using hardware. As a result, it is possible to reliably eliminate verification failure.

More specifically, it is possible to eliminate omission of verification even when a range that cannot be covered by verification occurs in a newly created test pattern. As a result, the verification range is expanded and the coverage target value can be achieved. In addition, since the variable control by hardware direct control is performed after expanding the verification range as much as possible with new test patterns, the types of variables that need to be controlled can be reduced, even in actual machines where there are not enough terminals, pads, etc. for debugging Variable control is possible.

1: Actual machine to be tested
2: Test target software storage module,
10: Test pattern creation device,
11: Test pattern detection unit,
12: Library storage unit,
13: Test coverage measurement unit,
14: Coverage data storage unit,
15: Test pass / fail judgment unit,
16: Library optimization unit,
20: Software verification device,
21: Library optimization determination unit,
22: Verification information monitoring unit,
23: Variable information storage unit,
24: Variable control unit,
101: Existing test pattern,
103: Coverage data,
104: Verification priority information,
108: New test pattern,
201: variable combination information,
202: Coverage data,
203: Unimplemented variable combination information,
204: Variable control information.

Claims

In a test pattern creation device that creates a test pattern used for software verification,
A test pattern detection unit that divides the input test pattern into a predetermined unit and creates a library;
A test coverage measurement unit that measures the test coverage by verifying the test pattern in a library for each library;
A library optimization unit that changes the execution order of the libraries so that the test coverage is expanded based on the measured test coverage;
A test pattern creation device comprising:
In a test pattern creation device that creates a test pattern used for software verification,
A test pattern detection unit that divides the input test pattern into a predetermined unit and creates a library;
A test coverage measurement unit that measures the test coverage of each library in a test pattern that has been made into a library;
A library optimization unit that changes the control parameters of the library so that the test coverage is expanded based on the measured test coverage;
A test pattern creation device comprising:
In the test pattern creation device according to claim 1 or 2,
The library optimizing unit uses verification priority order information indicating a priority order of verification items in order to expand test coverage.
In the test pattern creation device according to claim 3,
The library optimization unit, if the test coverage does not expand even if the execution order of the library is changed, change the control parameter of the library,
If the test coverage does not expand even if the control parameters of the library are changed, the execution order of the library is changed.
In a software verification device that verifies software using a test pattern created by the test pattern creation device according to any one of claims 1 to 4,
A verification information monitoring unit that monitors the software verification status and extracts a combination of software variables used at the time of verification;
A variable information storage unit for storing the extracted variable combinations and extracting unexecuted variable combinations at the time of verification;
Using the unexecuted variable combination, a variable control unit that sets a software variable from an external terminal of the verification target device incorporating the software;
A software verification apparatus comprising:
The software verification device according to claim 5,
A test coverage measuring unit for measuring test coverage at the time of the software verification;
The variable control unit associates the unexecuted variable combination with the measured test coverage on a one-to-one basis, and sets a software variable using a variable combination at a location where a verification failure has occurred Verification device.
The software verification device according to claim 6,
The variable control unit, when setting the software variable, sets a variable corresponding thereto at a timing when a verification failure occurs.
The software verification device according to claim 5, wherein
A library optimization determination unit that determines whether or not further test coverage can be expanded by the library optimization unit for the created test pattern;
A software verification apparatus characterized by setting a software variable by the variable control unit when it is determined that the test coverage cannot be expanded.
In a test pattern creation method for creating a test pattern used for software verification,
Dividing the input test pattern into a predetermined unit and creating a library;
The step of verifying the test pattern that is made into a library and measuring the test coverage for each library,
Changing the execution order of the libraries so as to expand the test coverage based on the measured test coverage, or changing the control parameters of the library;
A test pattern creation method characterized by comprising:
The test pattern creation method according to claim 9,
A test pattern creation method using verification priority information indicating a priority order of verification items in order to expand the test coverage.
The test pattern creation method according to claim 10,
If the test coverage does not expand even if the execution order of the library is changed, the control parameter of the library is changed,
If the test coverage does not increase even if the control parameters of the library are changed, the execution order of the library is changed.
A software verification method for verifying software using a test pattern created by the test pattern creation method according to claim 9,
Monitoring the software verification status and extracting a combination of software variables used during verification;
Extracting the unexecuted variable combinations at the time of verification with reference to the extracted variable combinations;
Using the unexecuted variable combination to set a software variable from an external terminal of the verification target device incorporating the software;
A software verification method comprising:
The software verification method according to claim 12, wherein
Measuring test coverage during the software verification;
Correlating the unexecuted variable combination with the measured test coverage on a one-to-one basis,
A software verification method characterized in that a software variable is set using a variable combination at a place where a verification failure has occurred.
The software verification method according to claim 13,
A software verification method characterized in that, when setting the software variable, a variable corresponding thereto is set at a timing when a verification failure occurs.