JP2020524862A - Software test apparatus, software test method, and software test program - Google Patents

Abstract

The software testing apparatus uses the detection rule 101 that defines normal communication data to generate the verification software 103 for generating the expected value of the execution result that the verification target software 106 should output, and the verification software generation unit 102. Then, a test pattern is input to the verification software 103, the verification software 103 is executed, and an expected value of the execution result is generated as a test case 105. Verification for determining whether the verification target software 106 is acceptable or not by inputting the verification target software 106 to execute the verification target software 106 and comparing the obtained execution result with the expected value of the execution result of the test case 105. And an execution unit 107.

Description

The present invention relates to a software test device, a software test method, and a software test program.

In recent years, the number of cases where control devices are connected to networks has increased. In addition, IoT (Internet of Things) devices are becoming widespread. Therefore, control devices and IoT devices are increasingly targeted by cyber attacks.

In general, communication data is fixed in the control device and the IoT device. Therefore, in the control device and the IoT device, it is relatively easy to preset the detection rule defining the normal communication data. Therefore, a whitelist attack detection method that detects a cyber attack by setting detection rules in advance has been receiving attention. In the whitelist attack detection method, a detection rule is set in advance, and when the received communication data deviates from the detection rule, it is determined that the communication data is abnormal, thereby detecting a cyber attack. ..

The performance and quality of products that use the whitelist attack detection method largely depend on the number of types of applicable detection rules. When developing highly reliable attack detection software, it is necessary to test the attack detection software according to different detection rules for each system before introducing the attack detection software into the actual system. Therefore, it is necessary to create highly reliable test data according to each detection rule to be used in the mounting test.

In the prior art, test data was created manually. However, since the test data is manually generated according to the detection rule, there is a possibility that the test data may be leaked or an error may be mixed. Therefore, the generation of highly reliable test data without leakage and error is an issue.

Patent Document 1 proposes to automatically create test data instead of manually creating the test data. In Patent Document 1, test data is automatically generated from a design model such as a UML (Unified Modeling Language) class diagram and an activity diagram.

Patent Document 2 proposes to automatically create comprehensive test data. In Patent Document 2, test data is automatically generated from rules for actual operation related to train travel.

Patent Document 3 proposes to automatically generate test data in a model-based design of safety-oriented software. In Patent Document 3, test data is automatically generated from a specification model using model checking or other formal analysis technology.

JP, 2010-267023, A JP, 2014-046800, A JP, 2017-033562, A

Patent Document 1 proposes to automatically generate test data from a design model. However, Patent Document 1 has a problem that it is necessary to create a design model for test data.

Patent Document 2 proposes to automatically create comprehensive test data. However, in Patent Document 2, since the expected value of the test data is not generated, the determination of the test result cannot be automatically performed, and there is a problem that it is necessary to manually perform the determination.

Patent Document 3 proposes to automatically generate test data from a specification model using a formal analysis technique. However, Patent Document 3 has a problem that it is necessary to create a specification model for test data.

INDUSTRIAL APPLICABILITY According to the present invention, the verification result of the software to be verified can be automatically obtained only by making a simple input, without the need to generate the test data according to the detection rule used in the mounting test. , A software testing device, a software testing method, and a software testing program.

The present invention provides verification software for generating an expected value of an execution result to be output by the verification target software by using a detection rule that defines normal communication data with which a device into which the verification target software is installed communicates. The verification software generator that generates the test pattern, the test pattern generator that inputs the test pattern to the verification software, executes the verification software, and generates the expected value of the execution result as a test case Executing the verification pattern by inputting the test pattern into the verification target software, executing the verification target software, and comparing the obtained execution result with the expected value of the test case execution result And a software testing device.

According to the software testing device of the present invention, it is possible to automatically obtain the verification result of the software to be verified only by making a simple input.

1 is a block diagram showing an example of the overall configuration of a software test device according to a first embodiment of the present invention. FIG. 3 is a block diagram showing a hardware configuration example of the software test apparatus according to the first embodiment of the present invention. 5 is a flowchart showing an operation example of the software test apparatus according to the first embodiment of the present invention. 5 is a flowchart showing an operation example of a verification software generation unit in the software test apparatus according to the first embodiment of the present invention. It is the figure which showed the example of the detection rule which concerns on Embodiment 1 of this invention. It is the figure which showed the example of the rule list tree produced|generated by the test device which concerns on Embodiment 1 of this invention. It is the figure which showed the example of the decision tree produced|generated by the test apparatus which concerns on Embodiment 1 of this invention. It is the figure which showed the example of the verification software produced|generated by the test apparatus which concerns on Embodiment 1 of this invention. 9 is a flowchart showing an operation example of a verification software generation unit in the test apparatus according to the second embodiment of the present invention. It is a block diagram showing an example of whole composition of a test device concerning a 3rd embodiment of the present invention. 9 is a flowchart showing an operation example of the test apparatus according to the third embodiment of the present invention.

Embodiments of a test apparatus according to the present invention will be described below with reference to the drawings.

Embodiment 1.
FIG. 1 shows an example of the overall configuration of a software test apparatus 100 according to the first embodiment. As shown in FIG. 1, the software test apparatus 100 is configured to include a verification software generation unit 102, a test case generation unit 104, and a verification execution unit 107. The software testing device 100 is a device that verifies the software to be verified 106 and determines whether the software is acceptable or not. Here, as the verification target software 106, for example, attack detection software that detects abnormal communication data due to a cyber attack or the like will be described as an example. The verification target software 106 is software that is installed in a device such as a control device and an IoT device and determines whether or not the communication data with which the device communicates is correct.

The verification execution unit 107 inputs the test case 105 and the verification target software 106, and outputs the verification result 108.

The verification software generation unit 102 inputs the detection rule 101 and generates verification software 103. The verification software 103 is software used as a test oracle in which only defined communication data is positive.

The test case generation unit 104 generates the test case 105 by inputting the test pattern into the verification software 103 and executing the test pattern.

When the test of the verification target software 106 is performed, when the test pattern is input to the verification target software 106, the expected value of the execution result is necessary as a judgment criterion for judging whether or not the execution result is correct. is there. The test case 105 is data that is an expected value of the execution result. Therefore, when the test pattern is input to the verification target software 106 and the execution result matches the execution result of the test case 105, it can be determined that the verification target software 106 is operating correctly.

The details of each unit of the software test apparatus 100 will be described later.

FIG. 2 shows a hardware configuration example of the software test apparatus 100 shown in FIG. As shown in FIG. 2, the software test apparatus 100 according to the first embodiment is composed of a computer 200.

The computer 200 includes, as hardware, a processor 201, an auxiliary storage device 202, a memory 203, a display device 204, and an operating device 205.

The auxiliary storage device 202 stores a program that implements the functions of the verification software generation unit 102, the test case generation unit 104, and the verification execution unit 107 illustrated in FIG. 1. Further, the auxiliary storage device 202 stores the detection rule 101 and the verification target software 106 input to the software testing device 100. Further, the auxiliary storage device 202 stores the verification software 103, the test case 105, and the verification result 108 output from each unit of the software test apparatus 100.

As described above, the functions of the verification software generation unit 102, the test case generation unit 104, and the verification execution unit 107 illustrated in FIG. 1 are realized by programs. Programs for realizing the functions of the verification software generation unit 102, the test case generation unit 104, and the verification execution unit 107 are loaded into the memory 203 and executed by the processor 201.

FIG. 2 schematically illustrates a state in which the processor 201 is executing a program that implements the functions of the verification software generation unit 102, the test case generation unit 104, and the verification execution unit 107.

The display device 204 assists the operation device 205 and displays the data of the verification result 108.

The operation device 205 is used to perform a data input operation of each data such as the detection rule 101 and the verification target software 106.

Next, details of the verification software generation unit 102, the test case generation unit 104, and the verification execution unit 107 shown in FIG. 1 will be described.

The verification software generation unit 102 generates verification software 103 that determines only the defined communication data as positive based on the detection rule 101 that defines normal communication data.

The test case generation unit 104 inputs a boundary value or an arbitrary test pattern by probabilistic input to the verification software 103, causes the verification software 103 to execute, and obtains an execution result. The test case generation unit 104 generates a pair of a test pattern input to the verification software 103 and an execution result as a test case 105. The execution result included in the test case 105 is an expected value of the execution result obtained when the same test pattern is input to the verification target software 106. Therefore, hereinafter, the execution result in the test case 105 will be referred to as “expected value of execution result” or “expected value”.

The verification execution unit 107 inputs the same test pattern as the test pattern input to the verification software 103 to the verification target software 106, compares the execution result with the expected value in the test case 105, and compares the comparison result with the verification result 108. Output as.

Next, an operation example of the software test apparatus 100 will be described with reference to FIG. However, FIG. 3 shows an example S1000 of the operation flow of the software test apparatus 100, and the operation flow of the software test apparatus 100 does not necessarily have to be as shown in FIG.

In FIG. 3, first, in step S1001, the detection rule 101 is input to the software test apparatus 100.

Next, in step S1002, the verification software generation unit 102 generates verification software 103 from the detection rule input in step S1001. The detailed processing flow of the verification software generation operation will be described later.

Next, in step S1003, the test case generation unit 104 inputs the test pattern to the verification software 103 generated in step S1002.

Next, in step S1004, the test case generation unit 104 executes the verification software 103, and generates a set of the execution result and the test pattern input in step S1003 as a test case 105. The execution result included in the test case 105 is used as the “expected value of the execution result” as described above.

Next, in step S1005, the test case generation unit 104 determines whether all the test cases 105 have been generated in step S1004. If it is determined in step S1004 that all test cases 105 have been generated, the process advances to step S1006. On the other hand, if all the test cases 105 are being generated, the process returns to step S1003. The processes from step S1003 to step S1005 are repeated until generation of all the test cases 105 is completed.

In step S1006, the verification execution unit 107 inputs the test pattern in the test case 105 generated by the processing in steps S1003 to S1005 into the verification target software 106.

In step S1007, the verification execution unit 107 compares the execution result obtained by executing the verification target software 106 with the expected value of the execution result in the test case 105. If the execution result of the verification target software 106 and the expected value of the execution result in the test case 105 match, the comparison result is “match”, and if they do not match, the comparison result is output as “mismatch”. ..

In step S1008, the verification execution unit 107 determines whether the verification has been executed for all test cases 105 in step S1007. When it is determined that the verification has been executed for all the test cases 105, the process proceeds to step S1009. On the other hand, if the verification is being executed, the process returns to step S1006. The processes of steps S1006 to S1008 are repeated until the verification of all the test cases 105 is completed.

In step S1009, the verification execution unit 107 outputs all the comparison results as the verification result 108, and ends the processing of the operation flow of FIG.

Next, the detailed processing flow of the operation of “verification software generation” in step S1002 shown in FIG. 3 will be described using FIG.

In FIG. 4, in step S2001, the detection rule 101 is input to the verification software generation unit 102. An example of the detection rule 101 is shown in FIG. The detection rule 101 shown in FIG. 5 includes information such as transmission source information, transmission destination information, data length, and payload as definition items of normal communication data. In the example of FIG. 5, the detection rule 101 includes N detection rules from rule 1 to rule N. In the verification software 103 and the verification target software 106, the N pieces of communication data included in the detection rule 101 are positive, and the others are determined to be abnormal data.

In step S2002, the verification software generation unit 102 creates the rule list tree 400 by analyzing the detection rule 101 input in step S2001. An example of the rule list tree is shown in FIG. The rule list tree 400 shown in FIG. 6 represents a rule list with a definition item as a leaf and each rule as a tree.

In step S2003, the verification software generation unit 102 combines the definition items of all trees from the rule list tree 400 generated in step S2002 for each common item.

In step S2004, the verification software generation unit 102 builds the decision tree 500. An example of the decision tree 500 is shown in FIG. The decision tree 500 shown in FIG. 7 is obtained by converting the rule list tree 400 to which the definition items are combined in step S2003 into a decision tree structure, and adding unmatched leaves to each branch.

In step S2005, the verification software generation unit 102 generates a program that becomes the verification software 103 from the decision tree 500 generated in step S2004. FIG. 8 shows an example of the program 600 that becomes the verification software 103.

In step S2006, the verification software generation unit 102 outputs the program 600 generated in step S2005 as the verification software 103, and ends the processing.

The verification software 103 composed of the program 600 of FIG. 8 will be described. In the verification software 103, if the test data matches any one of the N detection rules from rule 1 to rule N shown in FIG. 5, the test data is determined to be positive. On the other hand, if the test data does not match any of these N detection rules, it is determined that the test data is abnormal and may be under cyber attack. Specifically, if the transmission source information of the test data is the transmission source information 1, the transmission destination information is the transmission destination information 1, the data length is the data length 1, and the payload is the payload 1, it matches Rule 1. The test data can be determined to be positive.

As described above, in the first embodiment, the verification software generation unit 102 generates the verification software 103 shown in FIG. 8 from the detection rule 101. The test case generation unit 104 inputs M test patterns A, B, C,..., M to the verification software 103, and outputs the respective execution results as expected values Aout, Bout, Cout, of the execution results. ..., get as Mout. The verification execution unit 107 inputs the test patterns A, B, C,..., M to the verification target software 106 to be tested, and outputs the respective execution results as execution results Aout′, Bout′, Cout′,... ··· Obtain as Mout′. The verification execution unit 107 compares the expected values Aout, Bout, Cout,..., Mout of the execution results with the execution results Aout′, Bout′, Cout′,. If the number of coincidences is equal to or more than a preset threshold value, the verification target software 106 is determined to be acceptable, and if the number of coincidences is less than the threshold value, the verification target software 106 is determined to be unacceptable. The threshold value may be set to M or may be set to a value smaller than M.

As described above, according to the software test apparatus 100 according to the first embodiment, the test of the verification target software 106 can be executed fully automatically only by inputting the detection rule 101 and the verification target software 106. Therefore, it is not necessary to manually generate a test case as in the conventional case, and the test time can be reduced.

In the software test apparatus 100 according to the first embodiment, the test cases are fully automatically generated, so that the possibility of leakage of test cases and mixing of errors due to human intervention can be reduced.

Since the software testing apparatus 100 according to the present embodiment generates the test case 105 from the detection rule 101 and executes the verification test of the verification target software 106, even if the content of the verification target software 106 is a black box and unknown. , It is possible to carry out a verification test.

The software test apparatus 100 according to the present embodiment can obtain the test case 105 as the expected value of the execution result by using the verification software 103, and thus automatically obtains the pass/fail judgment result of the verification target software 106. Therefore, it is not necessary to manually make a pass/fail judgment.

Embodiment 2.
A software test apparatus according to the second embodiment of the present invention will be described with reference to FIG. FIG. 9 shows a detailed processing flow of the operation of “verification software generation” in step S1002 shown in FIG. 3 described above.

The overall configuration of the software test apparatus according to the second embodiment is the same as the configuration of the software test apparatus 100 according to the first embodiment shown in FIG. The hardware configuration example of the second embodiment is the same as the hardware configuration of the first embodiment shown in FIG. The operation of the software test apparatus according to the second embodiment is basically the same as that of the first embodiment shown in FIG. The second embodiment is different from the first embodiment only in that the operation flow of FIG. 9 is performed instead of FIG. 4 of the first embodiment. The format verification property generated here is used to verify the match between the communication data to be inspected and the detection rule 101 of FIG. In the formal verification property, the N detection rules defined in the detection rule 101 of FIG. 5 are formally converted. For example, from the rule 1 of the detection rule 101 in FIG. 5, the format is “rule 1 == sender == sender information 1 && receiver == destination information 1 && length == data length 1 && command == payload 1” A validation property is created. Here, sender, receiver, length, and command are the contents of the communication data to be inspected.

9, step S3001, step S3002, step S3003, step S3005, step S3006, and step S3008 are respectively step S2001, step S2002, step S2003, step S2004, step S2005, and step S2006 in FIG. 4 of the first embodiment. Therefore, the description thereof is omitted here. That is, in FIG. 9, the step of “form verification property generation” of step S3004 is added to FIG. The formal verification property is a property for formally verifying the operation of the verification software 103.

9, in step S3004, the verification software generation unit 102 generates a formal verification property corresponding to the detection rule 101 from the rule list tree 400 of FIG. 6 generated in step S3002.

In step S3007, the verification software generation unit 102 uses the formal verification property generated in step S3004 for the program 600 generated in step S3006 to determine whether the detection rule 101 in FIG. 5 is correctly reflected in the program 600. Verify whether or not.

As described above, also in the second embodiment, similarly to the first embodiment, the acceptance/rejection determination of the verification target software 106 can be automatically performed only by inputting the detection rule 101 and the verification target software 106. Therefore, the same effect as that of the first embodiment can be obtained.

Further, in the second embodiment, the verification software generation unit 102 generates the formal verification property according to the detection rule 101. The formal verification using the formal verification property proves that the verification software 103 has no bug. Therefore, the highly reliable test case 105 can be generated.

Embodiment 3.
FIG. 10 shows an example of the overall configuration of a software test apparatus 100A according to the third embodiment of the present invention.

As shown in FIG. 10, the software testing apparatus 100A has a model checking unit 702 and a detection rule generating unit 703 added to the configuration of the first embodiment shown in FIG.

The model checking unit 702 inputs the communication model 701 and performs model checking.

The detection rule generation unit 703 outputs the detection rule 101 from the model-checked communication model output from the model checking unit 702.

The details of the model checking unit 701 and the detection rule generating unit 703 will be described later.

The hardware configuration example of the software test apparatus 100A according to the third embodiment is different from the hardware configuration of the software test apparatus 100 according to the first embodiment in FIG. 2 in that a model checking unit 702 and a detection rule generating unit 703 are provided. The model checking unit 702 and the detection rule generation unit 703 are realized by a program, like the verification software generation unit 102. As described above, the hardware configuration example of the software test apparatus 100A according to the third embodiment is basically the same as the hardware configuration of the software test apparatus 100 according to the first embodiment of FIG. , The description is omitted. The communication model 701 is stored in the auxiliary storage device 202 or the memory 203.

Next, details of the communication model 701, the model checking unit 702, and the detection rule generating unit 703 in the software testing apparatus 100A according to the third embodiment shown in FIG. 10 will be described. Since other configurations are the same as those in the first embodiment, the description thereof will be omitted here.

The communication model 701 is a design model of communication data communicated by a device such as a control device or an IoT device into which the verification target software 106 is installed. The communication model 701 is described by, for example, a block diagram or a state transition diagram, and includes information necessary for implementing a communication function.

The model checking unit 702 performs model checking whether the communication model 701 has an error according to a preset model checking rule. For example, the model checking unit 702 performs format verification by model checking whether deadlock occurs in the communication sequence.

The detection rule generation unit 703 receives the communication model 701 after model checking as an input, and generates the detection rule 101 defining normal communication data. The detection rule generation unit 703 extracts the specifications of the communication function included in the communication model 701 and defines the extracted communication function as a definition item of the detection rule 101.

Next, the operation of the software test apparatus 100A will be described with reference to FIG. However, FIG. 11 shows an example S4000 of the operation of the software test apparatus 100A, and the operation flow of the software test apparatus 100A may not necessarily be as shown in FIG.

In FIG. 11, first, in step S4001, the communication model 701 is input to the software test apparatus 100A.

Next, in step S4002, the model checking unit 702 formally verifies the communication model 701 input in step S4001 by model checking.

Next, in step S4003, the detection rule generation unit 703 generates the detection rule 101 from the communication model 701 model-checked in step S4002. An example of the generated detection rule 101 is shown in FIG. 5, for example.

The detection rule 101 generated in step S4003 is input to the verification software generation unit 102.

The processes of steps S4004 to S4011 of FIG. 11 are the same as steps S1002 to S1009 of the first embodiment of FIG. 3, respectively, and therefore description thereof will be omitted here.

As described above, also in the third embodiment, the acceptance/rejection determination of the verification target software 106 can be automatically performed only by inputting the communication model 701 and the verification target software 106. The same effect as that of the first embodiment can be obtained.

Further, in the third embodiment, since the detection rule 101 is generated from the communication model 701 after the model checking is performed, the generation of the detection rule 101 from the communication model 701 and the verification test of the verification target software 106 are performed. Can be realized with a fully automatic and highly reliable test scheme.

Lastly, a supplementary description of the hardware configurations of the software test apparatuses 100 and 100A of FIG. 2 will be given.

The processor 201 is an IC (Integrated Circuit) that performs processing. The processor 201 is, for example, a CPU (Central Processing Unit) or a DSP (Digital Signal Processor).

The auxiliary storage device 202 is, for example, a ROM (Read Only Memory), a flash memory, or an HDD (Hard Disk Drive).

The memory 203 is, for example, a RAM (Random Access Memory).

The display device 204 is, for example, a display, a lamp, or a voice operator.

The operation device 205 is, for example, a mouse, a keyboard, or a touch panel.

An OS (Operating System) is also stored in the auxiliary storage device 202. Then, at least part of the OS is loaded into the memory 203. While executing the OS, the processor 201 executes a program that implements the functions of the verification software generation unit 102, the test case generation unit 104, the verification execution unit 107, the model checking unit 702, and the detection rule generation unit 703. ..

When the processor 201 executes the OS, task management, memory management, file management, communication control, etc. are performed.

Further, the software test apparatuses 100 and 100A may include a plurality of processors that replace the processor 201. These multiple processors share the execution of programs that implement the functions of the verification software generation unit 102, the test case generation unit 104, the verification execution unit 107, the model checking unit 702, and the detection rule generation unit 703. Each processor is an IC that performs processing, like the processor 201.

Information and data indicating the results of the processes of the verification software generation unit 102, the test case generation unit 104, the verification execution unit 107, the model checking unit 702, and the detection rule generation unit 703 are stored in the memory 203 and the auxiliary storage. It is stored as a file in the device 202 or a register or cache memory in the processor 201.

Further, the programs that implement the functions of the verification software generation unit 102, the test case generation unit 104, the verification execution unit 107, the model checking unit 702, and the detection rule generation unit 703 are magnetic disk, flexible disk, optical disk, It may be stored in a portable storage medium such as a compact disc, a Blu-ray (registered trademark) disc, or a DVD.

In addition, the “software” generation unit 102, the test case generation unit 104, the verification execution unit 107, the model checking unit 702, and the detection rule generation unit 703 are replaced by “circuit” or “process” or “procedure”. Or “processing”.

In addition, the software test apparatuses 100 and 100A may be realized by an electronic circuit such as a logic IC (Integrated Circuit), a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array). In that case, the verification software generation unit 102, the test case generation unit 104, the verification execution unit 107, the model checking unit 702, and the detection rule generation unit 703 are each realized as a part of an electronic circuit.

The processor and the electronic circuits described above are also collectively referred to as a processing circuit.

100, 100A software testing device, 101 detection rule, 102 verification software generation unit, 103 verification software, 104 test case generation unit, 105 test case, 106 verification target software, 107 verification execution unit, 108 verification result, 201 processor, 202 auxiliary storage device, 203 memory, 204 display device, 205 operating device, 701 communication model, 702 model checking unit, 703 detection rule generation unit.

Claims (6)

Generation of verification software for generating verification software for generating an expected value that the verification software should output as an execution result using a detection rule that defines normal communication data in a device in which the verification software is installed Department,
A test case generation unit that generates a test case by inputting a test pattern into the verification software and executing the verification software,
By inputting the same test pattern as the test pattern to the verification target software, by comparing the actual execution result obtained by executing the verification target software and the expected value by the test case, And a verification execution unit that determines whether the verification target software is acceptable or not. According to a preset model checking rule, a model checking unit for checking whether or not there is an error in the communication model which is the design model of the normal communication data,
The software test apparatus according to claim 1, further comprising: a detection rule generation unit that generates the detection rule based on the communication model after the model inspection. The verification software generation unit,
As a definition item that defines the normal communication data, the detection rule including at least one information of transmission source information, transmission destination information, data length, and payload is input,
Analyzing the detection rule detection rule, the definition item as a leaf, each rule as a tree, to generate a rule list tree,
From the rule list tree, combine the definition items of all trees for each common item to generate a decision tree,
Generating the verification software from the decision tree,
The software test apparatus according to claim 1 or 2. The verification software generation unit,
Generate a formal verification property according to the detection rule from the rule list tree,
Formally verifying the verification software generated from the decision tree using the formal verification property,
The software testing device according to claim 3. Generation of verification software for generating verification software for generating an expected value that the verification software should output as an execution result using a detection rule that defines normal communication data in a device in which the verification software is installed Steps,
A test case generation step of generating a test case of the expected value by inputting a test pattern into the verification software and executing the verification software;
By inputting the same test pattern as the test pattern to the verification target software, by comparing the actual execution result obtained by executing the verification target software and the expected value by the test case, And a verification execution step of judging whether the verification target software is acceptable or not. On the computer,
Generation of verification software for generating verification software for generating an expected value that the verification software should output as an execution result using a detection rule that defines normal communication data in a device in which the verification software is installed Steps,
A test case generation step of generating a test case of the expected value by inputting a test pattern into the verification software and executing the verification software;
By inputting the same test pattern as the test pattern into the verification target software and comparing the actual execution result obtained by executing the verification target software with the expected value of the test case, A software test program for executing the verification execution step of judging whether the verification target software is acceptable or not.

Images