JP2023020006A - Test condition determination apparatus and test condition determination method - Google Patents

Abstract

To provide a technology for easily conducting a security test.SOLUTION: A test condition determination apparatus includes an analysis unit and a calculation unit. The analysis unit extracts contents of predetermined items from a specification document regarding a test object on which a security test is conducted. The calculation unit calculates a threshold to determine whether the test object is normal with a test condition, on the basis of the extracted contents and information on the test condition which can be conducted as the security test.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a test condition determination device and a test condition determination method.

In fuzzing, which is one of the security tests, if the test target is a device, by monitoring the behavior of the device to which incorrect data is input intentionally, it is determined whether the device is normal or abnormal. For example, in Patent Document 1, another unit of an in-vehicle ECU (Electronic Control Unit) monitors the output signal from the ECU, and based on the monitoring result and the test conditions preset in the output signal, the ECU is normal. There have been proposed techniques for judging whether or not there is abnormality. Further, for example, in Patent Document 2, side channel information of a test object (for example, electromagnetic waves, current, heat, sound information, etc.) is measured, and an abnormality is detected by comparing the measurement result with the side channel information during normal or abnormal conditions. A technique for determining is proposed.

Japanese Patent No. 6718960 JP 2019-572255 A

In a fuzzing test performed using a black box, it is difficult for the user to determine the threshold for judging whether the test object is normal or abnormal. For example, when a test is executed and a phenomenon that seems to be delayed occurs in the communication function to be tested, it is necessary to determine whether the phenomenon is an "abnormality" in the specifications and a problem in the test object. It is difficult to determine whether the test is "normal" and there is no problem with the test object. Therefore, there is a problem that security tests such as fuzzing cannot be easily performed.

Therefore, the present disclosure has been made in view of the above-described problems, and aims to provide a technology that enables easy security testing.

The test condition determination device according to the present disclosure includes an analysis unit that extracts the contents of a predetermined item from the specifications of the test target on which the security test is performed, the extracted contents, and the security test and a calculation unit for calculating a threshold value for determining whether the test object is normal or abnormal under the test conditions, based on information on test conditions that can be implemented.

According to the present disclosure, since a threshold value for determining whether a test target is normal or abnormal is calculated under test conditions, security testing can be easily performed.

1 is a block diagram showing the configuration of a test condition determination device according to Embodiment 1; FIG. 4 is a diagram showing the flow of information in the test condition determination device according to Embodiment 1; FIG. FIG. 3 is a diagram showing an example of specifications according to the first embodiment; FIG. 4 is a diagram showing an example of read data information according to Embodiment 1; FIG. It is an example of test case information according to the first embodiment. 4 is a flow chart showing processing of the test condition determination device according to the first embodiment; 4 is a diagram showing an example of extracted values according to Embodiment 1; FIG. 4 is a diagram showing an example of analysis data according to Embodiment 1; FIG. 4 is a diagram showing an example of threshold information according to Embodiment 1; FIG. 4 is a diagram showing an example of threshold information according to Embodiment 1; FIG. FIG. 9 is a block diagram showing the configuration of a test condition determination device according to Embodiment 2; FIG. 10 is a diagram showing the flow of information in the test condition determination device according to the second embodiment; FIG. 10 is a diagram showing an example of description of a specification sheet according to Embodiment 2; 9 is a flow chart showing processing of the test condition determination device according to the second embodiment; FIG. 10 is a diagram showing an example of extracted values according to Embodiment 2; FIG. FIG. 10 is a diagram showing an example of inquiry data according to Embodiment 2; FIG. FIG. 10 is a diagram showing an example of terminology data according to Embodiment 2; FIG. FIG. 10 is a diagram showing an example of analysis data according to Embodiment 2; FIG. FIG. 11 is a block diagram showing the hardware configuration of a test condition determination device according to another modified example; FIG. 11 is a block diagram showing the hardware configuration of a test condition determination device according to another modified example;

<Embodiment 1>
When fuzzing, which is one of security tests (also called security inspections), is performed on communication devices, two tests are assumed. The first test is to check whether the processing capability of the test target is normal by sending data to the test target that increases the communication load so as to impair the availability of the test target. The data that increases the communication load is, for example, valid data that is repeatedly transmitted in large amounts to the test target. The second test is to send invalid data with grammatical errors that deviate from the communication specification to see if the test subject can correctly process the invalid data.

In the following, an example will be described in which fuzzing for transmitting data that increases the communication load, such as the first test, is performed on the test target. In such a case, the test condition determination device according to the first embodiment calculates a threshold for determining whether the test target is normal or abnormal in fuzzing, which is a security test, as described in detail below. It is possible.

FIG. 1 is a block diagram showing the configuration of the test condition determination device according to the first embodiment. The test condition determination device of FIG. 1 is applied to, for example, a security inspection device that performs a security test.

The test condition determination device of FIG. Note that the read data information storage unit 3 and the test case information storage unit 4 may be provided outside the test condition determination device without being provided in the test condition determination device.

The analysis unit 1 performs a process of reading and extracting the contents of predetermined items from the specifications of the test target on which the security test is to be performed. In the following description, the contents to be extracted are values including numerical values and characters, but are not limited to this.

Based on the values extracted by the analysis unit 1 and information on one or more test conditions that can be implemented as a security test, the calculation unit 2 sets a threshold for determining whether the test target is normal or abnormal under the test conditions. calculate. In the following description, test conditions that can be implemented as security tests are sometimes referred to as test cases.

The read data information holding unit 3 holds, as read data information, items defined in advance as items whose values should be read and extracted by the analysis unit 1 . The test case information holding unit 4 holds, as test case information, information on test conditions that can be implemented as a security test.

FIG. 2 is a diagram showing the flow of information in the test condition determination device according to the first embodiment.

The analysis unit 1 extracts the values of the items defined by the read data information from the specifications based on the specifications given from the outside and the read data information held by the read data information holding unit 3, and extracts the values. Output the value obtained as analysis data. FIG. 3 is a diagram showing an example of specifications according to the first embodiment. FIG. 4 is a diagram showing an example of the read data information according to the first embodiment. The read data information in FIG. including.

The calculation unit 2 in FIG. 2 generates and outputs threshold information based on the analysis data output from the analysis unit 1 and the test case information held by the test case information holding unit 4 . FIG. 5 is an example of test case information according to the first embodiment, and the test case information in the example of FIG. 5 includes protocol, protocol layer, and transmittable data size for each test case. The threshold information in FIG. 2 includes a test case used for transmission to the test target and a threshold for determining whether the test target is normal or abnormal in the test case. As the threshold, for example, a transmission data size threshold and a transmission rate threshold, which is the number of packets transmitted per second (packet/s), are used.

FIG. 6 is a flow chart showing processing of the test condition determination device according to the first embodiment.

First, in step S1, the analysis unit 1 searches the specifications for items defined as synonyms of the read data information. For example, if the specification is the specification in FIG. 3 and the item to be retrieved is the "message size" defined in the synonyms of the read data information in FIG. Search for "message size" from the character string before ":". Items used for retrieval may be synonyms of the read data information as described above, or may be read data of the read data information. When the item used for retrieval is the read data of the read data information, the read data is used instead of the synonyms in the following description.

At step S2, the analysis unit 1 extracts the value of the synonym retrieved from the specification as an extraction value. For example, in step S1, when "message size" defined as a synonym in FIG. 4 is retrieved from the specification in FIG. Values "4" and "100" are extracted as extraction values. Note that learning such as AI (Artificial Intelligence) may be used for the search in step S1 and the extraction in step S2.

The analysis unit 1 performs the search in step S1 and the extraction in step S2 for all the read data in FIG. FIG. 7 is a diagram showing an example of extracted values obtained by searching in step S1 and extracting in step S2 from the specifications in FIG. 3 and the read data information in FIG.

In step S3, the analysis unit 1 selects extracted values that match the reading conditions defined in the read data information from among the extracted values obtained in step S2, generates analysis data, and calculates the analysis data. Output to part 2. For example, the extracted values of "maximum reception size" in FIG. 7 are "4" and "100", and the reading condition in FIG. Among them, the maximum value "100" is selected. FIG. 8 is a diagram showing an example of analysis data generated by selection in step S3 from the read data information of FIG. 4 and the extracted values of FIG.

In step S4, the calculation unit 2 selects a test case from the test case information for transmission to the test target based on the extracted value of the "communication protocol" of the analysis data and the protocol and hierarchy of the test case information. Determine the test cases to be used. In the first embodiment, the calculation unit 2 determines a test case of a protocol in a layer higher than or in the same layer as the extracted value of "communication protocol" of the analysis data as a test case to be used in the security test.

For example, the extracted value of "communication protocol" of the analysis data of FIG. 8 is "TCP", and the layer of "TCP" of the test case information of FIG. All test cases are determined as test cases to be used in the security test. For example, if the extraction value of "communication protocol" in the analysis data of FIG. Test cases 1 to 5 are determined as test cases to be used in the security test.

In step S5, the calculation unit 2 determines whether the test target is normal or abnormal based on the extracted value of the "maximum reception size" of the analysis data and the value of the "maximum" "data size" of the test case. Determine the transmission data size threshold for Here, the "maximum reception size" of the analysis data is the maximum size that can be normally received by the test target. Therefore, the calculation unit 2 determines the "maximum" "data size" that does not exceed the "maximum reception size" as the threshold of the transmission data size for determining whether the test target is normal or abnormal.

FIG. 9 is a diagram showing an example of threshold information generated from the test case information of FIG. 5 and the analysis data of FIG. For example, No. in FIG. The “maximum” “data size” of test case 1 is 1514 bytes, and the “maximum reception size” of the analysis data in FIG. 8 is 100 bytes. In this case, the "maximum" "data size" in FIG. 5 exceeds the "maximum reception size" in FIG. 100 bytes, No. 1 is determined as the threshold value of the transmission data size for determination.

Further, for example, No. in FIG. The “maximum data size” of test case 3 is 60 bytes, and the “maximum reception size” of the analysis data of FIG. 8 is 100 bytes. In this case, the "maximum" "data size" in FIG. 5 does not exceed the "maximum reception size" in FIG. 60 bytes which is "size", No. 3 is determined as the threshold of the transmission data size for determination.

At step S6 in FIG. 6, the calculator 2 determines whether or not there is an extracted value in the "minimum reception interval" of the analysis data. If it is determined that there is an extracted value, the process proceeds to step S7, and if it is determined that there is no extracted value, the process proceeds to step S8.

In step S7, the calculation unit 2 calculates a transmission rate threshold for determining whether the test target is normal or abnormal based on the "minimum reception interval" of the analysis data. For example, since the "minimum reception interval" of the analysis data in FIG. 8 is 10 ms, the calculator 2 calculates 100 packets/s (=1 s/10 ms) as the transmission rate threshold as shown in FIG. After step S7, the process of FIG. 6 ends.

In step S8, the calculation unit 2 calculates a transmission rate threshold for determining whether the test target is normal or abnormal, based on the "maximum number of connections" and the "restricted time" of the analysis data. Here, the "maximum number of connections" is the maximum number of simultaneous communications performed by the test target. The "constraint time" is the minimum time required for the test object to perform some processing, and the time occupied by communication must not exceed the "constraint time" in order to maintain normal operation. Therefore, the value obtained by dividing the "restricted time" by the "maximum number of connections" is substantially synonymous with the "minimum reception interval."

In view of this, the calculator 2 uses the value obtained by dividing the "restricted time" by the "maximum number of connections" as the "minimum reception interval". For example, if there is no extracted value for the "minimum reception interval" in the analysis data of FIG. used as Then, as shown in FIG. 10, the calculator 2 calculates 250 packets/s (=1 s/4 ms) as the transmission rate threshold.

Note that in step S7, if there are extracted values for the "maximum number of connections" and "restricted time" in the analysis data, in step S7, the calculation unit 2 may perform the same processing as in step S8. .

<Summary of Embodiment 1>
According to the test condition determination apparatus according to the first embodiment as described above, based on the extracted value extracted from the specification and the information on the test conditions that can be implemented as a security test, the test target is determined under the test conditions. Calculate a threshold value for determining normality or abnormality. According to such a configuration, security tests such as fuzzing can be easily performed.

<Embodiment 2>
FIG. 11 is a block diagram showing the configuration of a test condition determination device according to the second embodiment. Hereinafter, among the constituent elements according to the second embodiment, constituent elements that are the same as or similar to the above-described constituent elements are denoted by the same or similar reference numerals, and different constituent elements will be mainly described.

The test condition determination device of FIG. That is, the configuration of FIG. 11 is the same as the configuration of FIG. 1 with an inquiry unit 5 added.

If extracted values for at least some of the predetermined items are not extracted from the specification, that is, if the extracted values are insufficient, the inquiry unit 5 asks the user about at least some of the items. make an inquiry to For the inquiry unit 5, an interface device such as a display device with a touch panel is used, for example. Items to be queried hereinafter may be read data or synonyms.

FIG. 12 is a diagram showing the flow of information in the test condition determination device according to the second embodiment. The information flow in the test condition determination device according to the second embodiment includes the same information flow as in the first embodiment. In the second embodiment, in addition to this, when the extracted value read from the specification is insufficient, the analysis unit 1 asks the user about the read data for which the extracted value is not extracted. Data is output to the inquiry unit 5 . Inquiry unit 5 inquires of the user about missing reading data based on the inquiry data from analysis unit 1 , and outputs terminology data including terms answered by the user to analysis unit 1 . The analysis unit 1 extracts values to be extracted values from the specifications based on the terminology data.

FIG. 13 is a diagram showing a description example of a specification sheet according to the second embodiment. The specifications in FIG. 13 do not have the reception intervals described in FIG. 3, and describe the maximum number of terminals instead of the maximum number of connections described in FIG. An example of read data information and an example of test case information according to the second embodiment are the same as those in FIGS. 4 and 5 of the first embodiment.

FIG. 14 is a flow chart showing processing of the test condition determination device according to the first embodiment. Note that steps S11, S12, and S19 to S24 in FIG. 14 are the same as steps S1, S2, and S3 to S8 in FIG. 6 of the first embodiment, respectively. Therefore, steps S13 to S18 will be mainly described below.

In step S11, the analysis unit 1 searches for synonyms defined in the read data information from the specifications. At step S12, the analysis unit 1 extracts the value of the synonym retrieved from the specification as an extraction value.

FIG. 15 is a diagram showing an example of extracted values obtained by searching in step S11 and extracting in step S12 from the specifications in FIG. According to the specifications of FIG. 13, in the read data information of FIG. is not searched. Therefore, in FIG. 15, the extracted values of "minimum reception interval" and "maximum number of connections" are NULL, which means none.

In step S13, the analysis unit 1 determines whether or not extraction values have been extracted for all of the read data defined in the read data information. If it is determined that the extracted values have been extracted for all of the read data, the process proceeds to step S19; if it is determined that the extracted values have not been extracted for at least part of the read data, the process goes to step S14.

At step S14, the analysis unit 1 outputs inquiry data to the inquiry unit 5 for inquiring of the user about the read data for which the extraction value has not been extracted. FIG. 16 is a diagram showing an example of inquiry data output in step S14 based on the results of the extracted values in FIG. The inquiry data in the example of FIG. 16 includes read data for which no extraction value was extracted in the example of FIG. 15, that is, "minimum reception interval" and "maximum number of connections."

In step S15 of FIG. 14, the inquiry unit 5 notifies the user of the read data included in the inquiry data from the analysis unit 1. FIG. For example, if a term synonymous with the read data exists in the specification, the inquiry unit 5 inputs the term. The user is notified to input such as.

At step S16, the inquiry unit 5 receives a term that is a reply from the supplier to the inquiry. Then, the inquiry unit 5 outputs to the analysis unit 1 terminology data in which the inquired read data and the term from the user in response to the inquiry are linked. FIG. 17 is a diagram showing an example of term data for the inquiry data of FIG. The example of FIG. 17 indicates that the user has input the term "NULL" for the read data "minimum reception interval" and the term "maximum number of terminals" for the "maximum number of connections". .

In step S17, based on the read data and terms included in the term data, the analysis unit 1 performs learning using the terms as synonyms of the read data using AI or the like. Based on the learning result, the analysis unit 1 changes the read data information by adding terms from the user to the synonyms of the read data in the read data information held in the read data information holding unit 3. do.

FIG. 18 is a diagram showing an example of learned read data information obtained by changing the read data information in FIG. 4 and the terminology data in FIG. 17 in step S17. In the example of FIG. 18, "maximum number of terminals" in FIG. 17 is added as a synonym for "maximum number of connections" in the read data information in FIG.

In step S18, the analysis unit 1 performs the same processing as the search in step S11 and the extraction in step S12 to search the terms of the terminology data from the specifications and extract the values of the terms retrieved from the specifications. Extract as a value. Note that the processing in step S17 for changing the read data information held in the read data information holding unit 3 and the processing in step S18 for extracting the value of the term data from the specification are performed in the reverse order of the above order. may be performed in

After that, in steps S19 to S24, the same processes as in steps S3 to S8 of the first embodiment are performed.

<Summary of Embodiment 2>
According to the test condition determination apparatus according to the second embodiment as described above, an inquiry is made to the user regarding the read data for which the extraction value is not extracted from the specification. According to such a configuration, even if no extracted value is extracted from the specification, it is possible to obtain a threshold value for determining whether the test object is normal or abnormal under the test conditions.

Further, according to the second embodiment, learning is performed based on at least part of the read data for which an inquiry is made and terms that are answers from the user to the inquiry, and read data information is acquired based on the learning result. change. According to such a configuration, it is possible to reduce the possibility that the extraction value is not extracted from the specification as the learning progresses.

<Modification>
In the above description, the thresholds for determining whether the test object is normal or abnormal under the test conditions are the transmission data size threshold and the transmission rate threshold, but the thresholds are not limited to these. For example, the threshold for determining whether the test object is normal or abnormal under the test conditions may be a threshold for a parameter such as transmission time.

<Other Modifications>
The analysis unit 1 and the calculation unit 2 described above are hereinafter referred to as “analysis unit 1 and the like”. The analysis unit 1 and the like are implemented by a processing circuit 81 shown in FIG. That is, the processing circuit 81 includes the analysis unit 1 for extracting the content of a predetermined item from the specifications of the test target on which the security test is to be performed, a calculation unit 2 for calculating a threshold value for determining whether the test object is normal or abnormal under the test conditions, based on the condition information. Dedicated hardware may be applied to the processing circuit 81, or a processor that executes a program stored in a memory may be applied. Processors include, for example, central processing units, processing units, arithmetic units, microprocessors, microcomputers, and DSPs (Digital Signal Processors).

When the processing circuit 81 is dedicated hardware, the processing circuit 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each function of each unit such as the analysis unit 1 may be realized by a circuit in which processing circuits are distributed, or the functions of each unit may be collectively realized by one processing circuit.

When the processing circuit 81 is a processor, the functions of the analysis unit 1 and the like are realized by combining with software and the like. Software and the like correspond to, for example, software, firmware, or software and firmware. Software or the like is written as a program and stored in memory. As shown in FIG. 20, a processor 82 applied to a processing circuit 81 reads out and executes a program stored in a memory 83 to implement the functions of each section. That is, when the test condition determination device is executed by the processing circuit 81, the step of extracting the content of a predetermined item from the specification of the test target on which the security test is to be performed; and a step of calculating a threshold value for judging whether the test target is normal or abnormal under the test conditions based on information on test conditions that can be implemented as a security test. A memory 83 is provided for storing the . In other words, it can be said that this program causes a computer to execute the procedures and methods of the analysis unit 1 and the like. Here, the memory 83 is, for example, a non-volatile or Volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), drive devices thereof, etc., or any storage medium that will be used in the future There may be.

A configuration in which each function of the analysis unit 1 and the like is realized by either hardware or software has been described above. However, the configuration is not limited to this, and a configuration in which part of the analysis unit 1 and the like is realized by dedicated hardware and another part is realized by software or the like may be employed. For example, some of the functions may be realized by a processing circuit 81 as dedicated hardware, an interface, etc., and the processing circuit 81 as a processor 82 may read out and execute a program stored in the memory 83 for the rest. It is possible to realize the function by

As described above, the processing circuit 81 can implement each of the functions described above using hardware, software, etc., or a combination thereof. Note that the above also applies to the inquiry unit 5 .

Moreover, the test condition determination apparatus described above can also be applied to a test condition determination system constructed as a system by combining a plurality of devices. In this case, each function or each component of the test condition determination apparatus described above may be distributed to each device that constructs the system, or may be concentrated in any one of the devices. good.

It should be noted that it is possible to freely combine each embodiment and each modification, and to modify or omit each embodiment and each modification as appropriate.

1 analysis unit, 2 calculation unit, 5 inquiry unit.

Claims (4)

an analysis unit that extracts the contents of predetermined items from the specifications of the test target on which the security test is performed;
a calculation unit that calculates a threshold value for determining whether the test target is normal or abnormal under the test conditions, based on the extracted content and information on test conditions that can be implemented as the security test; Test condition determination device. The test condition determination device according to claim 1,
test condition determination, further comprising an inquiry unit that inquires of the user about at least some of the predetermined items when the contents of at least some of the predetermined items are not extracted from the specification document; Device. The test condition determination device according to claim 2,
The analysis unit is
A test condition determination device that learns based on the at least some of the items for which the inquiry has been made and the user's response to the inquiry, and changes the at least some of the items based on the learning result. . Extract the content of predetermined items from the specifications of the test target on which the security test is performed,
A test condition determination method for calculating a threshold value for determining whether the test target is normal or abnormal under the test conditions, based on the extracted contents and information on test conditions that can be implemented as the security test.

Images