WO2019168167A1 - Verification method, verification device, computer program, and verification system - Google Patents

Abstract

A verification method is provided with: a step S2 for receiving side channel information from a test device to which fuzz data is given; and a step S3 for verifying an operation of the test device on the basis of the received side channel information.

Description

Verification method, verification device, computer program, and verification system

The present disclosure relates to a verification method, a verification device, a computer program, and a verification system.

If an electronic device has security vulnerabilities, it may operate abnormally according to illegally input data from a third party called a so-called cyber attack. An abnormal operation refers to the inability to operate or an operation different from the original operation.

Therefore, it is important to detect a software defect in the electronic device that causes such abnormal operation as a security measure for the electronic device. For this purpose, a fuzzing test is known. The fuzzing test is a black box test method in which various data are comprehensively sent to an electronic device to be tested to detect the presence or absence of abnormal operation.

Recently, there are various electronic devices that operate by receiving data input from the outside. Some electronic devices do not have an output unit such as a display unit, and the operation status does not appear outside. In addition, there are devices that are not frequently operated externally, such as an ECU (Electronic Control Unit) for emergency braking operation of a vehicle. If the frequency of external operation is low, even if the electronic device performs an internal abnormal operation, the abnormal operation rarely appears as an external abnormality.

When such electronic devices are subject to fuzzing tests, it is difficult to detect the presence or absence of abnormal operations because abnormal operations are unlikely to appear outside. Therefore, it may be difficult to verify an electronic device by a conventional fuzzing test.

In this regard, the above paper (Non-Patent Document 1) published at the "Cryptography and Information Security Symposium" in 2017 is based on a hardware simulator that performs the same operation as an electronic device that is the subject of a fuzzing test. The verification method of the target electronic device (test device) is proposed by comparing the operation of the hardware simulator when the same data as the target electronic device is input with the operation of the target electronic device. However, with this method, cost increases and complexity due to the use of a hardware simulator are problems. Therefore, it is desired to simplify the verification of the test device.

According to an embodiment, a verification method includes receiving side channel information from a test device provided with fuzz data, verifying operation of the test device based on the received side channel information, and Is provided.

According to another embodiment, the verification apparatus performs an operation of a test device based on an interface that receives input of side channel information from a test device to which fuzz data is given, and side channel information input through the interface. And a processing unit for verifying.

According to another embodiment, the computer program is a computer program for causing a computer to function as a verification device that verifies the operation of the test device, and the computer is configured to receive side channel information from the test device to which fuzz data is given. Based on the side channel information input via the input unit, and a processing unit for verifying the operation of the test device.

According to another embodiment, the verification system measures the operation of the test device based on the measurement device that measures the side channel information from the test device to which the fuzz data is given and the side channel information that is input from the measurement device. A verification device for verification.

FIG. 1 is a diagram illustrating an example of a configuration of a verification system and a verification method. FIG. 2 is a diagram for explaining the configuration of the verification apparatus. FIG. 3 is a block diagram illustrating processing executed by the control unit of the verification apparatus. FIG. 4 is a diagram illustrating a specific example of a normal-time electromagnetic wave database. FIG. 5 is a flowchart showing an example of the flow of abnormal operation detection processing in the verification device. FIG. 6 is a diagram illustrating a specific example of the abnormal electromagnetic wave database. FIG. 7 is a flowchart showing another example of the abnormal operation detection process flow in the verification apparatus. FIG. 8 is a diagram showing another example of the abnormal time electromagnetic wave database. FIG. 9 is a flowchart showing another example of the flow of the abnormal operation detection process in the verification device. FIG. 10 is a diagram illustrating an example of the input signal (A) to the test device and the detected electromagnetic wave waveform (B) in the first embodiment. FIG. 11 is an enlarged view of the input signal (A) and the detected electromagnetic wave waveform (B) after the sixth frame in FIG. FIG. 12 is an enlarged view of an electromagnetic wave detected immediately after the sixth frame input in FIG. FIG. 13 is a diagram illustrating another example of the input signal (A) to the test device and the detected electromagnetic wave waveform (B) in the second embodiment. It is a figure for demonstrating an example of arrangement | positioning of a probe. It is a figure showing an example of the normal time electromagnetic wave database for every position with respect to a test device. It is a flowchart showing the other example of the flow of the abnormal operation detection process in a verification apparatus. FIG. 17 is a block diagram illustrating processing executed by the control unit of the verification apparatus according to the seventh embodiment. FIG. 18 is a diagram schematically showing the structure of the classifier of FIG. 17 and the learning method of the classifier.

[1. Overview of Verification Method, Verification Device, and Computer Program]

(1) The verification method included in the present embodiment includes a step of receiving side channel information from a test device to which fuzz data is given, and verifies the operation of the test device based on the received side channel information. Steps. The side channel information here is synonymous with the side channel information acquired for decryption in a decryption method called a side-channel attack. That is, the side channel information is information obtained from a channel (side channel) other than the regular input / output channel of data in the test device. The side channel information includes, for example, electromagnetic waves, power consumption, temperature, sound, and the like. The verification of the operation of the test device may be the same as the verification performed in fuzzing, for example, verification of an unknown vulnerability in the test device. The verification of the unknown vulnerability is, for example, detection of abnormal operation.

The method of verifying the operation based on the side channel information can avoid an increase in cost compared to the case of separately providing a hardware simulator for verification different from the test device, and the operation of the hardware simulator and the operation of the test device. Since it is not necessary to verify the identity of each other, the verification work can be simplified.

(2) Preferably, the step of verifying the operation of the test device includes detecting an abnormal operation of the test device. The abnormal operation is different from the operation instructed by the control signal, for example, an arithmetic operation that performs an operation to divide by zero, an arithmetic operation to calculate a square root of a negative number, an operation that accompanies a buffer overflow, and a watch Such as the expiration of a dog timer. Thereby, the abnormal operation of the test device can be easily detected.

(3) Preferably, the detection of the abnormal operation is at least one of detection of the presence / absence of the abnormal operation and detection of the type of the abnormal operation. By this method, it is possible to easily detect at least one of detection of the presence or absence of abnormal operation and detection of the type of abnormal operation.

(4) Preferably, the side channel information is at least one of electromagnetic waves leaking from the operating test device and power consumption of the test device. By this method, an abnormal operation of the test device can be detected with high accuracy with a simple apparatus using electromagnetic waves or power consumption of the test device as side channel information.

(5) Preferably, in the verification method, the verifying step includes a step of comparing the received side channel information with reference side channel information stored in a database. By this method, an abnormal operation can be detected with high accuracy by a simple process.

(6) Preferably, the database stores at least one reference side channel information during abnormal operation and during normal operation, and is stored in the database with the received side channel information in the comparing step. By comparing with the reference side channel information, the presence or absence of abnormal operation in the test device is detected in the verification step. By this method, the presence or absence of abnormal operation can be easily detected.

(7) Preferably, the database stores side channel information for each type of abnormal operation, and the received side channel information is compared with reference side channel information for each type of abnormal operation in the comparing step. Accordingly, the type of abnormal operation in the test device is detected in the verification step. By this method, it is possible to easily detect the type of abnormal operation.

(8) Preferably, the database stores reference side channel information for each type of test device, and in the comparing step, the received side channel information and the reference side channel information stored in the database are included. The reference side channel information corresponding to the type of the test device is compared. The type of the test device may be a product type of the test device or a manufacturer type of the test device, but is not limited thereto. By this method, an abnormal operation can be detected with higher accuracy according to the type of the test device.

(9) Preferably, the side channel information is time-series data, and the step of comparing is defined in the received side channel information that is a part less than the whole of the received side channel information. The information according to the period is compared with the reference side channel information stored in the database. As a result, the detection process can be facilitated rather than using the entire received side channel information, and the detection accuracy can be improved by limiting the range used for the detection process.

(10) Preferably, the side channel information is information measured from a specific position with respect to the test device, and the database stores reference side channel information for each position with respect to the test device. The received side channel information is compared with the reference side channel information stored in the database according to the position with respect to the test device that has received the channel information. The position with respect to the test device is, for example, the position where the memory of the test device is mounted, the position where the CPU is mounted, or the like. The side channel information measured from a specific position with respect to the test device is, for example, leaked electromagnetic waves, power consumption, heat, sound, and the like. As a result, for each position of the test device, it is possible to detect an abnormal operation of the apparatus mounted at that position.

(11) Preferably, in the step of verifying, when side channel information is input, learning learned to output side channel information during abnormal operation or side channel information during normal operation The model includes inputting the received side channel information and obtaining its output. By this method, an abnormal operation can be detected with high accuracy by a simple process.

(12) Preferably, the learning model is provided with a first model for classifying the type of operation based on the feature amount obtained from the side channel information, and the side channel information at the time of abnormal operation is given from the feature amount. Or a second model that determines whether the side channel information is during normal operation. The first model is, for example, a one-dimensional convolutional neural network (1D-CNN). The second model is a classification algorithm such as One class SVM (Support Vector Vector Machine). The learning model is learned by a learning method called so-called transfer learning. By using a learning model that has been machine-learned in this way, it is possible to detect an abnormal operation with high accuracy by simple processing.

(13) Preferably, the detected abnormal operation includes a calculation abnormality. The operation abnormality includes, for example, an operation that divides by zero, an operation that calculates a square root of a negative number, and the like. By this method, an operation abnormality in the test device is easily detected using the channel information.

(14) Preferably, the test device includes an in-vehicle control device. Thereby, operation | movement of a vehicle-mounted control apparatus can be verified using this verification method.

(15) The verification apparatus included in the present embodiment includes an interface that receives input of side channel information from a test device to which fuzz data is given, and a test device based on side channel information input through the interface. And a processing unit for verifying the operation. Since this verification apparatus employs the verification methods (1) to (14), it has the same effects as the verification methods described in (1) to (14).

(16) A computer program included in the present embodiment is a computer program for causing a computer to function as a verification device that verifies the operation of a test device, and the computer is connected to a side channel from a test device to which fuzz data is given. It functions as an input unit that receives input of information and a processing unit that verifies the operation of the test device based on side channel information input via the input unit. Since this computer program causes the computer to execute the verification methods (1) to (14), the computer program has the same effects as the verification methods described in (1) to (14).

(16) The verification system included in the present embodiment includes a measurement device that measures side channel information from a test device to which fuzz data is given, and a test device that is based on side channel information input from the measurement device. A verification device for verifying the operation. Since this verification system is a system that executes the verification methods (1) to (14), it has the same effects as the verification methods described in (1) to (14).

[2. Example of Verification Method, Verification Device, and Computer Program]

[First Embodiment]
<System configuration>
FIG. 1 is a diagram showing an example of a configuration of a verification system 100 according to the present embodiment. Referring to FIG. 1, a verification system 100 is an example of a verification apparatus 1 that executes a fuzz test, an input apparatus 3 that can input a control signal that is fuzz data to a test device D, and side channel information from the test device. And an electromagnetic wave measurement probe (hereinafter referred to as probe) 5.

The verification device 1 verifies the operation of the test device D by executing a fuzz test. Verification of the operation of the test device D includes detecting an abnormal operation in the test device D. The abnormal operation is, for example, an operation other than the operation specified for the input control signal. The test device D may be any device as long as it is a device that receives an input of a control signal that is fuzz data from the outside and executes an operation defined for the input control signal. Preferably, the test device D is a device that does not have a function of outputting a result of the operation, such as a display. Preferably, the test device D is a device that is mounted on another device and has a low operation frequency, for example. The other device is, for example, a vehicle, and the test device D is, for example, an ECU (Electronic Control Unit) that is an in-vehicle control device. In the following description, it is assumed that the test device D is an ECU.

The ECU is connected to an in-vehicle network that adopts a communication standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), Ethernet (registered trademark), or MOST (Media Oriented System Transport: MOST is a registered trademark) The specified operation is executed in accordance with a control signal (a CAN message or communication frame in the case of CAN) input via the in-vehicle network.

The input device 3 is connected to the test device D by wire or wireless, and can input a CAN message, which is a control signal according to the CAN communication standard, to the ECU as an example. The input device 3 may be a device independent of the verification device 1 or may be included in the verification device 1. In the following description, it is assumed that the input device 3 is an independent device from the verification device 1 and is connected to the verification device 1 so as to be communicable. The input device 3 inputs a control frame to the ECU according to the CAN message from the verification device 1.

The probe 5 acquires side channel information, which is information indicating an operation state, from the test device D without contacting the test device D, or is acquired by being connected to the test device D by wire or wireless. It is an example of an apparatus. The probe 5 receives an electromagnetic wave leaking from the test device D, which is an example of side channel information. The probe 5 can communicate with the verification device 1 by wire or wirelessly, and inputs a signal indicating the received electromagnetic wave to the verification device 1.

The verification apparatus 1 is composed of a general PC (personal computer) or the like. Using the signal input from the probe 5, an abnormal operation detection process for detecting the presence or absence of an abnormal operation of the ECU that is the test device D is executed.

FIG. 2 is a diagram for explaining the configuration of the verification apparatus 1. With reference to FIG. 2, the verification device 1 includes a control unit 11, a storage unit 12, a communication unit 13, a probe interface (I / F) 14, and a communication unit 13.

The control unit 11 includes a CPU (Central Processing Unit). The CPU of the control unit 11 includes one or a plurality of large scale integrated circuits (LSIs). In a CPU including a plurality of LSIs, the plurality of LSIs cooperate to realize the function of the CPU.

The CPU of the control unit 11 can read out an application including one or a plurality of programs stored in the storage unit 12 and execute various processes. The application can be transferred in a state of being recorded on a recording medium such as a CD-ROM or a DVD-ROM, or can be transferred by downloading from a computer device such as a server computer.

The storage unit 12 includes a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), or a nonvolatile memory element such as a ROM, and a volatile memory element such as a RAM (Random Access Memory). The nonvolatile memory element has a storage area for storing an application or data necessary for executing the application. The volatile memory element has a storage area for storing an update program to be described later. The storage unit 12 stores a database 121. The database 121 stores reference electromagnetic wave waveforms used for detection processing.

The communication unit 13 communicates with the input device 3 under the control of the control unit 11 according to the execution of the application, and instructs the input device 3 to input a control signal to the test device D.

The probe I / F 14 receives a signal transmitted from the probe 5 to the verification device 1. The signal received by the probe I / F 14 is converted into a digital signal by the AD converter 16 and input to the control unit 11. Note that the probe I / F 14 and the AD converter 16 can be substituted by connecting a commercially available oscilloscope to the verification apparatus 1.

<Verification method>
FIG. 1 further shows a method for detecting an abnormal operation in the test device D, which is an example of a verification method in the verification system 100. For example, the abnormal operation in the test device D is detected based on electromagnetic waves leaking from the test device D operating in accordance with the input control signal.

Specifically, referring to FIG. 1, the input device 3 according to the control of the verification device 1 inputs a communication frame F, which is a control signal, to the ECU that is the test device D (step S1). The ECU executes an operation instructed by the input communication frame F.

During the operation of the ECU, the probe 5 is brought close to the ECU to a range where electromagnetic waves leaking from the ECU can be received. Conversely, the probe 5 may be fixed, and the ECU that is the test device D may be arranged in a range in which the leaking electromagnetic wave can be received with respect to the probe 5. For example, the probe 5 may be plate-shaped, and the test device D may be disposed on the plate-shaped probe 5.

Probe 5 receives electromagnetic waves leaking from the ECU (step S2). A signal indicating the received electromagnetic wave is input from the probe 5 to the verification apparatus 1. The verification device 1 detects an abnormal operation of the ECU by executing an abnormal operation detection process using a signal indicating the electromagnetic wave input from the probe 5 (step S3). The detection result is notified to the user, for example, by being displayed on the display unit 15 of the verification device 1.

<Abnormal operation detection processing>
The application stored in the storage unit 12 of the verification apparatus 1 is a program that causes the control unit 11 to execute processing for detecting abnormal operation in the test device D using a signal indicating the electromagnetic wave input from the probe 5. .

FIG. 3 is a block diagram showing an abnormal operation detection process executed by the control unit 11. Referring to FIG. 3, the abnormal operation detection process includes a reading process 111, a detection process 112, and a display process 114.

Read processing 111 is processing in which the control unit 11 reads the reference electromagnetic wave waveform from the database 121. Preferably, in the reading process 111, the control unit 11 reads the reference electromagnetic wave waveform in accordance with the type of ECU used as the test device D and the attribute of the manufacturer. Preferably, in the reading process 111, the control unit 11 reads the reference electromagnetic wave waveform in accordance with a control signal input from the input device 3 to the test device D.

FIG. 4 is a diagram illustrating a specific example of a normal electromagnetic wave database 121A, which is an example of the database 121 stored in the storage unit 12 in the present embodiment. Referring to FIG. 4, in the normal electromagnetic wave database 121A, for each CAN message input from the input device 3 to the ECU that is the test device D, an electromagnetic wave waveform leaking from the ECU that normally operates in accordance with the CAN message (normal time) Electromagnetic wave waveform) is stored. In the example of FIG. 4, the electromagnetic wave waveforms for each input CAN message are shown for the identification numbers 1 to 4. The database 121 can store an electromagnetic wave waveform for a CAN message (normal CAN message) in which the ECU performs some operation and an electromagnetic wave waveform for a CAN message (other CAN message) that the ECU does not originally operate. . For example, in the example of FIG. 4, “AAABBBCC”, “DDDEEEFF”, and “GGGHHHII” are categorized and registered as normal CAN data, and “JJJJKKLL” is categorized and registered as other CAN messages. Is done.

The detection process 112 is a process for detecting an abnormal operation of the ECU. The detection process 112 includes a comparison process 113, and the control unit 11 executes the comparison process 113 in the detection process 112. The comparison process 113 is a process for comparing the reference electromagnetic wave waveform read from the database 121 by the reading process 111 and the electromagnetic wave waveform (hereinafter also referred to as a measurement waveform) indicated by the signal input from the probe 5.

In the comparison process 113, a general pattern matching method may be employed. As an example, in the comparison process 113, the control unit 11 calculates a correlation coefficient between the reference electromagnetic wave waveform and the measurement waveform. That the correlation coefficient is equal to or greater than the threshold means that the measured waveform matches the reference electromagnetic wave waveform. Conversely, a correlation coefficient less than the threshold means that the measured waveform does not match the reference electromagnetic wave waveform.

In this embodiment, since the reference electromagnetic wave waveform is a normal electromagnetic wave waveform, it is detected whether or not the measured waveform matches the normal electromagnetic wave waveform by comparing the correlation coefficient with a threshold value. When the measured waveform matches the electromagnetic wave waveform at the normal time, it means that the ECU which is the test device D performs a normal operation, that is, no abnormal operation occurs. The fact that the measured waveform does not match the normal electromagnetic wave waveform means that the ECU is not operating normally, that is, an abnormal operation is occurring. Therefore, the comparison process 113 in the first embodiment is a process for specifying whether or not there is an abnormal operation in the test device D.

The display process 114 is a process for displaying the detection result in the detection process 112 on the display unit 15. In the present embodiment, since the presence or absence of an abnormal operation in the test device D is detected in the detection process 112, a process for displaying at least one detection result of the presence or absence of an abnormal operation on the display unit 15 is performed.

<Processing flow>
FIG. 5 is a flowchart showing an example of the flow of the abnormal operation detection process executed by the control unit 11 in the present embodiment. Referring to FIG. 5, first, control unit 11 receives an input of a signal indicating an electromagnetic wave from probe 5 (step S <b> 101). From the input signal, an electromagnetic wave waveform (waveform A) which is a measurement waveform is specified.

Next, the control unit 11 reads the corresponding normal electromagnetic wave waveform (waveform B) as the reference electromagnetic wave waveform from the normal electromagnetic wave database 121A (step S103). In step S103, a normal electromagnetic wave waveform corresponding to the CAN message input from the input device 3 to the test device D is read.

The control unit 11 identifies the presence or absence of abnormal operation in the test device D by comparing the waveforms A and B (step S105). That is, when the waveform A that is the measurement waveform matches the waveform B that is the reference electromagnetic wave waveform (YES in step S107), the control unit 11 indicates that no abnormal operation is detected as a detection result of the abnormal operation of the test device D. A message indicating “normal” is displayed on the display unit 15 (step S109).

When the waveforms A and B do not match (NO in step S107), the control unit 11 displays, as the detection result of the abnormal operation of the test device D, a message indicating “abnormal” indicating that the abnormal operation has been detected. (Step S111).

<Effect of Embodiment>
In the verification system 100, the operation of the test device D is captured by electromagnetic waves. Therefore, even if the test device D is a device that does not output by operation, such as an ECU, or a device that does not have a function of performing output by operation, a hardware simulator other than the test device D is not required. In addition, the operation can be easily detected. Thereby, an apparatus structure can be made easy and an increase in cost can be suppressed. In addition, since it is not necessary to verify the identity of the operation of the test device D and the hardware simulator, the detection work can be simplified.

Further, in the verification system 100 according to the present embodiment, the presence / absence of the abnormal operation of the test device D is detected because the presence / absence of the abnormal operation of the test device D is detected by comparing the detected electromagnetic wave waveform with the normal-time electromagnetic wave waveform. Can be detected easily and with high accuracy.

In the first embodiment, the database 121 may store an abnormal electromagnetic wave waveform instead of the normal electromagnetic wave waveform. In this case, the control unit 11 of the verification apparatus 1 detects an abnormal operation by the reverse detection method described above. That is, the control unit 11 detects abnormal operation when the electromagnetic wave waveform measured from the test device D input from the probe 5 includes a waveform that matches the abnormal electromagnetic wave waveform that is the reference electromagnetic wave waveform, and does not include it. In such a case, an abnormal operation is not detected, that is, a normal operation is detected. That is, the verification of the operation of the test device D may be to detect the normal operation of the test device D. That is, the database 121 stores at least one of the normal-time electromagnetic wave waveform and the abnormal-time electromagnetic wave waveform, or both as the reference electromagnetic wave waveform, and the control unit 11 determines the measured electromagnetic wave waveform and the reference electromagnetic wave waveform. As a verification of the operation of the test device D, the presence / absence of an abnormal operation and / or the presence / absence of a normal operation can be easily detected with high accuracy. When the abnormal electromagnetic waveform is stored in the database 121, the normal CAN message and other CAN messages may not be categorized.

[Second Embodiment]
The verification of the operation of the test device D is not limited to the detection of the presence / absence of abnormal operation and / or the presence / absence of normal operation. The verification of the operation of the test device D may be detection of the type of abnormal operation. In this case, the database 121 stored in the storage unit 12 is an abnormal electromagnetic wave database 121B as shown in FIG. FIG. 6 shows another example of the database 121 stored in the storage unit 12 and shows a specific example of the abnormal electromagnetic wave database 121B.

Referring to FIG. 6, the electromagnetic wave database 121B at the time of abnormality stores an electromagnetic wave waveform (abnormality electromagnetic wave waveform) leaked from the ECU when an abnormality occurs for each type of abnormality. In the example of FIG. 6, an error to perform division by zero (division by zero), an error to calculate a negative square root (negative square root), an error due to a buffer overflow (buffer overflow), and expiration of the watchdog timer, The electromagnetic wave waveform when each of these occurs is stored. Note that the timing of FIG. 6 is not used in the second embodiment, but will be described in a fourth embodiment to be described later.

FIG. 7 is a flowchart showing an example of the flow of the abnormal operation detection process executed by the control unit 11 in the present embodiment. Referring to FIG. 7, first, control unit 11 receives an input of a signal indicating an electromagnetic wave from probe 5 (step S201). From the input signal, an electromagnetic wave waveform (waveform A) which is a measurement waveform is specified.

Next, the control unit 11 reads out each abnormal electromagnetic wave waveform (waveform B) stored in the abnormal electromagnetic wave database 121B (step S203), and uses the electromagnetic wave waveform (waveform A) input in step S201 for each abnormal time. By performing pattern matching with the electromagnetic wave waveform (waveform B), an abnormal electromagnetic wave waveform that matches the waveform A is specified (step S205). That is, the type of the corresponding abnormality is specified.

When there is no abnormal electromagnetic wave waveform that matches the electromagnetic wave waveform (waveform A) input in step S201 (NO in step S207), the control unit 11 determines that the abnormal operation is “database” as the detection result of the abnormal operation of the test device D. A message indicating “abnormality not existing in” is displayed on the display unit 15 (step S209). In step S209, a normal message indicating that no abnormal operation is detected may be output.

On the other hand, when there is an abnormal electromagnetic wave waveform that matches the waveform A (YES in step S207), the control unit 11 displays a message indicating the type of abnormality corresponding to the abnormal electromagnetic wave waveform as a detection result of the abnormal operation of the test device D. Is displayed on the display unit 15 (step S211).

<Effect of Embodiment>
In the verification system 100 according to the present embodiment, the type of abnormal operation of the test device D is easily detected because the type of abnormal operation of the test device D is detected by comparing the detected electromagnetic wave waveform and the electromagnetic wave waveform at the time of abnormality. In addition, it can be detected with high accuracy.

[Third Embodiment]
By storing in advance the detailed electromagnetic wave waveform corresponding to the type of the test device in the database 121, it is possible to detect the presence / absence of abnormal operation and / or the type of abnormal operation in more detail. For example, the reference electromagnetic wave waveform stored in the abnormal electromagnetic wave database 121B may be an electromagnetic wave waveform for each type of abnormality for each ECU type or manufacturer. This is because different types of ECUs and manufacturers have different ways of generating an abnormality, and electromagnetic wave waveforms may be different even with the same type of abnormality.

FIG. 8 is another example of the database 121 stored in the storage unit 12, and is a diagram showing a specific example of the abnormal electromagnetic wave database 121C. Referring to FIG. 8, in the abnormal electromagnetic wave database 121C, an input of a CAN message is assumed for each attribute (manufacturer) of the ECU that is the test device S from the input device 3 and each CAN message input to the ECU. The cause (type) of abnormal operation and the determination result (NG) are stored. In the example of FIG. 8, for the identification numbers 1 and 2, the electromagnetic wave waveform for each input CAN message and the type of abnormal operation when the electromagnetic wave waveform is generated for the ECU manufactured by company A, and the determination result are NG. ,It is shown.

FIG. 9 is a flowchart showing an example of the flow of the abnormal operation detection process executed by the control unit 11 in the present embodiment. Referring to FIG. 9, first, control unit 11 receives an input of a signal indicating an electromagnetic wave from probe 5 (step S301). From the input signal, an electromagnetic wave waveform (waveform A) which is a measurement waveform is specified.

Next, the control unit 11 reads the abnormal electromagnetic wave waveform (waveform B) corresponding to the type of the test device as the reference electromagnetic wave waveform from the abnormal electromagnetic wave database 121C (step S303). Note that the type of the test device is specified by a user input to the control unit 11, for example. In step S303, an abnormal electromagnetic wave waveform stored as an abnormal wave waveform assumed at the time of CAN message input according to the CAN message input from the input device 3 to the test device D is read. Here, a plurality of abnormal electromagnetic wave waveforms may be read.

The control unit 11 identifies the presence or absence of an abnormal operation assumed in the test device D by comparing the waveforms A and B (step S305). That is, when the waveform A that is the measurement waveform matches the waveform B that is the reference electromagnetic wave waveform (YES in step S307), the control unit 11 detects that the abnormal operation is detected as the detection result of the abnormal operation of the test device D. Is displayed on the display unit 15 (step S309).

As described above, when a plurality of abnormal electromagnetic wave waveforms corresponding to the CAN message input to the test device D in step S303 are read, in step S305, the control unit 11 determines the electromagnetic wave waveform (waveform A) and each waveform. By pattern matching with the abnormal electromagnetic wave waveform, the abnormal electromagnetic wave waveform matching the waveform A is specified. That is, a corresponding abnormality type is specified from a plurality of assumed abnormality types. In this case, when there is an abnormal electromagnetic wave waveform that matches the electromagnetic wave waveform (waveform A) input in step S301 (YES in step S307), the control unit 11 detects the abnormal operation as a result of the abnormal operation of the test device D. A message indicating the type of abnormality corresponding to the electromagnetic wave waveform is displayed on the display unit 15 (step S309).

On the other hand, when there is no abnormal electromagnetic wave waveform that matches the waveform A (NO in step S307), the control unit 11 indicates that the abnormal operation is “abnormality not existing in the database” as the detection result of the abnormal operation of the test device D. Is displayed on the display unit 15 (step S311). In step S311, a normal message indicating that no abnormal operation is detected may be output.

<Effect of Embodiment>
In the verification system 100 according to the present embodiment, the detected electromagnetic wave waveform and the prepared electromagnetic wave waveform are prepared by preparing in advance an electromagnetic wave waveform corresponding to the type of abnormal operation assumed for each input control signal. Thus, the type of abnormal operation of the test device D can be detected in more detail.

In the above example, it is assumed that the abnormal electromagnetic wave waveform for each type of test device D is stored in the database 121, but in combination with the first embodiment, the normal time for each type of test device D The electromagnetic wave waveform may be stored in the database 121. In this case, the presence or absence of abnormal operation can be detected with high accuracy according to the type of the test device D.

[Fourth Embodiment]
As shown in FIG. 6, the database 121 may further store information defining the comparison timing. For example, in FIG. 6, information defining the timing for comparison with the abnormal electromagnetic wave waveform is stored in the abnormal electromagnetic wave waveform database 121 </ b> B. The timing of the comparison is, for example, an elapsed time from the input of the control signal, an elapsed time from the time when a specified feature point of the electromagnetic wave waveform input from the probe 5 is detected, and the like. Information defining the timing of comparison may be stored in a database different from the database 121.

The time from when the control signal is input to when the operation according to the control signal (for example, division) is started is defined by the test device D software. Therefore, the time is known in advance by the software designer. The comparison timing defined by the information stored in the database 121 is based on the above time. For example, the timing of comparison is when the above time has elapsed since the input of the control signal, or when a time shorter than the above time by a predetermined period since the input of the control signal. In the former case, the comparison timing is the timing at which the operation according to the control signal is started. In the latter case, the comparison timing is slightly before the operation according to the control signal is started.

Since the above timing is defined as the comparison timing, in this embodiment, the control unit 11 of the verification apparatus 1 obtains the electromagnetic wave waveform obtained from the signal at the defined timing among the signals input from the probe 5. That is, the presence / absence of abnormal operation and / or abnormality is determined only by comparing the waveform of the electromagnetic wave waveform measured from the test device D at the position defined by the database 121 and the waveform stored in the database 121. The type of operation can be specified.

As a result, it is possible to specify the presence / absence of abnormal operation and / or the type of abnormal operation with higher accuracy than matching the entire measured electromagnetic wave waveform with the reference electromagnetic wave waveform, and to suppress the processing amount at high speed. Enables processing.

Further, the control unit 11 of the verification apparatus 1 determines whether or not the position where the waveform that matches the reference electromagnetic wave waveform appears among the electromagnetic wave waveforms input from the probe 5 is a specified position. The presence / absence of an operation and / or the type of abnormal operation can be specified. In this case, for each position of the electromagnetic wave waveform input from the probe 5, a database (normal electromagnetic wave waveform database 121A or abnormal electromagnetic wave waveform database 121B, 121C) storing a reference electromagnetic wave waveform used for comparison processing is prepared. May be. In this case, the control unit 11 of the verification apparatus 1 reads the reference electromagnetic wave waveform from the corresponding database and uses it for the comparison process for each position used for the comparison process among the electromagnetic wave waveforms input from the probe 5.

As a result, even if a test device outputs the same electromagnetic wave waveform even in abnormal operation, the abnormal operation of the test device can be detected with high accuracy if the timing at which the electromagnetic wave waveform is output differs depending on the abnormal operation. it can.

[Fifth Embodiment]
Side channel information is not limited to electromagnetic waves. As another example, the power consumption of the test device D may be used. In this case, the verification system 100 includes a measuring device for measuring the current value instead of the probe 5. The measuring instrument is, for example, a device that can be inserted into a power supply line of the test device D, and can measure a current value by detecting a voltage difference between both ends of a minute resistance.

In this case, the database 121 stores a change in power consumption per unit time while the test device D is operating as a reference. The control unit 11 of the verification apparatus 1 calculates a change in power consumption per unit time in the test device D by multiplying the current value obtained from the measuring instrument by a voltage value acquired in advance. The control unit 11 compares the calculated change in the power consumption with the change in the reference current value stored in the database 121 and the change in the measured current value, so that the presence or absence of abnormal operation, or Detect the type of abnormal operation.

From the above, the side channel information is at least one of electromagnetic waves leaking from the test device D and power consumption of the test device D. Alternatively, the side channel information may be other than electromagnetic waves and power consumption. Still another example of the side channel information may be heat (temperature) or sound (frequency, volume) generated during the operation of the test device D.

In the verification system 100, the side channel information is not limited to electromagnetic waves, but can be captured by other information. Thereby, the abnormal operation of the test device D can be detected with high accuracy by a simple apparatus.

[Sixth Embodiment]
More specifically, the leaked electromagnetic wave waveform differs for each device (memory, CPU, etc.) mounted on the test device D. The same applies to the power consumption, heat, sound, and the like shown in the fifth embodiment. Therefore, preferably, the probe 5 is disposed at a position corresponding to a device that detects the presence or absence of abnormal operation in the test device D.

FIG. 14 is a diagram for explaining an example of the arrangement of the probes 5. With reference to FIG. 14, the test device D includes, as an example, a CPU 51, a memory 52, and an interface (I / F) 53. FIG. 14 exemplifies a configuration in which the CPU 51, the memory 52, and the I / F 53 chip are arranged on one substrate. In the example of FIG. 14, when the coordinates are set with the substrate surface as the XY plane, the CPU 51 is arranged in the second quadrant, the memory 52 is arranged in the third quadrant, and the I / F 53 is arranged in the fourth quadrant.

When detecting the presence or absence of abnormal operation of the CPU 51 of the test device D, the probe 5 is placed in the second quadrant position 5A to receive the leaking electromagnetic wave. When detecting the presence or absence of abnormal operation of the memory 52, the probe 5 is placed in the position 5B of the third quadrant to receive the leaking electromagnetic wave. When detecting the presence / absence of abnormal operation of the I / F 53, the probe 5 is placed in the fourth quadrant position 5C to receive the leaking electromagnetic wave.

In this case, the database 121 stores a reference electromagnetic wave waveform corresponding to a device that detects the presence or absence of abnormal operation. Or the database which stores a reference electromagnetic wave waveform according to the device which detects the presence or absence of abnormal operation may be prepared. FIG. 15 is a diagram illustrating another example of the database 121 stored in the storage unit 12 and an example of a normal electromagnetic wave database 121D for each position with respect to the test device D. Referring to FIG. 15, the normal-time electromagnetic wave database 121 </ b> D has an electromagnetic wave waveform that leaks from a normally operating device (normal time) for each position corresponding to the device that detects the presence or absence of abnormal operation in the test device D. Electromagnetic wave waveform) is stored. In the example of FIG. 15, the normal-time electromagnetic wave waveforms of the CPU 51, the memory 52, and the I / F 53 are stored. In addition, in the example of FIG. 15, illustration of each concrete electromagnetic wave waveform is abbreviate | omitted.

FIG. 16 is a flowchart showing an example of the flow of the abnormal operation detection process executed by the control unit 11 in the present embodiment. Referring to FIG. 16, first, control unit 11 accepts selection of a position (for example, one of the first quadrant to the fourth quadrant) according to a device that detects presence / absence of abnormal operation in test device D. (Step S400). In step S400, a user input from an input device (not shown) may be received. When a detector (not shown) that can detect the position of the probe 5 and the verification device 1 are connected, in step S400, the detection result of the position of the probe 5 (for example, any one of positions 5A to 5C) is detected. Input may be accepted.

After the position is selected in step S400, the same processing as steps S101 to S111 in FIG. 5 is performed. That is, the control unit 11 receives an input of a signal indicating an electromagnetic wave from the probe 5 (step S401), and compares the input electromagnetic wave waveform with a reference electromagnetic wave waveform stored in the normal time electromagnetic wave database 121D (step S403). , S405). In the present embodiment, in step S403, the control unit 11 reads the reference electromagnetic wave waveform corresponding to the selected position from the normal electromagnetic wave database 121D. The control unit 11 outputs “normal” when the input electromagnetic wave waveform matches the reference electromagnetic wave waveform, and outputs “abnormal” when it does not (steps S407 to S411).

15 and 16 are examples in the case where the reference electromagnetic wave waveform is a normal electromagnetic wave waveform. The reference electromagnetic wave waveform may be an abnormal electromagnetic wave waveform. In this case, as in the second embodiment or the third embodiment, “abnormal” is output when the input electromagnetic wave waveform matches the reference electromagnetic wave waveform corresponding to the selected position. Is done.

In the above example, the case where electromagnetic waves are measured as side channel information has been described, but the same applies to cases where the side channel information includes power consumption, heat, sound, and the like. When the side channel information is electromagnetic waves or heat, the CPU 51, the memory 52, the I / F 53, etc. are 1 in addition to the configuration in which a plurality of chips are arranged on one substrate as shown in FIG. Even in the case of a single chip, side channel information can be measured for each position. Therefore, the configuration of the test device D is not limited to the configuration of FIG.

In the verification system 100 according to the present embodiment, by using the side channel information detected from the position corresponding to the apparatus that detects the presence or absence of abnormal operation in the test device D, it is more accurate and more detailed. An abnormal operation of the test device D can be detected.

[Seventh Embodiment]
FIG. 17 is a block diagram illustrating an abnormal operation detection process executed by the control unit 11 of the verification apparatus 1 according to the seventh embodiment. With reference to FIG. 17, in the verification apparatus 1 according to the seventh embodiment, the detection process 112 includes a determination process 116 instead of the comparison process 113 illustrated in FIG. The determination process 116 is a process for determining whether the measurement waveform is a normal electromagnetic wave waveform or an abnormal electromagnetic wave waveform using the classifier 122.

A classifier 122 is stored in the storage unit 12 of the verification device 1 according to the seventh embodiment. The classifier 122 is a learning model learned to output a normal-time electromagnetic wave waveform or an abnormal-time electromagnetic wave waveform when an electromagnetic wave waveform is given. As shown in FIG. 17, the classifier 122 includes a first model 122A and a second model 122B.

The first model 122A is learned using normal-time electromagnetic wave waveforms with respect to a plurality of CAN messages. When a certain electromagnetic wave waveform is given, the first model 122A is machine-learned to output information indicating the CAN message corresponding to the electromagnetic wave waveform. It is a learning model. FIG. 18 is a diagram schematically showing the structure of the classifier 122. In the example of FIG. 18, the first model 122A is a one-dimensional convolutional neural network (1D-CNN). The first model 122A may be another deep learning model obtained by executing deep learning using a plurality of the above combinations.

Referring to FIG. 18, the first model 122A includes an input layer 61 that receives an input of an electromagnetic wave waveform, a convolution layer 62, a pooling layer 63, a total coupling layer 64, and an output layer 65.

The convolution layer 62 performs a filtering process on the input electromagnetic wave waveform to extract a feature amount. The pooling layer 63 aggregates the feature values obtained by the convolution layer 62. The total connection layer 64 combines the aggregated results in the pooling layer 63. The output layer 65 outputs information indicating the corresponding CAN message that is the classification result based on the coupling result in the all coupling layer 64.

In the first model 122A, for each CAN message, a plurality of normal electromagnetic wave waveforms corresponding to the CAN message are given as input values and learned. Specifically, referring to FIG. 18, each of CAN messages M1 to M4 is input to the ECU serving as test device D (step S501). The messages M1 to M4 are CAN messages for instructing operations such as vehicle speed acquisition, engine speed acquisition, and air flow meter value acquisition, for example.

The normal electromagnetic wave waveform from the test device D when each CAN message M1 to M4 is input is measured (step S503). In the example of FIG. 18, it is assumed that the normal-time electromagnetic wave waveforms a1 to a4 are measured with respect to the input of the CAN message M1.

The normal-time electromagnetic wave waveforms a1 to a4 measured with respect to the input of the CAN message M1 are set as input values to the first model 122A (step S505). The first model 122A receives the normal-time electromagnetic wave waveforms a1 to a4 from the input layer 61, and processes the convolutional layer 62, the pooling layer 63, and the total coupling layer 64 for each input waveform. Through the processing in, the probabilities of the CAN messages M1 to M4 are output from the output layer 65. In the case of the example of FIG. 18, a certain waveform (for example, a normal electromagnetic wave waveform a1) is input from the input layer 61, and the input waveform is classified into four classes of CAN messages M1 to M4. The output is 95%, the probability of being a CAN message M2 is 5%, the probability of being a CAN message M3 is 3%, and the probability of being a CAN message M4 is 2%.

Next, the coefficient of the function used in the calculation in each of the layers 62 to 64 is adjusted so as to increase the probability of the CAN message M1 corresponding to the inputted normal electromagnetic wave waveform a1 (step S507). In the case of the example of FIG. 18, for example, the total coupling layer is set so that the probability (95%) of the CAN message M1 output from the output layer 65 when the normal-time electromagnetic wave waveform a1 is input from the input layer 61 is higher. The weighting factor at 64 is changed.

The first model 122A is learned by performing steps S501 to S507 of FIG. 18 for each of the CAN messages M1 to M4 for the combination of the CAN message and the normal electromagnetic wave waveform measured when the CAN message is input. The accuracy of classifying the input electromagnetic wave waveform into corresponding CAN messages can be improved.

The second model 122B is a learning model that is machine-learned so as to output a determination result of whether the target electromagnetic wave waveform is a normal electromagnetic wave waveform or an abnormal electromagnetic wave waveform based on the feature quantity of the electromagnetic wave waveform. The second model 122B is a classification algorithm such as One class SVM (Support Vector Vector Machine). In the example of FIG. 18, the second model 122B is One class SVM. The second model 122B may be another deep learning model obtained by executing deep learning using a plurality of feature amounts obtained from the normal electromagnetic wave waveform.

The second model 122B is learned by a so-called transfer learning method as shown in FIG. 18 as an example. That is, learning of the first model 122A is the first-stage learning, and as the second-stage learning, the second model 122B is calculated from each electromagnetic wave waveform in the entire coupling layer 64 of the learned first model 122A. The above-described feature quantity group of the electromagnetic wave waveform is given as an input value. Specifically, referring to FIG. 18, calculation is performed for a layer including all coupling layers 64 when a normal-time electromagnetic wave waveform (for example, waveform a <b> 1) in input layer 61 of learned first model 122 </ b> A is input. The feature amount F is given to the second model 122B as an input value (step S509). The layer having the total coupling layer 64 is, for example, the second or third layer from the pooling layer 63.

The second model 122B stores the feature value F input to the second model 122B in step S509 as normal values obtained from the normal electromagnetic wave waveform a1 of the CAN message M1. And the 2nd model 122B sets the boundary surrounding the input feature-value F as the identification boundary B which is a boundary of the normal value and abnormal value of the feature-value from the measured electromagnetic wave waveform about the CAN message M1. (Step S511).

The second model 122B is learned and set for each CAN message by performing steps S509 and S511 of FIG. 18 for each of the CAN messages M1 to M4 with respect to each normal electromagnetic wave waveform measured when the CAN message is input. The accuracy of the identification boundary B can be improved.

The classifier 122 may be stored in the storage unit 12 in advance. Preferably, the control unit 11 further executes a learning process 115 for creating the classifier 122. The learning process 115 is a process represented by steps S501 to S511. In the learning process 115, at least a part of steps S501 to S511 may be executed by another device.

The determination process 116 inputs a measured waveform to the learned classifier 122, and determines from the output value whether it is a normal electromagnetic wave waveform or an abnormal electromagnetic wave waveform. That is, when the measured waveform is input to the classifier 122, the learned first model 122A calculates the feature value and gives it to the second model 122B. The learned second model 122B calculates the divergence between the identification boundary B set by learning and the feature value, and outputs information indicating the determination result of whether the electromagnetic wave waveform at normal time is the electromagnetic wave waveform at abnormal time based on the divergence. . The output information is once written in the storage unit 12.

The divergence may be, for example, data indicating whether the input feature quantity is within or outside the range surrounded by the identification boundary B. The determination process 116 reads the output value from the classifier 122 from the storage unit 12 and detects whether the measurement waveform is a normal electromagnetic wave waveform or an abnormal electromagnetic wave waveform based on the value. For example, when it is within the range surrounded by the identification boundary B, it is determined that the electromagnetic wave waveform is normal, that is, normal operation, and when it is outside the above range, it is determined that the electromagnetic wave waveform is abnormal, that is, the operation is abnormal. The determination result is once written in the storage unit 12.

By using the determination process 116 described above to detect whether the measurement waveform is a normal electromagnetic wave waveform or an abnormal electromagnetic wave waveform, the measurement waveform and the verification apparatus 1 according to the first to sixth embodiments are used. Even if random noise is included in the reference electromagnetic wave waveform, it becomes possible to detect with high accuracy. Further, even when an interrupt process or the like occurs during execution of a program of a device to be verified (for example, an ECU), the feature point interval of the measurement waveform can be detected with high accuracy.

It should be noted that the machine learning model shown in the classifier 122 may be an auto encoder as another example. The auto encoder is a learning model that is machine-learned using the feature quantity obtained by compressing the input waveform and reducing it after reducing the dimension, and the feature quantity of the input waveform. This is a machine-learned model. The learned auto encoder outputs a waveform that is almost the same as the input waveform. Therefore, when an unlearned waveform, that is, an abnormal electromagnetic wave waveform that is not a normal electromagnetic wave waveform is given as an input value, substantially the same waveform is not output. Thereby, it determines with it being an electromagnetic wave waveform at the time of abnormality.

[Example 1]
FIG. 10 is a diagram illustrating a measurement result of Example 1 in which the inventors verified the operation of the test device D using the verification apparatus 1 of the present application. In Example 1, a commercially available head-up display was used as the test device D. 10A is a control signal (CAN message) input to the test device D, and FIG. 10B is an electromagnetic wave waveform measured by the probe 5 from the test device D. 11 is an enlarged view of a part (dotted line part) of FIG. 10, and FIG. 12 is an enlarged view of a part (solid line part) of FIG.

In the measurement of the first embodiment, when the CAN messages F1 to F6 for inputting the CAN messages F1 to F6 for causing the head up display used as the test device D to operate abnormally are input, When the message F6 is input, the one having the vulnerability of abnormal operation is used.

FIG. 11 is an enlarged view of the CAN message F6 and the electromagnetic wave waveform when the CAN message F6 is input, which is a range surrounded by a dotted line in FIG. In the first embodiment, the time after the elapse of t [us] from the input of the CAN message F6 is defined as the comparison timing. Therefore, the control unit 11 of the verification apparatus 1 compares the electromagnetic wave waveform (comparison waveform) of the portion surrounded by the solid line in FIG. 11 with the reference electromagnetic wave waveform stored in the database 121.

FIG. 12 is an enlarged view of the comparative waveform of FIG. The control unit 11 of the verification device 1 compares the comparison waveform with the reference electromagnetic wave waveform, and detects that the abnormal waveform AW1 is included in the comparison waveform.

The head-up display used as the test device D in the measurement of Example 1 ends abnormally due to abnormal operation. However, in the measurement of Example 1, abnormal operation was detected using the electromagnetic wave waveform input from the probe 5 before abnormal termination due to abnormal operation. Therefore, it was verified that the operation of the test device D can be verified with high accuracy using the verification apparatus 1.

[Example 2]
FIG. 13 is a diagram illustrating a measurement result of Example 2 in which the inventors verified the operation of the test device D using the verification apparatus 1 of the present application. In the second embodiment, an arithmetic unit equipped with software is used as the test device D, and a control signal for performing an operation for dividing by a number other than zero (normal division) is input, and an operation for dividing by zero (division by zero) ) Is input when a control signal is input. FIGS. 13A and 13B show the measurement results of the electromagnetic wave waveform of the test device D in each.

The arithmetic unit used as the test device D executes division according to the control signal from the normal state (the state where division is not executed), and returns to the normal state when the division is completed. In this case, the database 121 stores the electromagnetic wave waveform being divided as the reference electromagnetic wave waveform, and the control unit 11 of the verification apparatus 1 patterns the electromagnetic wave waveform measured from the input point of the control signal with the reference electromagnetic wave waveform. Match.

Referring to FIG. 13A, when performing normal division, that is, when the test device D is performing normal operation, electromagnetic waves indicating that division is being performed for a period At1 from the input time of the control signal. A waveform W1 is detected, and an electromagnetic wave waveform W2 indicating a normal state is detected after a period At1. The period At1 was 3.5 [μs].

On the other hand, during the division by zero, that is, when an abnormal operation occurs in the test device D, an abnormal waveform AW2 indicating that the division is being executed is detected for the period At2 from the input time of the control signal. An electromagnetic wave waveform W2 indicating a normal state was detected after the passage of At2. The period At2 was 1.0 [μs].

In the second embodiment, the electromagnetic wave waveform indicating that the division is being performed is continued for a period At2 shorter than the period At1 indicating the normal division execution time, and thereafter the electromagnetic wave waveform W2 is detected, whereby the test device Division by zero at D, ie, an abnormal operation was detected. As described above, in Example 2, division by zero, that is, a specific abnormal operation was detected by the test device D using the electromagnetic wave waveform input from the probe 5. Therefore, it was verified that the operation of the test device D can be verified using the verification apparatus 1 according to the position where the electromagnetic wave waveform that matches the reference electromagnetic wave waveform among the electromagnetic wave waveforms input from the probe 5 appears.

The present invention is not limited to the above embodiment, and various modifications are possible.

DESCRIPTION OF SYMBOLS 1 Verification apparatus 3 Input apparatus 5 Probe 5A, 5B, 5C Probe position 11 Control part 12 Memory | storage part 13 Communication part 14 Probe interface 15 Display part 16 AD converter 51 CPU
52 Memory 53 Interface (I / F)
61 Input layer 62 Convolutional layer 63 Pooling layer 64 Total coupling layer 65 Output layer 100 Verification system 111 Reading process 112 Detection process 113 Comparison process 114 Display process 121 Database 121A Normal electromagnetic wave database 121B Abnormal electromagnetic wave database 121C Abnormal electromagnetic wave database 121D Normal Time electromagnetic wave database 122 Classifier 122A First model 122B Second model At1 period At2 period AW1 Abnormal waveform AW2 Abnormal waveform a1 to a4 Normal electromagnetic wave waveform B Identification boundary D Test device F Features M1 to M4 CAN message F1 to F6 CAN message W1 Electromagnetic waveform W2 Electromagnetic waveform

Claims (17)

1. Receiving side channel information from a test device provided with fuzz data;
   Verifying operation of the test device based on the received side channel information;
   A verification method comprising:
2. The verification method according to claim 1, wherein the step of verifying the operation of the test device includes detecting an abnormal operation of the test device.
3. The detection of the abnormal operation is at least one of detection of the presence or absence of the abnormal operation and detection of the type of the abnormal operation.
   The verification method according to claim 2.
4. The side channel information is at least one of electromagnetic waves leaking from the operating test device and power consumption of the test device.
   The verification method according to any one of claims 1 to 3.
5. The step of verifying comprises comparing the received side channel information with reference side channel information stored in a database;
   The verification method according to any one of claims 1 to 4.
6. The database stores reference side channel information of at least one of abnormal operation and normal operation,
   Detecting the presence or absence of abnormal operation in the test device in the verifying step by comparing the received side channel information with the reference side channel information stored in the database in the comparing step;
   The verification method according to claim 5.
7. The database stores reference side channel information for each type of abnormal operation,
   Detecting the type of abnormal operation in the test device in the verifying step by comparing the received side channel information with the reference side channel information for each type of abnormal operation in the comparing step;
   The verification method according to claim 5.
8. The database stores reference side channel information for each type of test device,
   In the comparing step, the received side channel information is compared with reference side channel information corresponding to a type of the test device among reference side channel information stored in the database.
   The verification method according to any one of claims 5 to 7.
9. The side channel information is time series data,
   In the comparing step, information corresponding to a specified period of the side channel information, which is a part smaller than the whole of the received side channel information, and a reference side channel stored in the database Compare with information,
   The verification method according to any one of claims 5 to 8.
10. The side channel information is information measured from a specific position with respect to the test device,
    The database stores reference side channel information for each position relative to the test device,
    In the comparing step, the received side channel information is compared with reference side channel information according to a position with respect to the test device that has received the side channel information, which is stored in the database.
    The verification method according to any one of claims 5 to 9.
11. The step of verifying, when side channel information is input, is received by a learning model learned to output whether it is side channel information during abnormal operation or side channel information during normal operation. 5. The verification method according to claim 1, further comprising a step of inputting the side channel information and obtaining an output thereof.
12. The learning model is
    A first model for classifying the type of motion based on the feature amount obtained from the side channel information;
    The verification according to claim 11, further comprising: a second model that is provided with the feature amount, and that determines whether the feature amount is side channel information during abnormal operation or side channel information during normal operation. Method.
13. The abnormal operation includes calculation abnormality,
    The verification method according to any one of claims 1 to 12.
14. The test device includes an in-vehicle control device,
    The verification method according to any one of claims 1 to 13.
15. An interface that accepts input of side channel information from a test device given fuzz data;
    A processing unit for verifying the operation of the test device based on the side channel information input via the interface;
    A verification apparatus comprising:
16. A computer program for causing a computer to function as a verification device for verifying the operation of a test device,
    The computer,
    An input unit for receiving input of side channel information from the test device to which fuzz data is given; and
    Based on the side channel information input via the input unit, function as a processing unit that verifies the operation of the test device,
    Computer program.
17. A measuring device for measuring side channel information from a test device provided with fuzz data;
    A verification device for verifying the operation of the test device based on the side channel information input from the measurement device;
    A verification system comprising:

Images