RU2805014C1 - Method for generating adversarial examples for intrusion detection system of industrial control system - Google Patents

Abstract

FIELD: computer engineering.

SUBSTANCE: invention is aimed at increasing the variability of the values of the selected features to increase the likelihood of generating effective adversarial examples that allow you to evade detection of an attack by an intrusion detection system. It is achieved due to the fact that when generating adversarial examples for an intrusion detection system of an industrial control system, the received data, which has the same distribution as the training data used by the intrusion detection system of an industrial control system, is additionally passed through a network device with an MTU different from MTU of industrial control system.

EFFECT: increasing the variability of the values of the selected features to increase the likelihood of generating effective adversarial examples that allow you to evade detection of an attack by an intrusion detection system.

1 cl, 2 dwg

Description

The invention relates to the field of information technology, in particular to information security, and can be used for training intrusion detection systems (IDS).

As part of the event “Methods for detecting and countering attacks involving the introduction of bookmarks and malicious code in machine learning models,” developed by the Research Center for Trusted Artificial Intelligence of the ISP RAS, approaches to implementing adversarial attacks against intrusion detection models are known. The present invention also relates to the field of adversarial learning, develops approaches proposed by the Center for Trusted Artificial Intelligence of the ISP RAS and describes a method for generating examples that implement adversarial evasion attacks against an intrusion detection system of an industrial control system.

Terms used in the text of the description of the invention:

An intrusion detection system (IDS) is a software system designed to detect unauthorized and malicious activity on a computer network or on a separate node.

Network devices are electronic devices necessary for the operation of a computer network, for example: router, switch, hub, patch panel, etc.

MTU (maximum transmission unit) is the maximum useful data block of one packet that can be transmitted by a network device (an industrial control system network as a collection of network devices) without fragmentation.

Nowadays, much attention around the world is paid to information security, in particular, the security of industrial control systems. To solve these problems, the development and implementation of intrusion detection systems, signature and non-signature, is underway. The disadvantage of signature-based IDS is the inability to detect new, previously unknown computer attacks. To detect new attacks, the use of non-signature IDS is justified, among which systems based on the use of machine learning methods are most widely used.

However, IDS of industrial control systems based on machine learning are vulnerable in certain aspects. A machine learning model can be manipulated through samples intentionally created by an attacker. An attacker, by specially selecting an example of input data, can force the model to make mistakes: a slight change in an example of input data will lead to an error in the response of the machine learning model (classifier). A set of examples of model input data on which the model makes mistakes, i.e. misclassifies is called adversarial sampling. The possibility of implementing adversarial attacks poses a potential security threat to a system based on machine learning.

Thus, conducting research in the field of adversarial learning is fundamentally important and relevant for ensuring information security. Developing methods for generating adversarial examples will make it possible to simulate the actions of a potential attacker that could lead to an IDS error and thereby allow one to evade detection of an attack and develop adequate defense mechanisms.

Known from US20210067549A1 A method for detecting and responding to an intrusion into a computer network, comprising: generating a set of adversarial training data that includes initial examples and adversarial examples by disrupting one or more initial examples using an integrated gradient attack to generate adversarial examples; encoding the source and adversarial samples to generate corresponding source samples and adversarial graph representations based on aggregation of node neighborhoods; training a graph neural network to detect anomalous activity on a computer network using an adversarial training set and performing a security action in response to the detected anomalous activity.

The main disadvantage of this method is a generalized representation of the method for generating adversarial examples, based on the use of the so-called “integrated gradient attack”. The specific steps to obtain an adversarial example from the original example are not disclosed; the subset of variable features of the feature space is not indicated; the well-known limitations in the field of generation of signs of network traffic sessions on the part of the attacker are not taken into account, for example, the impossibility of directly arbitrarily changing the values of signs of network traffic sessions: session duration, statistics of packet lengths, inter-packet delays, etc.

The closest in technical essence and performed functions, chosen as a prototype, is the Method for generating adversarial examples for an industrial control system according to the patent US20210319113A1, including the following steps:

1) listen with a sample generator to the traffic of the industrial control system to obtain data having the same distribution as the training data used by the intrusion detection system of the industrial control system, mark the received data and accept the marked anomalous data as initial examples of the attack;

2) analyze the traffic of an industrial control system, extract features including the source IP address, source port number, destination IP address, destination port number, time interval between packets, packet transmission time and packet destination code;

3) create and train a machine learning classifier based on the features extracted in step 2;

4) transform the problem of training an intrusion detection system of an industrial control system into an optimization problem using the classifier created in step 3, and solve the optimization problem to obtain adversarial examples, and the optimization problem is as follows:

x* = arg min g(x) and

d(x*, x0) < dmax,

where x0 is the initial example of the attack;

x* – generated adversarial example;

g(x) is the probability that example x* will be identified as an anomalous example (an example of an attack);

d(x*, x0) – distance between the adversarial example and the original attack example;

dmax is the maximum Euclidean distance allowed by the industrial control system, indicating that the adversarial example does not have a harmful effect if the distance is exceeded;

5) test the adversarial example created in step 4 in a real industrial control system, and if the adversarial example successfully bypasses the intrusion detection system of the industrial control system and retains the ability to attack, it is accepted as an effective adversarial example, and if the adversarial example cannot evade from the intrusion detection system of an industrial control system or to preserve the possibility of attack, the adversarial example is discarded.

The disadvantage of the prototype method is the limited variability of feature values of the feature space chosen by the authors, since the stage of listening to traffic of an industrial control system is performed under specific network settings, and not in a wide range of permissible network settings. The consequence of searching for adversarial examples in a limited area of feature definition is the relatively low probability of finding such adversarial examples.

The technical problem is the low probability of finding adversarial examples due to the limited variability of the values of the selected features.

The technical result is to increase the variability of the values of the selected features to increase the likelihood of generating effective adversarial examples that allow you to evade detection of an attack from the IDS.

The technical problem is solved and the technical result is realized due to the fact that the following sequence of actions is used in the method for generating adversarial examples for the intrusion detection system of an industrial control system:

- listening with a generator of traffic samples through a network device with the MTU of the industrial control system to obtain data having the same distribution as the training data used by the intrusion detection system of the industrial control system, marking the received data and taking the marked anomalous data as initial examples of the attack;

- further analyze the traffic of the industrial control system, extract features including the source IP address, source port number, destination IP address, destination port number, time interval between packets, packet transmission time and packet destination code;

- after which a machine learning classifier is created and trained based on the features obtained from traffic analysis;

- transform the problem of training an intrusion detection system of an industrial control system into an optimization problem using the created classifier and solve the optimization problem to obtain adversarial examples, wherein the optimization problem is as follows:

x* = arg min g(x) and

d(x*, x0) < dmax,

where x0 is the initial example of the attack;

x* – generated adversarial example;

g(x) is the probability that example x* will be identified as an anomalous example (an example of an attack);

d(x*, x0) – distance between the adversarial example and the original attack example;

dmax is the maximum Euclidean distance allowed by the industrial control system, indicating that the adversarial example does not have a harmful effect if the distance is exceeded;

- then the created adversarial example is tested in a real industrial control system and, based on the result, is accepted or not accepted as an effective adversarial example.

A distinctive feature of the method is that the received data, having the same distribution as the training data used by the intrusion detection system of the industrial control system, is additionally passed through a network device with an MTU different from the MTU of the industrial control system.

Passing a session of network traffic through a device with an MTU different from the MTU of an industrial control system, for example, the MTU was 1520 bytes, and a session of network traffic is additionally passed through a device with an MTU of 400 bytes, leads to a change in the distribution of packet lengths in the session, inter-packet delays, and session duration , the rate of arrival of packets, the characteristics of the specified distributions, etc. In this way, an increase in the variability of the values of the attributes of the original examples is achieved, the search area is expanded, and the probability of finding adversarial examples and effective adversarial examples increases. At the same time, an important fact when implementing the proposed method is taking into account known restrictions in the field of generating signs of network traffic sessions on the part of the attacker, namely the impossibility of directly arbitrarily changing the values of individual signs of network traffic sessions, because modification of the values of network traffic session attributes by an attacker is performed by an additional action, implemented in practice - passing traffic through a network device with a different MTU.

The analysis of the state of the art made it possible to establish that there are no analogues characterized by sets of features identical to all the features of the claimed method. Consequently, the claimed invention meets the patentability condition of “novelty”.

The listed new set of essential features provides an expansion of the capabilities of the prototype method due to the fact that network traffic sessions corresponding to adversarial examples are additionally passed through a network device with an MTU different from the MTU of the industrial control system.

The results of a search for known solutions in this and related fields of technology in order to identify features that coincide with the features of the claimed invention that are distinctive from the prototypes, showed that they do not follow explicitly from the prior art. The prior art determined by the applicant does not reveal the impact of the essential features of the claimed invention on achieving the specified technical result. Therefore, the claimed invention meets the patentability requirement of “inventive step”.

The “industrial applicability” of the method is determined by the technical feasibility of implementing this method.

The claimed method is illustrated by drawings:

fig. 1 is a flow diagram of a method for generating adversarial examples for an intrusion detection system of an industrial control system.

fig. 2 – scheme for generating adversarial examples for an intrusion detection system of an industrial control system.

In block 1 (Fig. 1), the sample generator (Fig. 2, block 2) listens to the traffic of the industrial control system (Fig. 2, block 3) to obtain data having the same distribution as the training data used by the industrial intrusion detection system control systems (Fig. 2, block 4).

In block 2 (Fig. 1), the data received in block 1 (Fig. 1) is additionally passed through a network device (Fig. 2, block 1) with an MTU different from the MTU of the industrial control system (Fig. 2, block 3), in As a result, the packet sizes of this data become different from the packet sizes in an industrial control system.

In block 3 (Fig. 1), the received data is marked and the marked anomalous data is taken as initial examples of the attack (Fig. 2, block 5).

In block 4 (Fig. 1), the traffic of an industrial control system is analyzed, features are extracted (Fig. 2, block 6), including the source IP address, source port number, destination IP address, destination port number, time interval between packets, packet transmission time and packet destination code.

In block 5 (Fig. 1), a machine learning classifier (Fig. 2, block 7) is created and trained based on the features (Fig. 2, block 6) extracted in block 4 (Fig. 1).

In block 6 (Fig. 1), the task of training an adversarial intrusion detection system of an industrial control system is converted into an optimization problem using a classifier (Fig. 2, block 7) created in block 5 (Fig. 1).

In block 7 (Fig. 1), the optimization problem is solved to obtain adversarial examples (Fig. 2, block 8), and the optimization problem is as follows:

x* = arg min g(x) and

d(x*, x0) < dmax,

where x0 is the initial example of the attack;

x* – generated adversarial example;

g(x) is the probability that example x* will be identified as an anomalous example (an example of an attack);

d(x*, x0) – distance between the adversarial example and the original attack example;

dmax is the maximum Euclidean distance allowed by the industrial control system, indicating that the adversarial example does not have a harmful effect if the distance is exceeded;

In block 8 (Fig. 1), an adversarial example (Fig. 2, block 8), created in block 7 (Fig. 1), is tested in a real industrial control system (Fig. 2, block 3).

In block 9 (Fig. 1), the conditions are checked whether the adversarial example bypasses the intrusion detection system of the industrial control system and whether it retains the possibility of attack. If the conditions are met, then it is accepted (Fig. 1, block 10) as an effective adversarial example (Fig. 2, block 9), and if the conditions are not met, the adversarial example is discarded (Fig. 1, block 11).

The claimed method is confirmed by an example of a program in the python programming language with the stated result achieved. The program builds a classifier and searches for adversarial examples for data obtained from industrial control system traffic with an MTU of 1520 - this data corresponds to the prototype, and data obtained from industrial control system traffic and additionally passed through a network device with an MTU of 400 – these data correspond to the additional action in the claimed method. The number of adversarial examples detected is then compared. It is expected that the additional action in the claimed method will increase the number of detected effective adversarial examples, and, therefore, increase the likelihood of generating effective adversarial examples that allow one to evade detection of an attack from the IDS. The indicated result corresponds to the declared technical result.

Below is a description of the program and the actions performed.

Loading required libraries:

import math

import pickle

from typing import List

import numpy as np

import pandas as pd

from sklearn import model_selection

from sklearn.ensemble import RandomForestClassifier

from sklearn.metrics import accuracy_score, confusion_matrix, , precision_score, recall_score

from sklearn.model_selection import train_test_split

Next, they listen to traffic through the network device with the MTU of the industrial control system (1520 bytes) to obtain data that has the same distribution as the training data used by the intrusion detection system of the industrial control system, and mark the received data. In the example, the marked received data is written to the file 'Results_1520.csv' and then read in the program. Labels corresponding to attacks (the 'GlobalLabel' attribute) are marked as "Attack", labels indicating the absence of an attack are marked as "Benign". Based on this data, a dataframe df_1520 is created:

df_1520 = pd.read_csv('Results_1520.csv')

Converting the nominal values of the 'GlobalLabel' attribute in the df_1520 dataframe into digital values ("Attack" is translated to "1", "Benign" to "0"):

df_1520['GlobalLabel']

= df_1520['GlobalLabel'].apply(lambda x: 0 if x == 'Benign' else 1)

Making the 'Packet Time' attribute equal to 1 / 'Flow Packets/s' and multiplying it by 100000 to get a value greater than 1:

df_1520['Packet Time'] = df_1520['Flow Packets/s'].apply(lambda x: 100000/x)

Removing dots in the values of the 'Source IP' and 'Destination IP' characteristics:

df_1520['Source IP'] = df_1520['Source IP'].apply(lambda x: x.replace('.',''))

df_1520['Destination IP'] = df_1520['Destination IP'].apply(lambda x: x.replace('.',''))

Feature selection:

• source IP address - 'Source IP';

• source port number - 'Source Port';

• Destination IP address - 'Destination IP';

• destination port number -'Destination Port';

• time interval between packets - 'Flow IAT Mean';

• packet transmission time - 'Packet Time':

webattack_features = ['Source IP',

'Source Port'

'Destination IP',

'Destination Port',

'Flow IAT Mean',

'Packet Time']

View selected features:

df_1520[webattack_features]

Source IP Source Port Destination IP Destination Port Flow IAT Mean Packet Time 0 8314919850 60514 1721803 443 1.634757e+05 16155.244716 1 8314919850 60516 1721803 443 5.279682e+05 50841.377774 2 8314919850 60628 1721803 443 7.404261e+05 71194.815457 3 8314919850 60630 1721803 443 1.372046e+06 131239.151936 4 8314919850 60786 1721803 443 1.007281e+06 95132.083668 ... ... ... ... ... ... ... 17941 8314919850 54480 1721803 443 3.237820e+04 2698.183333 17942 8314919850 54650 1721803 443 8.489444e+02 80.426316 17943 8314919850 54652 1721803 443 5.656823e+05 53590.957786 17944 8314919850 54734 1721803 443 5.591972e+05 52976.573547 17945 8314919850 54816 1721803 443 1.038160e+04 865.133333

17946 rows × 6 columns

Formation of the target vector of the training sample:

y = df_1520['GlobalLabel'].values

Formation of a matrix of object-features of the training sample:

X = df_1520[webattack_features].values

View the dimension of the matrix of feature objects and the target vector:

print(X.shape, y.shape)

(17946, 6) (17946,)

Dividing the training set into training (X_train, y_train) and testing (X_test, y_test):

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, shuffle=True, random_state=42)

Building a random forest classifier:

RFmodel = RandomForestClassifier(max_depth=5, n_estimators=5, max_features=3)

Training the classifier on the training set:

RFmodel.fit(X_train, y_train)

Formation of a vector of model predictions on a test sample:

y_pred = RFmodel.predict(X_test)

Obtaining the error matrix of the RFmodel on the original test sample:

matrix = confusion_matrix(y_test, y_pred)

print(matrix)

array([[1731, 61],

[ 178, 2517]])

A function that calculates and displays quality metrics:

def print_metrics(y_eval: np.ndarray, y_pred: np.ndarray, average: str = 'binary') -> List[float]:

accuracy = accuracy_score(y_eval, y_pred)

precision = precision_score(y_eval, y_pred, average=average)

recall = recall_score(y_eval, y_pred, average=average)

f1 = f1_score(y_eval, y_pred, average=average)

print('Accuracy =', accuracy)

print('Precision =', precision)

print('Recall =', recall)

print('F1 =', f1)

Output of RFmodel quality metrics on the initial test sample:

print_metrics(y_test, y_pred)

Accuracy = 0.9467350122576331

Precision = 0.9763382467028704

Recall = 0.9339517625231911

F1 = 0.954674758202162

Search for adversarial examples according to the prototype method.

Formation of the initial state of the adversarial sample:

X_test_evasion_attack = X_test.copy()

The task of training an adversarial intrusion detection system for an industrial control system is formulated as an optimization problem according to the prototype method. All sessions labeled “attack” in which the model does not make mistakes are taken from the test sample. For simplicity, the optimization problem in this example is solved by brute force: for each session the value of the “Packet Time” attribute is changed by brute force (from initial to initial+500). If it was possible to change the classifier's answer, such an example is considered adversarial. All adversarial examples are output. Changing the value of the "Packet Time" attribute in practice can be carried out by introducing a delay at the physical layer, so such a change preserves the effect of the attack and the adversarial examples detected in the presented way are effective adversarial examples.

for i in range(0, X_test.shape[0]):

if (y_test[i] == 1) and (RFmodel.predict(X_test[[i]]) == 1):

X_test_attack = X_test[[i]]

j = math.ceil(X_test[i, 5])

for Packet_Time in range(j, j + 500):

X_test_attack[0, 5] = Packet_Time

pred = RFmodel.predict(X_test_attack)

if pred[0] < 1:

print(i, Packet_Time)

X_test_evasion_attack[i, 5] = Packet_Time

break

61 53083

261 22362

400 22362

598 22362

745 53083

831 22362

893 22362

948 22362

987 22362

1003 22362

1128 22362

1257 22362

1475 22362

1592 53083

1681 22362

1827 53343

1839 22362

1851 22362

1887 22362

2202 2800

2378 22362

2421 22362

2578 22362

2725 1720

2851 53083

2978 22362

3018 22362

3033 53343

3041 22362

3135 53083

3247 22362

3488 53083

3551 53083

3555 22362

3701 2800

3852 22362

3918 53083

4055 22362

4100 22362

4117 22362

4118 53083

4237 22362

4265 22362

4291 22362

4295 53343

4377 22362

4444 22362

4452 53343

4463 53083

The total number of detected effective adversarial examples (according to the prototype method) is 49.

Comparison of model predictions for one of the vectors of the original test sample and the adversarial example:

print("Example line of initial test sample:")

print(X_test[[61]])

pred = RFmodel.predict(X_test[[61]])

print("Model prediction for the original test sample: ", pred[0])

print("Adversarial example:")

print(X_test_evasion_attack[[61]])

y_pred_evasion_attack = RFmodel.predict(X_test_evasion_attack[[61]])

print("Model prediction for adversarial example: ", y_pred_evasion_attack[0])

Example of a line from the original test sample:

[[' 8314919850' 39372 ' 1721803' 443 557563.11111111 52821.76845083388]]

Model prediction for initial test set: 1

Adversarial example:

[[' 8314919850' 39372 ' 1721803' 443 557563.11111111 53083]]

Model prediction for adversarial example: 0

The adversarial example "fools" the model: while maintaining the attack property, the classifier changed the answer from 1 ("attack") to 0 ("not an attack").

Formation of a vector of model predictions on a test sample with added adversarial examples:

y_pred_evasion_attack = RFmodel.predict(X_test_evasion_attack)

Obtaining the error matrix of the RFmodel on the test set with added adversarial examples:

matrix = confusion_matrix(y_test, y_pred_evasion_attack)

matrix

array([[1731, 61],

[227, 2468]])

Output of RFmodel quality metrics on the test set with added adversarial examples:

print_metrics(y_test, y_pred_evasion_attack)

Accuracy = 0.9358145754401604

Precision = 0.9758797943851325

Recall = 0.9157699443413729

F1 = 0.9448698315467076

The quality of the model has decreased because adversarial examples added to the test set "fool" the model.

Next, steps are performed corresponding to the additional action in the claimed method.

Data derived from the industrial control system traffic, having the same distribution as the training data used by the industrial control system intrusion detection system, is further passed through a network device with an MTU of 400 bytes and a different MTU from the industrial control system of 1520 bytes. In the example, the marked received data is written to the file 'Results_400.csv' and then read in the program. Labels corresponding to attacks (the 'GlobalLabel' attribute) are marked as "Attack", the absence of an attack is marked as "Benign". Based on this data, a dataframe df_400 is created:

df_400 = pd.read_csv('Results_400.csv')

Converting the nominal values of the 'GlobalLabel' attribute in the df_400 dataframe into digital values ("Attack" is translated to "1", "Benign" - to "0"):

df_400['GlobalLabel'] = df_400['GlobalLabel'].apply(lambda x: 0 if x == 'Benign' else 1)

Making the 'Packet Time' attribute equal to 1 / 'Flow Packets/s' and multiplying it by 100000 to get a value greater than 1:

df_400['Packet Time'] = df_400['Flow Packets/s'].apply(lambda x: 100000/x)

Removing dots in the values of the 'Source IP' and 'Destination IP' characteristics:

df_400['Source IP'] = df_400['Source IP'].apply(lambda x: x.replace('.',''))

df_400['Destination IP'] = df_400['Destination IP'].apply(lambda x: x.replace('.',''))

View the webattack_features of the df_400 dataframe:

df_400[webattack_features]

Source IP Source Port Destination IP Destination Port Flow IAT Mean Packet Time 0 8314919850 60514 1721803 443 1.634757e+05 16155.244716 1 8314919850 60516 1721803 443 5.279682e+05 50841.377774 2 8314919850 60628 1721803 443 7.404261e+05 71194.815457 3 8314919850 60630 1721803 443 1.372046e+06 131239.151936 4 8314919850 60786 1721803 443 1.007281e+06 95132.083668 ... ... ... ... ... ... ... 9725 8314919850 54480 1721803 443 3.237820e+04 2698.183333 9726 8314919850 54650 1721803 443 8.489444e+02 80.426316 9727 8314919850 54652 1721803 443 5.656823e+05 53590.957786 9728 8314919850 54734 1721803 443 5.591972e+05 52976.573547 9729 8314919850 54816 1721803 443 1.038160e+04 865.133333

9730 rows × 6 columns

Formation of the target vector of the training sample:

y_400 = df_400['GlobalLabel'].values

Formation of a matrix of object-features of the training sample:

X_400 = df_400[webattack_features].values

View the dimension of the matrix of feature objects and the target vector:

print(X_400.shape, y_400.shape)

(9730, 6) (9730,)

Dividing the training set into training (X_400_train, y_400_train) and testing (X_400_test, y_400_test):

X_400_train, X_400_test, y_400_train, y_400_test = train_test_split(X_400, y_400, test_size=0.25, shuffle=True, random_state=42)

Building a random forest classifier:

RFmodel_400 = RandomForestClassifier(max_depth=5, n_estimators=5, max_features=3)

Training the classifier on the training set:

RFmodel_400.fit(X_400_train, y_400_train)

Formation of a vector of model predictions on a test sample:

y_400_pred = RFmodel_400.predict(X_400_test)

Obtaining the error matrix of the RFmodel_400 model on the initial test sample:

matrix = confusion_matrix(y_400_test, y_400_pred)

print(matrix)

array([[1167, 64],

[28, 1174]])

Output of quality metrics for the RFmodel_400 model on the initial test sample:

print_metrics(y_400_test, y_400_pred)

Accuracy = 0.9621866009042335

Precision = 0.9483037156704361

Recall = 0.9767054908485857

F1 = 0.962295081967213

Finding adversarial examples.

Formation of the initial state of the adversarial sample:

X_400_test_evasion_attack = X_400_test.copy()

All sessions labeled “attack” in which the model does not make mistakes are taken from the test sample. For each session, the value of the "Packet Time" attribute is changed by brute force (from initial to initial+500). If it was possible to change the classifier's answer, such an example is considered adversarial. All adversarial examples are output.

for i in range(0, X_400_test.shape[0]):

if (y_400_test[i] == 1) and (RFmodel_400.predict(X_400_test[[i]]) == 1):

X_400_test_attack = X_400_test[[i]]

j = math.ceil(X_400_test[i, 5])

for Packet_Time in range(j, j + 500):

X_400_test_attack[0, 5] = Packet_Time

pred = RFmodel_400.predict(X_400_test_attack)

if pred[0] < 1:

print(i, Packet_Time)

X_400_test_evasion_attack[i, 5] = Packet_Time

break

40 26658

117 26658

186 26658

217 26658

231 26658

280 26658

321 26658

353 26658

355 2981

475 26658

500 26658

525 53758

552 26658

571 2252

581 26658

637 26658

652 53758

689 26658

720 2252

736 26658

934 26658

935 2981

941 26658

1032 26658

1184 26658

1226 26658

1229 26658

1243 2252

1287 26658

1291 26658

1605 26658

1644 26658

1655 26658

1704 26658

1778 2981

1799 26658

1856 26658

1868 26658

1889 26658

1916 26658

1953 26658

1975 26658

1984 26658

1991 26658

2014 26658

2017 3554

2039 26658

2091 26658

2187 26658

2197 26658

2265 26658

2278 26658

2320 26658

2327 26658

2331 26658

2353 26658

2365 26658

2371 26658

2410 26658

2425 26658

The total number of effective adversarial examples detected after performing the additional action of the claimed method is 60.

Comparison of model predictions for one of the vectors of the original test sample and the adversarial example:

print("Example line of initial test sample:")

print(X_400_test[[40]])

pred_400 = RFmodel_400.predict(X_400_test[[40]])

print("Model prediction for the original test sample: ", pred_400[0])

print("Adversarial example:")

print(X_400_test_evasion_attack[[40]])

y_400_pred_evasion_attack = RFmodel_400.predict(X_400_test_evasion_attack[[40]])

print("Model prediction for adversarial example: ", y_400_pred_evasion_attack[0])

Example of a line from the original test sample:

[[' 8314919850' 42692 ' 1721803' 443 280166.66666667 26542.105274403366]]

Model prediction for initial test set: 1

Adversarial example:

[[' 8314919850' 42692 ' 1721803' 443 280166.66666667 26658]]

Model prediction for adversarial example: 0

The adversarial example "fools" the model: while maintaining the attack property, the classifier changed the answer from 1 ("attack") to 0 ("not an attack").

Formation of a vector of model predictions on a test sample with added adversarial examples:

y_400_pred_evasion_attack = RFmodel_400.predict(X_400_test_evasion_attack)

Obtaining the error matrix of the RFmodel_400 model on the test set with added adversarial examples:

matrix = confusion_matrix(y_400_test, y_400_pred_evasion_attack)

print(matrix)

array([[1167, 64],

[88, 1114]])

Output of quality metrics of the RFmodel_400 model on the test set with added adversarial examples:

print_metrics(y_400_test, y_400_pred_evasion_attack)

Accuracy = 0.9375256884504727

Precision = 0.9456706281833617

Recall = 0.9267886855241264

F1 = 0.9361344537815126

The quality of the model has decreased because adversarial examples added to the test set "fool" the model.

Conclusions.

When generating adversarial examples according to the prototype method, 49 effective adversarial examples were found. After performing an additional action of the claimed method, namely additionally passing traffic through a network device with an MTU different from the MTU of the industrial control system, it was possible to detect another 60 effective adversarial examples. The total number of detected effective adversarial examples in the claimed method was 109 examples.

An additional action in the claimed method - passing traffic through a network device with an MTU different from the MTU of the industrial control system - leads to greater variability in the feature values of the original examples, expands the search area, and increases the likelihood of finding adversarial examples and effective adversarial examples.

Claims (14)

A method for generating adversarial examples for an intrusion detection system of an industrial control system, including the following sequences of actions: listen with a sample generator to traffic through a network device with a maximum useful data unit of one packet (MTU), which is transmitted by a protocol without fragmentation of an industrial control system to obtain data that has the same distribution as the training data used by the intrusion detection system of an industrial control system, mark the received data and take the labeled anomalous data as initial attack examples; perform traffic analysis of an industrial control system, extract features including source IP address, source port number, destination IP address, destination port number, time interval between packets, packet transmission time and packet destination code; create and train a machine learning classifier based on features obtained from traffic analysis; transform the problem of training an intrusion detection system of an industrial control system into an optimization problem using the created classifier and solve the optimization problem to obtain adversarial examples, wherein the optimization problem is as follows: x* = arg min g(x) and d(x*, x0) < dmax, where x0 is the initial example of the attack; x* – generated adversarial example; g(x) is the probability that example x* will be identified as an anomalous example (an example of an attack); d(x*, x0) – distance between the adversarial example and the original attack example; dmax is the maximum Euclidean distance allowed by the industrial control system, indicating that the adversarial example does not have a harmful effect if the distance is exceeded; testing the created adversarial example in a real industrial control system and, based on the result, accepting or not accepting it as an effective adversarial example; characterized in that the received data having the same distribution as the training data used by the intrusion detection system of the industrial control system is further passed through a network device with an MTU different from the MTU of the industrial control system.