KR102279983B1 - Network Intrusion Detection Method using unsupervised deep learning algorithms and Computer Readable Recording Medium on which program therefor is recorded - Google Patents

Abstract

The present invention relates to an unsupervised network intrusion detection method using a deep learning algorithm and a recording medium in which a program for executing the same is recorded.
An unsupervised method for network intrusion detection using a deep learning algorithm according to an example of the present invention, and a recording medium on which a program for executing the same is recorded, pre-processes a data packet input from the outside in a one-hot encoding method a preprocessing step of generating an original vector expressed as a vector value; A compression and restoration step of compressing the original vector to a lower dimension using an autoencoder method and then restoring it to the original dimension to generate a restored vector expressed as a vector value; a loss value calculation step of calculating a loss value by calculating a difference value between the original vector and the restored vector; and a determination step of determining the data packet as abnormal data if the calculated loss value is greater than the threshold, and determining the data packet as normal data if the calculated loss value is less than the threshold. It is determined using an operating characteristic curve, ROC curve), and a point having a high true positive rate and a low false positive rate in the receiver operating characteristic curve may be selected as a threshold value.

Description

Network Intrusion Detection Method using unsupervised deep learning algorithms and Computer Readable Recording Medium on which program therefor is recorded}

The present invention relates to an unsupervised network intrusion detection method using a deep learning algorithm and a recording medium in which a program for executing the same is recorded.

Intrusion Detection Systems (IDS) are used to protect systems and information assets from various types of network attacks.

Typically, experts analyze the traffic and logs collected through IDS to identify the exact cause and pattern of the attack. This analysis process can take anywhere from several days to several months, depending on the type of threat. If the response is delayed, there is a problem in that the recovery cost greatly increases.

Accordingly, in order to solve the problem of intrusion detection, research on automating the detection of security threats using machine learning techniques is in progress.

Machine learning techniques are divided into supervised learning or unsupervised learning.

When there are multiple classes to classify in the data, supervised learning algorithms learn classification patterns using pre-annotated class labels.

On the other hand, unsupervised learning algorithms learn classification patterns without class labels. IDS technology using machine learning uses signature-based learning methods and anomaly-based methods.

Signature-based learning methods use supervised learning algorithms (SVMs, decision trees, neural networks, KNNs, etc.) to learn intrusion patterns.

Due to the availability of classified data sets such as KDD Cup 99 and Kyoto 2006+, much research has been done on signal-based approaches.

Roshan and Huang [2014] developed an intrusion detection system based on the progressive SVM algorithm. Mohammed et al. [2016] developed an IDS using a least-squares support vector (SVM) based on a mutual information-based algorithm.

Sahu and Mehtre [2015] analyzed the performance of an intrusion detection system based on the J48 decision tree algorithm.

Kazuya et al. [2011] proposed an intrusion detection system that combines several classifiers obtained by learning various traffic data sets to cope with dynamic changes in network traffic trends. Beaver et al. [2013] proposed an intrusion detection system based on the adaptive enhancement of multiple classifiers.

However, the use of the signature-based approach is limited to practical uses. First, it is difficult to detect new types of attacks whose patterns belong to pre-trained attack classes. Second, updating the classifier requires labeled data that requires non-critical manual work.

To solve the shortcomings of the signature-based approach, many exception-based operations have been proposed. Anomaly detection using one-class SVM and clustering method considers the majority of data densely located within the normal region as normal, and some outliers as abnormal.

Song et al. [2011b] proposed a first-class SVM and obtained a detection rate of 93.5% and a false positive rate of 7.33% when applied to the Kyoto 2006+ set.

Ishida et al. [2011] proposed an intrusion detection scheme to identify attack traffic by combining OptiGrid clustering and grid-based cluster labeling algorithms.

Hassen and Bourouis [2015] tested the KDD'99 and Kyoto 2006+ datasets with an online self-learning SVM and obtained an F1-score and an accuracy of 0.98.

Several papers applied autoencoder for network anomaly detection and verified the validity of autoencoder in terms of feature extraction and classification functions. In this section, we first describe our previous work using autoencoders and introduce some work using deep learning algorithms for IDS.

Autoencoder algorithms have been used in IDS, mainly to reduce the feature dimension for efficient computation. For example, Javaid et al. [2016] developed a network IDS by combining sparse autoencoder and soft-max regression (SMR) methods.

Aygun et al. [2017] proposed two IDS models for detecting unknown attack types. One uses an autoencoder and the other uses an autoencoder with noise removed. The model additionally includes a novel probabilistic method to determine the threshold, which is a key parameter for accurate detection. Each model was evaluated with the NSL-KDD data set and achieved detection accuracies of 88.28% and 88.65%, respectively.

Li et al. [2015] proposed an IDS based on the combination of autoencoder and DBN. In this method, an autoencoder is used to reduce the data dimension, and a DBN composed of multiple layers of RBM and an additional layer of BP neural network is used as a class between the malicious class and the normal class. The proposed model was evaluated using the KDD Cup 99 data set, and the results showed that combining an autoencoder and DBN can achieve better detection accuracy with DBN alone.

Tao et al. [2016] proposed a data fusion approach based on Fisher scores and deep autoencoders. The main purpose of this is to reduce the dimension of the data. It has been confirmed that the integration of deep autoencoders as feature extraction methods can improve the accuracy of classification algorithms such as back propagation neural networks and support vector machines.

Shone et al. [2018] proposed an asymmetric deep autoencoder (NDAE) that provides unsupervised feature learning and data-dimensional decay capabilities. Based on the proposed NDAE, the authors developed a novel NIDS model that combines cumulative NDAE (for better functional learning) and Random Forest (RF) (reducing analysis overhead). Evaluation results using the KDD Cup 99 and NSL-KDD data sets showed that the proposed method achieves accuracies of 97.85% and 85.42% for each data set.

Mirsky et al. [2018] Deep learning based on autoencoder function was used to detect online and unsupervised malicious traffic by developing a NIDS model called Kitsune.

Zhang et al. [2018] proposed a deep learning approach using a sparse stack autoencoder device and binary tree ensemble method for network intrusion detection, and evaluated its performance using the NSL-KDD dataset. The evaluation results showed that the proposed approach achieved an average F1 score of 91.97%.

Diro et al. [2018] proposed a distributed NIDS using a stack autoencoder and soft-max regression for pre-training, especially for classification in fog-to-things computing environments.

Madani et al. [2018] proposed a framework for evaluating the robustness of machine learning-based IDS against intentional data poisoning, and analyzed and compared the performance of autoencoder-based and PCA-based models under hostile pollution attacks. The results showed that autoencoder-based IDS can provide more stable search performance than PCA-based IDS.

Li et al. [2017] studied a deep learning model to differentiate the traffic flow of mobile applications, and proposed a VEAN (Variational Autoencoder Network)-based traffic identification model.

Alom et al. [2015] proposed the use of a DBN consisting of a stack of restricted Boltzmann machines (RBMs) for network operation intrusion detection. The proposed approach was evaluated with the NSL-KDD data set, and according to the results, it achieved a detection accuracy of 97.5% for only 37 detection units.

Salama et al. [2011] proposed a hybrid model of IDS combining RBM-based DBN and SVM classifier. After reducing the dimension of the data using DBN, SVM is applied to classify the data.

Kim et al. [2016] proposed IDS based on recurrent neural networks (RNN), and applied LSTM (long short term memory) structure to RNN.

Tang et al. [2016] designed an IDS based on deep neural networks (DNN) for flow-based anomaly detection in a software-based networking environment.

In this way, normal/abnormal data labels are required to identify network threats using conventional machine learning techniques. However, using deep learning or machine learning techniques to solve problems requires a large amount of data, so it is very important for data labeling. There is a problem that requires a lot of effort and time.

The present invention classifies network data as normal or abnormal using an autoencoder, which is a deep learning algorithm, and uses only normal data to train a deep learning network. Deep learning that classifies and collects normal data from a vast amount of learning data An object of the present invention is to provide an unsupervised method for detecting network intrusion using an algorithm and a recording medium on which a program for executing the method is recorded.

An unsupervised method for network intrusion detection using a deep learning algorithm according to an example of the present invention, and a recording medium on which a program for executing the same is recorded, preprocesses data packets input from the outside in a one-hot encoding method a preprocessing step of generating an original vector expressed as a vector value; A compression and restoration step of compressing the original vector to a lower dimension using an autoencoder method and then restoring it to the original dimension to generate a restored vector expressed as a vector value; a loss value calculation step of calculating a loss value by calculating a difference value between the original vector and the restored vector; and a determination step of determining the data packet as abnormal data if the calculated loss value is greater than the threshold, and determining the data packet as normal data if the calculated loss value is less than the threshold. It is determined using an operating characteristic curve, ROC curve), and a point having a high true positive rate and a low false positive rate in the receiver operating characteristic curve may be selected as a threshold value.

For example, in the determination step, the threshold value may be a true positive rate of 0.8 or more and a false positive rate of 0.3 or less in the receiver manipulation characteristic curve.

In the pre-processing step, the data packet input from the outside contains natural language or characters that are non-numeric data, and before pre-processing the data packet in a one-hot encoding method, non-numeric data is converted to numeric data (numerical data). data), and then pre-processed by a one-hot encoding method, the data packet in the numeric data state may be coded as 0 and 1 and displayed as a vector value.

In the compression and decompression step, the autoencoder method includes an encoder step that compresses the original vector to a lower dimension than the original vector, and a decoder step that restores the original vector to the same dimension as the original vector to generate the original vector. may include

In the step of calculating the loss value, at least one of a cross-entropy error (CEE) and a root mean square error (RMSE) may be used to calculate a difference between the original vector and the restored vector.

An unsupervised method for detecting network intrusion using a deep learning algorithm according to the present invention and a recording medium on which a program for executing the same is recorded using an autoencoder after preprocessing a data packet input from the outside into a vector value Thus, it is possible to compress and restore the data packet, calculate a loss value from the difference value, and determine whether the data packet is normal data or abnormal data, thereby reducing the effort and time required for data labeling.

1 is a diagram for explaining the concept of an unsupervised network intrusion detection system using a deep learning algorithm according to an example of the present invention.
FIG. 2 is a diagram for explaining the concept of an autoencoder used in the compression and decompression step S2 shown in FIG. 1 .
3 shows an example of the ROC curve of the SGD algorithm.
4 shows an example of the ROC curve of the RMSPROP algorithm.
5 shows an example of the ROC curve of the ADAM algorithm.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it goes without saying that the present invention is not limited to these embodiments and may be modified in various forms.

In the drawings, in order to clearly and briefly describe the present invention, the illustration of parts irrelevant to the description is omitted, and the same reference numerals are used for the same or extremely similar parts throughout the specification. In addition, in the drawings, the thickness, width, etc. are enlarged or reduced in order to make the description more clear, and the thickness and width of the present invention are not limited to those shown in the drawings.

1 is a diagram for explaining the concept of an unsupervised network intrusion detection system and method using a deep learning algorithm according to an example of the present invention.

An unsupervised network intrusion detection system and method using a deep learning algorithm according to an example of the present invention is a pre-processing step (S1), a compression and restoration step (S2), a loss value calculation step (S3), as shown in FIG. ) and the determination step (S4) may be performed.

In the pre-processing step S1, an original vector expressed as a vector value may be generated by pre-processing a data packet input from the outside in a one-hot encoding method.

In this pre-processing step (S1), the data packet input from the outside may include natural language or characters that are non-numerical data, and the data packet is converted into a one-hot encoding method. Before pre-processing, the non-numeric data is converted to numeric data, and then pre-processed by the one-hot encoding method, so that the data packets in the numeric data state are coded with 0 and 1. It can be expressed as a vector value.

In the compression and decompression step S2, the original vector is compressed to a lower dimension by an autoencoder method, and then restored to the original dimension to generate a restored vector represented by a vector value.

Here, the autoencoder method may include an encoder step of compressing the original vector to a lower dimension than the original vector and a decoder step of restoring the original vector to the same dimension as the original vector.

The loss value calculation step S3 may calculate a loss value by calculating a difference value between the original vector and the restored vector. In order to calculate the difference between the original vector and the restored vector, a cross-entropy error (CEE) (cross-entropy error, CEE) or a root mean square error (RMSE) is used. can

In the determination step S4, if the calculated loss value is greater than the threshold, the data packet may be determined as abnormal data, and if the calculated loss value is less than the threshold, the data packet may be determined as normal data.

Here, the threshold is determined using a receiver operating characteristic curve (ROC curve), and the point in the receiver operating characteristic curve having a high true positive rate and a low false positive rate is the may be selected as a threshold.

For example, the threshold value may be selected such that the true positive rate is 0.8 or more and the false positive rate is 0.3 or less in the receiver manipulation characteristic curve.

Hereinafter, each step performed by the unsupervised network intrusion detection system using a deep learning algorithm according to an example of the present invention will be described in more detail.

To this end, first, datasets will be described.

Data collected in the real world, including IDS, is usually not balanced between classes. The intrusion data rate is known to be about 1% of all traffic data [Song et al., 2011a, Roshan and Huang, 2014].

However, KDD Cup 99 [2007] and Kyoto 2006 + [Song et al., 2011a], which are mainly used to evaluate machine learning-based IDS performance, contain much less normal data than abnormal data. Due to the environment specifically designed for attack data collection, the attack data rate for all data in KDD Cup 99 is about 74%, and the attack data rate for the Kyoto 2006+ dataset is about 95%.

Unbalanced data sets caused the following problems. When experiencing IDS performance using data sets with unequal class sizes, it is imperative to define an evaluation metric that is appropriate for the problem. For example, accuracy is not the best metric when most of the data belongs to a particular class. Therefore, in this invention, we present all key performance scores in the evaluation section.

The KDD Cup99 data set [2007] is a large set mainly used to evaluate network IDS performance. However, the data set was collected in 1999 and may not contain the latest network intrusion patterns. In addition, this data set is collected in a virtual network environment that is different from the patterns observed in the real network system.

The Kyoto 2006+ data set [Song et al., 2006, Song et al., 2011a] contains network traffic data collected from November 2006 to December 2015. This data set can also be used for large data analysis of 19.683 GB. Therefore, in this invention, the approach was applied to the Kyoto 2006+ data set.

In the Kyoto 2006+ dataset, various network threats collected through the honeypot network are combined with Kyoto University's regular servers. The data set consists of a total of 24 features [Song et al., 2006].

Of these, 14 features are selected from 41 features of the KDD CUP99 data set. These features include Duration, Service, Source Bytes, Destination Bytes, Count, Srv rate, Serror rate, Srv Serror rate, Dst host count, Dst host srv count, src port rate equal to Dst host, Dst host serror rate, Dst host srv serror rate, Flag, etc.

An additional 10 features were designated for identification purposes. Three features (IDS detection, malware detection, and shellcode detection) indicate whether a packet was detected by a specific IDS mounted on the front end of the honeypot network. It also contains information such as source IP address, source port number, destination IP address, destination port number, start time, and protocol.

The whole data set consists of three classes.

First, every packet entering the bait server called Honeypot is considered an attack, redefined as 'known attack (-1)' and 'shellcode-based unknown attack (-2)', and all packets in the normal server are considered 'normal' (1)'.

In the pre-processing step S1, an original vector expressed as a vector value may be generated by pre-processing a data packet input from the outside in a one-hot encoding method.

To this end, noise may be filtered in the first pre-processing step S1. When the IDS Detection, Malware Detection and Shell Code Detection fields are triggered, instances with normal (1) packets are considered noise. Instances should be labeled with the -1 (attack) label, as these functions indicate whether an intrusion, malicious software, or shellcode was detected, respectively. This discrepancy arises from the assumption that all incoming packets to the normal server are considered normal.

The second step removes some features. IDS detection, malware detection, and shellcode detection are eliminated because they are strong attack indicators. The source IP address and destination IP address are removed as these values are deleted as random IPs during the initial data collection process. Start time is not critical and should be removed.

Third, non-numeric data is changed to numeric data and input to an autoencoder device. Services, flags and protocols are categorical data. Service function has 85 values and Flag has 13 values. protocol indicates a network protocol with a value of tcp, udp, or icmp. Each number of non-numeric data is converted to numeric data, which is later encoded with one-hot encoding. For numeric category data, the size of the number does not indicate the size of the data itself.

Therefore, we encode the data using one-time encoding so that this number does not affect the training weighting process. The protocol function is 3 vectors: [1,0,0] for tcp(0), [0,1,0] for udp(1), and [0, 0,1] for icmp(2) is displayed

Similarly, the categorical service and flag functions translate to 85 and 13, respectively. The source port number and destination port number are between 0 and 65535, so converting to a 65536-dimensional vector would make the data size too large.

Therefore, well-known ports (0 ~ 1023), registered ports (1024 ~ 49151), and dynamic ports (49152 ~ 65536) are expressed as three vectors.

The last step is to normalize the feature values between 0 and 1. Numeric categorical data is automatically converted to 0 and 1 due to one-hot encoding. Duration, source byte, and destination byte attribute values can be normalized using MinMax multipliers after statistical outliers are converted to thresholds for upper outliers.

The Dst_host_count and Dst_host_srv_count rate functions can be converted to a value between 0 and 1 by dividing the value by the maximum value (eg 100 in this case). The final characteristic selected for evaluation may be the following list - Duration, Service, Source bytes, Destination bytes, Count, Same srv rate, Serror rate, Srv serror rate, Dst host count, Dst host srv count, Dst host same src port rate, Dst host serror rate, Dst host srv serror rate, Flag, Source Port Number, Destination Port Number, Protocol.

These functions are expressed in 119 dimensions by combining 12 normal functions and 107 one-hot encoded category functions. Finally, in the present invention, duplicates of a normalized data set can be removed using drop duplicates of the Python Pandas library.

Then, the present invention aggregates Known Attack (-1) and Unknown Attack (-2) into one abnormal class, and can be composed of two classes: normal class normal (1) and abnormal class abnormal (-1). .

FIG. 2 is a diagram for explaining the concept of an autoencoder used in the compression and decompression step S2 shown in FIG. 1 .

In Fig. 2, white circles indicate bias and black circles indicate network nodes.

In the compression and decompression step S2, the original vector is compressed to a lower dimension by an autoencoder method, and then restored to the original dimension to generate a restored vector represented by a vector value.

The autoencoder may be a deep learning algorithm consisting of an input that reconstructs a compression vector to the original vector size and an encoding network that compresses a decoding network.

More specifically, as shown in FIG. 2 , when a network log represented by 119 vectors is given to a trained autoencoder device, an output may be generated through compression and reconstruction.

In the compression and decompression step (S2), the autoencoder method restores the original vector to the same dimension as the encoder step of compressing the original vector to a lower dimension than the original vector and the original vector, It may include a decoder (Decoder) step for generating the vector of the present application.

In the encoder step, as shown in FIG. 2, the encoder network converts a 119-dimensional original vector into a 512-dimensional value, and then compresses it into 256, 128, and 64 dimensions, so that it can be compressed to a lower dimension than the original vector. there is. In the decoder (Decoder) step, the decoding network can restore the 64 dimensions to 128, 256, 512, and 119 dimensions to have the same dimension as the original vector.

Then, in order to calculate the difference between the original vector and the restored vector in the loss value calculation step S3, a cross-entropy error (CEE) or a root mean square error (RMSE) is used. Thus, the loss value can be calculated.

As an example of the evaluation for this purpose, a total of two data sets were created for evaluation.

First, the training set was randomly undersampled from the normal data set.

Second, to avoid class imbalance, the test set was randomly drawn from a data set consisting of equal proportions of normal and abnormal data.

Table 1 below shows the number of logs in each data set. For reference, in Table 1, note that there is no abnormal data in the training set.

In Table 1, the number of logarithms of normal and abnormal classes for each evaluation data set is expressed.

Dataset Normal class Abnormal class Total training set 50284 0 50284 test set 95752 95752 191504

Table 2 shows the training configuration. For evaluation, the weights were initialized according to the weight distribution and the bias weights were initialized to zero. RMSPROP, SGD, and ADAM were used as optimization algorithms, and Root Mean Square Error (RMSE) and cross-entropy error (CEE) were used as loss functions. Epoch was set to 6 and batch sizes were tested at 32 and 64 respectively. The configurations used for training are shown in Table 2.

Parameter Value Optimization algorithm Rmsprop, SGD, Adam Loss function MSE, cross entropy error Epochs 6 Batch Size 32, 64

A receiver operating characteristic curve (ROC curve) is then drawn using the various parameters, and then the loss function that best classifies normal and abnormal (i.e. intrusive) classes can be selected as a threshold value. .

3 shows an example of the ROC curve of the SGD algorithm, where (a) and (b) are the results of batch sizes 32 and 64 for the MSE function, and (c) and (d) are the cross-entropy functions.

As shown in (c) and (d) of Fig. 3, cross entropy has better performance than the MSE shown in Fig. 3 (a) and (b) for the loss function, and the difference in batch size is important only in MSE. do. Although there are some differences depending on the loss function and batch size, the closer the positive rate is to 90%, the higher the false positive rate, so it can be seen that the overall performance is good.

4 shows an example of the ROC curve of the RMSPROP algorithm.

Cross entropy is not applicable, only MSE was evaluated. There is a slight difference depending on the batch size, but you can see that the performance is not good.

5 shows an example of the ROC curve of the ADAM algorithm.

Figures 5 (a) and (b) are the results of batch sizes 32 and 64 for the MSE function, and Figure 5 (c) and (d) are the cross-entropy functions. It is better to use MSE as the loss function, but we can confirm that it has the worst performance of the three algorithms used so far.

Cross-entropy error (CEE) and root mean square error (RMSE) between input and output data reconstructed by a trained autoencoder model were calculated.

To classify the input log as normal or abnormal, we determined a threshold representing the cross-entropy error (CEE) and the root mean square error (RMSE) that best classifies the normal and nonstationary classes. First, we use the receiver operating characteristic (ROC) curve as various parameters to check whether it can be used as a good classification parameter. It can be seen that the curve of FIG. 3 showing the best performance and cross entropy shows better performance than the root mean square error (RMSE).

However, depending on which optimization algorithm is used, loss functions should be used as features to improve performance.

In the determination step S4, through the above receiver operation characteristic (ROC) curve, if the calculated loss value is greater than the threshold, the data packet is determined as abnormal data, and if the calculated loss value is less than the threshold, the data packet is converted to normal data. can be identified as

Here, as the threshold value, a point having a high true positive rate and a low false positive rate on the receiver operation characteristic (ROC) curve may be selected as the threshold value.

For example, the threshold value may be selected such that the true positive rate is 0.8 or more and the false positive rate is 0.3 or less in the receiver manipulation characteristic curve.

Tables 3-5 below show the recall, precision, specificity (true negative rate), and false positives predicting anomalous grade (i.e. network intrusion) when 0.5, 0.95, and F1 measurements reach their maximum (bold). ) and F-measurements.

Recall Precision Specificity False
Positive Threshold F _One 0.50 1.00 1.00 0.00 11.79 0.67 0.55 1.00 1.00 0.00 11.26 0.71 0.60 1.00 1.00 0.00 10.66 0.75 0.65 1.00 0.99 0.01 10.20 0.79 0.70 1.00 0.99 0.01 10.08 0.82 0.75 1.00 0.99 0.01 9.56 0.86 0.80 1.00 0.89 0.01 9.03 0.89 0.85 1.00 0.98 0.02 8.29 0.92 0.90 0.99 0.88 0.12 4.45 0.94 0.95 0.98 0.62 0.38 2.22 0.97

Table 3 shows the evaluation results for SGD.

Recall Precision Specificity False
Positive Threshold F _One 0.50 1.00 1.00 0.00 8.94 0.67 0.55 1.00 1.00 0.00 8.00 0.71 0.60 1.00 0.99 0.01 6.77 0.75 0.65 1.00 0.99 0.01 5.36 0.79 0.70 1.00 0.99 0.01 4.61 0.82 0.75 1.00 0.98 0.02 3.88 0.86 0.80 1.00 0.92 0.08 3.08 0.89 0.85 0.99 0.86 0.14 2.45 0.92 0.90 0.98 0.70 0.30 1.68 0.94 0.95 0.97 0.37 0.63 1.20 0.96

Table 4 shows the evaluation results for RMSPROP.

Recall Precision Specificity False
Positive Threshold F _One 0.50 1.00 1.00 0.00 7.34 0.67 0.55 1.00 1.00 0.00 6.08 0.71 0.60 1.00 0.99 0.01 5.20 0.75 0.65 1.00 0.98 0.02 4.09 0.79 0.70 1.00 0.97 0.03 3.53 0.83 0.75 1.00 0.92 0.08 3.09 0.86 0.80 0.99 0.88 0.12 2.49 0.89 0.85 0.99 0.81 0.19 2.22 0.92 0.90 0.99 0.74 0.23 1.91 0.94 0.95 0.99 0.44 0.56 1.39 0.96

Table 5 shows the evaluation results for ADAM. Table 3-5 shows the best parameter values for ROC curve graph modification in each optimization algorithm. When the recall value is 0.95, it can be seen that the F1 measurement value is 0.97, which is the highest when using the SGD algorithm. Although the F1 measurement value is a smaller number than the other two algorithms, it can be seen that the SGD algorithm has a much larger difference than the other algorithms by using the false positive rate.

As such, the present invention proposes an unsupervised deep learning approach for detecting network intrusions. An autoencoder network compresses the input and reconstructs it to its original size as learned from a given training set. The present invention assumes that if the network is trained with normal data, the attack is unlikely to be restored to the input value. Experiments with the training set consisting of 100% normal data confirm that the approach of the present invention is effective for IDS (F-score 0.97).

The approach proposed according to the present invention offers several advantages. First, since only normal data is used in the learning method for implementing the present invention, much effort is not required to label the training data. Second, it can solve the problem of zero-day attacks where intrusion patterns are not yet discovered.

The features, structures, effects, etc. as described above are included in at least one embodiment of the present invention, and are not necessarily limited to one embodiment. Furthermore, features, structures, effects, etc. illustrated in each embodiment can be combined or modified for other embodiments by those of ordinary skill in the art to which the embodiments belong. Accordingly, the contents related to such combinations and modifications should be interpreted as being included in the scope of the present invention.

Claims (6)

In an unsupervised network intrusion detection method using a deep learning algorithm in which each step is performed by a computing system,
A pre-processing step of pre-processing a data packet input from the outside in a one-hot encoding method to generate an original vector expressed as a vector value;
a compression and restoration step of compressing the original vector to a lower dimension using an autoencoder method and then restoring the original dimension to generate a restored vector expressed as a vector value;
a loss value calculation step of calculating a loss value by calculating a difference value between the original vector and the restored vector; and
a determining step of determining the data packet as abnormal data if the calculated loss value is greater than a threshold value, and determining the data packet as normal data if the calculated loss value is less than the threshold value;
The threshold value is determined using a receiver operating characteristic curve (ROC curve), and in the receiver operating characteristic curve, the true positive rate is 0.8 or more and the false positive rate is 0.3 or less. An unsupervised network intrusion detection method using a selected deep learning algorithm. delete According to claim 1,
In the pretreatment step,
The data packet input from the outside includes natural language or characters that are non-numeric data, and before the data packet is pre-processed by the one-hot encoding method, the non-numeric data is numeric data (numerical data). data), then preprocessed by the one-hot encoding method, and data packets in the numeric data state are coded as 0 and 1 and displayed as a vector value. An unsupervised method of network intrusion detection using a deep learning algorithm. According to claim 1,
In the compression and decompression step,
The autoencoder method is
An encoder step of compressing the original vector to a lower dimension than the original vector;
An unsupervised network intrusion detection method using a deep learning algorithm, comprising a decoder step of restoring the original vector to the same dimension as the original vector and generating the restored vector. According to claim 1,
In the step of calculating the loss value,
Busy using a deep learning algorithm using at least one of a cross-entropy error (CEE) and a root mean square error (RMSE) in order to calculate the difference between the original vector and the restored vector A method of detecting network intrusion in a diagram manner. A computer-readable recording medium in which a computer program for executing the unsupervised network intrusion detection method using the deep learning algorithm according to any one of claims 1 to 5 by a computer is recorded.

Images