DE202023100327U1 - A system for analyzing the relevance of features and for reducing features on the Microsoft Azure Machine Learning Studio (MAMLS) - Google Patents

Abstract

A system (100) for feature relevance analysis and feature reduction on Microsoft Azure Machine Learning Studio (MAMLS), the system (100) comprising:
a dataset collection module (102) for creating a dataset (UNSW NB-15) of modern attack vectors;
a feature extraction module (104) coupled to the data set collection module (102) for processing the generated data set to generate feature sets resulting in the generation of multiple classes of features, the multiple classes of features including multiple flow features, base features, content features , time features and additional features include;
a feature reduction module (106) connected to the feature extraction module (104) to reduce the extracted features based on the relevance level to improve the prediction rate of the attack; and
a classification module (108) coupled to the feature reduction module (106) and comprising a two-class neural network for predicting the generated multiple classes of features in a network instance into a plurality of classes as anomalous or normal.

Description

FIELD OF THE INVENTION

The present invention relates to the field of feature reduction systems. In particular, the present invention relates to a system for feature relevance analysis and feature reduction on MAMLS.

BACKGROUND OF THE INVENTION

The entities within a network topology are constantly at risk due to the proliferation of modern attack vectors that threaten the confidentiality, integrity and availability of information. The practical problems associated with any intrusion detection system (IDS) are usually related to generating large numbers of false positives when exposed to newer types of attacks. A prerequisite for effective training of the IDS is the use of a modern dataset.

CN104050787A discloses a method and apparatus for detecting anomalies with categorical attributes. The method includes detecting a plurality of events related to user activities within a security system, the events being defined by a plurality of attributes, at least one attribute being categorical, and a data spacing between the events being a function of the event attributes is, evaluating the detected events using a density-based anomaly detection method f(r), where r is a size of a neighborhood around a data point, comparing a value of the evaluated expression to a marginal threshold (msg(r)), and setting an alarm , when the value is detected as exceeding the threshold.

US8887281B2 discloses a system and method for detecting intrusions into the operation of a computer system, comprising a sensor configured to collect information about the operation of the computer system, to format the information into a data set, and to transmit the data set. A database is configured to receive the data set from the sensor and store the data set. A detection model generator is configured to request records from the database, generate an intrusion detection model, and submit the intrusion detection model to the database. A detector is configured to receive a data set from the sensor and classify the data set as normal operation or an attack in real time. A data analysis device is configured to request data sets from the database and perform a data processing function on the data sets.

Traditional datasets like KDD cup 99 or its successor NSL-KDD no longer seem to be the ideal choice due to the lack of modern attack scenarios.

• • Im KDD-Cup 99 gibt es zahlreiche sich wiederholende Datensätze, die dazu führen, dass maschinelle Lerntechniken auf bestimmte Angriffsarten ausgerichtet sind.
• • Die Wahrscheinlichkeitsverteilung der Trainings- und Testdatensätze ist unterschiedlich, so dass es zu Verzerrungen kommt, anstatt normale und anomale Netzwerkinstanzen auszugleichen. NSL-KDD enthält keine modernen Angriffsmuster.

However, the KDD-Cup-99 and NSL-KDD records have significant disadvantages:

• • In the KDD-Cup 99, there are numerous repetitive datasets that result in machine learning techniques being targeted for specific types of attacks.
• • The probability distribution of the training and test datasets is different, so bias occurs instead of balancing normal and anomalous network instances. NSL-KDD does not contain modern attack patterns.

Therefore, in order to overcome the above disadvantages, there is a need to develop a system for feature relevance analysis and feature reduction on MAMLS. To do this, the UNSW NB-15 dataset is used to evaluate the efficiency of two-class neural networks available as a module on Microsoft Azure Machine Learning Studio (MAMLS) to perform predictive analytics.

The technical advance disclosed by the present invention overcomes the limitations and disadvantages of existing and conventional systems and methods.

SUMMARY OF THE INVENTION

The present invention relates generally to a feature relevance analysis and feature reduction system on Microsoft Azure Machine Learning Studio.

The aim of the present invention is to develop a system for feature relevance analysis and feature extraction;

Another objective of the present invention is to evaluate the efficiency of two-class neural networks available as a module on MAMLS to perform predictive analysis; and

Another objective of the present invention is to continuously update many modern attack vectors from the CVE (Common Vulnerability Exposure) website, which contains entries of publicly known cybersecurity vulnerabilities.

• ein Datensammelmodul zur Erstellung eines Datensatzes (UNSW NB-15) mit modernen Angriffsvektoren;
• ein Merkmalsextraktionsmodul, das mit dem Datensatzsammelmodul verbunden ist, um den erzeugten Datensatz zu verarbeiten, um Merkmalssätze zu erzeugen, die zur Erzeugung von mehreren Klassen von Merkmalen führen, wobei die mehreren Klassen von Merkmalen mehrere Flussmerkmale, Basismerkmale, Inhaltsmerkmale, Zeitmerkmale und zusätzliche Merkmale umfassen;
• ein Modul zur Merkmalsreduzierung, das mit dem Merkmalsextraktionsmodul verbunden ist, um die extrahierten Merkmale auf der Grundlage des Relevanzniveaus zu reduzieren, um die Vorhersagerate des Angriffs zu verbessern; und
• ein Klassifizierungsmodul, das mit dem Modul zur Merkmalsreduzierung verbunden ist und ein neuronales Zweiklassennetz zur Vorhersage der erzeugten Mehrfachklassen von Merkmalen in einer Netzinstanz in eine Vielzahl von Klassen als anomal oder normal umfasst.

In one embodiment, a system for feature relevance analysis and feature reduction on Microsoft Azure Machine Learning Studio (MAMLS), the system comprising:

• a data collection module to create a dataset (UNSW NB-15) of modern attack vectors;
• a feature extraction module connected to the data set collection module to process the generated data set to generate feature sets resulting in the generation of multiple classes of features, the multiple classes of features including multiple flow features, base features, content features, time features and additional features ;
• a feature reduction module connected to the feature extraction module to reduce the extracted features based on the relevance level to improve the prediction rate of the attack; and
• a classification module coupled to the feature reduction module and comprising a two-class neural network for predicting the generated multiple classes of features in a network instance into a plurality of classes as anomalous or normal.

In one embodiment, the UNSW NB-15 cybersecurity record is created to generate both normal activity and attack patterns.

In one embodiment, the flow characteristics shows basic information of each network packet related to internet protocol (ips), port numbers and a protocol involved in the transaction, where the basic characteristics are more towards time-to-live (TTL), Service, packet counts, and so on, where the content features correspond to source and destination window sizes, the timing features relate to jitter (delays) of packets in the network, arrival time between packets, round-trip time, etc., and the additional features focus primarily on the number of records or network instances with the same IPs, port numbers, flows in the FTP session, same service, etc.

In one embodiment, the neural network includes a cross-entropy loss function to optimize the log-likelihood of the data set.

In one embodiment, the data set is standardized by the neural network after the min-max classification has been performed.

In one embodiment, the feature reduction module includes a meta-estimator for selecting features based on scores of a threshold, wherein the feature with a score less than the threshold is irrelevant and eliminated.

In order to further clarify the advantages and features of the present invention, a more detailed description of the invention will be made by reference to specific embodiments thereof illustrated in the attached figure. It is understood that this figure represents only typical embodiments of the invention and therefore should not be considered as limiting its scope. The invention will be described and illustrated with additional specificity and detail with the accompanying figure.

character list

• 1 ein Blockdiagramm eines Systems zur Merkmalsrelevanzanalyse und Merkmalsreduktion auf Microsoft Azure Machine Learning Studio (MAMLS) zeigt.

These and other features, aspects and advantages of the present invention will be better understood when the following detailed description is read with reference to the accompanying figure, in which like characters represent like parts throughout the figure, wherein:

• 1 shows a block diagram of a system for feature relevance analysis and feature reduction on Microsoft Azure Machine Learning Studio (MAMLS).

Those skilled in the art will understand that the elements in the figure are shown for simplicity and are not necessarily drawn to scale. For example, the flowcharts illustrate the method in terms of key steps that may aid in understanding aspects of the present disclosure. Furthermore, one or more components of the device in the figure may be represented by conventional symbols, and the figure only shows the specific details relevant to understanding the embodiments of the present disclosure, not to encircle the figure with details to overload, which are easily recognizable to those skilled in the art familiar with the present description.

DETAILED DESCRIPTION

For the purposes of promoting an understanding of the invention, reference will now be made to the embodiment illustrated in the Figure and specific language will be used to describe the same. It should be understood, however, that no limitation on the scope of the invention is intended, and such alterations and further modifications to the illustrated system and such further applications of the principles of the invention set forth therein are contemplated as would occur to those skilled in the art invention would normally come to mind.

Those skilled in the art will understand that the foregoing general description and the following detailed description are exemplary and explanatory of the invention and are not to be taken as limiting.

When this specification refers to "an aspect," "another aspect," or the like, it means that a particular feature, structure, or characteristic described in connection with the embodiment is present in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment," "in another embodiment," and similar phrases throughout this specification may or may not all refer to the same embodiment.

The terms "comprises," "including," or other variations thereof are intended to cover non-exclusive inclusion, such that a method or method that includes a list of steps includes not only those steps, but may also include other steps that are not expressly stated or pertaining to any such process or method. Likewise, any device or subsystem or element or structure or component preceded by "comprises...a" does not, without further limitation, exclude the existence of other devices or other subsystem or other element or other structure or other component or additional device or additional subsystems or additional elements or additional structures or additional components.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The system, methods, and examples provided herein are for purposes of illustration only and are not intended to be limiting.

Embodiments of the present invention are described in detail below with reference to the attached figure.

1 shows a block diagram of a system (100) for feature relevance analysis and feature reduction on Microsoft Azure Machine Learning Studio (MAMLS), the system (100) comprising: a data set collection module (102), a feature extraction module (104), a feature reduction module (106) and a classification module (108).

The dataset collection module (102) to create a dataset (UNSW NB-15) with modern attack vectors. The UNSW NB-15 record is created for cyber security to generate both normal activity and attack patterns.

The feature extraction module (104) interfaces with the dataset collection module (102) to process the generated dataset and create feature sets that result in the generation of multiple classes of features, the multiple classes of features including flow features, base features, content features, time features, and additional Features include. The flow characteristics reveal basic information of each network packet in terms of internet protocol (Ips), port numbers and a protocol in the transaction, with the basic characteristics leaning more towards time-to-live (TTL), service, packet count, etc., with the Content characteristics correspond to source and destination window sizes, timing characteristics relate to jitter (delays) of packets in the network, arrival time between packets, round trip time, etc., and additional characteristics focus primarily on the number of records or network instances containing them IPs, port numbers, flows in the FTP session, same service, etc.

In one embodiment, the Argus tool is used to process raw pcap files and create feature sets, resulting in the creation of five feature classes. Table 1: Flow characteristics SL No. characteristic Description 1. script Source IP address 2. Sports Source port number 3 dstip Destination IP address 4 sport Port number of the target 5 proto Protocol used in the transaction Table 2: Basic characteristics SL No. characteristic Description 6 Country State of the associated log 7 major duration of the transaction 8th sbytes Bytes from source to destination 9 dbytes Bytes from destination to source 10 quarterly Lifetime from source to destination 11 German Lifetime from target to source 12 castle Packets dropped by the source 13 doss Packets dropped by destination 14 service Service used, e.g.: http, smtp, ftp, etc. 15 load source bits per second 16 dload Target bits per second 17 spkts Number of packets from source to destination 18 dpkts Number of packets from destination to source Table 3: Content characteristics SL No. characteristic Description 19 swing Window display of source tcp 20 dwin Target tcp window display 21 stcpb Consecutive number of the source tcp 22 dtcpb Consecutive number of the target tcp 23 smeansz Mean of the size of the flow packets transmitted by the source 24 dmeansz Mean of the size of the flow packets transmitted by the destination 25 trans_depth Specifies the depth of the http request-response transaction 26 res_bdy_len Data transmitted by the http service Table 4: Time characteristics SL No. characteristic Description 27 sjit jitter at the source 28 djit jitter at destination 29 voice Record start time 30 Itime Record last time 31 sinpt Time of arrival between packets at the source 32 dintpt Arrival time between packages at destination 33 tcprtt The sum of synack and ackdat 34 synack The time elapsed between SYN and SYN_ACK packets 35 acknowledgment The time between SYN_ACK and ACK packets Table 5 Additional features SL No. characteristic Description 36 is_sm_ips_ports If the source ip is equal to the destination ip and sport is equal to dport then the value is 1 or else 0 37 ct_state_ttl The states (6) are assigned values for the source and destination lifetimes 38 ct_flw _http_mthd Number of get and post methods in the http service 39 is_ftp_login ftp session logged in by user or not 40 ct_ftp_cmd Number of commands in the ftp session 41 ct_srv_src Number of connections to the same service and source address in 100 connections as last time 42 ct_srv_dst Number of connections to the same service and destination address in 100 connections as last time 43 ct_dst_ltm Number of connections with the same destination address in 100 connections since the last time 44 ct_src_ltm Number of connections with the same source address in 100 connections last time 45 ct_src_dport_ltm Number of connections with the same source and destination port in 100 connections the last time 46 ct_dst_sport_ltm Number of connections to the same destination and source port in 100 connections last time 47 ct_dst_src_ltm Number of connections with the same source and destination address in 100 connections since the last time

The feature reduction module (106) is connected to the feature extraction module (104) to reduce the extracted features based on the relevance level to improve the prediction rate of the attack. The feature reduction module (106) includes a meta-estimator for selecting features based on scores of a threshold, where the feature whose score is below the threshold is irrelevant and eliminated.

The classification module (108) interfaces with the feature reduction module (106) which comprises a two-class neural network for predicting the generated multiple classes of features in a network instance into a plurality of classes as anomalous or normal. The neural network includes a cross-entropy loss function to optimize the log-likelihood of the dataset. The data set is standardized by the neural network after the min-max classification has been carried out.

In one embodiment, the number of hidden nodes used for both feature relevance and feature reduction tasks is 100. The default learning rate in MAMLS is set to 0.1.

t_i bezieht sich auf die Zielausgabe und y_i auf die berechnete Ausgabe.The neural network architecture uses a cross-entropy loss function that optimizes the log-likelihood of the training data according to equation.1: $$\text{E}=-{\displaystyle \sum _{i=1}^n{t}_i\text{log}}\left(y_i\right)+\left(1-t_i\right)\text{log}\left(1-y_i\right)$$

t _i refers to the target output and y _i to the calculated output.

X_i ist der i-te Datenpunkt. min und max stehen für den Minimal- bzw. Maximalwert eines bestimmten Merkmals.To standardize the data, the neural network model uses the equations given in Eq. 2 defined min-max normalization: $$A-\frac{\left[Xi-\text{at least}\left(X\right)\right]}{\left[\text{Max}\left(X\right)-\text{at least}\left(X\right)\right]}$$

X _i is the ith data point. min and max represent the minimum and maximum values, respectively, of a particular feature.

According to one embodiment, Select From Model is a meta-estimator found in the ScikitLearn library and is primarily used to select features based on their scores. Twenty-three features were part of the feature reduction task. A two-class neural network was then used as a classifier. Various combinations of features greater or less than 23 features were also tried, but none gave better results than the combination of time, addition, base, and flow (TABF). Table 6 23 features selected by Select From Model 1. spkts 9. tcprtt 17.ct_src_dport_ltm 2nd sentence 10. synack 18.ct_srv_dst 3rd German 11. dmean 19. answer_body_len 4. load 12. sbytes 20. is_ftp_login 5. ackdat 13. Castle 21. dload 6. Sinpt 14. trans_depth 22. synack 7. djit 15. ct_srv_src 23. ct_state_ttl 8.dwin 16. ct_ftp_cmd

$$Pr\ddot{\text{a}}zision=\frac{TP}{TP+FP}$$

$$R\ddot{\text{u}}ckruf=\frac{TP}{TP+FN}$$

$$F1-Ergebnis=\frac{2XPr\ddot{\text{a}}zisionXR\ddot{\text{u}}ckruf}{Pr\ddot{\text{a}}zision+R\ddot{\text{u}}ckruf}$$

In one embodiment, 31 feature combinations are examined and their contribution to the classification result in terms of accuracy, precision, recognition, F1 score and area under the curve (AUC) is explained. Equations 3-6 can be used to calculate performance indicators. $$Geba\mathrm{and}iGkeit=\frac{TP+TN}{TP+TN+fP+fN}$$

$$P\mathrm{right}\ddot{\text{a}}\mathrm{e.g}isiOn=\frac{TP}{TP+fP}$$

$$R\ddot{\text{and}}ck\mathrm{right}\mathrm{and}f=\frac{TP}{TP+fN}$$

$$f1-E\mathrm{right}Gebnis=\frac{2XP\mathrm{right}\ddot{\text{a}}\mathrm{e.g}isiOnXR\ddot{\text{and}}ck\mathrm{right}\mathrm{and}f}{P\mathrm{right}\ddot{\text{a}}\mathrm{e.g}isiOn+R\ddot{\text{and}}ck\mathrm{right}\mathrm{and}f}$$

According to one embodiment, a combination of TABF (39 features in total) gave the best prediction result, but cannot be considered practical due to the 39 features and thus offers ample scope for reducing the features.

A random sample is preferable for splitting the data. The 80:20 rule is followed to achieve a lower estimation error, in line with the widely accepted principle that "better safe than sorry". It is more appropriate when the datasets are heterogeneous in nature, such as B. At UNSW NB-15, where the attack instances (119,341) outweigh the normal instances (56,000). A 10-fold cross-validation is used.

Intruder Detection Systems (IDS) always deal with numerous features that, if improperly selected, can affect the performance of IDS. Based on this claim, it is extremely important to perform a feature analysis to understand the performance of IDS. Feature analysis could be of great help in discerning the importance of each feature, and feature reduction provides ample scope for using fewer features to improve attack detection rate. Analysis of feature relevance has not yet been performed for the UNSW NB-15 dataset, although some authors have performed feature reduction to improve the classification result.

Not every feature in a network record appears to be relevant to intrusion detection. As an integral part of this work, a detailed study was carried out to recognize the importance of each and every feature, without which the feature selection becomes absurd.

The founders of UNSW NB-15 heavily emphasized the mechanism of derivation of 47 traits, but not much was explained about the applicability of trait classes.

To name a few, combinations such as BAFC and TABF yielded good accuracy but cannot be considered the best subset of traits as there are 39 traits that are often considered impractical in the study

Feature Relevance Analysis and Feature Reduction of UNSW NB-15 331 Machine Learning.

Relevance of features alone cannot guarantee optimal results, especially when 39 features are involved. Therefore, feature reduction is essential. The idea behind performing the feature reduction was to examine whether there is a feature subset with fewer than 39 features. This paved the way for feature reduction, which eventually led to better predictions by using only 23 features retrieved from Select From Model. As explained above, despite the use of classifiers such as logistic regression, the highest accuracy was reported as 89.26, but the experimental sequence using MAMLS resulted in an accuracy of 97%, which is significantly higher than the existing work.

It is worth noting that the MAMLS neural network model was quite successful in making optimal predictions with 23 features that have not been achieved so far using the UNSW NB-15 dataset, which obviously makes this work special.

This work attempted to thoroughly examine the UNSW NB-15 dataset by considering its five feature classes. A methodical investigation of this data set was necessary useful to understand the differences in prediction rates produced by different feature class combinations.

Feature relevance helped explore the features of the data set, while feature reduction helped improve the classification result.

According to an alternative embodiment, a multi-class classification task can be examined to determine the relevance of feature classes for predicting a particular attack type. Feature reduction can then be performed to achieve better results. Since MAMLS also offers various modules other than detectors, the performance evaluation of these detectors could be a possible starting point for data scientists.

The figure and the preceding description give examples of embodiments. Those skilled in the art will understand that one or more of the elements described may well be combined into a single functional element. Alternatively, certain elements can be broken down into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, the order of the processes described herein may be changed and is not limited to the manner described herein. Additionally, the actions of a flowchart need not be performed in the order shown; Also, not all actions have to be carried out. Also, the actions that are not dependent on other actions can be performed in parallel with the other actions. The scope of the embodiments is in no way limited by these specific examples. Numerous variations are possible, regardless of whether they are explicitly mentioned in the description or not, e.g. B. Differences in structure, dimensions and use of materials. The scope of the embodiments is at least as broad as indicated in the following claims.

Advantages, other benefits, and solutions to problems have been described above with respect to particular embodiments. However, the benefits, advantages, problem solutions, and components that can cause an advantage, benefit, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or component of any or all claims.

reference list

100

A system (100) for feature relevance analysis and feature reduction on Microsoft Azure Machine Learning Studio (MAMLS).

102

Record acquisition module

104

Feature extraction module

106

Feature reduction module

108

Classification module.

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• CN 104050787A [0003]
• US8887281B2 [0004]

Claims (6)

A system (100) for feature relevance analysis and feature reduction on Microsoft Azure Machine Learning Studio (MAMLS), the system (100) comprising: a dataset collection module (102) for creating a dataset (UNSW NB-15) of modern attack vectors; a feature extraction module (104) coupled to the data set collection module (102) for processing the generated data set to generate feature sets resulting in the generation of multiple classes of features, the multiple classes of features including multiple flow features, base features, content features , time features and additional features include; a feature reduction module (106) connected to the feature extraction module (104) to reduce the extracted features based on the relevance level to improve the prediction rate of the attack; and a classification module (108) coupled to the feature reduction module (106) and comprising a two-class neural network for predicting the generated multiple classes of features in a network instance into a plurality of classes as anomalous or normal. The system after claim 1 , where the UNSW NB-15 cybersecurity record is created to generate both normal activities and attack patterns. The system after claim 1 , where the flow characteristics reveal basic information of any network packet in terms of Internet protocol (Ips), port numbers and a protocol in the transaction, with the basic characteristics leaning more towards time-to-live (TTL), service, packet count, etc., where the content characteristics correspond to source and destination window sizes, the timing characteristics relate to jitter (delays) of packets in the network, arrival time between packets, round trip time, etc., and the additional characteristics focus primarily on the number of records or network instances with the same IPs, port numbers, flows in the FTP session, same service, etc. system after claim 1 , where the neural network includes a cross-entropy loss function to optimize the log-likelihood of the dataset. system after claim 1 , where the dataset is standardized by the neural network after performing the min-max classification. system after claim 1 wherein the feature reduction module (106) comprises a meta-estimator to select features based on scores of a threshold, wherein the feature with a score less than the threshold is irrelevant and eliminated.

Images