CN114048468A - Intrusion detection method, intrusion detection model training method, device and medium - Google Patents

Intrusion detection method, intrusion detection model training method, device and medium Download PDF

Info

Publication number
CN114048468A
CN114048468A CN202111401951.6A CN202111401951A CN114048468A CN 114048468 A CN114048468 A CN 114048468A CN 202111401951 A CN202111401951 A CN 202111401951A CN 114048468 A CN114048468 A CN 114048468A
Authority
CN
China
Prior art keywords
intrusion detection
training
data
detection model
clusters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111401951.6A
Other languages
Chinese (zh)
Inventor
顾荣松
马改妮
常丽杉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Topsec Technology Co Ltd
Beijing Topsec Network Security Technology Co Ltd
Beijing Topsec Software Co Ltd
Original Assignee
Beijing Topsec Technology Co Ltd
Beijing Topsec Network Security Technology Co Ltd
Beijing Topsec Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Topsec Technology Co Ltd, Beijing Topsec Network Security Technology Co Ltd, Beijing Topsec Software Co Ltd filed Critical Beijing Topsec Technology Co Ltd
Priority to CN202111401951.6A priority Critical patent/CN114048468A/en
Publication of CN114048468A publication Critical patent/CN114048468A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G06F21/554—Detecting local intrusion or implementing counter-measures involving event detection and direct action
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Abstract

The embodiment of the application provides an intrusion detection method, an intrusion detection model training device and a medium, wherein the method comprises the following steps: acquiring an initial data set, wherein the initial data set comprises n pieces of network data; clustering the initial data set to obtain a plurality of clusters; and respectively acquiring partial network data from each cluster of at least partial clusters in the plurality of clusters as training data, and training the constructed intrusion detection model based on the training data to acquire a target intrusion detection model. According to the method and the device, more effective characteristics in the network data can be extracted, so that the related information of the network data is fully extracted, accurate automatic classification of the network data is realized, and the detection accuracy of the intrusion detection model is effectively improved.

Description

Intrusion detection method, intrusion detection model training method, device and medium
Technical Field
The embodiment of the application relates to the field of intrusion detection, in particular to an intrusion detection method, an intrusion detection model training device and an intrusion detection model training medium.
Background
In the related art, there is a technical solution for completing intrusion detection based on a machine learning (neural network, support vector machine, etc.) technique. However, in the face of increasingly complex network environments, many intrusion detection technologies based on feature extraction and classifier design are labeled by human beings in the training process, and data information of network data is also not sufficiently extracted, resulting in low detection accuracy.
Therefore, how to improve the accuracy of intrusion detection becomes an urgent problem to be solved.
Disclosure of Invention
The embodiment of the application provides an intrusion detection method, an intrusion detection model training device and an intrusion detection model training medium, and more effective features in network data can be extracted through some embodiments of the application, so that relevant information of the network data is fully extracted, accurate automatic classification of the network data is further realized, and the detection accuracy of the intrusion detection model is effectively improved.
In a first aspect, some embodiments of the present application provide a method of intrusion detection model training, the method including: acquiring an initial data set, wherein the initial data set comprises n pieces of network data; clustering the initial data set to obtain a plurality of clusters; and respectively acquiring partial network data from each cluster of at least partial clusters in the plurality of clusters as training data, and training the constructed intrusion detection model based on the training data to acquire a target intrusion detection model.
Therefore, different from the technical scheme of manually marking training data in the training process and training according to unprocessed original data in the related art, the embodiment of the application improves the selection method of the training data, namely, the obtained network data is clustered according to a clustering algorithm and some component training data are respectively selected from clusters (or called various types) or partial clusters (or called partial classification) obtained by clustering, various different types of data can be better extracted by respectively selecting the training data from different clusters, richer features can be mined when a model is trained based on the data, more effective data features can be extracted, the classification performance of the model is improved, the detection accuracy of intrusion detection is improved, and the network attack risk can be reduced through the technical scheme of the embodiment of the application, and the network security is improved.
With reference to the first aspect, in some embodiments of the present application, before the obtaining the target intrusion detection model, the method further includes: respectively acquiring partial network data from each cluster of the at least partial clusters as test data, wherein the test data is network data different from the training data; and testing and adjusting the intrusion detection model to be tested obtained through the training according to the test data to obtain the target intrusion detection model.
Therefore, according to the embodiment of the application, the test data are obtained, the intrusion detection model to be detected is tested through the test data, the training condition of the intrusion detection model to be detected can be checked, and whether the model can carry out intrusion detection or not is confirmed through the accuracy of the model.
With reference to the first aspect, in some embodiments of the present application, the clustering the initial data set to obtain a plurality of clusters includes: randomly dividing the initial data set into C clusters, and determining an initial clustering center of each cluster in the C clusters; obtaining a clustering result corresponding to a first cycle according to a clustering cost function and the initial clustering center, wherein the clustering result corresponding to the first cycle comprises a first clustering center determined for each cluster; repeating the following process until the plurality of clusters are obtained: adjusting cluster classification results according to the clustering cost function and the ith clustering center to obtain clustering results corresponding to the ith cycle, wherein i is an integer greater than 1; and the clusters are clustering results obtained by the last cycle when the cycle is terminated.
Therefore, the initial data set is clustered, and data with high reliability can be obtained, so that the quality of training data is improved, the training data with high reliability is reused to train a model, more effective characteristics of network data can be extracted, the performance of a classifier can be improved, and the detection accuracy is improved.
With reference to the first aspect, in some embodiments of the present application, the target intrusion detection model includes a self-encoding module and a classification module; training the constructed intrusion detection model based on the training data comprises the following steps: training a self-coding module to be trained according to the training data to obtain the self-coding module; and training the classification module to be trained according to the training data to obtain the classification module.
Some embodiments of the application train the self-coding module and the classification module respectively based on training data, and can improve the classification precision of the whole target intrusion detection model.
With reference to the first aspect, in some embodiments of the present application, the self-encoding module includes an encoder and a decoder; the training of the self-coding module to be trained according to the training data to obtain the self-coding module comprises the following steps: inputting the training data into the encoder for encoding to obtain an encoded data set; inputting the coding data set into the decoder for reconstruction to obtain an ith generation training data set; and repeating the steps until the ith generated training data set meets a preset condition, and obtaining the self-encoding module.
With reference to the first aspect, in some embodiments of the present application, the encoder includes a first convolutional layer, a pooling layer, and a second convolutional layer in this order, where the first convolutional layer has a size of 3 × 3 × 6, the pooling layer has a size of 2 × 2, and the second convolutional layer has a size of 2 × 2 × 3.
According to some embodiments of the method, a sample selection method is improved, a clustering algorithm is used for automatically obtaining training samples with high reliability, the generalization capability and classification performance of the model are effectively improved, and finally the detection accuracy of the detection model is effectively improved.
In a second aspect, in some embodiments of the present application, there is provided a method of intrusion detection, the method comprising: acquiring a to-be-detected network data set; inputting the network data set to be detected into the target intrusion detection model obtained by the training method in the first aspect and all embodiments thereof, and obtaining an intrusion detection result.
In a third aspect, in some embodiments of the present application, there is provided an apparatus for intrusion detection, which is applied to a target intrusion detection model, where the target intrusion detection model includes: an encoder configured to obtain a primary characteristic of the network data set to be detected from the network data set to be detected; a classification module configured to obtain intrusion detection results based on the primary features.
With reference to the third aspect, in some embodiments of the present application, the encoder is implemented based on a pooling layer and a convolutional layer, wherein the first convolutional layer is connected to the pooling layer and the pooling layer is connected to the second convolutional layer.
Therefore, according to the embodiment of the application, by constructing the target intrusion detection model, richer network data characteristics can be mined, and classification is performed according to the richer characteristics, so that higher detection accuracy is obtained.
In a fourth aspect, in some embodiments of the present application, there is provided a training apparatus for an intrusion detection model, the training apparatus comprising: a data acquisition module configured to acquire an initial data set, wherein the initial data set comprises n pieces of network data; a data clustering module configured to cluster the initial data set to obtain a plurality of clusters; and the model training module is configured to respectively acquire partial network data from each of at least partial clusters in the plurality of clusters as training data, train the constructed intrusion detection model based on the training data and acquire a target intrusion detection model.
In combination with the fourth aspect, in some embodiments of the application, the model training module is further configured to: respectively acquiring partial network data from each cluster of the at least partial clusters as test data, wherein the test data is network data different from the training data; and testing and adjusting the intrusion detection model to be tested obtained through the training according to the test data to obtain the target intrusion detection model.
In combination with the fourth aspect, in some embodiments of the present application, the clustering module is further configured to: randomly dividing the initial data set into C clusters, and determining an initial clustering center of each cluster in the C clusters; obtaining a clustering result corresponding to a first cycle according to a clustering cost function and the initial clustering center, wherein the clustering result corresponding to the first cycle comprises a first clustering center determined for each cluster; repeating the following process until the plurality of clusters are obtained: adjusting cluster classification results according to the clustering cost function and the ith clustering center to obtain clustering results corresponding to the ith cycle, wherein i is an integer greater than 1; and the clusters are clustering results obtained by the last cycle when the cycle is terminated.
With reference to the fourth aspect, in some embodiments of the present application, the target intrusion detection model includes a self-encoding module and a classification module; the model training module is further configured to: training a self-coding module to be trained according to the training data to obtain the self-coding module; and training the classification module to be trained according to the training data to obtain the classification module.
With reference to the fourth aspect, in some embodiments of the present application, the self-encoding module includes an encoder and a decoder; the model training module is further configured to: inputting the training data into the encoder for encoding to obtain an encoded data set; inputting the coding data set into the decoder for reconstruction to obtain an ith generation training data set; and repeating the steps until the ith generated training data set meets a preset condition, and obtaining the self-encoding module.
With reference to the fourth aspect, in some embodiments of the present application, the encoder includes a first convolutional layer, a pooling layer, and a second convolutional layer in this order, where the first convolutional layer has a size of 3 × 3 × 6, the pooling layer has a size of 2 × 2, and the second convolutional layer has a size of 2 × 2 × 3.
In a fifth aspect, in some embodiments of the present application, there is provided an electronic device comprising: a processor, a memory, and a bus; the processor is connected to the memory via the bus, and the memory stores computer readable instructions for implementing the method according to the first and second aspects and any embodiment thereof when the computer readable instructions are executed by the processor.
In a sixth aspect, in some embodiments of the present application, there is provided a computer readable storage medium having stored thereon a computer program for, when executed, implementing the method as described in the first and second aspects and any of the embodiments thereof.
Drawings
Fig. 1 is a schematic diagram illustrating an intrusion detection system according to an embodiment of the present application;
FIG. 2 is a flowchart of a method for intrusion detection according to an embodiment of the present application;
fig. 3 is a schematic diagram illustrating a target intrusion detection model according to an embodiment of the present application;
fig. 4 is a schematic diagram illustrating a composition of an intrusion detection model to be trained according to an embodiment of the present application;
FIG. 5 is a second flowchart of a method for intrusion detection according to an embodiment of the present application;
fig. 6 is a block diagram illustrating an intrusion detection apparatus according to an embodiment of the present application;
fig. 7 is a schematic composition diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, as presented in the figures, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
In some embodiments of the present application, initial data sets composed of collected network data are clustered into a plurality of clusters, then part of the network data are respectively selected from the plurality of clusters as training data, and the constructed intrusion detection model is trained based on the training data, so as to obtain a target intrusion detection model with a better classification effect. It can be understood that the target intrusion detection model obtained through training can perform feature mining and classification on the acquired network data to obtain whether the network data is safe or not, and can further obtain the danger category when the network data is judged to be dangerous data.
For example, in some embodiments of the present application, first, the intrusion detection device clusters a plurality of pieces of acquired network data to obtain a plurality of clusters, and obtains data with high reliability; then extracting data with a certain proportion from each cluster of the plurality of clusters as training data; and respectively inputting the training data into the constructed intrusion detection model and the classification module to train the two models, thereby obtaining a target intrusion detection model for classifying the network data to be detected. Immediately, the obtained network data set to be detected is classified according to the target intrusion detection model, a classification result of the network data set to be detected is obtained, the generalization capability and the classification capability of the model can be effectively improved, and therefore the detection accuracy of the target intrusion detection model is effectively improved.
The method steps in the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 provides a block diagram of a system for intrusion detection in some embodiments of the present application, including a network terminal 110 and an intrusion detection device 120.
The network terminal 110 is capable of accessing a network and generating network data. For example, the network terminal 110 generates various network data in the course of accessing the internet network. In some embodiments of the present application, the network terminal 110 further needs to communicate with the intrusion detection device 120, so that the intrusion detection device 120 can perform intrusion detection on the acquired network data belonging to the network terminal 110. It is understood that the target intrusion detection model obtained after the training is finished is set on the intrusion detection device 120 in the embodiments of the present application.
In order to obtain the target intrusion detection model set on the intrusion detection device 120, it is necessary to first train the constructed intrusion detection model. For example, the training process may include: the intrusion detection device 120 acquires network data generated in the network terminal 110, then clusters the acquired network data (i.e., an initial data set) by a clustering algorithm to obtain a plurality of clusters, acquires a set amount (e.g., 10%) of data from the plurality of clusters as training data, respectively, trains the constructed intrusion detection model based on the training data, and obtains a target detection model.
It should be noted that the network terminal 110 may be any terminal capable of generating network data. As some specific examples of the present application, the network terminal 110 may be a computer, a mobile phone, a tablet computer, a server, and the like. The embodiments of the present application are not limited thereto.
Different from the method for automatically acquiring training data through clustering in the embodiment of the application, the method is characterized in that in the related technology, the acquired network data are manually marked to obtain the training data, and at least because the acquisition mode of the training data cannot sufficiently extract the data information of the network data, the detection accuracy of a target intrusion detection model obtained through the training of an intrusion detection model based on the data is low. In some embodiments of the application, a clustering algorithm is used to select some network data with higher reliability as training data to train the model, so that the target intrusion detection model in the application has higher detection accuracy.
The intrusion detection method provided by some embodiments of the present application is exemplarily described below by taking an intrusion detection device as an example. It can be understood that the technical solution of the intrusion detection method according to the embodiment of the present application can be applied to any security device, for example, a firewall product.
The following first describes a process of training a constructed intrusion detection model to obtain a target intrusion detection model according to some embodiments of the present application.
As shown in fig. 2, some embodiments of the present application provide an intrusion detection model training method, including: s210, acquiring an initial data set; s220, clustering the initial data set to obtain a plurality of clusters; and S230, respectively acquiring partial network data from each of at least partial clusters of the plurality of clusters as training data, and training the constructed intrusion detection model based on the training data to acquire a target intrusion detection model.
The following exemplarily illustrates the implementation of the above steps.
In some embodiments of the present application, the initial data set referred to by S210 includes n pieces of network data. Clustering n pieces of network data through a Fuzzy C-means (FCM) algorithm to obtain a plurality of clusters. For example, by randomly initializing the membership matrix U0Given a fuzzy index m and a threshold epsilon (or number of iterations t)max). Then calculating a clustering center C, and updating a membership matrix U until | | | U(t+1)-U(t)I | < epsilon (or t ═ t)max) And obtaining a plurality of clusters.
For example, the specific steps are as follows:
the method comprises the following steps: and randomly dividing the initial data set into C clusters, and determining the initial clustering center of each cluster in the C clusters.
That is, the initial data set X is divided into C clusters, each having a center CiEach sample xj(j＝1,…,n)Belonging to a certain class CiDegree of membership. Randomly initializing a membership matrix U to satisfyThe following equation:
Figure BDA0003364460090000081
wherein u isijAnd representing the membership degree of the jth data to the ith initial clustering center, and n represents the number of the initial data sets.
Initial cluster center for each of the C clusters: ci(i＝1,…,c)The formula is as follows:
Figure BDA0003364460090000091
wherein, CiDenotes the initial cluster center, u, of each clusterijRepresenting degree of membership, x, of the jth data to the ith initial cluster centerjThe jth sample is represented, n represents the number of the initial data sets, and m is a membership factor and generally takes a value of 2.
Step two: and obtaining a clustering result corresponding to the first circulation according to the clustering cost function and the initial clustering center, wherein the clustering result corresponding to the first circulation comprises the first clustering center determined for each cluster.
The clustering cost function is as follows:
Figure BDA0003364460090000092
wherein, CiAnd representing the initial clustering center of each cluster, and J represents the result value of the clustering cost function.
That is, after the initial clustering center is obtained, the initial clustering center is brought into the clustering cost function, a prediction result corresponding to the minimum value in the calculation result values of the clustering cost function is taken, and a clustering result corresponding to the first cycle is determined.
Step three: the following process is repeated until a plurality of clusters are obtained: and adjusting cluster classification results according to the clustering cost function and the ith clustering center to obtain clustering results corresponding to the ith cycle, wherein i is an integer greater than 1, and the clusters are obtained by the last cycle when the cycle is terminated.
That is, in the process of multiple cycles, the membership matrix U is updated according to the initial clustering center of each cluster, and is as follows:
Figure BDA0003364460090000101
wherein, CiDenotes the initial cluster center, u, of each clusterijRepresenting the membership degree of the jth data to the ith initial clustering center, wherein m is a membership factor and generally takes the value of 2, CtThe initial cluster center for each cluster during the t-th cycle is indicated.
Stopping if the following condition is met, and when J no longer changes, the algorithm has converged to a better result.
||U(t+1)-U(t)I | < epsilon or t ═ tmax
Therefore, the initial data set is clustered, and data with high reliability can be obtained, so that the quality of training data is improved, the training data with high reliability is reused to train a model, more effective characteristics of network data can be extracted, the performance of a classifier can be improved, and the detection accuracy is improved.
In some embodiments of the present application, the network data of the first set ratio (i.e., the partial network data involved in S230) is acquired as the training data from each of some or all of the obtained clusters.
The first setting ratio may be set according to actual conditions. As a specific example of the present application, the first set ratio may be 10%. As another specific example of the present application, the first set ratio may be 20%. The embodiments of the present application are not limited thereto.
In some embodiments of the present application, partial network data is acquired from each of at least partial clusters as test data, respectively, where the test data is network data different from the training data. And then testing and adjusting the to-be-tested intrusion detection model obtained through training according to the test data to obtain a target intrusion detection model.
That is to say, test data of a second set proportion is obtained correspondingly from each cluster, after training of the constructed intrusion detection model is completed by using the training data, the intrusion detection model to be tested is obtained, then the test data is input into the intrusion detection model to be tested for testing, and under the condition that the test meets the condition, the target intrusion detection model is obtained.
The second setting ratio may be set according to actual conditions. As a specific example of the present application, the second set ratio may be 30%. As another specific example of the present application, the second set ratio may be 20%. The embodiments of the present application are not limited thereto.
Therefore, according to the embodiment of the application, the test data are obtained, the intrusion detection model to be detected is tested through the test data, the training condition of the intrusion detection model to be detected can be checked, and whether the model can carry out intrusion detection or not is confirmed through the accuracy of the model.
In some embodiments of the present application, after obtaining the training data and the test data, data preprocessing is required on the training data and the test data before model training. For example: and performing vector-matrix transformation on the training data and the test data, converting the 118-dimensional feature vectors into 11 x 11 feature vectors, filling the blank spaces of the matrix with 0, and then generating a normalized feature matrix.
For example, the step of preprocessing the training data and the test data illustratively includes:
vector data extracted from training data and test data is converted into a matrix form, that is, a 118-dimensional feature vector is converted into a feature matrix of 11 × 11 × 11. In addition, a feature matrix of 11 × 11 × 11 to 9 × 9 × 9 is also required. In order to ensure that the sizes of input training data and test data meet the requirements of convolution operation, zero padding processing is carried out on the input data by using 0 padding operation. And then, carrying out normalization processing on the data, and taking the processed data as training data input in the constructed intrusion detection model.
In some embodiments of the present application, the target intrusion detection model includes a self-encoding module and a classification module. S230 includes: and training the self-coding module to be trained according to the training data to obtain the self-coding module. And training the classification module to be trained according to the training data to obtain the classification module.
The following is an exemplary illustration of the structure of the intrusion network detection module employed in some examples of the present application, and an exemplary illustration of the training process is provided in conjunction with the structure.
As shown in fig. 3, the intrusion detection model constructed by some embodiments of the present application includes a self-encoding module 310 and a classification module 320.
The self-encoding module 310 of fig. 3 includes an encoder (first convolutional layer 311, pooling layer 312, and second convolutional layer 313) and a decoder (second deconvolution layer 403, inverse pooling layer 402, and first deconvolution layer 401). Inputting the training data into an encoder to be encoded to obtain an encoded data set, then inputting the encoded data set into a decoder to be reconstructed to obtain an ith generation training data set, repeating the steps until the ith generation training data set meets a preset condition, and obtaining a self-encoding module.
For example, training data is input into the self-encoding module for intrusion type prediction. And comparing the generated training data set obtained after each feature is reconstructed with the original input training data, and learning the parameters of the whole self-coding module by minimizing a mean square error function E (loss function). And training data as input to the fully connected layer (i.e., the classification module) for training.
The preset condition may be that i in the ith generated training data set satisfies a preset number of cycles, or that the degree of similarity between the ith generated training data set and the input training data reaches a predetermined similarity, where the predetermined similarity may be 90% or 95%, and the preset number of cycles may be 1000 times or 2000 times. The embodiments of the present application are not limited thereto.
As a specific embodiment of the present application, training data is input into the first convolution layer 311, and subjected to dimensionality reduction through the pooling layer 312, and then the dimensionality-reduced data is input into the second convolution layer 313 to encode the data, so as to obtain an encoded data set. The encoded data set is then input into the second deconvolution layer 403, reconstructed through the deconvolution layer 402 and the first deconvolution layer 401, and the i-th generated training data set is obtained. And repeating the steps, and stopping training when the similarity degree between the ith generated training data set and the input training data reaches a set value or i meets the set cycle number to obtain the self-encoding module after training is finished.
As another specific embodiment of the present application, the specific steps of training the constructed intrusion detection model are as follows:
the method comprises the following steps: inputting the training data obtained after normalization into the constructed intrusion detection model, and inputting the training data into an encoder, wherein the encoding formula is as follows:
hk＝σ(X*wk+bk)
wherein, wkRepresenting the weight under k convolution kernels, bkDenotes the deviation under k convolution kernels, X denotes the input training data, hkRepresenting the encoded data set, and sigma represents the sigmoid function.
The sigmoid function is as follows:
σ(x)＝σ(x)(1-σ(x))
step two: h obtained in the step onekAnd (3) performing characteristic reconstruction, wherein a reconstruction formula is as follows:
Figure BDA0003364460090000131
wherein the content of the first and second substances,
Figure BDA0003364460090000132
representing a convolutional array wkThe flip in two dimensions (rot180), c denotes the bias and Y denotes the i-th generated training data set after reconstruction.
Step three: and when the training data is similar to the features of the ith generated training data set as much as possible, the training of the self-editing module is completed. Specifically, a minimum mean square error function is used as the loss function E, and the formula is as follows:
Figure BDA0003364460090000133
step four: and inputting the training data into the full-connection layer, and taking the maximum value as a detection result. The formula is as follows:
Figure BDA0003364460090000134
where l represents a prediction tag. The output result represents the probability that the characteristic data j belongs to the class c, XjAnd WkRespectively representing the jth input vector and the kth weight.
The cost function is also needed to measure the difference between the training output value and the real value of the deep neural network after the full connection layer. In the application, Logistic regression is selected as a cost function, and softmax regression can be used as the cost function. The embodiments of the present application are not limited thereto.
As another specific embodiment of the present application, the encoder includes a first convolutional layer, a pooling layer, and a second convolutional layer in this order, where the size of the first convolutional layer is 3 × 3 × 6, the size of the pooling layer is 2 × 2, and the size of the second convolutional layer is 2 × 2 × 3. The decoder is symmetrical to the structure of the encoder.
Therefore, the embodiment of the application improves the sample selection method, automatically obtains the training sample with high reliability by using the clustering algorithm, effectively improves the generalization capability and classification performance of the model, and finally effectively improves the detection accuracy of the detection model.
In some embodiments of the present application, before obtaining the target intrusion detection model, the method further includes: and optimizing the constructed intrusion detection model by using a BP algorithm.
That is, the BP algorithm is applied to optimize the trained model, and the formula is as follows:
Figure BDA0003364460090000141
where σ h denotes the sensitivity of the hidden layer and δ y denotes the sensitivity of the output layer.
In other embodiments of the present application, the detailed procedure for fine-tuning the trained intrusion detection model using the BP algorithm is as follows.
The method comprises the following steps: a forward propagation process. The reconstructed representation x' is obtained using the training data x as input data.
Step two: and according to the error back propagation, performing back updating of the weight and the bias by using a gradient descent algorithm.
Therefore, different from the technical scheme of manually marking training data in the training process and training according to unprocessed original data in the related art, the embodiment of the application improves the selection method of the training data, namely, the obtained network data is clustered according to a clustering algorithm and some component training data are respectively selected from clusters (or called various types) or partial clusters (or called partial classification) obtained by clustering, various different types of data can be better extracted by respectively selecting the training data from different clusters, richer features can be mined when a model is trained based on the data, more effective data features can be extracted, the classification performance of the model is improved, the detection accuracy of intrusion detection is improved, and the network attack risk can be reduced through the technical scheme of the embodiment of the application, and the network security is improved.
The following exemplary description describes a process of detecting network data based on a trained target intrusion detection model.
As shown in fig. 4, some embodiments of the present application provide an intrusion detection model that includes: an encoder 410 and a classification module 320. For example, encoder 410 includes first convolutional layer 311, pooling layer 312, and second convolutional layer 313.
In some embodiments of the present application, the encoder 410 is configured to obtain a primary feature of the network data set to be detected from the network data set to be detected; a classification module 320 configured to obtain intrusion detection results based on the preliminary features.
That is, the encoder 410 in the target intrusion detection model obtains some primary features of the network data set to be detected based on the pooling layer and the convolutional layer, and inputs some primary features into the classification module 320 for classification, so as to obtain an intrusion detection result.
Specifically, the network data set to be detected is input into the first convolution layer 311 in the encoder 410, and feature extraction is performed on the network data set to be detected through the first convolution layer 311. The extracted features are then input into the pooling layer 312 for dimensionality reduction to preserve the main features and reduce feature dimensionality. And then inputting the features subjected to dimension reduction into the second convolution layer 313 for further extraction to obtain the primary features of the network data set to be detected. The primary features are then input into a classification module 320 for classification to obtain a network data classification result (i.e., intrusion detection result).
It is understood that encoder 410 is implemented based on a pooling layer and a convolutional layer, wherein first convolutional layer 311 is connected to pooling layer 312 and pooling layer 312 is connected to second convolutional layer 313.
As one specific example of this application, the classification module 320 may be a fully-connected layer, where the last layer in the fully-connected layer is a cost function, and after the second convolutional layer 313, the high-level logical reasoning is done through the fully-connected layer, i.e., the neurons of the fully-connected layer are connected to all outputs of the previous layer. And finishing final classification by the full connection layer to obtain an intrusion detection result.
As another specific example in the present application, the size of the first convolution layer is 3 × 3 × 6, the size of the pooling layer is 2 × 2, and the size of the second convolution layer is 2 × 2 × 3.
Therefore, according to the embodiment of the application, by constructing the target intrusion detection model, richer network data characteristics can be mined, and classification is performed according to the richer characteristics, so that higher detection accuracy is obtained.
The above describes the structure of a target intrusion detection model in the present application, and the following describes a method for training an intrusion detection model in the present application.
In some embodiments of the present application, a to-be-detected network data set is obtained, and the to-be-detected network data set is input into a target intrusion detection model for classification, so as to obtain attack detection results of all network data.
That is, as shown in fig. 4, the network data set to be detected is input into the first convolution layer 311 in the encoder 410, and feature extraction is performed on the network data set to be detected through the first convolution layer 311. The extracted features are then input into the pooling layer 312 for dimensionality reduction to preserve the main features and reduce feature dimensionality. And then inputting the features subjected to dimension reduction into the second convolution layer 313 for further extraction to obtain the primary features of the network data set to be detected. The primary features are then input into a classification module 320 (e.g., full connectivity layer) for classification, and a network data classification result (i.e., intrusion detection result) is obtained.
It should be noted that the network data set to be detected may be one piece of network data or multiple pieces of network data. The embodiments of the present application are not limited thereto.
The method of intrusion detection in the embodiment of the present application is described above, and a specific embodiment of intrusion detection in the embodiment of the present application will be described below.
As a specific embodiment of the present application, a system for intrusion detection includes: the system comprises a data preprocessing module, a Clustering Convolution Auto-encoder Network (Clustering-CAE) model construction module and a model optimization and prediction module.
As shown in fig. 5, some embodiments of the present application provide a method of intrusion detection comprising: s510, data preselection; s520, preprocessing data; s530, constructing a CAE model; s540, optimizing and predicting a model; and S550, ending.
The data preselection module is configured to execute S510, first select some network data with higher reliability by using a clustering algorithm, and then divide the network data into two parts, namely training data and test data.
For example: the data pre-selection process comprises the steps of firstly inputting KDDCUP 99 data sets to be detected (each data comprises 41 characteristics and can be divided into a normal type and 39 abnormal types, wherein the 39 abnormal types can be divided into 4 categories, namely Dos, Probe, U2R and R2L, the 4 abnormal types and the normal types form five categories of labels together.
The data preprocessing module is configured to execute S520, convert the data from the vector to a matrix form for storage, and perform normalization processing. S530, constructing a CAE model, and training the CAE model by using training data.
And the model optimizing and predicting module is configured to execute S540 and S550, and predict the whole network data set by using the CAE model obtained by the final training.
Therefore, compared with the traditional network intrusion detection technology, due to the excellent hierarchical characteristics and automatic classification characteristics of the network intrusion detection technology, the detection accuracy can be improved and the risk of network attack is reduced while data information is fully mined.
While a specific embodiment of a method of intrusion detection is described above, an apparatus for intrusion detection model training will be described below.
As shown in fig. 6, an apparatus 600 for training an intrusion detection model includes: a data acquisition module 610, a data clustering module 620, and a model training module 630.
In some embodiments of the present application, there is provided a training apparatus 600 for an intrusion detection model, the training apparatus comprising: a data acquisition module 610 configured to acquire an initial data set, wherein the initial data set includes n pieces of network data; a data clustering module 620 configured to cluster the initial data set to obtain a plurality of clusters; a model training module 630, configured to obtain partial network data from each of at least partial clusters of the multiple clusters as training data, and train the constructed intrusion detection model based on the training data to obtain a target intrusion detection model.
In some embodiments of the present application, the model training module 630 is further configured to: respectively acquiring partial network data from each cluster of the at least partial clusters as test data, wherein the test data is network data different from the training data; and testing and adjusting the intrusion detection model to be tested obtained through the training according to the test data to obtain the target intrusion detection model.
In some embodiments of the present application, the data clustering module 620 is further configured to: randomly dividing the initial data set into C clusters, and determining an initial clustering center of each cluster in the C clusters; obtaining a clustering result corresponding to a first cycle according to a clustering cost function and the initial clustering center, wherein the clustering result corresponding to the first cycle comprises a first clustering center determined for each cluster; repeating the following process until the plurality of clusters are obtained: adjusting cluster classification results according to the clustering cost function and the ith clustering center to obtain clustering results corresponding to the ith cycle, wherein i is an integer greater than 1; and the clusters are clustering results obtained by the last cycle when the cycle is terminated.
In some embodiments of the present application, the target intrusion detection model includes a self-encoding module and a classification module; model training module 630 is further configured to: training a self-coding module to be trained according to the training data to obtain the self-coding module; and training the classification module to be trained according to the training data to obtain the classification module.
In some embodiments of the present application, the self-encoding module comprises an encoder and a decoder; model training module 630 is further configured to: inputting the training data into the encoder for encoding to obtain an encoded data set; inputting the coding data set into the decoder for reconstruction to obtain an ith generation training data set; and repeating the steps until the ith generated training data set meets a preset condition, and obtaining the self-encoding module.
In some embodiments of the present application, the encoder includes a first convolutional layer having a size of 3 × 3 × 6, a pooling layer having a size of 2 × 2, and a second convolutional layer having a size of 2 × 2 × 3 in this order.
In the embodiment of the present application, the module shown in fig. 6 can implement each process in the method embodiments of fig. 1 to 5. The operations and/or functions of the respective modules in fig. 6 are respectively for implementing the corresponding flows in the method embodiments in fig. 1 to 5. Reference may be made specifically to the description of the above method embodiments, and a detailed description is appropriately omitted herein to avoid redundancy.
As shown in fig. 7, an embodiment of the present application provides an electronic device 700, including: a processor 710, a memory 720 and a bus 730, wherein the processor is connected to the memory through the bus, and the memory stores computer readable instructions, which when executed by the processor, are used for implementing the method according to any of the above embodiments, and specifically refer to the description of the above embodiments of the method, and the detailed description is omitted here to avoid redundancy.
Wherein the bus is used for realizing direct connection communication of the components. The processor in the embodiment of the present application may be an integrated circuit chip having signal processing capability. The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The Memory may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Read Only Memory (EPROM), an electrically Erasable Read Only Memory (EEPROM), and the like. The memory stores computer readable instructions that, when executed by the processor, perform the methods described in the embodiments above.
It will be appreciated that the configuration shown in fig. 7 is merely illustrative and may include more or fewer components than shown in fig. 7 or have a different configuration than shown in fig. 7. The components shown in fig. 7 may be implemented in hardware, software, or a combination thereof.
Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed, the method in any of the above-mentioned all embodiments is implemented, in particular, refer to the description in the above-mentioned method embodiments, and in order to avoid repetition, detailed description is appropriately omitted here.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (12)

1. A method of intrusion detection model training, the method comprising:
acquiring an initial data set consisting of network data, wherein the initial data set comprises n pieces of network data, and n is an integer greater than 1;
clustering the initial data set to obtain a plurality of clusters;
and respectively acquiring partial network data from each cluster of at least partial clusters in the plurality of clusters as training data, and training the constructed intrusion detection model based on the training data to acquire a target intrusion detection model.
2. The method of claim 1, wherein prior to said obtaining a target intrusion detection model, the method further comprises:
respectively acquiring remaining part of network data from each of the at least part of clusters as test data, wherein the test data is network data different from the training data;
and testing and adjusting the to-be-tested intrusion detection model obtained through training according to the test data to obtain the target intrusion detection model.
3. The method of claim 1, wherein clustering the initial data set to obtain a plurality of clusters comprises:
randomly dividing the initial data set into C clusters, and determining an initial clustering center of each cluster in the C clusters;
obtaining a clustering result corresponding to a first cycle according to a clustering cost function and the initial clustering center, wherein the clustering result corresponding to the first cycle comprises a first clustering center determined for each cluster;
repeating the following process until the plurality of clusters are obtained:
adjusting cluster classification results according to the clustering cost function and the ith clustering center to obtain clustering results corresponding to the ith cycle, wherein i is an integer greater than 1;
and the clusters are clustering results obtained by the last cycle when the cycle is terminated.
4. The method of any one of claims 1-3, wherein the target intrusion detection model comprises a self-encoding module and a classification module;
training the constructed intrusion detection model based on the training data comprises the following steps:
training a self-coding module to be trained according to the training data to obtain the self-coding module; and training the classification module to be trained according to the training data to obtain the classification module.
5. The method of claim 4, wherein the self-encoding module comprises an encoder and a decoder;
the training of the self-coding module to be trained according to the training data to obtain the self-coding module comprises the following steps:
inputting the training data into the encoder for encoding to obtain an encoded data set;
inputting the coding data set into the decoder for reconstruction to obtain an ith generation training data set;
and repeating the steps until the ith generated training data set meets a preset condition, and obtaining the self-encoding module.
6. The method of claim 5, wherein the encoder comprises, in order, a first convolutional layer having a size of 3 x 6, a pooling layer having a size of 2 x 2, and a second convolutional layer having a size of 2 x 3.
7. A method of intrusion detection, the method comprising:
acquiring a to-be-detected network data set;
inputting the network data set to be detected into a target intrusion detection model obtained by the method according to any one of claims 1 to 6, and obtaining intrusion detection results.
8. An intrusion detection apparatus, applied to a target intrusion detection model, the target intrusion detection model comprising:
an encoder configured to obtain a primary characteristic of the network data set to be detected from the network data set to be detected;
a classification module configured to obtain intrusion detection results based on the primary features.
9. The apparatus of claim 8, wherein the encoder is implemented based on a pooling layer and a convolutional layer, wherein a first convolutional layer is connected to the pooling layer and the pooling layer is connected to a second convolutional layer.
10. An intrusion detection model training apparatus, comprising:
a data acquisition module configured to acquire an initial data set, wherein the initial data set comprises n pieces of network data;
a data clustering module configured to cluster the initial data set to obtain a plurality of clusters;
and the model training module is configured to respectively acquire partial network data from each of at least partial clusters in the plurality of clusters as training data, train the constructed intrusion detection model based on the training data and acquire a target intrusion detection model.
11. An electronic device, comprising: a processor, a memory, and a bus;
the processor is connected to the memory via the bus, the memory storing computer readable instructions for implementing the method of any one of claims 1-7 when the computer readable instructions are executed by the processor.
12. A computer-readable storage medium, having stored thereon a computer program which, when executed, implements the method of any one of claims 1-7.
CN202111401951.6A 2021-11-19 2021-11-19 Intrusion detection method, intrusion detection model training method, device and medium Pending CN114048468A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111401951.6A CN114048468A (en) 2021-11-19 2021-11-19 Intrusion detection method, intrusion detection model training method, device and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111401951.6A CN114048468A (en) 2021-11-19 2021-11-19 Intrusion detection method, intrusion detection model training method, device and medium

Publications (1)

Publication Number Publication Date
CN114048468A true CN114048468A (en) 2022-02-15

Family

ID=80210621

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111401951.6A Pending CN114048468A (en) 2021-11-19 2021-11-19 Intrusion detection method, intrusion detection model training method, device and medium

Country Status (1)

Country Link
CN (1) CN114048468A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114584377A (en) * 2022-03-04 2022-06-03 奇安信科技集团股份有限公司 Flow anomaly detection method, model training method, device, equipment and medium
CN114884755A (en) * 2022-07-12 2022-08-09 广东瑞普科技股份有限公司 Network security protection method and device, electronic equipment and storage medium
CN115062721A (en) * 2022-07-01 2022-09-16 中国电信股份有限公司 Network intrusion detection method and device, computer readable medium and electronic equipment
CN115080963A (en) * 2022-07-07 2022-09-20 济南开耀网络技术有限公司 Intelligent financial data protection method based on cloud computing and server
CN116112288A (en) * 2023-04-07 2023-05-12 天翼云科技有限公司 Network intrusion detection method, device, electronic equipment and readable storage medium

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114584377A (en) * 2022-03-04 2022-06-03 奇安信科技集团股份有限公司 Flow anomaly detection method, model training method, device, equipment and medium
CN115062721A (en) * 2022-07-01 2022-09-16 中国电信股份有限公司 Network intrusion detection method and device, computer readable medium and electronic equipment
CN115062721B (en) * 2022-07-01 2023-10-31 中国电信股份有限公司 Network intrusion detection method and device, computer readable medium and electronic equipment
CN115080963A (en) * 2022-07-07 2022-09-20 济南开耀网络技术有限公司 Intelligent financial data protection method based on cloud computing and server
CN115080963B (en) * 2022-07-07 2023-04-04 上海量化森林科技有限公司 Intelligent financial data protection method and server based on cloud computing
CN114884755A (en) * 2022-07-12 2022-08-09 广东瑞普科技股份有限公司 Network security protection method and device, electronic equipment and storage medium
CN114884755B (en) * 2022-07-12 2022-09-16 广东瑞普科技股份有限公司 Network security protection method and device, electronic equipment and storage medium
CN116112288A (en) * 2023-04-07 2023-05-12 天翼云科技有限公司 Network intrusion detection method, device, electronic equipment and readable storage medium
CN116112288B (en) * 2023-04-07 2023-08-04 天翼云科技有限公司 Network intrusion detection method, device, electronic equipment and readable storage medium

Similar Documents

Publication Publication Date Title
CN114048468A (en) Intrusion detection method, intrusion detection model training method, device and medium
CN111738351B (en) Model training method and device, storage medium and electronic equipment
CN109389171B (en) Medical image classification method based on multi-granularity convolution noise reduction automatic encoder technology
CN105975573A (en) KNN-based text classification method
CN113095370B (en) Image recognition method, device, electronic equipment and storage medium
CN109635010B (en) User characteristic and characteristic factor extraction and query method and system
CN114019370B (en) Motor fault detection method based on gray level image and lightweight CNN-SVM model
CN113869208A (en) Rolling bearing fault diagnosis method based on SA-ACWGAN-GP
CN110851654A (en) Industrial equipment fault detection and classification method based on tensor data dimension reduction
CN112364974B (en) YOLOv3 algorithm based on activation function improvement
CN116596095B (en) Training method and device of carbon emission prediction model based on machine learning
CN116595463A (en) Construction method of electricity larceny identification model, and electricity larceny behavior identification method and device
CN112149909A (en) Ship oil consumption prediction method and device, computer equipment and storage medium
CN111371611B (en) Weighted network community discovery method and device based on deep learning
CN113052577A (en) Method and system for estimating category of virtual address of block chain digital currency
CN111915595A (en) Image quality evaluation method, and training method and device of image quality evaluation model
CN110135428B (en) Image segmentation processing method and device
CN113487223B (en) Risk assessment method and system based on information fusion
CN114003900A (en) Network intrusion detection method, device and system for secondary system of transformer substation
CN111783688B (en) Remote sensing image scene classification method based on convolutional neural network
CN110991247B (en) Electronic component identification method based on deep learning and NCA fusion
CN117011274A (en) Automatic glass bottle detection system and method thereof
CN113541985A (en) Internet of things fault diagnosis method, training method of model and related device
CN114387524B (en) Image identification method and system for small sample learning based on multilevel second-order representation
CN116958622A (en) Data classification method, device, equipment, medium and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination