CN111683108B

CN111683108B - Method for generating network flow anomaly detection model and computer equipment

Info

Publication number: CN111683108B
Application number: CN202010823315.1A
Authority: CN
Inventors: 吕麒; 李伟超; 汪漪; 金波
Original assignee: Southwest University of Science and Technology; Peng Cheng Laboratory
Current assignee: Southwest University of Science and Technology; Peng Cheng Laboratory
Priority date: 2020-08-17
Filing date: 2020-08-17
Publication date: 2020-11-17
Anticipated expiration: 2040-08-17
Also published as: CN111683108A; WO2022037191A1

Abstract

The invention provides a method for generating a network flow abnormity detection model and computer equipment, wherein the method for generating the network flow abnormity detection model comprises the following steps: training the first network model based on a source domain to obtain a trained first network model, wherein the trained first network model comprises a source domain feature extractor and a classifier; training based on the target domain, the source domain feature extractor and the discriminator pair to obtain a target domain feature extractor; and generating a network flow abnormity detection model according to the target domain feature extractor and the classifier. According to the invention, the features extracted by the target domain feature extractor on the target domain are similar to the features extracted by the source domain feature extractor on the source domain through training, and further, the classifier obtained based on the source domain training in the network flow anomaly detection model can be used for anomaly detection on the target domain and has high accuracy.

Description

Method for generating network flow anomaly detection model and computer equipment

Technical Field

The present application relates to the field of network data detection technologies, and in particular, to a method for generating a network flow anomaly detection model and a computer device.

Background

Network attacks are a serious problem of increasingly close contact in the current society, and with the development and continuous expansion of the application range of networks, network intrusion means are changed day by day, so that the damage is more and more serious. Intrusion refers to an attempt to access an associated computer system or to disrupt the operation of the system in an illegal or unauthorized manner. The anomaly detection can well detect new network intrusion behaviors.

The existing anomaly detection method does not consider the influence of the network data characteristic scene change on the algorithm performance, model training and model detection are carried out on the same data set, only a model trained on a certain data set can be described, and the method is effective for detecting the data set. In a new scene, the model needs to be adjusted, and the adjustment of the model depends on a large amount of marked data, so that the method is not suitable for an environment with less data and no label.

Therefore, the prior art is in need of improvement.

Disclosure of Invention

The invention provides a method for generating a network flow anomaly detection model and computer equipment, wherein the features extracted by a trained target domain feature extractor on a target domain are similar to the features extracted by a source domain feature extractor on a source domain, so that the classifier obtained based on the source domain training in the network flow anomaly detection model can be used for carrying out anomaly detection on the target domain and has high accuracy.

In a first aspect, an embodiment of the present invention provides a method for generating a network flow anomaly detection model, including:

training a first network model based on a source domain to obtain a trained first network model, wherein the trained first network model comprises a source domain feature extractor and a classifier;

training a second network model based on a target domain, the source domain feature extractor and a discriminator to obtain a target domain feature extractor;

and generating a network flow abnormity detection model according to the target domain feature extractor and the classifier.

In a second aspect, an embodiment of the present invention provides an anomaly detection method for a network flow, which is applied to a network flow anomaly detection model, where the network flow anomaly detection model includes a target domain feature extractor and a classifier, and the anomaly detection method for the network flow includes:

the network flow abnormity detection model acquires a to-be-detected network flow in a target domain;

the target domain feature extractor extracts a to-be-detected feature vector corresponding to the to-be-detected network flow, wherein the target domain feature extractor is a target domain feature extractor in the method for generating the network flow anomaly detection model;

and the classifier classifies the feature vector to be detected to obtain an anomaly detection result corresponding to the feature vector to be detected, wherein the classifier is the classifier in the method for generating the network flow anomaly detection model.

In a third aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the following steps when executing the computer program:

generating a network flow abnormity detection model according to the target domain feature extractor and the classifier;

or, the network flow abnormity detection model acquires the network flow to be detected in the target domain;

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the following steps:

Compared with the prior art, the embodiment of the invention has the following advantages:

the invention provides a method for generating a network flow anomaly detection model, which comprises the following steps: training a first network model based on a source domain to obtain a trained first network model, wherein the trained first network model comprises a source domain feature extractor and a classifier; training a second network model based on a target domain, the source domain feature extractor and a discriminator to obtain a target domain feature extractor; and generating a network flow abnormity detection model according to the target domain feature extractor and the classifier. In the invention, data in a target domain has no label, a second network model is trained in a countermeasure generating mode to obtain a target domain feature extractor, so that the target domain feature extractor can map the data on the target domain to a feature space similar to a source domain to minimize the spatial distance between the feature space of the target domain and the feature of the source domain, and the feature extracted by the target domain feature extractor on the target domain is similar to the feature extracted by the source domain feature extractor on the source domain, thereby completing the adaptation process from the source domain to the target domain; furthermore, when the classifier obtained by the source domain training is used in a new scene, the new scene does not need to have a large amount of labeled data for secondary training. The classifier obtained based on source domain training in the network flow anomaly detection model can be used for carrying out anomaly detection on the target domain, and the accuracy is high.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic diagram of an application field of a method for generating a network flow anomaly detection model according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of a method for generating a network flow anomaly detection model according to an embodiment of the present invention;

FIG. 3 is a diagram illustrating a process of determining normal network flows and abnormal network flows from a source domain according to an embodiment of the present invention;

FIG. 4 is a diagram illustrating a first network stream converted into a first string according to an embodiment of the present invention;

fig. 5 is a schematic diagram of a format obtained by parsing a first network flow through packet grabbing (wirehards) according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a first three-dimensional tensor stored as a Numpy zip (NPZ) file according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a process of extracting flow feature vectors by a convolutional neural network according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a convolutional neural network according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of a recurrent neural network in an embodiment of the present invention;

FIG. 10 is a diagram illustrating the generation of an abnormal network flow by a vector generator when the abnormal network flow in the source domain is insufficient according to an embodiment of the present invention;

FIG. 11 is a diagram illustrating a structure of a vector generator according to an embodiment of the present invention;

FIG. 12 is a diagram illustrating a process for training a second network model according to an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of a network flow anomaly detection model according to an embodiment of the present invention;

fig. 14 is a schematic stage diagram of a method for generating a network flow anomaly detection model in an embodiment of the present invention in a specific implementation;

fig. 15 is a flowchart illustrating an anomaly detection method for network flows according to an embodiment of the present invention;

fig. 16 is a flowchart illustrating an anomaly detection method for network flows according to an embodiment of the present invention;

fig. 17 is an internal structural view of a computer device in the embodiment of the present invention.

Detailed Description

The invention provides a method for generating a network flow anomaly detection model and computer equipment, and in order to make the purpose, technical scheme and effect of the invention clearer and clearer, the invention is further described in detail below by referring to the attached drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The inventor finds that, in the prior art, although a good classifier can be trained on a large data set through a deep learning method, the trained model cannot be directly popularized to a new scene with different data distribution characteristics. A typical solution is to train the model first and then further tune (Fine-tuning) the model to the task-specific data set. However, this is extremely difficult and expensive, especially in the field of network anomaly detection, and it is often difficult to obtain enough data with tags to tune deep neural networks with huge numbers of parameters. That is to say, the existing anomaly detection method does not consider the influence of the network data characteristic scene change on the algorithm performance, the model training and the model detection are performed on the same data set, only a model trained on a certain data set can be described, and the method is effective for the detection of the data set. In a new scene, the model needs to be adjusted, and the adjustment of the model depends on a large amount of marked data, so that the method is not suitable for an environment with less data and no label.

In order to solve the above problem, in an embodiment of the present invention, a first network model is trained based on a source domain to obtain a trained first network model, where the trained first network model includes a source domain feature extractor and a classifier; training a second network model based on a target domain, the source domain feature extractor and a discriminator to obtain a target domain feature extractor; and generating a network flow abnormity detection model according to the target domain feature extractor and the classifier. The data in the target domain has no label, a second network model is trained in a countermeasure generating mode to obtain a target domain feature extractor, so that the target domain feature extractor can map the data in the target domain to a feature space similar to that in the source domain, the spatial distance between the feature space in the target domain and the feature in the source domain is minimized, the features extracted by the target domain feature extractor on the target domain are similar to the features extracted by the source domain feature extractor on the source domain, and the adaptation process from the source domain to the target domain is completed; furthermore, when the classifier obtained by the source domain training is used in a new scene, the new scene does not need to have a large amount of labeled data for secondary training. The classifier obtained based on source domain training in the network flow anomaly detection model can be used for carrying out anomaly detection on the target domain, and the accuracy is high.

The embodiment provides a generation method of a network flow anomaly detection model and computer equipment, wherein the generation method of the network flow anomaly detection model can be applied to a scene shown in FIG. 1. In this scenario, first, the terminal device 1 may collect a source domain and a target domain, and input the source domain and the target domain into the server 2, so that the server 2 trains the first network model and the second network model according to the source domain and the target domain. The server 2 may pre-select and store a first network model and a second network model, respond to the input source domain and target domain of the terminal device 1 to train the first network model and the second network model to obtain a target domain feature extractor and a classifier, and generate a network flow anomaly detection model according to the target domain feature extractor and the classifier.

It is to be understood that, in the application scenario described above, although the actions of the embodiment of the present invention are described as being performed partially by the terminal device 1 and partially by the server 2, the actions may be performed completely by the server 2 or completely by the terminal device 1. The invention is not limited in its implementation to the details of execution, provided that the acts disclosed in the embodiments of the invention are performed.

Further, after the network flow anomaly detection model is generated, the network flow anomaly detection model is applied to an electronic device for detecting whether the network flow to be detected, which is acquired by the electronic device from the target domain, is anomalous, and the electronic device includes a PC, a server, a mobile phone, a tablet computer, a palm computer, a Personal Digital Assistant (PDA), and the like.

It should be noted that the above application scenarios are only presented to facilitate understanding of the present invention, and the embodiments of the present invention are not limited in any way in this respect. Rather, embodiments of the present invention may be applied to any scenario where applicable.

The invention will be further explained by the description of the embodiments with reference to the drawings.

Referring to fig. 2, the embodiment provides a method for generating a network flow anomaly detection model, including:

s1, training the first network model based on the source domain to obtain a trained first network model, wherein the trained first network model comprises a source domain feature extractor and a classifier.

In the embodiment of the present invention, the first network model is a deep learning network model, the traffic in the source domain is labeled traffic, the label is used to indicate that the network traffic in the source domain is normal traffic or abnormal traffic, the first network model is trained through the source domain to obtain a trained first network model, and the trained first network model includes a source domain feature extractor and a classifier.

Specifically, step S1 includes:

and S11, inputting normal network flows in training data and abnormal network flows in the training data into the first network model, and generating a first detection score corresponding to the normal network flows and a second detection score corresponding to the abnormal network flows through the first network model, wherein the training data comprises a plurality of training groups, and each training group comprises normal network flows from a source domain and abnormal network flows from the source domain.

In this embodiment of the present invention, the first network model includes a first sub-network and a second sub-network, the first sub-network is configured to extract a flow feature vector of each network flow (including a normal network flow and an abnormal network flow), and the second sub-network is configured to classify the extracted flow feature vector and output a score corresponding to the flow feature vector. Inputting normal network flow and abnormal network flow of a source domain into the first sub-network to obtain a normal flow feature vector corresponding to the normal network flow and an abnormal flow feature vector corresponding to the abnormal network flow, and inputting the normal flow feature vector and the abnormal flow feature vector into the second sub-network to obtain a first detection score corresponding to the normal flow feature vector and a second detection score corresponding to the abnormal flow feature vector. The first detection score is a score obtained by the second sub-network based on the normal flow feature vector, and the second detection score is a score obtained by the second sub-network based on the abnormal flow feature vector. A detailed process of inputting the normal network flow in the training data and the abnormal network flow in the training data into the first network model and generating the first detection score corresponding to the normal network flow and the second detection score corresponding to the abnormal network flow by the first network model will be described in detail later.

In an embodiment of the invention, each training set in the training data comprises normal network flows from a source domain and abnormal network flows from the source domain; from the source domain, normal network flows and abnormal network flows can be determined.

Specifically, referring to fig. 3, the process of determining each normal network flow and each abnormal network flow from the source domain is as follows:

11. acquiring a source domain;

12. dividing a large Packet Capture (PCAP) file in a source domain to obtain a PCAP file with network flow as a division granularity;

21. filtering each network flow to filter out the network flows which can not identify the labels;

22. adding a label to each filtered network flow to obtain each first network flow and each second network flow data, wherein the first network flow is the network flow added with a normal label, and the second network flow is the network flow added with an abnormal label;

31. generating first three-dimensional tensors with preset sizes according to the first network flows, and generating second three-dimensional tensors with preset sizes according to the second network flows;

41. and storing each first three-dimensional tensor as a Numpy zip (NPZ) file to obtain each normal network stream, and storing each second three-dimensional tensor as an NPZ file to obtain each abnormal network stream. Normal network flows and abnormal network flows are inputs to the first network model.

How to get normal network flow and abnormal network flow in the training data is described in detail next. Before step S11, the method further includes:

and M, determining each abnormal network flow and each normal network flow based on the source domain.

In the embodiment of the present invention, a source domain includes a plurality of network flows, and firstly, a label of each network flow may be determined according to a description file of a data set, where the label includes a normal label and an abnormal label, and a label is added to each network flow to obtain a first network flow and a second network flow, where the first network flow is a network flow to which a normal label is added, and the second network flow is a network flow to which an abnormal label is added. And secondly, generating a first three-dimensional tensor according to the first network flow to obtain normal flow, and generating a second three-dimensional tensor according to the second network flow to obtain abnormal network flow.

Specifically, the step M includes:

m1, extracting each first network flow and each second network flow in the source domain.

In the embodiment of the present invention, a network flow corresponding to a source domain is captured, the captured network flow corresponding to the source domain is usually stored through a PCAP file, the network flow corresponding to the source domain is usually large, for example, the size of the network flow corresponding to the source domain is several G to several tens G, the network flow corresponding to the source domain includes thousands of data packets, and usually, the data packets of a certain network within a certain time period are collected. The network flow corresponding to the source domain is stored through the PCAP file, so that the PCAP file corresponding to the source domain can be obtained, the PCAP file corresponding to the source domain is cut, and each sub-PACP file is obtained and corresponds to one cut network flow.

Specifically, a pkt2flow tool is used to cut captured packets (a PCAP file corresponding to a source domain includes a plurality of packets) in units of streams (five-tuple: source IP, source port, destination IP, destination port, protocol) to obtain a plurality of sub-PCAP files, each sub-PCAP file represents one stream, and the file names of the sub-PCAP files may be named by the five-tuple of each PCAP file.

And for each cut sub PCAP file, filtering the sub PCAP file with the type which cannot be identified in each sub PCAP file to obtain a filtered sub PACP file. The sub PCAP file of the unrecognizable type means that it is not recognizable that the sub PCAP file is normal or abnormal network traffic.

For the filtered sub-PACP files, adding labels to the PCAP files according to the description files of the data set; the data set comprises network traffic stored in a PCAP file and a Comma-separated Values (CSV) file marked in units of streams; the CSV file records whether each sub-PCAP file is normal traffic or abnormal traffic, and this operation is to write code according to the description of the CSV file to add a tag to each sub-PCAP file, where the tag includes a normal tag and an abnormal tag. The addition of the tag changes the file name of the child PCAP file, that is, whether the child PCAP file is normal traffic or abnormal traffic can be determined by the file name of the child PCAP file. For convenience of description, the sub-PCAP file to which the normal tag is added is denoted as a first network flow, and the sub-PCAP file to which the abnormal tag is added is denoted as a second network flow.

M2, generating first three-dimensional tensors of a preset size according to the first network flows, and taking the first three-dimensional tensors as the normal network flows.

In this embodiment of the present invention, for each first network flow, a plurality of first network packets in the first network flow are extracted, and a first three-dimensional tensor is generated according to each first network packet.

Specifically, step M2 includes:

m21, for each first network flow, extracting each first network packet corresponding to the first network flow.

In the embodiment of the invention, a plurality of first network data packets in the first network flow are extracted; specifically, for a first network flow, a packet object is obtained through an rdpcap () function in a Scapy packet, and this operation can obtain each packet object corresponding to the first network flow, that is, a plurality of first network data Packets.

M22, obtaining, according to each first network data packet, each first three-dimensional tensor with a preset size corresponding to the first network flow, and taking each first three-dimensional tensor as each normal network flow.

In the embodiment of the present invention, first, each first network packet is converted into a first character string respectively corresponding to each first network packet, a first two-dimensional tensor can be obtained according to each first character string, a first three-dimensional tensor can be obtained according to a first two-dimensional tensor corresponding to each first character string, and the preset size can be used to represent the size of the first two-dimensional tensor and the number of the first two-dimensional tensors.

Specifically, step M22 includes:

and M221, performing serialization processing on each first network data packet to obtain first character strings respectively corresponding to the first network data packets.

In the embodiment of the invention, each first network data packet is firstly serialized to obtain a first character string, the first character string is a character string in a hexadecimal number form, the value of the hexadecimal number is in a [0,255] interval, and the hexadecimal number is consistent with the gray value range of an image, that is, the first character string corresponding to the first network data packet can indirectly represent the gray value of the image. In practical application, referring to fig. 4 and 5, fig. 4 is a format obtained by converting the first network packet into the first character string, and fig. 5 is a format obtained by parsing the first network packet through a grab packet (wirehardks), it can be seen that a value of each field obtained by parsing the first network packet is completely consistent with a field value parsed through the wirehardks, that is, it is practical in the embodiment of the present invention to convert the first network packet into the first character string.

And M222, generating first three-dimensional tensors with preset sizes according to the first character strings, and taking the first three-dimensional tensors as the normal network streams.

In the embodiment of the present invention, the preset size includes the number of packets and the size of each intercepted packet, the number of network packets is denoted as pkt _ num, and the size of each intercepted packet is denoted as pkt _ size. Sequentially reading each first network data packet, and for each first network data packet, generating a first two-dimensional tensor according to a first character string corresponding to the first network data packet and the size of each intercepted data packet, wherein the size of the first two-dimensional tensor is determined according to the size of each intercepted data packet, each intercepted data packet comprises pkt _ size effective characters, the effective characters refer to the number part in the character string, and \ x is divided into effective characters, such as \ xff; other symbols in the character string are used as identifiers, are not converted, and are only used for identification of character string analysis. A valid character has a meaning corresponding to a hexadecimal number, such as \ xff, and a meaning corresponding to a hexadecimal number, stored in one byte.

For example, pkt _ size is 484, and a first two-dimensional tensor of 22 × 22 in size is generated from the first 484 significant characters in the first string. If the number of effective characters in the first character string exceeds 484 bytes, only the first 484 bytes of the first character string are taken to generate a first two-dimensional tensor, if the number of effective characters in the first character string is less than 484 bytes, 0 is added to the end of the first character string to obtain a first character string with the size of 484 bytes, and then a first two-dimensional tensor with the size of 22 x 22 is generated according to the first character string with 0 added.

And generating a first three-dimensional tensor according to the first two-dimensional tensor corresponding to each first character string. The number of the first character strings is the same as the number of the first data packets, and a first three-dimensional tensor is generated according to each first character string and the number of the network data packets. If the number of the first data packets is larger than the number of the network data packets, only the number of the first network data packets is selected to generate a first three-dimensional tensor; if the number of the first network data packets is smaller than the number of the network data packets (the number of the first character strings is smaller than the number of the network data packets), generating a zero matrix when generating the first three-dimensional tensor, so that the size of the first three-dimensional tensor after the zero matrix is added is a preset size.

For example, assuming that pkt _ size is 484 and pkt _ num is 10, the predetermined size is 10 × 22, i.e., the size of the first three-dimensional tensor is 10 × 22; assuming that, for a first network flow, the first network flow includes 15 first network packets, a first three-dimensional tensor is generated according to the first 10 first network packets in the 15 first network packets, and the size of the first three-dimensional tensor is: 10*22*22. Assuming that, for a first network flow comprising 8 first network packets, a three-dimensional tensor is generated based on the 8 first network packets, and a zero matrix is directly generated using np.

In the embodiment of the invention, the first three-dimensional tensor is stored in an NPZ form to obtain a normal network flow; referring to fig. 6, fig. 6 is a schematic diagram of the first three-dimensional tensor stored in NPZ form.

M3, generating second three-dimensional tensors of the preset size according to the second network flows, and taking the second three-dimensional tensors as the abnormal network flows.

In this embodiment of the present invention, for each second network flow, each second network packet in the second network flow is extracted, and a second three-dimensional tensor is generated according to each second network packet.

Specifically, step M3 includes:

m31, for each first network flow, extracting each first network packet corresponding to the first network flow.

In the embodiment of the invention, a plurality of second network data packets in the second network flow are extracted; specifically, for the second network stream, a packet object is obtained through an rdpcap () function in a Scapy packet, and this operation can obtain each packet object corresponding to the second network stream, that is, a plurality of second network data Packets.

M32, obtaining, according to each first network data packet, each first three-dimensional tensor with a preset size corresponding to the first network flow, and taking each first three-dimensional tensor as each normal network flow.

In the embodiment of the present invention, first, each second network packet is converted into a corresponding second string, a second two-dimensional tensor can be obtained according to each second string, a second three-dimensional tensor can be obtained according to a corresponding second two-dimensional tensor of each second string, and the preset size can be used to represent the size of the second two-dimensional tensor and the number of the second two-dimensional tensors.

Specifically, step M32 includes:

m321, performing serialization processing on each second network data packet to obtain second character strings respectively corresponding to each second network data packet;

and M322, generating a second three-dimensional tensor of the preset size according to each second character string, and taking the second three-dimensional tensor as the abnormal network flow.

In this embodiment of the present invention, the execution process of "performing serialization processing on each second network data packet to obtain a second character string corresponding to each second network data packet" includes: the execution process of "performing serialization processing on each first network data packet to obtain the first character string corresponding to each first network data packet" is consistent, and further, the specific description of step M321 may refer to the description of step M221 above.

In this embodiment of the present invention, the executing process of "generating the second three-dimensional tensor with the preset size according to each second character string, and using the second three-dimensional tensor as the abnormal network flow" includes: the execution process of "generating a first three-dimensional tensor with a preset size according to each first character string, and using the first three-dimensional tensor as the normal network stream" is consistent, and further, for a specific description of step M322, reference may be made to the description of step M222 above.

Next, a specific process of inputting a normal network flow in training data and an abnormal network flow in the training data into the first network model, and generating a first detection score corresponding to the normal network flow and a second detection score corresponding to the abnormal network flow by the first network model will be described in detail.

In this embodiment of the present invention, the first network model includes a first sub-network and a second sub-network, and the first sub-network is used to extract normal flow feature vectors corresponding to normal network flows and abnormal flow feature vectors corresponding to abnormal network flows. And inputting the normal flow feature vector and the abnormal flow feature vector into a second sub-network to obtain a first detection score corresponding to the normal flow feature vector and a second detection score corresponding to the abnormal flow feature vector.

For ease of illustration, normal network flows and abnormal network flows are collectively referred to as network flows. The first sub-network comprises a Convolutional Neural Network (CNN) and a recurrent neural network (GRU), and the CNN is used for learning the spatial characteristics of the network flow, and the GRU is used for learning the time sequence characteristics of the network flow. Specifically, referring to fig. 7, each network flow is in the form of a three-dimensional tensor (n × m), and may be divided into n two-dimensional vectors (m × m), which are packet feature vectors of a data packet; specifically, for the first three-dimensional tensor, the first three-dimensional tensor includes first two-dimensional vectors respectively corresponding to the first network data packets, that is, packet vectors respectively corresponding to the first network data packets; and for the second three-dimensional tensor, the second three-dimensional tensor comprises the packet vectors respectively corresponding to the second network data packets.

Inputting each packet vector with the size of m × m into a CNN to obtain packet feature vectors corresponding to each packet vector, splicing each packet feature vector into a feature vector by using an np.concatenate () function, inputting the spliced feature vectors into a GRU, and learning the time sequence features of the spliced feature vectors to obtain the stream feature vectors corresponding to the network stream. For normal network flows, normal flow feature vectors corresponding to the normal network flows are obtained through CNNs and GRUs; and for the abnormal network flow, obtaining an abnormal flow characteristic vector corresponding to the abnormal network flow through the CNN and the GRU.

In the embodiment of the present invention, the entries of the CNN are in the form of three-dimensional tensors (n × M), and in step M2, a first three-dimensional tensor of a preset size is generated, and in step M3, a second three-dimensional tensor of a preset size is generated. The first three-dimensional tensor is an input item of a normal network flow, namely a CNN, and the second three-dimensional tensor is an input item of the CNN.

Since the CNN requires that the input data have a fixed size, the preset size includes the number of packets and the size of each intercepted packet, and the number of packets and the size of each intercepted packet have a great influence on the algorithm. Some attack types, such as DoS attacks, may have a greater relationship with the packet header data and the first few packets in a stream, and some attack types, such as XSS attacks, may have a greater relationship with the data of the load, and therefore determining which portion of the original network stream data to characterize learning may have a significant impact on the detection accuracy of the algorithm.

In the embodiment of the present invention, by comprehensively considering each index through comprehensive experimental result analysis of various types of attacks on a plurality of data sets and statistical results of streams and data packets in the data sets, the preset size may be set as: the number of packets is 6 and the size of each packet intercepted is 484. Such a stream is finally processed into a three-dimensional tensor of 6 × 22, which is input into the CNN, and may be optimized according to the characteristics of the data during actual use.

In the prior art, HAST-NAD firstly proposes to use a Convolutional Neural Network (CNN) to learn the spatial characteristics of network streams, and then uses a cyclic neural network (LSTM) to learn the time sequence characteristics between the network streams, and unlike HAST-NAD, the application does not carry out One-Hot Encoding (One-Hot Encoding). Meanwhile, GRUs are selected instead of LSTMs according to the method, because the cost of the GRUs is lower than that of the LSTMs, the effect is almost the same, and the GRUs are selected to capture the time sequence characteristics of the network flow finally in consideration of the requirement of the network flow detection on efficiency.

In an embodiment of the present invention, referring to fig. 8, a Convolutional Neural Network (CNN) includes three convolutional layers, two pooling layers, and one wire layer, and an activation function among the convolutional layers uses ReLU; inputting network flow into CNN, obtaining different scale characteristic diagrams at each layer of CNN, wherein, the number in front of @ represents channel number, the number behind @ represents characteristic diagram size, and the essence of characteristic diagram is matrix obtained after characteristic extraction. The method includes inputting a three-dimensional tensor into the CNN, specifically, sequentially inputting two-dimensional vectors in the three-dimensional tensor into the CNN, where each two-dimensional vector in the three-dimensional tensor is a packet vector corresponding to each network data packet corresponding to a network flow. The final output of CNN is the packet feature vector corresponding to each packet vector. Referring to fig. 9, a recurrent neural network (GRU) includes two GRU layers and one scatter layer, the input items of the GRU are concatenated packet feature vectors, the output of the GRU network is a flow feature vector, and the flow feature vector is a one-dimensional feature vector.

For ease of explanation, the normal flow feature vector and the abnormal flow feature vector are collectively referred to as a flow feature vector. The second sub-network is essentially a classifier for determining whether the extracted features are abnormal, and the output result of the second sub-network is a floating point number in the interval of [0,1], that is, the first detection score and the second detection score are both floating point numbers in the interval of [0,1 ].

In the embodiment of the present invention, the normal network flow is a network flow to which a normal tag is added, the normal tag is represented by 0, the abnormal network flow is a network flow to which an abnormal tag is added, and the abnormal tag is represented by 1; the first detection score is a detection score corresponding to normal network flow, and the second detection score is a detection score corresponding to abnormal network flow; that is, the first network model obtains a first detection score according to the normal network flow, and the first network model obtains a second detection score according to the abnormal network flow.

In the embodiment of the invention, because the space-time characteristics are obtained by the first sub-network, the network structure of the second sub-network is simpler, and the second sub-network comprises a full-connection layer and a Sigmoid layer.

S12, training the first network model according to the first detection score and the second detection score until a first preset condition is met, so as to obtain a trained first network model.

In an embodiment of the present invention, the first network model includes a first sub-network and a second sub-network, and therefore, in the training, the first sub-network and the second sub-network are trained according to the first detection score and the second detection score to obtain a trained first sub-network, i.e., a source domain feature extractor, and a trained second sub-network, i.e., a classifier. The normal flow feature vectors and the abnormal flow feature vectors are input into the second subnetwork simultaneously in one iteration during the training process, so that the trained second subnetwork (classifier) can be used to distinguish whether the input is a normal network flow or an abnormal network flow, respectively.

The process of training the first network model according to the first detection score and the second detection score is as follows: modifying parameters of a first sub-network and parameters of a second sub-network based on the first detection score and the second detection score. Specifically, a classification loss function value is calculated according to the first detection score and the second detection score, and the parameters of the first sub-network and the parameters of the second sub-network are modified according to the classification loss function value.

In the prior art, abnormal detection is a typical data imbalance problem, that is, abnormal network flow in training data is far less than normal network flow, if special processing is not performed on the unbalanced training data, a neural network is directly used for training, due to strong learning capability of the neural network, normal data flow can be fitted, and due to little learning on abnormal data flow, the trained classifier is difficult to detect the abnormal data flow, so that serious data deviation (Bias) is generated, and extremely low abnormal detection rate is caused.

In this embodiment of the present invention, the first network model further includes a vector generator, and when the abnormal network flow in the source domain is insufficient, random noise is input into the vector generator to obtain the abnormal network flow.

In the embodiment of the present invention, referring to fig. 10, each normal network flow and each abnormal network flow in the source domain are loaded first in the training process, and when the abnormal network flow in the source domain is insufficient, the abnormal network flow generated by using the vector generator is complemented, so that the normal network flow and the abnormal network flow actually input into the second sub-network can maintain a fixed ratio regardless of the ratio of the normal network flow and the abnormal network flow in the source domain. It should be noted that when there are also abnormal network flows in the source domain, the abnormal network flows are not generated by the vector generator. That is, the whole training process is essentially divided into two phases, the first phase extracts normal network flows and abnormal network flows from the source domain, and the second phase extracts normal network flows from the source domain and generates abnormal network flows by using the vector generator. The detection scores output by the second subnetwork in fig. 10 include: a first detection score corresponding to the normal flow feature vector, and a second detection score corresponding to the abnormal flow feature vector.

In the embodiment of the present invention, the network structure of the vector generator is as shown in fig. 11, the vector generator includes 4 deconvolution layers, wherein after each deconvolution layer, normalization is performed using BatchNorm2d, the activation function uses ReLU, and the final output of the vector generator is a vector that is isomorphic with the vector read in the NPZ file, that is, the structure of the normal network flow obtained in step M2 and the structure of the abnormal network flow obtained in step M3 are the same.

In the embodiment of the present invention, the training process of the first network model may be implemented by the following algorithm.

Inputting: normal network flows extracted from a source domain

Abnormal network flow extracted from source domain

The noise Z;

and (3) outputting: the first sub-network:

the second sub-network:

；

starting;

iterate from 1 to N;

respectively loading a batch of normal network flows from training data

Abnormal network flow

；

Through a first sub-network

Extraction of

To obtain

；

If it is not

Is equal to the size of a batch of data;

through the first sub-network

Extraction of

To obtain

；

Otherwise, use

Extraction of

To obtain

Wherein, in the step (A),

is a vector generator;

will be provided with

And

is inputted into

The classification loss was calculated:

；

loss of classification

Back-transfer, simultaneous update

And

the parameters of (1);

loss of the generator is calculated:

；

loss of the generator:

back-transfer, simultaneous update

The parameters of (1);

outputting the trained classifier:

and a source domain feature extractor:

；

and (6) ending.

In the embodiment of the invention, the normal network flow and the abnormal network flow extracted from the source domain are real data, the correct classification priority of the real data can be improved by introducing a hyper-parameter gamma, and the value range of the gamma is (0, 1)]When the gamma is smaller than 1, the composition,

the priority of correctly classifying the real data is improved. It should be noted that, in the present application, the abnormal network flow generated by the vector generator is labeled as 1, and the normal network flow extracted from the source domain is labeled as 0, which is the opposite of the common cross entropy function in GAN, which labels the true sample as 1 and the generated sample as 0 by default. Therefore, the temperature of the molten metal is controlled,

the corresponding classification loss function is shown in equation (1).

（1）

Wherein the content of the first and second substances,

in order to classify the function of the loss,

is a hyper-parameter which is the parameter,

is composed of

Or

，

Based on normal or abnormal network flow extracted from source domain for second sub-network

The score obtained when the input is a normal network flow extracted from the source domain

When the temperature of the water is higher than the set temperature,

for the first detection score, when the input is an abnormal network flow extracted from the source domain

When the temperature of the water is higher than the set temperature,

is a second detection score;

according to a vector generator for a second subnetwork

Generated abnormal network flow

The resulting score.

In an embodiment of the invention, the network parameters of the first sub-network and the second sub-network are modified according to the classification loss function values calculated by the classification loss function,until a first preset condition is met, obtaining a trained first network model, wherein the trained first network model comprises a source domain feature extractor corresponding to a first sub-network

And a classifier corresponding to the second subnetwork

。

In the embodiment of the invention, when the abnormal network flow from the source domain is insufficient in the training data, the abnormal network flow generated by the vector generator is used. In order to enable the vector generator to generate an abnormal network flow around the real data (network flow extracted from the source domain), in an embodiment of the invention, the vector generator is trained.

The goal of the training vector generator is to make

Normal network flow and abnormal network flow can be well distinguished, and the abnormal network flow generated by the vector generator is closely distributed around the normal network flow but not distributed in a different way. Ideally, the situation is

The network flows with the same distribution can be identified as normal network flows, and the network flows with different distributions can be identified as abnormal network flows. If the abnormal network flow is not closely distributed around the normal network flow, the classifier

It can be easily distinguished, for example, that for a first abnormal network flow a and a second abnormal network flow B, if a is distributed near the normal network flow and B is distributed at a location far from the normal network flow, then compared to B,

it is easier to distinguish that a is an abnormal network flow. To enable the vector generator to generate a wrap around trueThe abnormal network flow around the real data (the network flow extracted from the source domain) is trained to the vector generator through the surrounding loss and the dispersion loss, so that the vector generator can generate the abnormal network flow around the real data (the network flow extracted from the source domain).

Specifically, the abnormal network flow generated by the vector generator is input into a first network model, a generation score is obtained through the first network model, and the vector generator is trained according to the generation score to obtain a trained vector generator. The generated score is used for representing the score obtained by the second sub-network according to the abnormal network flow generated by the vector generator.

Specifically, the value of the surround loss is calculated from the generated score, and the value of the surround loss can be calculated by the formula (2)

。

（2）

Wherein the content of the first and second substances,

is a vector generator

The abnormal network flow that is generated is,

is that the second sub-network is based on a vector generator

Generated abnormal network flow

The resulting fraction, i.e. the number of generated components,

in order to be a hyper-parameter,

。

the generated abnormal network flow can be dispersed as far as possible through the dispersion loss, and the distance between the data point of the generated abnormal network flow and the centroid of the abnormal network flow is maximized, so that the data point is encouraged to cover the whole boundary. Calculating a discrete loss value by equation (3)

。

（3）

Wherein the content of the first and second substances,

is the centroid corresponding to the generated abnormal network flow,

is a vector generator

The generated abnormal network flow.

The loss function corresponding to the vector generator can be described by formula (4) by comprehensively considering the surround loss and the dispersion loss.

（4）

Wherein the content of the first and second substances,

is the corresponding loss function value of the vector generator,

is a hyper-parameter used to adjust the weights of the surround and dispersion losses.

In the embodiment of the invention, in the training process, network parameters of the first sub-network and the second sub-network are modified through classifying the loss function values, and meanwhile, the network parameters of the vector generator are modified through the loss function values corresponding to the vector generator until a first preset condition is met, so that the source domain feature extractor, the classifier and the trained vector generator are obtained.

In the embodiment of the present invention, the first preset condition includes that the classification loss function value meets a preset requirement, or the training times reach a preset number. The preset requirement may be determined according to the accuracy of the classifier, which is not described in detail herein, and the preset number may be a maximum number of training times of the second sub-network, for example, 4000 times, etc. Therefore, after the classification loss function value is calculated, whether the classification loss function value meets a preset requirement is judged, and if the classification loss function value meets the preset requirement, the training is ended; if the classification loss function value does not meet the preset requirement, judging whether the training times of the second sub-network reach the prediction times, if not, modifying the network parameters of the first sub-network and the second sub-network through the classification loss function value, and simultaneously modifying the network parameters of the vector generator through the loss function value corresponding to the vector generator; and if the preset times are reached, ending the training. Therefore, whether training is finished or not is judged through the classification loss function value and the training times, and the phenomenon that the training enters a dead cycle because the classification loss function value cannot meet the preset requirement can be avoided.

S2, training the second network model based on the target domain, the source domain feature extractor and the discriminator to obtain the target domain feature extractor.

In the embodiment of the present invention, the source domain and the target domain are both network flows in nature, the network flow in the source domain is a labeled network flow, and the network flow in the target domain is a network flow without a label; in the existing anomaly detection method, model training and model detection are performed on the same data set, and only a model trained on a certain data set can be described, so that the method is effective for detecting the data set. In a new scene, the model needs to be adjusted, and the adjustment of the model depends on a large amount of marked data, so that the method is not suitable for an environment with less data and no label. In the embodiment of the invention, the data in the target domain for training has no label.

In the embodiment of the invention, a classifier is trained through training data extracted from a source domain, and the classifier is migrated into a target domain to perform anomaly detection on the target domain; that is, the migration of the domain is done through the mapping of latent features, which in turn is done through the target domain feature extractor, which is optimized through the process of competitive training. In step S2, the features extracted in the target domain by the target domain feature extractor are made similar to the features extracted in the source domain by the source domain feature extractor through the antagonistic training.

Referring to fig. 12, the process of training the second network model includes: source domain feature extractor

Extracting source domain feature vectors from a source domain, extracting target domain feature vectors from a target domain by the second network model, and inputting the source domain feature vectors and the target domain feature vectors into a discriminator

Passing through discriminator

Outputting a prediction score, the prediction score comprising: a first prediction score corresponding to the source domain feature vector and a second prediction score corresponding to the target domain feature vector, and then model parameters of a second network model are modified according to the first prediction score and the second prediction score to obtain a target domain feature extractor

。

In the embodiment of the invention, the source domain feature extractor

Is obtained by training the first subnetwork through step S1.At the beginning of training, the initial model parameters of the second network model are the same as the model parameters of the source domain feature extractor, and the structure of the second network model is the same as that of the source domain feature extractor. The initial model parameters of the second network model are model parameters of the second network model when the second network model is not trained, that is, the source domain feature extractor is adopted

The model parameters of the first network model are used for carrying out parameter initialization on the second network model, and in the training process, the source domain feature extractor is used for extracting the source domain features

Only the model parameters of the second network model are updated. The first prediction score is used for representing a source domain feature score corresponding to a source domain feature vector output by the discriminator, and the second prediction score is used for representing a target domain feature score corresponding to a target domain feature vector output by the discriminator.

Specifically, step S2 includes:

and S21, the source domain feature extractor extracts the source domain feature vector corresponding to the source domain.

In the embodiment of the invention, the source domain feature extractor

The process of extracting the source domain feature vector is identical to the step of extracting the normal flow feature vector by the first sub-network (identical to the step of extracting the abnormal flow feature vector by the first sub-network). Specifically, a network flow is obtained from a source domain, a three-dimensional tensor with a preset size is extracted from the obtained network flow, and a source domain feature extractor

And outputting the source domain feature vector according to the extracted three-dimensional tensor.

And S22, extracting the target domain feature vector corresponding to the target domain by the second network model.

In the embodiment of the present invention, the source domain feature extractor includes a CNN and a GRU, and similarly, the second network model also includes a CNN and a GRU, specifically, the CNN includes three convolution layers, two pooling layers and a linear layer, the GRU includes two GRU layers and a Flatten layer, the network structure of the CNN is shown in fig. 8, the network structure of the GRU is shown in fig. 9, and the CNN and the GRU are cascaded to obtain the second network model.

In the embodiment of the invention, the second network model and the source domain feature extractor

The structure is the same, and similarly, a network flow is obtained from the target domain, a three-dimensional tensor with a preset size is extracted from the network flow corresponding to the target domain, and the second network model outputs a target domain feature vector according to the three-dimensional tensor from the target domain.

S23, inputting the source domain feature vector and the target domain feature vector into the discriminator to generate a first prediction score corresponding to the source domain feature vector and a second prediction score corresponding to the target domain feature vector.

In the embodiment of the invention, the discriminator

Is to distinguish features from the source domain and the target domain, i.e. a discriminator

The goal of (1) is to distinguish between source domain feature vectors and target domain feature vectors. The source domain feature vector is labeled 1 and the target domain feature vector is labeled 0. The first prediction score is used for representing a source domain feature score corresponding to a source domain feature vector output by the discriminator, the second prediction score is used for representing a target domain feature score corresponding to a target domain feature vector output by the discriminator, and the discriminator

It is possible to distinguish whether the input features come from the target domain or the source domain.

S24, training the second network model based on the first prediction score and the second prediction score until a second preset condition is met, so as to obtain the target domain feature extractor.

In the embodiment of the invention, a target domain loss function value corresponding to the second network model is calculated through the first prediction score and the second prediction score, and the parameters of the second network model are adjusted according to the target domain loss function value until a second preset condition is met, so that the target domain feature extractor is obtained

。

In the embodiment of the present invention, the target domain loss function value may be calculated by equation (5).

（5）

Wherein the content of the first and second substances,

is the extracted three-dimensional tensor in the source domain,

is the extracted three-dimensional tensor in the target domain,

is the feature vector of the target domain and,

is the second detection score, i.e. the target domain feature score corresponding to the target domain feature vector, and D is the discriminator.

In the embodiment of the invention, during the process of training the second network model, the discriminator

The model parameters of (2) are also updated. Specifically, the discriminator is calculated by a first prediction score and a second prediction score

Adjusting parameters of the second network model according to the discriminant loss function value until a second preset condition is met to obtain a target domain feature extractor

。

In the embodiment of the invention, the discriminator

The corresponding discriminant loss function is shown in equation (6).

（6）

Wherein the content of the first and second substances,

is the extracted three-dimensional tensor in the source domain,

is a feature vector of the source domain and,

is a first detection score, namely a source domain feature score corresponding to the source domain feature vector,

is the extracted three-dimensional tensor in the target domain,

is the feature vector of the target domain and,

is the second detection score, i.e. the target domain feature score corresponding to the target domain feature vector.

In the embodiment of the invention, the input is

The data in (1) is three-dimensional tensor extracted from a source domain (including a first three-dimensional tensor corresponding to normal network flow and a second three-dimensional tensor corresponding to abnormal network flow), the input data of a second network model is an unlabeled three-dimensional tensor from a target domain, and then target domain feature vectors output by the second network model and source domain feature vectors output by a source domain feature extractor are input into a discriminator

. Having the second network model attempt to extract sums from the target domain by antagonistic training

Extracting similar features from the source domain to spoof the discriminator, subject to a countermeasure optimization, such that

And

all approach to 0.5, i.e. as the discriminator

When the extracted feature is from the source domain or the target domain, the training process is completed.

And S3, generating a network flow abnormity detection model according to the target domain feature extractor and the classifier.

In the embodiment of the present invention, referring to fig. 13, the network flow anomaly detection model includes: a target domain feature extractor and a classifier, wherein the classifier is obtained through the training of the step S1, and the target domain feature extractor is obtained through the training of the step S2.

Referring to fig. 14, in a specific implementation, a method for generating a network flow anomaly detection model may be divided into three stages. A first stage of training a classifier based on the source domain

And source field specificSign extractor

When the abnormal network flow from the source domain is insufficient, the abnormal network flow is generated through the Gaussian noise and vector generator such that the normal network flow and the abnormal network flow input to the source domain feature extractor are balanced, preventing the classifier from balancing the normal network flow and the abnormal network flow, in consideration of the number imbalance of the normal network flow and the abnormal network flow from the source domain

The occurrence of a normal sample data Bias (Bias) results in an extremely low anomaly detection rate. In the second stage, a method of the antagonistic domain adaptation is used for training a target domain feature extractor corresponding to a target domain

And mapping the data on the target domain to the feature space similar to the source domain to minimize the spatial distance between the feature space of the target domain and the features of the source domain, so that the features extracted by the target domain feature extractor on the target domain are similar to the features extracted by the source domain feature extractor on the source domain, thereby completing the adaptation process from the source domain to the target domain. A third stage of training the classifier of the first stage

And the second-stage trained target domain feature extractor

And finally, realizing a network flow abnormity detection model capable of carrying out abnormity detection on the target domain.

In the embodiment of the invention, a first network model is trained based on a source domain to obtain a trained first network model, wherein the trained first network model comprises a source domain feature extractor and a classifier; training a second network model based on a target domain, the source domain feature extractor and a discriminator to obtain a target domain feature extractor; and generating a network flow abnormity detection model according to the target domain feature extractor and the classifier. The data in the target domain has no label, a second network model is trained in a countermeasure generating mode to obtain a target domain feature extractor, so that the target domain feature extractor can map the data in the target domain to a feature space similar to that in the source domain, the spatial distance between the feature space in the target domain and the feature in the source domain is minimized, the features extracted by the target domain feature extractor on the target domain are similar to the features extracted by the source domain feature extractor on the source domain, and the adaptation process from the source domain to the target domain is completed; furthermore, when the classifier obtained by the source domain training is used in a new scene, the new scene does not need to have a large amount of labeled data for secondary training. The classifier obtained based on source domain training in the network flow anomaly detection model can be used for carrying out anomaly detection on the target domain, and the accuracy is high.

Based on the above generation method of the network flow anomaly detection model, the present invention further provides a network flow anomaly detection method, where the network flow anomaly detection method applies the network flow anomaly detection model obtained by the generation method of the network flow anomaly detection model according to the above embodiment, and the network flow anomaly detection model includes a target domain feature extractor and a classifier, as shown in fig. 15, the network flow anomaly detection method includes:

k1, the network flow abnormity detection model obtains the network flow to be detected in the target domain.

K2, extracting the feature vector to be detected corresponding to the network flow to be detected by the target domain feature extractor, wherein the target domain feature extractor is the target domain feature extractor in the above network flow anomaly detection method;

and K3, classifying the feature vector to be detected by the classifier to obtain an anomaly detection result corresponding to the feature vector to be detected, wherein the classifier is the classifier in the anomaly detection method for the network flow.

In the embodiment of the present invention, first, a target domain is preprocessed to obtain a network flow to be detected, a process of preprocessing the target domain to obtain the network flow to be detected is the same as a process of obtaining a normal network flow and an abnormal network flow based on a source domain in steps M1 to M3, and further, a specific process of preprocessing the target domain to obtain the network flow to be detected may refer to the description in steps M1 to M3.

In the embodiment of the present invention, in specific implementation, referring to fig. 16, the spatio-temporal features of the network flow to be detected are extracted by the target domain feature extractor to obtain a feature vector to be detected, the feature vector to be detected is input to the classifier, the output of the classifier is a floating point type number of [0,1], and the floating point type number can obtain a tag for representing an anomaly detection result through a binary function. Through the binary function, the label corresponding to the floating point type number smaller than or equal to 0.5 is 0, the label corresponding to the floating point type number larger than 0.5 is 1, the label of 0 indicates that the anomaly detection result of the network flow to be detected is normal, and the label of 1 indicates that the anomaly detection result of the network flow to be detected is abnormal.

In the embodiment of the invention, the features extracted by the target domain feature extractor on the target domain are similar to the features extracted by the source domain feature extractor on the source domain, so that the classifier trained by the source domain can be used for carrying out anomaly detection on the target domain, and the accuracy is high.

In one embodiment, the present invention provides a computer device, which may be a terminal, having an internal structure as shown in fig. 17. The computer device comprises a processor, a memory, a network model interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network model interface of the computer device is used for communicating with an external terminal through network model connection. The computer program is executed by a processor to implement a method of generating a network flow anomaly detection model, or a method of anomaly detection for a network flow. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the illustration in fig. 17 is merely a block diagram of a portion of the structure associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

The embodiment of the invention provides computer equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the computer equipment is characterized in that the processor executes the computer program and realizes the following steps:

the target domain feature extractor extracts a to-be-detected feature vector corresponding to the to-be-detected network flow, wherein the target domain feature extractor is a target domain feature extractor in the above network flow anomaly detection method;

the classifier classifies the feature vector to be detected to obtain an anomaly detection result corresponding to the feature vector to be detected, wherein the classifier is the classifier in the anomaly detection method for the network flow.

An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the following steps:

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method for generating a network flow anomaly detection model is characterized by comprising the following steps:

the training of the first network model based on the source domain to obtain the trained first network model specifically includes:

determining each abnormal network flow and each normal network flow based on the source domain;

inputting normal network flows in training data and abnormal network flows in the training data into the first network model, and generating a first detection score corresponding to the normal network flows and a second detection score corresponding to the abnormal network flows through the first network model, wherein the training data comprises a plurality of training groups, and each training group comprises normal network flows from a source domain and abnormal network flows from the source domain;

training the first network model according to the first detection score and the second detection score until a first preset condition is met to obtain a trained first network model;

the first network model comprises a vector generator;

when the abnormal network flow in the source domain is insufficient, the determining the abnormal network flow and the normal network flow based on the source domain further includes:

and inputting random noise into the vector generator to obtain abnormal network flow.

2. The method for generating the network flow anomaly detection model according to claim 1, wherein the determining each anomaly network flow and each normal network flow based on the source domain specifically includes:

extracting each first network flow and each second network flow in the source domain;

generating first three-dimensional tensors with preset sizes according to the first network flows, and taking the first three-dimensional tensors as the normal network flows;

and generating second three-dimensional tensors of the preset size according to the second network flows, and taking the second three-dimensional tensors as the abnormal network flows.

3. The method for generating the network flow abnormality detection model according to claim 2, wherein the generating, according to the first network flows, first three-dimensional tensors of a preset size, and taking the first three-dimensional tensors as the normal network flows specifically includes:

for each first network flow, extracting each first network data packet corresponding to the first network flow;

and obtaining first three-dimensional tensors with preset sizes corresponding to the first network flows according to the first network data packets, and taking the first three-dimensional tensors as the normal network flows.

4. The method according to claim 3, wherein obtaining, according to each first network data packet, each first three-dimensional tensor with a preset size corresponding to the first network flow, and using each first three-dimensional tensor as the normal network flow specifically includes:

carrying out serialization processing on each first network data packet to obtain first character strings respectively corresponding to the first network data packets;

and generating first three-dimensional tensors with preset sizes according to the first character strings, and taking the first three-dimensional tensors as the normal network streams.

5. The method for generating the network flow abnormality detection model according to claim 2, wherein the generating of the second three-dimensional tensors of the preset size according to the second network flows and taking the second three-dimensional tensors as the abnormal network flows specifically includes:

for each second network flow, extracting each second network data packet corresponding to the second network flow;

and obtaining a second three-dimensional tensor of a preset size corresponding to the second network flow according to each second network data packet, and taking each second three-dimensional tensor as each abnormal network flow.

6. The method according to claim 5, wherein the obtaining, according to each second network data packet, a second three-dimensional tensor of a preset size corresponding to the second network flow, and using each second three-dimensional tensor as each abnormal network flow specifically includes:

carrying out serialization processing on each second network data packet to obtain second character strings respectively corresponding to each second network data packet;

and generating second three-dimensional tensors of the preset size according to the second character strings, and taking the second three-dimensional tensors as the abnormal network streams.

7. The method according to claim 1, wherein the training a second network model based on a target domain, the source domain feature extractor, and a discriminator to obtain a target domain feature extractor specifically includes:

the source domain feature extractor extracts a source domain feature vector corresponding to the source domain;

extracting a target domain feature vector corresponding to a target domain by the second network model;

inputting the source domain feature vector and the target domain feature vector into the discriminator to generate a first prediction score corresponding to the source domain feature vector and a second prediction score corresponding to the target domain feature vector;

and training the second network model based on the first prediction score and the second prediction score until a second preset condition is met to obtain a target domain feature extractor.

8. The method according to claim 7, wherein initial model parameters of the second network model are the same as model parameters of the source domain feature extractor, and the structure of the second network model is the same as that of the source domain feature extractor, and the initial model parameters of the second network model are model parameters of the second network model when the second network model is not trained.

9. The method for detecting the network flow abnormity is applied to a network flow abnormity detection model, wherein the network flow abnormity detection model comprises a target domain feature extractor and a classifier, and specifically comprises the following steps:

the target domain feature extractor extracts a to-be-detected feature vector corresponding to the to-be-detected network flow, wherein the target domain feature extractor is the target domain feature extractor of any one of claims 1 to 8;

the classifier classifies the feature vector to be detected to obtain an anomaly detection result corresponding to the feature vector to be detected, wherein the classifier is the classifier according to any one of claims 1 to 8.

10. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps in the method for generating a network flow anomaly detection model according to any one of claims 1 to 8 or the method for detecting an anomaly of a network flow according to claim 9 when executing the computer program.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps in the method for generating the network flow abnormality detection model according to any one of claims 1 to 8 or the method for detecting the abnormality of the network flow according to claim 9.