CN113055384A - SSDDQN network abnormal flow detection method - Google Patents

SSDDQN network abnormal flow detection method Download PDF

Info

Publication number
CN113055384A
CN113055384A CN202110271456.1A CN202110271456A CN113055384A CN 113055384 A CN113055384 A CN 113055384A CN 202110271456 A CN202110271456 A CN 202110271456A CN 113055384 A CN113055384 A CN 113055384A
Authority
CN
China
Prior art keywords
network
neural network
current
function value
flow
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110271456.1A
Other languages
Chinese (zh)
Inventor
董仕
夏元俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhoukou Normal University
Original Assignee
Zhoukou Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhoukou Normal University filed Critical Zhoukou Normal University
Priority to CN202110271456.1A priority Critical patent/CN113055384A/en
Publication of CN113055384A publication Critical patent/CN113055384A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425—Traffic logging, e.g. anomaly detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/147—Network analysis or design for predicting network behaviour

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Security & Cryptography (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Computer Hardware Design (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a semi-supervised Double Deep Q-Network (SSDDQN) Network abnormal flow detection method based on Deep reinforcement learning Double Deep Q-Network, and relates to the technical field of computer Network safety. The method comprises the following steps: the method comprises the steps of obtaining a training sample of computer network flow data, establishing a neural network, training the neural network by using the training sample in an SSDDQN mode, updating parameters of the neural network, and finally performing anomaly detection on the network flow data by using the trained neural network. The SSDDQN mode of the invention can not only reduce the manual marking cost and improve the learning performance, but also ensure that the neural network is simpler and faster and is easier to be deployed in a harsh network environment, and simultaneously improve the detection accuracy of unknown attacks.

Description

SSDDQN network abnormal flow detection method
Technical Field
The invention relates to the technical field of computer Network safety, in particular to a method for detecting Network abnormal flow of a semi-supervised Double Deep Q-Network (SSDDQN).
Background
Nowadays, a great number of abnormal behaviors causing malicious consequences such as network failures, abuses, attacks and the like exist in the internet, and the behaviors are often reflected in network traffic, wherein abnormal situations such as worms, port views, denial of service attacks, distributed denial of service attacks and the like are particularly common. These anomalies tend to waste network resources, resulting in degraded performance of network devices and end hosts, and even security concerns for a large number of network users.
At present, most of network traffic intrusion detection is mainly based on misuse detection or a machine learning algorithm: the misuse detection is to distinguish the abnormal behavior from the behavior under the normal condition according to the known characteristics to realize the detection of the intrusion behavior according to the known characteristics, but the detection effect of the unknown attack type is very poor and the false alarm rate is very high, and the maintenance of the characteristics is mostly completed by adopting a manual mode; the traditional machine learning detection algorithm depends on manual extraction of flow characteristics and labeling, the manual intervention is serious, and the classification effect depends on the quality of the manually extracted characteristics to a great extent. Both of the above rely on labor, which is extremely costly.
Deep learning is used as a high-level branch of machine learning, complex data can be processed, data characteristics can be automatically learned only through training, but due to a complex network, prediction cannot be quickly trained, and therefore the deep learning cannot be deployed in a harsh environment with real-time response.
Disclosure of Invention
The method for detecting the network abnormal flow of the SSDDQN can solve the problems in the prior art.
The invention provides a method for detecting network abnormal flow of SSDDQN, which comprises the following steps:
step 1, obtaining sample data from computer network flow data, wherein the sample data comprises training samples;
step 2, establishing a neural network, and enabling the flow characteristics s in the current flow data in the training sampletInputting the label A of all flow characteristics into a neural network, predicting all Q function values under the current flow characteristics according to each label, obtaining the maximum Q function value through a greedy strategy algorithm, and obtaining the maximum Q function value through the maximum Q functionNumerical value is obtained to obtain predicted flow label a 'under current flow characteristic't；
Step 3, labeling a 'with the predicted flow'tWith the true label a in the training sample* tComparing, if consistent, obtaining the reward rtThe reward value is 1; if not, award rtIs 0;
step 4, receiving the flow characteristic s of the next stage in the training samplet+1Predicting the corresponding label A' by an unsupervised learning algorithm, calculating all Q function values by a target network, obtaining the maximum Q function value, and replaying the reward r in the set according to the Q function value and experiencetCalculating a target Q function value;
step 5, passing the current flow characteristic stAnd predicted traffic tag a'tCalculating a current Q function value when the current neural network is trained, obtaining a loss function through the current Q function value and a target Q function value, updating parameters of the current neural network through a back propagation algorithm, and periodically copying network parameters to the target network to obtain the trained neural network;
and 6, inputting the flow data to be detected into the trained neural network for abnormal flow detection.
Preferably, step 3 is carried out to obtain st、rt、a't、st+1It is then placed in the empirical playback set, and step 4, when used, is randomly taken from the empirical playback set.
Preferably, step 2 specifically comprises:
the flow characteristics s in the current flow data in the training sample are measuredtAnd label a ═ of all traffic characteristics0,a1,…,at,…,an) Inputting into a neural network, and calculating all Q function values:
Q(s,a)＝E[Rt|st＝s,at＝a]
wherein, E represents the value of the expected value,
Figure BDA0002974671020000031
in return for time t, γ isAttenuation factor, avoiding R in continuous taskstF, changing the value of the time step to be infinity, wherein T is a time step, and n is the number of tags;
predicting all Q function values under the current flow characteristics according to each label:
Q(st,a)＝[Q(st,a0),Q(st,a1),...,Q(st,at),...,Q(st,an)]
and (3) solving a maximum Q function value through a greedy strategy algorithm:
Policy(st)＝argamax(Q(st,a))
obtaining a predicted flow label a 'under the current flow characteristic through the maximum Q function value't。
Preferably, step 4 specifically includes:
receiving the flow characteristic s of the next stage in the training samplet+1Predicting the label A 'by an unsupervised learning algorithm, and using the predicted label A' and the traffic characteristics st+1All Q function values are calculated and the maximum value is found:
Q'st+1＝maxQ'(st+1A')
the reward r in the experience playback settCalculating a target Q function value:
Q*＝rt+γ·Q'st+1。
preferably, step 5 specifically includes:
by current flow characteristic stAnd predicted traffic tag a'tCalculating the current Q function value during the current neural network training:
Q'st＝Q'(st,a't)
and solving a loss function through the current Q function value and the target Q function value:
L＝1/n·∑n(Q'st-Q*)2
and finally, updating the parameters of the current neural network through a back propagation algorithm, and periodically copying the network parameters to the target network.
Preferably, step 1 also applies to training samplesForming a new training sample after sampling, and 2, obtaining the flow characteristic s in the current flow data in the new training sampletAnd all the flow characteristics label A input neural network to calculate all the Q function values.
Preferably, the sample data further includes a test sample, and when the trained neural network is obtained in step 5, the test sample is input into the neural network to test the performance of the neural network.
The method for detecting the network abnormal flow of the SSDDQN has the following advantages that:
1. and (5) manually marking the cost. At present, the scale of marking data is far from keeping up with the application requirement, and the manual marking cost is extremely high. With the labeled data, the algorithm can be trained on the basis, and the higher the quality of data labeling is, the more accurate the learning result is. And the CNN, RNN and DBN which are commonly used at present all adopt a supervised learning mode, a large amount of marking cost is needed, and the manual marking cost can be reduced by adopting a semi-supervised learning mode, so that the learning performance is improved.
2. Is easier to implement. Most of the current flow anomaly detection uses an open data set, namely, off-line detection. But with the expansion of network data in these years, the kinds and the number of attack traffic are increased, even unknown variants are caused, and therefore, the detection of the traffic is more difficult, and the importance of real-time detection is highlighted. Because the neural networks with functions of deep reinforcement learning strategies, values or Q functions and the like for classification are simpler and faster, the neural networks are easier to deploy in a harsh network environment. Furthermore, the reward function for detection is very flexible and does not need to be differentiated.
3. And (4) unknown attack detection. Most of the current simulation works, training the model by using a public and famous data set, and only detecting existing attacks but not detecting unknown attacks. And adopting semi-supervised deep reinforcement learning, adopting unsupervised learning algorithm clustering, predicting the characteristic label of the target network, and generating a Q function value by the target network so as to improve the accuracy of unknown attack detection.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a network abnormal traffic detection method of SSDDQN in the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the present invention provides a network abnormal traffic detection method of SSDDQN, which includes the following steps:
step 1, obtaining sample data from computer network flow data, dividing the sample data into training samples and testing samples, and forming new training samples after sampling the training samples in small batches.
Step 2, establishing a deep neural network, wherein the deep neural network has three hidden layers, the number of neurons of each hidden layer is 100, and the receiving state in the sampled new training sample, namely the flow characteristic s in the current flow datatAnd actions, i.e. label a ═ a for all traffic characteristics0,a1,…,at,…,an) Input to the neural network to calculate all Q function values:
Q(s,a)＝E[Rt|st＝s,at＝a]
wherein, E represents the value of the expected value,
Figure BDA0002974671020000051
the time t is used as a return, gamma is used as an attenuation factor, and R is avoided during the continuous tasktAnd f, T is a time step, and n is the number of tags.
Predicting all Q function values under the current flow characteristics according to each label:
Q(st,a)＝[Q(st,a0),Q(st,a1),...,Q(st,at),...,Q(st,an)]
and (3) solving a maximum Q function value through a greedy strategy algorithm:
Policy(st)＝argamax(Q(st,a))
the greedy strategy algorithm is to gradually approach a given target from a certain initial solution of the problem so as to obtain a better solution as fast as possible. The algorithm stops when a certain step in the algorithm is reached and can no longer proceed. That is, the greedy policy algorithm does not consider the overall optimum, but rather makes the best choice currently viewed.
Obtaining a predicted flow label a 'under the current flow characteristic through the maximum Q function value't。
Step 3, labeling a 'with the predicted flow'tWith real label a in training sample sampled in small batch* tComparing, if consistent, obtaining the reward rtThe reward value is 1; if not, award rtThe value of (d) is 0. Will predict traffic label a'tAnd a prize rtPutting the data into an empirical playback set, wherein the empirical playback set is used for solving the problems of correlation and non-static distribution among data, and the predicted traffic label a 'obtained at the current moment is used'tPrize rtWhen the storage is needed in the next stage, the storage is randomly taken out.
Step 4, receiving the flow characteristic s of the next stage in the training samplet+1Predicting the label A 'by an unsupervised learning algorithm, and using the predicted label A' and the traffic characteristics st+1All Q function values are calculated and the maximum value is found:
Q'st+1＝maxQ'(st+1A')
the reward r in the experience playback settCalculating a target Q function value:
Q*＝rt+γ·Q'st+1
wherein the unsupervised learning algorithm comprises: k-means, hierarchical clustering, Gaussian mixture model GMM and the like.
Taking K _ means as an example, the calculation method of the target Q function value is as follows:
(1) at St+1In the method, k samples are randomly selected
Figure BDA0002974671020000061
(2) Initialize l cluster centers { x1,x2,...,xl}；
(3) Calculating Euclidean distance from each object to each cluster center
Figure BDA0002974671020000062
(4) Determining x from the nearest mean vectorlCluster marking of (2): lambda [ alpha ]l＝argminldlk；
(5) Sample xlDividing into corresponding clusters: cλl＝Cλl∪{xl}；
(6) The label a' of the last cluster is output.
Step 5, passing the current flow characteristic stAnd predicted traffic tag a'tTo calculate the current Q function value when the current neural network is trained:
Q'st＝Q'(st,a't)
the above-mentioned method for calculating the current Q function value predicted during the training of the current neural network is the same as the calculation direction of the target Q function value, and a description thereof is not repeated.
And solving a loss function through the current Q function value and the target Q function value:
L＝1/n·∑n(Q'st-Q*)2
finally, updating parameters of the current neural network through a back propagation algorithm, and periodically copying the network parameters to a target network to obtain the trained neural network, wherein the parameters comprise weight values
Figure BDA0002974671020000071
Offset value
Figure BDA0002974671020000072
And so on, where ρ is the learning rate.
And 6, performing performance test on the trained neural network by using the test sample.
And 7, inputting the flow data to be detected into a neural network for abnormal flow detection.
To verify the effectiveness of the method of the invention, experiments were carried out using the public and well-known data set NSL-KDD. The experimental platform is an associative desktop computer, the system is Windows10, the processor is Intel (R) core (TM) i7-8700 CPU @3.20GHz, and the RAM is 16 GB. Because 23 feature labels are contained in the training sample of the NSL-KDD data set and 38 feature labels are contained in the testing sample, the performance that the intrusion detection framework reduces the labor cost and improves the detection accuracy rate can be embodied due to the unbalanced distribution of the data set labels. The experiment mainly comprises the following steps:
1. and (6) sampling data. The original flow data is sampled and divided into a training sample and a testing sample, and the training sample and the testing sample mainly comprise a current state (current flow characteristic), an action (real label) and a next-stage state (next-stage flow characteristic).
2. All Q-function values and rewards are calculated. Calculating all Q function values in the current neural network by inputting current flow characteristics and all flow characteristic labels, obtaining a predicted flow label when the Q function value is maximum through a greedy strategy algorithm, comparing the predicted flow label with a real label, and if the predicted flow label is consistent with the real label, rewarding is 1, and if the predicted flow label is inconsistent with the real label, rewarding is 0. And finally, calculating the current Q function value during the current neural network training through the predicted flow label and the current flow characteristic.
3. And calculating a final Q function value. By inputting in an unsupervised learning algorithmOne-stage traffic features direct prediction of their labels A ', using the predicted labels A' and traffic features st+1All the Q function values are calculated, and the maximum value is the final Q function value.
4. A loss function is calculated. And obtaining a target Q function value by multiplying the final Q function value by the attenuation factor and adding the obtained reward, calculating a loss function by the target Q function value and the current Q function value during the current neural network training so as to update the neural network parameters, and storing the training model.
5. The neural network is evaluated by evaluation criteria. Inputting a test sample, loading a local neural network model, and calculating an evaluation result through a series of evaluation criteria.
The analysis experiment result shows that compared with the traditional machine learning, the accuracy of the deep reinforcement learning is improved by about 7 percent and the F1-score value is improved by about 8 percent compared with the common deep reinforcement learning.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (7)

1. A network abnormal flow detection method of SSDDQN is characterized by comprising the following steps:
step 1, obtaining sample data from computer network flow data, wherein the sample data comprises training samples;
step 2, establishing a neural network, and enabling the flow characteristics s in the current flow data in the training sampletAnd all the labels A of the traffic characteristics are input into the neural network, and prediction is carried out according to each labelObtaining the maximum Q function value through a greedy strategy algorithm according to all Q function values under the current flow characteristic, and obtaining a predicted flow label a 'under the current flow characteristic according to the maximum Q function value't；
Step 3, labeling a 'with the predicted flow'tWith the true label a in the training sample* tComparing, if consistent, obtaining the reward rtThe reward value is 1; if not, award rtIs 0;
step 4, receiving the flow characteristic s of the next stage in the training samplet+1Predicting the corresponding label A' by an unsupervised learning algorithm, calculating all Q function values by a target network, obtaining the maximum Q function value, and replaying the reward r in the set according to the Q function value and experiencetCalculating a target Q function value;
step 5, passing the current flow characteristic stAnd predicted traffic tag a'tCalculating a current Q function value when the current neural network is trained, obtaining a loss function through the current Q function value and a target Q function value, updating parameters of the current neural network through a back propagation algorithm, and periodically copying network parameters to the target network to obtain the trained neural network;
and 6, inputting the flow data to be detected into the trained neural network for abnormal flow detection.
2. The method for detecting abnormal network traffic of SSDDQN as claimed in claim 1, wherein step 3 is obtaining st、rt、a't、st+1It is then placed in the empirical playback set, and step 4, when used, is randomly taken from the empirical playback set.
3. The method for detecting network abnormal traffic of SSDDQN according to claim 1, wherein step 2 specifically comprises:
the flow characteristics s in the current flow data in the training sample are measuredtAnd label a ═ of all traffic characteristics0,a1,…,at,…,an) Transfusion systemEntering the neural network, calculating all Q function values:
Q(s,a)＝E[Rt|st＝s,at＝a]
wherein, E represents the value of the expected value,
Figure FDA0002974671010000021
the time t is used as a return, gamma is used as an attenuation factor, and R is avoided during the continuous tasktF, changing the value of the time step to be infinity, wherein T is a time step, and n is the number of tags;
predicting all Q function values under the current flow characteristics according to each label:
Q(st,a)＝[Q(st,a0),Q(st,a1),...,Q(st,at),...,Q(st,an)]
and (3) solving a maximum Q function value through a greedy strategy algorithm:
Policy(st)＝argamax(Q(st,a))
obtaining a predicted flow label a 'under the current flow characteristic through the maximum Q function value't。
4. The method for detecting network abnormal traffic of SSDDQN according to claim 1, wherein step 4 specifically comprises:
receiving the flow characteristic s of the next stage in the training samplet+1Predicting the label A 'by an unsupervised learning algorithm, and using the predicted label A' and the traffic characteristics st+1All Q function values are calculated and the maximum value is found:
Q'st+1＝maxQ'(st+1A')
the reward r in the experience playback settCalculating a target Q function value:
Q*＝rt+γ·Q'st+1。
5. the method for detecting network abnormal traffic of SSDDQN as claimed in claim 4, wherein step 5 specifically comprises:
by current flow characteristic stAnd predicted traffic tag a'tCalculating the current Q function value during the current neural network training:
Q'st＝Q'(st,a't)
and solving a loss function through the current Q function value and the target Q function value:
L＝1/n·∑n(Q'st-Q*)2
and finally, updating the parameters of the current neural network through a back propagation algorithm, and periodically copying the network parameters to the target network.
6. The method according to claim 1, wherein step 1 further samples the training samples to form new training samples, and step 2 combines the traffic features s in the current traffic data in the new training samplestAnd all the flow characteristics label A input neural network to calculate all the Q function values.
7. The method according to claim 1, wherein the sample data further includes a test sample, and when the trained neural network is obtained in step 5, the test sample is input into the neural network to test the performance of the neural network.
CN202110271456.1A 2021-03-12 2021-03-12 SSDDQN network abnormal flow detection method Pending CN113055384A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110271456.1A CN113055384A (en) 2021-03-12 2021-03-12 SSDDQN network abnormal flow detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110271456.1A CN113055384A (en) 2021-03-12 2021-03-12 SSDDQN network abnormal flow detection method

Publications (1)

Publication Number Publication Date
CN113055384A true CN113055384A (en) 2021-06-29

Family

ID=76512392

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110271456.1A Pending CN113055384A (en) 2021-03-12 2021-03-12 SSDDQN network abnormal flow detection method

Country Status (1)

Country Link
CN (1) CN113055384A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113537383A (en) * 2021-07-29 2021-10-22 周口师范学院 Method for detecting abnormal flow of wireless network based on deep migration reinforcement learning
CN114374541A (en) * 2021-12-16 2022-04-19 四川大学 Abnormal network flow detector generation method based on reinforcement learning

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110673488A (en) * 2019-10-21 2020-01-10 南京航空航天大学 Double DQN unmanned aerial vehicle concealed access method based on priority random sampling strategy
CN110958135A (en) * 2019-11-05 2020-04-03 东华大学 Method and system for eliminating DDoS (distributed denial of service) attack in feature self-adaptive reinforcement learning
CN111741002A (en) * 2020-06-23 2020-10-02 广东工业大学 Method and device for training network intrusion detection model
CN111800414A (en) * 2020-07-03 2020-10-20 西北工业大学 Convolutional neural network-based traffic anomaly detection method and system
CN112364980A (en) * 2020-11-09 2021-02-12 北京计算机技术及应用研究所 Deep neural network training method based on reinforcement learning under weak supervision scene

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110673488A (en) * 2019-10-21 2020-01-10 南京航空航天大学 Double DQN unmanned aerial vehicle concealed access method based on priority random sampling strategy
CN110958135A (en) * 2019-11-05 2020-04-03 东华大学 Method and system for eliminating DDoS (distributed denial of service) attack in feature self-adaptive reinforcement learning
CN111741002A (en) * 2020-06-23 2020-10-02 广东工业大学 Method and device for training network intrusion detection model
CN111800414A (en) * 2020-07-03 2020-10-20 西北工业大学 Convolutional neural network-based traffic anomaly detection method and system
CN112364980A (en) * 2020-11-09 2021-02-12 北京计算机技术及应用研究所 Deep neural network training method based on reinforcement learning under weak supervision scene

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LEI XI；等: ""A Novel Multi-Agent DDQN-AD Method-Based Distributed Strategy for Automatic Generation Control of Integrated Energy Systems"", 《IEEE TRANSACTIONS ON SUSTAINABLE ENERGY》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113537383A (en) * 2021-07-29 2021-10-22 周口师范学院 Method for detecting abnormal flow of wireless network based on deep migration reinforcement learning
CN114374541A (en) * 2021-12-16 2022-04-19 四川大学 Abnormal network flow detector generation method based on reinforcement learning

Similar Documents

Publication Publication Date Title
Fan et al. Watching a small portion could be as good as watching all: Towards efficient video classification
US11689549B2 (en) Continuous learning for intrusion detection
Yu et al. An automatically tuning intrusion detection system
CN113179263A (en) Network intrusion detection method, device and equipment
WO2021139279A1 (en) Data processing method and apparatus based on classification model, and electronic device and medium
CN112153002B (en) Alarm information analysis method, device, computer equipment and storage medium
CN111709028B (en) Network security state evaluation and attack prediction method
CN113055384A (en) SSDDQN network abnormal flow detection method
CN113204745B (en) Deep learning back door defense method based on model pruning and reverse engineering
US11941867B2 (en) Neural network training using the soft nearest neighbor loss
CN113225346A (en) Network operation and maintenance situation assessment method based on machine learning
GSR et al. Hybrid optimization enabled deep learning technique for multi-level intrusion detection
CN114124460B (en) Industrial control system intrusion detection method and device, computer equipment and storage medium
US20230206601A1 (en) Device and method for classifying images and accessing the robustness of the classification
Liu et al. Handling concept drift in global time series forecasting
CN112613032B (en) Host intrusion detection method and device based on system call sequence
CN117077870B (en) Water resource digital management method based on artificial intelligence
Ibrahim et al. Modeling an intrusion detection using recurrent neural networks
CN114915496B (en) Network intrusion detection method and device based on time weight and deep neural network
Kushardianto et al. 2-step prediction for detecting attacker in vehicle to vehicle communication
CN113835973B (en) Model training method and related device
CN115952343A (en) Social robot detection method based on multi-relation graph convolutional network
Deekshitha et al. URL Based Phishing Website Detection by Using Gradient and Catboost Algorithms
CN115393925A (en) Face attraction classification method, system, equipment and medium
CN116524289A (en) Model training method and related system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210629