CN109818976A - A kind of anomalous traffic detection method and device - Google Patents
A kind of anomalous traffic detection method and device Download PDFInfo
- Publication number
- CN109818976A CN109818976A CN201910197232.3A CN201910197232A CN109818976A CN 109818976 A CN109818976 A CN 109818976A CN 201910197232 A CN201910197232 A CN 201910197232A CN 109818976 A CN109818976 A CN 109818976A
- Authority
- CN
- China
- Prior art keywords
- flow
- model
- network flow
- information
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application provides a kind of anomalous traffic detection method and device, when network flow is non-encrypted flow, if then storing flow information and its classification information by characteristic matching abnormal flow, if not when either encrypting flow when abnormal flow, the classification of network flow is then obtained using machine learning model, and stores flow information and classification information；The class label of the network flow stored is modified；Learn and updates the machine learning model.This programme constructs abnormal traffic detection system by the way of combining based on feature and machine learning, and feedback mechanism and self-study mechanism are introduced on this basis, it can constantly be trained using the data on flows in local network context, optimize machine learning model, it can not only be applied to the scene of encryption flow, the abnormal flow never occurred before can also effectively detecting.In addition this programme directly updates machine learning model in user local on-line training, is not necessarily to outgoing data, protects privacy of user.
Description
Technical field
This application involves network communication technology field, in particular to a kind of anomalous traffic detection method and device.
Background technique
Network flow refers to the data of transmission over networks, and most network flows are all usually in a network environment
Proper network flow, and these network flows have certain mode.And Abnormal network traffic is then those and proper network
Flow rate mode differs biggish network flow, these Abnormal network traffics are likely to the flow of aggression generation.If
It can recognize that abnormal flow, then can effectively take precautions against, identify, dissolve attack.Abnormal traffic detection system
System is a kind of system that abnormal process is identified from the all-network flow of a network, it is the one of network security management
A basis a, it has also become important research content in network safety filed.
In the prior art, a kind of abnormal traffic detection scheme be based on feature, namely from be identified manually for
Condition code is extracted in the flow of abnormal flow, and the identification of abnormal flow is then carried out using these condition codes.However because
Need using in flow data content and condition code matched, so the program can be only applied to the field of non-encrypted flow
Scape, meanwhile, the identification method based on condition code can not identify the abnormal flow never occurred before those.
Another scheme is then to generate classification using natural language processing N-Gram model extraction feature in the prior art
Model detects flow.But the program can not equally handle encryption flow, moreover, the disaggregated model that the program uses
Lower generation online is needed, then puts and uses on line, Automatic Optimal update can not be carried out, if it is desired to optimizing update just needs
It periodically obtains the data on flows in user network environment and is sent to the server in outer net and be used to generate new classification
The problem of then model updates the network equipment of user again, this relates to user information confidentiality is easy to be resisted, existing
It is difficult expansion application in reality.Therefore still lack a kind of effective abnormal traffic detection scheme in the prior art.
Summary of the invention
In view of this, the application provides a kind of anomalous traffic detection method and device, not occur to encryption flow and
Network flow detected.
Specifically, the application is achieved by the following technical solution:
A kind of anomalous traffic detection method, which comprises
When network flow to be detected is non-encrypted flow, judge whether the network flow matches default feature；
If matching default feature, it is determined as abnormal flow and stores the flow information and classification letter of the network flow
Breath；
If mismatching default feature, alternatively, being mentioned from the network flow when the network flow is encryption flow
Take out metadata；
The metadata is input to preset machine learning model to obtain the classification of the network flow, described in storage
The flow information and classification information of network flow；
It is modified according to classification information of the revision directive to the network flow stored；
According to the flow information and classification information of the network flow stored, learns and update the machine learning model.
A kind of abnormal traffic detection device, described device include:
First layer detection unit, for judging the network flow when network flow to be detected is non-encrypted flow
Whether feature is preset in matching；If matching default feature, it is determined as abnormal flow and triggers storage unit；If mismatched pre-
If feature, then second layer detection unit is triggered；
Second layer detection unit, for the triggering according to the first layer detection unit, or when the network flow is
When encrypting flow, metadata is extracted from the network flow；The metadata is input to preset machine learning model
To obtain the classification of the network flow, storage unit is triggered；
Storage unit, for storing the flow information and classification information of the network flow；
Feedback unit, for being modified according to classification information of the revision directive to the network flow that storage unit stores；
Unit, the flow information and classification information of the network flow for being stored according to storage unit, learn and more
The new machine learning model.
By the above technical solution provided by the present application as it can be seen that when network flow to be detected is non-encrypted flow, first lead to
The mode for crossing characteristic matching is judged, flow information and its classification information are stored if judging abnormal flow, if not
When abnormal flow (at this time may be normal discharge be also likely to be unknown abnormal flow) either encryption flow when, then using pre-
If machine learning model obtain the classification of network flow, and store flow information and classification information；According to revision directive to institute
The class label of the network flow of storage is modified；Learn and updates the machine learning model.This programme is used based on spy
The mode that machine learning of seeking peace combines the detection and classification that construct abnormal traffic detection system to realize abnormal flow, and
Feedback mechanism and self-study mechanism are introduced on the basis of this, can constantly be instructed using the data on flows in local network context
Practice, optimization machine learning model, realizes an identification, feedback, study, the complete closed loop updated, detection can be automatically performed
Update with taxon upgrades, and can not only be applied to the scene of encryption flow, before can also effectively detecting never
The abnormal flow occurred.In addition, directly in user local on-line training, update machine learning model in this programme, without outer
Data are sent out, to protect privacy of user.In addition, in this programme for encryption flow and non-encrypted flow detection it is asynchronous into
Row, can reduce detection system influence caused by network performance to the full extent.
Detailed description of the invention
Fig. 1 is a kind of flow chart of anomalous traffic detection method shown in the application；
Fig. 2 is a kind of flow chart of anomalous traffic detection method shown in the application；
Fig. 3 is a kind of schematic diagram of anomalous traffic detection method shown in the application；
Fig. 4 is a kind of flow chart of anomalous traffic detection method shown in the application；
Fig. 5 is a kind of flow chart of anomalous traffic detection method shown in the application；
Fig. 6 is a kind of schematic diagram of abnormal traffic detection device shown in the application.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistent with the application.On the contrary, they be only with it is such as appended
The example of the consistent device and method of some aspects be described in detail in claims, the application.
It is only to be not intended to be limiting the application merely for for the purpose of describing particular embodiments in term used in this application.
It is also intended in the application and the "an" of singular used in the attached claims, " described " and "the" including majority
Form, unless the context clearly indicates other meaning.It is also understood that term "and/or" used herein refers to and wraps
It may be combined containing one or more associated any or all of project listed.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the application
A little information should not necessarily be limited by these terms.These terms are only used to for same type of information being distinguished from each other out.For example, not departing from
In the case where the application range, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as
One information.Depending on context, word as used in this " if " can be construed to " ... when " or " when ...
When " or " in response to determination ".
Referring to Figure 1, Fig. 1 is a kind of flow chart of anomalous traffic detection method shown in the application.As an example, our
Method can be used for intrusion detections and the guard systems such as IPS (Intrusion Prevention System).This method may include
Following steps:
It is pre- to judge whether the network flow matches when network flow to be detected is non-encrypted flow by step S101
If feature.
Because network flow can be divided into encryption flow and non-encrypted flow, two layers of detection list is devised in this method
Member, this step are in first layer, and first layer is the detection unit (i.e. first detection unit) based on feature, all non-encrypted
Flow can all flow into the unit and be handled.
Step S102 is determined as abnormal flow and stores the flow letter of the network flow if matching default feature
Breath and classification information.
In first layer, the online lower common abnormal flow collected can be first passed through in advance, and analysis is extracted its feature and marked
Its attack type is infused, is then matched with network flow to be detected.If it is pre- that network flow to be detected matches some
If feature, then the flow is regarded as into attack type corresponding with this feature, its corresponding classification information can be assigned, stamp class
Distinguishing label.And if not matching any default feature, " normal discharge " can be regarded as.It should be noted that herein
" normal discharge " be also likely to be unknown abnormal flow, in other words, first layer may recognize that herein be abnormal flow it
Outer flow, is all temporarily considered as normal discharge.
The flow information and its type (i.e. classification information) of network flow by first layer detection unit can all be stored
Get up.Specifically, if first layer is identified as abnormal flow, training sample can be stored as machine learning mould
Type training updates.If unidentified first layer is abnormal flow, the second layer is continued to flow into, conduct finally can be also stored in order to
Training sample.
It should be noted that the equipment such as IPS would not allow for it to pass through if first layer is the discovery that abnormal flow.And such as
Fruit first layer is not the discovery that abnormal flow, then while being sent to the second layer and further being detected, the equipment such as IPS also can
The network flow is allowed to pass through, to reduce detection system influence caused by network performance.
The advantages of this layer of detection unit is that speed ratio is very fast, and Detection accuracy height is (for the flow for being matched to some feature
Data), but there is also disadvantages: on the one hand since the feature quantity of extraction is limited, thus the layer can not detect it is certain
The abnormal flow of type, on the other hand, the detection based on feature premise is that see the content of data on flows, but encrypt
The data content of flow is sightless, so also can not just be detected with this detection mode to encryption flow, at this time
It needs to be detected with following second layer detection unit.
Step S103, if mismatching default feature, alternatively, when the network flow is encryption flow, from the net
Metadata is extracted in network flow.
As an example, the metadata may include one or more of:
Session persistence, mean packet length, transmission data volume, reception data volume, the time interval sequence of data packet, etc.
Deng.Because these metadata be it is unrelated with load contents, it is all manageable for encrypting flow and non-encrypted flow.With
On only list a part of metadata, also have other metadata in practice.Definition for metadata, the present embodiment is not
Limited, those skilled in the art can according to different demands different scenes and voluntarily select, design, can make here
These selections and design are all without departing from spirit and scope of the invention.
The metadata is input to preset machine learning model to obtain the class of the network flow by step S104
Not, the flow information and classification information of the network flow are stored.
The second layer, i.e. second detection unit are the detection units based on machine learning.Encrypt flow and by first layer
Detection unit is determined as normal flow all and can flow into the detection unit continuing to detect.The but processing stream of the detection unit
Journey relative data stream be it is asynchronous, i.e., the detection unit will not block other operation flows when being detected.For tool
The machine learning model of body, the present embodiment are simultaneously not limited, and those skilled in the art can be according to Bu Tongxuqiu different scenes
And voluntarily select, design, these selections and design that can be used here are all without departing from spirit of the invention and protection model
It encloses.
Step S105 is modified according to classification information of the revision directive to the network flow stored.
This step belongs to feedback unit.Due to the limitation of machine learning, abnormal flow is examined using machine learning
Survey and classification may have the case where a degree of wrong report or misclassification, this just needs feedback mechanism to be made up.
Therefore shown in Figure 2, in this embodiment or some other embodiment of the present invention as example, referred to according to amendment
Order is modified the classification information of the network flow stored, can specifically include:
Step S201 shows the flow information and classification information of stored network flow；
Step S202 receives the revision directive to the classification information；
Step S203 modifies the classification information according to the revision directive.
In the present solution, the classification information of the network flow stored can be by being manually modified.Such as it can be by certain
The flow information and type (i.e. classification information) of class flow show that user can change its type on the administration interface of user,
Then system will record current change, and system can be with these modification records come Optimized model, to improve when learning, update
The accuracy rate of model.
Step S106 learns according to the flow information and classification information of the network flow stored and updates the machine
Learning model.
This step belongs to unit.Unit is one and is constantly remembered using the flow that front has correctly been detected and classified
It records to update, optimize the unit of machine learning model.Since the network flow data of tape label is fewer, and from the net of user
These datas on flows are collected in network environment then concentrate and be trained usually extremely difficult, be not allowed to even, and such as
Fruit is used to that the data that are trained are few, then the detection performance of machine learning model will be poor, thus the invention proposes
User side is trained automatically, i.e., trains, updates automatically the side of machine learning model using the data on flows in user network environment
Method, so as to realize the performance of continuous lifting system entirety.
The schematic diagram of the present embodiment can be with further reference to shown in Fig. 3, and in Fig. 3, non-encrypted flow is input to first (layer)
Detection unit, and encrypt flow and be then input to second (layer) detection unit；First detection unit if it is judged that be abnormal flow,
Then flow information and classification information are stored to storage unit, if not abnormal flow, then continued to be input to the second inspection
Survey unit；Second detection unit is detected and is classified to the network flow of input, then deposits flow information and classification information
It stores up to storage unit；Storage unit is modified the classification information stored by feedback unit；Unit utilizes storage
The flow information and classification information that unit is stored are updated optimization to second detection unit.
The present embodiment constructs abnormal traffic detection system by the way of combining based on feature and machine learning, realizes
The detection and classification of abnormal flow, and feedback mechanism and self-study mechanism are introduced on this basis, it can constantly utilize this
Data on flows in ground network environment trains, optimizes machine learning model, realizes an identification, feedback, study, updates
Complete closed loop, can be automatically performed detection and the upgrading of taxon updates, and can not only be applied to the scene of encryption flow, also
The abnormal flow never occurred before can effectively detecting.In addition, in this programme directly user local on-line training,
Machine learning model is updated, trains update when using, outgoing data is not necessarily to, to protect privacy of user.In addition, this programme
In for encrypt flow and non-encrypted flow the asynchronous progress of detection, detection system can be reduced to the full extent to network performance
Caused by influence.
In addition, the control to abnormal flow can be further applied after completing to the detection of abnormal flow,
Make it that can not endanger the safety of user network.Therefore shown in Figure 4, in the present embodiment or other some embodiments of the invention
In, the method can also include:
Step S401 obtains control rule according to the flow information of the network flow stored and classification information；
Step S402 controls the network flow according to the control rule.
As an example, in this embodiment or some other embodiment of the present invention, the control rule can specifically include:
The source IP of network flow and or destination IP pass through restriction rule.
Flow usually has the information such as source IP, source port, destination IP, destination port, agreement, be identified as abnormal flow it
Afterwards, system can limit the inflow of the flow with identical sources IP.Can certainly further limit any source IP or
Destination IP is passing through for the flow of the above-mentioned source IP identified or destination IP.
Machine learning model and unit are illustrated again below:
As an example, second layer detection unit can be divided into two parts: abnormal traffic detection model (the first model),
With abnormal flow disaggregated model (the second model).Abnormal traffic detection model can will be abnormal with unsupervised machine learning model
Flow is identified from network flow.Abnormal flow disaggregated model can be with having the machine learning model of supervision to abnormal flow
Being classified, (abnormal flow caused by different types of network attack is also different, so classification here can be with
Finger classifies to the corresponding network attack of abnormal flow).
Therefore in this embodiment or some other embodiment of the present invention, the machine learning model specifically may include first
Model and the second model, wherein first model is preset unsupervised machine learning model, second model is default
Supervised machine learning model；
It is corresponding shown in Figure 5, the metadata is input to preset machine learning model to obtain the network
The classification of flow, can specifically include following steps:
The metadata is input to first model, obtains the output of first model by step S501；
Step S502 is deposited if determining that the network flow is not abnormal flow according to the output of first model
Store up the flow information and classification information of the network flow；
Step S503, if determining that the network flow is abnormal flow according to the output of first model, by institute
It states metadata and inputs second model, obtain the output of second model；
Step S504 obtains the classification of the network flow according to the output of second model, stores the network flow
The flow information and classification information of amount.
Abnormal traffic detection model (i.e. the first model) uses a unsupervised machine learning model, such as cluster, LDA
(linear discriminent analysis) etc..It is entered into the first model after the metadata extracted is done normalized, the first mould
Type can judge whether the network flow is abnormal according to these data.If it is abnormal flow, then the flow is given different
Normal flow disaggregated model is handled.
Abnormal flow disaggregated model (i.e. the second model) uses the machine learning model for having supervision, such as SVM,
Random forest, deep neural network etc..The model is used to identify the specific attack type for generating abnormal flow.By first number of extraction
According to inputting the second model.Metadata can also be referred to as statistical data, these data are all to unite from different perspectives to flow
Meter.These metadata can be generally normalized with z-score, even if also each metadata all in a mean value is
0, among the normal distribution that variance is 1.Metadata generally compares less, is probably in more than 40 or so namely input vector
40 dimension left and right.Output is a probability vector, and the length of the vector is the type attacked.Assuming that there is the attack of 5 classes, then output is
Flow is identified as the probability of corresponding classification by one 5 dimensional vector, each element representation in vector, such as [0.1,0.7,0.1,
0.1,0.0] mean that the probability for being identified as the first kind is 0.1, the probability for being identified as the second class is 0.7, and model can choose generally
That maximum classification of rate is as classification results.It then (is an array, the first attack is the of array in a mapping table
One element, second attack be exactly array second element, and so on) in find corresponding attack title.
Unit is one and is constantly updated using the discharge record that front has correctly been detected and classified, optimizes engineering
Practise the unit of model.The flow for wherein correctly having detected and having classified may include two classes, one kind be first layer detection unit institute into
Row classification, another kind of to be that user carried out artificial modified.Correctly detect and classify network flow flow information and
Classification information can be all stored in a specific memory space, and unit meeting not timing obtains data from the storage unit
To be learnt.
Because a complete learning process may consume longer time, unit is relative to other Business Streams
Journey be it is asynchronous, other operation flows will not be blocked.Two models in second detection unit be when initial it is online under train
Two universal models, with the operation of system on line, unit can be constantly according to the data in current network environment
The two models are updated respectively, to make model that there is preferably detection and classification performance in current network environment.The two
What the learning process of model carried out automatically respectively, completely without manual intervention.When the model learnt reaches preset condition
When just terminate learning process, then asynchronous refresh.
As an example, in specifically study renewal process, it can be according to the stream for the network flow that storage unit is stored
Information and classification information are measured, continues to train on the basis of the first model and second model respectively original Parameter File, then use
The new Parameter File that training obtains replaces original Parameter File.
Each model is incremental update, i.e., is trained with the data newly obtained to model.Incremental update can be simple
Understand are as follows: assuming that reaching the position that height is 2 meters, training next time after certain training now in the position that height is 1 meter
At 2 meters, this position starts for meeting, rather than is trained since 1 meter.Each training be all on the basis of trained before into
Capable.So-called trained model is exactly the parameter of model, such as parameter a=10, b=20.These parameters can be with certain lattice
Formula is stored, and is typically stored in one file.It updates the corresponding file of Shi Huiyong new model and replaces original model
File.Model can read the model file after replacing, and replace existing parameter of the same name with the parameter in model file.
Following is apparatus of the present invention embodiment, can be used for executing embodiment of the present invention method.For apparatus of the present invention reality
Undisclosed details in example is applied, embodiment of the present invention method is please referred to.
Fig. 6 is referred to, Fig. 6 is a kind of schematic diagram of abnormal traffic detection device shown in the application.As an example, this dress
It sets and can be used for intrusion detections and the guard systems such as IPS (Intrusion Prevention System).The present apparatus can wrap
It includes:
First layer detection unit 601, for judging the network flow when network flow to be detected is non-encrypted flow
Whether amount matches default feature；If matching default feature, it is determined as abnormal flow and triggers storage unit；If mismatched
Default feature, then trigger second layer detection unit；
Second layer detection unit 602 for the triggering according to the first layer detection unit, or works as the network flow
When to encrypt flow, metadata is extracted from the network flow；The metadata is input to preset machine learning mould
Type triggers storage unit to obtain the classification of the network flow；
Storage unit 603, for storing the flow information and classification information of the network flow；
Feedback unit 604, for being repaired according to classification information of the revision directive to the network flow that storage unit stores
Just；
Unit 605, the flow information and classification information of the network flow for being stored according to storage unit, study is simultaneously
Update the machine learning model.
In this embodiment or some other embodiment of the present invention, the machine learning model may include the first model and
Second model, first model are preset unsupervised machine learning model, and second model has intendant to be preset
Device learning model；
The metadata is being input to preset machine learning model to obtain the net by the second layer detection unit
When the classification of network flow, it is specifically used for:
The metadata is input to first model, obtains the output of first model；If according to described
The output of one model determines that the network flow is not abnormal flow, then triggers storage unit；If according to first model
Output determine that the network flow is abnormal flow, then the metadata is inputted into second model, obtains described second
The output of model；The classification of the network flow is obtained according to the output of second model, triggers storage unit.
Abnormal traffic detection model (i.e. the first model) uses a unsupervised machine learning model, such as cluster, LDA
(linear discriminent analysis) etc..It is entered into the first model after the metadata extracted is done normalized, the first mould
Type can judge whether the network flow is abnormal according to these data.If it is abnormal flow, then the flow is given different
Normal flow disaggregated model is handled.
Abnormal flow disaggregated model (i.e. the second model) uses the machine learning model for having supervision, such as SVM,
Random forest, deep neural network etc..The model is used to identify the specific attack type for generating abnormal flow.
In this embodiment or some other embodiment of the present invention, the metadata may include one or more of:
Session persistence, mean packet length, transmission data volume, reception data volume, the time interval sequence of data packet, etc.
Deng.
In this embodiment or some other embodiment of the present invention, the feedback unit can specifically include:
Show subelement, the flow information and classification information of the network flow for showing storage unit storage；
Command reception subelement, for receiving the revision directive to the classification information；
Instruction execution subelement, for modifying the classification information according to the revision directive.
In other words, the classification information of the network flow stored can be by being manually modified.Such as it can be by certain class
The flow information and type (i.e. classification information) of flow show that user can change its type, so on the administration interface of user
System will record current change afterwards, and system can be with these modification records come Optimized model, to improve mould when learning, update
The accuracy rate of type.
In this embodiment or some other embodiment of the present invention, described device can also include:
Flow controlling unit, the flow information and classification information of the network flow for being stored according to storage unit obtain control
System rule；The network flow is controlled according to the control rule.
After completing to the detection of abnormal flow, the control to abnormal flow can be further applied, it is made
The safety of user network can not be endangered.
In this embodiment or some other embodiment of the present invention, the control rule can specifically include:
The source IP of network flow and or destination IP pass through restriction rule.
Flow usually has the information such as source IP, source port, destination IP, destination port, agreement, be identified as abnormal flow it
Afterwards, system can limit the inflow of the flow with identical sources IP.Can certainly further limit any source IP or
Destination IP is passing through for the flow of the above-mentioned source IP identified or destination IP.
The present embodiment constructs abnormal traffic detection system by the way of combining based on feature and machine learning, realizes
The detection and classification of abnormal flow, and feedback mechanism and self-study mechanism are introduced on this basis, it can constantly utilize this
Data on flows in ground network environment trains, optimizes machine learning model, realizes an identification, feedback, study, updates
Complete closed loop, can be automatically performed detection and the upgrading of taxon updates, and can not only be applied to the scene of encryption flow, also
The abnormal flow never occurred before can effectively detecting.In addition, in this programme directly user local on-line training,
Machine learning model is updated, trains update when using, outgoing data is not necessarily to, to protect privacy of user.In addition, this programme
In for encrypt flow and non-encrypted flow the asynchronous progress of detection, detection system can be reduced to the full extent to network performance
Caused by influence.
About the device in above-described embodiment, wherein each unit module execute the concrete mode of operation in correlation
It is described in detail in the embodiment of method, details are not described herein again.
The function of each unit and the realization process of effect are specifically detailed in the above method and correspond to step in above-mentioned apparatus
Realization process, details are not described herein.
For device embodiment, since it corresponds essentially to embodiment of the method, so related place is referring to method reality
Apply the part explanation of example.The apparatus embodiments described above are merely exemplary, wherein described be used as separation unit
The unit of explanation may or may not be physically separated, and component shown as a unit can be or can also be with
It is not physical unit, it can it is in one place, or may be distributed over multiple network units.It can be according to actual
The purpose for needing to select some or all of the modules therein to realize application scheme.Those of ordinary skill in the art are not paying
Out in the case where creative work, it can understand and implement.
The foregoing is merely the preferred embodiments of the application, not to limit the application, all essences in the application
Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the application protection.
Claims (12)
1. a kind of anomalous traffic detection method, which is characterized in that the described method includes:
When network flow to be detected is non-encrypted flow, judge whether the network flow matches default feature；
If matching default feature, it is determined as abnormal flow and stores the flow information and classification information of the network flow；
If mismatching default feature, alternatively, being extracted from the network flow when the network flow is encryption flow
Metadata；
The metadata is input to preset machine learning model to obtain the classification of the network flow, stores the network
The flow information and classification information of flow；
It is modified according to classification information of the revision directive to the network flow stored；
According to the flow information and classification information of the network flow stored, learns and update the machine learning model.
2. the method according to claim 1, wherein the machine learning model includes the first model and the second mould
Type, first model are preset unsupervised machine learning model, and second model is preset Supervised machine learning
Model；
The metadata is input to preset machine learning model to obtain the classification of the network flow, comprising:
The metadata is input to first model, obtains the output of first model；
If determining that the network flow is not abnormal flow according to the output of first model, the network flow is stored
Flow information and classification information；
If determining that the network flow is abnormal flow according to the output of first model, the metadata is inputted into institute
The second model is stated, the output of second model is obtained；
The classification that the network flow is obtained according to the output of second model, store the network flow flow information and
Classification information.
3. method according to claim 1 or 2, which is characterized in that the metadata includes one or more of:
Session persistence, mean packet length send data volume, receive data volume, the time interval sequence of data packet.
4. the method according to claim 1, wherein the classification according to revision directive to the network flow stored
Information is modified, comprising:
Show the flow information and classification information of stored network flow；
Receive the revision directive to the classification information；
The classification information is modified according to the revision directive.
5. the method according to claim 1, wherein the method also includes:
Control rule is obtained according to the flow information of the network flow stored and classification information；
The network flow is controlled according to the control rule.
6. according to the method described in claim 5, it is characterized in that, the control rule includes:
The source IP of network flow and or destination IP pass through restriction rule.
7. a kind of abnormal traffic detection device, which is characterized in that described device includes:
First layer detection unit, for whether judging the network flow when network flow to be detected is non-encrypted flow
Match default feature；If matching default feature, it is determined as abnormal flow and triggers storage unit；If mismatching default spy
Sign, then trigger second layer detection unit；
Second layer detection unit, for the triggering according to the first layer detection unit, or when the network flow is encryption
When flow, metadata is extracted from the network flow；The metadata is input to preset machine learning model to obtain
The classification of the network flow is taken, storage unit is triggered；
Storage unit, for storing the flow information and classification information of the network flow；
Feedback unit, for being modified according to classification information of the revision directive to the network flow that storage unit stores；
Unit, the flow information and classification information of the network flow for being stored according to storage unit learn and update institute
State machine learning model.
8. device according to claim 7, which is characterized in that the machine learning model includes the first model and the second mould
Type, first model are preset unsupervised machine learning model, and second model is preset Supervised machine learning
Model；
The metadata is being input to preset machine learning model to obtain the network flow by the second layer detection unit
When the classification of amount, it is specifically used for:
The metadata is input to first model, obtains the output of first model；If according to first mould
The output of type determines that the network flow is not abnormal flow, then triggers storage unit；If according to the defeated of first model
Determine that the network flow is abnormal flow out, then the metadata is inputted into second model, obtains second model
Output；The classification of the network flow is obtained according to the output of second model, triggers storage unit.
9. device according to claim 7 or 8, which is characterized in that the metadata includes one or more of:
Session persistence, mean packet length send data volume, receive data volume, the time interval sequence of data packet.
10. device according to claim 7, which is characterized in that the feedback unit includes:
Show subelement, the flow information and classification information of the network flow for showing storage unit storage；
Command reception subelement, for receiving the revision directive to the classification information；
Instruction execution subelement, for modifying the classification information according to the revision directive.
11. device according to claim 7, which is characterized in that described device further include:
Flow controlling unit, the flow information and classification information of the network flow for being stored according to storage unit obtain control rule
Then；The network flow is controlled according to the control rule.
12. device according to claim 11, which is characterized in that the control rule includes:
The source IP of network flow and or destination IP pass through restriction rule.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910197232.3A CN109818976B (en) | 2019-03-15 | 2019-03-15 | Abnormal flow detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910197232.3A CN109818976B (en) | 2019-03-15 | 2019-03-15 | Abnormal flow detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109818976A true CN109818976A (en) | 2019-05-28 |
| CN109818976B CN109818976B (en) | 2021-09-21 |
Family
ID=66609132
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910197232.3A Active CN109818976B (en) | 2019-03-15 | 2019-03-15 | Abnormal flow detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109818976B (en) |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110247930A (en) * | 2019-07-01 | 2019-09-17 | 北京理工大学 | A kind of refined net method for recognizing flux based on deep neural network |
| CN110781950A (en) * | 2019-10-23 | 2020-02-11 | 新华三信息安全技术有限公司 | Message processing method and device |
| CN110782014A (en) * | 2019-10-23 | 2020-02-11 | 新华三信息安全技术有限公司 | Neural network increment learning method and device |
| CN110839042A (en) * | 2019-11-22 | 2020-02-25 | 上海交通大学 | Flow-based self-feedback malicious software monitoring system and method |
| CN111143169A (en) * | 2019-12-30 | 2020-05-12 | 杭州迪普科技股份有限公司 | Abnormal parameter detection method and device, electronic equipment and storage medium |
| CN111294332A (en) * | 2020-01-13 | 2020-06-16 | 交通银行股份有限公司 | Traffic anomaly detection and DNS channel anomaly detection system and method |
| CN111582235A (en) * | 2020-05-26 | 2020-08-25 | 瑞纳智能设备股份有限公司 | Alarm method, system and equipment for monitoring abnormal events in station in real time |
| CN111935144A (en) * | 2020-08-10 | 2020-11-13 | 武汉思普崚技术有限公司 | Method and system for analyzing traffic safety |
| CN112118268A (en) * | 2020-09-28 | 2020-12-22 | 北京嘀嘀无限科技发展有限公司 | Network flow judgment method and system |
| CN112134898A (en) * | 2020-09-28 | 2020-12-25 | 北京嘀嘀无限科技发展有限公司 | Network flow judgment method and system |
| CN112235230A (en) * | 2019-07-15 | 2021-01-15 | 北京观成科技有限公司 | Malicious traffic identification method and system |
| CN112511457A (en) * | 2019-09-16 | 2021-03-16 | 华为技术有限公司 | Data stream type identification method and related equipment |
| CN112954689A (en) * | 2021-02-07 | 2021-06-11 | 中国科学院计算技术研究所 | Lightweight network intrusion detection system and method for Bluetooth wireless transmission |
| CN112995052A (en) * | 2021-04-25 | 2021-06-18 | 北京世纪好未来教育科技有限公司 | Flow control method and related device |
| CN113538049A (en) * | 2021-07-14 | 2021-10-22 | 北京明略软件系统有限公司 | Abnormal flow identification system |
| CN113595967A (en) * | 2020-04-30 | 2021-11-02 | 深信服科技股份有限公司 | Data identification method, equipment, storage medium and device |
| CN113810343A (en) * | 2020-06-15 | 2021-12-17 | 深信服科技股份有限公司 | Method, device and equipment for detecting function injection attack and readable storage medium |
| CN113872939A (en) * | 2021-08-30 | 2021-12-31 | 济南浪潮数据技术有限公司 | Flow detection method, device and storage medium |
| CN114422242A (en) * | 2022-01-19 | 2022-04-29 | 闪捷信息科技有限公司 | Abnormal traffic identification method, client and server |
| CN114465823A (en) * | 2022-04-08 | 2022-05-10 | 杭州海康威视数字技术股份有限公司 | Industrial Internet terminal encrypted flow data security detection method, device and equipment |
| CN115134276A (en) * | 2022-05-12 | 2022-09-30 | 亚信科技（成都）有限公司 | Ore digging flow detection method and device |
| CN115242427A (en) * | 2022-06-08 | 2022-10-25 | 浪潮通信信息系统有限公司 | Network flow abnormity detection method and system |
| CN116915512A (en) * | 2023-09-14 | 2023-10-20 | 国网江苏省电力有限公司常州供电分公司 | Method and device for detecting communication flow in power grid |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104735074A (en) * | 2015-03-31 | 2015-06-24 | 江苏通付盾信息科技有限公司 | Malicious URL detection method and implement system thereof |
| CN106060043A (en) * | 2016-05-31 | 2016-10-26 | 北京邮电大学 | Abnormal flow detection method and device |
| CN107360159A (en) * | 2017-07-11 | 2017-11-17 | 中国科学院信息工程研究所 | A kind of method and device for identifying abnormal encryption flow |
| CN107451476A (en) * | 2017-07-21 | 2017-12-08 | 上海携程商务有限公司 | Webpage back door detection method, system, equipment and storage medium based on cloud platform |
| CN107749859A (en) * | 2017-11-08 | 2018-03-02 | 南京邮电大学 | A kind of malice Mobile solution detection method of network-oriented encryption flow |
| US9998480B1 (en) * | 2016-02-29 | 2018-06-12 | Symantec Corporation | Systems and methods for predicting security threats |
| CN108173708A (en) * | 2017-12-18 | 2018-06-15 | 北京天融信网络安全技术有限公司 | Anomalous traffic detection method, device and storage medium based on incremental learning |
| CN108833360A (en) * | 2018-05-23 | 2018-11-16 | 四川大学 | A kind of malice encryption flow identification technology based on machine learning |
| CN109151880A (en) * | 2018-11-08 | 2019-01-04 | 中国人民解放军国防科技大学 | Mobile application flow identification method based on multilayer classifier |
-
2019
- 2019-03-15 CN CN201910197232.3A patent/CN109818976B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104735074A (en) * | 2015-03-31 | 2015-06-24 | 江苏通付盾信息科技有限公司 | Malicious URL detection method and implement system thereof |
| US9998480B1 (en) * | 2016-02-29 | 2018-06-12 | Symantec Corporation | Systems and methods for predicting security threats |
| CN106060043A (en) * | 2016-05-31 | 2016-10-26 | 北京邮电大学 | Abnormal flow detection method and device |
| CN107360159A (en) * | 2017-07-11 | 2017-11-17 | 中国科学院信息工程研究所 | A kind of method and device for identifying abnormal encryption flow |
| CN107451476A (en) * | 2017-07-21 | 2017-12-08 | 上海携程商务有限公司 | Webpage back door detection method, system, equipment and storage medium based on cloud platform |
| CN107749859A (en) * | 2017-11-08 | 2018-03-02 | 南京邮电大学 | A kind of malice Mobile solution detection method of network-oriented encryption flow |
| CN108173708A (en) * | 2017-12-18 | 2018-06-15 | 北京天融信网络安全技术有限公司 | Anomalous traffic detection method, device and storage medium based on incremental learning |
| CN108833360A (en) * | 2018-05-23 | 2018-11-16 | 四川大学 | A kind of malice encryption flow identification technology based on machine learning |
| CN109151880A (en) * | 2018-11-08 | 2019-01-04 | 中国人民解放军国防科技大学 | Mobile application flow identification method based on multilayer classifier |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110247930A (en) * | 2019-07-01 | 2019-09-17 | 北京理工大学 | A kind of refined net method for recognizing flux based on deep neural network |
| CN112235230B (en) * | 2019-07-15 | 2023-05-02 | 北京观成科技有限公司 | Malicious traffic identification method and system |
| CN112235230A (en) * | 2019-07-15 | 2021-01-15 | 北京观成科技有限公司 | Malicious traffic identification method and system |
| CN112511457A (en) * | 2019-09-16 | 2021-03-16 | 华为技术有限公司 | Data stream type identification method and related equipment |
| US11838215B2 (en) | 2019-09-16 | 2023-12-05 | Huawei Technologies Co., Ltd. | Data stream classification method and related device |
| CN110781950B (en) * | 2019-10-23 | 2023-06-30 | 新华三信息安全技术有限公司 | Message processing method and device |
| CN110781950A (en) * | 2019-10-23 | 2020-02-11 | 新华三信息安全技术有限公司 | Message processing method and device |
| CN110782014A (en) * | 2019-10-23 | 2020-02-11 | 新华三信息安全技术有限公司 | Neural network increment learning method and device |
| CN110839042A (en) * | 2019-11-22 | 2020-02-25 | 上海交通大学 | Flow-based self-feedback malicious software monitoring system and method |
| US11709912B2 (en) | 2019-12-30 | 2023-07-25 | Hangzhou Dptech Technologies Co., Ltd. | Abnormality detection |
| CN111143169B (en) * | 2019-12-30 | 2024-02-27 | 杭州迪普科技股份有限公司 | Abnormal parameter detection method and device, electronic equipment and storage medium |
| CN111143169A (en) * | 2019-12-30 | 2020-05-12 | 杭州迪普科技股份有限公司 | Abnormal parameter detection method and device, electronic equipment and storage medium |
| CN111294332A (en) * | 2020-01-13 | 2020-06-16 | 交通银行股份有限公司 | Traffic anomaly detection and DNS channel anomaly detection system and method |
| CN113595967A (en) * | 2020-04-30 | 2021-11-02 | 深信服科技股份有限公司 | Data identification method, equipment, storage medium and device |
| CN111582235B (en) * | 2020-05-26 | 2023-04-07 | 瑞纳智能设备股份有限公司 | Alarm method, system and equipment for monitoring abnormal events in station in real time |
| CN111582235A (en) * | 2020-05-26 | 2020-08-25 | 瑞纳智能设备股份有限公司 | Alarm method, system and equipment for monitoring abnormal events in station in real time |
| CN113810343B (en) * | 2020-06-15 | 2023-05-12 | 深信服科技股份有限公司 | Method, device and equipment for detecting function injection attack and readable storage medium |
| CN113810343A (en) * | 2020-06-15 | 2021-12-17 | 深信服科技股份有限公司 | Method, device and equipment for detecting function injection attack and readable storage medium |
| CN111935144A (en) * | 2020-08-10 | 2020-11-13 | 武汉思普崚技术有限公司 | Method and system for analyzing traffic safety |
| CN112118268A (en) * | 2020-09-28 | 2020-12-22 | 北京嘀嘀无限科技发展有限公司 | Network flow judgment method and system |
| CN112134898A (en) * | 2020-09-28 | 2020-12-25 | 北京嘀嘀无限科技发展有限公司 | Network flow judgment method and system |
| CN112954689A (en) * | 2021-02-07 | 2021-06-11 | 中国科学院计算技术研究所 | Lightweight network intrusion detection system and method for Bluetooth wireless transmission |
| CN112995052B (en) * | 2021-04-25 | 2021-08-06 | 北京世纪好未来教育科技有限公司 | Flow control method and related device |
| CN112995052A (en) * | 2021-04-25 | 2021-06-18 | 北京世纪好未来教育科技有限公司 | Flow control method and related device |
| CN113538049A (en) * | 2021-07-14 | 2021-10-22 | 北京明略软件系统有限公司 | Abnormal flow identification system |
| CN113872939A (en) * | 2021-08-30 | 2021-12-31 | 济南浪潮数据技术有限公司 | Flow detection method, device and storage medium |
| CN114422242A (en) * | 2022-01-19 | 2022-04-29 | 闪捷信息科技有限公司 | Abnormal traffic identification method, client and server |
| CN114465823B (en) * | 2022-04-08 | 2022-08-19 | 杭州海康威视数字技术股份有限公司 | Industrial Internet terminal encrypted flow data security detection method, device and equipment |
| CN114465823A (en) * | 2022-04-08 | 2022-05-10 | 杭州海康威视数字技术股份有限公司 | Industrial Internet terminal encrypted flow data security detection method, device and equipment |
| CN115134276A (en) * | 2022-05-12 | 2022-09-30 | 亚信科技（成都）有限公司 | Ore digging flow detection method and device |
| CN115134276B (en) * | 2022-05-12 | 2023-12-08 | 亚信科技（成都）有限公司 | Mining flow detection method and device |
| CN115242427A (en) * | 2022-06-08 | 2022-10-25 | 浪潮通信信息系统有限公司 | Network flow abnormity detection method and system |
| CN116915512A (en) * | 2023-09-14 | 2023-10-20 | 国网江苏省电力有限公司常州供电分公司 | Method and device for detecting communication flow in power grid |
| CN116915512B (en) * | 2023-09-14 | 2023-12-01 | 国网江苏省电力有限公司常州供电分公司 | Method and device for detecting communication flow in power grid |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109818976B (en) | 2021-09-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109818976A (en) | A kind of anomalous traffic detection method and device | |
| US11031135B2 (en) | Determination of cybersecurity recommendations | |
| EP3107026B1 (en) | Event anomaly analysis and prediction | |
| Rastegari et al. | Evolving statistical rulesets for network intrusion detection | |
| CN106790008B (en) | Machine learning system for detecting abnormal host in enterprise network | |
| Sharma et al. | Analysis of machine learning techniques based intrusion detection systems | |
| Hijazi et al. | A Deep Learning Approach for Intrusion Detection System in Industry Network. | |
| CN105637519A (en) | Cognitive information security using a behavior recognition system | |
| Arora et al. | Evaluation of machine learning algorithms used on attacks detection in industrial control systems | |
| Castellanos et al. | A modular hybrid learning approach for black-box security testing of CPS | |
| CN107111610A (en) | Mapper component for neural language performance identifying system | |
| CN107111609A (en) | Lexical analyzer for neural language performance identifying system | |
| Thames et al. | Cybersecurity for Industry 4.0 and advanced manufacturing environments with ensemble intelligence | |
| Musa et al. | A review on intrusion detection system using machine learning techniques | |
| Jain et al. | Hidden markov model based anomaly intrusion detection | |
| Dehlaghi-Ghadim et al. | Anomaly detection dataset for industrial control systems | |
| KR20190107523A (en) | System and method for handling network failure using syslog | |
| Kim et al. | Unknown payload anomaly detection based on format and field semantics inference in cyber-physical infrastructure systems | |
| Gupta et al. | Genetic algorithm technique used to detect intrusion detection | |
| Efiong et al. | A contrived dataset of substation automation for cybersecurity research in the smart grid networks based on IEC61850 | |
| Müller et al. | CyPhERS: A cyber-physical event reasoning system providing real-time situational awareness for attack and fault response | |
| CN113992419A (en) | User abnormal behavior detection and processing system and method thereof | |
| TWI667587B (en) | Information security protection method | |
| Bukola et al. | Auto-immunity dendritic cell algorithm | |
| Malek et al. | User Behaviour based Intrusion Detection System Overview |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |