CN108173708A - Anomalous traffic detection method, device and storage medium based on incremental learning - Google Patents
Anomalous traffic detection method, device and storage medium based on incremental learning Download PDFInfo
- Publication number
- CN108173708A CN108173708A CN201711363640.9A CN201711363640A CN108173708A CN 108173708 A CN108173708 A CN 108173708A CN 201711363640 A CN201711363640 A CN 201711363640A CN 108173708 A CN108173708 A CN 108173708A
- Authority
- CN
- China
- Prior art keywords
- data
- training
- abnormal
- grader
- incremental learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0823—Errors, e.g. transmission errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Networks & Wireless Communication (AREA)
- Evolutionary Computation (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of anomalous traffic detection method based on incremental learning, device and storage medium detection methods, obtain the data on flows of user terminal；The data on flows is carried out abnormality detection using the flow detection grader built in advance in abnormality detecting apparatus；When detecting abnormal data, based on the abnormal data, training sample data are obtained, and on-line training is carried out to the flow detection grader using the training sample data.The present invention is by increasing the diversity of training sample, improving the generalization ability of grader.
Description
Technical field
The present invention relates to exception of network traffic detection techniques, and in particular to a kind of abnormal traffic detection based on incremental learning
Method, apparatus and storage medium.
Background technology
Exception of network traffic refers to that network flow deviates the situation of its normal trace, such as:It occupies the operation behavior of resource, attack
Hit the safety that the exception that sexual behaviour etc., especially attack generate will threaten whole network.The purpose of Traffic anomaly detection
It seeks to find these exceptions in time, and makes quick reflection.
The method of current flux abnormality detection includes the detection method based on statistical analysis and the detection based on machine learning
Method.
Method based on statistical analysis can be analyzed data traffic sampling according to time series, from data distribution,
Multiple dimensions such as changes in flow rate, child resource occupancy situation are for statistical analysis, and the feature of extraction description flow recycles these spies
Sign data analyze some threshold value results by grader and are used as discrimination standard；It can also be by analyzing data pack load part
Character string identifies abnormal abnormal flow.
Detection method based on machine learning usually models composition and classification device, then respectively to normal and abnormal flow
Prediction belongs to normal and abnormal probability respectively, takes the big person of probability as final category result.
Above two scheme is had the following problems:
1. real-time is poor, the either selection of the threshold value based on the statistical analysis still model based on machine learning
Structure is all to need first to analyze data under line, then disposes production environment on line again, and data on flows is all the time
All continually changing, such model or threshold value obviously easily generate erroneous judgement or even failure, also inevitable even if timing updates
Hysteresis quality in having time.
2. model is built based on a large amount of positive negative datas, since negative data is rare, model generalization energy force difference.
Invention content
The purpose of the invention is to provide the anomalous traffic detection method based on incremental learning, device and storage medium,
This method increases the diversity of training sample, improves the generalization ability of grader.
One side according to the present invention provides a kind of anomalous traffic detection method based on incremental learning,
Obtain the data on flows of user terminal；
The data on flows is carried out abnormality detection using the flow detection grader built in advance in abnormality detecting apparatus；
When detecting abnormal data, based on the abnormal data, training sample data are obtained, and utilize the trained sample
Notebook data carries out on-line training to the flow detection grader.
Optionally, it is described to be based on the abnormal data in the method for the invention, training sample data are obtained, including：
Extract the valid data in the abnormal data；
Mode excavation is carried out to the valid data of extraction；
The data that mode excavation obtains are normalized, obtain training sample data.
Optionally, in the method for the invention, when detecting abnormal data, the method further includes：Send out alarm.
Optionally, in the method for the invention, the method further includes：
Acquire historical traffic data；
Based on the historical traffic data, history training sample data are obtained；
Using the history training sample data, off-line training is carried out to the flow detection grader, off-line training is more
New flow detection grader real-time synchronization is to the exception monitoring equipment.
Optionally, in the method for the invention, using deep learning algorithm or transfer learning algorithm, the flow is examined
It surveys grader and carries out online or off-line training.
Other side according to the present invention provides a kind of abnormal traffic detection device based on incremental learning, including number
According to acquisition module and online incremental learning module,
The data acquisition module, for obtaining the data on flows of user terminal；
The online incremental learning module, for utilizing the flow detection grader pair built in advance in abnormality detecting apparatus
The data on flows carries out abnormality detection；
When detecting abnormal data, based on the abnormal data, training sample data are obtained, and utilize the trained sample
Notebook data carries out on-line training to the flow detection grader.
Optionally, in device of the present invention, described when detecting abnormal data, the method further includes：Send out police
Report.
Optionally, in device of the present invention, described device further includes：Offline incremental learning module, the data acquisition
Module is additionally operable to acquisition historical traffic data；
The offline incremental learning module for being based on the historical traffic data, obtains history training sample data；Profit
With the history training sample data, off-line training, the newer flow inspection of off-line training are carried out to the flow detection grader
Grader real-time synchronization is surveyed to the exception monitoring equipment.
The second aspect according to the present invention, a kind of abnormal traffic detection equipment based on incremental learning, including：Storage
Device, processor and the computer program that can be run on the memory and on the processor is stored in, the computer journey
Sequence realizes the step of anomalous traffic detection method as described above based on incremental learning when being performed by the processor.
In terms of third according to the present invention, a kind of computer readable storage medium, the computer readable storage medium
On be stored with the abnormal traffic detection program based on incremental learning, the abnormal traffic detection program based on incremental learning is located
The step of reason device realizes anomalous traffic detection method as described above based on incremental learning when performing.
Compared with prior art, effect of the invention is as follows：
Anomalous traffic detection method provided by the invention based on incremental learning, device and storage medium, by increasing online
The mode of study is measured, new samples is captured in real time, increases the diversity of training sample, in actual production, to ensure that model is general
The continuous enhancing of change ability, final realize predict accurate, real-time purpose.
The present invention is in a manner that offline incremental learning and online incremental learning are combined, to off-line training and on-line training
The update of grader real-time synchronization, further increase the diversity of sample.
Above description is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, below the special specific embodiment for lifting the present invention.
Description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention, for those of ordinary skill in the art, without having to pay creative labor, may be used also
To obtain other attached drawings according to these attached drawings.
Fig. 1 is the flow chart of the anomalous traffic detection method on-line training the present invention is based on incremental learning；
Fig. 2 is the present invention is based on abnormal data, obtains the flow chart of training sample data；
Fig. 3 is the network structure that the present invention uses convolutional neural networks algorithm；
Fig. 4 is the flow chart of the anomalous traffic detection method off-line training the present invention is based on incremental learning；
Fig. 5 is the flow chart of specific example of the present invention；
Fig. 6 is the functional block diagram of the abnormal traffic detection device the present invention is based on incremental learning.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other without making creative work
Embodiment shall fall within the protection scope of the present invention.
Shown in Figure 1, in order to solve the problems in the prior art, the present invention provides a kind of exception based on incremental learning
Flow rate testing methods,
Step S100：Obtain the data on flows of user terminal；
Step S200：The data on flows is carried out using the flow detection grader built in advance in abnormality detecting apparatus
Abnormality detection；
Step S300：When detecting abnormal data, based on the abnormal data, training sample data are obtained, and utilize
The training sample data carry out on-line training to the flow detection grader.
Possibly/optionally, the present embodiment is described based on the abnormal data, training sample data is obtained, such as Fig. 2 institutes
Show, including：
Step S301：Extract the valid data in the abnormal data；
Step S302：Mode excavation is carried out to the valid data of extraction；
Step S303：The data that mode excavation obtains are normalized, obtain training sample data.
Possibly/optionally, the present embodiment, when detecting abnormal data, the method further includes：Send out alarm.
Possibly/optionally, the present embodiment using deep learning algorithm or transfer learning algorithm, is examined the flow
It surveys grader and carries out on-line training.The transfer learning is that a kind of application has knowledge the problem of different but related field is carried out
The method of solution, it be suitable for tape label sample it is less even without situation.It can also be used for overcoming certain schemes only
The drawbacks of known type is abnormal can be detected, such as Traffic Anomaly caused by normal data and a kind of type flaw attack can be trained, is used
Exception in another source of detection.Therefore it can be reached using transfer learning algorithm and improve model generalization ability effect.
The specific embodiment of the present invention, classifier training process of the present invention use the convolution god in deep learning algorithm
Through network algorithm, it includes the neural net layers of three types:
Convolutional layer 5：The characteristic present of study identification input data.
Sample level 6：Also pond layer is, concrete operations and the operation of convolutional layer are essentially identical, only the volume of down-sampling
Product core is only takes maximum value, average value of corresponding position etc. (maximum pond, average pond), and repairing without backpropagation
Change.
Full articulamentum 7：Each unit of one layer of front is connected with below one layer.
The present invention is trained grader by building convolutional neural networks (CNN) model realization, the convolutional Neural
The whole network structure of network (CNN) model,
Convolutional layer 5：Enhance original signal feature, and reduce noise.Map number of convolutional layer is specified in netinit
, map, that is, characteristic pattern, and the size of the map of convolutional layer is determined by the size of convolution kernel and last layer input map, it is assumed that
The map sizes of last layer be n*n, convolution kernel size be k*k, then the map sizes of this layer are (n-k+1) * (n-k+1).
Sample level 6：Calculation amount can be reduced, while keep image rotation invariance by carrying out sub-sampling to image.It is to upper
A sampling processing of one layer of map, sample mode here is to carry out aggregate statistics, area to the neighboring community domain of last layer map
Domain size is scale*scale.
Full articulamentum 7：It is connected entirely using softmax, obtained activation value, output category result.
Such as：As shown in figure 3, input 28*28, the feature that 4 sizes are 24 * 24 is obtained after the processing of convolutional layer 5
Figure obtains the characteristic pattern that 4 sizes are 12*12 after the processing of sample level 6, again passes by the processing of convolutional layer 5 and obtains 12 greatly
Small is the characteristic pattern of 8*8, again passes by the processing of sample level 6 and obtains the characteristic pattern that 12 sizes are 4*4, finally by connecting entirely
It connects layer 7 and exports 26 1*1, the value of the parameter is the specific embodiment of the present invention, and the present invention is to the value of each parameter
It is not restricted.
As can be known from Fig. 3, each layer has multiple characteristic patterns, and each characteristic pattern passes through a kind of extraction input of convolution filter
A kind of feature, each characteristic pattern have multiple neurons.
In the second embodiment of the present invention, the present invention provides a kind of anomalous traffic detection method based on incremental learning：
The method is further included and is classified to the flow detection in addition to carrying out on-line training to the flow detection grader
Device carries out off-line training, as shown in figure 4, the specific method of the off-line training is as follows：
Step S01:Acquire historical traffic data；
Step S02:Based on the historical traffic data, history training sample data are obtained；To the historical accumulation sample of collection
Example sample carries out valid data (feature) and extracts, carries out mode excavation, the number obtained to mode excavation to the valid data of extraction
According to the series of preprocessing operation such as being normalized, history training sample is obtained.The processing side of the history training sample data
For method with being based on the abnormal data, the method for obtaining training sample data is identical.
Step S03：Using the history training sample data, off-line training is carried out to the flow detection grader, from
The newer flow detection grader real-time synchronization of line training is to the exception monitoring equipment.Using deep learning algorithm or migration
Learning algorithm carries out off-line training, is set by flow detection grader described in appropriate repetitive exercise, and with online abnormality detection
Flow detection grader real-time synchronization update in standby.
The specific method that on-line training is carried out to the flow detection grader is identical with the first embodiment, therefore herein no longer
It repeats.
The present embodiment will combine concrete application example, and the present embodiment the method is illustrated, it should be pointed out that this
A large amount of technical details disclosed in embodiment are not used to uniquely limit the present invention for explaining the present invention.
Here is the example of the anomalous traffic detection method of the present invention based on incremental learning, as shown in figure 5,
On-line training study is as follows：
Step S001：Obtain the data on flows of user terminal；The data on flows is that the data that production environment generates refer mainly to
Some datas on flows that user terminal generates during using abnormality detecting apparatus.
Step S002：The data on flows is carried out using the flow detection grader built in advance in abnormality detecting apparatus
Abnormality detection；
Step S003：When detecting abnormal data, based on the abnormal data, training sample data are obtained, and utilize
The training sample data carry out on-line training to the flow detection grader.
It is described to be based on the abnormal data, training sample data are obtained, including：
Step S301：The valid data in the abnormal data are extracted, to new samples data prediction, by sample flow number
According to from different protocol layers (network layer, transport layer, application layer) using natural language processing N-Gram models to every layer data into
Row extracts and (extracts valid data).
Step S302：Mode excavation is carried out to the valid data of extraction；Based on Aprior algorithms to the data that have extracted into
Row mode excavation come reduce invalid information and reduce calculation amount.
Step S303：The data that mode excavation obtains are normalized, i.e., data vectorization is handled, and is trained
Sample data.
It is described that on-line training is carried out to the flow detection grader using the training sample data, including：
Continue to use on original model and new model is formed based on deep learning algorithm optimization model, further, by structure
The new model real-time synchronization built is to off-line model.
Off-line training study is as follows：
Step S01：Historical traffic data is collected, collects the sample sample of historical accumulation；
Step S02:Based on the historical traffic data, history training sample data are obtained；First sample is labeled, so
Afterwards by forming initial training sample to data prediction；Preprocess method includes：By sample data on flows from different agreements
Layer is (i.e.：Network layer, transport layer, application layer) every layer data is extracted using natural language processing N-Gram models, it is described
It extracts as valid data, then mode excavation is carried out to reduce invalid information and reduction to the data extracted based on Aprior algorithms
Calculation amount finally carries out data vector processing.
Step S03：Using the history training sample data, off-line training is carried out to the flow detection grader, from
The newer flow detection grader real-time synchronization of line training is to the exception monitoring equipment.Using deep learning algorithm or migration
Learning algorithm carries out off-line training, by flow detection grader described in appropriate repetitive exercise.Entire off-line learning stage, data
It is ready under line, and new samples can be continuously increased and be trained, realizes offline incremental learning, each newer new mould
Type real-time synchronization is to exception monitoring equipment.
Online Increment Learning Algorithm proposed by the invention, combining classification device on-line training and grader off-line training lead to
After constantly increasing sample online, disaggregated model generalization ability can be effectively improved, table 1 below and table 2 are using difference respectively
The training corpus of scale carries out the accuracy rate result of open test.
1 incremental learning test result one of table
| Training data type (scale) | Test data type (scale) | Test result | |
| First group | Normal (20000)+SQL injection (20000) | Cross-site scripting attack (2000) | 57.83% |
| Second group | Normal (20000)+SQL injection (20000)+cross-site scripting attack (10000) | Cross-site scripting attack (2000) | 75.42% |
2 incremental learning test result two of table
| Training data type (scale) | Test data type (scale) | Test result | |
| First group | Normal (20000)+SQL injection (10000) | SQL injection (2000) | 67.63% |
| Second group | Normal (20000)+SQL injection (20000) | SQL injection (2000) | 73.12% |
First group of experimental data in table 1：Include the SQL injection flow number in normal normal discharges data, web attacks
According to two classification based trainings of progress and open test.Second group of experiment：It is increased on the basis of first group of experiment in web attacks
Cross-site scripting attack has carried out open test；Two groups of tests are tested mainly for cross site scripting data on flows, the results showed that
By being continuously increased training sample diversity, disaggregated model generalization ability can be improved.
First group of experimental data in table 2：Comprising the SQL injection data on flows in normal discharge data, web attacks, carry out
Two classification based trainings and open test.Second group of experiment：SQL injection sample in WEB attacks is increased on the basis of first group of experiment
This quantity carries out open test；Two groups of tests are tested mainly for SQL injection data on flows, the results showed that in certain item
By being continuously increased training samples number under part, the accuracy of disaggregated model can be improved.
In the third embodiment of the present invention, a kind of abnormal traffic detection device based on incremental learning is provided, including number
According to acquisition module 1 and online incremental learning module 2,
The data acquisition module 1, for obtaining the data on flows of user terminal；
The online incremental learning module 2, for utilizing the flow detection grader built in advance in abnormality detecting apparatus
The data on flows is carried out abnormality detection；When detecting abnormal data, based on the abnormal data, number of training is obtained
According to, and on-line training is carried out to the flow detection grader using the training sample data.The online incremental learning mould
Deep learning module is equipped in block 2, the deep learning module is used for using deep learning algorithm or transfer learning algorithm profit
On-line training is carried out to the flow detection grader with the training sample data.
Possibly/optionally, the present embodiment is described to be based on the abnormal data, obtains training sample data, including：
Extract the valid data in the abnormal data；
Mode excavation is carried out to the valid data of extraction；
The data that mode excavation obtains are normalized, obtain training sample data.
Possibly/optionally, the present embodiment, described device further includes：Alarm module 4, the alarm module 4 are used to work as
It detects to send out alarm during abnormal data.
In the fourth embodiment of the present invention, a kind of abnormal traffic detection device based on incremental learning, the dress are provided
It puts in addition to including data acquisition module 1 and online incremental learning module 2, further includes：Offline incremental learning module 3, such as Fig. 6 institutes
Show：
The data acquisition module 1 is additionally operable to acquisition historical traffic data；
The offline incremental learning module 3 for being based on the historical traffic data, obtains history training sample data；
Using the history training sample data, off-line training, the newer flow of off-line training are carried out to the flow detection grader
Grader real-time synchronization is detected to the exception monitoring equipment.It is described to be based on the historical traffic data, obtain history training sample
Notebook data includes：The historical accumulation sample sample of collection is carried out valid data (feature) extract, to the valid data of extraction into
Row mode excavation the series of preprocessing operation such as is normalized the data that mode excavation obtains, obtains history training sample.
Deep learning module is equipped in the offline incremental learning module 3, for using deep learning algorithm or migration
Learning algorithm carries out off-line training using the history training sample data to the flow detection grader, by appropriate iteration
The training flow detection grader.Deep learning module and the online increment in the offline incremental learning module 3
A deep learning module can be shared by practising the deep learning module of module 2.
The effect of the data acquisition module 1 and online incremental learning module 2 is identical with 3rd embodiment, therefore herein no longer
It repeats.
In the fifth embodiment of the present invention, a kind of abnormal traffic detection equipment based on incremental learning is provided, including：It deposits
Reservoir, processor and it is stored in the computer program that can be run on the memory and on the processor, the computer
The step of anomalous traffic detection method such as based on incremental learning is realized when program is performed by the processor.Specific steps are such as
Under：
Step S100：Obtain the data on flows of user terminal；
Step S200：The data on flows is carried out using the flow detection grader built in advance in abnormality detecting apparatus
Abnormality detection；
Step S300：When detecting abnormal data, based on the abnormal data, training sample data are obtained, and utilize
The training sample data carry out on-line training to the flow detection grader.
It is illustrated due to being sliced quick instantiation method to network in first, second embodiment, aunt's sheet
Details are not described herein for embodiment.
In the sixth embodiment of the present invention, a kind of computer readable storage medium, the computer-readable storage are provided
The abnormal traffic detection program based on incremental learning, the abnormal traffic detection program based on incremental learning are stored on medium
The step of anomalous traffic detection method such as based on incremental learning is realized when being executed by processor.It is as follows：
Step S100：Obtain the data on flows of user terminal；
Step S200：The data on flows is carried out using the flow detection grader built in advance in abnormality detecting apparatus
Abnormality detection；
Step S300：When detecting abnormal data, based on the abnormal data, training sample data are obtained, and utilize
The training sample data carry out on-line training to the flow detection grader.
It is illustrated due to being sliced quick instantiation method to network in first, second embodiment, aunt's sheet
Details are not described herein for embodiment.
In the present embodiment, the storage medium can include but is not limited to for：ROM, RAM, disk or CD etc..
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
God and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (10)
1. a kind of anomalous traffic detection method based on incremental learning, it is characterised in that：
Obtain the data on flows of user terminal；
The data on flows is carried out abnormality detection using the flow detection grader built in advance in abnormality detecting apparatus；
When detecting abnormal data, based on the abnormal data, training sample data are obtained, and utilize the number of training
On-line training is carried out according to the flow detection grader.
2. according to the method described in claim 1, it is characterized in that:It is described to be based on the abnormal data, obtain number of training
According to, including：
Extract the valid data in the abnormal data；
Mode excavation is carried out to the valid data of extraction；
The data that mode excavation obtains are normalized, obtain training sample data.
3. method according to claim 1 or 2, it is characterised in that:When detecting abnormal data, the method is also wrapped
It includes：Send out alarm.
4. according to the method described in claim 1, it is characterized in that：The method further includes：
Acquire historical traffic data；
Based on the historical traffic data, history training sample data are obtained；
Using the history training sample data, off-line training is carried out to the flow detection grader, off-line training is newer
Flow detection grader real-time synchronization is to the exception monitoring equipment.
5. the method according to claim 1 or 4, it is characterised in that：Using deep learning algorithm or transfer learning algorithm,
Online or off-line training is carried out to the flow detection grader.
6. a kind of abnormal traffic detection device based on incremental learning, it is characterised in that：Including data acquisition module and online increasing
Measure study module,
The data acquisition module, for obtaining the data on flows of user terminal；
The online incremental learning module, for using the flow detection grader built in advance in abnormality detecting apparatus to described
Data on flows carries out abnormality detection；
When detecting abnormal data, based on the abnormal data, training sample data are obtained, and utilize the number of training
On-line training is carried out according to the flow detection grader.
7. device according to claim 6, it is characterised in that:Described device further includes：Alarm module, the alarm mould
Block, for sending out alarm when detecting abnormal data.
8. the device described according to claim 6 or 7, it is characterised in that：Described device further includes：Offline incremental learning module,
The data acquisition module is additionally operable to acquisition historical traffic data；
The offline incremental learning module for being based on the historical traffic data, obtains history training sample data；Using institute
History training sample data are stated, off-line training, off-line training newer flow detection point are carried out to the flow detection grader
Class device real-time synchronization is to the exception monitoring equipment.
9. a kind of abnormal traffic detection equipment based on incremental learning, it is characterised in that：Including：Memory, processor and storage
On the memory and the computer program that can run on the processor, the computer program are held by the processor
The step of method as described in any one of claim 1 to 5 is realized during row.
10. a kind of computer readable storage medium, which is characterized in that be stored on the computer readable storage medium based on increasing
The abnormal traffic detection program of study is measured, the abnormal traffic detection program based on incremental learning is realized when being executed by processor
The step of method as described in any one of claim 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711363640.9A CN108173708A (en) | 2017-12-18 | 2017-12-18 | Anomalous traffic detection method, device and storage medium based on incremental learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711363640.9A CN108173708A (en) | 2017-12-18 | 2017-12-18 | Anomalous traffic detection method, device and storage medium based on incremental learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108173708A true CN108173708A (en) | 2018-06-15 |
Family
ID=62522268
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711363640.9A Pending CN108173708A (en) | 2017-12-18 | 2017-12-18 | Anomalous traffic detection method, device and storage medium based on incremental learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108173708A (en) |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109117941A (en) * | 2018-07-16 | 2019-01-01 | 北京思特奇信息技术股份有限公司 | Alarm prediction method, system, storage medium and computer equipment |
| CN109361658A (en) * | 2018-09-26 | 2019-02-19 | 杭州安恒信息技术股份有限公司 | Abnormal flow information storage means, device and electronic equipment based on industry control industry |
| CN109413028A (en) * | 2018-08-29 | 2019-03-01 | 集美大学 | SQL injection detection method based on convolutional neural networks algorithm |
| CN109462521A (en) * | 2018-11-26 | 2019-03-12 | 华北电力大学 | A kind of network flow abnormal detecting method suitable for source net load interaction industrial control system |
| CN109583904A (en) * | 2018-11-30 | 2019-04-05 | 深圳市腾讯计算机系统有限公司 | Training method, impaired operation detection method and the device of abnormal operation detection model |
| CN109597687A (en) * | 2018-10-31 | 2019-04-09 | 东软集团股份有限公司 | Data synchronous resource allocation methods, device, storage medium and electronic equipment |
| CN109639734A (en) * | 2019-01-24 | 2019-04-16 | 大连理工大学 | A kind of anomalous traffic detection method with computing resource adaptivity |
| CN109670307A (en) * | 2018-12-04 | 2019-04-23 | 成都知道创宇信息技术有限公司 | A kind of SQL injection recognition methods based on CNN and massive logs |
| CN109787958A (en) * | 2018-12-15 | 2019-05-21 | 深圳先进技术研究院 | Network flow real-time detection method and detection terminal, computer readable storage medium |
| CN109802868A (en) * | 2019-01-10 | 2019-05-24 | 中山大学 | A kind of mobile application real-time identification method based on cloud computing |
| CN109818976A (en) * | 2019-03-15 | 2019-05-28 | 杭州迪普科技股份有限公司 | A kind of anomalous traffic detection method and device |
| CN109934412A (en) * | 2019-03-18 | 2019-06-25 | 无锡雪浪数制科技有限公司 | Real-time device abnormal detector and method based on Time series forecasting model |
| CN110097037A (en) * | 2019-05-22 | 2019-08-06 | 天津联图科技有限公司 | Intelligent monitoring method, device, storage medium and electronic equipment |
| CN110224990A (en) * | 2019-07-17 | 2019-09-10 | 浙江大学 | A kind of intruding detection system based on software definition security architecture |
| CN110380922A (en) * | 2019-05-29 | 2019-10-25 | 兴业证券股份有限公司 | The full link stress test method and storage medium of transaction system |
| CN110830515A (en) * | 2019-12-13 | 2020-02-21 | 支付宝(杭州)信息技术有限公司 | Flow detection method and device and electronic equipment |
| CN111371742A (en) * | 2020-02-21 | 2020-07-03 | 重庆邮电大学 | SVDD (singular value decomposition and direct data decomposition) -based network slice physical node anomaly detection method |
| CN111435364A (en) * | 2019-01-14 | 2020-07-21 | 阿里巴巴集团控股有限公司 | Electronic medical record quality inspection method and device |
| CN111835541A (en) * | 2019-04-18 | 2020-10-27 | 华为技术有限公司 | Model aging detection method, device, equipment and system |
| CN112529204A (en) * | 2019-09-17 | 2021-03-19 | 华为技术有限公司 | Model training method, device and system |
| CN113259331A (en) * | 2021-04-29 | 2021-08-13 | 上海电力大学 | Unknown abnormal flow online detection method and system based on incremental learning |
| WO2022011977A1 (en) * | 2020-07-15 | 2022-01-20 | 中国科学院深圳先进技术研究院 | Network anomaly detection method and system, terminal and storage medium |
| CN114374605A (en) * | 2022-01-12 | 2022-04-19 | 重庆邮电大学 | Dynamic adjustment and migration method for service function chain in network slice scene |
| CN114465962A (en) * | 2019-09-16 | 2022-05-10 | 华为技术有限公司 | Data stream type identification method and related equipment |
| CN117879970A (en) * | 2024-02-23 | 2024-04-12 | 南京妙怀晶科技有限公司 | Network security protection method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070061882A1 (en) * | 2005-09-13 | 2007-03-15 | Honeywell International Inc. | Instance based learning framework for effective behavior profiling and anomaly intrusion detection |
| CN103745229A (en) * | 2013-12-31 | 2014-04-23 | 北京泰乐德信息技术有限公司 | Method and system of fault diagnosis of rail transit based on SVM (Support Vector Machine) |
| EP2784729A1 (en) * | 2013-03-25 | 2014-10-01 | Amadeus | Method and system for detecting anomaly in passenger flow |
| CN105577685A (en) * | 2016-01-25 | 2016-05-11 | 浙江海洋学院 | Intrusion detection independent analysis method and system in cloud calculation environment |
| CN106612289A (en) * | 2017-01-18 | 2017-05-03 | 中山大学 | Network collaborative abnormality detection method based on SDN |
| CN107391569A (en) * | 2017-06-16 | 2017-11-24 | 阿里巴巴集团控股有限公司 | Identification, model training, Risk Identification Method, device and the equipment of data type |
-
2017
- 2017-12-18 CN CN201711363640.9A patent/CN108173708A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070061882A1 (en) * | 2005-09-13 | 2007-03-15 | Honeywell International Inc. | Instance based learning framework for effective behavior profiling and anomaly intrusion detection |
| EP2784729A1 (en) * | 2013-03-25 | 2014-10-01 | Amadeus | Method and system for detecting anomaly in passenger flow |
| CN103745229A (en) * | 2013-12-31 | 2014-04-23 | 北京泰乐德信息技术有限公司 | Method and system of fault diagnosis of rail transit based on SVM (Support Vector Machine) |
| CN105577685A (en) * | 2016-01-25 | 2016-05-11 | 浙江海洋学院 | Intrusion detection independent analysis method and system in cloud calculation environment |
| CN106612289A (en) * | 2017-01-18 | 2017-05-03 | 中山大学 | Network collaborative abnormality detection method based on SDN |
| CN107391569A (en) * | 2017-06-16 | 2017-11-24 | 阿里巴巴集团控股有限公司 | Identification, model training, Risk Identification Method, device and the equipment of data type |
Cited By (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109117941A (en) * | 2018-07-16 | 2019-01-01 | 北京思特奇信息技术股份有限公司 | Alarm prediction method, system, storage medium and computer equipment |
| CN109413028A (en) * | 2018-08-29 | 2019-03-01 | 集美大学 | SQL injection detection method based on convolutional neural networks algorithm |
| CN109413028B (en) * | 2018-08-29 | 2021-11-30 | 集美大学 | SQL injection detection method based on convolutional neural network algorithm |
| CN109361658B (en) * | 2018-09-26 | 2021-04-23 | 杭州安恒信息技术股份有限公司 | Industrial control industry-based abnormal flow information storage method and device and electronic equipment |
| CN109361658A (en) * | 2018-09-26 | 2019-02-19 | 杭州安恒信息技术股份有限公司 | Abnormal flow information storage means, device and electronic equipment based on industry control industry |
| CN109597687A (en) * | 2018-10-31 | 2019-04-09 | 东软集团股份有限公司 | Data synchronous resource allocation methods, device, storage medium and electronic equipment |
| CN109462521B (en) * | 2018-11-26 | 2020-11-20 | 华北电力大学 | Network flow abnormity detection method suitable for source network load interaction industrial control system |
| CN109462521A (en) * | 2018-11-26 | 2019-03-12 | 华北电力大学 | A kind of network flow abnormal detecting method suitable for source net load interaction industrial control system |
| CN109583904B (en) * | 2018-11-30 | 2023-04-07 | 深圳市腾讯计算机系统有限公司 | Training method of abnormal operation detection model, abnormal operation detection method and device |
| CN109583904A (en) * | 2018-11-30 | 2019-04-05 | 深圳市腾讯计算机系统有限公司 | Training method, impaired operation detection method and the device of abnormal operation detection model |
| CN109670307A (en) * | 2018-12-04 | 2019-04-23 | 成都知道创宇信息技术有限公司 | A kind of SQL injection recognition methods based on CNN and massive logs |
| CN109787958A (en) * | 2018-12-15 | 2019-05-21 | 深圳先进技术研究院 | Network flow real-time detection method and detection terminal, computer readable storage medium |
| CN109787958B (en) * | 2018-12-15 | 2021-05-25 | 深圳先进技术研究院 | Network flow real-time detection method, detection terminal and computer readable storage medium |
| CN109802868A (en) * | 2019-01-10 | 2019-05-24 | 中山大学 | A kind of mobile application real-time identification method based on cloud computing |
| CN109802868B (en) * | 2019-01-10 | 2022-05-06 | 中山大学 | Mobile application real-time identification method based on cloud computing |
| CN111435364B (en) * | 2019-01-14 | 2023-04-18 | 阿里巴巴集团控股有限公司 | Electronic medical record quality inspection method and device |
| CN111435364A (en) * | 2019-01-14 | 2020-07-21 | 阿里巴巴集团控股有限公司 | Electronic medical record quality inspection method and device |
| CN109639734A (en) * | 2019-01-24 | 2019-04-16 | 大连理工大学 | A kind of anomalous traffic detection method with computing resource adaptivity |
| CN109818976B (en) * | 2019-03-15 | 2021-09-21 | 杭州迪普科技股份有限公司 | Abnormal flow detection method and device |
| CN109818976A (en) * | 2019-03-15 | 2019-05-28 | 杭州迪普科技股份有限公司 | A kind of anomalous traffic detection method and device |
| CN109934412A (en) * | 2019-03-18 | 2019-06-25 | 无锡雪浪数制科技有限公司 | Real-time device abnormal detector and method based on Time series forecasting model |
| CN111835541A (en) * | 2019-04-18 | 2020-10-27 | 华为技术有限公司 | Model aging detection method, device, equipment and system |
| CN110097037A (en) * | 2019-05-22 | 2019-08-06 | 天津联图科技有限公司 | Intelligent monitoring method, device, storage medium and electronic equipment |
| CN110380922A (en) * | 2019-05-29 | 2019-10-25 | 兴业证券股份有限公司 | The full link stress test method and storage medium of transaction system |
| CN110224990A (en) * | 2019-07-17 | 2019-09-10 | 浙江大学 | A kind of intruding detection system based on software definition security architecture |
| CN114465962A (en) * | 2019-09-16 | 2022-05-10 | 华为技术有限公司 | Data stream type identification method and related equipment |
| CN114465962B (en) * | 2019-09-16 | 2024-01-05 | 华为技术有限公司 | Data stream type identification method and related equipment |
| US11838215B2 (en) | 2019-09-16 | 2023-12-05 | Huawei Technologies Co., Ltd. | Data stream classification method and related device |
| CN112529204A (en) * | 2019-09-17 | 2021-03-19 | 华为技术有限公司 | Model training method, device and system |
| WO2021052394A1 (en) * | 2019-09-17 | 2021-03-25 | 华为技术有限公司 | Model training method, apparatus, and system |
| CN110830515A (en) * | 2019-12-13 | 2020-02-21 | 支付宝(杭州)信息技术有限公司 | Flow detection method and device and electronic equipment |
| CN111371742B (en) * | 2020-02-21 | 2022-04-29 | 重庆邮电大学 | SVDD (singular value decomposition and direct data decomposition) -based network slice physical node anomaly detection method |
| CN111371742A (en) * | 2020-02-21 | 2020-07-03 | 重庆邮电大学 | SVDD (singular value decomposition and direct data decomposition) -based network slice physical node anomaly detection method |
| WO2022011977A1 (en) * | 2020-07-15 | 2022-01-20 | 中国科学院深圳先进技术研究院 | Network anomaly detection method and system, terminal and storage medium |
| CN113259331A (en) * | 2021-04-29 | 2021-08-13 | 上海电力大学 | Unknown abnormal flow online detection method and system based on incremental learning |
| CN113259331B (en) * | 2021-04-29 | 2022-10-11 | 上海电力大学 | Unknown abnormal flow online detection method and system based on incremental learning |
| CN114374605A (en) * | 2022-01-12 | 2022-04-19 | 重庆邮电大学 | Dynamic adjustment and migration method for service function chain in network slice scene |
| CN114374605B (en) * | 2022-01-12 | 2024-01-05 | 西安盈科思泰网络技术有限公司 | Dynamic adjustment and migration method for service function chain in network slice scene |
| CN117879970A (en) * | 2024-02-23 | 2024-04-12 | 南京妙怀晶科技有限公司 | Network security protection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108173708A (en) | Anomalous traffic detection method, device and storage medium based on incremental learning | |
| CN111882446B (en) | Abnormal account detection method based on graph convolution network | |
| CN108509976A (en) | The identification device and method of animal | |
| CN108257114A (en) | A kind of transmission facility defect inspection method based on deep learning | |
| CN107480611A (en) | A kind of crack identification method based on deep learning convolutional neural networks | |
| CN104052612B (en) | A kind of Fault Identification of telecommunication service and the method and system of positioning | |
| CN108229580A (en) | Sugared net ranking of features device in a kind of eyeground figure based on attention mechanism and Fusion Features | |
| CN107909564A (en) | A kind of full convolutional network image crack detection method based on deep learning | |
| CN110288032B (en) | Vehicle driving track type detection method and device | |
| CN110197205A (en) | A kind of image-recognizing method of multiple features source residual error network | |
| CN105608446A (en) | Video stream abnormal event detection method and apparatus | |
| CN109272500A (en) | Fabric classification method based on adaptive convolutional neural networks | |
| CN106485528A (en) | The method and apparatus of detection data | |
| CN108562821B (en) | Method and system for determining single-phase earth fault line selection of power distribution network based on Softmax | |
| CN105653450A (en) | Software defect data feature selection method based on combination of modified genetic algorithm and Adaboost | |
| CN110134961A (en) | Processing method, device and the storage medium of text | |
| CN107133343A (en) | Big data abnormal state detection method and device based on time series approximate match | |
| CN109120632A (en) | Network flow abnormity detection method based on online feature selection | |
| CN110414780A (en) | A kind of financial transaction negative sample generation method based on generation confrontation network | |
| CN111833310B (en) | Surface defect classification method based on neural network architecture search | |
| CN108334943A (en) | The semi-supervised soft-measuring modeling method of industrial process based on Active Learning neural network model | |
| CN108628164A (en) | A kind of semi-supervised flexible measurement method of industrial process based on Recognition with Recurrent Neural Network model | |
| CN110263934A (en) | A kind of artificial intelligence data mask method and device | |
| CN112763215B (en) | Multi-working-condition online fault diagnosis method based on modular federal deep learning | |
| CN107145778A (en) | A kind of intrusion detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180615 |