CN108494746A - A kind of network port Traffic anomaly detection method and system - Google Patents
A kind of network port Traffic anomaly detection method and system Download PDFInfo
- Publication number
- CN108494746A CN108494746A CN201810187959.9A CN201810187959A CN108494746A CN 108494746 A CN108494746 A CN 108494746A CN 201810187959 A CN201810187959 A CN 201810187959A CN 108494746 A CN108494746 A CN 108494746A
- Authority
- CN
- China
- Prior art keywords
- port
- traffic
- domain name
- event
- flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a kind of network port Traffic anomaly detection method and system.This method is：1) the communications and liaison session log flow in target data platform is read out and is summarized according to source port number, destination slogan grouping, then counted the flow indicator data of each port, constitute the flow sequence of corresponding ports；2) according to the flow sequence per Single port, the input vector of the port is constituted, input LSTM networks obtain the traffic prediction value of port moment t；The traffic prediction value of port moment t is compared with observation；It imposes a condition if the two deviation is more than, it is determined that the Traffic Anomaly of the port；3) qualitative to the Traffic Anomaly progress of the port according to recent whole traffic logs of the port and preset rules, judge the Traffic Anomaly event of the port；If can not judge, the traffic log of extraction is inputted into trained machine learning model and is classified to the Traffic Anomaly of the port, identifies the Traffic Anomaly event of the port.
Description
Technical field
The present invention relates to the fields such as big data, network security, deep learning, are related to a kind of network port Traffic anomaly detection
Method and system, using the passive analysis means of wide area network flow to the network exception events such as DDoS, Botnet, viral transmission into
Row is found and portrait.
Background technology
The internet of today is faced with many security threats.For example, distributed denial of service (DDoS, Distributed
Denial of Service) it attacks and heavy losses was caused to the website of many organizations and equipment.DDoS refer to by means of
Client/server technology, multiple computers are joined together as Attack Platform, mobilize DDoS to attack one or more targets
It hits, to double up the power of Denial of Service attack.
Ddos attack is usually initiated by some Botnet.Botnet is made of the host for being infected bot program
A controllable network.Attacker sends out zombie host by order and control channel (C＆C, Command and Control)
Instruction is sent, to carry out the network attacks such as information stealth, Denial of Service attack and crime.Occur from late nineteen nineties in last century,
Botnet structure and form develop to the Fen BushiC ＆C based on P2P, used domain name from initial simple centralization C＆C
Then develop into domain name from initial fixation domain name to automatically generate (Domain Generation Automation).
In order to cope with the threat of Botnet, government, enterprise and scientific research institution, which are uniting, is detected Botnet
And strike.The important means of Botnet detection is that internet traffic is acquired and is analyzed, and note abnormalities feature, to sieve
Choosing and locking Botnet member, and then take attack action.It is stiff that discovery is not only facilitated to effective analysis of internet traffic
Corpse network members, while can also detect other malicious network traffics, such as malice domain name request, malicious file is propagated, malice chain
Receiving is asked and ddos attack.
Discovery to exception of network traffic is predicted dependent on to normal flowed fluctuation.If seen on a wide area network
The flow-rate fluctuation of each network port is examined, then whole flows can be regarded as to the time series of 65536 dimensions.The present invention experience be,
There is the periodicity of multiple frequency for this sequence, and there is also accidental fluctuations.Hostile network event is usually brought significantly
Flowed fluctuation.In order to which the time series to this complexity of network flow models, it may be considered that use Recognition with Recurrent Neural Network
(RNN)。
Long Short-Term Memory Neural Network (long in short-term Memory Neural Networks) abbreviation LSTM is
A kind of type that RNN is special, can learn long-term Dependency Specification.LSTM is carried by Hochreiter＆Schmidhuber (1997)
Go out, and is improved and promoted by Alex Graves in the recent period.Speech recognition, phonetic synthesis, hand-written disjunctor word identification,
The fields such as time series forecasting, image header generation, end-to-end machine translation, LSTM obtains quite huge success, and obtains
Extensive use is arrived.LSTM causes the gradient in neural network training process to be moved back by design deliberately to avoid relying on for a long time
Change and dissipates, feature of the energy serialized data in context.
Invention content
The present invention proposes a kind of network port Traffic anomaly detection method and system, is named as cPortMon, for real-time
It was found that network port Traffic Anomaly.CPortMon is the subsystem of cNetS network security monitoring analysis systems.CNetS systems exist
A large amount of backbone network entrance on-premise network flow collection probes, and it is stored in basic data big data platform.
The technical scheme is that：
A kind of network port Traffic anomaly detection method, step include：
1) the communications and liaison session log flow in target data platform is read out and according to source port number, destination slogan
Grouping summarizes, and then counts the flow indicator data of each port, constitutes the flow sequence of corresponding ports；
2) according to the flow sequence per Single port, the input vector of the port is constituted, input LSTM networks obtain the port
The traffic prediction value of moment t；The traffic prediction value of port moment t and the observation of port moment t are compared；If
The two deviation, which is more than, to impose a condition, it is determined that the Traffic Anomaly of the port；
3) for the port of Traffic Anomaly, discovery module is threatened to extract the recent whole of the port from the target data platform
Traffic log, it is qualitative to the Traffic Anomaly progress of the port according to the traffic log of extraction and preset rules, judge the port
Traffic Anomaly event；If can not be qualitative to the progress of the Traffic Anomaly of the port according to preset rules, by the flow of extraction
Daily record inputs trained machine learning model and classifies to the Traffic Anomaly of the port, identifies the Traffic Anomaly of the port
Event.
Further, the setting condition is：
Wherein, o (t) is the observation of port moment t, and p (t) is the predicted value of port moment t；O (τ) is the observation of port time instant τ,
P (τ) is the predicted value of port time instant τ, and T is observation cycle length, and m is natural number, k1,k2For proportionality coefficient.
Further, k1,k2Value be 2.
Further, threaten discovery module judge the port Traffic Anomaly event whether be Botnet event side
Method is：When occurring individual host in the traffic log of extraction and initiating list SYN packets for the identical port of a large amount of hosts and connect, then recognize
Constant flow source is scanning source；When occurring being directed to the scanning source of identical network port on a large scale, assert that Traffic Anomaly event is to live
Jump Botnet event.
Further, the method for determining the corpse machine in Botnet event is：Discovery module will be threatened to regard as sweeping
The host for retouching source is determined as corpse machine, and the host for periodically or quasi-periodically initiating request to same non-known domain name is determined as
Corpse machine；The non-known domain name is the domain name except the known domain name list of setting.
Further, judge whether a traffic sources periodically or quasi-periodically initiate same non-known domain name the side of request
Method is：The request analysis set of domains of the traffic sources is obtained, and filters out known domain name；Then in request analysis set of domains
Remaining each domain name executes following operation：
If 61) analysis request sequence of events number N of the traffic sources to current domain name ddLess than threshold value k5, then ignore to domain
All analysis request events of name d, terminate the processing to domain name d；Otherwise, it enters step 62)；
62) all intervals are directed to, the traffic sources gather all analysis request events of domain name d with DBSCAN algorithms
Class is gathered analysis request spacing value is identical for one kind, if a cluster result C, is met | C |>k6Nd, then assert the traffic sources
Request periodically is initiated to domain name d, it is the domain name request period to take the mean value u of the analysis request spacing value in cluster result C, into
Enter step 64)；Otherwise, it enters step 63)；k6Value is 0.9~0.98；
63) in the event of multiple cluster result Ci, i=1,2 ... Nc, between the analysis request in these cluster results
It is denoted as u every the mean value of valuei；Take umin=minuiIf each uiBe approximately or be equal to uminMultiple；Then assert traffic sources standard
Request, period u periodically are initiated to domain name dmin；Otherwise terminate the processing to domain name d；
64) judge that the traffic sources periodically or quasi-periodically initiate request to domain name d, domain name d is master control domain name.
A kind of network port Traffic anomaly detection system, which is characterized in that including flow collection module, flow cytometer showed mould
Block, volume forecasting module, abnormal judgment module and threat discovery module；Wherein,
Flow collection module, for acquiring the communications and liaison session log flow in target data platform；
Flow cytometer showed module, for being carried out to the communications and liaison session log flow of acquisition according to source port number, destination slogan
Grouping summarizes, and then counts the flow indicator data of each port, constitutes the flow sequence of corresponding ports；
Volume forecasting module inputs LSTM for according to the flow sequence per Single port, constituting the input vector of the port
Network obtains the traffic prediction value of port moment t；
Abnormal judgment module, for carrying out pair the observation of the traffic prediction value of port moment t and port moment t
Than imposing a condition if the two deviation is more than, it is determined that the Traffic Anomaly of the port；
Discovery module is threatened, for the port for Traffic Anomaly, the recent of the port is extracted from the target data platform
Whole traffic logs, it is qualitative to the Traffic Anomaly progress of the port according to the traffic log of extraction and preset rules, judge this
The Traffic Anomaly event of port；If can not be qualitative to the progress of the Traffic Anomaly of the port according to preset rules, by extraction
Traffic log inputs trained machine learning model and classifies to the Traffic Anomaly of the port, identifies the flow of the port
Anomalous event.
CPortMon structures are as shown in Figure 1.The Realtime Analysis of cPortMon are to the communications and liaison in big data platform
Session log flow is read out, and is summarized respectively according to source port number, destination slogan grouping, is counted with predetermined period length T
The flow indicators such as number, byte number are connected, flow sequence is constituted.It is recommended that T is set as 1 hour length.
Long-term observation and record are carried out to the flow sequence of each port all directions, constitute observation sequence for existing for a long time
Line training LSTM networks：
O=o (t), t=O, T, 2T ...
LSTM is based on history observation, and the prediction value sequence generated to each port flow from moment mT is：
P=p (t)=p (o (t-T), o (t-2T) ...), t=mT, (m+1) T ...
It is recommended that mT is more than a week period length.As the observation o (t) of moment t port is much larger than corresponding predicted value p
(t), then port flow anomalous event is alerted:
Threshold value k1,k2Control exception-triggered condition, it is proposed that be set as 2.
Exceptional condition is triggered under two situations with above-mentioned formula, the prediction error of LSTM networks itself has been carried out centainly
Tolerance, can reduce prediction error caused by judge by accident.
Other modules of cNetS also play key player.CNetS overall architectures are as shown in Figure 2.CNetS uses Mon-
Mine frameworks carry out module design.Mon generic modules monitor disparate networks entity based on initial data real-time, Mine
The exception that generic module is then found based on Mon modules, carries out deep excavation.In addition to cPortMon, other Mon generic modules are such as
CHostMon, cNameMon, cLinkMon etc. are respectively from IP, domain name, and URL is angularly monitored network flow, and in real time
It was found that corresponding Traffic Anomaly event.After the anomalous event that all kinds of Mon modules are found, related data can submit to threat and find
Module, for further verifying the presence with qualitative hostile network event.It is different according to the classification of malicious event, these events
Related data can be dispatched to multiple modules of tracing to the source of excavating and execute further information excavation again.As Botnet event can be assigned to
CBotMine, malicious file communication events distribute to cMalMon, and ddos attack event distributes to cDoSMon.Threaten information bank to
Discovery module is threatened to provide intelligence supports, and all kinds of Mine modules then provide intelligence feedback to threat information bank.In addition, threatening feelings
Also it supports to import information from external data source in report library.
After cPortMon has found port flow exception, discovery module is threatened to extract related port from data management platform
Whole traffic log in the recent period carries out event qualitative.Qualitatively strategy first submits preset rules to judge, can not judge, can carry
Trained machine learning model (Stochastic Decision-making forest) is classified for friendship.Threaten discovery module structure as shown in Figure 3.
For Botnet, the present invention proposes following rule：
When individual host occur and initiating list SYN packets for the identical port of a large amount of hosts and connect, flow can be directly assert
Source (individual host for initiating the identical port of a large amount of hosts the connection of list SYN packets) is scanning source.
When the scanning source for occurring being directed to identical network port is more than given threshold, regard as enlivening Botnet propagation
Event.
The loophole for relating to the port in information bank is threatened, and known Botnet utilizes the record of this loophole, then will
Botnet communication events correspond to specific known Botnet.
CBotMine is responsible for the further analysis relevant Traffic Anomaly event of Botnet, and performed task includes Bot
Detection, Bot portraits and master control are traced to the source, as shown in Figure 4.
Bot detection functions extract corpse machine list from abnormal flow.Mainly pass through following two criterions：
The host in scanning source is regarded as by threat discovery module；
The host of request is periodically or quasi-periodically initiated same non-known domain name.
Periodic regularity finds that algorithm acts on the parsing time interval sequence to same domain name.For asking for the traffic sources
Analysis set of domains is solved, filters known domain name with Alexa10000, and following operation is executed to remaining each domain name：
1. if to the analysis request sequence of events number N of current domain name ddLess than threshold value k5, then ignore and own to this domain name
Analysis request event, exits calculating process.Otherwise, 2 are entered step.It is recommended that k5Value is between 50 to 100.
2. being directed to all intervals, clustered with DBSCAN algorithms, range error is set 1 minute.If most spacing values are poly-
For a kind of C so that | C |>k6Nd, then assert periodicity, it is the domain name request period to take such mean value u, enters step 4.Otherwise, into
Enter step 3.It is recommended that k6Value is between 0.9 to 0.98.
3. in the event of multiple apparent class Ci, i=1,2 ... Nc, the mean value of these classes is denoted as ui.Take umin=
minuiIf each uiIt is approximately uminMultiple, i.e., to arbitrary uiIt is satisfied by one of following two condition：
「ui/umin」-ui/umin＜ ∈
Also assert periodically, be uminFor the period, and for caused by primitive event shortage of data the phenomenon that lack leading class, into
Enter step 4.If being unsatisfactory for assert periodic condition, calculating is exited.It is recommended that ∈ value ranges are below 0.1.
4. assert the entitled master control domain name of the current field, threat information bank is submitted to.In addition, can detect whether the domain name is DGA
Domain name, not within the scope of being discussed herein.
Bot portrait modules calculate the Bot that Bot detection modules are found as follows：
1. sending out the disease time to find.The time of origin of abnormal behaviour is analyzed, determines that earliest time is infection time.
2. health status is drawn a portrait.Before sending out the disease time, the communication ownership place distribution of the host, protocol type point are counted
Cloth, local known port frequency distribution, the features such as long-range known port frequency distribution.
3. the source of infection traces.It is assumed that infection time is close with the hair disease time, without incubation period, then in the premise of hair disease time
Take all events in the time window that length is 5 minutes.For port intrusive mood Botnet, investigation corresponding port is scanned
Event and the event that is successfully connected assert that successfully connection source is the source of infection.For other kinds of Botnet, in the time window
Detection does not meet the session of health status in mouthful, using far-end IP as doubtful scanning source.
4. applying harmful behavior retrospect.For port intrusive mood Botnet, its externally scanning rule is excavated, finds successfully end
Mouth scan event, is added and increases victim's list newly.
5. suspected infection source and victim's list feed back to threat information bank.
Master control traces to the source mould based on known Bot progress communications and liaison relationship calculating, attempts to find upper level master control.Utilize following standard
Then：
The master control domain name found in Bot detection modules, analytic value regard as master control IP address.If master cannot be passed through
Dynamic parsing and passive monitoring mode find master control domain name mapping value, then observe the connection that current Bot is initiated after analysis request
Address.
If multiple Bot recognize to more generation communications and liaison relationships of the same non-known port Pc of the same host Hc
Determine Hc:Pc is master control address.
If it is known that the same non-known port Pc of the same host Hc communications and liaison relationship periodically occurs for Bot.Its
In, periodic regularity finds the corresponding realization in algorithm multiplexing Bot detection modules.
Compared with prior art, the positive effect of the present invention is：
Research about abnormal flow excavation is mostly towards enterprise's Intranet environment, and the research of the present invention is then directed to extensively
Domain net environment.A small number of organizations such as only large-scale operator, CERT can carry out flow analysis monitoring under wan environment.
In such a scenario, the statistical nature of flow tends to be apparent, and contingency event then being smoothed contributes to the hair of anomalous event
It is existing.
The present invention has many advantages：
1. the present invention is not caused the discovery of new threat by signature value matching operation, therefore it can be found that unknown threat.
2. the present invention is entirely the mode passively observed to the analysis of flow, internet will not be interfered this province, it is right
Botnet is invisible.
3. the method for the present invention can relatively easily expand on more massive cluster, to realize the prison to greater flow
It surveys.
4.cNetS uses Mon-Mine separate type frameworks, and each module is clear in job responsibility, is conducive to engineering developme, maintenance and liter
Grade.Mon generic modules are suitable for using simple and quick Stream Processing task, Mine generic modules to be applicable in complicated offline mining task,
Not only it ensure that system throughput performance, but also complicated algorithm supported to realize.
Description of the drawings
Fig. 1 is cPortMon module diagrams；
Fig. 2 is cNetS integrated stand compositions；
Fig. 3 is to threaten discovery module execution flow chart；
Fig. 4 is cBotMine module diagrams.
Specific implementation mode
In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, in conjunction with attached drawing and specific embodiment party
The present invention is described in further detail for formula.
The flow collection module of cNetS is realized using high-performance server, loads polylith 10Ge network interface cards, runs DPDK frames
Realize high speed flow collection.Network flow is exported in backbone network router, is shunted and is introduced by mirror-image fashion.Flow collection mould
Flow is aggregated into NetFlow abstract formats and exports and give cPortMon and cHostMon modules by block, for DNS response bag derived fields
The abstract fields such as name, sourcesink IP, timestamp, which export, gives cNameMon modules, is plucked for HTTP request packet export URL, sourcesink IP etc.
It wants information to export and gives cLinkMon modules.
Abstract flow is transferred to each Mon generic modules by Apache Kafka.Each Mon generic modules were both available
SparkStreaming handles data on flows in real time, also can simultaneously store the output copy of Kafka to Hadoop platform, and
Later stage carries out data access by Hive, executes offline mining task.
The real-time traffic processing routine of cPortMon add up each port day part flow, and after day part will
Summarized results is stored.The offline excavation program of cPortMon reads day part summarized results.Except the stream in nearest 24 periods
It measures outside summarized results, time series is built to the flow of each port, and nearest using the progress volume forecasting of TensorFlow frames
The flow in 24 periods.Predicted flow rate compares with the flow value in nearest 24 periods, if deviation is excessive, is determined as Traffic Anomaly
Event is submitted to and discovery module is threatened to be handled.
It is above to implement to be merely illustrative of the technical solution of the present invention rather than be limited, the ordinary skill people of this field
Member can be modified or replaced equivalently technical scheme of the present invention, without departing from the spirit and scope of the present invention, this hair
Bright protection domain should be subject to described in claims.
Claims (10)
1. a kind of network port Traffic anomaly detection method, step include：
1) the communications and liaison session log flow in target data platform is read out and is grouped according to source port number, destination slogan
Summarize, then count the flow indicator data of each port, constitutes the flow sequence of corresponding ports；
2) according to the flow sequence per Single port, the input vector of the port is constituted, input LSTM networks obtain port moment t
Traffic prediction value；The traffic prediction value of port moment t and the observation of port moment t are compared；If the two
Deviation, which is more than, to impose a condition, it is determined that the Traffic Anomaly of the port；
3) for the port of Traffic Anomaly, discovery module is threatened to extract recent whole flows of the port from the target data platform
Daily record, it is qualitative to the Traffic Anomaly progress of the port according to the traffic log of extraction and preset rules, judge the stream of the port
Measure anomalous event；If can not be qualitative to the progress of the Traffic Anomaly of the port according to preset rules, by the traffic log of extraction
It inputs trained machine learning model to classify to the Traffic Anomaly of the port, identifies the Traffic Anomaly thing of the port
Part.
2. the method as described in claim 1, which is characterized in that the setting condition is： Wherein, o (t) is the observation of port moment t, and p (t) is port moment t's
Predicted value；O (τ) is the observation of port time instant τ, and p (τ) is the predicted value of port time instant τ, and T is observation cycle length, and m is certainly
So number, k1,k2For proportionality coefficient.
3. method as claimed in claim 2, which is characterized in that k1,k2Value be 2.
4. the method as described in claim 1, which is characterized in that discovery module is threatened to judge the Traffic Anomaly event of the port
Whether it is that the method for Botnet event is：It is directed to the identical port of a large amount of hosts when occurring individual host in the traffic log of extraction
When initiating the connection of list SYN packets, then assert that traffic sources are scanning source；It is more than setting when there is the scanning source for identical network port
When threshold value, assert that Traffic Anomaly event is to enliven Botnet event.
5. method as claimed in claim 4, which is characterized in that the method for determining the corpse machine in Botnet event is：
The host for threatening discovery module to regard as scanning source is determined as corpse machine, it will be periodically or quasi-periodically to same non-known domain
The host that name initiates request is determined as corpse machine；The non-known domain name is the domain name except the known domain name list of setting.
6. method as claimed in claim 5, which is characterized in that judge a traffic sources whether periodically or quasi-periodically to same
The method that non-known domain name initiates request is：The request analysis set of domains of the traffic sources is obtained, and filters out known domain name；So
Following operation is executed to remaining each domain name in request analysis set of domains afterwards：
If 61) analysis request sequence of events number N of the traffic sources to current domain name ddLess than threshold value k5, then ignore to domain name d's
All analysis request events terminate the processing to domain name d；Otherwise, it enters step 62)；
62) all intervals are directed to, the traffic sources cluster all analysis request events of domain name d with DBSCAN algorithms,
Gather analysis request spacing value is identical for one kind, if a cluster result C, meet | C |>k6Nd, then assert the traffic sources period
Property request is initiated to domain name d, it is the domain name request period to take the mean value u of the analysis request spacing value in cluster result C, into step
It is rapid 64)；Otherwise, it enters step 63)；k6Value is 0.9~0.98；
63) in the event of multiple cluster result Ci, i=1,2 ... Nc, the analysis request spacing value in these cluster results
Mean value be denoted as ui；Take umin=minuiIf each uiBe approximately or be equal to uminMultiple；Then assert traffic sources paracycles
Property request, period u are initiated to domain name dmin；Otherwise terminate the processing to domain name d；
64) judge that the traffic sources periodically or quasi-periodically initiate request to domain name d, domain name d is master control domain name.
7. a kind of network port Traffic anomaly detection system, which is characterized in that including flow collection module, flow cytometer showed module,
Volume forecasting module, abnormal judgment module and threat discovery module；Wherein,
Flow collection module, for acquiring the communications and liaison session log flow in target data platform；
Flow cytometer showed module, for being grouped to the communications and liaison session log flow of acquisition according to source port number, destination slogan
Summarize, then count the flow indicator data of each port, constitutes the flow sequence of corresponding ports；
Volume forecasting module inputs LSTM networks for according to the flow sequence per Single port, constituting the input vector of the port
Obtain the traffic prediction value of port moment t；
Abnormal judgment module, for comparing the traffic prediction value of port moment t and the observation of port moment t, such as
Both fruits deviation, which is more than, to impose a condition, it is determined that the Traffic Anomaly of the port；
Discovery module is threatened, for the port for Traffic Anomaly, the recent whole of the port is extracted from the target data platform
Traffic log, it is qualitative to the Traffic Anomaly progress of the port according to the traffic log of extraction and preset rules, judge the port
Traffic Anomaly event；If can not be qualitative to the progress of the Traffic Anomaly of the port according to preset rules, by the flow of extraction
Daily record inputs trained machine learning model and classifies to the Traffic Anomaly of the port, identifies the Traffic Anomaly of the port
Event.
8. system as claimed in claim 7, which is characterized in that the setting condition is： Wherein, o (t) is the observation of port moment t, and p (t) is port moment t's
Predicted value；O (τ) is the observation of port time instant τ, and p (τ) is the predicted value of port time instant τ, and T is observation cycle length, and m is certainly
So number, k1,k2For proportionality coefficient.
9. system as claimed in claim 7, which is characterized in that discovery module is threatened to judge the Traffic Anomaly event of the port
Whether it is that the method for Botnet event is：It is directed to the identical port of a large amount of hosts when occurring individual host in the traffic log of extraction
When initiating the connection of list SYN packets, then assert that traffic sources are scanning source；It is more than setting when there is the scanning source for identical network port
When threshold value, assert that Traffic Anomaly event is to enliven Botnet event.
10. system as claimed in claim 9, which is characterized in that further include a Bot detection modules, for determining corpse net
Corpse machine in network event, method are：The host for threatening discovery module to regard as scanning source is determined as corpse machine, by the period
Property or quasi periodic request is initiated to same non-known domain name host be determined as corpse machine；The non-known domain name is setting
Domain name except known domain name list.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810187959.9A CN108494746B (en) | 2018-03-07 | 2018-03-07 | Method and system for detecting abnormal flow of network port |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810187959.9A CN108494746B (en) | 2018-03-07 | 2018-03-07 | Method and system for detecting abnormal flow of network port |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108494746A true CN108494746A (en) | 2018-09-04 |
| CN108494746B CN108494746B (en) | 2020-08-25 |
Family
ID=63341847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810187959.9A Active CN108494746B (en) | 2018-03-07 | 2018-03-07 | Method and system for detecting abnormal flow of network port |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108494746B (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109582555A (en) * | 2018-12-04 | 2019-04-05 | 北京锐安科技有限公司 | Data exception detection method, device, detection system and storage medium |
| CN109768995A (en) * | 2019-03-06 | 2019-05-17 | 国网甘肃省电力公司电力科学研究院 | A kind of network flow abnormal detecting method based on circular prediction and study |
| CN109800782A (en) * | 2018-12-11 | 2019-05-24 | 国网甘肃省电力公司金昌供电公司 | A kind of electric network fault detection method and device based on fuzzy knn algorithm |
| CN109995592A (en) * | 2019-04-09 | 2019-07-09 | 中国联合网络通信集团有限公司 | Quality of service monitoring method and equipment |
| CN110040107A (en) * | 2019-03-18 | 2019-07-23 | 百度在线网络技术（北京）有限公司 | Vehicle intrusion detection and prediction model training method, device and storage medium |
| CN110149343A (en) * | 2019-05-31 | 2019-08-20 | 国家计算机网络与信息安全管理中心 | A kind of abnormal communications and liaison behavioral value method and system based on stream |
| CN110493253A (en) * | 2019-09-02 | 2019-11-22 | 四川长虹电器股份有限公司 | A kind of Botnet analysis method of the home router based on raspberry pie design |
| CN110519290A (en) * | 2019-09-03 | 2019-11-29 | 南京中孚信息技术有限公司 | Anomalous traffic detection method, device and electronic equipment |
| CN110730175A (en) * | 2019-10-16 | 2020-01-24 | 杭州安恒信息技术股份有限公司 | Botnet detection method and detection system based on threat information |
| TWI684889B (en) * | 2018-10-04 | 2020-02-11 | 安碁資訊股份有限公司 | Method for evaluating domain name and server using the same |
| CN111092852A (en) * | 2019-10-16 | 2020-05-01 | 平安科技（深圳）有限公司 | Network security monitoring method, device, equipment and storage medium based on big data |
| CN111209163A (en) * | 2020-01-03 | 2020-05-29 | 中国工商银行股份有限公司 | Application system anomaly detection method and system |
| CN111224924A (en) * | 2018-11-27 | 2020-06-02 | 北京金山云网络技术有限公司 | Traffic processing method and device, electronic equipment and storage medium |
| CN111343136A (en) * | 2018-12-19 | 2020-06-26 | 福建雷盾信息安全有限公司 | Network abnormal behavior analysis and detection method based on flow behavior characteristics |
| CN111818097A (en) * | 2020-09-01 | 2020-10-23 | 北京安帝科技有限公司 | Traffic monitoring method and device based on behaviors |
| CN111935064A (en) * | 2020-05-28 | 2020-11-13 | 南京南瑞信息通信科技有限公司 | Industrial control network threat automatic isolation method and system |
| CN112073355A (en) * | 2019-05-25 | 2020-12-11 | 福建雷盾信息安全有限公司 | Vulnerability analysis method based on network flow |
| CN113572653A (en) * | 2020-04-29 | 2021-10-29 | 华为技术有限公司 | Method, device and equipment for obtaining flow prediction range and storage medium |
| US11290329B2 (en) | 2020-04-30 | 2022-03-29 | Hewlett Packard Enterprise Development Lp | Configuring a network based on a centroid configuration of a group of network entities |
| CN114928560A (en) * | 2022-05-16 | 2022-08-19 | 珠海市鸿瑞信息技术股份有限公司 | Big data based network flow and equipment log cooperative management system and method |
| CN115952465A (en) * | 2023-03-10 | 2023-04-11 | 畅捷通信息技术股份有限公司 | Sensor data anomaly detection method and device and computer storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060042788A (en) * | 2004-11-10 | 2006-05-15 | 한국전자통신연구원 | Method for analyzing security condition by representing network events in graphs and apparatus thereof |
| CN104486324A (en) * | 2014-12-10 | 2015-04-01 | 北京百度网讯科技有限公司 | Method and system for identifying network attack |
| CN106453392A (en) * | 2016-11-14 | 2017-02-22 | 中国人民解放军防空兵学院 | Whole-network abnormal flow identification method based on flow characteristic distribution |
| CN106789297A (en) * | 2016-12-29 | 2017-05-31 | 淮海工学院 | Predicting network flow system and its method for predicting based on neutral net |
| CN107592312A (en) * | 2017-09-18 | 2018-01-16 | 济南互信软件有限公司 | A kind of malware detection method based on network traffics |
-
2018
- 2018-03-07 CN CN201810187959.9A patent/CN108494746B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060042788A (en) * | 2004-11-10 | 2006-05-15 | 한국전자통신연구원 | Method for analyzing security condition by representing network events in graphs and apparatus thereof |
| CN104486324A (en) * | 2014-12-10 | 2015-04-01 | 北京百度网讯科技有限公司 | Method and system for identifying network attack |
| CN106453392A (en) * | 2016-11-14 | 2017-02-22 | 中国人民解放军防空兵学院 | Whole-network abnormal flow identification method based on flow characteristic distribution |
| CN106789297A (en) * | 2016-12-29 | 2017-05-31 | 淮海工学院 | Predicting network flow system and its method for predicting based on neutral net |
| CN107592312A (en) * | 2017-09-18 | 2018-01-16 | 济南互信软件有限公司 | A kind of malware detection method based on network traffics |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11095672B2 (en) | 2018-10-04 | 2021-08-17 | Acer Cyber Security Incorporated | Method for evaluating domain name and server using the same |
| TWI684889B (en) * | 2018-10-04 | 2020-02-11 | 安碁資訊股份有限公司 | Method for evaluating domain name and server using the same |
| CN111224924A (en) * | 2018-11-27 | 2020-06-02 | 北京金山云网络技术有限公司 | Traffic processing method and device, electronic equipment and storage medium |
| CN109582555A (en) * | 2018-12-04 | 2019-04-05 | 北京锐安科技有限公司 | Data exception detection method, device, detection system and storage medium |
| CN109800782A (en) * | 2018-12-11 | 2019-05-24 | 国网甘肃省电力公司金昌供电公司 | A kind of electric network fault detection method and device based on fuzzy knn algorithm |
| CN111343136A (en) * | 2018-12-19 | 2020-06-26 | 福建雷盾信息安全有限公司 | Network abnormal behavior analysis and detection method based on flow behavior characteristics |
| CN109768995A (en) * | 2019-03-06 | 2019-05-17 | 国网甘肃省电力公司电力科学研究院 | A kind of network flow abnormal detecting method based on circular prediction and study |
| CN109768995B (en) * | 2019-03-06 | 2021-08-13 | 国网甘肃省电力公司电力科学研究院 | Network flow abnormity detection method based on cyclic prediction and learning |
| CN110040107A (en) * | 2019-03-18 | 2019-07-23 | 百度在线网络技术（北京）有限公司 | Vehicle intrusion detection and prediction model training method, device and storage medium |
| CN109995592A (en) * | 2019-04-09 | 2019-07-09 | 中国联合网络通信集团有限公司 | Quality of service monitoring method and equipment |
| CN112073355A (en) * | 2019-05-25 | 2020-12-11 | 福建雷盾信息安全有限公司 | Vulnerability analysis method based on network flow |
| CN110149343A (en) * | 2019-05-31 | 2019-08-20 | 国家计算机网络与信息安全管理中心 | A kind of abnormal communications and liaison behavioral value method and system based on stream |
| CN110149343B (en) * | 2019-05-31 | 2021-07-16 | 国家计算机网络与信息安全管理中心 | Abnormal communication behavior detection method and system based on flow |
| CN110493253B (en) * | 2019-09-02 | 2021-06-22 | 四川长虹电器股份有限公司 | Botnet analysis method of home router based on raspberry group design |
| CN110493253A (en) * | 2019-09-02 | 2019-11-22 | 四川长虹电器股份有限公司 | A kind of Botnet analysis method of the home router based on raspberry pie design |
| CN110519290A (en) * | 2019-09-03 | 2019-11-29 | 南京中孚信息技术有限公司 | Anomalous traffic detection method, device and electronic equipment |
| CN111092852A (en) * | 2019-10-16 | 2020-05-01 | 平安科技（深圳）有限公司 | Network security monitoring method, device, equipment and storage medium based on big data |
| CN110730175B (en) * | 2019-10-16 | 2022-12-06 | 杭州安恒信息技术股份有限公司 | Botnet detection method and detection system based on threat information |
| CN110730175A (en) * | 2019-10-16 | 2020-01-24 | 杭州安恒信息技术股份有限公司 | Botnet detection method and detection system based on threat information |
| CN111209163A (en) * | 2020-01-03 | 2020-05-29 | 中国工商银行股份有限公司 | Application system anomaly detection method and system |
| CN113572653A (en) * | 2020-04-29 | 2021-10-29 | 华为技术有限公司 | Method, device and equipment for obtaining flow prediction range and storage medium |
| CN113572653B (en) * | 2020-04-29 | 2023-03-21 | 华为技术有限公司 | Method, device and equipment for obtaining flow prediction range and storage medium |
| US11290329B2 (en) | 2020-04-30 | 2022-03-29 | Hewlett Packard Enterprise Development Lp | Configuring a network based on a centroid configuration of a group of network entities |
| CN111935064A (en) * | 2020-05-28 | 2020-11-13 | 南京南瑞信息通信科技有限公司 | Industrial control network threat automatic isolation method and system |
| CN111818097A (en) * | 2020-09-01 | 2020-10-23 | 北京安帝科技有限公司 | Traffic monitoring method and device based on behaviors |
| CN114928560A (en) * | 2022-05-16 | 2022-08-19 | 珠海市鸿瑞信息技术股份有限公司 | Big data based network flow and equipment log cooperative management system and method |
| CN114928560B (en) * | 2022-05-16 | 2023-01-31 | 珠海市鸿瑞信息技术股份有限公司 | Big data based network flow and equipment log cooperative management system and method |
| CN115952465A (en) * | 2023-03-10 | 2023-04-11 | 畅捷通信息技术股份有限公司 | Sensor data anomaly detection method and device and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108494746B (en) | 2020-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108494746A (en) | A kind of network port Traffic anomaly detection method and system | |
| Ujjan et al. | Towards sFlow and adaptive polling sampling for deep learning based DDoS detection in SDN | |
| Karatas et al. | Deep learning in intrusion detection systems | |
| US11336669B2 (en) | Artificial intelligence cyber security analyst | |
| Gogoi et al. | MLH-IDS: a multi-level hybrid intrusion detection method | |
| Haddadi et al. | Benchmarking the effect of flow exporters and protocol filters on botnet traffic classification | |
| CN110149343A (en) | A kind of abnormal communications and liaison behavioral value method and system based on stream | |
| CN105471854B (en) | A kind of adaptive boundary method for detecting abnormality based on multistage strategy | |
| CN103957203B (en) | A kind of network security protection system | |
| CN102821002A (en) | Method and system for network flow anomaly detection | |
| CN102857486A (en) | Next-generation application firewall system and defense method | |
| Liu et al. | The detection method of low-rate DoS attack based on multi-feature fusion | |
| CN104734916B (en) | A kind of high-efficiency multi-stage anomalous traffic detection method based on Transmission Control Protocol | |
| Fallahi et al. | Automated flow-based rule generation for network intrusion detection systems | |
| CN106254318A (en) | A kind of Analysis of Network Attack method | |
| Aung et al. | An analysis of K-means algorithm based network intrusion detection system | |
| Wheelus et al. | Towards a big data architecture for facilitating cyber threat intelligence | |
| Do et al. | Classifying anomalies for network security | |
| Viegas et al. | A resilient stream learning intrusion detection mechanism for real-time analysis of network traffic | |
| Thi et al. | Federated learning-based cyber threat hunting for apt attack detection in SDN-enabled networks | |
| CN110336806A (en) | A kind of covert communications detection method of combination session behavior and correspondence | |
| Klymash et al. | Concept of intelligent detection of DDoS attacks in SDN networks using machine learning | |
| Tariq et al. | Botnet classification using centralized collection of network flow counters in software defined networks | |
| Campbell et al. | Intrusion detection at 100G | |
| Gupta et al. | A categorical survey of state-of-the-art intrusion detection system-Snort |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |