CN110149343A - A kind of abnormal communications and liaison behavioral value method and system based on stream - Google Patents

A kind of abnormal communications and liaison behavioral value method and system based on stream Download PDF

Info

Publication number
CN110149343A
CN110149343A CN201910469616.6A CN201910469616A CN110149343A CN 110149343 A CN110149343 A CN 110149343A CN 201910469616 A CN201910469616 A CN 201910469616A CN 110149343 A CN110149343 A CN 110149343A
Authority
CN
China
Prior art keywords
port
flow
highest priority
network
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910469616.6A
Other languages
Chinese (zh)
Other versions
CN110149343B (en
Inventor
李志辉
严寒冰
丁丽
温森浩
姚力
朱芸茜
王小群
陈阳
李世淙
徐剑
王适文
肖崇蕙
贾子骁
张帅
吕志泉
韩志辉
马莉雅
雷君
周彧
周昊
高川
楼书逸
文静
吕卓航
杜飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BEIJING RUICHI XINAN TECHNOLOGY Co Ltd
National Computer Network and Information Security Management Center
Original Assignee
BEIJING RUICHI XINAN TECHNOLOGY Co Ltd
National Computer Network and Information Security Management Center
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BEIJING RUICHI XINAN TECHNOLOGY Co Ltd, National Computer Network and Information Security Management Center filed Critical BEIJING RUICHI XINAN TECHNOLOGY Co Ltd
Priority to CN201910469616.6A priority Critical patent/CN110149343B/en
Publication of CN110149343A publication Critical patent/CN110149343A/en
Application granted granted Critical
Publication of CN110149343B publication Critical patent/CN110149343B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1416—Event detection, e.g. attack signature detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425—Traffic logging, e.g. anomaly detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The present invention provides a kind of abnormal communications and liaison behavioral value method and system based on stream, belongs to network security anomalous event passive finding field.Detection system of the invention includes: the configuration management module for configuring white list IP, highest priority IP and general objectives IP, it obtains and the data acquisition module and memory module of storage network flow data information, the highest priority abnormality detection module and general objectives abnormality detection module and anomaly assessment module that highest priority and general objectives are detected respectively.Detection method carries out discharge model building using different methods to important network node and common network node, Network anomaly detection is carried out respectively, it is associated with the network event of important goal and general goals again, excavates and has the network intrusions behavior centainly endangered and abnormal communications and liaison behavior.The present invention has good ability of discovery to various types of Traffic Anomaly behaviors, small to the computation complexity of data on flows, and anomaly strong real-time.

Description

A kind of abnormal communications and liaison behavioral value method and system based on stream
Technical field
The present invention relates to network security anomalous event passive finding field, be it is a kind of based on full dose flow data, for interior The method and system that portion network communications and liaison outside ip address system, is carried out abnormality detection to full dose communications and liaison behavior.
Background technique
With the fast development of computer and network technology, Internet user's scale is growing, China Internet network letter The report of breath center (CNNIC) publication is shown, by December, 2015, for China's netizen's scale up to 6.88 hundred million, internet is universal Rate is 50.3%, meanwhile, the whole nation uses the enterprise of internet office up to 89.0%.Internet has become in people's production and living not The important infrastructure that can or lack.
At the same time, network security problem becomes increasingly conspicuous, and the network safety event frequently occurred brings huge to internet It is big to threaten.Wherein, it is the important threat of information system that network attack and network, which are stolen secret information,.The DDoS that on May 9th, 2009 occurs (divides The service of cloth refusal) attack storm video suspension event, lead to southern six province's carrier server all to smash, telecommunications is in south The network of six provinces, side is paralysed substantially；With in 2009, South Korea's prevailing governmental website suffers New DDoS Attack, including Blue House 25 mechanisms, Korean government, bank and media website paralysis；Famous Spamhaus event in 2013, attack traffic reach Unprecedented 300Git/s, entire European network state are all influenced；On December 31st, 2015, since serious DDoS is attacked It hits, the British Broadcasting Corporation website (BBC) and iPlayer service are forced offline, website paralysis several hours；On January 29th, 2016, The bank system of web of two branches in Hong Kong and Shanghai of the Hong Kong and Shanghai Banking Corporation, Britain, while it being subject to a series of ddos attack, just It is easy to act as the account day that day is system, causes serious influence to the service of whole system.According to famous release mechanism kappa this The report of first quarter ddos attack in 2016 that base laboratory is delivered, 74 national resources by ddos attack, wherein China, South Korea and the U.S. are impacted the most serious.
Ddos attack is initiated to target of attack by the host or Botnet that are largely controlled, it can be to by the letter of attacker Breath system causes serious harm, and most probably influences the proper network activity of other Internet users, generates great network Security incident.China's internet scale is huge, and there is the host largely controlled by wooden horse or bot program, China national interconnections The internet security of net emergency center (CNCERT/CC) publication threatens report display, in March, 2016, domestic more than 1,960,000, China The corresponding host of IP address is controlled by wooden horse or bot program, and therefore, China faces serious potential network attack and threatens.
Cause the abnormal network behavior of network safety event, such as network attack with steal secret information, often relate to network flow Exception, therefore can by abnormality detection, find Network anomalous behaviors, in conjunction with the relevant technologies carry out exception response processing, ensure Network is normal, safeguards network security, and therefore, exception of network traffic detection has a very important significance.
It is existing to be specifically included that using the relevant technologies for doing Network anomalous behaviors detection based on NetFlow
(1) based on the method for data mining.In order to application data mining technology to user behavior carry out abnormality detection it is necessary to Using in data mining association analysis and sequential mining, extract correlation present in the performed order of user under normal circumstances Property, the historical behavior mode of each user is established, the differentiation for actually detected user behavior in the process affords the basis of comparison.It is logical Cross and normal user's training data and current user operation data excavated, obtain respectively user historical behavior mode and After current behavior mode, so that it may judge whether user behavior is abnormal by model comparision.
(2) method neural network based.Neural network (neural networks) is mentioned using adaptive learning technology The feature for taking abnormal behaviour needs to learn training dataset to obtain normally performed activity mode, and training data mark is Normal data and invasion two class of data, the neural network after training can be normal or invasion event recognition.
(3) based on the method for machine learning.This method for detecting abnormality realizes abnormality detection by machine learning, will be abnormal Detection, which is attributed to, to learn the interim sequence of discrete data to obtain the behavioural characteristic of individual, system and network.
In above technology, it is unable to satisfy abnormality detection real-time and universal requirement, for true network In environment, the randomness of flow and the cycle characteristics of variation targetedly develop various existing Network Abnormal intrusion behaviors Method for detecting abnormality realizes that anomaly quickly finds and tracks.
Summary of the invention
Insufficient for the abnormality detection real-time of current internal network, the problem of the universal difference of detection, the present invention proposes A kind of abnormal communications and liaison behavioral value method and system based on stream, relies on stream (flow) data, to important in internal network Network node and common network node carry out Network anomaly detection using different method progress discharge model buildings respectively, then It is associated with the network event of important goal and general goals, excavates and has the network intrusions behavior centainly endangered and abnormal communications and liaison row For.
The present invention is directed to Intranet, discloses a kind of abnormal communications and liaison behavioral value system based on stream, including match Set management module, data acquisition module, data memory module, highest priority abnormality detection module, general objectives abnormality detection mould Block and anomaly assessment module.
Configuration management module provides white list IP configuration, highest priority IP configuration, general objectives IP configuration, it is understood that there may be leakage The port numbers configuration in hole etc., supports the static configuration function of system data, while providing abnormality detection model training task and matching It sets and is configured with abnormality detection task, realize that abnormality detecting process is controllable.Highest priority is IP to be protected is weighed in internal network Location is reported and submitted by user or is monitored analysis network flow data and obtains, and the IP address that highest priority is removed in internal network is general mesh Mark.To in the opposite end IP of target access, legal, stable, safe opposite end IP is added in system white list, in flow Directly ignore the flow of opposite end IP in white list during model foundation and Traffic anomaly detection.Configuration management module also according to The port of the service and opening disposed on known highest priority server configures the stabilization port of corresponding highest priority, so as to When highest priority Traffic Anomaly models and detects, it is no longer necessary to be calculated and be modeled to port is stablized.White list and stabilization Port configuration needs are simple direct, facilitate user's real time modifying to configure, are issued to corresponding detecting and alarm in time.Highest priority and General objectives Traffic anomaly detection engine, can inspect periodically the configuration file issued, timely update and white list and stablize port, change Become detection behavior.
Data acquisition module docks router or other network shunt equipment, receives the netflow number from router According to, primitive network flow is handled, by solution data packet and flow restore, thus flowed four-tuple (source IP address, Purpose IP address, source port, destination port), time started, end time, protocol type, TCP flag bit, packet number, byte number Etc. information be written to data memory module by a plurality of streaming data information in a manner of kafka.
Data memory module reads streaming data information from kafka message queue, every fluxion to highest priority it is believed that Breath is all stored, and to general objectives using 5 minutes or 1 hour as minimum particle size, counts the flow-time sequence on each port Column.In flow-time sequence store five-tuple (source IP address, purpose IP address, source port, destination port, protocol type) and Packet number, byte number.For traffic characteristics such as packet number, byte numbers, the value changed over time is recorded, each value recorded is one In the statistical value of minimum particle size.
Highest priority abnormality detection module is used for: (1) communication time of communication object is obtained from history streaming data information With communicate byte number, construct the stabilized communication object model of highest priority；(2) multidimensional characteristic vectors of highest priority are constructed, Middle characteristic item includes stream direction, flow network protocol type, service port number and flow indicator, is counted from history streaming data information The value of each characteristic item；The hypothesis for carrying out normal distribution and two kinds of regularities of distribution of logarithm normal distribution to the statistical value of each characteristic item is examined It tests, for the characteristic item of Normal Distribution rule, calculates mean value and standard deviation as statistical threshold, for obeying lognormal The characteristic item of the regularity of distribution calculates logarithmic average and logarithm standard deviation as statistical threshold, establishes threshold model；(3) threshold is utilized Value model detects band detection flows.When detecting, each characteristic item in each time window is obtained in measurement of discharge to be checked Observation its mean value and standard deviation or logarithmic average and logarithm standard deviation are calculated according to the Statistical Distribution of characteristic item, Then it is compared with statistical threshold corresponding in threshold model, calculates departure degree；Departure degree is equal by measurement of discharge to be checked Difference between value/logarithmic average and mean value/logarithmic average of threshold model ,/logarithm standard deviation poorer than upper threshold value model Plays, Multiple obtained determines.
General objectives abnormality detection module is used for: (1) being detected, wrapped using the changes in flow rate model based on time series It includes: to the time series of port flow, subtracting tendency component and cyclical component therein, obtain random fluctuation feature, with Machine fluctuation characteristic meets the definition of normal distribution, according to confidence level, calculates random fluctuation feature using normal distribution hypothesis testing The coefficient for deviateing standard deviation, finds flow and uprushes a little；(2) it is detected using flow data polymerization model, the flow data is poly- Molding type chooses different grouping from five-tuple and carries out varigrained building, is divided according to selected granularity stream data information Group, then converging operation is carried out to byte number and packet number, abnormal behaviour is found by sequence.
Anomaly assessment module synthesis highest priority abnormality detection result and general objectives abnormality detection result, in conjunction with its exception Every traffic characteristic when generation, using respective feature weight, comprehensive assessment exception level and Exception Type generate abnormal inspection Observe and predict announcement.
The present invention is directed to Intranet network, proposes a kind of abnormal communications and liaison behavioral value method based on stream, point Following steps:
Step 1: the stabilization port of highest priority, white list and highest priority in configuration internal network, by internal network Middle other IP nodes for removing highest priority are as general objectives.Highest priority reports and submits or monitors analysis network fluxion according to user According to configuration,；IP in white list is the IP address in the external network of legitimate secure, and the stabilization port of highest priority is according to right The port of highest priority server opening is answered to configure.
Step 2: configuring netflow data on the network equipments such as the critical point router of internal network or network shunt and receive Collection generates, sending function, the server network flow data of crawl being sent to where data acquisition module, data acquisition module Block is using libcap, dpdkcap etc., to the data packet for flowing through server system network interface, as NetFlow, NetStream, The flow data of the formats such as sflow, are grabbed in real time, are parsed to the flow data of crawl, and streaming data information is obtained.
Step 3: to the streaming data information of acquisition, being filtered according to white list IP, for highest priority IP, store every Streaming data information.
Step 4: from the streaming data information of storage, counting the communication of different port and opposite end IP for same highest priority Duration and communication byte number, the port and peer IP address that access using clustering algorithm, when will be long and data transmission are biggish Port and opposite end IP, which are used as, to be stablized port and stablizes peer IP address.
Step 5: to each highest priority, traffic characteristic, including stream direction, flow network agreement are extracted from streaming data information Type, service port number and flow indicator；It whether is to stablize port or steady using extracted traffic characteristic and Correspondent Node Determine opposite end IP and combines the feature vector to form the different dimensions of highest priority；It is right according to the history streaming data information of highest priority The value of each characteristic item of feature vector is counted.
Stream direction refers to, when the source address that highest priority is initiated as connection, the direction of the data stream is outflow, instead It, the direction of the data stream is to flow into；Flow network protocol type refers to that the flow data that highest priority is communicated with opposite end is in network The agreement run on layer；Service port number refers to, when highest priority is communicated with opposite end, the port numbers of highest priority side； Flow indicator refers to, the uninterrupted of highest priority and opposite end, including attribute have byte number, network packet number and opposite end IP number.
Step 6: judging that the regularity of distribution of each traffic characteristic, counting statistics threshold value establish the threshold model of traffic characteristic.It is right Statistical value on traffic characteristic carries out the hypothesis testing of normal distribution and two kinds of regularities of distribution of logarithm normal distribution, for obeying just The traffic characteristic of the state regularity of distribution calculates its mean value and standard deviation as statistical threshold, for obeying logarithm normal distribution rule Traffic characteristic, calculate its logarithmic average and logarithm standard deviation as statistical threshold.
Step 7: using the threshold model of the traffic characteristic for the highest priority established, treating detection flows and carry out irrelevance meter It calculates, and then judges whether measurement of discharge to be checked is abnormal flow.It is special to obtain in measurement of discharge to be checked each flow in each time window The observation of sign calculates its mean value and standard deviation or logarithmic average and logarithm according to the Statistical Distribution of the traffic characteristic Standard deviation is compared with threshold value corresponding in threshold model, calculates departure degree；Departure degree is equal by measurement of discharge to be checked Difference between value/logarithmic average and mean value/logarithmic average of threshold model, than standard deviation/logarithmic scale in upper threshold value model The multiple of difference determines.
Step 8: for the traffic flow information of general objectives, according to the IP segment limit of general objectives, the exception of required detection Type and preset Packet Filtering rule, are screened, to reduce the data volume of required by task.
Step 9: by the way of multi-process/multithreading, using 5 minutes or 1 hour as minimum particle size, calculating general objectives Flow on each port forms the time series of each port.Flow-time sequence is according to five-tuple (source IP address, purpose IP address, source port, destination port, protocol type) divided, the feature of storage include packet number, byte number etc., recorded Packet number and byte number be the value changed over time, each value recorded is a statistical value in minimum particle size
Step 10: to the flow-time sequence of each port, subtracting the vector of its tendency, then subtract its periodic feature Vector meets the definition of normal distribution using remaining vector as the random fluctuation feature of general objectives, according to confidence level, answers Its coefficient for deviateing standard deviation is calculated with normal distribution hypothesis testing, flow is found and uprushes a little.
Step 11: by 1 hour/1 day dimension statistics general objectives IP address and its communications and liaison opposite end of particular port, in access The quantity and the connection frequency of portion's IP node, and opposite end IP is ranked up, it finds extremely different with IP sections of scannings there are port scan Normal peer IP address.To the opposite end IP found, is detected using flow data polymerization model, judge whether there is abnormal behaviour； The flow data polymerization model is according to five-tuple (source IP address, purpose IP address, source port, destination port, protocol type) Stream data information is grouped, and carries out converging operation to byte number and packet number, then find exceptional value by sequence.
Step 12: to the opposite end IP for accessing multiple general objectives, being scanned analysis and DDoS analysis, detected whether end The behavior of mouth scanning and logon attempt.
Step 13: the abnormal aggression event of association highest priority and general objectives carries out comprehensive assessment to anomalous event.
The method of the present invention and system have the following advantages that and good effect compared with traditional business monitoring technology:
(1) five-tuple data are based on, data-handling efficiency is high；To source IP address, source port, purpose IP address, destination Mouthful etc. using fining analysis method, to various types of Traffic Anomaly behaviors, including DDoS, port scan, IP address detection, Abnormal flow, which is uprushed etc., has good ability of discovery, and building is based on IP communications and liaison behavior model, complicated to the calculating of data on flows It spends small, sufficiently carries out multiple Data Management Analysis, anomaly strong real-time.
(2) it is deployed on the portal, anomalous event finds that range is comprehensive, and this method being capable of network communications and liaison to full dose Behavioral data is retained, and is handled and is analyzed, and can be analyzed one by one its abnormal index feature to every four-tuple record, connect The flow collection number being deployed on network critical point is received, all abnormal behaviours to be analyzed can be covered.
(3) strong using distributed storage, load balancing and the detection method of parallelization, data-handling capacity.Side of the present invention Data are stored and are separated with abnormality detection by method, can to different Traffic Anomaly behaviors, provide different magnitude of data on flows into Row detection；It can will realize that IP address-based Detection task divides for the detection method of highest priority IP address simultaneously, do To the single Detection task of single IP, to realize that detection process parallelization is handled.
Detailed description of the invention
Fig. 1 is that the present invention is based on a business processing flow figures of the abnormal communications and liaison behavioral value method of stream；
Fig. 2 is that time series models construct schematic diagram in present system；
Fig. 3 is the allomeric function structural schematic diagram of detection system of the invention.
Specific embodiment
Illustrate technical solution of the present invention with reference to the accompanying drawings and examples.
The purpose of the present invention is realizing the real-time and general detection of Intranet exception communications and liaison behavior, quickly to find Various exceptions, maintenance inner-mesh network safety.Firstly, illustrate the abnormal communications and liaison behavioral value method of the invention based on stream, one The operation flow of realization is as shown in Figure 1, illustrate to implement step below.
Step 1: configuration highest priority IP, highest priority source are made of two parts: being reported and submitted and monitored analysis and obtain in outside.
1) highest priority reported and submitted based on outside.Report and submit fixation composition portion of the highest priority as highest priority collection in outside Point, it is the main services target of this system；For the highest priority that outside is reported and submitted, attack detecting is provided and detection of stealing secret information.
The highest priority that outside is reported and submitted is divided into two classes:
(a) highest priority illustrated with service type:, can be directly by configuring highest priority for this kind of target Network behavior constrains to detect the abnormal behaviour of such highest priority.
(b) do not have the highest priority that service type illustrates: for this kind of highest priority, needing to call active probe function Module detects target, analyzes the types of network services of highest priority, and network behavior is arranged for it and constrains.
2) highest priority based on monitoring discovery.Monitoring Focus target refers to through port scan, routing tracking, domain name solution The analysis means such as analysis find highest priority.
Monitoring Focus target detects highest priority with the mode passively combined using active:
(a) passive part is responsible for obtaining the IP address and service type of doubtful highest priority；
(b) active part is to the service type of highest priority and port as detection and verifying.
The part that monitoring analysis obtains is as the supplement part for reporting and submitting highest priority.The underlying attribute of active probe target is The key node of Internet service, comprising: domestic main dns server, domestic main ntp server, backbone network by Node, domestic main mail server, famous game server, mobile internet service device etc..
For example, a threshold value, to general service device, the whether stable length in certain port of statistical server is arranged when monitoring analysis Phase provides externally service, the WEB server of 80/443 port, the mail server of open 25/110 port is such as opened, if service Certain port of device is stable long-term to be occurred in particular time range, then it is assumed that this server is stable server, as Server ip is set highest priority IP by highest priority server.
Other node IPs of highest priority IP will be removed in local area network, as general objectives.General objectives node may produce Raw a large amount of random open port flows, and there is the case where access opposite end is emphasis destination server.
In the opposite end IP of local area network destination node access, the opposite end IP of part is stable and safety.These are right The flow of end IP does not need actually to be detected, and these flows should occupy the suitable of the flow of the access of a target Big specific gravity.So needing these are legal, stable, safe opposite end IP to be added in system white list, in discharge model Directly ignore the flow of opposite end IP in white list during establishing with Traffic anomaly detection.In highest priority Traffic anomaly detection In, the stabilized communication object model of highest priority can be constructed.Stable port can be constructed in the model, which is to pass through stream What data information modeled, and in actual environment, which service may be deployed on certain known highest priority server, it is open Which port.In this case, stablizing port should be configured according to known case, without being calculated and being built Mould.White list and stable port configuration needs are simple direct, facilitate user's real time modifying to configure, are issued to corresponding detection in time Engine.Highest priority and general objectives Traffic anomaly detection engine, can inspect periodically the configuration file issued, and timely update white name Singly and stablize port, changes detection behavior.
Step 2: data packet crawl refers to using operating system physical layer interface, such as: libpcap, winpcap, bpf (Bai Ke Sharp Packet Filter) etc., crawl analysis in real time is carried out to the data packet for flowing through system network interface.Stream based on data packet crawl Acquisition technique is measured, all detailed informations of each data packet from data link layer to application layer can be got.Relative to will be entire Data packet crawl analysis, the TcpDump realized based on interface is that the head of data packet is intercepted to analysis, the head of data packet The summary info of data packet is contained, TcpDump can provide enough flow informations for most analytical technology.
Based on the acquisition technique of data packet crawl, the collected data on flows of institute can provide enough thin for flow analysis Information is saved, can analyze all detailed informations for getting each data packet from data link layer to application layer, but it is to system Performance requirement is higher, and obvious network performance can be brought to consume.
Rapidly obtaining flow data is timely to detect abnormal communications and liaison security incident to provide the foundation guarantee.It is flowed in the present invention The acquisition of information is carried out on backbone routers, and router acquisition network flow has following several mechanism:
1) data packet is received, whether detect in flow table has matched five-tuple (source IP, destination IP, source port, destination Mouthful, protocol type) etc., have, packet information is added in flow table, otherwise creation stream record information, is added in flow table.
2) when receiving stream end mark (Transmission Control Protocol), terminate current stream, send current stream information.
3) it when flow table is full, and needs to be inserted into new stream node, then refreshes current flow table, export all stream notes Record.
4) every the flow table of update in 5 minutes, specified data acquisition interface address is sent by the stream record in flow table.
Data acquisition interface, timing calling have stream acquisition system order and flow data are cached to this by particular category format Ground, data query are delayed 2 days, can guarantee that flow data obtains substantially.Data acquisition module solves the network flow data of acquisition Analysis obtains the information such as data four-tuple (source IP, destination IP, source port, destination port), packet number, byte number, and a plurality of stream is recorded In a manner of kafka, it is written to data memory module.
Step 3: the streaming data information that data memory module storage is collected, the historical data of storage about 2-4 weeks or so, application Hive or the format of file storage, at least contain source IP address, purpose IP address, source port, destination port, packet number, byte Number, time etc..Network biggish for data volume, can carry out sampling according to certain rule or white list filtering, use are white When list IP is filtered, filters out source IP or destination IP is the streaming data information in white list IP.
The present invention uses the retention mode of multi-layer, for highest priority IP and general objectives IP storage different grain size, no With the streaming data information of format, to meet the detection demand to different target.For highest priority IP, store every fluxion it is believed that Breath.Then it is carried out abnormality detection according to 4 to 7 pairs of highest priorities of below step.For general objectives IP, made with 5 minutes or 1 hour For minimum particle size, the flow-time sequence on each port is counted, step 8 and 9 specifically are stored in the data of general objectives IP Middle explanation.Then it is carried out abnormality detection according to step 10 to 12 pairs of general objectives.
Step 4: from the streaming data information of storage, when obtaining different port and opposite end IP and the communication of same highest priority Long and communication byte number is clustered to obtain based on k-means and stablizes port and stablize peer IP address.Stablize port and stablizes opposite end IP address refers in server operation on line, opens more port and connects more opposite end.
Since two attributive character of communication object are respectively duration tn and byte number pbtmum, the two is not the same amount The attribute of guiding principle carries out similarity measurement using formula (1), takes logarithm to average flow rate value, then Euler's formula recycled to carry out Distance calculates.One sample representation, one port communicated with highest priority or opposite end IP, including duration and byte number two categories Property, give sample xi=(tni,pbtnumi) and xj=(tnj,pbtnumj), define distance function dist (xi,xj) are as follows:
Initial center point is set, for data sample to be divided into expected 4 classifications: occurring to long-time stable and flow Biggish classification is measured, selects tn value and pbtnum value in sample maximum as initial center；Occur to long-time stable and flows General object is measured, selects tn value in sample maximum, the conduct initial center near pbtnum mean value；Occur to the short time and flows Measure biggish, pbtnum maximum in selection sample, the conduct initial center near tn mean value；Occur to the short time and flow is general , select tn value in sample minimum, pbtnum value is the smallest to be used as initial center.
It is 4 that number of clusters is clustered in the embodiment of the present invention, for the 4 cluster sample sets got, according to formula (2), if Set 4 initial cluster center { u1, u2, u3, u4, first run iteration, which is carried out, by range formula (1) calculates each sample to 4 Sample is divided into nearest class cluster, then recalculates the center of 4 class clusters respectively by the distance of initial cluster center Value, if a central value does not update, clustering algorithm terminates, otherwise, into next round iteration.
Wherein, max expression is maximized, and mean expression takes mean value, and min expression is minimized, and i is in the cluster set of place Sample number.
After the Iterative classification of clustering algorithm, 4 classifications and class center are finally obtained, wherein long-time stable goes out In the existing and biggish classification of flow is to stablize port and stablize peer IP address.
Step 5: constructing the multidimensional traffic characteristic of target.
To the streaming data information of highest priority from the definition of following dimension: stream direction, flow network protocol type, service Port numbers, flow indicator etc., in conjunction with whether stable port/opposite end IP, form multiple feature vectors, to each feature vector unite Meter forms and the feature of highest priority flow different dimensions is described.
The traffic characteristic of the streaming data information of highest priority: stream direction, flow network protocol type, service port number, flow The concrete meaning of index is as follows:
Flow direction: the flow direction based on data packet in a stream determines the direction of this stream, i.e., when highest priority is as connection When the source address of initiation, the direction of this stream is outflow, conversely, the direction of this stream is to flow into.The dimension includes two attributes: It flows into, outflow；
Flow network protocol type: the flow data that target is communicated with opposite end, the agreement run in network layer include three Attribute: TCP, UDP, ICMP；
Service port number: highest priority is communicated with serviced opposite end, the port numbers of highest priority side, range of attributes: 0 ~65535.
Flow indicator: marking uninterrupted, and mainstream includes byte number, network packet number, IP number etc. at present, therefore the dimension Degree mainly includes attribute: byte number, network packet number, opposite end IP number.
To the attribute in each dimension, combined crosswise is at 4 tuple multidimensional characteristics are constituted, i.e., < stream direction, protocol type, port Number, flow indicator >, realize the careful division to network flow.Meanwhile attribute is not distinguished in each dimension addition, for realizing right The upper volume of other dimensions combination such as flows the addition of direction dimension and does not distinguish attribute, i.e. the statistical nature direction of not distinguishing stream, is applicable in In the data of all flow directions.It counts the flow value of each traffic characteristic item: every stream in target histories flow data is noted down, according to Its affiliated time window, communication port, Correspondent Node, flow value update the statistical value of corresponding traffic characteristic item, complete to each spy Levy the traffic statistics of item.
Step 6: establishing the threshold model of traffic characteristic item.Two kinds of distributions are carried out to statistical value of the flow in each feature Rule --- the hypothesis testing of normal distribution and logarithm normal distribution calculates it for the characteristic item of Normal Distribution rule Mean value and standard deviation calculate its logarithmic average and logarithm standard deviation, are counted for obeying the characteristic item of logarithm normal distribution rule The mean value and standard deviation or logarithmic average and logarithm standard deviation of calculation are exactly statistical threshold, establish the threshold value mould of each highest priority Type.
Step 7: using the threshold model of the traffic characteristic item for the highest priority established, to the flow measurement to be checked of the highest priority Amount carries out irrelevance calculating, generates sliding window and deviates vector.
For the measurement of discharge to be checked observed, observation time window Ws, the time window size and the threshold established are set The time window size of value model is consistent, and every time from observed volume sequence, takes out an observation time window Ws, statistics The flow value of each characteristic item in the window.To the observation of characteristic item each in time window, with this feature in threshold model Threshold value is compared, according to the Statistical Distribution of this feature, the mean value and standard deviation or logarithmic average of calculating observation data and Logarithm standard deviation, the departure degree relative to the statistical threshold in threshold model.Departure degree by measurement of discharge to be checked mean value/it is right Difference between number mean value and mean value/logarithmic average of threshold model ,/logarithm standard deviation poorer than upper threshold value model Plays are obtained Multiple determine.When departure degree is to abnormal data is judged as after a certain amount of, according to common normal state point in the present invention Cloth hypothesis testing algorithm, general 3 times of deviation are judged as abnormal data.
Step 8: according to the IP segment limit of general objectives, the Exception Type of required detection, as HTTP retains full dose extremely 80 port flows, FTP retain 21 port flows and other Packet Filterings rule of full dose extremely, are waited using sql inquiry Filter mode screens the network data of general objectives, to reduce the data volume of required by task.Screening rule is as follows:
1) streaming data information for inside end slogan less than 10000 filters out wherein outflow flow and is greater than 300 bytes Streaming data information；
2) white list and black name are used if there are more web servers for inside for the uplink traffic of 80/443 port Folk prescription formula, there are the flows of suspected attack for fetching portion；
3) for the flow greater than 10000 ports, often in addition to the part such as 11211 (memcache), 27017 (mongodb) See outside port, remaining only obtains the streaming data information for being greater than 600 bytes as random port.
Step 9: by the way of multi-process/multithreading, using 5 minutes/1 hour as minimum particle size, calculating general objectives Flow on each port forms the time series of each port.
Since inner open port range is 0-65535, quantity is larger, will be each by the way of epoll multi-process Computing unit is packaged, and is put into the shared section key of system, is read the data and its treatment process by other processes, maximally utilize CPU forms about 60000 time serieses.Flow-time sequence be one include stream information sequence, by five-tuple (source IP Location, purpose IP address, source port, destination port, protocol type) it is divided, and have very common field, such as uplink and downlink Byte number, uplink and downlink number-of-packet etc., the value that the traffic characteristics such as packet number, byte number are recorded are the sizes changed over time, often A value is a statistical value in minimum particle size.
Step 10: to single time series, there is tendency feature and periodic feature, remove outside the two, it is remaining Fluctuation vector of the vector as time series, randomness meet the definition of normal distribution.As shown in Fig. 2, calculating random fluctuation The process of feature is as follows:
Step1) original time series Y subtracts trend variable, and trend variable initial value is 0；
Step2) period subsequence is smooth, using loess (the local weighted Return Law) to each subsequence time in period Point calculates smooth value, including missing point.In the embodiment of the present invention period be day, time granularity be 5 minutes granularities, then one day when Between have 12*24=288 period subsequence in sequence；
It Step3 is) window size with each period mileage strong point number 288, time series C smoothed out to Step2 obtains every Sliding Mean Number on a time granularity forms new time series, while being loess to new time series and smoothly being faced Shi Liuliang sequence L.When in the presence of doing sliding average, beginning and end data are lost situation, flat by 288 loess in Step2 Period subsequence after cunning predicts beginning and end of data；
Step4 period subsequence, i.e. periodic component S=C-L) are obtained；
Step5) original time series remove periodicity, original flow Y- periodic component S；
Step6) to going the time series progress loess after the period smooth, trend subsequence, i.e. trend component T are obtained；
Step7) surplus of flow is R=Y-T-S.
R is exactly random fluctuation feature, according to confidence level, is using what normal distribution hypothesis testing calculated its deviation standard deviation Number, finds flow and uprushes a little, it is a little exactly abnormal flow that flow, which is uprushed,.
Step 11: individual flow data can by five-tuple (source IP address, purpose IP address, source port, destination port, Protocol type) it is divided, and have very common field, such as uplink and downlink byte number, uplink and downlink number-of-packet.Flow data Flow data that acquisition module receives daily record has hundreds and thousands of ten thousand, to each flow data carry out analysis efficiency one by one compared with It is low, as soon as and a flow data be not easy to find out abnormal behaviour, so polymerization of the stream data on different dimensions extremely has It is necessary.By the traffic behavior of certain IP and serve port, it is more easier to find Traffic Anomaly behavior.By 1 hour/1 day dimension The communications and liaison opposite end of general objectives IP address and its particular port is counted, accesses the quantity and the connection frequency of internal IP node, and right Opposite end IP is ranked up, and finds that there are port scan is abnormal and the peer IP address of IP sections of scan abnormalities.To the opposite end IP found, It is detected using flow data polymerization model, judges whether there is abnormal behaviour.
Flow data polymerization model is grouping-polymerization-sequence to the basic information of collected flow data.Flow data polymerization What model needed to do is exactly to be grouped according to five-tuple information convection current, and carry out converging operation to fields such as byte, data packets, Extreme exceptional value is found by sequence again.General objectives Traffic anomaly detection pays close attention to the outflow flow of target, so, source Ip (sip) is general objectives, schematically illustrates part grouping-converging operation below:
It is grouped according to sport, byte number, number-of-packet in polymerization grouping.
It is grouped according to sip, sport, byte number, number-of-packet in polymerization grouping.
According to the C section of sip, sport is grouped, byte number, number-of-packet in polymerization grouping.
According to sip, dip, sport is grouped, byte number, number-of-packet in polymerization grouping.
It is grouped according to sip, dip, sport, dport, byte number, number-of-packet in polymerization grouping.
According to the C section of sip, dip, sport, dport is grouped, byte number, number-of-packet in polymerization grouping.
Wherein sip is source ip, and sport is source port, and dip is purpose ip, and dport is destination port.Pay close attention to the C section of sip Flow aggregation information is same network segment ip to be permeated and be counted because of many network attacks, the C section flow of sip is poly- Molding type is able to reflect abnormal behaviour caused by corresponding attack.
Based on above-mentioned grouping-converging operation, diversified flow data polymerization model is constructed from different Packet granularities. These flow data polymerization models pay close attention to the flow aggregated data of different grain size level, from sip, dip, sport, the difference of dport Combination is to pay close attention to various forms of abnormal behaviours.Such as sip, the combination of sport focuses more on some tool to some general objectives The traffic conditions of body service, to discover whether there is abnormal behaviour to the service of general objectives；The combination of sip, dip, sport are more Some specific attack host is paid attention in the traffic conditions of some specific service of some general objectives, to find the opposite end Whether abnormal behaviour is had to the service of general objectives.
Step 12: to the peer IP address for accessing multiple general objectives, detecting that there are port scan and logon attempts etc. Behavior.
1) scanning analysis.Scanning analysis engine analyzes the message in pcap file to be analyzed, respectively with (source IP, Destination IP), (source IP, destination port), (destination IP, destination port) etc. for key carry out message amount statistics, be more than for quantity The key assignments of given threshold marks exception, is recorded in corresponding construction body, carries out further anomaly analysis or direct output abnormality, Output abnormality type includes:
It is the scanning behavior of IPs a large amount of to fixed port 1. IP is scanned.
2. port scan, to the full port scan of a certain IP.
2) DDoS is analyzed.Statistic record with (destination IP, destination port) be key message amount, while calculate its peak value and Total message number, and the type of message including TCP SYN, TCP ACK, UDP etc. is analyzed, output peak value is more than the threshold of setting Value, and meet the abnormal events information of DDoS feature.
Step 13: the linear weighted function assessment based on abnormal class: on the one hand, assessment abnormal flow corresponds to various classical networks The known network attack that a possibility that attack occurs, i.e. discovery target are subjected to；On the other hand, the threat of abnormal flow totality is assessed Property, the unknown network that discovery target is subjected to is attacked.In terms of known network attacks threat level assessment, attacked in conjunction with known network Abnormal flow feature takes big weight to the feature of strong correlation, and weak relevant feature takes small weight, and the weight of uncorrelated features takes Zero.The initial weight parameter of each characteristic item is both configured to 1, for the traffic characteristic of strong correlation, weighting parameter value is double；For Incoherent traffic characteristic, weighting parameter value take zero, and linear summation obtains AnomalyValue.
In view of the assessed value AnomalyValue value range after weighting is still very wide, and often focus first on abnormal prestige That coerces is qualitative, then carries out quantitative analysis again, therefore, using formula (3), takes logarithm to anomaly assessment value and be rounded, obtain Exception level:
AnomalyLevel=int (ln (AnomalyValue+1)) (3)
It is abnormal detecting traffic statistics value based on threshold model, and weight the higher situation of threat level after assessment Under, it analyzes in abnormal time window, distribution of the flow on port, according to total flow accounting, finds out flow port outstanding, tie It closes distribution of the flow on stabilized communication port set and these ports on opposite end and judges it according to crosscheck method A possibility that for abnormal, promotes anomaly assessment grade if finding the high communication port of a collection of dubiety.Equally, if hair The high Correspondent Node of existing a batch dubiety, then promote anomaly assessment grade.
Compared with the prior art, the method for the present invention is by utilizing flow data, for internal network nodes, opposite customization Using abnormality detection model, anomaly result is made to have comprehensibility, from Traffic Anomaly event is macroscopically found, provides stream Measure the method for detecting abnormality of communications and liaison behavior all standing.Varigrained abnormality detection is done to general objectives and highest priority, is realized The effective use of system computing capacity has certain real-time and availability.
Corresponding, the abnormal communications and liaison behavioral value system provided by the invention based on stream is arranged in Intranet On separate server or distributed server, as shown in figure 3, the system includes configuration management module, data acquisition module, data Memory module, highest priority abnormality detection module, general objectives abnormality detection module and anomaly assessment module.It is main in Fig. 3 Illustrate abnormality detection module.
Configuration management module is mainly used for: (1) configuring white list IP address information, including IP address, domain name, white list class Type mainly has general service device IP, common WEB application etc., such as 8.8.8.8DNS server.(2) highest priority IP address is configured, To need the IP address of internal network and its normal open port laid special stress on protecting.(3) general objectives IP address is configured, for except highest priority IP Outer all IP address of internal network or IP address section.In addition, configuration management module also configures the port numbers there may be loophole, branch The static configuration function of system data is held, while providing the configuration of abnormality detection model training task and matching with abnormality detection task It sets, realizes that abnormality detecting process is controllable.
Data acquisition module, data sending terminal configuration of routers netflow/netstream function support two kinds of v9, v5 Format flow data, configuration purpose IP address are the server where data acquisition module.Data acquisition module utilizes operating system Physical layer interface, such as: libpcap, winpcap, bpf etc., or the DPDK trawl performance mode provided using intel utilize it Characteristic carries out high-performance network interface card and catches packet.Data four-tuple (source IP, destination IP, source port, the mesh that data acquisition module will parse Port) and other relevant streaming data informations, be written to the kafka message queue of hive database.Data acquisition module The streaming data information that parsing obtains includes source IP, destination IP, source port, destination port, protocol type, packet number, byte number, time Deng.
Data memory module completes the business such as data retention, data check, data retrieval.Data memory module obtains in real time The streaming data informations such as four-tuple, packet number, the byte number of kafka message queue call the unserializing method of java, generate structure The stream of change records, including source IP address, purpose IP address, source port, destination port, the time started, the end time, protocol number, The fields such as tcp flag bit, packet number, byte number are persisted in database according to 4000 every batch of.Data memory module is using more The retention mode of level stores different grain size, different-format to highest priority IP, general objectives IP and its port numbers etc. Data, to meet the data requirements of detection module.For highest priority ip, every stream information is stored, for general objectives IP, system Count the daily flow-time sequence of each port.Meanwhile this module provides IP address-based data retrieval mode, to reach Requirement of real-time.
That there are the behaviors of larger statistic bias is different to identify with normal activity by analysis for highest priority abnormality detection module Often.Method based on statistical analysis sampled targets behavioural characteristic first, such as the intensity of performance, no of distribution, the measurement of Audit data With the distribution etc. of audit actions.The series of parameters of description goal behavior is obtained by calculation, forms detection behavior profile.Due to Network event (as grouping reaches) is dynamic, and the behavior profile acquired every time and goal behavior profile are merged to obtain by system Normally performed activity profile.When detecting by relatively judging with normal behaviour profile, when beyond the threshold value of setting, carry out different Often alarm.
Highest priority abnormality detection module includes four parts, is the building of stabilized communication object model, the mesh of target respectively Target multidimensional characteristic statistical threshold model, noise data are eliminated, deviate vector generation.
1) the stabilized communication object model of target is constructed.Count the communication time and flow mean value of each communication object: for The communication port of target and opposite end define communication time and average flow rate value based on time window.The given duration observed For the historical traffic of t, the time window with stronger cycle effect, such as 1 day are set；It is a length of when historical traffic is divided into n On the time window of wt, to each time window, statistics wherein all port/opposite end IP flow value；Then it establishes global Port/opposite end IP dictionary, count the number tn of port/opposite end IP time window of each appearance, meanwhile, count the end The total flow byte number of mouth/opposite end IP；Finally calculate mean value pbtnum of the port/opposite end flow on time window, port/right The attribute vector at end is { communication time, average communication byte number }, i.e., { tn, pbtnum }.Then it is identified using clustering algorithm In stabilization port and stablize peer IP address.
2) the multidimensional characteristic statistical threshold model of target is constructed.Count the flow value of each traffic characteristic item: to target histories Every stream record in flow data updates corresponding flow according to its affiliated time window, communication port, Correspondent Node, flow value The statistical value of characteristic item completes the traffic statistics to each characteristic item.Judge the regularity of distribution of each characteristic item, counting statistics threshold value: The hypothesis testing of two kinds of regularities of distribution (normal distribution and logarithm normal distribution) is carried out to statistical value of the flow in each feature, For the characteristic item of Normal Distribution rule, its mean value, standard deviation are calculated, for obeying the feature of logarithm normal distribution rule , its logarithmic average, logarithm standard deviation are calculated, statistical threshold is obtained.
3) noise data is eliminated.For the noise data of suppressing exception, while normal data is accidentally deleted as little as possible, devise base In the noise cancelling alorithm of maximal density income, which assumes the integrated distribution on certain section of the normal data in data, Abnormal data sparse distribution outside normal interval, i.e. normal data are distributed in the big section of density, this and the flow observed Data distribution is consistent.
4) it generates and deviates vector.To the observation of characteristic item each in time window, with the spy in corresponding threshold model The threshold value of sign is compared, and according to the Statistical Distribution of this feature, calculates the logarithm of its statistical value or statistical value, relative to The departure degree of threshold value, wherein departure degree is by mean value/logarithmic average of measurement of discharge to be checked and mean value/logarithm of threshold model The difference of mean value, the multiple of/logarithm standard deviation poorer than upper threshold value model Plays determine.
Opposite highest priority, the quantity of general objectives is relatively large, in the limited situation of resource, needs to study new inspection Method of determining and calculating promotes detectability.Highest priority Traffic anomaly detection is once analyzed for the flow data of a highest priority； And general objectives Traffic anomaly detection is the abnormality detection for all targets, whether when detection will not be highest priority to target It is distinguish.General objectives abnormality detection module includes constructing different grouping granularity by the way of grouping-polymerization-sequence Flow data polymerization model, the changes in flow rate model based on time series etc..
General objectives abnormality detection module includes as follows:
1) the flow data polymerization model of different grouping granularity.Individual streaming data information can by five-tuple (source IP address, Purpose IP address, source port, destination port, protocol type) it is divided, and very common field is had, such as uplink and downlink word Joint number, uplink and downlink number-of-packet etc..The flow data record that data acquisition module receives daily has hundreds and thousands of ten thousand, to each The analysis efficiency that flow data carries out one by one is lower, and a flow data is not easy to find out abnormal behaviour, so stream data Polymerization of the information on different dimensions just very it is necessary to.By the traffic behavior of certain IP and serve port, it is more easier to send out Existing Traffic Anomaly behavior.Since present system pays close attention to Dos attack and secret stealing behavior is.So aggregate target is general Flow data byte number and number-of-packet, especially byte number, the size of direct response data flow are selected, is most common poly- Close target.Flow data polymerization model can choose different grouping from five-tuple and carry out constructed by varigrained building, according to institute It selects granularity stream data information to be grouped, then converging operation is carried out to byte number and packet number, abnormal behaviour is found by sequence. It is generally believed that stolen close exception is most likely to occur when flow polymerization model finds big flow behavior, so using sequence side Method finds extreme byte value, detects stolen close abnormal behaviour.
2) the changes in flow rate model based on time series.Flow data polymerization model is mainly the time window provided at one Flow data polymerization is carried out under mouthful, in general, this time window can be set to 1 day, or is used as required finer Granularity, 12 hours, 6 hours etc..But not this dimension of having time in the result of above-mentioned flow data polymerization model.And different Often detection in the model based on time series be it is very necessary, the model based on time series can reflect flow at any time or The situation of change of time window reflects Traffic Anomaly behavior.Therefore, it using the time series of each port flow, subtracts therein Tendency component and cyclical component obtain random fluctuation feature, since random fluctuation feature meets the definition of normal distribution, root According to confidence level, the coefficient that random fluctuation feature deviates standard deviation is calculated using normal distribution hypothesis testing, flow is found and uprushes a little, Note abnormalities behavior.
Anomaly assessment module is in terms of known network attacks threat level assessment, in conjunction with the abnormal flow of known network attack Feature takes big weight to the feature of strong correlation, and weak relevant feature takes small weight, and the weight of uncorrelated features takes zero.By each spy The initial weight parameter of sign item is both configured to 1, and for the traffic characteristic of strong correlation, weighting parameter value is double；For incoherent Traffic characteristic, weighting parameter value take zero, and linear summation obtains threat level.It includes wrapping that the output of anomaly assessment module, which tests and analyzes, Include event information, evidence information, related data, doubtful event etc..
In Fig. 3, PCAP retains the crawl and information collection that system realizes network flow data packet, attack disaggregated model master DDoS, crawler, detection scanning, password attempt login etc. are detected, universal network discharge model refers to the multidimensional that highest priority uses Characteristic statistics threshold model.

Claims (10)

1. a kind of abnormal communications and liaison behavioral value system based on stream, is arranged on the server of Intranet, feature exists In, the system include configuration management module, data acquisition module, data memory module, highest priority abnormality detection module and General objectives abnormality detection module；
Configuration management module is used for: (1) configuring highest priority IP and general objectives IP；Highest priority is reported and submitted or is monitored by user The IP address in the internal network that network flow data obtains is analyzed, the IP address that highest priority is removed in internal network is general mesh Mark；(2) white list IP is configured, legal, the safe opposite end IP of access internal network is recorded in white list；(3) emphasis mesh is configured The stabilization port for marking IP configures stable port according to the service of known highest priority IP exploitation and port；
The router or network shunt equipment of data acquisition module docking internal network；Receiving router or network shunt equipment are matched Netflow/netstream function is set, acquisition network flow data is sent to data acquisition module；Data acquisition module is to acquisition Flow data is parsed, and the streaming data information that parsing obtains is written to data memory module in a manner of kafka；The stream Data information include source IP address, purpose IP address, source port, destination port, the time started, the end time, protocol type, TCP flag bit, packet number and byte number；
Data memory module reads streaming data information from kafka message queue, all to every streaming data information of highest priority It is stored, to general objectives using 5 minutes or 1 hour as minimum particle size, counts the flow-time sequence of each port；It is described Flow-time sequence storage five-tuple (source IP address, purpose IP address, source port, destination port, protocol type) and wrap Number, byte number, the packet number and byte number recorded is the value changed over time, and each value recorded is one in minimum particle size Statistical value；
Highest priority abnormality detection module is used for: (1) obtaining the communication time of communication object from history streaming data information and lead to Believe byte number, constructs the stabilized communication object model of highest priority；(2) multidimensional characteristic vectors of highest priority are constructed, wherein special Levying item includes stream direction, flow network protocol type, service port number and flow indicator, and each spy is counted from history streaming data information Levy the value of item；The hypothesis testing of normal distribution and two kinds of regularities of distribution of logarithm normal distribution is carried out to the statistical value of each characteristic item, For the characteristic item of Normal Distribution rule, mean value and standard deviation are calculated as statistical threshold, for obeying lognormal point The characteristic item of cloth rule calculates logarithmic average and logarithm standard deviation as statistical threshold, establishes threshold model；(3) threshold value is utilized Model detects band detection flows；When detecting, each characteristic item in each time window is obtained in measurement of discharge to be checked Observation calculates its mean value and standard deviation or logarithmic average and logarithm standard deviation, so according to the Statistical Distribution of characteristic item It is compared afterwards with statistical threshold corresponding in threshold model, calculates departure degree；Departure degree by measurement of discharge to be checked mean value/ Difference between logarithmic average and mean value/logarithmic average of threshold model ,/logarithm standard deviation poorer than upper threshold value model Plays, institute The multiple of acquisition determines；
General objectives abnormality detection module is used for: (1) being detected using the changes in flow rate model based on time series, comprising: To the time series of port flow, tendency component and cyclical component therein are subtracted, obtains random fluctuation feature, random wave Dynamic feature meets the definition of normal distribution, according to confidence level, calculates random fluctuation feature using normal distribution hypothesis testing and deviates The coefficient of standard deviation finds flow and uprushes a little；(2) it is detected using flow data polymerization model, the flow data polymerize mould Type chooses different grouping from five-tuple and carries out varigrained building, is grouped according to selected granularity stream data information, Converging operation is carried out to byte number and packet number again, abnormal behaviour is found by sequence.
2. detection system according to claim 1, which is characterized in that the data memory module, which provides, is based on IP address Data retrieval mode, be also filtered based on white list stream data information.
3. detection system according to claim 1 or 2, which is characterized in that the system further includes anomaly assessment module, Anomaly assessment module synthesis highest priority abnormality detection result and general objectives abnormality detection result, items when to abnormal occurring Feature is weighted summation, comprehensive assessment exception level and Exception Type, generates abnormality detection report.
4. a kind of abnormal communications and liaison behavioral value method based on stream, for Intranet characterized by comprising
Step 1: the highest priority and general objectives of analysis network flow data configuration internal network are reported and submitted or monitored according to user, is matched Set the stabilization port of white list IP and highest priority IP；
Wherein, highest priority is the IP address node for needing emphasis to monitor in internal network, and emphasis mesh will be removed in internal network Target IP node is as general objectives；IP in white list is the IP address in the external network of legitimate secure；Highest priority IP Stabilization port be to be configured according to the open service of corresponding IP node and port；
Step 2: the critical point router or network shunt equipment of internal network acquire net using netflow/netstream function Network flow data and the server being sent to where data acquisition module, data acquisition module grab network flow data packet in real time and go forward side by side Row parsing, obtains streaming data information, and the mode of kafka is written to data memory module；
The streaming data information include source IP address, purpose IP address, source port, destination port, the time started, at the end of Between, protocol type, TCP flag bit, packet number and byte number；；
Step 3: the streaming data information obtained to step 2 is filtered according to white list IP, and for highest priority IP, storage is every Streaming data information；
Step 4: for each highest priority, different port and opposite end IP are obtained from history streaming data information for the emphasis mesh Target communication time and communication byte number are obtained using clustering algorithm and are stablized port and stablize peer IP address；
Step 5: to each highest priority, traffic characteristic, including stream direction, flow network protocol class are extracted in stream data information Type, service port number and flow indicator；It whether is to stablize port or stabilization using extracted traffic characteristic and Correspondent Node Opposite end IP combines the feature vector to form the different dimensions of highest priority；According to the history streaming data information of highest priority, to spy Each characteristic value of sign vector is counted；
Wherein, stream direction refers to, when the source address that highest priority is initiated as connection, the direction of the data stream is outflow, Conversely, the direction of the data stream is to flow into；Flow network protocol type refers to that the flow data that highest priority is communicated with opposite end is in net The agreement run in network layers；Service port number refers to, when highest priority is communicated with opposite end, the port of highest priority side Number；Flow indicator refers to, the uninterrupted of highest priority and opposite end, including attribute have byte number, network packet number and opposite end IP Number；
Step 6: to each highest priority, judging the regularity of distribution of each traffic characteristic, calculate the statistical threshold of each traffic characteristic, build Vertical threshold model；
The hypothesis testing that normal distribution and two kinds of regularities of distribution of logarithm normal distribution are carried out to the statistical value on traffic characteristic, for The traffic characteristic of Normal Distribution rule calculates its mean value and standard deviation as statistical threshold, for obeying lognormal point The traffic characteristic of cloth rule calculates its logarithmic average and logarithm standard deviation as statistical threshold；
Step 7: using the threshold model of the traffic characteristic for the highest priority established, treats detection flows and carry out irrelevance calculating, And then judge whether measurement of discharge to be checked is abnormal flow；
The observation for obtaining each traffic characteristic in each time window in measurement of discharge to be checked, according to the statistical distribution of traffic characteristic Rule calculates its mean value and standard deviation or logarithmic average and logarithm standard deviation, then with statistics threshold corresponding in threshold model Value is compared, and calculates departure degree；Departure degree by measurement of discharge to be checked mean value/logarithmic average and threshold model mean value/ Difference between logarithmic average ,/logarithm standard deviation poorer than upper threshold value model Plays, multiple obtained determine；
Step 8: for the traffic flow information of general objectives, according to IP segment limit, the Exception Type of required detection and preset Packet Filtering rule is screened；
Step 9: by the way of multi-process or multithreading, using 5 minutes or 1 hour as minimum particle size, it is each to calculate general objectives Flow on a port forms the flow-time sequence of each port；
The flow-time sequence is according to five-tuple (source IP address, purpose IP address, source port, destination port, protocol class Type) it is divided, the feature of storage includes packet number, byte number, and the packet number and byte number recorded is the value changed over time, institute Each value of record is a statistical value in minimum particle size；
Step 10: to each port flow time series of general objectives, tendency component and cyclical component therein are subtracted, Using residual components as the random fluctuation feature of general objectives, random fluctuation feature meets the definition of normal distribution, according to confidence Degree calculates its coefficient for deviateing standard deviation using normal distribution hypothesis testing, finds flow and uprush a little；
Step 11: by 1 hour or 1 day dimension, counting the Correspondent Node of general objectives IP, access the number of internal network IP node Amount and the connection frequency, and are ranked up Correspondent Node, find that there are port scan is abnormal and the opposite end IP of IP sections of scan abnormalities； To the opposite end IP found, is detected using flow data polymerization model, judge whether there is abnormal behaviour；The flow data polymerization Model is carried out according to five-tuple (source IP address, purpose IP address, source port, destination port, protocol type) stream data information Grouping, and converging operation is carried out to byte number and packet number, then exceptional value is found by sequence；
Step 12: to the opposite end IP for accessing multiple general objectives, being scanned analysis and DDoS analysis, detected whether that port is swept Retouch the behavior with logon attempt；
Step 13: the abnormal aggression event of association highest priority and general objectives carries out comprehensive assessment to anomalous event.
5. according to the method described in claim 4, it is characterized in that, passing through monitoring analysis network fluxion in the step 1 According to configuration highest priority when, the flow data for flowing through internal network interface is acquired, parse streaming data information, obtain source IP, Destination IP, source port, destination port and protocol type；If being counted after time T acquisition to the streaming data information of acquisition The IP address externally serviced for a long time is selected in analysis according to preset threshold, and selected IP address is highest priority server.
6. according to the method described in claim 4, it is characterized in that, obtaining stabilized end using clustering algorithm in the step 4 Mouth and the method for stablizing peer IP address include:
Each be used as a sample with highest priority IP communication object, all have communication time tn with communicate byte number pbtnum Two attributive character, if two object samples are xi=(tni,pbtnumi) and xj=(tnj,pbtnumj), i, j are two samples Number, tni、tnjIt is the communication time of two samples, pbtnum respectivelyi、pbtnumjIt is the communication byte of two samples respectively It counts, then the similitude of two samples distance dist (xi,xj) measure；
Sample is divided into 4 classes, initial center is set, comprising: to long-time stable appearance and the biggish classification of flow, selects tn Value and all maximum sample of pbtnum value are as initial center；To long-time stable appearance and the general object of flow, tn is selected Value is maximum and is located at the sample of pbtnum mean value as initial center；Occur to the short time and flow is biggish, selects pbtnum Value is maximum and is located at the sample of tn mean value as initial center；To the short time occur and flow in general, selection tn value with The all the smallest sample of pbtnum value is as initial center；
Using clustering algorithm, each sample is calculated to the distance of 4 initial cluster centers, sample is divided into apart from nearest class In cluster, class cluster center is then updated, cluster process is iterated, until 4 class cluster centers no longer change, wherein long-time stable In appearance and the biggish classification of flow is to stablize port and stablize peer IP address.
7. the method according to claim 4, which is characterized in that in the step 5, traffic characteristic is constituting weight When the feature vector of the different dimensions of point target, it is the traffic characteristic for not distinguishing attribute that wherein one or more, which are arranged,.
8. the method according to claim 4, which is characterized in that in the step 8, to the data of general objectives The rule that stream information is screened includes:
1) flow data for port numbers less than 10000 filters out the streaming data information that wherein outflow flow is greater than 300 bytes；
2) for the uplink traffic of 80/443 port, using white and black list mode, there are the flows of suspected attack for acquisition；
3) port numbers are greater than with 10000 port, other than common port, remaining only obtains as random port and be greater than 600 The streaming data information of byte, common port include 11211,27017.
9. according to the method described in claim 4, it is characterized in that, in the step 10, to every time of a port Sequence Y executes operation below:
1) time series Y is subtracted into trend variable, trend variable initial value is 0；
2) the local weighted Return Law loess of application carries out each period subsequence smooth；Using day as the period, it was with 5 minutes Time granularity then has 12*24=288 period subsequence in one day；
3) it is window size with each period mileage strong point number 288, each time granularity is obtained to smoothed out All Time sequence C On Sliding Mean Number, form new time series, while loess is done to new time series and smoothly obtains L；
4) periodic component S=C-L is obtained；
5) time series Y is subtracted into periodic component S, carries out periodicity；
6) to going the time series progress loess after periodicity smooth, trend component T is obtained；
7) residual components R=Y-T-S, as random fluctuation feature flow.
10. the method according to claim 4, which is characterized in that in the step 11, chosen from five-tuple Different grouping constructs flow data polymerization model, comprising:
It is grouped according to sport, byte number, packet number in polymerization grouping；
It is grouped according to sip, sport, byte number, packet number in polymerization grouping；
According to the C section of sip, sport is grouped, byte number, packet number in polymerization grouping；
According to sip, dip, sport is grouped, byte number, packet number in polymerization grouping；
It is grouped according to sip, dip, sport, dport, byte number, packet number in polymerization grouping；
According to the C section of sip, dip, sport, dport is grouped, byte number, packet number in polymerization grouping；
Wherein sip is source ip, and sport is source port, and dip is purpose ip, and dport is destination port；From sip, dip, sport, The various combination of dport detects various forms of abnormal behaviours.
CN201910469616.6A 2019-05-31 2019-05-31 Abnormal communication behavior detection method and system based on flow Active CN110149343B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910469616.6A CN110149343B (en) 2019-05-31 2019-05-31 Abnormal communication behavior detection method and system based on flow

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910469616.6A CN110149343B (en) 2019-05-31 2019-05-31 Abnormal communication behavior detection method and system based on flow

Publications (2)

Publication Number Publication Date
CN110149343A true CN110149343A (en) 2019-08-20
CN110149343B CN110149343B (en) 2021-07-16

Family

ID=67590131

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910469616.6A Active CN110149343B (en) 2019-05-31 2019-05-31 Abnormal communication behavior detection method and system based on flow

Country Status (1)

Country Link
CN (1) CN110149343B (en)

Cited By (51)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110933082A (en) * 2019-11-29 2020-03-27 深信服科技股份有限公司 Method, device and equipment for identifying lost host and storage medium
CN110943974A (en) * 2019-11-06 2020-03-31 国网上海市电力公司 DDoS (distributed denial of service) anomaly detection method and cloud platform host
CN110943883A (en) * 2019-11-13 2020-03-31 深圳市东进技术股份有限公司 Network flow statistical method, system, gateway and computer readable storage medium
CN110995769A (en) * 2020-02-27 2020-04-10 上海飞旗网络技术股份有限公司 Deep data packet detection method and device and readable storage medium
CN111092865A (en) * 2019-12-04 2020-05-01 全球能源互联网研究院有限公司 Security event analysis method and system
CN111092900A (en) * 2019-12-24 2020-05-01 北京北信源软件股份有限公司 Method and device for monitoring abnormal connection and scanning behavior of server
CN111182087A (en) * 2019-12-18 2020-05-19 哈尔滨工业大学（威海） Flow playback method based on single network card binding multiple IPs
CN111556066A (en) * 2020-05-08 2020-08-18 国家计算机网络与信息安全管理中心 Network behavior detection method and device
CN111683102A (en) * 2020-06-17 2020-09-18 绿盟科技集团股份有限公司 FTP behavior data processing method, and method and device for identifying abnormal FTP behavior
CN111800411A (en) * 2020-07-02 2020-10-20 支付宝(杭州)信息技术有限公司 Privacy-protecting business prediction model joint updating method and device
CN111818049A (en) * 2020-07-08 2020-10-23 宝牧科技（天津）有限公司 Botnet flow detection method and system based on Markov model
CN111818050A (en) * 2020-07-08 2020-10-23 腾讯科技（深圳）有限公司 Target access behavior detection method, system, device, equipment and storage medium
CN112087350A (en) * 2020-09-17 2020-12-15 中国工商银行股份有限公司 Method, device, system and medium for monitoring network access line flow
CN112181696A (en) * 2020-08-31 2021-01-05 五八到家有限公司 Abnormal information processing method, equipment and storage medium
CN112257760A (en) * 2020-09-30 2021-01-22 北京航空航天大学 Method for detecting abnormal network communication behavior of host based on time sequence die body
CN112329839A (en) * 2020-11-03 2021-02-05 北京理工大学 Encrypted flow refined classification method based on one-way burst flow characteristics
US20210058424A1 (en) * 2019-08-21 2021-02-25 Nokia Solutions And Networks Oy Anomaly detection for microservices
CN112565225A (en) * 2020-11-27 2021-03-26 北京百度网讯科技有限公司 Method and device for data transmission, electronic equipment and readable storage medium
CN113037748A (en) * 2021-03-08 2021-06-25 中国科学院信息工程研究所 C and C channel hybrid detection method and system
CN113079143A (en) * 2021-03-24 2021-07-06 北京锐驰信安技术有限公司 Flow data-based anomaly detection method and system
CN113259367A (en) * 2021-05-28 2021-08-13 苏州联电能源发展有限公司 Industrial control network flow multistage anomaly detection method and device
CN113271322A (en) * 2021-07-20 2021-08-17 北京明略软件系统有限公司 Abnormal flow detection method and device, electronic equipment and storage medium
CN113656535A (en) * 2021-08-31 2021-11-16 上海观安信息技术股份有限公司 Abnormal session detection method and device and computer storage medium
CN113765921A (en) * 2021-09-08 2021-12-07 沈阳理工大学 Abnormal flow grading detection method for industrial Internet of things
WO2021244415A1 (en) * 2020-06-03 2021-12-09 华为技术有限公司 Network failure detection method and apparatus
CN113810335A (en) * 2020-06-12 2021-12-17 武汉斗鱼鱼乐网络科技有限公司 Method and system for identifying target IP, storage medium and equipment
CN113824740A (en) * 2021-11-23 2021-12-21 山东云天安全技术有限公司 Port detection method, electronic device and computer-readable storage medium
CN113852603A (en) * 2021-08-13 2021-12-28 京东科技信息技术有限公司 Method and device for detecting abnormality of network traffic, electronic equipment and readable medium
CN113904795A (en) * 2021-08-27 2022-01-07 北京工业大学 Rapid and accurate flow detection method based on network security probe
CN113923051A (en) * 2021-11-12 2022-01-11 国网河南省电力公司漯河供电公司 Novel intranet abnormal IP (Internet protocol) discovery technology
CN114244727A (en) * 2021-12-15 2022-03-25 国网辽宁省电力有限公司沈阳供电公司 Instant generation method and system for power Internet of things communication panorama
CN114301694A (en) * 2021-12-29 2022-04-08 赛尔网络有限公司 Network abnormal flow analysis method, device, equipment and medium
CN114301668A (en) * 2021-12-28 2022-04-08 北京安天网络安全技术有限公司 Flow detection method and device, electronic equipment and computer readable storage medium
CN114401516A (en) * 2022-01-11 2022-04-26 国家计算机网络与信息安全管理中心 5G slice network anomaly detection method based on virtual network traffic analysis
CN114499987A (en) * 2021-12-29 2022-05-13 云南电网有限责任公司信息中心 Network abnormal IP and port hybrid detection method based on relative density
CN114615078A (en) * 2022-03-30 2022-06-10 中国农业银行股份有限公司 DDoS attack detection method, device and equipment
CN114650232A (en) * 2020-12-02 2022-06-21 中盈优创资讯科技有限公司 Network quality analysis method and device based on QOS queue flow
CN114760126A (en) * 2022-04-08 2022-07-15 沈阳化工大学 Industrial control network flow real-time intrusion detection method
CN114760131A (en) * 2022-04-15 2022-07-15 中国人民解放军国防科技大学 Feature extraction method, device and equipment for return programming flow
CN114760152A (en) * 2022-06-14 2022-07-15 湖南警察学院 Cloud data center virtualization node network security early warning method
CN114826895A (en) * 2022-04-24 2022-07-29 金祺创（北京）技术有限公司 Large-scale backbone intranet NAT (network Address translation) flow big data intelligent analysis alarm positioning method and monitoring system
CN115037654A (en) * 2022-05-09 2022-09-09 维沃移动通信有限公司 Flow statistical method and device, electronic equipment and readable storage medium
CN115103000A (en) * 2022-06-20 2022-09-23 北京鼎兴达信息科技股份有限公司 Method for restoring and analyzing business session of railway data network based on NetStream
CN115174190A (en) * 2022-06-29 2022-10-11 武汉极意网络科技有限公司 Information security management and control system and method based on network traffic
CN115361231A (en) * 2022-10-19 2022-11-18 中孚安全技术有限公司 Access baseline-based host abnormal traffic detection method, system and equipment
CN116094842A (en) * 2023-04-07 2023-05-09 北京豪密科技有限公司 State recognition system and method of network cipher machine
CN116633752A (en) * 2023-04-07 2023-08-22 南京和子祥企业管理有限公司 Analysis management system based on big data
CN116668085A (en) * 2023-05-05 2023-08-29 山东省计算中心（国家超级计算济南中心） Flow multi-process intrusion detection method and system based on lightGBM
TWI823657B (en) * 2022-11-02 2023-11-21 中華電信股份有限公司 Monitoring system and monitoring method for abnormal behavior of user equipment
CN117395070A (en) * 2023-11-16 2024-01-12 国家计算机网络与信息安全管理中心 Abnormal flow detection method based on flow characteristics
TWI835113B (en) 2022-04-08 2024-03-11 彰化商業銀行股份有限公司 System for executing task based on an analysis result of records for achieving device joint defense and method thereof

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070043851A1 (en) * 2005-08-16 2007-02-22 Netdevices, Inc. Facilitating a user to detect desired anomalies in data flows of networks
CN107231345A (en) * 2017-05-03 2017-10-03 成都国腾实业集团有限公司 Networks congestion control methods of risk assessment based on AHP
CN107277039A (en) * 2017-07-18 2017-10-20 河北省科学院应用数学研究所 A kind of network attack data analysis and intelligent processing method
CN107426040A (en) * 2017-09-20 2017-12-01 华中科技大学 A kind of Forecasting Methodology of network behavior
CN107483455A (en) * 2017-08-25 2017-12-15 国家计算机网络与信息安全管理中心 A kind of network node abnormality detection method and system based on stream
CN108494746A (en) * 2018-03-07 2018-09-04 长安通信科技有限责任公司 A kind of network port Traffic anomaly detection method and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070043851A1 (en) * 2005-08-16 2007-02-22 Netdevices, Inc. Facilitating a user to detect desired anomalies in data flows of networks
CN107231345A (en) * 2017-05-03 2017-10-03 成都国腾实业集团有限公司 Networks congestion control methods of risk assessment based on AHP
CN107277039A (en) * 2017-07-18 2017-10-20 河北省科学院应用数学研究所 A kind of network attack data analysis and intelligent processing method
CN107483455A (en) * 2017-08-25 2017-12-15 国家计算机网络与信息安全管理中心 A kind of network node abnormality detection method and system based on stream
CN107426040A (en) * 2017-09-20 2017-12-01 华中科技大学 A kind of Forecasting Methodology of network behavior
CN108494746A (en) * 2018-03-07 2018-09-04 长安通信科技有限责任公司 A kind of network port Traffic anomaly detection method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
贺亮等: "基于通联累积量的动态网络异常检测算法", 《通信技术》 *

Cited By (82)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11811801B2 (en) * 2019-08-21 2023-11-07 Nokia Solutions And Networks Oy Anomaly detection for microservices
US20210058424A1 (en) * 2019-08-21 2021-02-25 Nokia Solutions And Networks Oy Anomaly detection for microservices
CN110943974A (en) * 2019-11-06 2020-03-31 国网上海市电力公司 DDoS (distributed denial of service) anomaly detection method and cloud platform host
CN110943883A (en) * 2019-11-13 2020-03-31 深圳市东进技术股份有限公司 Network flow statistical method, system, gateway and computer readable storage medium
CN110943883B (en) * 2019-11-13 2023-01-31 深圳市东进技术股份有限公司 Network flow statistical method, system, gateway and computer readable storage medium
CN110933082A (en) * 2019-11-29 2020-03-27 深信服科技股份有限公司 Method, device and equipment for identifying lost host and storage medium
CN111092865A (en) * 2019-12-04 2020-05-01 全球能源互联网研究院有限公司 Security event analysis method and system
CN111182087A (en) * 2019-12-18 2020-05-19 哈尔滨工业大学（威海） Flow playback method based on single network card binding multiple IPs
CN111182087B (en) * 2019-12-18 2022-01-28 哈尔滨工业大学（威海） Flow playback method based on single network card binding multiple IPs
CN111092900B (en) * 2019-12-24 2022-04-05 北京北信源软件股份有限公司 Method and device for monitoring abnormal connection and scanning behavior of server
CN111092900A (en) * 2019-12-24 2020-05-01 北京北信源软件股份有限公司 Method and device for monitoring abnormal connection and scanning behavior of server
CN110995769B (en) * 2020-02-27 2020-06-05 上海飞旗网络技术股份有限公司 Deep data packet detection method and device
CN110995769A (en) * 2020-02-27 2020-04-10 上海飞旗网络技术股份有限公司 Deep data packet detection method and device and readable storage medium
CN111556066A (en) * 2020-05-08 2020-08-18 国家计算机网络与信息安全管理中心 Network behavior detection method and device
WO2021244415A1 (en) * 2020-06-03 2021-12-09 华为技术有限公司 Network failure detection method and apparatus
CN113810335B (en) * 2020-06-12 2023-08-22 武汉斗鱼鱼乐网络科技有限公司 Method and system for identifying target IP, storage medium and equipment
CN113810335A (en) * 2020-06-12 2021-12-17 武汉斗鱼鱼乐网络科技有限公司 Method and system for identifying target IP, storage medium and equipment
CN111683102B (en) * 2020-06-17 2022-12-06 绿盟科技集团股份有限公司 FTP behavior data processing method, and method and device for identifying abnormal FTP behavior
CN111683102A (en) * 2020-06-17 2020-09-18 绿盟科技集团股份有限公司 FTP behavior data processing method, and method and device for identifying abnormal FTP behavior
CN111800411A (en) * 2020-07-02 2020-10-20 支付宝(杭州)信息技术有限公司 Privacy-protecting business prediction model joint updating method and device
CN111800411B (en) * 2020-07-02 2021-04-02 支付宝(杭州)信息技术有限公司 Privacy-protecting business prediction model joint updating method and device
CN111818050A (en) * 2020-07-08 2020-10-23 腾讯科技（深圳）有限公司 Target access behavior detection method, system, device, equipment and storage medium
CN111818050B (en) * 2020-07-08 2024-01-19 腾讯科技（深圳）有限公司 Target access behavior detection method, system, device, equipment and storage medium
CN111818049A (en) * 2020-07-08 2020-10-23 宝牧科技（天津）有限公司 Botnet flow detection method and system based on Markov model
CN112181696A (en) * 2020-08-31 2021-01-05 五八到家有限公司 Abnormal information processing method, equipment and storage medium
CN112181696B (en) * 2020-08-31 2023-05-12 五八到家有限公司 Abnormality information processing method, apparatus, and storage medium
CN112087350A (en) * 2020-09-17 2020-12-15 中国工商银行股份有限公司 Method, device, system and medium for monitoring network access line flow
CN112257760A (en) * 2020-09-30 2021-01-22 北京航空航天大学 Method for detecting abnormal network communication behavior of host based on time sequence die body
CN112257760B (en) * 2020-09-30 2022-06-03 北京航空航天大学 Method for detecting abnormal network communication behavior of host based on time sequence die body
CN112329839A (en) * 2020-11-03 2021-02-05 北京理工大学 Encrypted flow refined classification method based on one-way burst flow characteristics
CN112329839B (en) * 2020-11-03 2022-02-08 北京理工大学 Encrypted flow refined classification method based on one-way burst flow characteristics
CN112565225B (en) * 2020-11-27 2022-08-12 北京百度网讯科技有限公司 Method and device for data transmission, electronic equipment and readable storage medium
CN112565225A (en) * 2020-11-27 2021-03-26 北京百度网讯科技有限公司 Method and device for data transmission, electronic equipment and readable storage medium
CN114650232B (en) * 2020-12-02 2024-03-12 中盈优创资讯科技有限公司 Network quality analysis method and device based on QOS queue flow
CN114650232A (en) * 2020-12-02 2022-06-21 中盈优创资讯科技有限公司 Network quality analysis method and device based on QOS queue flow
CN113037748A (en) * 2021-03-08 2021-06-25 中国科学院信息工程研究所 C and C channel hybrid detection method and system
CN113079143A (en) * 2021-03-24 2021-07-06 北京锐驰信安技术有限公司 Flow data-based anomaly detection method and system
CN113259367B (en) * 2021-05-28 2022-05-06 苏州联电能源发展有限公司 Industrial control network flow multistage anomaly detection method and device
CN113259367A (en) * 2021-05-28 2021-08-13 苏州联电能源发展有限公司 Industrial control network flow multistage anomaly detection method and device
CN113271322A (en) * 2021-07-20 2021-08-17 北京明略软件系统有限公司 Abnormal flow detection method and device, electronic equipment and storage medium
CN113271322B (en) * 2021-07-20 2021-11-23 北京明略软件系统有限公司 Abnormal flow detection method and device, electronic equipment and storage medium
CN113852603A (en) * 2021-08-13 2021-12-28 京东科技信息技术有限公司 Method and device for detecting abnormality of network traffic, electronic equipment and readable medium
CN113852603B (en) * 2021-08-13 2023-11-07 京东科技信息技术有限公司 Abnormality detection method and device for network traffic, electronic equipment and readable medium
CN113904795A (en) * 2021-08-27 2022-01-07 北京工业大学 Rapid and accurate flow detection method based on network security probe
CN113656535B (en) * 2021-08-31 2023-11-14 上海观安信息技术股份有限公司 Abnormal session detection method and device and computer storage medium
CN113656535A (en) * 2021-08-31 2021-11-16 上海观安信息技术股份有限公司 Abnormal session detection method and device and computer storage medium
CN113765921B (en) * 2021-09-08 2023-04-07 沈阳理工大学 Abnormal flow grading detection method for industrial Internet of things
CN113765921A (en) * 2021-09-08 2021-12-07 沈阳理工大学 Abnormal flow grading detection method for industrial Internet of things
CN113923051A (en) * 2021-11-12 2022-01-11 国网河南省电力公司漯河供电公司 Novel intranet abnormal IP (Internet protocol) discovery technology
CN113824740B (en) * 2021-11-23 2022-03-04 山东云天安全技术有限公司 Port detection method, electronic device and computer-readable storage medium
CN113824740A (en) * 2021-11-23 2021-12-21 山东云天安全技术有限公司 Port detection method, electronic device and computer-readable storage medium
CN114244727A (en) * 2021-12-15 2022-03-25 国网辽宁省电力有限公司沈阳供电公司 Instant generation method and system for power Internet of things communication panorama
CN114301668A (en) * 2021-12-28 2022-04-08 北京安天网络安全技术有限公司 Flow detection method and device, electronic equipment and computer readable storage medium
CN114301694B (en) * 2021-12-29 2024-03-15 赛尔网络有限公司 Network abnormal flow analysis method, device, equipment and medium
CN114301694A (en) * 2021-12-29 2022-04-08 赛尔网络有限公司 Network abnormal flow analysis method, device, equipment and medium
CN114499987A (en) * 2021-12-29 2022-05-13 云南电网有限责任公司信息中心 Network abnormal IP and port hybrid detection method based on relative density
CN114401516B (en) * 2022-01-11 2024-05-10 国家计算机网络与信息安全管理中心 5G slice network anomaly detection method based on virtual network traffic analysis
CN114401516A (en) * 2022-01-11 2022-04-26 国家计算机网络与信息安全管理中心 5G slice network anomaly detection method based on virtual network traffic analysis
CN114615078A (en) * 2022-03-30 2022-06-10 中国农业银行股份有限公司 DDoS attack detection method, device and equipment
CN114760126A (en) * 2022-04-08 2022-07-15 沈阳化工大学 Industrial control network flow real-time intrusion detection method
CN114760126B (en) * 2022-04-08 2023-09-19 沈阳化工大学 Industrial control network flow real-time intrusion detection method
TWI835113B (en) 2022-04-08 2024-03-11 彰化商業銀行股份有限公司 System for executing task based on an analysis result of records for achieving device joint defense and method thereof
CN114760131A (en) * 2022-04-15 2022-07-15 中国人民解放军国防科技大学 Feature extraction method, device and equipment for return programming flow
CN114760131B (en) * 2022-04-15 2024-03-01 中国人民解放军国防科技大学 Feature extraction method, device and equipment for return type programming flow
CN114826895A (en) * 2022-04-24 2022-07-29 金祺创（北京）技术有限公司 Large-scale backbone intranet NAT (network Address translation) flow big data intelligent analysis alarm positioning method and monitoring system
CN115037654B (en) * 2022-05-09 2024-01-09 维沃移动通信有限公司 Flow statistics method, device, electronic equipment and readable storage medium
CN115037654A (en) * 2022-05-09 2022-09-09 维沃移动通信有限公司 Flow statistical method and device, electronic equipment and readable storage medium
CN114760152A (en) * 2022-06-14 2022-07-15 湖南警察学院 Cloud data center virtualization node network security early warning method
CN114760152B (en) * 2022-06-14 2022-08-19 湖南警察学院 Cloud data center virtualization node network security early warning method
CN115103000A (en) * 2022-06-20 2022-09-23 北京鼎兴达信息科技股份有限公司 Method for restoring and analyzing business session of railway data network based on NetStream
CN115103000B (en) * 2022-06-20 2023-09-26 北京鼎兴达信息科技股份有限公司 Method for restoring and analyzing business session of railway data network based on NetStream
CN115174190A (en) * 2022-06-29 2022-10-11 武汉极意网络科技有限公司 Information security management and control system and method based on network traffic
CN115174190B (en) * 2022-06-29 2024-01-26 武汉极意网络科技有限公司 Information security management and control system and method based on network traffic
CN115361231A (en) * 2022-10-19 2022-11-18 中孚安全技术有限公司 Access baseline-based host abnormal traffic detection method, system and equipment
TWI823657B (en) * 2022-11-02 2023-11-21 中華電信股份有限公司 Monitoring system and monitoring method for abnormal behavior of user equipment
CN116633752A (en) * 2023-04-07 2023-08-22 南京和子祥企业管理有限公司 Analysis management system based on big data
CN116094842A (en) * 2023-04-07 2023-05-09 北京豪密科技有限公司 State recognition system and method of network cipher machine
CN116094842B (en) * 2023-04-07 2023-06-06 北京豪密科技有限公司 State recognition system and method of network cipher machine
CN116668085B (en) * 2023-05-05 2024-02-27 山东省计算中心（国家超级计算济南中心） Flow multi-process intrusion detection method and system based on lightGBM
CN116668085A (en) * 2023-05-05 2023-08-29 山东省计算中心（国家超级计算济南中心） Flow multi-process intrusion detection method and system based on lightGBM
CN117395070A (en) * 2023-11-16 2024-01-12 国家计算机网络与信息安全管理中心 Abnormal flow detection method based on flow characteristics
CN117395070B (en) * 2023-11-16 2024-05-03 国家计算机网络与信息安全管理中心 Abnormal flow detection method based on flow characteristics

Also Published As

Publication number Publication date
CN110149343B (en) 2021-07-16

Similar Documents

Publication Publication Date Title
CN110149343A (en) A kind of abnormal communications and liaison behavioral value method and system based on stream
CN114257386B (en) Training method, system, equipment and storage medium for detection model
Xia et al. An efficient network intrusion detection method based on information theory and genetic algorithm
Elsayed et al. Machine-learning techniques for detecting attacks in SDN
CN106973038B (en) Network intrusion detection method based on genetic algorithm oversampling support vector machine
CN101924757B (en) Method and system for reviewing Botnet
CN107579956B (en) User behavior detection method and device
Husain et al. Development of an efficient network intrusion detection model using extreme gradient boosting (XGBoost) on the UNSW-NB15 dataset
CN106357622B (en) Exception flow of network based on software defined network detects system of defense
CN108289088A (en) Abnormal traffic detection system and method based on business model
Hassan Network intrusion detection system using genetic algorithm and fuzzy logic
CN108494746A (en) A kind of network port Traffic anomaly detection method and system
CN106663169A (en) System and method for high speed threat intelligence management using unsupervised machine learning and prioritization algorithms
RU2682108C1 (en) Method of using options of countermeasure of network and stream computer intelligence and network attacks and system therefor
Akbar et al. Intrusion detection system methodologies based on data analysis
CN107294966A (en) A kind of IP white list construction methods based on Intranet flow
CN109361673A (en) Network anomaly detection method based on data on flows sample statistics and balance comentropy estimation
Nehinbe Log Analyzer for Network Forensics and Incident Reporting
Badajena et al. Incorporating hidden Markov model into anomaly detection technique for network intrusion detection
Chawla et al. Discrimination of DDoS attacks and flash events using Pearson’s product moment correlation method
Odusami et al. A survey and meta‐analysis of application‐layer distributed denial‐of‐service attack
CN115225384B (en) Network threat degree evaluation method and device, electronic equipment and storage medium
Al-Sanjary et al. Comparison and detection analysis of network traffic datasets using K-means clustering algorithm
CN116996286A (en) Network attack and security vulnerability management framework platform based on big data analysis
Hnamte et al. An extensive survey on intrusion detection systems: Datasets and challenges for modern scenario

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant