CN1507233A - Firm gateway system and its attack detecting method - Google Patents
Firm gateway system and its attack detecting method Download PDFInfo
- Publication number
- CN1507233A CN1507233A CNA021553823A CN02155382A CN1507233A CN 1507233 A CN1507233 A CN 1507233A CN A021553823 A CNA021553823 A CN A021553823A CN 02155382 A CN02155382 A CN 02155382A CN 1507233 A CN1507233 A CN 1507233A
- Authority
- CN
- China
- Prior art keywords
- data
- traffic characteristic
- real
- dog
- central host
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
- Computer And Data Communications (AREA)
Abstract
The firm gateway system includes several firm gateway devices, several real-time monitors, one central host computer and one data warehouse. The firm gateway devices are connected to the real-time monitors and the real-time monitors to the central host computer. The firm gateway devices complete data acquisition and abnormal response; the real-time monitors detect the flux characteristic of data, detect attack and control the data filtered by the firm gateway devices; the central host computer obtains the flux characteristic from the data warehouse, generate the flux model and distributes to the real-time monitors; and the data warehouse is used to store the flux characteristics of the real-time monitors. The present invention also discloses the method of applying the firm gateway system to detect attack.
Description
Technical field
The invention belongs to network security technology, relate in particular to DOS/DDOS and detect attack technology.
Background technology
Along with the development of Internet, the means of network attack are increasing.Denial of service (Denial ofService is called for short DOS) is exactly wherein a kind of, the availability that adopts the attack means of this kind mode directly threatening network.Modal dos attack be exactly the assailant at short notice, by sending thousands of junk data or illegal request, flood destination server or objective network, the service request of validated user is rejected.Dos attack comes down to a kind of intrusion behavior to network or system, and still the operation of this intrusion behavior and normal service usually can't be distinguished, and for instance: to a large amount of page request of www server may be exactly a Denial of Service attack.Current, to walk out of at basic enterprising one of DOS and to have showed a kind of evolution version d DOS (Distributed Denial ofService) new, that have more threat, this kind attack pattern has proposed new challenge to network security.DDOS attacks and utilizes the distributed connection characteristics of Internet, produces large-scale packet inundation by the computer that is distributed in all over the world, and one or more computer is attacked.
In the prior art, have following safe practice and product, yet the ability that these products detections DOS/DDOS attacks is very limited:
1, fire compartment wall:
Fire compartment wall is as a kind of safeguard protection product of computer network, employing be a kind of isolated controlling technology: between internal network and unsafe external network such as Internet, place obstacles, stop the unauthorized access of external network internal resource.The major technique that realizes fire compartment wall has Packet Filtering, application gateway and agency service.The Packet Filtering technology is packet to be implemented selectively to pass through in network layer; Application gateway then is the protocol filtering that is based upon on the network application layer; The both determines whether allowing specific packet to pass through by specific logic determines, and agency service then is that the link of the application layer inside and outside the fire compartment wall is fetched realization by two chains that end at acting server.The core of these technology all is the certain rule of configuration in fire compartment wall, check then whether these rules are satisfied to find invasion, and the formulation of rule and change generally is to be finished by hand by the professional.Fire compartment wall can only detect those attacks that can estimate, and for the attack of the unknown, particularly DOS/DDOS attacks and can't effectively detect.And the controlling object of fire compartment wall is single packet, and behavior is difficult to find and control for lot of data.For example, the SNY flood attack during DOS/DDOS attacks, the individual data in the attack all are fully legal on agreement, attack yet a large amount of this packets has just constituted once.
2, intruding detection system (IDS) product:
Intrusion behavior is meant confidentiality, integrality, the availability of those destruction of computer systems or network system or walks around the attempt of security mechanism.IDS product role is exactly by the incident in supervisory computer system or the network system, analyzes the sign of finding invasion.The IDS product is exactly some softwares or hardware product, can finish the process that monitors and analyze automatically, thereby detect intrusion behavior.Many IDS products have been arranged at present, although they adopt supervision different with analytical technology, they all meet IDS common treatment model as shown in Figure 1.
As seen from Figure 1, the IDS product all has 3 most basic functional modules: data source, analysis engine and response, data source module and analysis engine are located on monitor host 11 usually.
The data source module is responsible for collecting data, for example individual host or network etc. from the different levels of system.Accordingly, there are Host Based surveillance 12 and based on network surveillance 13.These two surveillances send response message to respond module 14, respond module 14 control fire compartment wall 15 and router one 6 filtering datas.Analysis engine is a real functional module of being responsible for detecting intrusion behavior in the intruding detection system, and it is by analyzing the incident that data source is submitted to, and whether judgement invasion has taken place, and gives respond module with the result notification that detects.The position of analysis engine can be positioned at individual host, also can be positioned at any interface of internal network.
The detection method that analysis engine adopted of IDS product comprises following two kinds:
A, based on the detection method of knowledge:
Detection method based on knowledge is used the known attack or the known bugs of system, takes out feature, attack whether occurs detecting by judging these features in data.Defective based on the detection method of knowledge is to detect known attack means, when occurring at the attack means of new leak or during at the new attack mode of known bugs, need manually or by other machine learning system to draw corresponding feature, as the foundation that detects, just can make product or system possess the ability that detects the new attack means., usually can't detect owing to lack corresponding feature description for novel attack, thereby need constantly, upgrade timely, could guarantee the completeness of detectability.
B, based on the detection method of behavior:
Based on the detection method of behavior usually with the visit of the activity density of objective network or destination host, file, I/O activity, login frequency, CPU time take and network connection etc. as the object of analysis, pattern (or vector) by learning progressively to set up user behavior judges whether invasion according to user's the behavior or the normal degree of resource behaviour in service.Key based on the detection method of behavior is how to determine normal behavior.In the practical application, the shadow that this detection method very easily is subjected to network environment to, false drop rate is than higher.
But the IDS product detects and has following defective when DOS/DDOS attacks:
(1) surveillance that no matter is based on main frame still is based on network surveillance, and they all are easy to be subjected to DOS/DDOS and attack.In fact Host Based surveillance is exactly a process that operates on the individual host, and the operation of this process need consume the resource of protected main frame, thereby also is the target that DOS/DDOS attacks.In the based on network surveillance in monitor host and the network status of other main frame be the same, if be subjected to the DOS/DDOS attack, can't continue the data on the collection network, also just make the intruding detection system state that paralyses.
(2) respond module is responsible for taking a series of measure to stop attack protection system resource when attacking generation.The most frequently used active response mode is that blocking-up is attacked, and IDS does not have entering of ability blocking-up assailant, can only be blocking-up some to be suspect to be assailant's IP address, by sending TCP RET order or reconfiguring router and fire compartment wall solves.Yet this measure is very dangerous, at first the transmission of response command may be intercepted and captured by the hacker, next sends a large amount of TCP RET orders may be exactly to be used to initiate the DDOS attack, and a large amount of false-alarms cause the performance that can influence network to the frequent configuration of router and protecting wall greatly.
As seen, IDS product itself just very easily is subjected to DOS/DDOS and attacks, and must rely on other safety product to resist attack after attacking generation, self lacks effective countermeasure.
3, the application data digging technology detects the protection scheme of DOS/DDOS:
Data mining technology is exactly to extract inherent, intrinsic, previous unknown and process final intelligible information from mass data.Can use the information that is extracted to form a prediction or disaggregated model, perhaps find out the similitude between data-base recording.The object information of excavating out can help to make stronger decision-making.Thereby data mining technology has been introduced in the existing intruding detection system as a kind of data analysis means.Be that example is introduced the maintenance data digging technology and detected ubiquitous safety defect in method that DOS/DDOS attacks and this scheme below with DuDe.
The English full name of DuDe is Defense Under Denial-of-Service; i.e. defence under dos attack; it be of Columbia University when dos attack takes place to the protect networks scheme, wherein use data mining technology as the method that detects DOS.Referring to shown in Figure 2, system is by the data collection engine (DCA that is distributed in internal network, Data Collection Agent) and data fusion device (DFA, Data Fusion Agent) forms, DFA utilizes mining algorithm to produce discharge model, DCA detects the flow that enters in real time according to this discharge model, notes abnormalities and is correspondingly processed.DFA uses machine learning algorithm and data mining algorithm to produce model, thereby these data sets that are used for training must can abundant network activity that is virtually reality like reality.
This method needs a large amount of data normal data through mark to come training pattern with the attack data.Because it is very high to produce the cost of above-mentioned training data, and need a large amount of manual workings, and, can not well work usually when trained model is used to another environment in an environment, must re-construct training data.Therefore this method is portable poor.Structurally, DCA is actually a software that operates on the Web-Server, and all flows that need enter Web-Server all must just lack the ability that resists for DOS/DDOS attack itself through DCA.
4, other resists the product that DOS/DDOS attacks:
At present, there are many network companies to be devoted to research and develop special-purpose DOS/DDOS attack detecting product replenishing in the world as existing safety measure, mainly contain the AttackMitigator of TopLayer company, the Peakflow DOS of Arbor company, the CaptIO Security Device of Capus company, the Vantage System of Asta Network company and the Mazu Enforcer of Mazu company etc.A common feature of these products is exactly to be an independent equipment; the obvious like this integral status that can not detect protected network; though this defective can obtain remedying to a certain degree by increase a plurality of equipment in network; but can't share information between these equipment, reach the purpose of common defence.For example, when an Equipment Inspection when attack taking place, can't be with other equipment of the result notification that detects.
With regard to the detection method of using, they have all utilized existing Intrusion Detection Technique.Except the AttackMitigator of TopLayer company detects some dos attack by the mechanism of pre-configured known attack, all the other companies have all adopted the abnormality detection mechanism based on behavior, as mentioned above, these two kinds of methods all exist very big safety defect.
Summary of the invention
In view of this, main purpose of the present invention is to provide a kind of robust gateway system, comprise a plurality of robust gateway equipment, a plurality of real-time watch-dog, a central host and a data warehouse, robust gateway equipment, real-time watch-dog, central host connect and compose the tri-level logic structure, each robust gateway equipment and corresponding real-time watch-dog connection, each real-time watch-dog is connected with central host, and central host is connected with data warehouse, wherein:
Each robust gateway equipment all places between external network and the internal network, with respect to internal network and the transparent setting of external network, has data acquisition and data filter function;
The discharge model that each real-time watch-dog all has the detection extracting data traffic characteristic that collects from robust gateway equipment, send according to central host judges whether the present flow rate feature unusual, to robust gateway issue response policy with the function of control robust gateway to filtering from the network data of outside;
Central host has the traffic characteristic that sends according to from data warehouse, generates the function that discharge model and continuous self adaptation generate new discharge model, and simultaneously, the policy configurations that this equipment also will be stored generation traffic characteristic thereon is published on the real-time watch-dog;
Data warehouse obtains the traffic characteristic of the detection data on the real-time watch-dog by central host, is kept in this data warehouse, and when central host generated discharge model, central host obtained traffic characteristic from this data warehouse.
Wherein, described robust gateway equipment comprises:
The function of obtaining raw data packets from INTERNET is finished in the packet capturing storehouse that disposes in this equipment;
The fire compartment wall that disposes in this equipment according to the response policy that obtains from real-time watch-dog, will filter from the initial data that INTERNET obtains, and be sent to internal network.
Wherein, described robust gateway comprises with respect to internal network and the transparent setting of external network:
Robust gateway has and is arranged at the phase same network segment and two interfaces internal network and external network.
Wherein, described real-time watch-dog comprises:
The traffic characteristic extraction module: this module utilizes association algorithm that the detection data that robust gateway equipment transmits are carried out feature extraction according to the strategy of the generation traffic characteristic that obtains from central host, obtains traffic characteristic;
Detect and attack module: this module obtains traffic characteristic from the traffic characteristic extraction module, and from central host acquisition discharge model, utilize the discharge model and the traffic characteristic that obtain to finish detection attack function, finish the function of issue response policy to robust gateway according to testing result.
Wherein, described real-time watch-dog further comprises:
It is visual that warning visualization model, this module will detect the testing result of attacking module.
Wherein, described central host comprises:
The database broker module, finish from data warehouse reading flow measure feature, be sent to central host the model generation module function and finish the traffic characteristic that receives on each real-time watch-dog, be stored in the function of data warehouse;
The model generation module according to the traffic characteristic that is obtained by the database broker module, according to the configuration that produces traffic characteristic, generates discharge model according to clustering algorithm, and this model is sent in each real-time watch-dog.
Wherein, described central host further comprises:
The model visualization module, reception comes the model of self model generation module, shows this model on central host.
Wherein, described each real-time watch-dog is connected with central host and comprises:
Each real-time watch-dog is connected to hub, is connected to central host by hub.
Wherein, described each real-time watch-dog is connected with central host and comprises:
Each real-time watch-dog is connected to the internal network switch, is connected to central host by the internal network switch.
The present invention also provides a kind of method of utilizing the robust gateway system network data to be detected attack; wherein; each robust gateway equipment is connected with protected network in the robust gateway system; each robust gateway equipment is connected with separately real-time watch-dog respectively; each real-time watch-dog is connected with central host; central host is connected with data warehouse, it is characterized in that adopting the self-adapting detecting method based on data mining, and this method may further comprise the steps:
Each robust gateway equipment in A, the robust gateway system is from detecting data from acquisition the data of external network, to detect data respectively and be sent to separately real-time watch-dog, watch-dog utilizes association algorithm to obtain traffic characteristic respectively from detect data according to the policy configurations that produces traffic characteristic in real time;
B, central host obtain traffic characteristic from data warehouse, according to the policy configurations that generates discharge model, utilize clustering algorithm to handle traffic characteristic, the discharge model that generation is made up of normal cluster, this discharge model is sent to each real-time watch-dog, the traffic characteristic that will detect data with each real-time watch-dog respectively sends to central host, and central host stores these traffic characteristics in the data warehouse into;
C, each real-time watch-dog detect the traffic characteristic that detects data according to this discharge model, according to testing result control robust gateway the primitive network data are filtered.
Wherein, before described steps A, the robust gateway system initialization carries out the self study process, generates discharge model, comprising:
Real-time watch-dog in the robust gateway system is according to the policy configurations that produces traffic characteristic, utilize association algorithm to obtain the traffic characteristic of the network data collect, these traffic characteristics are stored in the data warehouse by central host, central host is according to the policy configurations that produces discharge model, obtain traffic characteristic from data warehouse, utilize clustering algorithm to handle traffic characteristic and generate the discharge model of forming by normal cluster, be stored in the central host, and be published to each enforcement watch-dog.
Wherein, in steps A, described from comprising from obtaining to detect data the data of external network:
Robust gateway equipment obtains the data collection strategy configuration from the real-time watch-dog of correspondence, collects from the primitive network data that external network obtains according to this configuration and obtains described detection data.
Wherein, described step B and step C with step C preceding step B in the back or step B preceding step C in the back or step B and step C order simultaneously carry out.
Wherein, describedly utilize association algorithm to obtain described traffic characteristic to comprise:
Determining the size of time window according to the parameter in the policy configurations that produces traffic characteristic, is that unit utilizes association algorithm to obtain traffic characteristic with the time window.
Wherein, described is that unit utilizes association algorithm acquisition traffic characteristic to comprise with the time window:
A1, be linkage record with the network data recovery in the unit interval window of collecting;
A2, utilize association algorithm to excavate the Frequent Item Sets in the linkage record in this unit interval window and calculate corresponding number, obtain described traffic characteristic.
Wherein, described according to the policy configurations that generates discharge model, utilize clustering algorithm to generate discharge model and comprise:
Calculate the distance between the traffic characteristic that obtains, according to the similarity parameter of setting in the policy configurations that generates discharge model, utilize clustering algorithm will have homophylic traffic characteristic and be summed up as cluster, the cluster that will meet traffic characteristic number in the policy configurations that generates discharge model constitutes discharge model as normal cluster with normal cluster.
Wherein, in step C, the described traffic characteristic that detects data the detection according to discharge model comprises:
C1, calculate the present flow rate feature whether in discharge model within the scope of each cluster, if then testing result is the normal discharge feature for this traffic characteristic, otherwise testing result is the abnormal flow feature for this traffic characteristic;
C2, according to testing result to robust gateway equipment issue response policy, robust gateway equipment filters the primitive network data according to response policy, comprising:
When the number of abnormal flow feature in the testing result is in the monitoring scope that response policy sets, enter listen mode by response policy control robust gateway equipment, robust gateway equipment is directly transmitted all primitive network data and is left intact;
When the number of abnormal flow feature in the testing result is in the suspicious scope that response policy sets, enter str mode by response policy control robust gateway equipment, robust gateway equipment carries out bandwidth constraints to the detection data with abnormal flow feature;
When abnormal flow feature in the testing result outnumber the suspicious scope that response policy sets the time, by to answering policy control robust gateway equipment to enter the active defense pattern, robust gateway equipment blocks the detection data with abnormal flow feature.
As seen, in the present invention, robust gateway equipment filters the initial data in the network according to data collection strategy, gives real-time watch-dog with the detection data that obtain after filtering; The detection data that real-time watch-dog will receive revert to linkage record, utilize the association mining in the data digging method, extract traffic characteristic; Watch-dog is invaded judgement according to discharge model and current traffic characteristic in real time, makes real-time warning, and produces corresponding response policy, and control robust gateway equipment filters the primitive network data.The present invention has realized detecting the function of DOS/DDOS, its system has transparent gateway setting, not being vulnerable to DOS/DDOS attacks, the tri-level logic composition mode of its system makes that the detection task is carried out respectively, reduced internal network pressure, the traffic characteristic that central host gathers each real-time watch-dog produces discharge model, makes to detect attack more comprehensively, accurately; Its method has been utilized association algorithm and the clustering algorithm in the data mining algorithm, has reduced the workload of data processing, and makes that detecting the accuracy of attacking improves.
Description of drawings
Fig. 1 is the common treatment model schematic diagram of IDS product.
Fig. 2 is the Dude architectural schematic.
Fig. 3 is robust gateway system topology figure.
Fig. 4 is the configuration schematic diagram of robust gateway equipment.
Fig. 5 is the configuration schematic diagram of real-time watch-dog.
Fig. 6 is the configuration schematic diagram of central host.
Fig. 7 divides schematic diagram for the robust gateway system function module.
Fig. 8 generates schematic diagram for discharge model.
Embodiment
The present invention adopts the robust gateway system that is made up of a plurality of robust gateway equipment, a plurality of real-time watch-dog, central host and data warehouse to detect DOS/DDOS and attacks, its detection method has been utilized association algorithm and the clustering algorithm in the data mining algorithm, according to discharge model the traffic characteristic that detects data is detected, produce response policy according to testing result, control robust gateway equipment filters the primitive network data.
Referring to shown in Figure 3, Fig. 3 is robust gateway system topology figure of the present invention, and this system constitutes the tri-level logic structure by central host, real-time watch-dog and robust gateway, and system comprises:
(1) a plurality of robust gateway 31: each robust gateway all is between router and the fire compartment wall 33, and the interface place network segment of each robust gateway by fire compartment wall and internal network is with identical by the interface place network segment of router and external network, with this assurance robust gateway being set is transparent setting with respect to internal network and external network, and then can hide whole robust gateway system, making it not be vulnerable to DOS/DDOS attacks, in the present embodiment, being provided with that the network segment is identical adopts the ARP proxy technology to realize.These robust gateway one ends connect exterior I NTERNET by router, and an end connects internal network by fire compartment wall, and an end connects real-time watch-dog 32 by network interface independently.Each robust gateway is all finished the function of data acquisition and exception response, referring to shown in Figure 4, and thick arrow representative of data flow among this figure, thin arrow is represented control flows, comprises in the robust gateway:
Netfilter41, the Linux fire compartment wall that present embodiment adopted, Netfilter obtains response policy from real-time watch-dog and disposes this fire compartment wall (1), and this fire compartment wall filters (5) to the raw data packets of obtaining from INTERNET, is delivered to network.
(2) with the real-time one to one watch-dog 32 of robust gateway: the robust gateway that each real-time watch-dog is corresponding with it is connected, and all real-time watch-dogs are connected with the hub that is independent of internal network (Hub) 36, the last connection of Hub central host.Wherein, all real-time watch-dogs also can be connected with the switch that is independent of internal network, connect central host on the switch.Each real-time watch-dog is all finished traffic characteristic and is extracted, judges that the present flow rate feature is whether unusual and control the function that robust gateway filters according to discharge model.Referring to shown in Figure 5, thick arrow representative of data flow among this figure, thin arrow is represented control flows, and watch-dog comprises in real time:
Traffic characteristic extraction module 51: this module obtains to produce the strategy (3) of traffic characteristic from central host, the detection data that robust gateway obtains are carried out feature extraction (1), obtain the traffic characteristic of these detection data, this traffic characteristic is carried out following operation: this traffic characteristic is delivered to central host (4), and the database broker module of central host transmits this traffic characteristic and be stored in the data warehouse; This traffic characteristic is delivered to detection attacks module (5), detect the detected object of data as this with this traffic characteristic;
Detect and attack module 52: this module obtains discharge model (2) from central host, with the discharge model is that standard detects obtaining traffic characteristic from the traffic characteristic extraction module, issue response policy to robust gateway (6) according to testing result, and testing result is shown (8) on warning visualization model 53;
Warning visualization model 53: this module receives to detect and attacks the detection attack result that module sends that it is visual on real-time watch-dog to detect attack result;
In real time also preserving the data collection strategy configuration on the watch-dog, required relevant parameter when comprising data collection in this configuration, watch-dog sends to robust gateway (7) with this configuration in real time, the type of control robust gateway collection primitive network data.
(3) central host 34: this central host is connected with real-time watch-dog by Hub, controls the work of real-time watch-dog, and produces the configuration and the discharge model of traffic characteristic to each real-time watch-dog issue.The central host major function is: obtain the traffic characteristic of each real-time watch-dog from data warehouse, utilize data mining algorithm, produce discharge model and in time be distributed to each real-time watch-dog according to the traffic characteristic that obtains.Referring to shown in Figure 6, thick arrow representative of data flow among this figure, thin arrow is represented control flows, and central host comprises:
Database broker module 61: this module is finished the database broker function, obtain traffic characteristic (1) from real-time watch-dog, this module is (3) in the data warehouse that this traffic characteristic is stored into central host is connected, this module reads discharge model and produces needed traffic characteristic from data warehouse, be delivered to model generation module (2);
Model generation module 62: this module obtains traffic characteristic from the database broker module, produce model according to the model generation strategy that passes over, this model is sent to real-time watch-dog as the standard (4) that detects attack, and this model is sent to model visualization module (5);
Model visualization module 63: this module receives the model of self model generation module, on central host to this model visualization;
In central host, also preserve the policy configurations 64 that produces traffic characteristic, required parameter when having traffic characteristic to produce in this configuration, central host is delivered to real-time watch-dog (6) with this configuration, controls each real-time watch-dog and generates traffic characteristic according to identical rule; Also there is model generation strategy configuration in the central host, required parameter when having discharge model to generate in this configuration, this configuration is sent to the model generation module, the allocation models generation.
(4) data warehouse 35: this data warehouse is connected with central host by hub (or switch), by the database broker module on the central host, obtain the traffic characteristic of the detection data on each real-time watch-dog from central host, be kept in this data warehouse; When central host generated discharge model, the database broker module of central host obtained the traffic characteristic that each real-time watch-dog is submitted from this data warehouse, to generate discharge model.In the embodiment of the invention, adopt oracle database.
Referring to shown in Figure 7, in the above firm network management system, the functional relationship of robust gateway, real-time watch-dog and central host is:
Robust gateway obtains primitive network data (1), from the primitive network data, collect out detection data (3) according to obtaining data collection strategy by real-time watch-dog, these detection data are sent to real-time watch-dog (2), watch-dog is according to the strategy (6) that obtains to produce traffic characteristic from central host in real time, detect the extracting data traffic characteristic from this, this traffic characteristic is delivered to central host (4), and the database broker module on the central host transmits the traffic characteristic of each real-time watch-dog and be stored in the data warehouse; Central host obtains the traffic characteristic (7) that each real-time watch-dog is submitted from data warehouse, according to the strategy that produces traffic characteristic these traffic characteristics are handled, generate discharge model, this discharge model is sent to real-time watch-dog as detecting the standard (5) of attacking; In real-time watch-dog, traffic characteristic according to discharge model and detection data detects attack, to robust gateway (8), robust gateway sends out (9) with these policy filtering primitive network data with the network data after filtering according to testing result issue response policy.
Describing the robust gateway system below in detail utilizes data mining algorithm to carry out the method for attack detecting.
Wherein, data mining algorithm comprises two classes: association algorithm and clustering algorithm, and association algorithm refers to: excavate the valuable association knowledge that interknits between the data of description item from lot of data; Clustering algorithm refers to: one group of individuality is classified as some classifications according to similitude, and the distance of sorting out between the individuality that principle need guarantee to belong to same classification is as much as possible little, and the distance between different classes of individuality is big as much as possible.
This attack detection method specifically comprises:
1, the central host in the robust gateway system obtains traffic characteristic from data warehouse, and the traffic characteristic that utilizes data mining algorithm to handle acquisition generates discharge model, is stored in the central host as the present flow rate model, uses during for attack detecting;
Referring to shown in Figure 8, adopt data mining algorithm to generate discharge model among the present invention and specifically comprise:
Adopt the Single-Linkage algorithm, traffic characteristic is considered as vector, attributes such as COS wherein calculate the distance between these vectors as vector components, will condense together apart near vector, obtain having the cluster of similar traffic characteristic thus; Wherein, each cluster is represented that by such center vector and radius the form of center vector is identical with the form of traffic characteristic; Wherein, the near setting of distance is as the criterion with the parameter in the policy configurations that generates discharge model, and the form ginseng of described traffic characteristic is shown in Table 2;
According to the hypothesis of " normal data than attack data many ", will comprise the vectorial cluster of major part and be considered as normal cluster, and the cluster that will comprise the minority vector is considered as unusual cluster; Each normal cluster constitutes discharge model, and wherein, the setting of the number of vectors in the normal cluster is as the criterion with the parameter in the policy configurations that generates discharge model.
Wherein, the k1 among Fig. 8, k2 and q are mutual incoherent parameter.
Above step has illustrated the process that robust gateway flow system flow model produces.The robust gateway system produces strategy execution in step 1 regularly according to model in detecting attack process, produce new discharge model.
2, the robust gateway system carries out attack detecting according to the discharge model that produces in the step 1 to network data, and each robust gateway and real-time watch-dog thereof are all carried out following steps:
2.1, robust gateway obtains the primitive network data, collects out the detection data according to data collection strategy from the primitive network data;
2.2, robust gateway will detect data and send to its real-time watch-dog, watch-dog utilizes association algorithm to obtain traffic characteristic from detect data according to the configuration in the strategy that produces traffic characteristic in real time, comprising:
Network data recovery in the time window that real-time watch-dog will be collected is the linkage record of table 1 form, according to the policy configurations that produces traffic characteristic, utilize association algorithm to excavate the Frequent Item Sets in these linkage records and calculate corresponding number with similitude, these Item Sets are respectively as the traffic characteristic of the network data in this time window, the form ginseng of traffic characteristic is shown in Table 2, wherein, total number is the sum of linkage record in this time window, the number of the linkage record that number comprises for this traffic characteristic; Network data in each time window is repeated above operation, obtain the traffic characteristic in each time window; Further, these traffic characteristics are passed through the database broker module stores of central host in data warehouse;
| Time | Source IP address | Purpose IP address | COS | Protocol type | TCP connection status sign |
Table 1
| COS | Protocol type | TCP connection status mark | Number | Total number |
Table 2
2.3, in real time watch-dog obtains the present flow rate model from central host, with this discharge model the traffic characteristic that detects data is compared, and detects attack, comprising:
Traffic characteristic is considered as vector, calculates the vector of current traffic characteristic correspondence and the distance of each cluster in the present flow rate model, do not describe vector in the scope, be the abnormal flow feature in each cluster; The number of the abnormal flow feature in detecting data is in the monitoring scope that response policy sets the time, and robust gateway equipment is not done filtration treatment to the primitive network data; The number of the abnormal flow feature in detecting data is in the suspicious scope that response policy sets the time, produce suspicious warning, watch-dog makes robust gateway enter " str mode " work by the configuration response strategy in real time, and the flow with abnormal flow feature is carried out bandwidth constraints; Abnormal flow feature in detecting data outnumber the suspicious scope that response policy sets the time, watch-dog configuration response strategy makes robust gateway enter " active defense pattern " in real time, robust gateway blocks those flows with abnormal flow feature;
Network data is being detected in the process of attack, constantly having new discharge model to generate, detecting when attacking for step 2 and use in the described mode of step 1.
In the above step, the execution sequence of step 2 and step 1 comprises: step 2 preceding step 1 after, step 1 is carried out in back and step 1 and step 2 simultaneously in preceding step 1, just, the execution sequence of step 2 and step 1 does not influence enforcement of the present invention.
Wherein, when the robust gateway system is installed in the new protected network; at first carry out the robust gateway system initialization; this initialization procedure comprises: obtain traffic characteristic according to the method described in step 2.1～2.2; utilize the method described in the step 1 to handle these traffic characteristics and generate discharge model; be stored in the central host with this discharge model, the detection attack and the self adaptation that are used as after the initialization generate discharge model.
As seen, the invention provides a kind of robust gateway system, this system is by robust gateway equipment, watch-dog, central host and data warehouse are formed in real time, and robust gateway is finished to the collection of primitive network data with according to the function of response policy filtration primitive network data; The watch-dog robust gateway corresponding with it is connected in real time, finishes the function that the traffic characteristic that detects data extracted and detected according to discharge model the present flow rate feature; Central host links to each other with each real-time watch-dog, obtains the traffic characteristic that each real-time watch-dog transmits, and utilizes clustering algorithm constantly to generate new discharge model, is distributed to each real-time watch-dog.Data bins stock implements the traffic characteristic of watch-dog by the storage of the database broker on the central host from each, and provides traffic characteristic when central host generates discharge model.The present invention also provides the detection attack method of a kind of robust gateway system, this method utilizes association algorithm to extract the traffic characteristic that detects data, compare with this traffic characteristic and discharge model, the traffic characteristic of the cluster scope in the excess flow model is the abnormal flow feature, number and system's setting according to the abnormal flow feature, produce corresponding response policy, according to these policy filtering primitive network data.This system and method thereof have well realized detecting the function of DOS/DDOS, have the following advantages:
1 〉, robust gateway itself has very high disguise, can not found that self can not suffer the attack of DOS/DDOS by the hacker of outside.
2 〉, each robust gateway links to each other with a real-time watch-dog by an Ethernet interface, finishes the detection task by real-time watch-dog, can alleviate the Processing tasks of firm network so greatly, the throughput of raising system.
3 〉, in real time watch-dog constitutes a network that is independent of internal network by Hub separately with central host, has so not only reduced the pressure of communication to internal network, no longer takies its bandwidth, and good hidden system, the enhancing of self-defense ability.
4 〉, robust gateway, in real time watch-dog and central host have constituted 3 layers of basic logical construction of system, system can expand arbitrarily based on the clear and definite structure of this logic division of labor.
5 〉, the ingenious associating of robust gateway used association mining algorithm and the cluster mining algorithm in the data mining, without any need for training data.Owing to utilize association algorithm that data set is abstract in traffic characteristic, significantly reduced the data volume that clustering algorithm is handled, not only overcome the slow shortcoming of the clustering algorithm speed of service, increased Useful Information for clustering processing again.
6 〉, the robust gateway system discharge model that can adaptive generation is used to detect.
Claims (17)
1, a kind of robust gateway system, it is characterized in that this system comprises a plurality of robust gateway equipment, a plurality of real-time watch-dog, a central host and a data warehouse, robust gateway equipment, real-time watch-dog, central host connect and compose the tri-level logic structure, each robust gateway equipment and corresponding real-time watch-dog connection, each real-time watch-dog is connected with central host, central host is connected with data warehouse, wherein:
Each robust gateway equipment all places between external network and the internal network, with respect to internal network and the transparent setting of external network, has data acquisition and data filter function;
The discharge model that each real-time watch-dog all has the detection extracting data traffic characteristic that collects from robust gateway equipment, send according to central host judges whether the present flow rate feature unusual, to robust gateway issue response policy with the function of control robust gateway to filtering from the network data of outside;
Central host has the traffic characteristic that sends according to from data warehouse, generates the function that discharge model and continuous self adaptation generate new discharge model, and simultaneously, the policy configurations that this equipment also will be stored generation traffic characteristic thereon is published on the real-time watch-dog;
Data warehouse obtains the traffic characteristic of the detection data on the real-time watch-dog by central host, is kept in this data warehouse, and when central host generated discharge model, central host obtained traffic characteristic from this data warehouse.
2, system according to claim 1 is characterized in that described robust gateway equipment comprises:
The function of obtaining raw data packets from INTERNET is finished in the packet capturing storehouse that disposes in this equipment;
The fire compartment wall that disposes in this equipment according to the response policy that obtains from real-time watch-dog, will filter from the initial data that INTERNET obtains, and be sent to internal network.
3, system according to claim 1 is characterized in that described robust gateway comprises with respect to internal network and the transparent setting of external network:
Robust gateway has and is arranged at the phase same network segment and two interfaces internal network and external network.
4, system according to claim 1 is characterized in that described real-time watch-dog comprises:
The traffic characteristic extraction module: this module utilizes association algorithm that the detection data that robust gateway equipment transmits are carried out feature extraction according to the strategy of the generation traffic characteristic that obtains from central host, obtains traffic characteristic;
Detect and attack module: this module obtains traffic characteristic from the traffic characteristic extraction module, and from central host acquisition discharge model, utilize the discharge model and the traffic characteristic that obtain to finish detection attack function, finish the function of issue response policy to robust gateway according to testing result.
5, system according to claim 4 is characterized in that described real-time watch-dog further comprises:
It is visual that warning visualization model, this module will detect the testing result of attacking module.
6, system according to claim 1 is characterized in that described central host comprises:
The database broker module, finish from data warehouse reading flow measure feature, be sent to central host the model generation module function and finish the traffic characteristic that receives on each real-time watch-dog, be stored in the function of data warehouse;
The model generation module according to the traffic characteristic that is obtained by the database broker module, according to the configuration that produces traffic characteristic, generates discharge model according to clustering algorithm, and this model is sent in each real-time watch-dog.
7, system according to claim 6 is characterized in that described central host further comprises:
The model visualization module, reception comes the model of self model generation module, shows this model on central host.
8, system according to claim 1 is characterized in that described each real-time watch-dog is connected with central host to comprise:
Each real-time watch-dog is connected to hub, is connected to central host by hub.
9, system according to claim 1 is characterized in that described each real-time watch-dog is connected with central host to comprise:
Each real-time watch-dog is connected to the internal network switch, is connected to central host by the internal network switch.
10, a kind of method of utilizing the robust gateway system network data to be detected attack; wherein; each robust gateway equipment is connected with protected network in the robust gateway system; each robust gateway equipment is connected with separately real-time watch-dog respectively; each real-time watch-dog is connected with central host; central host is connected with data warehouse, it is characterized in that adopting the self-adapting detecting method based on data mining, and this method may further comprise the steps:
Each robust gateway equipment in A, the robust gateway system is from detecting data from acquisition the data of external network, to detect data respectively and be sent to separately real-time watch-dog, watch-dog utilizes association algorithm to obtain traffic characteristic respectively from detect data according to the policy configurations that produces traffic characteristic in real time;
B, central host obtain traffic characteristic from data warehouse, according to the policy configurations that generates discharge model, utilize clustering algorithm to handle traffic characteristic, the discharge model that generation is made up of normal cluster, this discharge model is sent to each real-time watch-dog, the traffic characteristic that will detect data with each real-time watch-dog respectively sends to central host, and central host stores these traffic characteristics in the data warehouse into;
C, each real-time watch-dog detect the traffic characteristic that detects data according to this discharge model, according to testing result control robust gateway the primitive network data are filtered.
11, method according to claim 10 is characterized in that before described steps A the robust gateway system initialization carries out the self study process, generated discharge model, comprising:
Real-time watch-dog in the robust gateway system is according to the policy configurations that produces traffic characteristic, utilize association algorithm to obtain the traffic characteristic of the network data collect, these traffic characteristics are stored in the data warehouse by central host, central host is according to the policy configurations that produces discharge model, obtain traffic characteristic from data warehouse, utilize clustering algorithm to handle traffic characteristic and generate the discharge model of forming by normal cluster, be stored in the central host, and be published to each enforcement watch-dog.
12, method according to claim 10 is characterized in that in steps A, and is described from comprising from obtaining to detect data the data of external network:
Robust gateway equipment obtains the data collection strategy configuration from the real-time watch-dog of correspondence, collects from the primitive network data that external network obtains according to this configuration and obtains described detection data.
13, method according to claim 10, it is characterized in that described step B and step C with step C preceding step B in the back or step B preceding step C in the back or step B and step C order simultaneously carry out.
14,, it is characterized in that describedly utilizing association algorithm to obtain described traffic characteristic to comprise according to claim 10 or 11 described methods:
Determining the size of time window according to the parameter in the policy configurations that produces traffic characteristic, is that unit utilizes association algorithm to obtain traffic characteristic with the time window.
15, method according to claim 14 is characterized in that described is that unit utilizes association algorithm acquisition traffic characteristic to comprise with the time window:
A1, be linkage record with the network data recovery in the unit interval window of collecting;
A2, utilize association algorithm to excavate the Frequent Item Sets in the linkage record in this unit interval window and calculate corresponding number, obtain described traffic characteristic.
16, according to claim 10 or 11 described methods, it is characterized in that described policy configurations according to the generation discharge model, utilize clustering algorithm to generate discharge model and comprise:
Calculate the distance between the traffic characteristic that obtains, according to the similarity parameter of setting in the policy configurations that generates discharge model, utilize clustering algorithm will have homophylic traffic characteristic and be summed up as cluster, the cluster that will meet traffic characteristic number in the policy configurations that generates discharge model constitutes discharge model as normal cluster with normal cluster.
17, method according to claim 10 is characterized in that in step C, and the described traffic characteristic that detects data the detection according to discharge model comprises:
C1, calculate the present flow rate feature whether in discharge model within the scope of each cluster, if then testing result is the normal discharge feature for this traffic characteristic, otherwise testing result is the abnormal flow feature for this traffic characteristic;
C2, according to testing result to robust gateway equipment issue response policy, robust gateway equipment filters the primitive network data according to response policy, comprising:
When the number of abnormal flow feature in the testing result is in the monitoring scope that response policy sets, enter listen mode by response policy control robust gateway equipment, robust gateway equipment is directly transmitted all primitive network data and is left intact;
When the number of abnormal flow feature in the testing result is in the suspicious scope that response policy sets, enter str mode by response policy control robust gateway equipment, robust gateway equipment carries out bandwidth constraints to the detection data with abnormal flow feature;
When abnormal flow feature in the testing result outnumber the suspicious scope that response policy sets the time, enter the active defense pattern by response policy control robust gateway equipment, robust gateway equipment blocks the detection data with abnormal flow feature.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 02155382 CN1257632C (en) | 2002-12-11 | 2002-12-11 | Firm gateway system and its attack detecting method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 02155382 CN1257632C (en) | 2002-12-11 | 2002-12-11 | Firm gateway system and its attack detecting method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1507233A true CN1507233A (en) | 2004-06-23 |
| CN1257632C CN1257632C (en) | 2006-05-24 |
Family
ID=34235881
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 02155382 Expired - Fee Related CN1257632C (en) | 2002-12-11 | 2002-12-11 | Firm gateway system and its attack detecting method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1257632C (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100384149C (en) * | 2005-11-11 | 2008-04-23 | 上海交通大学 | Method for detecting and monitoring gusty abnormal network flow |
| CN101192917B (en) * | 2006-11-24 | 2010-05-12 | 凹凸科技（中国）有限公司 | Method and system for network access control based on NAT |
| CN1777179B (en) * | 2004-11-19 | 2010-09-01 | 微软公司 | Method and system for distributing security policies |
| CN101355463B (en) * | 2008-08-27 | 2011-04-20 | 成都市华为赛门铁克科技有限公司 | Method, system and equipment for judging network attack |
| CN101548269B (en) * | 2006-10-20 | 2011-11-16 | 韦里佐内服务公司 | Method, computer program product, and device for network reconnaissance flow identification |
| CN101316268B (en) * | 2008-07-04 | 2011-12-14 | 中国科学院计算技术研究所 | Detection method and system for exception stream |
| CN101267353B (en) * | 2008-04-24 | 2011-12-21 | 北京大学 | A load-independent method for detecting network abuse |
| CN101588358B (en) * | 2009-07-02 | 2012-06-27 | 西安电子科技大学 | System and method for detecting host intrusion based on danger theory and NSA |
| CN101729301B (en) * | 2008-11-03 | 2012-08-15 | 中国移动通信集团湖北有限公司 | Monitor method and monitor system of network anomaly traffic |
| CN101789931B (en) * | 2009-12-31 | 2012-12-05 | 暨南大学 | Network intrusion detection system and method based on data mining |
| CN107948587A (en) * | 2017-11-15 | 2018-04-20 | 中国联合网络通信集团有限公司 | Methods of risk assessment, the apparatus and system of monitoring device |
| US10484406B2 (en) | 2015-01-22 | 2019-11-19 | Cisco Technology, Inc. | Data visualization in self-learning networks |
-
2002
- 2002-12-11 CN CN 02155382 patent/CN1257632C/en not_active Expired - Fee Related
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1777179B (en) * | 2004-11-19 | 2010-09-01 | 微软公司 | Method and system for distributing security policies |
| CN100384149C (en) * | 2005-11-11 | 2008-04-23 | 上海交通大学 | Method for detecting and monitoring gusty abnormal network flow |
| CN101548269B (en) * | 2006-10-20 | 2011-11-16 | 韦里佐内服务公司 | Method, computer program product, and device for network reconnaissance flow identification |
| CN101192917B (en) * | 2006-11-24 | 2010-05-12 | 凹凸科技（中国）有限公司 | Method and system for network access control based on NAT |
| CN101267353B (en) * | 2008-04-24 | 2011-12-21 | 北京大学 | A load-independent method for detecting network abuse |
| CN101316268B (en) * | 2008-07-04 | 2011-12-14 | 中国科学院计算技术研究所 | Detection method and system for exception stream |
| CN101355463B (en) * | 2008-08-27 | 2011-04-20 | 成都市华为赛门铁克科技有限公司 | Method, system and equipment for judging network attack |
| CN101729301B (en) * | 2008-11-03 | 2012-08-15 | 中国移动通信集团湖北有限公司 | Monitor method and monitor system of network anomaly traffic |
| CN101588358B (en) * | 2009-07-02 | 2012-06-27 | 西安电子科技大学 | System and method for detecting host intrusion based on danger theory and NSA |
| CN101789931B (en) * | 2009-12-31 | 2012-12-05 | 暨南大学 | Network intrusion detection system and method based on data mining |
| US10484406B2 (en) | 2015-01-22 | 2019-11-19 | Cisco Technology, Inc. | Data visualization in self-learning networks |
| CN107948587A (en) * | 2017-11-15 | 2018-04-20 | 中国联合网络通信集团有限公司 | Methods of risk assessment, the apparatus and system of monitoring device |
| CN107948587B (en) * | 2017-11-15 | 2019-12-27 | 中国联合网络通信集团有限公司 | Risk assessment method, device and system for monitoring equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1257632C (en) | 2006-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105208037B (en) | A kind of DoS/DDoS attack detectings and filter method based on lightweight intrusion detection | |
| CN1160899C (en) | Distributed dynamic network security protecting system | |
| CN102882884B (en) | Honeynet-based risk prewarning system and method in information production environment | |
| CN110149350A (en) | A kind of associated assault analysis method of alarm log and device | |
| CN1257632C (en) | Firm gateway system and its attack detecting method | |
| CN1655518A (en) | Network security system and method | |
| US20070180107A1 (en) | Security incident manager | |
| CN1725709A (en) | Method of linking network equipment and invading detection system | |
| JP2020530638A (en) | Malware Host NetFlow Analysis System and Method | |
| CN1578227A (en) | Dynamic IP data packet filtering method | |
| CN1889573A (en) | Active decoy method and system | |
| Vidal et al. | Alert correlation framework for malware detection by anomaly-based packet payload analysis | |
| US9961047B2 (en) | Network security management | |
| WO2012028375A2 (en) | Method and system for classifying traffic | |
| Landress | A hybrid approach to reducing the false positive rate in unsupervised machine learning intrusion detection | |
| CN1564530A (en) | Network safety guarded distributing invading detection and internal net monitoring system and method thereof | |
| CN1417690A (en) | Application process audit platform system based on members | |
| Sun | A New Perspective on Cybersecurity Protection: Research on DNS Security Detection Based on Threat Intelligence and Data Statistical Analysis | |
| CN113162897A (en) | Industrial control network security filtering system and method | |
| CN1694411A (en) | Network invading detection system with two-level decision structure and its alarm optimization method | |
| CN116319114A (en) | Method and system for network intrusion detection | |
| CN1602470A (en) | Protecting against malicious traffic | |
| CN116781380A (en) | Campus network security risk terminal interception traceability system | |
| CN107608752A (en) | The threat information response examined oneself based on virtual machine and method of disposal and system | |
| Sulaiman et al. | Big data analytic of intrusion detection system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| C56 | Change in the name or address of the patentee | ||
| CP03 | Change of name, title or address |
Address after: 100049 No. 19, Yuquanlu Road, Beijing, Shijingshan District Patentee after: University OF CHINESE ACADEMY OF SCIENCES Address before: 100039, Yuquanlu Road, Beijing No. 19 (a) Patentee before: GRADUATE University OF CHINESE ACADEMY OF SCIENCES |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20151120 Address after: 100195 Beijing city Haidian District minzhuang Road No. 87 C Patentee after: INSTITUTE OF INFORMATION ENGINEERING, CHINESE ACADEMY OF SCIENCES Address before: 100049 No. 19, Yuquanlu Road, Beijing, Shijingshan District Patentee before: University of Chinese Academy of Sciences |
|
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20060524 Termination date: 20191211 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |