CN101562539B - Self-adapting network intrusion detection system - Google Patents
Self-adapting network intrusion detection system Download PDFInfo
- Publication number
- CN101562539B CN101562539B CN2009101038804A CN200910103880A CN101562539B CN 101562539 B CN101562539 B CN 101562539B CN 2009101038804 A CN2009101038804 A CN 2009101038804A CN 200910103880 A CN200910103880 A CN 200910103880A CN 101562539 B CN101562539 B CN 101562539B
- Authority
- CN
- China
- Prior art keywords
- module
- network
- message
- data base
- detection module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention provides a self-adapting network intrusion detection system, and the system comprises a message pre-processing and forwarding module, a misuse detection module, an abnormality respondingmodule and an output device. The message pre-processing and forwarding module is used for extracting non-encrypted messages in network connecting objects and forwarding the messages to the misuse det ection module which is used for forwarding the detected abnormal network connecting objects to the abnormality responding module. The system is characterized by also comprising a real-time database, afeature data arranging and storing module, a historical database, a semi-supervised self-adaptive learning module and an abnormality detection module which is used for analyzing the network connectin g feature vectors in the real-time database and forwarding the detected abnormal network connection to the abnormality responding module. The beneficial effect of the system is as follows: the system uses the semi-supervised self-adaptive learning module to establish the rule of abnormality detection according to the data in the historical database, a lot of artificially-marked training data is not needed, thus reducing the deploying cost of the abnormality detection system.
Description
Technical field
The present invention relates to a kind of computer network intrusion detection system, particularly the self-adapting network intrusion detection system of an integrated abnormality detection technology and misuse detection technique.
Background technology
Network invasion monitoring is as one of present topmost Active Networks safety measure, it is discerned and responds by computer is connected with hostile network on the Internet resources, replenish effectively and perfect such as safety measures such as access control, data encryption, fire compartment wall, virus preventions, improve the integrality of information security foundation structure, become link indispensable in the information system security solution.The network invasion monitoring technology is divided into misuse detection technique and abnormality detection technology two classes by its operation principle.Wherein misapply detection technique and be the basis based on the data message characteristic matching, this detection technique accuracy rate height, but its problem is can not find new intrusion model and situation occurs failing to report.The abnormality detection technology is then based on data such as network connection features, system call feature, network flow characteristic and system's time delay features, set up the descriptive model of proper network behavior, when having great departing from, User Activity and normal behaviour be considered to invasion, this detection technique can be found the new network invasion, but there is the rate of false alarm height, needs the problem of a large amount of training samples.
Because the running status of information system is continuous evolution, reflects that so the regularity of distribution of the characteristic of its running status also can change naturally thereupon.In order to obtain desirable detection performance, just require abnormality detection system must regularly dynamically update training sample, and dynamically update the abnormality detection rule on this basis.Yet traditional pass through the method inefficiency that digerait's manual type is collected training sample, and cause the application and the high problem of lower deployment cost of abnormality detection system.
Summary of the invention
The technical problem to be solved in the present invention is: a self-adapting network intrusion detection system is provided, it can be in conjunction with abnormality detection technology and misuse detection technique, automatically adapt to the variation of running environment with the realization detection system, and reduce the application of whole system and the cost of deployment.
For solving the problems of the technologies described above, technical scheme of the present invention is as follows: system comprises message preliminary treatment and forwarding module, the misuse detection module, the exception response module, output device, this message preliminary treatment and forwarding module are used for extracting the non-encrypted message in the network connecting object and are forwarded to the misuse detection module, the misuse detection module is used for detected unusual network connecting object is forwarded to the exception response module, its key is, this system also comprises real-time data base, characteristic arrangement and memory module, historical data base, semi-supervised adaptive learning modules, the abnormality detection module
Message preliminary treatment and forwarding module also are used for extracting the network connection features vector of network connecting object and storing real-time data base into;
The misuse detection module also is used for storing the message testing result of non-encrypted message into real-time data base;
Characteristic arrangement and memory module are used for making regular check on and integrate the network connection features vector sum message status data in the real-time data base, and the network connection features vector after the arrangement is stored in the historical data base; Whether described message status comprises: connection identifier, message identification, detect and attribute such as testing result.
Semi-supervised adaptive learning modules utilizes the semi-supervised learning algorithm to generate the abnormality detection rule, and this abnormality detection rule is dynamically updated in the abnormality detection module according to the diverse network connection features vector in the historical data base;
The abnormality detection module is used for analyzing the network connection features vector in the real-time data base, detected unusual network connecting object is switched through send to the exception response module then.
Beneficial effect of the present invention: the present invention generates a large amount of training datas by the testing result of using the misuse detection module as the abnormality detection module, utilizes the semi-supervised learning technology to realize dynamically updating of abnormality detection rule, thereby improves the adaptive ability of system; Semi-supervised adaptive learning modules can be set up the abnormality detection rule according to the data in the historical data base, need not a large amount of handmarkings' training data, thereby has greatly reduced the lower deployment cost of abnormality detection system; System synthesis abnormality detection technology and misuse detection technique are analyzed network activity, thereby improve the detection performance of system.
Description of drawings
Fig. 1 is the structural representation among the present invention;
Fig. 2 is the structure chart of network connection features vector data;
Fig. 3 is characteristic arrangement and memory module workflow diagram;
Fig. 4 is the structure chart of message detected state data.
Embodiment
The present invention is further illustrated below in conjunction with the drawings and specific embodiments.
As shown in Figure 1 and Figure 2, comprise in the self-adapting network intrusion detection system: message preliminary treatment and forwarding module 1, real-time data base 2, characteristic arrangement and memory module 3, historical data base 4, semi-supervised adaptive learning modules 5, abnormality detection module 6, exception response module 7, misuse detection module 8, output device 9, input unit 10.Message preliminary treatment and forwarding module 1 are connected respectively to real-time data base 2 and misuse detection module 8, the network connection features vector that this message preliminary treatment and forwarding module are used for extracting stores real-time data base 2 into, simultaneously the non-encrypted message in the network connecting object is forwarded to misuse detection module 8; This misuse detection module 8 is dealt into exception response module 7 with detected intrusion event, and misuse detection module 8 also stores the testing result of message in the real-time data base (2) into.
Wherein message is handled and forwarding module 1, adopt the mode of protocal analysis that the network connecting object is analyzed, filtering out a part does not need the video analyzed, network traffics such as voice, extract non-encrypted and network connecting object that pay close attention to, set up its corresponding network connection features vector.At least comprise in the described network connection features vector: connection identifier, capture time stamp, source address, source port number, destination address, destination slogan, the agreement of employing, Exception Type.
Other data then can be according to the system configuration situation in the network connection features vector, use for reference the network attribute feature that adopts in the present abnormality detection algorithm, for example: connect length, connection type, mistake burst tree, the service content of duration, data message, information such as whether encrypt.
6 of abnormality detection modules detect the network connection features vector in the real-time data base 2 according to the abnormality detection rule, just testing result are submitted to exception response module 7 unusually if exist; Exception response module 7 is with the testing result of comprehensive abnormality detection module 6 and 8 pairs of network connection features of misuse detection module vector sum network message, generate warning information and submit to the system manager, by the system manager warning information is sent into output device 9, output device 9 can be facilities such as display, printer or journal file.
As Fig. 1, Fig. 2, Fig. 3, shown in Figure 4.In order to realize the automatic renewal of the abnormality detection rule in the abnormality detection module 6, characteristic arrangement and memory module 3 are used for checking and handling the result of the network connection features vector sum message detection in the real-time data base 2, then the disparate networks connection features vector data that obtains is written in the historical data base 4, as the training dataset of abnormality detection; Semi-supervised adaptive learning modules 5 generates the abnormality detection rule according to the training data in the historical data base 4, and with these abnormality detection Policy Updates in abnormality detection module 6.
Whether for writing down above-mentioned message status, the message status data in the real-time data base 2 comprise at least: connection identifier, message identification, detect and attribute such as testing result.If message has been sent into the abnormality detection module, testing result is not also returned for the testing result of empty then expression misuse detection module 8.
As shown in Figure 3, described characteristic arrangement and memory module 3 can be arranged to make regular check in per 1 second or per 2 seconds or per 3 seconds the network connection features vector in the real-time data base 2, and according to the following steps the data in the real-time data base 2 are handled:
S1, take out a network connecting object from real-time data base 2, require this moment the testing result of all messages in this connecting object all to return from misuse detection module 8, perhaps the message in this connection is not misapplied detection.
Whether S2, the message of judging this connecting object are admitted to misuse detection module 8, carry out corresponding operating if just change step S9 over to; Otherwise change step S3 over to.
S3, check the state of all messages in this connecting object, if the state that has a message is for unusual, show that misuse detection module 8 has detected intrusion behavior from message, mean that this connecting object must be unusual a connection, sends check result into step S4.
S4, judge in the relevant message of connecting object whether comprise the feature of known invasion, if comprise then jump to step S5; Otherwise this connection may be normal connection, then jumps to step S7.
S5, the exception class offset of corresponding network connection features vector is set in the real-time data base, and record is that testing result according to misuse detection module 8 is provided with, jumps to step S6.
S6, network connection features vector is removed from real-time data base 2, and be deposited in the historical data base 4, change step S1 over to and handle next network connecting object.
S7, judge whether the activity of current network connecting object finishes, finished just to forward to step S8 if network connects; Otherwise change step S1 over to.
S8, owing to connect and to finish, all messages all are normal condition, and promptly misuse detection system judges that this connection is normal, therefore can the exception class offset be set to normal connection, and this result of mark is that testing result according to misuse detection module 8 is provided with, and changes step S6 over to.
S9, since this connecting object in message send into the misuse detection module 8, if this connection activity finishes, then directly the exception class offset of this connection features vector is labeled as sky, promptly as the network connection features vector of no key words sorting, and commentaries on classics step S6; If do not finish just to change over to step S1.
Historical data base 4 also is connected with input unit 10, the network manager is according to output device 9 result displayed, other audit information in the coupling system connects network to be analyzed, and by input unit 10 auditing result is stored in the historical data base 4, described input unit is equipment such as keyboard and mouse.Wherein auditing result has comprised analysis and the judgement of system manager to the network behavior in the system, therefore can produce the network connection features vector that has key words sorting on a small quantity.The reliability of these network connection features vectors is higher in addition, therefore has bigger weights in the semi-supervised learning process.
A large amount of network connection features vectors are to generate according to the testing result of misapplying detection module 8, and lot of data is normal connection features data usually, a small amount of unusual connection features data.In addition, a large amount of networks connections are analyzed through abnormality detection module 6 and misuse detection module 8 in the system, so have the network connection features vector of a large amount of no key words sortings.
Semi-supervised adaptive learning modules 5 is the adaptive learning devices that adopt the semi-supervised learning algorithm, and learns and generate the abnormality detection rule on the training dataset in historical data base 4.Learner can adopt existing learner based on Bagging and Boosting (containing and acceleration) mode of learning commonly used.According to the characteristics that data constitute in 4 in the historical data base, semi-supervised adaptive learning modules 5 may adopt following three kinds of training modes:
C1: a spot of the network connection features vector of key words sorting arranged, do not have a network connection features vector of key words sorting in a large number.The network connection features vector that key words sorting is wherein arranged is the result that artificial audit produces, and may contain classification noise.
C2: a large amount of proper network connection features vectors, limited unusual network connection features vector, a large amount of no key words sorting network connection features vectors.Therefore the result that normal connection features data wherein detect from misuse may comprise classification noise.
C3: have the network connection features vector of key words sorting, a large amount of proper network connection features vector, limited unusual connection features vector sum not to have the network connection features vector of key words sorting in a large number on a small quantity.Wherein, have in the normal network connection features of the network connection features vector sum vector of key words sorting and may comprise classification noise.
Wherein situation C1 correspondence and is not disposed misuse detection system in the system, and the network connection features vector that therefore has classified information can only obtain by network manager's periodical audit; Network connection features vector among the situation C2 is all generated by misuse detection system, need not the network management personnel and participates in; Situation C3 then corresponding comprise the network connection features vector that audit process and misuse detection system generate simultaneously, be modal mode of operation.All only comprising partial data in above-mentioned three kinds of mode of operations has key words sorting information, therefore requires semi-supervised adaptive learning modules 5 can utilize part class indication information to finish the study of abnormality detection rule.
Abnormality detection module 6 will dynamically update according to the abnormality detection rule that semi-supervised adaptive learning modules 5 obtains and detect rule, and testing result is stored in the historical data base 4.The testing result of exception response module 7 comprehensive misuse detection modules 8 and 6 pairs of network messages of abnormality detection module and network connection features vector generates warning information, is submitted to the system manager then, and by output device 9 output testing results.
Claims (5)
1. self-adapting network intrusion detection system, comprise message preliminary treatment and forwarding module (1), misuse detection module (8), exception response module (7), output device (9), described message preliminary treatment and forwarding module (1) are used for extracting the non-encrypted message in the network connecting object and are forwarded to misuse detection module (8), described misuse detection module (8) is used for detected unusual network connecting object is forwarded to exception response module (7), it is characterized in that: this system also comprises real-time data base (2), characteristic arrangement and memory module (3), historical data base (4), semi-supervised adaptive learning modules (5), abnormality detection module (6)
Message preliminary treatment and forwarding module (1) also are used for extracting the network connection features vector of network connecting object and storing real-time data base (2) into;
Misuse detection module (8) also is used for storing the message testing result of non-encrypted message into real-time data base (2);
Characteristic arrangement and memory module (3) are used for making regular check on and integrate the network connection features vector sum message status in the real-time data base (2), whether store the network connection features vector after the arrangement into historical data base (4), described message status comprises: connection identifier, message identification, detect and testing result;
Semi-supervised adaptive learning modules (5) utilizes the semi-supervised learning algorithm to generate the abnormality detection rule, and the abnormality detection rule is dynamically updated in the abnormality detection module (6) according to the various data in the historical data base (4);
Abnormality detection module (6) is used for analyzing the network connection features vector in the real-time data base (2), then detected unusual network connecting object is forwarded to exception response module (7).
2. according to the described self-adapting network intrusion detection system of claim 1, it is characterized in that: described characteristic arrangement is handled the data in the real-time data base (2) according to the following steps with memory module (3):
S1, from real-time data base (2), take out a network connecting object, require this moment the testing result of all messages in this connecting object all to return from misuse detection module (8), perhaps the message in this connection is not misapplied detection;
Whether S2, the message of judging this connecting object send into misuse detection module (8), carry out corresponding operating if just change step S9 over to; Otherwise change step S3 over to;
S3, check the state of all messages in this connecting object, if the state that has a message is for unusual, show that misuse detection module (8) detects intrusion behavior from message, mean that this connecting object must be unusual a connection, sends check result into step S4;
S4, judge whether the relevant message of connecting object comprises the feature of known invasion, if comprise then jump to step S5; Otherwise this connection may be normal connection, then jumps to step S7;
S5, the exception class offset of corresponding network connection features vector is set in the real-time data base, and record is that testing result according to misuse detection module (8) is provided with, jumps to step S6;
S6, network connection features vector is removed from real-time data base (2), and be deposited in the historical data base (4), change step S1 over to and handle next network connecting object;
S7, judge whether the activity of current network connecting object finishes, finished just to forward to step S8 if network connects; Otherwise change step S1 over to;
S8, owing to connect and to finish, all messages all are normal condition, can judge that this connection is normal, therefore can the exception class offset be set to normal connection, and this result of mark is according to the testing result setting of misuse detection module (8), changes execution in step S6 over to;
S9, since this connecting object in message send into the misuse detection module (8), if this connection activity finishes, then directly the exception class offset of this connection features vector is labeled as sky, promptly as the network connection features vector of no key words sorting, and commentaries on classics step S6; If do not finish just to change over to step S1.
3. according to the described self-adapting network intrusion detection system of claim 1, it is characterized in that: the parameter of described network connection features vector comprises at least: connection identifier, capture time stamp, source address, source port number, destination address, destination slogan, the agreement of employing, Exception Type.
4. according to the described self-adapting network intrusion detection system of claim 1, it is characterized in that: the adaptive learning device that described semi-supervised adaptive learning modules (5) adopts is the learner of Bagging and Boosting learning model.
5. according to the described self-adapting network intrusion detection system of one of claim 1 to 4, it is characterized in that: historical data base (4) also is connected with input unit (10).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009101038804A CN101562539B (en) | 2009-05-18 | 2009-05-18 | Self-adapting network intrusion detection system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009101038804A CN101562539B (en) | 2009-05-18 | 2009-05-18 | Self-adapting network intrusion detection system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101562539A CN101562539A (en) | 2009-10-21 |
| CN101562539B true CN101562539B (en) | 2011-05-25 |
Family
ID=41221171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009101038804A Expired - Fee Related CN101562539B (en) | 2009-05-18 | 2009-05-18 | Self-adapting network intrusion detection system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101562539B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102291392B (en) * | 2011-07-22 | 2015-03-25 | 中国电力科学研究院 | Hybrid intrusion detection method based on Bagging algorithm |
| CN102591965B (en) * | 2011-12-30 | 2014-07-09 | 奇智软件（北京）有限公司 | Method and device for detecting black chain |
| CN105591977A (en) * | 2015-08-28 | 2016-05-18 | 杭州华三通信技术有限公司 | Message processing method and device |
| CN107169854B (en) * | 2016-03-07 | 2021-04-02 | 创新先进技术有限公司 | Data processing method and device |
| CN106302555A (en) * | 2016-11-10 | 2017-01-04 | 北京启明星辰信息安全技术有限公司 | A kind of network inbreak detection method and device |
| CN107948127A (en) * | 2017-09-27 | 2018-04-20 | 北京知道未来信息技术有限公司 | A kind of WAF detection methods and system based on feedback and supervised learning |
| CN108540473A (en) * | 2018-04-09 | 2018-09-14 | 华北理工大学 | A kind of data analysing method and data analysis set-up |
| CN110324323B (en) * | 2019-06-19 | 2024-01-19 | 全球能源互联网研究院有限公司 | New energy plant station network-related end real-time interaction process anomaly detection method and system |
-
2009
- 2009-05-18 CN CN2009101038804A patent/CN101562539B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN101562539A (en) | 2009-10-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101562539B (en) | Self-adapting network intrusion detection system | |
| US20210097624A1 (en) | Method and apparatus for increasing the density of data surrounding an event | |
| EP2011013B1 (en) | Merging multi-line log entries | |
| CN106101130B (en) | A kind of network malicious data detection method, apparatus and system | |
| CN108197261A (en) | A kind of wisdom traffic operating system | |
| CN108170580A (en) | A kind of rule-based log alarming method, apparatus and system | |
| US20130212257A1 (en) | Computer program and monitoring apparatus | |
| CN108737182A (en) | The processing method and system of system exception | |
| CN107423205A (en) | A kind of system failure method for early warning and system for anti-data-leakage system | |
| CN106708700A (en) | Operation and maintenance monitoring method and device applied to server side | |
| CN109660518A (en) | Communication data detection method, device and the machine readable storage medium of network | |
| Leszczyna et al. | Evaluation of open source SIEM for situation awareness platform in the smart grid environment | |
| CN103455313B (en) | Method and device for associating input information with output information of detected system | |
| CN108574627B (en) | SDN network multi-control-domain cooperative management method and system | |
| CN106209902A (en) | A kind of network safety system being applied to intellectual property operation platform and detection method | |
| CN108989084A (en) | Abnormal data monitoring method, apparatus and system | |
| CN114791846A (en) | Method for realizing observability aiming at cloud native chaos engineering experiment | |
| CN112001443A (en) | Network behavior data monitoring method and device, storage medium and electronic equipment | |
| JP6201079B2 (en) | Monitoring system and monitoring method | |
| CN113505044A (en) | Database warning method, device, equipment and storage medium | |
| CN103944779B (en) | A kind of WAP service features monitoring method and system | |
| CN110162444A (en) | A kind of system performance monitoring method and platform | |
| CN109245927A (en) | Warning system and method in cloud data system | |
| CN107682173A (en) | Fault automatic location method and system based on Trading Model | |
| CN108491259B (en) | Remote sensing algorithm flow Method of Scheduling Parallel and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110525 Termination date: 20120518 |