CN106657065A - Network abnormality detection method based on data mining - Google Patents
Network abnormality detection method based on data mining Download PDFInfo
- Publication number
- CN106657065A CN106657065A CN201611202992.1A CN201611202992A CN106657065A CN 106657065 A CN106657065 A CN 106657065A CN 201611202992 A CN201611202992 A CN 201611202992A CN 106657065 A CN106657065 A CN 106657065A
- Authority
- CN
- China
- Prior art keywords
- data
- cpp
- cluster
- detection
- call
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Computer Security & Cryptography (AREA)
- Artificial Intelligence (AREA)
- Genetics & Genomics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Physiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a network abnormality detection method based on data mining, which adopts a system standard input and output stdio, a system standard library stdlib and a system mathematical function library math and comprises the steps of: firstly, starting up a master program file detection.cpp, reading in data to be detected and carrying out preprocessing; calling a cluster analysis and generation module clust.cpp, using an individual obtained in the previous step as an initial center point of a cluster partitioning method, and by the module, carrying out partitioning on the data and generating a cluster; calling a data readability conversion module trap.cpp to carry out marking on the generated cluster, and determining types of normal data and abnormal data; and by an alert module alert.cpp, outputting information of the normal data and the abnormal data to a control console. According to the network abnormality detection method disclosed by the invention, a data mining technology is effectively applied to intrusion detection, and an application based on intrusion detection needs to improve an original clustering analysis algorithm, so that the algorithm can be suitable for an environment and a data type of intrusion detection, an aim of intrusion detection is fulfilled, a detection rate of intrusion detection is improved and a false alarm rate is reduced.
Description
Technical field
The invention belongs to Computer Applied Technology field, more specifically, more particularly to a kind of net based on data mining
Network method for detecting abnormality.
Background technology
Data mining is from being stored in database, to excavate in the mass data in data warehouse or other information storehouse and interesting know
The process of knowledge.Data mining is a large amount of fuzzy from database, in noisy initial data, excavate important according to certain rule
Information, extract valuable knowledge.Data mining technology is a cross discipline, is related to intrusion detection, intelligence database
With the multiple fields such as machine learning, data mining has become the research topic of hot topic.Application data is excavated in intrusion detection
Technology, can improve detection efficiency, enhance the adaptivity and autgmentability of system.Intrusion detection system based on data mining
System, can well protect the safety of computer network system, with very high researching value and realistic meaning.
The informatized service of 21 century brings earth-shaking change to human society, changes the information interchange in the whole world
Mode, with the development of computer and the communication technology, network becomes the key factor of world today's development.In recent years, computer
Network Development is extremely rapid, and information network has been related to the numerous areas such as government, military affairs, the culture and education of country, each weight of government
Big decision-making, commercial economy secret, bank capital flowing, energy statistics, scientific data etc. all contain substantial amounts of information flow, believe
Breath network has become the important guarantee of social development, is the symbol for embodying national comprehensive strength.With the networking of computer
And globalization, the every field of society generates qualitative leap in cybertimes, the study of people, work, lives and is all dissolved into
In network, by network people's shared resource.
Through the development of many decades, network environment has occurred that very big change, structure from simple to complex, application by
It is single to diversification, the normal table operation for safeguarding network also becomes extremely important problem.Various network technologies are in the time
The increase of extension spatially, number of users and equipment, attack takes place frequently, the destabilizing factor in these networks
The difficulty for making network management is increased.Serious by the criminal activity mouth benefit of network, the safety for ensureing computer network system is carved not
Rong Huan, network security has become the significant problem of national security.Only computer network security, the informationization of society could be normal
Development, the information of country could safety, the network life that could ensure the people do not encroached on, therefore, network security technology grinds
Study carefully with important social effect and realistic meaning.
At present, there are many unsafe factors in network application, be mainly shown as that leakage of information, information are distorted, illegally made
With Internet resources, invalid information infiltration etc., safety and the strick precaution of the network information seem more and more important.Computer network security is hidden
Trouble is more, and maintenance difficulties are big, and attack meanses become diversification, complication, intellectuality, it is therefore necessary to set up one safely and effectively
Protection system, just can guarantee that the stable operation of network health.
Traditional network security technology mainly has:Encryption technology, identity differentiate and authentication techniques, access control technology, anti-
Wall with flues technology etc., in the primary stage of evolution of computer networks, these safe practices serve certain effect, it is ensured that network
Normal operation and information exchange.In the face of current complicated network environment, traditional security protection technology mainly has following several
Individual problem:
1. these technologies belong to static security technology category, it is impossible to active tracing invader, while the safety in static defence
Strategy sacrifices the Partial rights of user, and this opening with network, sharing is disagreed；
2. the attack from internal system can not be prevented, it is helpless to the situation of authorized user's abusing computer and its resource；
3. due to the restriction of performance, it is impossible to provide real-time monitoring.
Therefore, the research emphasis of computer network security progressively turn to active tracing, active detecting by static security technology
Dynamic security technology.
The content of the invention
The invention aims to solve shortcoming present in prior art, and propose it is a kind of based on data mining
Network anomaly detection method.
For achieving the above object, the present invention provides following technical scheme：
A kind of network anomaly detection method based on data mining, including：
System standard input, output：stdio；
System standard storehouse:stdlib；
Systematic mathematical function library:math；
System standard input, output stream:iostream；
Specifically include following steps：
S1, first, starts master program file detection.cpp, reads in data to be detected and is pre-processed；
S2, to call program num.epp that quantizes, two files of data normalization program format.cpp to carry out data successively pre-
Process, improve the operational efficiency of algorithm below；
S3, the fitness for calling individual adaptation degree calculation procedure fitcal.cpp calculating individualities, for selection opertor part
Gambling disk is selected；
S4, call genetic operator program genetic.cpp；
S5, call cluster analysis and generation module clust.cpp, will back gained individuality as the initial of clustering method
Central point, thus module data are divided and are produced with cluster；
S6, call data readability modular converter trap.cpp to produce cluster be identified, determine normal data and exception
Data type；
S7, alarm module alert.cpp export the information of normal data and abnormal data to console.
Preferably, it is described to call comprising the following steps that for genetic operator program genetic.cpp in S4：
A first, calls selection opertor to select the higher individuality of fitness；
Then b, calls crossover operator moderately to intersect individuality, improves ability of searching optimum；
C is last, calls mutation operator moderately to make a variation individuality, improves local search ability, altogether iteration 4 times.
Preferably, in S5, the concrete grammar of the cluster is as follows：
Input：The data set A of density radius r, n bar record；Output：K cluster；Comprise the steps：
A, run-down data set A, For：Read each data i in sample set A；
B, the dot density of each data is calculated, and calculate the distance and D of each data point and distance and H；
If C, data object density value are less than density value Q, and D>H is then considered as the object for isolated point t；
D, the isolated point data removed in A, obtain new data set A', and record the number of samples m=n-t in A', and output is isolated
Point；
The algorithm of E, the acquisition initial cluster center of operational development, obtains k initial cluster center；
F, point set U is clustered, form k gathering UK。
Abnormality detection be according to the behavior of user or the normal degree of the normal behaviour in service of resource come judge whether invasion, can
To detect new attack behavior.The anomaly occurred in the communication of abnormality detection main detection, rather than known invasion row
For.Abnormality detection assume intrusion behavior be all it is abnormal, such as it is outside swarm into, internal upset operation and attack, based on this
It is individual it is assumed that the historical data of collection system normal behaviour, is the feature database that system sets up a normal behaviour with quantitative method,
So can just be considered as intrusion behavior with the different behavior of normal characteristics and operation in theory.Abnormality detection judges that system is provided
The service condition in source, monitors behavior of the user in system, by current movable feature and normally by running monitoring programme
The feature of behavior compares, if the difference degree of the feature of current movable feature and normal behaviour is sentenced more than the thresholds of setting
It is set to invasion, and sends alarm.For abnormality detection, the key of the method is the setting to abnormality degree threshold values and normal characteristics
Select.
The technique effect and advantage of the present invention：A kind of Network anomaly detection side based on data mining that the present invention is provided
Method, compared with conventional art, effectively application of the application data digging technology in intrusion detection, based on intrusion detection of the invention
Need to be improved original cluster algorithm so that algorithm can be applied to the environment and data type of intrusion detection,
Reach the purpose of intrusion detection, and improve the verification and measurement ratio of intrusion detection, reduce rate of false alarm；Meanwhile, invention introduces heredity is calculated
Method solves the defect that clustering algorithm converges on locally optimal solution, so as to further increase the verification and measurement ratio of intrusion detection, reduces
Rate of false alarm.
Specific embodiment
In order that the objects, technical solutions and advantages of the present invention become more apparent, below in conjunction with specific embodiment, to this
Invention is further elaborated.It should be appreciated that specific embodiment described herein is only to explain the present invention, not
For limiting the present invention.Based on the embodiment in the present invention, those of ordinary skill in the art are not before creative work is made
The every other embodiment for being obtained is put, the scope of protection of the invention is belonged to.
A kind of network anomaly detection method based on data mining, including：
System standard input, output：stdio；
System standard storehouse:stdlib；
Systematic mathematical function library:math；
System standard input, output stream:iostream；
Specifically include following steps：
S1, first, starts master program file detection.cpp, reads in data to be detected and is pre-processed；
S2, to call program num.epp that quantizes, two files of data normalization program format.cpp to carry out data successively pre-
Process, improve the operational efficiency of algorithm below；
S3, the fitness for calling individual adaptation degree calculation procedure fitcal.cpp calculating individualities, for selection opertor part
Gambling disk is selected；
S4, call genetic operator program genetic.cpp；
S5, call cluster analysis and generation module clust.cpp, will back gained individuality as the initial of clustering method
Central point, thus module data are divided and are produced with cluster；
S6, call data readability modular converter trap.cpp to produce cluster be identified, determine normal data and exception
Data type；
S7, alarm module alert.cpp export the information of normal data and abnormal data to console.
Specifically, it is described to call comprising the following steps that for genetic operator program genetic.cpp in S4：
A first, calls selection opertor to select the higher individuality of fitness；
Then b, calls crossover operator moderately to intersect individuality, improves ability of searching optimum；
C is last, calls mutation operator moderately to make a variation individuality, improves local search ability, altogether iteration 4 times.
Specifically, in S5, the concrete grammar of the cluster is as follows：
Input：The data set A of density radius r, n bar record；Output：K cluster；Comprise the steps：
A, run-down data set A, For：Read each data i in sample set A；
B, the dot density of each data is calculated, and calculate the distance and D of each data point and distance and H；
If C, data object density value are less than density value Q, and D>H is then considered as the object for isolated point t；
D, the isolated point data removed in A, obtain new data set A', and record the number of samples m=n-t in A', and output is isolated
Point；
The algorithm of E, the acquisition initial cluster center of operational development, obtains k initial cluster center；
F, point set U is clustered, form k gathering UK。
In sum：The present invention can not effectively detect the attack of UNKNOWN TYPE based on the method for misuse detection, based on different
Normal intrusion detection method does not rely on the feature database of intrusion detection, can detect the attack of UNKNOWN TYPE, it is necessary to research row
The method of effective abnormality detection.Data mining technology is present invention employs, intruding detection system is improved, can effectively be fitted
For the data mining algorithm of intruding detection system, further improve verification and measurement ratio and reduce rate of false alarm.When the attack to system is sent out
When raw, system is capable of detecting when attack, reports to the police to network manager in time, reaches the purpose of intrusion detection.
Finally it should be noted that：The preferred embodiments of the present invention are the foregoing is only, the present invention is not limited to,
Although being described in detail to the present invention with reference to the foregoing embodiments, for a person skilled in the art, it still may be used
To modify to the technical scheme described in foregoing embodiments, or equivalent is carried out to which part technical characteristic,
All any modification, equivalent substitution and improvements within the spirit and principles in the present invention, made etc., should be included in the present invention's
Within protection domain.
Claims (3)
1. a kind of network anomaly detection method based on data mining, including：
System standard input, output：stdio；
System standard storehouse:stdlib；
Systematic mathematical function library:math；
System standard input, output stream:iostream；
It is characterized in that：Specifically include following steps：
S1, first, starts master program file detection.cpp, reads in data to be detected and is pre-processed；
S2, to call program num.epp that quantizes, two files of data normalization program format.cpp to carry out data successively pre-
Process, improve the operational efficiency of algorithm below；
S3, the fitness for calling individual adaptation degree calculation procedure fitcal.cpp calculating individualities, for selection opertor part
Gambling disk is selected；
S4, call genetic operator program genetic.cpp；
S5, call cluster analysis and generation module clust.cpp, will back gained individuality as the initial of clustering method
Central point, thus module data are divided and are produced with cluster；
S6, call data readability modular converter trap.cpp to produce cluster be identified, determine normal data and exception
Data type；
S7, alarm module alert.cpp export the information of normal data and abnormal data to console.
2. a kind of network anomaly detection method based on data mining according to claim 1, it is characterised in that：In S4
In, it is described to call comprising the following steps that for genetic operator program genetic.cpp：
A first, calls selection opertor to select the higher individuality of fitness；
Then b, calls crossover operator moderately to intersect individuality, improves ability of searching optimum；
C is last, calls mutation operator moderately to make a variation individuality, improves local search ability, altogether iteration 4 times.
3. a kind of network anomaly detection method based on data mining according to claim 1, it is characterised in that：In S5
In, the concrete grammar of the cluster is as follows：
Input：The data set A of density radius r, n bar record；Output：K cluster；Comprise the steps：
A, run-down data set A, For：Read each data i in sample set A；
B, the dot density of each data is calculated, and calculate the distance and D of each data point and distance and H；
If C, data object density value are less than density value Q, and D>H is then considered as the object for isolated point t；
D, the isolated point data removed in A, obtain new data set A', and record the number of samples m=n-t in A', and output is isolated
Point；
The algorithm of E, the acquisition initial cluster center of operational development, obtains k initial cluster center；
F, point set U is clustered, form k gathering UK。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611202992.1A CN106657065A (en) | 2016-12-23 | 2016-12-23 | Network abnormality detection method based on data mining |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611202992.1A CN106657065A (en) | 2016-12-23 | 2016-12-23 | Network abnormality detection method based on data mining |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106657065A true CN106657065A (en) | 2017-05-10 |
Family
ID=58826561
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611202992.1A Pending CN106657065A (en) | 2016-12-23 | 2016-12-23 | Network abnormality detection method based on data mining |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106657065A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107341239A (en) * | 2017-07-05 | 2017-11-10 | 广东工业大学 | A kind of company-data analysis method and device |
| CN108737406A (en) * | 2018-05-10 | 2018-11-02 | 北京邮电大学 | A kind of detection method and system of abnormal flow data |
| CN109978070A (en) * | 2019-04-03 | 2019-07-05 | 北京市天元网络技术股份有限公司 | A kind of improved K-means rejecting outliers method and device |
| CN111460246A (en) * | 2019-12-19 | 2020-07-28 | 南京柏跃软件有限公司 | Real-time activity abnormal person discovery method based on data mining and density detection |
| CN113869455A (en) * | 2021-10-13 | 2021-12-31 | 平安科技（深圳）有限公司 | Unsupervised clustering method and device, electronic equipment and medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104753920A (en) * | 2015-03-01 | 2015-07-01 | 江西科技学院 | Quantum genetic algorithm based intrusion detection method |
-
2016
- 2016-12-23 CN CN201611202992.1A patent/CN106657065A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104753920A (en) * | 2015-03-01 | 2015-07-01 | 江西科技学院 | Quantum genetic algorithm based intrusion detection method |
Non-Patent Citations (2)
| Title |
|---|
| 宋先强等: "基于数据挖掘的网络异常检测方法的研究", 《电子技术研发》 * |
| 杨种学: "一种基于动态聚类的异常入侵检测方法", 《计算机工程与设计》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107341239A (en) * | 2017-07-05 | 2017-11-10 | 广东工业大学 | A kind of company-data analysis method and device |
| CN107341239B (en) * | 2017-07-05 | 2020-08-07 | 广东工业大学 | Cluster data analysis method and device |
| CN108737406A (en) * | 2018-05-10 | 2018-11-02 | 北京邮电大学 | A kind of detection method and system of abnormal flow data |
| CN108737406B (en) * | 2018-05-10 | 2020-08-04 | 北京邮电大学 | Method and system for detecting abnormal flow data |
| CN109978070A (en) * | 2019-04-03 | 2019-07-05 | 北京市天元网络技术股份有限公司 | A kind of improved K-means rejecting outliers method and device |
| CN111460246A (en) * | 2019-12-19 | 2020-07-28 | 南京柏跃软件有限公司 | Real-time activity abnormal person discovery method based on data mining and density detection |
| CN111460246B (en) * | 2019-12-19 | 2020-12-08 | 南京柏跃软件有限公司 | Real-time activity abnormal person discovery method based on data mining and density detection |
| CN113869455A (en) * | 2021-10-13 | 2021-12-31 | 平安科技（深圳）有限公司 | Unsupervised clustering method and device, electronic equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108566364B (en) | Intrusion detection method based on neural network | |
| Khan et al. | Malicious insider attack detection in IoTs using data analytics | |
| CN106657065A (en) | Network abnormality detection method based on data mining | |
| Zhu et al. | Data mining for network intrusion detection: a comparison of alternative methods | |
| CN108681966A (en) | A kind of information monitoring method and device based on block chain | |
| Murali et al. | A survey on intrusion detection approaches | |
| WO2004097601A1 (en) | System and method for determining a computer user profile from a motion-based input device | |
| CN111600905A (en) | Anomaly detection method based on Internet of things | |
| CN115883213B (en) | APT detection method and system based on continuous time dynamic heterogeneous graph neural network | |
| CN116957049B (en) | Unsupervised internal threat detection method based on countermeasure self-encoder | |
| CN105262715A (en) | Abnormal user detection method based on fuzzy sequential association pattern | |
| CN114598551A (en) | Information network security early warning system for dealing with continuous threat attack | |
| Tundis et al. | Challenges and available solutions against organized cyber-crime and terrorist networks | |
| Choksi et al. | Intrusion detection system using self organizing map: a survey | |
| Mehmood et al. | Privilege escalation attack detection and mitigation in cloud using machine learning | |
| CN117478403A (en) | Whole scene network security threat association analysis method and system | |
| CN109871711B (en) | Ocean big data sharing and distributing risk control model and method | |
| US20230164162A1 (en) | Valuable alert screening method efficiently detecting malicious threat | |
| Zhao et al. | A closed-loop hybrid supervision framework of cryptocurrency transactions for data trading in IoT | |
| Zhang | Analysis of Network Security Countermeasures From the Perspective of Improved FS Algorithm and ICT Convergence | |
| Xie et al. | Application of Big Data in Public Security Governance: Dilemma, Risk and Optimization Path | |
| Azanguezet Quimatio et al. | HOrBAC Optimization Based on Suspicious Behavior Detection Using Information Theory | |
| Wei et al. | Extracting novel attack strategies for industrial cyber-physical systems based on cyber range | |
| Sahifa | Implementation of intrusion detection systems to detect phishing in the banking industry | |
| López et al. | CGAPP: A continuous group authentication privacy-preserving platform for industrial scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170510 |