CN112583842A - Network security situation awareness system platform based on data stream processing - Google Patents
Network security situation awareness system platform based on data stream processing Download PDFInfo
- Publication number
- CN112583842A CN112583842A CN202011545847.XA CN202011545847A CN112583842A CN 112583842 A CN112583842 A CN 112583842A CN 202011545847 A CN202011545847 A CN 202011545847A CN 112583842 A CN112583842 A CN 112583842A
- Authority
- CN
- China
- Prior art keywords
- data
- situation
- module
- network
- acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a network security situation perception system platform based on data stream processing, which comprises a data acquisition module, a processing module, a recording module and a network situation visualization module, wherein the data acquisition module is used for acquiring data; the processing module is used for classifying the data acquired by the data acquisition module, processing the acquired and counted data information P at the later stage, and autonomously judging whether the data information P is an intruder or not without autonomously defending the intruder; the recording module is used for counting the operation data of the processing module, so that a later-stage operator can repair and process the operation data conveniently, and meanwhile, the operation data is stored in the database, and later-stage independent processing is facilitated. The data acquisition module of the network security situation awareness system platform based on data stream processing is divided into portal data acquisition and gateway data acquisition, and the data acquisition module enables the data acquisition amount to be large, so that the data can be acquired better, detection and analysis of a database can be performed conveniently at the later stage, and the network situation analysis correctness is enlarged.
Description
Technical Field
The invention relates to the technical field of network security situation awareness, in particular to a network security situation awareness system platform based on data stream processing.
Background
With the rapid development of computer networks, the weak links, various network attack modes and various kinds of automatic attack tools existing in the networks emerge endlessly, various network intrusions appear at the present, network security events frequently occur, the network security situation awareness technology bears important tasks in the face of large-scale network environments, the monitoring of network security is improved to a higher level, the network security situation awareness is taken as the hot direction of the information era, and the key effects are achieved in the places of accelerating disaster response speed, improving the anti-reactive capability, reducing harm loss and the like.
However, in the use process of the existing network security situation awareness system, due to the operation mode, the log records of the security devices are used as main data sources, the system is oriented to medium and small-scale networks, and limitations exist in the aspects of comprehensiveness of the data sources and calculation real-time performance.
Therefore, we propose a network security situation awareness system platform based on data stream processing so as to solve the problems proposed in the above.
Disclosure of Invention
The invention aims to provide a network security situation awareness system platform based on data stream processing, and aims to solve the problem that the existing network situation awareness system platform cannot well perform data-based arrangement statistics and technical processing in the background art.
In order to achieve the purpose, the invention provides the following technical scheme: a network security situation perception system platform based on data stream processing comprises a data acquisition module, a processing module, a recording module and a network situation visualization module;
the data acquisition module is used for acquiring and counting data information P;
the processing module is used for classifying the data acquired by the data acquisition module, processing the acquired and counted data information P at the later stage, and autonomously judging whether the data information P is an intruder or not without autonomously defending the intruder;
the recording module is used for counting the operation data of the processing module, so that a later-stage operator can conveniently repair and process the operation data, and meanwhile, the operation data is stored in the database, so that later-stage autonomous processing is facilitated;
and the network situation visualization module is used for visually and intuitively displaying the network situation through the display screen, so that information change statistics and situation analysis reports can be conveniently carried out on the network situation.
Preferably, the data acquisition module is based on a portal system server and a gateway, and the data acquisition is divided into active acquisition and passive acquisition, the active acquisition is that situation acquisition can be automatically and manually performed, the passive acquisition can be that updating and data transmission of a database can be manually performed, and in the acquisition process, if the network resources are classified and judged to be P-type data, the data acquisition module performs collection.
Preferably, the processing module calculates based on a hash function, and in the using process, analyzes the data information P to obtain the network condition of the data flow P, calls the hash function to calculate the network attribute information, and introduces the data flow P into different corresponding data lists according to the difference of hash values.
Preferably, the processing module contains situation prediction analysis understanding, the subprogram and the situation analysis system are both in an SOA structure in the understanding process, data discovered by the platform is uploaded in an XML format, the data is classified, and when the two data types are equal, the data are equipment data and link data respectively.
Preferably, the processing module includes a situation analysis module, in the analysis process, a data stream and preset characters in an occurrence time period are obtained, the preset fields are processed according to a preset BM algorithm, keywords T and P can be produced, in the calculation process, the characters are matched from right to left, if matching occurs when P is not equal to T and T is not equal to P mode, the mode P is moved backward until P is located on the right side of T, the mode P is used for aggregating the data stream into corresponding data nodes, and the data are aggregated integrally according to the preset fields.
Preferably, the recording module contains a network diary and a situation analysis data record, and can perform statistical listing on the aggregated data, analyze the listed data and send the analyzed data to the situation library, and the situation library automatically updates and learns the data.
Preferably, the network situation visualization module is divided into a security network board, a real-time tracking security board, an information change board, a situation analysis report board and a display area for each board, so that a user can observe the network situation more intuitively
Compared with the prior art, the invention has the beneficial effects that: the network security situation awareness system platform based on data stream processing;
1. the data acquisition module is divided into portal data acquisition and gateway data acquisition, and the data acquisition module enables the data acquisition amount to be large, so that the data can be better acquired, the detection and analysis of the database at the later stage are facilitated, and the network situation analysis correctness is enlarged;
2. and the situation analysis module adopts a BM algorithm to process the preset field, so that the acquired data stream and the preset field in the generation end time are processed by the BM algorithm in the analysis process, and the computed structure is more accurate due to the uniqueness of the BM algorithm, thereby facilitating the later-stage integral aggregation statistics of the data.
Drawings
FIG. 1 is a schematic structural diagram of a network security situation awareness system platform based on data stream processing according to the present invention;
FIG. 2 is a schematic diagram of a data acquisition module according to the present invention;
FIG. 3 is a schematic view of the structure of the present invention;
FIG. 4 is a schematic diagram of a data module acquisition architecture according to the present invention;
FIG. 5 is a schematic diagram of a data analysis and determination structure according to the present invention;
FIG. 6 is a schematic diagram of the situation analysis structure of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-6, the present invention provides a technical solution: a network security situation perception system platform based on data stream processing comprises a data acquisition module, a processing module, a recording module and a network situation visualization module;
the data acquisition module is used for acquiring and counting data information P;
the processing module is used for classifying the data acquired by the data acquisition module, processing the acquired and counted data information P at the later stage, and autonomously judging whether the data information P is an intruder or not without autonomously defending the intruder;
the recording module is used for counting the operation data of the processing module, so that a later-stage operator can conveniently repair and process the operation data, and meanwhile, the operation data is stored in the database, so that later-stage autonomous processing is facilitated;
and the network situation visualization module is used for visually and intuitively displaying the network situation through the display screen, so that information change statistics and situation analysis reports can be conveniently carried out on the network situation.
The data acquisition module is based on the portal system server and the gateway, and data acquisition is divided into active acquisition and passive acquisition, the active acquisition can be automatically and artificially carried out situation acquisition, the passive acquisition can be artificially updated and transmitted to the database, network resources are automatically classified and judged in the acquisition process and are collected if the network resources are P-type data, the data acquisition source is increased, the data volume of a large database is improved, the data comparability is higher, and the later-stage data are better subjected to statistical analysis.
The processing module calculates based on a hash function, analyzes the data information P to obtain the network condition of the data stream P in the using process, calls the hash function to calculate the network attribute information, and respectively introduces the data stream P into different corresponding data lists according to different hash values, so as to be convenient for analyzing the obtained data.
The processing module contains situation prediction analysis understanding, a subprogram and a situation analysis system are in an SOA structure in the understanding process, data discovered by the platform is uploaded in an XML format, the data is classified, and when the data are of two data types, namely equipment data and link data, the situation analysis system reads XML files from an appointed directory and writes the XML files into a database due to loose coupling between the systems, so that the situation prediction analysis understanding is convenient for later use, the data are checked and analyzed, and the data volume is more accurate.
The processing module comprises a situation analysis module, in the analysis process, data streams and preset characters in an occurrence time period are obtained, preset fields are processed according to a preset BM algorithm, keywords T and P can be produced, in the calculation process, the characters are matched from right to left, if the matching occurs when P is not equal to T and T is not equal to a P mode, the mode P moves backwards until P is located on the right side of T, the mode P is used for aggregating the data streams to corresponding data nodes, and the data are aggregated integrally according to the preset fields, so that the situation analysis is more accurate.
The recording module contains a network diary and a situation analysis data record, and can perform statistical listing on the aggregated data, analyze and send the listed data to the situation library, and automatically update and learn the situation library, so that the system platform achieves the effect of autonomous learning and upgrading.
The network situation visualization module is divided into a security network plate, a real-time tracking security plate, an information change plate, a situation analysis report plate and a display area corresponding to each plate, so that a user can observe the network situation more intuitively.
The working principle is as follows: when the network security situation awareness system platform based on data stream processing is used, firstly, when the platform is used, as shown in fig. 1, the platform is provided with a data acquisition module, a processing module and a recording analysis and network security situation visualization module, when the platform is used, firstly, data is acquired, as shown in fig. 2, the data sources are divided into multiple ways, so that the data acquisition amount is large, the platform is better processed based on the data stream, in the using process, the acquired data can be analyzed through a hash function, the acquired data are classified, the acquired data are respectively put into a database, as shown in fig. 3, if an invader is encountered, the processing module starts to operate, firstly, the invasion behavior is detected, whether the invasion is the invader is judged, if so, the identity of the invader is investigated and the invasion success rate of the invasion is judged, and if the other measures are needed to be taken to prevent the intrusion, calling a corresponding processing mode from the situation library to process the intrusion situation, wherein the processing situation can be reflected in the network diary, and the situation library automatically updates the database by adopting a BM algorithm in the process of normal use, so that the countermeasures in the database are gradually increased.
To thereby carry out a series of tasks, the contents of which are not described in detail in the present specification are prior art well known to those skilled in the art.
Although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that various changes in the embodiments and/or modifications of the invention can be made, and equivalents and modifications of some features of the invention can be made without departing from the spirit and scope of the invention.
Claims (7)
1. A network security situation awareness system platform based on data stream processing is characterized in that: the system comprises a data acquisition module, a processing module, a recording module and a network situation visualization module;
the data acquisition module is used for acquiring and counting data information P;
the processing module is used for classifying the data acquired by the data acquisition module, processing the acquired and counted data information P at the later stage, and autonomously judging whether the data information P is an intruder or not without autonomously defending the intruder;
the recording module is used for counting the operation data of the processing module, so that a later-stage operator can conveniently repair and process the operation data, and meanwhile, the operation data is stored in the database, so that later-stage autonomous processing is facilitated;
and the network situation visualization module is used for visually and intuitively displaying the network situation through the display screen, so that information change statistics and situation analysis reports can be conveniently carried out on the network situation.
2. The network security situation awareness system platform based on data stream processing according to claim 1, wherein: the data acquisition module is based on a portal system server and a gateway, and data acquisition is divided into active acquisition and passive acquisition, wherein the active acquisition is that situation acquisition can be automatically and manually carried out, the passive acquisition can be that updating and data transmission of a database can be manually carried out, and network resources are automatically classified and judged in the acquisition process and are collected if the data are P-type data.
3. The network security situation awareness system platform based on data stream processing according to claim 1, wherein: the processing module is used for calculating based on a hash function, analyzing according to the data information P to obtain the network condition of the data flow P in the using process, calling the hash function to calculate the network attribute information, and respectively importing the data flow P into different corresponding data lists according to different hash values.
4. The network security situation awareness system platform based on data stream processing according to claim 1, wherein: the processing module contains situation prediction analysis understanding, a subprogram and a situation analysis system are in an SOA structure in the understanding process, data discovered by the platform is uploaded in an XML format, the data is classified, and the data is of two data types, namely equipment data and link data.
5. The network security situation awareness system platform based on data stream processing according to claim 1, wherein: the processing module comprises a situation analysis module, in the analysis process, data streams and preset characters in an occurrence time period are obtained, preset fields are processed according to a preset BM algorithm, keywords T and P can be produced, in the calculation process, the characters are matched from right to left, if the matching occurs when P is not equal to T and T is not equal to a P mode, the mode P moves backwards until P is located on the right side of T, the mode P is used for aggregating the data streams to corresponding data nodes, and the data are aggregated integrally according to the preset fields.
6. The network security situation awareness system platform based on data stream processing according to claim 1, wherein: the recording module comprises a network diary and a situation analysis data record, and can perform statistical listing on the aggregated data, analyze and send the listed data to a situation library, and automatically update and learn the situation library.
7. The network security situation awareness system platform based on data stream processing according to claim 1, wherein: the network situation visualization module is divided into a security network plate, a real-time tracking security plate, an information change plate, a situation analysis report plate and a display area corresponding to each plate, so that a user can observe the network situation more intuitively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011545847.XA CN112583842A (en) | 2020-12-23 | 2020-12-23 | Network security situation awareness system platform based on data stream processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011545847.XA CN112583842A (en) | 2020-12-23 | 2020-12-23 | Network security situation awareness system platform based on data stream processing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112583842A true CN112583842A (en) | 2021-03-30 |
Family
ID=75139268
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011545847.XA Pending CN112583842A (en) | 2020-12-23 | 2020-12-23 | Network security situation awareness system platform based on data stream processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112583842A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113536311A (en) * | 2021-07-20 | 2021-10-22 | 国网新疆电力有限公司信息通信公司 | Network security situation sensing system and method based on AI technology |
| CN114826658A (en) * | 2022-03-15 | 2022-07-29 | 中国电子科技集团公司第三十研究所 | Controllable situation custom presentation method based on data middleboxes |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107958322A (en) * | 2017-10-09 | 2018-04-24 | 中国电子科技集团公司第二十八研究所 | A kind of urban network spatial synthesis governing system |
| CN109885562A (en) * | 2019-01-17 | 2019-06-14 | 安徽谛听信息科技有限公司 | A kind of big data intelligent analysis system based on cyberspace safety |
| CN110430212A (en) * | 2019-08-14 | 2019-11-08 | 杭州安恒信息技术股份有限公司 | The Internet of Things of multivariate data fusion threatens cognitive method and system |
| US20200259660A1 (en) * | 2019-02-07 | 2020-08-13 | Nebbiolo Technologies, Inc. | Trusted Virtual Process Execution Contexts Using Secure Distributed Ledger |
| CN111556066A (en) * | 2020-05-08 | 2020-08-18 | 国家计算机网络与信息安全管理中心 | Network behavior detection method and device |
| CN111882179A (en) * | 2020-07-09 | 2020-11-03 | 福建奇点时空数字科技有限公司 | Network security situation awareness system platform based on data stream processing |
-
2020
- 2020-12-23 CN CN202011545847.XA patent/CN112583842A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107958322A (en) * | 2017-10-09 | 2018-04-24 | 中国电子科技集团公司第二十八研究所 | A kind of urban network spatial synthesis governing system |
| CN109885562A (en) * | 2019-01-17 | 2019-06-14 | 安徽谛听信息科技有限公司 | A kind of big data intelligent analysis system based on cyberspace safety |
| US20200259660A1 (en) * | 2019-02-07 | 2020-08-13 | Nebbiolo Technologies, Inc. | Trusted Virtual Process Execution Contexts Using Secure Distributed Ledger |
| CN110430212A (en) * | 2019-08-14 | 2019-11-08 | 杭州安恒信息技术股份有限公司 | The Internet of Things of multivariate data fusion threatens cognitive method and system |
| CN111556066A (en) * | 2020-05-08 | 2020-08-18 | 国家计算机网络与信息安全管理中心 | Network behavior detection method and device |
| CN111882179A (en) * | 2020-07-09 | 2020-11-03 | 福建奇点时空数字科技有限公司 | Network security situation awareness system platform based on data stream processing |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113536311A (en) * | 2021-07-20 | 2021-10-22 | 国网新疆电力有限公司信息通信公司 | Network security situation sensing system and method based on AI technology |
| CN114826658A (en) * | 2022-03-15 | 2022-07-29 | 中国电子科技集团公司第三十研究所 | Controllable situation custom presentation method based on data middleboxes |
| CN114826658B (en) * | 2022-03-15 | 2023-05-23 | 中国电子科技集团公司第三十研究所 | Controllable situation custom presentation method based on data center |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103001811B (en) | Fault locating method and device | |
| US8150783B2 (en) | Security system for and method of detecting and responding to cyber attacks on large network systems | |
| US20190121969A1 (en) | Graph Model for Alert Interpretation in Enterprise Security System | |
| CN110213077A (en) | A kind of method, apparatus and system of determining electric power monitoring system security incident | |
| US8856313B2 (en) | Systems and methods for using provenance information for data retention in stream-processing | |
| CN106371986A (en) | Log treatment operation and maintenance monitoring system | |
| CN108809701A (en) | A kind of data center's wisdom data platform and its implementation | |
| CN112583842A (en) | Network security situation awareness system platform based on data stream processing | |
| CN106375339A (en) | Attack mode detection method based on event slide window | |
| CN111181799B (en) | Network traffic monitoring method and equipment | |
| CN112416872A (en) | Cloud platform log management system based on big data | |
| CN106104556A (en) | Log analysis system | |
| CN116232963B (en) | Link tracking method and system | |
| CN113612763A (en) | Network attack detection device and method based on network security malicious behavior knowledge base | |
| CN111800389A (en) | Port network intrusion detection method based on Bayesian network | |
| CN116166505A (en) | Monitoring platform, method, storage medium and equipment for dual-state IT architecture in financial industry | |
| CN118041699B (en) | Network intrusion positioning system based on artificial intelligence | |
| Lan et al. | Some special issues of network security monitoring on big data environments | |
| CN115776449A (en) | Train Ethernet communication state monitoring method and system | |
| CN114579407A (en) | Causal relationship inspection and micro-service index prediction alarm method | |
| Laue et al. | A SIEM architecture for multidimensional anomaly detection | |
| Laue et al. | A siem architecture for advanced anomaly detection | |
| CN111427749B (en) | Monitoring tool and method for ironic service in opentack environment | |
| CN117729043A (en) | Network security early warning method and system based on big data | |
| CN111882179A (en) | Network security situation awareness system platform based on data stream processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210330 |