CN109379390B - Network security baseline generation method based on full flow - Google Patents
Network security baseline generation method based on full flow Download PDFInfo
- Publication number
- CN109379390B CN109379390B CN201811589819.0A CN201811589819A CN109379390B CN 109379390 B CN109379390 B CN 109379390B CN 201811589819 A CN201811589819 A CN 201811589819A CN 109379390 B CN109379390 B CN 109379390B
- Authority
- CN
- China
- Prior art keywords
- data
- network
- security baseline
- network security
- application layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Small-Scale Networks (AREA)
Abstract
The invention discloses a network security baseline generation method based on full flow, which comprises the steps of collecting, analyzing and processing network full flow data, generating formatted data and storing the formatted data in a distributed database, and then carrying out operations such as aggregation, analysis, statistics and the like on the formatted data to generate a network security baseline, wherein the network security baseline can be used for identifying network intrusion behaviors. The network security baseline generation method based on the full flow can generate a more comprehensive network security baseline, can rapidly generate the security baseline in the whole regional network in a large scale, can rapidly and flexibly modify the network security baseline, has application universality, and can be widely applied to various network environments. In addition, the problem that a traditional network security baseline generation method needs to depend on cooperation of service experts, network security experts and network operation and maintenance personnel is solved, and the technical threshold for generating the network security baseline is greatly reduced.
Description
Technical Field
The invention relates to a network security baseline, in particular to a network security baseline generation method based on full flow.
Background
In the large background of the rapid development of the internet, more and more network security issues are exposed undoubtedly. With the frequent occurrence of security events of large enterprises, network security management becomes a hot topic of social attention. As an important link in network security management, network security baseline generation is widely applied to industries such as telecommunications, power, finance and the like which use communication networks and information systems based on IP networks and computer technologies. Network security baselines play an important fundamental role in improving the security of communication networks and information systems in these industries.
The network security baseline refers to the minimum security guarantee for a communication network element, i.e. the most basic security requirements that the communication network element needs to meet. The network security baseline specification is the minimum security configuration requirement of various systems and devices in the communication network unit. The network security baseline is a standard which is uniformly followed by related departments in an enterprise, and can be applied to each stage of the whole life cycle of network units such as design construction, network access detection, daily maintenance, compliance inspection, network quitting and the like. In the field of network security management, how to balance cost investment and risk is always a difficult problem, and the proposal of a network security baseline plays a role in ensuring the balance to be relatively stable. The construction and implementation of the network security baseline can ensure that the security protection of all systems and equipment in the communication network reaches the uniform and minimum required security level, facilitate the maintenance and management, improve the overall security protection level of the network and reduce the potential safety hazard.
The traditional network security baseline generation idea is as follows: aiming at mainstream network equipment, security equipment, an operating system, data and an application system and middleware of important network units applied in the current network, basic security configuration requirements and parameter thresholds which need to be followed for ensuring the basic security operation are determined. The network security baseline establishing method needs the cooperation of security experts, service experts and security operation and maintenance personnel, and manually sets various security baseline thresholds and conditions. The method has the problems of high threshold, difficult operation, inflexibility, easy error and the like.
Disclosure of Invention
In order to overcome the above disadvantages of the prior art, the present invention provides a network security baseline generation method based on full traffic, which generates formatted data by collecting, analyzing and processing network full traffic data and storing the formatted data in a distributed database, and then performs operations such as aggregation, analysis and statistics on the formatted data to generate a network security baseline, wherein the network security baseline can be used for identifying network intrusion behavior, and specifically comprises the following steps:
s1, deep analysis of network data: bypass collection is carried out on a network data packet through a mirror image port of the switch, deep analysis is carried out on partial application layer protocols, public data contents and partial deep analysis data information of the application layers in the network message are extracted, and formatted data are generated;
s2, data cleaning and extraction: cleaning the data of the formatted data generated after the acquisition and/or analysis, removing repeated or abnormal data, extracting all public data contents, and extracting key information of an application layer of the application layer data contents which are analyzed partially and deeply;
s3, distributed persistence: importing the extracted data information into a distributed database for persistence, wherein the data information is basic data for generating a network security baseline subsequently;
s4, receiving parameter input: receiving a security baseline generation parameter which is input by a user and comprises a starting time stamp, an ending time stamp and an application layer protocol type;
s5, data aggregation: extracting basic data in the time range from a distributed database according to initial timestamp and end timestamp parameters input by a user, generating an aggregated data set A according to network layer protocols, application layer protocols, source ip addresses, destination ip addresses, source mac addresses and destination mac address information in the basic data, and then aggregating again on the basis of the aggregated data set A according to the application layer protocols input by the user to construct an aggregated data set B of a specified application layer protocol;
s6, data analysis and calculation: and (3) generating a unique identification number ID through a character string formed by splicing all fields of each piece of data of the aggregation data set B through a secure hash algorithm (SHA1), and persisting the aggregation data set B with the ID into a distributed database, namely forming a network security baseline of a specified application layer protocol.
Further, the step S1 of deeply parsing the network data includes the following sub-steps:
s11, acquiring a device serial number, creating a processing sub-process, allocating a shared memory of the sub-process, and starting a write permission for a shared memory buffer area;
s12, initializing a sub-process;
s13, initializing a packet capturing interface and starting a packet capturing thread;
s14, initializing a storage thread and starting the storage thread;
s15, initializing a magnetic disk clearing thread, and starting a magnetic disk clearing function;
and S16, monitoring the subprocess, and restarting the subprocess if unexpected shutdown exists.
Further, in the step S1, in the deep network data analysis, the public data content includes: timestamp, source ip address, destination ip address, source port number, destination port number, source mac address, destination mac address, network layer protocol, and application layer protocol.
Further, in the data aggregation in step S5, if the application layer protocol input by the user is the IEC104 protocol, the aggregated data set B is generated from the request common body address, the response common body address, the message type, the type identifier, and the transmission reason of the IEC104 data in the base data.
Further, in the data analysis and calculation of step S6, if a security baseline of the application layer protocol originally exists in the distributed database, the data will be automatically updated or refined based on the unique identification number ID, and the user may also independently insert or delete a piece of designated security baseline data based on the unique identification number ID.
The method has the following characteristics:
1) full automation: the baseline can be automatically generated in one key mode, and a user only needs to set the time range of the network security baseline and click the baseline generation button, so that the establishment of a baseline result can be waited.
2) Monitoring the whole network: based on strong big data processing capacity, the network security baseline of the whole network full flow can be established without obstacles.
3) High-efficient accurate: based on big data distributed computing power, the establishment of the network security baseline can be completed in second level and the accuracy is high.
4) Modification: after the network security baseline is established, the user can manually modify the baseline at any time.
Compared with the prior art, the invention has the following positive effects:
1) a more comprehensive network security baseline can be generated;
2) the safety base line in the whole area network can be rapidly generated in a large scale;
3) the network security baseline can be quickly and flexibly modified;
4) the method has the universality of application, and can be widely applied to various network environments;
5) the problem that a traditional network security baseline generation method needs to depend on cooperation of a service expert, a network security expert and network operation and maintenance personnel is solved, and the technical threshold for generating the network security baseline is greatly reduced.
Drawings
The invention will now be described, by way of example, with reference to the accompanying drawings, in which:
FIG. 1 is a flow diagram of a network security baseline implementation principle;
fig. 2 is a flowchart of the working steps of deep parsing of network data.
Detailed Description
In order to more clearly understand the technical features, objects, and effects of the present invention, embodiments of the present invention will now be described with reference to the accompanying drawings.
The invention provides a network security baseline generation method based on full flow, which specifically comprises the following steps as shown in figure 1:
s1, deep analysis of network data: bypass collection is carried out on a network data packet through a mirror image port of the switch, deep analysis is carried out on partial application layer protocols, public data contents and partial deep analysis data information of the application layers in the network message are extracted, and formatted data are generated;
s2, data cleaning and extraction: cleaning the data of the formatted data generated after the acquisition and/or analysis, removing repeated or abnormal data, extracting all public data contents, and extracting key information of an application layer of the application layer data contents which are analyzed partially and deeply;
s3, distributed persistence: importing the extracted data information into a distributed database for persistence, wherein the data information is basic data for generating a network security baseline subsequently;
s4, receiving parameter input: receiving a security baseline generation parameter which is input by a user and comprises a starting time stamp, an ending time stamp and an application layer protocol type;
s5, data aggregation: extracting basic data in the time range from a distributed database according to initial timestamp and end timestamp parameters input by a user, generating an aggregated data set A according to network layer protocols, application layer protocols, source ip addresses, destination ip addresses, source mac addresses and destination mac address information in the basic data, and then aggregating again on the basis of the aggregated data set A according to the application layer protocols input by the user to construct an aggregated data set B of a specified application layer protocol;
s6, data analysis and calculation: and (3) generating a unique identification number ID through a character string formed by splicing all fields of each piece of data of the aggregation data set B through a secure hash algorithm (SHA1), and persisting the aggregation data set B with the ID into a distributed database, namely forming a network security baseline of a specified application layer protocol.
In a specific embodiment of the present invention, for the data cleaning and extraction in step S2, a specific processing method for the data content of the application layer parsed by the partial depth includes: and extracting log content information for a syslog protocol, extracting TLS version information for a ssl protocol, and extracting information such as a resource type, a query domain name, an A-type IP (Internet protocol), a PRT (reverse Domain name) type reverse domain name, an SRV (service name) type, an SRV (service request vector) type use protocol, a domain name where an SRV is located, a host name and the like for a dns protocol.
In a specific embodiment of the present invention, for step S3 distributed persistence, its distributed database may employ an Elasticsearch for performing persistence processing on the entire extracted data information.
The foregoing is illustrative of the preferred embodiments of this invention, and it is to be understood that the invention is not limited to the precise form disclosed herein and that various other combinations, modifications, and environments may be resorted to, falling within the scope of the concept as disclosed herein, either as described above or as apparent to those skilled in the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (4)
1. A network security baseline generation method based on full flow is characterized by comprising the following steps:
s1, deep analysis of network data: bypass collection is carried out on a network data packet through a mirror image port of the switch, deep analysis is carried out on partial application layer protocols, public data contents and partial deep analysis data information of the application layers in the network message are extracted, and formatted data are generated; wherein: the public data content includes: the method comprises the steps of time stamp, source ip address, destination ip address, source port number, destination port number, source mac address, destination mac address, network layer protocol and application layer protocol;
s2, data cleaning and extraction: cleaning the data of the formatted data generated after the acquisition and/or analysis, removing repeated or abnormal data, extracting all public data contents, and extracting key information of an application layer of the application layer data contents which are analyzed partially and deeply;
s3, distributed persistence: importing the extracted data information into a distributed database for persistence;
s4, receiving parameter input: receiving a security baseline generation parameter which is input by a user and comprises a starting time stamp, an ending time stamp and an application layer protocol type;
s5, data aggregation: extracting basic data in the time range from a distributed database according to initial timestamp and end timestamp parameters input by a user, generating an aggregated data set A according to network layer protocols, application layer protocols, source ip addresses, destination ip addresses, source mac addresses and destination mac address information in the basic data, and then aggregating again on the basis of the aggregated data set A according to the application layer protocols input by the user to construct an aggregated data set B of a specified application layer protocol;
s6, data analysis and calculation: and (3) generating a unique identification number ID through a character string formed by splicing all fields of each piece of data of the aggregation data set B through a secure hash algorithm, and persisting the aggregation data set B with the ID into a distributed database, namely forming a network security baseline of a specified application layer protocol.
2. The method for generating the full traffic-based network security baseline according to claim 1, wherein the step S1 of deeply parsing the network data includes the following sub-steps:
s11, acquiring a device serial number, creating a processing sub-process, allocating a shared memory of the sub-process, and starting a write permission for a shared memory buffer area;
s12, initializing a sub-process;
s13, initializing a packet capturing interface and starting a packet capturing thread;
s14, initializing a storage thread and starting the storage thread;
s15, initializing a magnetic disk clearing thread, and starting a magnetic disk clearing function;
and S16, monitoring the subprocess, and restarting the subprocess if unexpected shutdown exists.
3. The method for generating a full traffic based network security baseline according to claim 1, wherein in the step S5, if the application layer protocol inputted by the user is IEC104 protocol, the aggregated data set B is generated from the request common body address, the response common body address, the message type, the type identifier, and the transmission reason of IEC104 data in the basic data.
4. The method as claimed in claim 1, wherein in the step S6, if the security baseline of the application layer protocol exists in the distributed database, the data is automatically updated or refined based on the unique identifier ID, and the user can independently insert or delete a designated security baseline data based on the unique identifier ID.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811589819.0A CN109379390B (en) | 2018-12-25 | 2018-12-25 | Network security baseline generation method based on full flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811589819.0A CN109379390B (en) | 2018-12-25 | 2018-12-25 | Network security baseline generation method based on full flow |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109379390A CN109379390A (en) | 2019-02-22 |
| CN109379390B true CN109379390B (en) | 2021-04-27 |
Family
ID=65371770
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811589819.0A Active CN109379390B (en) | 2018-12-25 | 2018-12-25 | Network security baseline generation method based on full flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109379390B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110311838B (en) * | 2019-07-24 | 2021-05-04 | 绿盟科技集团股份有限公司 | Method and device for counting safety service flow |
| CN111130859B (en) * | 2019-12-10 | 2022-03-18 | 中国电子科技网络信息安全有限公司 | Industrial control network topological graph generation method based on full flow |
| CN111414394A (en) * | 2020-03-31 | 2020-07-14 | 上海观安信息技术股份有限公司 | Power grid company compliance checking and tracking method and system |
| CN112116078A (en) * | 2020-09-22 | 2020-12-22 | 工业互联网创新中心（上海）有限公司 | Information security baseline learning method based on artificial intelligence |
| CN112968842A (en) * | 2021-03-11 | 2021-06-15 | 东莞深证通信息技术有限公司 | Novel network flow acquisition and analysis method and system |
| CN114844831B (en) * | 2022-03-18 | 2024-02-27 | 奇安信科技集团股份有限公司 | Editing data routing method, device and equipment for behavior security base line |
| CN116074113B (en) * | 2023-03-06 | 2023-08-15 | 成都市以太节点科技有限公司 | Security protection method, device and storage medium based on business process constraint |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8730946B2 (en) * | 2007-10-18 | 2014-05-20 | Redshift Internetworking, Inc. | System and method to precisely learn and abstract the positive flow behavior of a unified communication (UC) application and endpoints |
| CN105141604A (en) * | 2015-08-19 | 2015-12-09 | 国家电网公司 | Method and system for detecting network security threat based on trusted business flow |
| CN107360118A (en) * | 2016-05-09 | 2017-11-17 | 中国移动通信集团四川有限公司 | A kind of advanced constant threat attack guarding method and device |
| CN107566372A (en) * | 2017-09-06 | 2018-01-09 | 南京南瑞集团公司 | The secure data optimization of collection method that feature based value is fed back under big data environment |
| JP2018100968A (en) * | 2016-12-19 | 2018-06-28 | 学校法人慶應義塾 | Flow rate measuring apparatus, method for measuring flow rate, and flow rate measurement program |
| CN108833397A (en) * | 2018-06-08 | 2018-11-16 | 武汉思普崚技术有限公司 | A kind of big data safety analysis plateform system based on network security |
-
2018
- 2018-12-25 CN CN201811589819.0A patent/CN109379390B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8730946B2 (en) * | 2007-10-18 | 2014-05-20 | Redshift Internetworking, Inc. | System and method to precisely learn and abstract the positive flow behavior of a unified communication (UC) application and endpoints |
| CN105141604A (en) * | 2015-08-19 | 2015-12-09 | 国家电网公司 | Method and system for detecting network security threat based on trusted business flow |
| CN107360118A (en) * | 2016-05-09 | 2017-11-17 | 中国移动通信集团四川有限公司 | A kind of advanced constant threat attack guarding method and device |
| JP2018100968A (en) * | 2016-12-19 | 2018-06-28 | 学校法人慶應義塾 | Flow rate measuring apparatus, method for measuring flow rate, and flow rate measurement program |
| CN107566372A (en) * | 2017-09-06 | 2018-01-09 | 南京南瑞集团公司 | The secure data optimization of collection method that feature based value is fed back under big data environment |
| CN108833397A (en) * | 2018-06-08 | 2018-11-16 | 武汉思普崚技术有限公司 | A kind of big data safety analysis plateform system based on network security |
Non-Patent Citations (1)
| Title |
|---|
| 《基于流量分析的工业控制系统网络安全基线确定方法研究》;李威等;《科技通报》;20180930;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109379390A (en) | 2019-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109379390B (en) | Network security baseline generation method based on full flow | |
| Perdisci et al. | Iotfinder: Efficient large-scale identification of iot devices via passive dns traffic analysis | |
| US10756949B2 (en) | Log file processing for root cause analysis of a network fabric | |
| US10084681B2 (en) | Method and system for monitoring server cluster | |
| US9665420B2 (en) | Causal engine and correlation engine based log analyzer | |
| EP2240854B1 (en) | Method of resolving network address to host names in network flows for network device | |
| US9584533B2 (en) | Performance enhancements for finding top traffic patterns | |
| US20190007292A1 (en) | Apparatus and method for monitoring network performance of virtualized resources | |
| CN101997925A (en) | Server monitoring method with early warning function and system thereof | |
| CN103152352A (en) | Perfect information security and forensics monitoring method and system based on cloud computing environment | |
| CN114598525A (en) | IP automatic blocking method and device for network attack | |
| US11431792B2 (en) | Determining contextual information for alerts | |
| CN112905548B (en) | Security audit system and method | |
| Vaarandi et al. | Using security logs for collecting and reporting technical security metrics | |
| CN104639391A (en) | Method for generating network flow record and corresponding flow detection equipment | |
| CN110896386B (en) | Method, device, storage medium, processor and terminal for identifying security threat | |
| CN111241104A (en) | Operation auditing method and device, electronic equipment and computer-readable storage medium | |
| US20190007285A1 (en) | Apparatus and Method for Defining Baseline Network Behavior and Producing Analytics and Alerts Therefrom | |
| CN110620690A (en) | Network attack event processing method and electronic equipment thereof | |
| CN115766258A (en) | Multi-stage attack trend prediction method and device based on causal graph and storage medium | |
| JP2016170568A (en) | Log management control system and log management control method | |
| CN114189348A (en) | Asset identification method suitable for industrial control network environment | |
| CN112714118A (en) | Network flow detection method and device | |
| CN114969450B (en) | User behavior analysis method, device, equipment and storage medium | |
| US11588678B2 (en) | Generating incident response action recommendations using anonymized action implementation data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |