CN111885040A - Distributed network situation perception method, system, server and node equipment - Google Patents
Distributed network situation perception method, system, server and node equipment Download PDFInfo
- Publication number
- CN111885040A CN111885040A CN202010694100.4A CN202010694100A CN111885040A CN 111885040 A CN111885040 A CN 111885040A CN 202010694100 A CN202010694100 A CN 202010694100A CN 111885040 A CN111885040 A CN 111885040A
- Authority
- CN
- China
- Prior art keywords
- network
- security
- data
- situation
- threat
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/552—Detecting local intrusion or implementing counter-measures involving long-term monitoring or reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1433—Vulnerability analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computer Networks & Wireless Communication (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention belongs to the technical field of network security, and particularly relates to a distributed network situation perception method, a distributed network situation perception system, a distributed network situation perception server and node equipment, wherein the method comprises the following steps: aiming at a network node security data source, carrying out data fusion by calling a HADOOP interface and utilizing a MapReduce model to obtain a security event in the current time period; network security situation assessment is carried out by quantifying security event threat risks; and predicting the security situation according to the identified attack stage in the quantization process and combining a network attack graph to obtain an attack intention. The invention expands the calculation and storage needing huge calculation capacity in the system to each node in the HADOOP cluster, utilizes the parallel calculation and storage capacity of the cluster to calculate and process, and utilizes MapReduce to realize parallel calculation, thereby realizing the distributed network security situation perception facing large-scale data, optimizing the network security situation perception storage scale and time efficiency, and improving the perception protection capacity for hidden, cooperative, large-scale and multi-stage attacks such as APT attack and the like.
Description
Technical Field
The invention belongs to the technical field of network security, and particularly relates to a distributed network situation perception method, a distributed network situation perception system, a distributed network situation perception server and node equipment.
Background
With the rapid development of information technology, the attacks and threats facing network space security are increasing day by day, and the traditional security products cannot meet the protection requirements more and more. The network security situation awareness technology is used as a new protection means, and is used for recognizing, understanding and predicting the security state and the development trend of the network in a complex and changeable network environment, so that managers can master the network security situation in time and protect threats which may appear in the future in advance. The method aims to realize the perception of the network security situation comprehensively, accurately and in real time. The Tim Bass of the air force communication and information center in the united states first proposed the concept of network situation awareness in 1999 and pointed out that this technology will become the focus of the next generation of network defense research. Over a decade of development, network security situation awareness has achieved many achievements.
At present, a great deal of research work is dedicated to network security situation awareness, but the network security situation awareness is mainly oriented to single-step and simple attacks, a network security sensor is used as a data source, and a hierarchical analysis mode is adopted for situation awareness, so that the situation awareness cannot accurately reflect the gradually-presented characteristics of large scale, cooperation, concealment, multi-stage and the like of novel APT attacks, Lesson attacks and the like. How to identify potential APT attacks from massive, complex and multi-source heterogeneous network security logs and flow and to mine nonlinear attack characteristics needs a server cluster with large-scale computing capability to support. The characteristics of APT attack and the complex data operation processing required by the existing large-scale data network and network multi-node equipment integration all bring many challenging problems to the perception and protection of network security situation.
Disclosure of Invention
Therefore, the invention provides a distributed network situation sensing method, a distributed network situation sensing system, a distributed network situation sensing server and a node device, which utilize the HADOOP to realize the distributed network security situation sensing, optimize the storage scale and timeliness of the network security situation sensing, and improve the sensing protection capability on hidden, cooperative, large-scale and multi-stage attacks such as APT attacks and the like, so as to accurately and timely sense the network attack action and ensure the network security and the reliability and stability.
According to the design scheme provided by the invention, the distributed network situation perception method comprises the following contents:
aiming at a network node security data source, carrying out data fusion by calling a HADOOP interface and utilizing a MapReduce model to obtain a security event in the current time period;
network security situation assessment is carried out by quantifying security event threat risks; and predicting the security situation according to the identified attack stage in the quantization process and combining a network attack graph to obtain an attack intention.
As the distributed network situation awareness method, further, the security data source comprises data of both attacking and defending parties and network environment data, wherein the attacking party comprises atom attack action information; the defender comprises a network protection strategy and security configuration information; the network environment data comprises information of a host, a server and terminal equipment, network connectivity information and network vulnerability scanning information.
As the distributed network situation perception method, firstly, the security data source is preprocessed, irregular data is eliminated by setting filtering rules, and data formatting is carried out through an XML public data model to obtain an XML file with a uniform format.
As the distributed network situation perception method, the invention further calls the HADOOP interface on the data integration server, completes the operation processing and storage of XML file clustering in the HADOOP, returns the result to the data integration server after the clustering is finished, and performs data fusion in the data integration server to obtain the security event of the current time period.
As the distributed network situation perception method, the security event threat risk is further quantized and divided into a system layer, a host layer and a service layer according to the scale and the hierarchical relationship of a network system; and according to the organization structure of the network system, adopting an evaluation strategy of from bottom to top, firstly, locally and then integrally.
As the distributed network situation awareness method, further, the evaluation strategy from bottom to top, local to whole, comprises the following contents: the method comprises the steps of taking a security event as a clue, combining network resource consumption, obtaining threat conditions of services provided by each host, carrying out statistical analysis on the severity, the occurrence frequency and the network bandwidth occupancy rate of the security event, and evaluating the security threat conditions of each service; comprehensively evaluating the safety condition of each host in the network system; and evaluating the security threat situation of the whole local area network system according to the network system structure.
As the distributed network situation perception method, further, a service threat index is obtained through the service attack severity, the attack occurrence frequency, the network bandwidth occupancy rate and the APT attack threat level vector in a given time period; obtaining a host threat index in a given time period through a service security threat vector of the host, weight vectors occupied by the service in all service centers of the host and the service threat index; and obtaining the threat index of the network system local area network in a given time period through the host threat index and the importance weight of the host in the local area network to be evaluated.
Further, the present invention also provides a distributed network situation awareness system, comprising: a data integration server, a HADOOP platform and a situation awareness visualization server connected with the data integration server and the HADOOP platform, wherein,
the data integration server is used for acquiring the security data source of each node of the network, performing data fusion by calling the HADOOP interface and utilizing a MapReduce model in the HADOOP platform, and acquiring the security event in the current time period;
and the situation awareness visualization server is used for calling the safety data on the data integration server and the HADOOP platform, quantifying the threat risk of the safety event, predicting the current situation and graphically displaying the current situation.
The invention has the beneficial effects that:
according to the invention, a network security situation perception framework is established, and data fusion processing is carried out on original security data through a HADOOP platform, so that network security situation evaluation and prediction are completed; expanding the calculation and storage needing huge calculation capacity in the system to each node in the HADOOP cluster, and calculating and processing by using the parallel calculation and storage capacity of the cluster; by using HDFS (Hadoop distributed file system) storage files and data in the HADOOP platform and using MapReduce to realize parallel computation, distributed network security situation perception oriented to large-scale data can be realized, the network security situation perception storage scale and time efficiency are optimized, and the perception protection capability on multi-stage attacks such as APT (advanced persistent threat) attacks and the like is improved.
Description of the drawings:
FIG. 1 is a schematic flow chart of a distributed network situation awareness method in an embodiment;
FIG. 2 is a diagram of an exemplary distributed situational awareness framework;
FIG. 3 is a schematic structural diagram of a distributed situation storage HDFS in an embodiment;
FIG. 4 is a diagram of a MapReduce framework of the distributed situation in the embodiment;
FIG. 5 is a schematic diagram of the working principle of the distributed situational awareness system in the embodiment;
FIG. 6 is a block diagram of an exemplary distributed situational awareness system framework;
FIG. 7 is a schematic diagram of a workflow of the distributed situational awareness system in an embodiment;
FIG. 8 is a diagram illustrating an exemplary network topology;
FIG. 9 is a schematic diagram illustrating visualization of network security situation in an embodiment;
FIG. 10 is a schematic diagram illustrating a real-time attack scenario in an embodiment;
FIG. 11 is a schematic diagram illustrating a display of real-time warning information in an embodiment.
The specific implementation mode is as follows:
in order to make the objects, technical solutions and advantages of the present invention clearer and more obvious, the present invention is further described in detail below with reference to the accompanying drawings and technical solutions.
The HADOOP is a framework platform capable of performing distributed parallel operation in a large number of clustered computers, can process data volume at a PB level, and has the advantages of high reliability, high efficiency and scalability. The main components comprise distributed storage HDFS and distributed computing MapReduce. The HDFS adopts a master/slave structure, is responsible for storage and management of cluster data, and has the characteristics of high data throughput and high fault tolerance rate. MapReduce is a distributed programming model, and comprises Map and Reduce operations. The embodiment of the invention, as shown in fig. 1, provides a distributed network situation awareness method, which includes the following contents:
s101, aiming at a network node security data source, performing data fusion by calling a HADOOP interface and utilizing a MapReduce model to obtain a security event in the current time period;
s102, evaluating the network security situation by quantifying the security event threat risk; and predicting the security situation according to the identified attack stage in the quantization process and combining a network attack graph to obtain an attack intention.
Referring to fig. 2, the embodiment of the present invention optimizes the storage size and timeliness of network security situation awareness by combining HADOOP. Firstly, a situation perception system framework is established, and the idea of distributed acquisition and structured processing mainly comprises four parts of data preprocessing, situation perception, situation display and situation decision. The method comprises the following steps that data preprocessing is carried out on collected information such as equipment performance, equipment logs and flow information, data cleaning, data transformation and the like are carried out on the information to form a situation basic information base; the situation perception takes a situation evaluation information base as a processing object, acquires knowledge of the situation evaluation information base, and uses the acquired knowledge for situation judgment; displaying the situation evaluation result by the situation display; situation decision is a series of actions taken by a network administrator based on the situation conclusions.
HDFS is a master-slave architecture that includes one NameNode and several DataNode nodes. The former manages access operations to files; the latter is responsible for storing data as shown in fig. 3. From the inside of the data, the file is divided into data blocks. In general, a file is divided into several data blocks according to the standard of 64M, and each data block is distributed as far as possible and stored on multiple DataNode nodes. The NameNode is responsible for operations such as file opening, closing, renaming and the like, and generates a mapping table between the DataNode and the data block. The DataNode is responsible for responding to read/write requests of files, and creating, copying and deleting data blocks. The MapReduce framework is used for processing large-scale data of the Internet, has a simple data processing interface and strong functions, can well package distributed parallel operation, and can be used in the application fields of data analysis, machine learning and the like. The core of MapReduce is Map and Reduce, which disassembles the program to be executed into Map and Reduce modes. An input < key, value > pair is converted to another or a collection of < key, value > pairs for output. The data processing process is shown in fig. 4, and the specific work flow is as follows:
and (3) Map end: 1. slicing: cutting an input text (default according to a maximum slice (Long maximum), a minimum slice (> < 1), and a block size (default 128M) to obtain the intermediate value of the maximum slice, the minimum slice and the block size) to form slices (map number), and executing in a cluster concurrently; 2. and (3) executing tasks: in general, the map function is performed as many times as there are rows in the input slice. Eventually a set of < key, value > (key-value pairs) will be formed; 3. writing the output of the maps into a buffer, wherein each map has an annular memory buffer (default 100), when a threshold value (0.8) is reached, overflowing to a disk, and merging and sequencing overflowing files; 4. partition (Partition): partitioning the result of each map, wherein the partition number is usually the number of reduce, and the partition rule is to perform Hash (random) partitioning on keys; 5. sorting: sorting the data of different partitions; 6. polymerization (Combiner): aggregating the sorted results by using a Combiner function; 7. after one map is finished, the HTTP is communicated, the TaskTracker obtains a message and reports the message to the JobTracker, and the Reduce obtains the message regularly; 8. copying: copying data from each map to a buffer; 9. merging and sequencing: merging all maps in one reduce by using heap sorting, and sorting according to key; 10. grouping: the same keys are grouped into groups, forming a set of k.list (V).
Reduce end: 1. and (3) executing tasks: executing a reduce function once on the reduce data of the same group; 2. and (3) outputting: one reduce corresponds to one file output.
Based on the above method, an embodiment of the present invention further provides a distributed network situation awareness system, as shown in fig. 5 and 6, including: a data integration server, a HADOOP platform and a situation awareness visualization server connected with the data integration server and the HADOOP platform, wherein,
the data integration server is used for acquiring the security data source of each node of the network, performing data fusion by calling the HADOOP interface and utilizing a MapReduce model in the HADOOP platform, and acquiring the security event in the current time period;
and the situation awareness visualization server is used for calling the safety data on the data integration server and the HADOOP platform, quantifying the threat risk of the safety event, predicting the current situation and graphically displaying the current situation.
In the embodiment of the invention, further, the security data source comprises data of both attacking and defending parties and network environment data, wherein the attacking party comprises atom attack action information; the defender comprises a network protection strategy and security configuration information; the network environment data comprises information of a host, a server and terminal equipment, network connectivity information and network vulnerability scanning information.
The method comprises the steps that through various safety sensors deployed in a network, safety elements influencing the network safety condition are collected, data support is provided for later situation analysis, and the situation perception is the premise; preprocessing the collected massive, redundant and heterogeneous situation elements, providing basic data for later situation analysis, and formatting alarm, log and system operation and maintenance information. The main data sources of situation analysis comprise an attacker, a defender and environmental information, wherein the attacker mainly acts as an atomic attack; the defender information comprises a summary of solutions such as network protection strategies, security configuration information and the like; the network environment information comprises connectivity information of various hosts, servers and terminal equipment in the network, a network topological structure and the like; and acquiring network vulnerability information by using a missing scanning tool Scanner, collecting network topology information by using a topology analysis tool, and obtaining network communication information according to firewall configuration information.
In the embodiment of the invention, further, aiming at the security data source, the security data source is preprocessed firstly, the irregular data is eliminated by setting the filtering rule, and the data formatting is carried out through the XML public data model to obtain the XML file with the uniform format.
As shown in fig. 7, in order to ensure the accuracy and normalization of data, the raw security data collected by the network security sensors deployed at each node of the network system are first subjected to data fusion to generate a security event, and the process is mainly completed in the data integration server. In the data fusion process, data are firstly cleaned, a filtering rule is set, and irregular data are removed. Such as field default, etc., the data formatting process is realized by adopting the common XML data model commonly used at present. Then, XML files with uniform formats are clustered, the algorithm needs to face massive original data during clustering, the operation amount is huge, in order to meet real-time processing of the XML files, the HADOOP interface is called on the data integration server at the stage, and operation and storage are completed in the HADOOP. At present, the research of the clustering algorithm in the HADOOP is mature, and the system also obtains a good effect. After the clustering is finished, the clustering result is transmitted back to the MySQL database of the data integration server, and the data volume is greatly reduced compared with the original alarm data. And finally, data fusion is carried out on the data integration algorithm, the calculation amount at the stage is relatively small, the read-write operation is directly carried out in the traditional database, and the algorithm is also completed in the data integration server. And finally, obtaining the security event in the current time period, wherein the security event can be realized by using C language, and the clustering algorithm is realized by using MapReduce language in HADOOP.
In the embodiment of the invention, further, the security event threat risk is quantified, and the quantification is divided into a system layer, a host layer and a service layer according to the scale and the hierarchical relationship of a network system; and according to the organization structure of the network system, adopting an evaluation strategy of from bottom to top, firstly, locally and then integrally.
After the data fusion obtains the security event, the system starts to perform a network security situation evaluation stage, and the module mainly operates in a situation analysis and visualization server. The module quantifies threat risks of different security events by accessing the security events in the MySQL database of the data integration server, considers that an actual network system can be decomposed into 3 layers of a system, a host and a service according to scale and hierarchical relation, and most attacks are directed at a certain service on the host in the system.
In the embodiment of the present invention, further, the evaluation strategy of the local part from bottom to top and then the whole part from bottom to top includes the following contents: the method comprises the steps of taking a security event as a clue, combining network resource consumption, obtaining threat conditions of services provided by each host, carrying out statistical analysis on the severity, the occurrence frequency and the network bandwidth occupancy rate of the security event, and evaluating the security threat conditions of each service; comprehensively evaluating the safety condition of each host in the network system; and evaluating the security threat situation of the whole local area network system according to the network system structure.
In the embodiment of the invention, further, a service threat index is obtained through the service attack severity, the attack occurrence frequency, the network bandwidth occupancy rate and the APT attack threat level vector in a given time period; obtaining a host threat index in a given time period through a service security threat vector of the host, weight vectors occupied by the service in all service centers of the host and the service threat index; and obtaining the threat index of the network system local area network in a given time period through the host threat index and the importance weight of the host in the local area network to be evaluated.
Threat of security event occurrence to serviceThe degree is related to the normal access amount, threat strength and attack severity of the service, and the normal access amount of the service in different periods is different, i.e. the same attack has different influence on the service in different periods, given an analysis time window delta t, defining the service S at the time tjHas a threat index of
Wherein:
(1)
for the normal access vector, h is the number of time periods divided into one day, such as 3 time periods: Δ t1＝Night(0:00-8:00)，Δt2＝OfficeHour(8:00-18:00)，Δt3Event (18:00-24:00), i.e.
Is initially set by the system administrator according to the normal average access quantity F of the protected network system at different time intervalsi(i is 1, …, h) is quantitatively assigned, 1,2,3,4,5 are respectively used for representing the visit amount, and the higher the value is, the larger the visit amount is, the higher the visit amount is, and then the normalization processing is carried out to obtain the result
Of elements, i.e.
(2)
The attack severity and occurrence number vectors at time t, respectively, elements thereof
For the time from t to t + Δ t in the ith period, for service SjU is the number of attack types within the time delta t, u is the sum of
The value of (a) is obtained by counting an attack event log database.
(3)
Threat level vectors, elements of which are respectively network bandwidth occupancy and APT attack
The network bandwidth occupancy rate of each time window in the ith time period and the threat level of the APT attack are shown, v is the number of analysis time windows in the ith time period,
the factor of (100) is to convert the network bandwidth occupancy into an integer to evaluate the threat of APT attack.
(4)
The larger the value, the higher the threat level, which should be highly valued by the administrator, and the calculation
The significance of (1) is to calculate the security threat values in a continuous period, draw a service level security situation map, compare the values, and thereby judge the service SjA security threat trend.
Host level: at time t host HkHas a threat index of
Wherein:
(1)
for time t host HkService security threat vector, element
For calculated service SiM is the host HkNumber of open services.
(2)
The weight vector of the service in all the services opened by the host computer is obtained according to the host computer HkImportance IM of provisioning servicesi(i ═ 1, …, m), with 1,2,3 respectively indicating the importance of the service: low, medium, high. Then, to importance IMiCarrying out normalization processing to obtain a vector
Of elements, i.e.
(3) Threat index
The larger the value is, the host H is representedkThe higher the threat level, the more significant it is to calculate a continuous period of time
The values are compared, thereby judging the host HkA trend of security threats during this period.
Network (system) level: the threat index of the network system LAN at time t is
Wherein:
(1)
security threat vector, element, for a host in a network system at time t
For calculated host HlN is the number of hosts in the network system.
(2)
The weight vector of the importance of the host in the local area network to be evaluated, the element value of which is determined according to the status ST of each host in the local area networki(i-1, …, n).
(3) Network system threat index RLThe larger the value, the higher the risk level, and the meaning of calculating R in a continuous periodLAnd comparing the values to judge the security threat trend of the network system in the period.
And generating a network attack graph by using the network topology and the vulnerability information, and predicting attack behaviors according to the attack stages identified in the quantization process and the generated attack graph. And then, predicting the security situation of the network, and identifying the attack intention according to the situation prediction result. The generation of the attack graph and the behavior prediction can be completed in a situation analysis and visualization server, wherein a situation behavior prediction algorithm can be realized on the HADOOP by using a MapReduce language.
In order to further verify the effectiveness of the technical scheme of the invention, a specific platform is set up for simulation test:
and (3) building an HADOOP platform with 5 nodes, wherein 1 master node and 4 slave nodes. The platform is based on a Ubuntu operating system, and the version is HADOOP 2.0.2. And the data acquisition module transmits data generated by the sensor to the server through a Socket by adopting C language. C is adopted to be crossed with Map/Reduce parallel language in situation understanding. The underlying program of the situation assessment and prediction module is developed by C language, wherein the visualization part is developed by Eclipse, and the system is based on a Windows7 system. When the data management is used, for the data scale, a mode of cross use of the traditional MySQL database and the HDFS is adopted, and data which needs to be stored and calculated in large quantity is stored on the HDFS. The monitored network topology and connectivity are constructed, and the interface is shown in fig. 8. The situation evaluation and prediction are performed to obtain a situation map as shown in fig. 9, where the situation change of each host includes the situation of the whole network. The obtained attack scene reconstruction graph is shown in fig. 10 through analysis, and the intrusion path of an attacker can be clearly seen through the graph. Real-time alerts for network security events are shown in fig. 11, which contains security events that occur in real-time.
Through the platform set up above, the network situation change trend is displayed clearly and visually, and it can be further verified that the technical scheme of the invention can realize distributed network security situation perception oriented to large-scale data, and can optimize the network security situation perception storage scale and time efficiency so as to improve the perception protection capability on hidden, collaborative, large-scale and multi-stage attacks such as APT attacks.
Unless specifically stated otherwise, the relative steps, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.
Based on the foregoing system, an embodiment of the present invention further provides a server, including: one or more processors; a storage device to store one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the system as described above.
Based on the system, the embodiment of the present invention further provides a computer-readable node device, on which a computer program is stored, where the program is executed by a processor to implement the system.
The device provided by the embodiment of the present invention has the same implementation principle and technical effect as the system embodiment, and for the sake of brief description, reference may be made to the corresponding content in the system embodiment for the part where the device embodiment is not mentioned.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing system embodiments, and are not described herein again.
In all examples shown and described herein, any particular value should be construed as merely exemplary, and not as a limitation, and thus other examples of example embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus, and system may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the system according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A distributed network situation awareness method is characterized by comprising the following contents:
aiming at a network node security data source, carrying out data fusion by calling a HADOOP interface and utilizing a MapReduce model to obtain a security event in the current time period;
network security situation assessment is carried out by quantifying security event threat risks; and predicting the security situation according to the identified attack stage in the quantization process and combining a network attack graph to obtain an attack intention.
2. The distributed network situation awareness method according to claim 1, wherein the secure data source includes data of both attacking and defending parties and network environment data, wherein the attacking party includes atomic attack action information; the defender comprises a network protection strategy and security configuration information; the network environment data comprises information of a host, a server and terminal equipment, network connectivity information and network vulnerability scanning information.
3. The distributed network situation awareness method according to claim 1 or 2, characterized in that for the secure data source, preprocessing is performed first, filtering rules are set, non-standard data are eliminated, and data formatting processing is performed through an XML public data model to obtain an XML file with a uniform format.
4. The distributed network situation awareness method according to claim 3, wherein an HADOOP interface is called on the data integration server, operation processing and storage of XML file clustering are completed in the HADOOP, a result is transmitted back to the data integration server after clustering is completed, and data fusion is performed in the data integration server to obtain a security event of a current time period.
5. The distributed network situational awareness method of claim 1, wherein the security event threat risk is quantified, and the quantification is divided into a system layer, a host layer and a service layer according to the network system scale and hierarchical relationship; and according to the organization structure of the network system, adopting an evaluation strategy of from bottom to top, firstly, locally and then integrally.
6. The distributed network situational awareness method of claim 5, wherein the bottom-up local-first-then-global evaluation strategy comprises the following: the method comprises the steps of taking a security event as a clue, combining network resource consumption, obtaining threat conditions of services provided by each host, carrying out statistical analysis on the severity, the occurrence frequency and the network bandwidth occupancy rate of the security event, and evaluating the security threat conditions of each service; comprehensively evaluating the safety condition of each host in the network system; and evaluating the security threat situation of the whole local area network system according to the network system structure.
7. The distributed network situation awareness method according to claim 5, wherein a service threat index is obtained by a service attack severity, an attack occurrence frequency, a network bandwidth occupancy rate and an APT attack threat level vector within a given time period; obtaining a host threat index in a given time period through a service security threat vector of the host, weight vectors occupied by the service in all service centers of the host and the service threat index; and obtaining the threat index of the network system local area network in a given time period through the host threat index and the importance weight of the host in the local area network to be evaluated.
8. A distributed network situational awareness system, comprising: a data integration server, a HADOOP platform and a situation awareness visualization server connected with the data integration server and the HADOOP platform, wherein,
the data integration server is used for acquiring the security data source of each node of the network, performing data fusion by calling the HADOOP interface and utilizing a MapReduce model in the HADOOP platform, and acquiring the security event in the current time period;
and the situation awareness visualization server is used for calling the safety data on the data integration server and the HADOOP platform, quantifying the threat risk of the safety event, predicting the current situation and graphically displaying the current situation.
9. A server, comprising: a memory, and one or more processors coupled to the memory; the processor is configured to execute the method of any one of claims 1-7 based on instructions stored in the memory.
10. A computer readable node device having stored thereon a computer program for execution by a processor, the computer program being adapted to perform the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010694100.4A CN111885040A (en) | 2020-07-17 | 2020-07-17 | Distributed network situation perception method, system, server and node equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010694100.4A CN111885040A (en) | 2020-07-17 | 2020-07-17 | Distributed network situation perception method, system, server and node equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111885040A true CN111885040A (en) | 2020-11-03 |
Family
ID=73154979
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010694100.4A Pending CN111885040A (en) | 2020-07-17 | 2020-07-17 | Distributed network situation perception method, system, server and node equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111885040A (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380514A (en) * | 2020-11-13 | 2021-02-19 | 支付宝(杭州)信息技术有限公司 | Biological identification security situation prediction method and device and electronic equipment |
| CN112528223A (en) * | 2020-12-11 | 2021-03-19 | 中国空间技术研究院 | Distributed situation perception consistency method and device |
| CN112685459A (en) * | 2020-11-16 | 2021-04-20 | 中国南方电网有限责任公司 | Attack source feature identification method based on K-means clustering algorithm |
| CN112764852A (en) * | 2021-01-18 | 2021-05-07 | 深圳供电局有限公司 | Operation and maintenance safety monitoring method and system for intelligent wave recording master station and computer readable storage medium |
| CN112769825A (en) * | 2021-01-07 | 2021-05-07 | 深圳市永达电子信息股份有限公司 | Network security guarantee method, system and computer storage medium |
| CN113642002A (en) * | 2021-07-28 | 2021-11-12 | 上海纽盾科技股份有限公司 | Rapid positioning situation perception method and system for cloud data security events |
| CN113746832A (en) * | 2021-09-02 | 2021-12-03 | 华中科技大学 | Multi-method mixed distributed APT malicious flow detection defense system and method |
| CN113949554A (en) * | 2021-10-13 | 2022-01-18 | 东南大学 | High-speed transmission method for distributed network global situation awareness data |
| CN114598534A (en) * | 2022-03-14 | 2022-06-07 | 葛晓磊 | Big data-based equipment detection early warning system |
| CN114745286A (en) * | 2022-04-13 | 2022-07-12 | 电信科学技术第五研究所有限公司 | Intelligent network situation perception system facing dynamic network based on knowledge graph technology |
| CN114915491A (en) * | 2022-06-20 | 2022-08-16 | 北京猎鹰安全科技有限公司 | Method and device for evaluating security state of network terminal and storage medium |
| CN115277249A (en) * | 2022-09-22 | 2022-11-01 | 山东省计算中心（国家超级计算济南中心） | Network security situation perception method based on cooperation of multi-layer heterogeneous network |
| CN116132311A (en) * | 2023-02-17 | 2023-05-16 | 成都工业职业技术学院 | Network security situation awareness method based on time sequence |
| CN116436666A (en) * | 2023-04-11 | 2023-07-14 | 山东省计算中心（国家超级计算济南中心） | Security situation awareness method for distributed heterogeneous network |
| CN117014224A (en) * | 2023-09-12 | 2023-11-07 | 联通(广东)产业互联网有限公司 | Network attack defense method and system based on Gaussian process regression |
| CN117130566A (en) * | 2023-10-27 | 2023-11-28 | 睿至科技集团有限公司 | Distributed storage method and storage platform |
| CN117375982A (en) * | 2023-11-07 | 2024-01-09 | 广州融服信息技术有限公司 | Network situation safety monitoring system |
| CN118590314A (en) * | 2024-08-02 | 2024-09-03 | 网思科技集团有限公司 | Network threat detection method, system and medium based on artificial intelligence |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102098180A (en) * | 2011-02-17 | 2011-06-15 | 华北电力大学 | Network security situational awareness method |
| CN108494810A (en) * | 2018-06-11 | 2018-09-04 | 中国人民解放军战略支援部队信息工程大学 | Network security situation prediction method, apparatus and system towards attack |
-
2020
- 2020-07-17 CN CN202010694100.4A patent/CN111885040A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102098180A (en) * | 2011-02-17 | 2011-06-15 | 华北电力大学 | Network security situational awareness method |
| CN108494810A (en) * | 2018-06-11 | 2018-09-04 | 中国人民解放军战略支援部队信息工程大学 | Network security situation prediction method, apparatus and system towards attack |
Non-Patent Citations (5)
| Title |
|---|
| 刘宇等: "一种松耦合网络安全态势感知模型", 《计算机工程》 * |
| 刘玉岭,冯登国,连一峰,陈恺,吴迪: ""基于时空维度分析的网络安全态势预测方法"", 《计算机研究与发展》 * |
| 王代远等: "基于攻击模式识别的高校网络安全态势评估方法", 《广西教育》 * |
| 管磊等: "基于大数据的网络安全态势感知技术研究", 《信息网络安全》 * |
| 陈秀真,郑庆华,管晓宏,林晨光: ""层次化网络安全威胁态势量化评估方法"", 《软件学报》 * |
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380514B (en) * | 2020-11-13 | 2022-11-22 | 支付宝(杭州)信息技术有限公司 | Biological identification security situation prediction method and device and electronic equipment |
| CN112380514A (en) * | 2020-11-13 | 2021-02-19 | 支付宝(杭州)信息技术有限公司 | Biological identification security situation prediction method and device and electronic equipment |
| CN112685459A (en) * | 2020-11-16 | 2021-04-20 | 中国南方电网有限责任公司 | Attack source feature identification method based on K-means clustering algorithm |
| CN112528223A (en) * | 2020-12-11 | 2021-03-19 | 中国空间技术研究院 | Distributed situation perception consistency method and device |
| CN112528223B (en) * | 2020-12-11 | 2024-05-31 | 中国空间技术研究院 | Distributed situation awareness consistency method and device |
| CN112769825A (en) * | 2021-01-07 | 2021-05-07 | 深圳市永达电子信息股份有限公司 | Network security guarantee method, system and computer storage medium |
| CN112769825B (en) * | 2021-01-07 | 2023-02-21 | 深圳市永达电子信息股份有限公司 | Network security guarantee method, system and computer storage medium |
| CN112764852A (en) * | 2021-01-18 | 2021-05-07 | 深圳供电局有限公司 | Operation and maintenance safety monitoring method and system for intelligent wave recording master station and computer readable storage medium |
| CN113642002A (en) * | 2021-07-28 | 2021-11-12 | 上海纽盾科技股份有限公司 | Rapid positioning situation perception method and system for cloud data security events |
| CN113642002B (en) * | 2021-07-28 | 2024-02-02 | 上海纽盾科技股份有限公司 | Rapid positioning situation awareness method and system for cloud data security event |
| CN113746832A (en) * | 2021-09-02 | 2021-12-03 | 华中科技大学 | Multi-method mixed distributed APT malicious flow detection defense system and method |
| CN113746832B (en) * | 2021-09-02 | 2022-04-29 | 华中科技大学 | Multi-method mixed distributed APT malicious flow detection defense system and method |
| CN113949554A (en) * | 2021-10-13 | 2022-01-18 | 东南大学 | High-speed transmission method for distributed network global situation awareness data |
| CN113949554B (en) * | 2021-10-13 | 2024-02-02 | 东南大学 | High-speed transmission method for global situation awareness data of distributed network |
| CN114598534B (en) * | 2022-03-14 | 2024-03-19 | 郑州市数字政通信息技术有限公司 | Equipment detection early warning system based on big data |
| CN114598534A (en) * | 2022-03-14 | 2022-06-07 | 葛晓磊 | Big data-based equipment detection early warning system |
| CN114745286B (en) * | 2022-04-13 | 2023-11-21 | 电信科学技术第五研究所有限公司 | Intelligent network situation awareness system oriented to dynamic network based on knowledge graph technology |
| CN114745286A (en) * | 2022-04-13 | 2022-07-12 | 电信科学技术第五研究所有限公司 | Intelligent network situation perception system facing dynamic network based on knowledge graph technology |
| CN114915491B (en) * | 2022-06-20 | 2023-12-26 | 北京猎鹰安全科技有限公司 | Evaluation method, device and storage medium for network terminal security state |
| CN114915491A (en) * | 2022-06-20 | 2022-08-16 | 北京猎鹰安全科技有限公司 | Method and device for evaluating security state of network terminal and storage medium |
| CN115277249B (en) * | 2022-09-22 | 2022-12-20 | 山东省计算中心（国家超级计算济南中心） | Network security situation perception method based on cooperation of multi-layer heterogeneous network |
| CN115277249A (en) * | 2022-09-22 | 2022-11-01 | 山东省计算中心（国家超级计算济南中心） | Network security situation perception method based on cooperation of multi-layer heterogeneous network |
| CN116132311A (en) * | 2023-02-17 | 2023-05-16 | 成都工业职业技术学院 | Network security situation awareness method based on time sequence |
| CN116132311B (en) * | 2023-02-17 | 2023-11-21 | 成都工业职业技术学院 | Network security situation awareness method based on time sequence |
| CN116436666B (en) * | 2023-04-11 | 2024-01-26 | 山东省计算中心（国家超级计算济南中心） | Security situation awareness method for distributed heterogeneous network |
| CN116436666A (en) * | 2023-04-11 | 2023-07-14 | 山东省计算中心（国家超级计算济南中心） | Security situation awareness method for distributed heterogeneous network |
| CN117014224B (en) * | 2023-09-12 | 2024-01-30 | 联通(广东)产业互联网有限公司 | Network attack defense method and system based on Gaussian process regression |
| CN117014224A (en) * | 2023-09-12 | 2023-11-07 | 联通(广东)产业互联网有限公司 | Network attack defense method and system based on Gaussian process regression |
| CN117130566A (en) * | 2023-10-27 | 2023-11-28 | 睿至科技集团有限公司 | Distributed storage method and storage platform |
| CN117375982A (en) * | 2023-11-07 | 2024-01-09 | 广州融服信息技术有限公司 | Network situation safety monitoring system |
| CN117375982B (en) * | 2023-11-07 | 2024-03-15 | 广州融服信息技术有限公司 | Network situation safety monitoring system |
| CN118590314A (en) * | 2024-08-02 | 2024-09-03 | 网思科技集团有限公司 | Network threat detection method, system and medium based on artificial intelligence |
| CN118590314B (en) * | 2024-08-02 | 2024-10-11 | 网思科技集团有限公司 | Network threat detection method, system and medium based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111885040A (en) | Distributed network situation perception method, system, server and node equipment | |
| CN109471846A (en) | User behavior auditing system and method on a kind of cloud based on cloud log analysis | |
| US10025813B1 (en) | Distributed data transformation system | |
| CN108039959B (en) | Data situation perception method, system and related device | |
| US20220078188A1 (en) | Change Monitoring and Detection for a Cloud Computing Environment | |
| DE112018005462T5 (en) | ANOMALY DETECTION USING COGNITIVE COMPUTING | |
| CN109362235B (en) | Method of classifying transactions at a network accessible storage device | |
| Li et al. | Safety risk monitoring of cyber-physical power systems based on ensemble learning algorithm | |
| US20100070981A1 (en) | System and Method for Performing Complex Event Processing | |
| CN110168523A (en) | Change monitoring to inquire across figure | |
| Liao et al. | Management and application of mobile big data | |
| CN112468347A (en) | Security management method and device for cloud platform, electronic equipment and storage medium | |
| Elagib et al. | Big data analysis solutions using MapReduce framework | |
| CN112306820B (en) | Log operation and maintenance root cause analysis method and device, electronic equipment and storage medium | |
| CN115544519A (en) | Method for carrying out security association analysis on threat information of metering automation system | |
| CN110677271B (en) | Big data alarm method, device, equipment and storage medium based on ELK | |
| WO2017037801A1 (en) | Monitoring system and method for monitoring | |
| CN117675691B (en) | Remote fault monitoring method, device, equipment and storage medium of router | |
| Xu et al. | Heterogeneous data-driven failure diagnosis for microservice-based industrial clouds towards consumer digital ecosystems | |
| Lee et al. | Detecting anomaly teletraffic using stochastic self-similarity based on Hadoop | |
| CN116545679A (en) | Industrial situation security basic framework and network attack behavior feature analysis method | |
| Sapegin et al. | Evaluation of in‐memory storage engine for machine learning analysis of security events | |
| Dong et al. | Security situation assessment algorithm for industrial control network nodes based on improved text simhash | |
| CN114756301A (en) | Log processing method, device and system | |
| Prashanthi et al. | Generating analytics from web log |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201103 |